Revised: Under review as a conference paper at ICLR

Original: A CTIVE T OPOLOGICAL M APPING BY M ETRIC -F REE E XPLORATION VIA T ASK AND M OTION I MITATION
Revised: A CTIVE T OPOLOGICAL M APPING BY M ETRIC -F REE E XPLORATION VIA T ASK AND M OTION I MITATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Topological map is an effective environment representation for visual navigation.
Revised: Topological map is an effective environment representation for visual navigation.

Original: It is a graph of image nodes and spatial neighborhood edges without metric information such as global or relative agent poses.
Revised: It is a graph of image nodes and spatial neighborhood edges without metric information such as global or relative agent poses.

Original: However, currently such a map construction relies on either less-efficient random exploration or more demanding training involving metric information.
Revised: However, currently such a map construction relies on either less-efficient random exploration or more demanding training involving metric information.

Original: To overcome these issues, we propose active topological mapping (ATM), consisting of an active visual exploration and a topological mapping by visual place recognition.
Revised: To overcome these issues, we propose active topological mapping (ATM), consisting of an active visual exploration and a topological mapping by visual place recognition.

Original: Our main novelty is the simple and lightweight active exploration policy that works entirely in the image feature space involving no metric information.
Revised: Our main novelty is the simple and lightweight active exploration policy that works entirely in the image feature space involving no metric information.

Original: More specifically, ATM’s metric-free exploration is based on task and motion planning (TAMP).
Revised: More specifically, ATM’s metric-free exploration is based on task and motion planning (TAMP).

Original: The task planner is a recurrent neural network using the latest local image observation sequence to hallucinate a feature as the next-step best exploration goal.
Revised: The task planner is a recurrent neural network using the latest local image observation sequence to hallucinate a feature as the next-step best exploration goal.

Original: The motion planner then fuses the current and the hallucinated feature to generate an action taking the agent towards the hallucinated feature goal.
Revised: The motion planner then fuses the current and the hallucinated feature to generate an action taking the agent towards the hallucinated feature goal.

Original: The two planners are jointly trained via deeplysupervised imitation learning from expert exploration demonstrations.
Revised: The two planners are jointly trained via deeply-supervised imitation learning from expert exploration demonstrations.

Original: Extensive experiments in both exploration and navigation tasks on the photo-realistic Gibson and MP3D datasets validate ATM’s effectiveness and generalizability.
Revised: Extensive experiments in both exploration and navigation tasks on the photo-realistic Gibson and MP3D datasets validate ATM’s effectiveness and generalizability.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Mobile agents often create maps to represent their surrounding environments [5].
Revised: Mobile agents often create maps to represent their surrounding environments [6].

Original: Typically, such a map is either topological or metrical (including hybrid ones).
Revised: Typically, such a map is either topological or metrical (including hybrid ones).

Original: We consider a topological map to be metric-free, which means it does not explicitly store global/relative position/orientation information with measurable geometrical accuracy [32, 31].
Revised: We consider a topological map to be metric-free, which means it does not explicitly store global/relative position/orientation information with measurable geometrical accuracy [39, 38].

Original: Instead, it is a graph that stores local sensor observations, such as RGB images, as graph nodes and the spatial neighborhood structure (and often navigation actions) as graph edges that connects observations taken from nearby locations.
Revised: Instead, it is a graph that stores local sensor observations, such as RGB images, as graph nodes and the spatial neighborhood structure (and often navigation actions) as graph edges that connects observations taken from nearby locations.

Original: Compared to metric mapswhich are often reconstructed by optimizing geometric constraints between landmarks and sensor poses from classic simultaneous localization and mapping (SLAM), topological maps have  attracted more attentions in visual navigation tasks due to its advantages in simplicity, flexibility, scalability, and interpretability [48, 9, 22, 33, 8].
Revised: While metric maps are often reconstructed by optimizing geometric constraints between landmarks and sensor poses from classic simultaneous localization and mapping (SLAM), topological maps have recently attracted attention in visual navigation tasks due to the simplicity, flexibility, scalability, and interpretability [58, 13, 27, 40, 12].

Original: A topological map used for visual navigation could be constructed in two ways.
Revised: A topological map used for visual navigation could be constructed in two ways.

Original: The first and simplest way is to let the agent explore the new environment through metric-free random walks , after which the map could be built by projecting the recorded observations into a feature space and adding edges between nearby or sequentially obtained features [48].
Revised: The first and simplest way is to let the agent explore the new environment through metric-free random walks , after which the map could be built by projecting the recorded observations into a feature space and adding edges between nearby or sequentially obtained features [58].

Original: However random walk is very inefficient especially in large or complex rooms, leading to repeated revisits of nearby locations in the same area.
Revised: However random walk is very inefficient especially in large or complex rooms, leading to repeated revisits of nearby locations in the same area.

Original: The other way is to design a navigation policy that controls the agent to more effectively explore the area while creating the map.
Revised: The other way is to design a navigation policy that controls the agent to more effectively explore the area while creating the map.

Original: It is known as active SLAM and often involves some metric information as either required input [35, 8] or intermediate estimations [9].
Revised: It is known as active SLAM and often involves some metric information as either required input [42, 12] or intermediate estimations [13].

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, could we combine the merits of the two ways by finding a metric-free (neither input nor estimates) exploration policy that discovers informative traversing trajectories in unknown environments for topological map construction after exploration?
Revised: 1, could we combine the merits of the two ways by finding a metric-free (neither input nor estimates) exploration policy that discovers informative traversing trajectories in unknown environments for topological map construction after exploration?

Original: To achieve this objective, we propose Active Topological Mapping (ATM) as shown in Fig.
Revised: To achieve this objective, we propose Active Topological Mapping (ATM) as shown in Fig.

Original: It contains two stages: active exploration through a learned metric-free policy, and topological mapping through visual place recognition (VPR) [41].
Revised: It contains two stages: active exploration through a learned metric-free policy, and topological mapping through visual place recognition (VPR) [51].

Original: The first stage adopts the task and motion planning formalism (TAMP) [21, 45] and imitation learning [53] from expert demonstrations which could come from either an oracle policy having full access to virtual environments, or simply a human 
Revised: The first stage adopts the task and motion planning formalism (TAMP) [26, 55] and imitation learning [63] from expert demonstrations which could come from either an oracle policy having full access to virtual environments, or simply a human expert in real world.

Revised: Our main novelty is to design such an imitation at both the task and the motion levels with joint end-to-end training.

Revised: Our task planner, a two-layer LSTM [31] network trained with deep supervision, conceives the next best goal feature to be

Revised: Under review as a conference paper at ICLR

Original: [Figure]
Revised: [Figure]

Original: Where to explore next?
Revised: Where to explore next?

Original: [Figure]
Revised: [Figure]

Original: forward

Original: T o p o l o g i c al
Revised: T o p o l o g [Figure] c a

Revised: [Figure]

Original: S t r a t e g y
Revised: S t r a t e g y

Original: M e t r i c a l S t r a t eg y left right
Revised: M e t r i c a l S t r a t eg y [Figure]

Revised: right explored by hallucination from the current and historical image features.

Revised: Our motion planner, a simple multi-layer perceptron, fuses the current and the hallucinated features and generates the best action that will move the agent to a location whose feature is closer to the goal.

Original: The second stage of ATM takes all observations recorded during the active exploration stage to create the topological map.
Revised: The second stage of ATM takes all observations recorded during the active exploration stage to create the topological map.

Original: This stage could be solved similar to [48], where nodes are first connected by the sequential visiting order, and then additional node connections are discovered by a binary classifier estimating the spatial adjacency between two nodes through their image similarity.
Revised: This stage could be solved similar to [58], where nodes are first connected by the sequential visiting order, and then additional node connections are discovered by a binary classifier estimating the spatial adjacency between two nodes through their image similarity.

Original: Differently, we adopt VPR, a classic technique in SLAM for loop closure detection, to discover additional edges more effectively.
Revised: Differently, we adopt VPR, a classic technique in SLAM for loop closure detection, to discover additional edges more effectively.

Original: We further train an action assigner to assign each newly-added edge with corresponding actions that will move the agent between the two connected nodes.
Revised: We further train an action assigner to assign each newly-added edge with corresponding actions that will move the agent between the two connected nodes.

Original: Finally, the topological map becomes our efficient environment representation for visual navigation as in [48].
Revised: Finally, the topological map becomes our efficient environment representation for visual navigation as in [58].

Original: We validate the efficacy of ATM on two tasks: exploration in which the goal is to maximize the explored area within a fixed step budget, and navigation in which the goal is to use ATM-constructed topological map to navigate the agent to a target image.
Revised: We validate the efficacy of ATM on two tasks: exploration in which the goal is to maximize the explored area within a fixed step budget, and navigation in which the goal is to use ATM-constructed topological map to navigate the agent to a target image.

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: • We propose a simple and effective framework named as active topological mapping (ATM) for efficient and lightweight visual navigation.
Revised: • We propose a simple and effective framework named as active topological mapping (ATM) for efficient and lightweight visual exploration.

Revised: The topological map it constructs can be used for efficient visual navigation.

Original: • We develop joint trainable feature-space task and motion planning (TAMP) networks to achieve metric-free and generalizable exploration.
Revised: • We develop joint trainable feature-space task and motion planning (TAMP) networks to achieve metric-free and generalizable exploration.

Original: • We design a deeply-supervised imitation learning strategy to train the feature-space TAMP networks with better data efficiency.
Revised: • We design a deeply-supervised imitation learning strategy to train the feature-space TAMP networks with better data efficiency.

Original: • We validate our method on the photo-realistic Gibson [59] and MP3D [6] datasets in both visual exploration and navigation.
Revised: • We validate our method on the photo-realistic Gibson [72] and MP3D [9] datasets in both visual exploration and navigation.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Topological map in  navigation.
Revised: Topological map in exploration and navigation.

Original: One of the most difficult aspects of developing a trustworthy robotic navigation system is to build an accurate spatial representation of  the environment.
Revised: Previous works tried to tackle navigation with endto-end learning of sensorimotor control by directly mapping visual observations to the action space [57]. However, such purely reactive RL-based methods that have no explicit memory struggle to navigate in complex scenarios [20, 74]. Newer methods that tackle this problem with scene memory [23, 29] often rely on metric information.

Original: The explicit metric map is commonly used for localization and navigation in the literature [18, 17, 5], but faces robustness and computation challenges especially in dynamic and complex scenes, due to the need of accurate geometric constraints during  map and pose optimization that is sensitive to outliers.
Revised: An explicit metric map is commonly used for localization and navigation in the literature [22, 21, 6], but may face robustness and computation challenges, especially in dynamic and complex scenes, due to the need for accurate geometric constraints during the map and pose optimization.

Original: Later, inspired by the animal and human psychology [58], researchers show thata simple and efficient topological map may aid robot navigation [9, 11, 7, 4, 24].
Revised: Later, inspired by the animal and human psychology [70], researchers show that topological map may aid robot navigation [13, 15, 11, 5, 29].

Original: In literature, many topological mapping solutions either uses a random walkthroughexploration sequence [48], or incrementally constructs a topological graph during the navigation task [9, 33].
Revised: In literature, many topological mapping solutions either uses a random walkthrough sequence [58], or incrementally constructs a topological graph during the navigation task [13, 40].

Original: However, random exploration is inefficient to create a comprehensive map given a limited exploration time.
Revised: However, random exploration is inefficient in creating a comprehensive map given a limited exploration time.

Original: And the existing exploring-while-mapping solutions still involves metric information either as required input or as intermediate estimation, which may not be conveniently and accurately available during training or inference.
Revised: And the existing exploring-while-mapping solutions still involve metric information either as required input or as intermediate estimation.

Original: Instead, we propose a two-stage solution that (1) learns an efficient and generalizable exploration policy to collect visual observations of a novel environment, and (2) uses VPR to construct a topological map for 
Revised: Instead, we propose a two-stage solution that (1) learns an efficient and generalizable exploration policy to collect visual observations of a novel environment, and (2) uses VPR to construct a topological map for future navigation.

Original: [Figure] future navigation. Similar exploration-before-navigation pipelines include [4] and [44], yet they still need metric information to create the topological map and has lower data-efficiency due to RL.
Revised: Similar exploration-before-navigation pipelines include [5]

Original: Learning-based navigation. Previous works enable end-to-end learning of sensorimotor control by directly mapping visual observations to the action space [47]. Purely reactive RL-based methods have no explicit memory and struggle to navigate in complex scenarios [16, 60], and more advanced techniques are proposed such as training with auxiliary tasks [27], episodic control [52], and scene memory [19]. However, RL-based methods still suffers from data inefficiency and poor generalizability.
Revised: Under review as a conference paper at ICLR

Original: Recently, NRNS [24] proposes to learn navigation strategies via watching passive videos, and does not need simulation nor RL. Specifically, NRNS uses a distance function to predict distances of unexplored nodes and uses greedy policy to choose the shortest-distance node. The key assumption is that the distance function can be learned from passive RGBD videos where SLAM is used to estimate relative poses.
Revised: [Figure]

Original: Our work differs from NRNS [24] in that we do not require metric data in either the inference or the training phase. Meanwhile, instead of navigation itself, we mainly focuses on how toefficiently explore an environment and create its topological map for future navigation.
Revised: and [54], yet they still need metric information to create the topological map and have lower data efficiency due to the use of RL.

Original: Exploration in navigation.
Revised: Exploration in navigation.

Original: While various studies have focused on passive map-building and targetdriven navigation policy learning, active SLAM has received less attention: how to actively control the camera for map construction [20].
Revised: While various studies have focused on passive map-building and target-driven navigation policy learning, active SLAM has received less attention: how to actively control the camera for map construction [25].

Original: Recent works [29, 10, 8] use learning-based methods to solve the problem of active exploration.
Revised: Recent works [35, 14, 12] use learning-based methods to solve the problem of active exploration.

Original: [8] learns a hierarchical policy for active exploration via both classic and RL methods.
Revised: [12] learns a hierarchical policy for active exploration via both classic and RL methods.

Original: [12] learns policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards.
Revised: [16] learns policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards.

Original: [56] proposes a curiosity-based RL method which uses episodic memory to form the novelty bonus.
Revised: [66] proposes a curiosity-based RL method which uses episodic memory to form the novelty bonus.

Original: Different from existing methods, we propose task and motion imitation which learns metric-free, generalizable, and data-efficient exploration.
Revised: Different from existing methods, we propose task and motion imitation which learns metric-free, generalizable, and data-efficient exploration.

Revised: Hallucinating future latent feature.

Revised: The idea of hallucinating future latent features has been discussed in other application domains.

Original: Task and  motion planning.
Revised: Previous work has utilized this idea of visual anticipation in video prediction/human action prediction [71, 73, 10, 24, 68], and researchers have applied similar ideas to robot motion and path planning [33, 37, 8, 60]. As stated in [71, 73, 68], visual features in the latent space provides an efficient way to encode semantic/high-level information of scenes, allowing us to do planning in the latent space, which is considered more computationally efficient when dealing with high-dimensional data as input [46, 32]. Different from previous robotics work, we take advantage of this efficient representation by adding deep supervision when anticipating the next visual feature, which was computationally intractable if we were to operate at the pixel level. Deeply-supervised Learning has been extensively explored [41, 67, 44, 45] during the past several years.

Revised: The main idea is to add extra supervisions to various intermediate layers of a deep neural network in order to more effectively train deeper neural networks.

Revised: In our work, we adopt similar idea to deeply supervise the training of feature hallucination and action generation

Original: Task and motion  planning (TAMP) divides a robotic planning problem into: the high-level task allocation (task planning) and the low-level action for the task execution (motion planning).
Revised: Task and motion planning. Task and motion planning (TAMP) divides a robotic planning problem into: the high-level task allocation (task planning) and the low-level action for the task execution (motion planning).

Original: TAMP for manipulation has sparked a lot of interest [13, 43], but TAMP for navigation has attracted far less attention [39, 57].
Revised: This framework is adopted in many robotic tasks such as manipulation [17, 53] exploration [7] and navigation [49, 69].

Revised: Such framework allows us to leverage high-level information about the scenes to tackle challenges in local control techniques [4].

Original: In this work, to perform active topological mapping of a novel environment, the agent firstly reason at the highest level about the regions to navigate: hallucinate the next best feature point to visit.
Revised: In this work, to perform active topological mapping of a novel environment, the agent firstly reasons at the highest level about the regions to navigate: hallucinate the next best feature point to visit.

Original: Afterwards, the agent takes an action to get to the target feature.
Revised: Afterward, the agent takes an action to get to the target feature.

Original: The whole procedure is totally implemented in feature space without any metric information.
Revised: The whole procedure is totally implemented in feature space without any metric information.

Original:  A CTIVE E XPLORATION BY I MITATION IN F EATURE S PACE
Revised:  A CTIVE E XPLORATION BY I MITATION IN F EATURE S PACE

Original: Our topological map is represented by a graph G =
Revised: Our topological map is represented by a graph G =

Original: ( I , A ) , where the graph nodes denoted by 
Revised: ( I , A ) , where the graph nodes denoted by I is a set of

Original: I is a set of RGB panoramic image observations collected by the agent at different locations
Revised: RGB panoramic image observations collected by the agent at different locations

Original: I =
Revised: I =

Original: { I 1 , I 2 , · · · , I N } (where N denotes the number of nodes), and the edges denoted by A is composed of a set of actions a ( I i ,I j
Revised: { I 1 , I 2 , · · · , I N } (where N denotes the number of nodes), and the edges denoted by A is composed of a set of actions a ( I i ,I j

Original: ) ∈
Revised: ) ∈

Original: A which connects two spatially adjacent observations
Revised: A which connects two spatially adjacent observations

Original: I i and I j .
Revised: I i and I j .

Original: In this work, each RGB panoramic image is of size 256 × 512 , and the action space consists of three basic actions: move_forward , turn_left , and turn_right .
Revised: In this work, each RGB panoramic image is of size 256 × 512 , and the action space consists of three basic actions: move_forward , turn_left , and turn_right .

Original: Active exploration aims at maximizing the topological map coverage over an environment given a certain step budget N .
Revised: Active exploration aims at maximizing the topological map coverage over an environment given a certain step budget N .

Original: The coverage of the topological map denoted by C is defined as the total area in the map that is known to be traversable or non-traversable.
Revised: The coverage of the topological map denoted by C is defined as the total area in the map that is known to be traversable or non-traversable.

Original: Mathematically, let π θ denote the policy network parameterized by θ , a t denote the action taken at step t , and ∆ C ( a t ) denote the gain in coverage introduced by taking action a t , the following objective function is optimized to obtain the optimal exploration policy π ∗ :
Revised: Mathematically, let π θ denote the policy network parameterized by θ , a t denote the action taken at step t , and ∆ C ( a t ) denote the gain in coverage introduced by taking action a t , the following objective function is optimized to obtain the optimal exploration policy π ∗ :

Original: [Equation]
Revised: [Figure]

Original: Task Planner

Original: Motion Planner ......
Revised: Motion Planner ......

Original: Inference 
Revised: Inference consuming a the M consumes the to [Equation]

Original: Learning from expert demonstrations.
Revised: Learning from expert demonstrations.

Original: In literature, most works solve Eq.
Revised: In literature, most works solve Eq.

Original: (1) by reinforcement learning to maximize the reward [12, 36], such solutions are not only data-hungry, but also require complicated training involving metric information.
Revised: (1) by reinforcement learning to maximize the reward [16, 43], such solutions are not only data-hungry, but also require complicated training involving metric information.

Original: Differently, we adopt imitation learning [53] to let our policy network π θ mimic the output of the expert policy ˜ π which could come from either an oracle policy having full access to virtual environments, or simply a human expert in real world.
Revised: Differently, we adopt imitation learning [63] to let our policy network π θ mimic the output of the expert policy ˜ π which could come from either an oracle policy having full access to virtual environments, or simply a human expert in real world.

Original: Hence, our objective is to minimize the difference between our policy network and the expert policy:
Revised: Hence, our objective is to minimize the difference between our policy network and the expert policy:

Original: [Equation]
Revised: [Equation]

Original: where L measures the discrepancy between two policies.
Revised: where L measures the discrepancy between two policies.

Original: We propose a novel task and motion imitation in feature space to solve Eq.
Revised: We propose a novel task and motion imitation in feature space to solve Eq.

Original: (2) which will be introduced in the following sections.
Revised: (2) which will be introduced in the following sections.

Original: We respectively introduce the feature extraction (3.1), the policy network π θ composed of a TaskPlanner denoted by π θ T (3.2) as well as a MotionPlanner denoted by π θ M (3.3), and the deeply-supervised learning strategy (3.4).
Revised: We respectively introduce the feature extraction (3.1), the policy network π θ composed of a TaskPlanner denoted by π θ T (3.2) as well as a MotionPlanner denoted by π θ M (3.3), and the deeply-supervised learning strategy (3.4).

Original: 3. F EATURE E XTRACTION
Revised: 3. F EATURE E XTRACTION

Original: We firstly encode each visual observation I t ∈
Revised: We firstly encode each visual observation I t ∈

Original: I ( t = 1 , 2 , ..., N ) with a feature extractor g ψ parameterized by ψ which uses the ImageNet [54] pre-trained ResNet18 backbone [25].
Revised: I ( t = 1 , 2 , ..., N ) with a feature extractor g ψ parameterized by ψ which uses the ImageNet [64] pre-trained ResNet18 backbone [30].

Original: The feature embedding f t ∈ R d ( d = 512) is obtained by f t = g ψ
Revised: The feature embedding f t ∈ R d ( d = 512) is obtained by f t = g ψ

Original: ( I t ) , (see Fig. 3).
Revised: ( I t ) , (see Fig. 3).

Original: Note that g ψ is jointly optimized with the task planner
Revised: Note that g ψ is jointly optimized with the task planner

Original: π θ T as well as the MotionPlanner π θ M via imitation learning.
Revised: π θ T as well as the MotionPlanner π θ M via imitation learning.

Original: 3. T ASK P LANNER FOR N EXT B EST F EATURE H ALLUCINATION
Revised: 3. T ASK P LANNER FOR N EXT B EST F EATURE H ALLUCINATION

Original: TaskPlanner π θ T parameterized by θ T takes the most recent m -step visual features F = { f t − m , · · · , f t } as input, and learns to hallucinate the next best feature to visit which is denoted by ˆ f t +1 , see Fig.
Revised: TaskPlanner π θ T parameterized by θ T takes the most recent m -step visual features F = { f t − m , · · · , f t } as input, and learns to hallucinate the next best feature to visit which is denoted by ˆ f t +1 , see Fig.

Original: In specific, π θ T is a two-layer LSTM [26]:
Revised: In specific, π θ T is a two-layer LSTM [31]:

Original: [Equation]
Revised: [Equation]

Revised: Under review as a conference paper at ICLR

Original: To save computation, π θ T only takes the most recent m -step features as input and we empirically find that m
Revised: To save computation, π θ T only takes the most recent m -step features as input and we empirically find that m

Original: = 10 achieves good performance.
Revised: = 10 achieves good performance.

Original: In other words, TaskPlanner is only equipped with a short-term scene memory, and it tries to extend the feature space as quickly as possible in order to guide the agent to perform efficient exploration.
Revised: In other words, TaskPlanner is only equipped with a short-term scene memory, and it tries to extend the feature space as quickly as possible in order to guide the agent to perform efficient exploration.

Original: 3. M OTION P LANNER FOR A CTION G ENERATION
Revised: 3. M OTION P LANNER FOR A CTION G ENERATION

Original: MotionPlanner π θ M parameterized by θ M takes the hallucinated feature ˆ
Revised: MotionPlanner π θ M parameterized by θ M takes the hallucinated feature ˆ

Original: f t +1 and the current feature f t as input, and outputs the action taking the agent towards the hallucinated goal (see Fig. 3).
Revised: f t +1 and the current feature f t as input, and outputs the action taking the agent towards the hallucinated goal (see Fig. 3).

Original: Specifically, π θ M is a multi-layer-perceptron (MLP) taking the concatenation of two features as input to classify the action:
Revised: Specifically, π θ M is a multi-layer-perceptron (MLP) taking the concatenation of two features as input to classify the action:

Original: [Equation]
Revised: [Equation]

Original: 3. D EEPLY -S UPERVISED I MITATION L EARNING S TRATEGY
Revised: 3. D EEPLY -S UPERVISED I MITATION L EARNING S TRATEGY

Original: Our imitation pipeline is shown in Fig.
Revised: Our imitation pipeline is shown in Fig.

Original: Given an expert exploration demonstration including a sequence of images and the corresponding expert actions E = {{ I 1 , a 1 } , { I 2 , a 2 } , · · · , { I N , a N }} , we adopt the deeply-supervised learning strategy [34] to jointly optimize the feature extractor g ψ , task planner π θ T , and MotionPlanner π θ M .
Revised: Given an expert exploration demonstration including a sequence of images and the corresponding expert actions E = {{ I 1 , a 1 } , { I 2 , a 2 } , · · · , { I N , a N }} , we adopt the deeply-supervised learning strategy [41] to jointly optimize the feature extractor g ψ , task planner π θ T , and MotionPlanner π θ M .

Original: Ultimately, our objective in Eq.
Revised: Ultimately, our objective in Eq.

Original: (2) becomes,
Revised: (2) becomes,

Original: [Equation]
Revised: [Equation]

Original: where L T is L 2 loss to measure the discrepancy between two features, and L M is cross entropy loss to make the model imitate the expert action category.
Revised: where L T is L 2 loss to measure the discrepancy between two features, and L M is cross entropy loss to make the model imitate the expert action category.

Original: The desired target feature f t +1 is obtained by f t +

Revised: The desired target feature f t +1 is obtained by f t +

Original: =
Revised: =

Original: g ψ ( I t +1 )
Revised: g ψ ( I t +1 )

Original: ( I t +1 is obtained from the expert demonstration E ), the desired action a t is also read from E , the hallucinated feature ˆ f t +1 is calculated by Eq.
Revised: ( I t +1 is obtained from the expert demonstration E ), the desired action a t is also read from E , the hallucinated feature ˆ f t +1 is calculated by Eq.

Original: (3), and the generated action ˆ a t is computed by Eq.
Revised: (3), and the generated action ˆ a t is computed by Eq.

Original: For each training iteration, we randomly clip m + 1 observations and the corresponding m actions from an expert exploration ( m = [-
-] and N ≫ m ), and feed them to g ψ , π θ T , and π θ M .
Revised: For each training iteration, we randomly clip m + 1 observations and the corresponding m actions from an expert exploration ( m = [-
-]10 and N ≫ m ), and feed them to g ψ , π θ T , and π θ M .

Original: Note that we supervise all the intermediate output, as illustrated in Fig.
Revised: Note that we supervise all the intermediate output, as illustrated in Fig.

Original: Remark.
Revised: Remark.

Original: Different from other works, we decompose the exploration policy π θ into a task planner
Revised: Different from other works, we decompose the exploration policy π θ into a task planner

Original: π θ T as well as a MotionPlanner π θ M , and jointly learn them by imitation in feature space.
Revised: π θ T as well as a MotionPlanner π θ M , and jointly learn them by imitation in feature space.

Original: TaskPlanner hallucinates high-level features and MotionPlanner generates low-level actions.
Revised: TaskPlanner hallucinates high-level features and MotionPlanner generates low-level actions.

Original: We supervise all the intermediate output of the network including task-level features ˆ f t , thus our TAMP imitation strategy can be considered as deeply-supervised learning [34].
Revised: We supervise all the intermediate output of the network including task-level features ˆ f t , thus our TAMP imitation strategy can be considered as deeply-supervised learning [41].

Original:  T OPOLOGICAL M APPING VIA V ISUAL P LACE R ECOGNITION
Revised:  T OPOLOGICAL M APPING VIA V ISUAL P LACE R ECOGNITION

Original: The topological map G =
Revised: The topological map G =

Original: ( I , A ) initialized by the active exploration experience in Sec.
Revised: ( I , A ) initialized by the active exploration experience in Sec.

Original: 3 is unidirectionally connected in the temporal domain.
Revised: 3 is unidirectionally connected in the temporal domain.

Original: Each node (a panoramic RGB image observation) is just connected with its preceding node and next node, failing to discover spatial adjacency.
Revised: Each node (a panoramic RGB image observation) is just connected with its preceding node and next node, failing to discover spatial adjacency.

Original: We propose to further complete the initial map G by adding edges to the node pairs if two nodes in a pair possess high visual similarity.
Revised: We propose to further complete the initial map G by adding edges to the node pairs if two nodes in a pair possess high visual similarity.

Original: Specifically, we adopt visual place recognition (VPR) [28, 3] method and geometric verification to measure “visual similarity” of a node pair.
Revised: Specifically, we adopt visual place recognition (VPR) [34, 3] method and geometric verification to measure “visual similarity” of a node pair.

Original: Given a room of interest and all the nodes’ images, for every node, we first used non-learning, 
Revised: Given a room of interest and all the nodes’ images, for every node, we first used non-learning, VLAD-based

Original: VLAD-based VPR system to pick out other images that are very similar to this node as candidates for further geometric verification.
Revised: VPR system to pick out other images that are very similar to this node as candidates for further geometric verification.

Original: After this VPR process, we conduct geometric verification by inspecting if  number of matching SIFT keypoints between the node’s image and a candidate’s image is larger than a threshold (in our case 100).
Revised: After this VPR process, we conduct geometric verification by inspecting if the number of matching SIFT keypoints between the node’s image and a candidate’s image is larger than a threshold (in our case 100).

Original: Passing the threhold means that we will add edges between these two nodes.
Revised: Passing the threshold means that we will add edges between these two nodes.

Original: Meanwhile, we train a model namedas ActionAssigner to assign an action list to each new edge.
Revised: Meanwhile, we train a model named ActionAssigner to assign an action list to each new edge.

Original: The architecture of ActionAssigner is similar to MotionPlanner , yet ActionAssigner predicts a sequence of actions with two node features as input, while MotionPlanner is a one-step action predictor (predict just one action).
Revised: The architecture of ActionAssigner is similar to MotionPlanner , yet ActionAssigner predicts a sequence of actions with two node features as input, while MotionPlanner is a one-step action predictor (predict just one action).

Original: We adopted VPR based method mainly due to its time efficiency compared to methods involving brute-force, such as that in SPTM [48] or brute-force geometric verification.
Revised: We adopted VPR based method mainly due to its time efficiency compared to methods involving brute force, such as that in SPTM [58] or brute-force geometric verification.

Original: See Appendix E for more details on the VPR process we used its time efficiency and our implementation of SPTM. 
Revised: See Appendix F for more details on the VPR process we used, its time efficiency, and our implementation of SPTM. Under review as a conference paper at ICLR

Original: Our completed topological map can perform image-goal navigation tasks.
Revised: Our completed topological map can perform image-goal navigation tasks.

Original: Given the image observations for the start and goal positions, we localize them on topological map by retrieving the node image with the highest inlier number (via VPR).
Revised: Given the image observations for the start and goal positions, we localize them on topological map by retrieving the node image with the highest inlier number (via VPR).

Original: Oncebeing localized, we apply Dijkstra’s algorithm [15] to find the shortest path between the two nodes.
Revised: Once localized, we apply Dijkstra’s algorithm [19] to find the shortest path between the two nodes.

Original: We can then smoothly navigate the agent from the start position to the goal position without metric information.
Revised: We can then smoothly navigate the agent from the start position to the goal position without metric information.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We test ATM on two datasets: Gibson [59] and Matterport3D (MP3D) [6] dataset on Habitat-lab platform [42].
Revised: We test ATM on two datasets: Gibson [72] and Matterport3D (MP3D) [9] dataset on Habitat-lab platform [52].

Original: The two datasets are collected in real indoor spaces by 3D scanning and reconstruction methods.
Revised: The two datasets are collected in real indoor spaces by 3D scanning and reconstruction methods.

Original: The agents can be equipped with multi-modality sensors to perform various robotic tasks.
Revised: The agents can be equipped with multi-modality sensors to perform various robotic tasks.

Original: The average room size of MP3D (100 m 2 ) is much larger than that of Gibson ( [14 m 2 , 85 m 2 ] ).
Revised: The average room size of MP3D (100 m 2 ) is much larger than that of Gibson ( [14 m 2 , 85 m 2 ] ).

Original: We run experiments on two tasks: (1) autonomous exploration proposed by Chen et al.
Revised: We run experiments on two tasks: (1) autonomous exploration proposed by Chen et al.

Original: [12], in which the target is to maximize an environment coverage within a fixed step budget (1000-step budget following [8]), and (2) image-goal navigation where the agent uses the constructed topological map to navigate from current observation to target observation.
Revised: [16], in which the target is to maximize an environment coverage within a fixed step budget (1000-step budget following [12]), and (2) image-goal navigation where the agent uses the constructed topological map to navigate from current observation to target observation.

Original: Regarding the exploration, we employ two evaluation metrics: (1) coverage ratio which is the percentage of covered area over all navigable area, and (2) absolute covered area ( m 2 ).
Revised: Regarding the exploration, we employ two evaluation metrics: (1) coverage ratio which is the percentage of covered area over all navigable area, and (2) absolute covered area ( m 2 ).

Original: We exactly follow the setting by ANS [8] that a point is covered by the agent if it lies within the agent’s field-of-view and is less then 3 .
Revised: We exactly follow the setting by ANS [12] that a point is covered by the agent if it lies within the agent’s field-of-view and is less then 3 .

Original: 2 m away.
Revised: 2 m away.

Original: Regarding the navigation, we adopt two evaluation metrics: shortest path length (SPL) and success rate (Succ.
Revised: Regarding the navigation, we adopt two evaluation metrics: shortest path length (SPL) and success rate (Succ.

Original: Rate)[2].
Revised: Rate)

Original: We again follow ANS [8] to train ATM on Gibson training dataset (72 scenes), and test ATM on Gibson validation dataset (14 scenes) and MP3D test dataset (18 scenes).
Revised: We again follow ANS [12] to train ATM on Gibson training dataset (72 scenes), and test ATM on Gibson validation dataset (14 scenes) and MP3D test dataset (18 scenes).

Original: Using the ATM trained on Gibson dataset to test on MP3D dataset helps to show ATM’s generalizability.
Revised: Using the ATM trained on Gibson dataset to test on MP3D dataset helps to show ATM’s generalizability.

Original: 5. E XPERIMENT C ONFIGURATION
Revised: 5. E XPERIMENT C ONFIGURATION

Original: Exploration setup.
Revised: Exploration setup.

Original: In exploration, we independently explore each scene 71 times, each time assigning the agent a random starting point (created by a random seed number).
Revised: In exploration, we independently explore each scene 71 times, each time assigning the agent a random starting point (created by a random seed number).

Original: We keep track of all the random seed numbers for result reproduction.
Revised: We keep track of all the random seed numbers for result reproduction.

Original: We use the Habitat-lab sliding function so that the agent does not stop when it collides with a wall, but instead slides along it.
Revised: We use the Habitat-lab sliding function so that the agent does not stop when it collides with a wall, but instead slides along it.

Original: In order to generate the initial 10 steps required by ATM, we constantly let the agent to execute move_forward action.
Revised: In order to generate the initial 10 steps required by ATM, we constantly let the agent to execute move_forward action.

Original: Once it collides the wall, it randomly chooses turn_left or turn_right action to continue to explore.
Revised: Once it collides the wall, it randomly chooses turn_left or turn_right action to continue to explore.

Original: Afterwards, we iteratively call TaskPlanner and MotionPlanner to actively explore the environment.
Revised: Afterwards, we iteratively call TaskPlanner and MotionPlanner to actively explore the environment.

Original: During ATM-guided exploration, we allow the agent to actively detect its distance with surrounding obstacles or walls.
Revised: During ATM-guided exploration, we allow the agent to actively detect its distance with surrounding obstacles or walls.

Original: When the agent’s forward-looking distance to obstacles is less than 2-step distances and the ATM predicted action is still move_forward , the agent will turn to a new direction (by randomly iteratively executing turn_left and turn_right action) with the longer forward-looking distance to new obstacles.
Revised: When the agent’s forward-looking distance to obstacles is less than 2-step distances and the ATM predicted action is still move_forward , the agent will turn to a new direction (by randomly iteratively executing turn_left and turn_right action) with the longer forward-looking distance to new obstacles.

Original: We experiment with two locomotion setups: the first one is with step-size 0.25 m and turning angle 10 ◦ , which follows the same setting established in [8] for comparing with baseline methods in the exploration task.
Revised: We experiment with two locomotion setups: the first one is with step-size 0.25 m and turning angle 10 ◦ , which follows the same setting established in [12] for comparing with baseline methods in the exploration task.

Original: The second one is with step-size 0.30 m and turn-angle 30 ◦ .
Revised: The second one is with step-size 0.30 m and turn-angle 30 ◦ .

Original: This helps us test ATM’s generalization capability under different locomotion configurations.
Revised: This helps us test ATM’s generalization capability under different locomotion configurations.

Original: Navigation setup.
Revised: Navigation setup.

Original: In navigation, we encourage the agent to visit enough positions for each room scene.
Revised: In navigation, we encourage the agent to visit enough positions for each room scene.

Original: Specifically, the agent has collected 2,000 images per room on Gibson and 5,000 images per room on MP3D (2,000/5,000-step ATM-guided exploration).
Revised: Specifically, the agent has collected 2,000 images per room on Gibson and 5,000 images per room on MP3D (2,000/5,000-step ATM-guided exploration).

Original: Expert demonstrations.
Revised: Expert demonstrations.

Original: We sample a set of anchor points across all navigable area inthe each room scene, then call Habitat-lab Pathfinder API to let the agent traverse all anchor points tocover the whole area. Pathfinder API generates the most economic action sequence (following shortest-path geodesic path) leading the agent from one anchor point to another.
Revised: To generate expert demonstrations, we place so-called anchor points relatively evenly over all navigable areas in each training room. The agent starts at a random anchor point and walks to the closest anchor point with help from Habitat’s API. We repeat this process until all the anchor points are visited.

Original: In practice, we use the provided episodes’ start/end points in Gibson as the anchor points since they densely cover all the navigable area (see Fig. 5). We collected 450 k training images and the associated actions inscenes from Gibson, which are used to train both ATM and ActionAssigner .
Revised: During this process, we record all the actions the agent takes, along with all the panoramic photos at each time step. Note that such a demonstration is not a strong demonstration, as this process is not directly related to the task of maximizing exploration or efficient navigation. For more details on expert demonstration, please refer to Appendix D and Fig.

Original: Training details.
Revised: Training details.

Original: In our implementation, the local observation sequence length is 10 (m=10) because we find it has achieved good performance-memory trade-off.
Revised: In our implementation, the local observation sequence length is 10 (m=10) because we find it has achieved good performance-memory trade-off.

Original: We experimentally tested m = 20 and got inferior performance.
Revised: We experimentally tested m = 20 and got inferior performance.

Original: ATM network architecture is in Appendix D (parameter size is just 16 M).
Revised: ATM network architecture is in Appendix E (parameter size is just 16 M).

Original: We train ATM with PyTorch [50].
Revised: We train ATM with PyTorch [61].

Original: The optimizer is Adam [30] with an initial learning rate 0.0005, but decays every 40 epochs with the decaying rate 0.5.
Revised: The optimizer is Adam [36] with an initial learning rate 0.0005, but decays everyepochs with the decaying rate 0.5.

Original: In total, we train 70 epochs.
Revised: In total, we train 70 epochs.

Original: We train all the ATM variants with the same hyperparameter setting for fair comparison.
Revised: We train all the ATM variants with the same hyperparameter setting for fair comparison.

Revised: Under review as a conference paper at ICLR

Original: 5. C OMPARISON M ETHODS
Revised: 5. C OMPARISON M ETHODS

Original: For exploration task, we compare ATM with six RL-based methods: 1. RL + 3LConv : An RL Policy with 3 layer convolutional network [42]; 2.
Revised: For exploration task, we compare ATM with six RL-based methods: 1. RL + 3LConv : An RL Policy withlayer convolutional network [52]; 2.

Original: RL + Res18 : RL Policy initialized with ResNet18 [25] and followed by GRU [14]; 3.
Revised: RL + Res18 : RL Policy initialized with ResNet18 [30] and followed by GRU [18]; 3.

Original: RL + Res18 + AuxDepth : adapted from [46] which uses depth map prediction as an auxiliary task.
Revised: RL + Res18 + AuxDepth : adapted from [56] which uses depth map prediction as an auxiliary task.

Original: The network architecture is the same as ANS [36] with one extra deconvolution layer for depth prediction; 4.
Revised: The network architecture is the same as ANS [43] with one extra deconvolution layer for depth prediction; 4.

Original: RL + Res18 + ProjDepth adapted from Chen et al.
Revised: RL + Res18 + ProjDepth adapted from Chen et al.

Original: [12] who project the depth image in an egocentric top-down in addition to the RGB image as input to the RL policy.
Revised: [16] who project the depth image in an egocentric top-down in addition to the RGB image as input to the RL policy.

Original: ANS (Active Neural SLAM [8]) jointly learns a local and global policy network to guide the agent to explore; 6.
Revised: ANS (Active Neural SLAM [12]) jointly learns a local and global policy network to guide the agent to explore; 6.

Original: OccAnt [55]: takes RGB, depth and camera as input to learn 2D top-down occupancy map to help exploration.
Revised: OccAnt [65]: takes RGB, depth and camera as input to learn 2D top-down occupancy map to help exploration.

Original: For ablation studies, we have following ATM variants:
Revised: For ablation studies, we have following ATM variants:

Original: RandomWalk
Revised: RandomWalk

Original: The agent randomly choose an action to execute at each step.
Revised: The agent randomly choose an action to execute at each step.

Original: It serves

Original:  as a baseline and helps us to know agent exploration capability without any active learning process.
Revised: It serves as a baseline and helps us to know agent exploration capability without any active learning process.

Original: Please note that RandomWalk is also the SPTM [48] exploration strategy.
Revised: Please note that RandomWalk is also the SPTM [58] exploration strategy.

Original: ATM_noFeatHallu.
Revised: ATM_noFeatHallu.

Original: We use the architecture of TaskPlanner to directly predict the next action.
Revised: We use the architecture of TaskPlanner to directly predict the next action.

Original: It discards task planning in feature space, but instead plans directly in action space.
Revised: It discards task planning in feature space, but instead plans directly in action space.

Original: Its performance helps us to understand if the hallucinated feature is truly necessary.
Revised: Its performance helps us to understand if the hallucinated feature is truly necessary.

Original: 3. ATM_FullHistory.
Revised: 3. ATM_withHistory.

Original: ATM is trained with only a short-memory (the latest m steps observations).
Revised: ATM is trained with only a short-memory (the latest m steps observations).

Original: To validate the influence of long-term memory, we train a new ATM variant by adding extra historical information: we evenly extract 10 observations among all historically explored observations excluding the latest m steps.
Revised: To validate the influence of long-term memory, we train a new ATM variant by adding extra historical information: we evenly extract 10 observations among all historically explored observations excluding the latest m steps.

Original: After feeding them to ResNet18 [25] to get their embedding, we simply use average pooling to get one 512-dimensional vector and feed it to TaskPlanner LSTM as the hidden state input.
Revised: After feeding them to ResNet18 [30] to get their embedding, we simply use average pooling to get one 512-dimensional vector and feed it to TaskPlanner LSTM as the hidden state input.

Original: ATM_LSTMActRegu.
Revised: ATM_LSTMActRegu.

Original: TaskPlanner hallucinates a next-best feature at each time step to deeply supervise (or regularize) the whole framework in feature space.
Revised: TaskPlanner hallucinates a next-best feature at each time step to deeply supervise (or regularize) the whole framework in feature space.

Original: As an alternative, we can predict action instead at each time step (regularize action).
Revised: As an alternative, we can predict action instead at each time step (regularize action).

Original: This ATM variant helps us to figure out whether supervising each step of TaskPlanner LSTM in feature space is helpful.
Revised: This ATM variant helps us to figure out whether supervising each step of TaskPlanner LSTM in feature space is helpful.

Original: ATM_noDeepSup.
Revised: ATM_noDeepSup.

Original: ATM without deeply-supervised learning.
Revised: ATM without deeply-supervised learning.

Original: We remove LSTM per-frame feature regularization in TaskPlanner and neighboring frame action regularization in MotionPlanner .
Revised: We remove LSTM per-frame feature supervision in TaskPlanner and neighboring frame action supervision in MotionPlanner .

Revised: In other words, we just keep the feature prediction and action classification between the latest step and future step.

Original: It helps to test the necessity of involving deeply-supervised learning strategy.
Revised: It helps to test the necessity of involving deeply-supervised learning strategy.

Original: 5. E VALUATION R ESULTS ON E XPLORATION
Revised: 5. E VALUATION R ESULTS ON E XPLORATION

Original: The quantitative results on exploration task are shown in Table 1.
Revised: The quantitative results on exploration task are shown in Table 1.

Original: We can observe from this table that ATM achieves comparable performance on both Gibson dataset with RL-based methods, and bestperforming result on MP3D dataset by outperforming all RL-based methods significantly (about 13% coverage ratio and 40 m 2 area improvement).
Revised: We can observe from this table that ATM achieves comparable performance on both Gibson dataset with RL-based methods, and best-performing result on MP3D dataset by outperforming all RL-based methods significantly (about 13% coverage ratio and 40 m 2 area improvement).

Original: Since the comparing RL-based methods [1, 46, 12, 8] build the map in metric space and requires millions of training images, ATM ismore desirable because (1) it is metric-free, and (2) it is lightweight (in terms of parameter size 16 M) and requires much less training data (just about 0.45 million images, in contrast with 10 million images required by most RL-based methods).
Revised: Since the comparing RL-based methods [1, 56, 16, 12] build the map in metric space and requires millions of training images, ATM is desirable because (1) it provides a metric-free option for exploration, and (2) it is lightweight (in terms of parameter size 16 M) and requires much less training data (just about 0.45 million images, in contrast with 10 million images required by most RL-based methods).

Original: The room scenes in MP3D dataset are much more complex and larger than those in Gibson dataset.
Revised: The room scenes in MP3D dataset are much more complex and larger than those in Gibson dataset.

Original: They contain various physical impediments (e.g. complex layout, furniture),  some room scenes even contain outdoor scenarios.
Revised: They contain various physical impediments (e.g. complex layout, furniture), and some rooms contain outdoor scenarios.

Original: Hence, ATM exhibits stronger generalizability in exploring novel scenes than RL-based methods .
Revised: Hence, ATM exhibits stronger generalizability in exploring novel scenes than RL-based methods .

Original: Moreover, performance gain is more obvious on both Gibson and MP3D datasets when we change the agent to a different locomotion setup (from 0.25/ 10 ◦ to 0.30/ 30 ◦ ), which also shows ATM is robust to different locomotion setups.
Revised: Moreover, performance gain is more obvious on both Gibson and MP3D datasets when we change the agent to a different locomotion setup (from 0.25/ 10 ◦ to 0.30/ 30 ◦ ), which also shows ATM is robust to different locomotion setups.

Original: On Gibson dataset, ATM achieves lower coverage ratio than ANS [8] but higher average covered area.
Revised: On Gibson dataset, ATM achieves lower coverage ratio than ANS [12] but higher average covered area.

Original: We find such performance difference is mainly caused by ATM stronger capability in exploring large areas than RL-based methods.
Revised: We find such performance difference is mainly caused by ATM stronger capability in exploring large areas than RL-based methods.

Original: In most cases, ATM actively reaches to new area within limited steps.
Revised: In most cases, ATM actively reaches to new area within limited steps.

Original: This result is also echoed by the coverage ratio curve between ATM and ANS [8] (Fig.
Revised: This result is also echoed by the coverage ratio curve between ATM and ANS [12] (Fig.

Original: III in Appendix C), from which we can see ATM quickly increases the coverage ratio during the early exploration stage (higher curve slope).
Revised: III in Appendix C), from which we can see ATM quickly increases the coverage ratio during the early exploration stage (higher curve slope).

Original: If we use a different locomotion setup (0.30/ 30 ◦ ), ATM coverage ratio stays as highest along the exploration process (left-most sub-figure of Fig. III).[Table] [Figure]
Revised: If we use a different locomotion setup (0.30/ 30 ◦ ), ATM coverage ratio stays as highest along the exploration process (left-most sub-figure of Fig. III).

Original: Comparison with random exploration.
Revised: Comparison with random exploration.

Original: RandomWalk serves as the baseline for our framework
Revised: RandomWalk serves as the baseline for our framework.

Original: It is also adopted by SPTM [48] to build topological map.
Revised: It is also adopted by SPTM [58] to build topological map.

Original: It involves no learning procedure and the agent randomly takes an action at each step to explore an environment.
Revised: It involves no learning procedure and the agent randomly takes an action at each step to explore an environment.

Original: From Table 1, we can see that 
Revised: From Table 1, we can see that RandomWalk

Revised: Under review as a conference paper at ICLR

Revised: [Table]

Original: RandomWalk dramatically reduces the exploration performance in terms of both coverage ratio and average coverage area.
Revised: [Table] [Table] [Figure] dramatically reduces the exploration performance in terms of both coverage ratio and average coverage area.

Original: The inferior performance of RandomWalk verifies the necessity of learning active exploration strategy in order to help agent to efficiently explore an environment.
Revised: The inferior performance of RandomWalk verifies the necessity of learning active exploration strategy in order to help agent to efficiently explore an environment.

Original: Figuredemonstrates the qualitative comparison between RandomWalk and ATM exploration result.
Revised: Figure 4 demonstrates the qualitative comparison between RandomWalk and ATM exploration result.

Original: Feature regularization and full history memory.
Revised: Feature regularization and with history memory .

Original: If we replace feature regularization involved in TaskPlanner with action regularization ( ATM_LSTMActRegu ), we have observed more performance drop on MP3D than on Gibson dataset ( 3% versus 0 .
Revised: If we replace feature regularization involved in TaskPlanner with action regularization ( ATM_LSTMActRegu ), we have observed more performance drop on MP3D than on Gibson dataset ( 3% versus 0 .

Original: 2% ), which shows adopting feature regularization improves the generalizability compared with action regularization.
Revised: 2% ), which shows adopting feature regularization improves the generalizability compared with action regularization.

Original: Moreover, introducing full history memory ( ATM_FullHistory ) to TaskPlanner (used as LSTM hidden state input) slightly improves the performance on Gibson dataset (less than 0 . 5% improvement), but significantly reduces the performance on MP3D dataset (more than 2% drop).
Revised: Moreover, introducing full history memory ( ATM_FullHistory ) to TaskPlanner (used as LSTM hidden state input) produces very similar results on Gibson dataset, but significantly reduces the performance on MP3D dataset (more than 2% drop).

Original: It thus shows using historical memory tends to encourage ATM to overfit training data, so that its generalizability is inevitably reduced.
Revised: It thus shows using historical memory tends to encourage ATM to overfit training data, so that its generalizability is inevitably reduced.

Original: We argue that such generalizability drop might lie in our over-simplified history memory modelling because we just evenly sample 10 nodes (image observations) from all historically visited nodes, which might be too simple to represent the whole history memory, or even confuses TaskPlanner if the agent has already explored many steps.
Revised: We argue that such generalizability drop might lie in our over-simplified history memory modelling because we just evenly sample 10 nodes (image observations) from all historically visited nodes, which might be too simple to represent the whole history memory, or even confuses TaskPlanner if the agent has already explored many steps.

Original: More elegant long-term history memory model remains to be explored.
Revised: More elegant long-term history memory model remains to be explored.

Original: Deeply-supervised learning and joint task and motion imitation.
Revised: Deeply-supervised learning and joint task and motion imitation.

Original: Removing deeply-supervised learning ( ATM_noDeepSup ) leads to obvious performance drop on both Gibson and MP3D dataset.
Revised: Removing deeply-supervised learning ( ATM_noDeepSup ) leads to obvious performance drop on both Gibson and MP3D dataset.

Original: In MP3D datast, it evens leads to worse performance than RandomWalk .
Revised: In MP3D datast, it evens leads to worse performance than RandomWalk .

Original: It thus shows the necessity of deep supervision in both feature space ( TaskPlanner ) and action space ( MotionPlanner ).
Revised: It thus shows the necessity of deep supervision in both feature space ( TaskPlanner ) and action space ( MotionPlanner ).

Original: Meanwhile, ATM_noFeatHallu leads to significant performance drop on both Gibson and MP3D datasets.
Revised: Meanwhile, ATM_noFeatHallu leads to significant performance drop on both Gibson and MP3D datasets.

Original: It thus 
Revised: It thus attests the advantage of our feature-space task and motion imitation strategy which jointly optimize TaskPlanner for high-level task allocation and MotionPlanner for low-level motion control.

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: attests the advantage of our feature-space task and motion imitation strategy which jointly optimize

Original: TaskPlanner for high-level task allocation and MotionPlanner for low-level motion control.

Original: We also visualize the comparison between ATM hallucinated next-step future feature and truly observed feature in Appendix Fig.
Revised: We also visualize the comparison between ATM hallucinated next-step future feature and truly observed feature in Appendix Fig.

Original: II (C).
Revised: II (C).

Original: We see that the hallucinated feature is more similar with the observed real feature when the agent is walking through a spacious area (in other words, the agent mostly takes move_forward action), than when the agent is walking along room corner, against the wall or through narrow pathway.
Revised: We see that the hallucinated feature is more similar with the observed real feature when the agent is walking through a spacious area (in other words, the agent mostly takes move_forward action), than when the agent is walking along room corner, against the wall or through narrow pathway.

Original: This may be due to the learned TaskPlanner most likely hallucinates feature moving the agent forward if the temporary egocentric environment allows.
Revised: This may be due to the learned TaskPlanner most likely hallucinates feature moving the agent forward if the temporary egocentric environment allows.

Original: This also matches expert exploration experience because experts (like humans) mostly prefer moving forward so as to explore as more area as possible.
Revised: This also matches expert exploration experience because experts (like humans) mostly prefer moving forward so as to explore as more area as possible.

Original: 5. E VALUATION R ESULTS ON N AVIGATION
Revised: 5. E VALUATION R ESULTS ON N AVIGATION

Original: The navigation result is in Table 2.
Revised: The navigation result is in Table 2.

Original: We compare ATM with most of methods compared in exploration task.
Revised: We compare ATM with most of methods compared in exploration task.

Original: CMP [23] builds up a topdown belief map for joint planning and mapping.
Revised: CMP [28] builds up a topdown belief map for joint planning and mapping.

Original: For OccAnt [55], we just report its result with model trained with RGB image (so as to be directly comparable with ATM).
Revised: For OccAnt [65], we just report its result with model trained with RGB image (so as to be directly comparable with ATM).

Original: We can see that ATM outperforms all comparing methods on the two datasets, with the largest performance gain on MP3D dataset (about 14% Succ.
Revised: We can see that ATM outperforms all comparing methods on the two datasets, with the largest performance gain on MP3D dataset (about 14% Succ.

Original: Rate, 12% SPL improvement).
Revised: Rate, 12% SPL improvement).

Original: Hence, we can conclude that our built topological map can be used  for image-goal navigation especially in large-scale scenes.
Revised: Hence, we can conclude that our built topological map can be used effectively for image-goal navigation.

Original: More importantly, ATM exhibits satisfactory generalizability in navigation owing to both active exploration in feature space and topological mapping in VPR.
Revised: More importantly, ATM exhibits satisfactory generalizability in navigation owing to both active exploration in feature space and topological mapping in VPR.

Original: In Fig.
Revised: In Fig.

Original: II (B), we can see VPR and ActionAssigner successfully add new edges (purple lines) to spatial-adjacent RGB images (nodes), resulting in a spatial-adjacency representative topological map that fits for navigation.
Revised: II (B), we can see VPR and ActionAssigner successfully add new edges (purple lines) to spatial-adjacent RGB images (nodes), resulting in a spatial-adjacency representative topological map.

Original: More detailed discussion on navigation is given in Appendix B.
Revised: More detailed discussion on navigation is given in Appendix B.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We propose a novel active topological mapping (ATM) framework for visual navigation, which is lightweight and computationally efficient.
Revised: We propose a novel active topological mapping (ATM) framework for visual navigation, which is lightweight and computationally efficient.

Original: Our ATM’s active exploration is metric-free, and can be learned via deeply-supervised task and motion imitation learning with superior data efficiency and generalizability compared to other RL-based methods.
Revised: Our ATM’s active exploration is metric-free, and can be learned via deeplysupervised task and motion imitation learning with superior data efficiency and generalizability compared to other RL-based methods.

Original: We also provide some real-world experiments to show our sim2real transferablity (in Appendix G).
Revised: We also provide some real-world experiments to show our sim2real transferablity (in Appendix H).

Original: Future works include designing elaborate historic memory modules and involving multi-modality sensors to further improve the performance of visual exploration and navigation. 
Revised: Future works include designing elaborate historic memory modules and involving multi-modality sensors to further improve the performance of visual exploration and navigation. Under review as a conference paper at ICLR

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: Please refer to Section. 5.1.
Revised: Please refer to Section. 5.1.

Original: on how to reproduce our expert demonstration dateset.
Revised: on how to reproduce our expert demonstration dateset.

Original: Appendix.
Revised: Appendix.

Original: D. provides details on how to implement all the neural networks used in our work, and Appendix.
Revised: E. provides details on how to implement all the neural networks used in our work, and Appendix.

Original: E. provides details on how to implement the VPR system used in the topological map construction
Revised: F. provides details on how to implement the VPR system used in the topological map construction.

Revised: All the code used in this paper are shared through Supplementary Material.

Original: A PPENDIX A M ORE R ESULTS ON E XPLORATION
Revised: A PPENDIX A M ORE R ESULTS ON E XPLORATION

Original: We show an extra exploration result on MP3D [6] q9vSo1VnCiC room scene in Fig.
Revised: We show an extra exploration result on MP3D [9] q9vSo1VnCiC room scene in Fig.

Original: I.
Revised: I.

Original: In this figure, we show both the room scene 3D visualization in top row and multiple exploration results in the two bottom rows.
Revised: In this figure, we show both the room scene 3D visualization in top row and multiple exploration results in the two bottom rows.

Original: Specifically, we show three efficient exploration results in the middle row, in which the agent begins to explore at different positions (marked by small red rectangle patch).
Revised: Specifically, we show three efficient exploration results in the middle row, in which the agent begins to explore at different positions (marked by small red rectangle patch).

Original: We further show two less-effective exploration results in the bottom row (last two sub-figures), which are mainly caused by repetitive visiting within a local area (yellow-to-red trajectory).
Revised: We further show two less-effective exploration results in the bottom row (last two sub-figures), which are mainly caused by repetitive visiting within a local area (yellow-to-red trajectory).

Original: A. Efficient 
Revised: A. Efficient Exploration

Original: [Figure]
Revised: [Figure]

Revised: B. Efficient Exploration C. Efficient Exploration D. Inefficient Exploration E. Local Repetitive Exploration F. Local Repetitive Exploration 3D Room Visualization

Original: B M ORE R ESULTS ON N AVIGATION
Revised: B M ORE R ESULTS ON N AVIGATION

Original: Visualizations of topological mapping.
Revised: Visualizations of topological mapping.

Original: We show a qualitative example of ATM in Fig.
Revised: We show a qualitative example of ATM in Fig.

Original: II.
Revised: II.

Original: We can see from exploration trajectory that agent actively traverses all areas across multiple rooms in the environment scene, VPR and ActionAssigner sucessfully add new edges (purple lines in the middle figure) to spatial-adjacent nodes, resulting in a complete and robust topological map.
Revised: We can see from exploration trajectory that agent actively traverses all areas across multiple rooms in the environment scene, VPR and ActionAssigner sucessfully add new edges (purple lines in the middle figure) to spatial-adjacent nodes, resulting in a complete and robust topological map.

Original: C C OVERAGE R ATIO C URVE
Revised: C C OVERAGE R ATIO C URVE

Original: The coverage ratio curve over 1000-step budget between ATM (two locomotion settings) and ANS [8] is shown in Fig.
Revised: The coverage ratio curve over 1000-step budget between ATM (two locomotion settings) and ANS [12] is shown in Fig.

Original: III, from which we can see ATM quickly increases the coverage ratio during the early exploration stage..
Revised: III, from which we can see ATM quickly increases the coverage ratio during the early exploration stage.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: D N EURAL N ETWORK A RCHITECTURE D. ATM N EURAL N ETWORK A RCHITECTURE
Revised: D M ORE D ETAILS ON E XPERT D EMONSTRATION

Original: Our ATM topological mapping neural network architecture is given in Table I, it consists of ResNet18 (as the image embedding module), LSTM layer (for task planner) and FC layer (for task planner and motion planner).
Revised: We sample a set of anchor points across all navigable area in the each room scene, then call Habitat-lab Pathfinder API to let the agent traverse all anchor points to cover the whole area.

Original: ResNet [Table]
Revised: Pathfinder API generates the most economic action sequence (following shortest-path geodesic path) leading the agent from one anchor point to another. In practice, we use the provided episodes’ start/end points in Gibson as the anchor points since they densely cover all the navigable area (see Fig. 5). We collected 450 k training images and the associated actions in 72 scenes from Gibson, which are used to train both ATM and ActionAssigner

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure]
Revised: Our ATM topological mapping neural network architecture is given in Table I, it consists of ResNet18 (as the image embedding module), LSTM layer (for task planner) and FC layer (for task planner and motion planner).

Original: D. A CTION A SSIGNER N ETWORK A RCHITECTURE
Revised: E. A CTION A SSIGNER N ETWORK A RCHITECTURE

Original: The ActionAssigner neural network is given in Table II.
Revised: The ActionAssigner neural network is given in Table II.

Original: The ActionAssigner also uses ResNet as the image embedding layer.
Revised: The ActionAssigner also uses ResNet18 as the image embedding layer.

Original: Then it uses a sequence of FC layers to predict a sequence of actions separately (sequence length is 6).
Revised: Then it uses a sequence of FC layers to predict a sequence of actions separately (sequence length is 6).

Original: Bidirectional LSTM is applied to model actions mutual dependencies.
Revised: Bidirectional LSTM is applied to model actions mutual dependencies.

Original: The parameter size 13 .
Revised: The parameter size 13 .

Original: 5 M. We train ActionAssigner with the same parameter setting as the famrwork of TaskPlanner and MotionPlanner (the network in Table I).
Revised: 5 M. We train ActionAssigner with the same parameter setting as the famrwork of TaskPlanner and MotionPlanner (the network in Table I).

Original: E M ORE D ETAILS ON V ISUAL P LACE R ECOGNITION E. D ETAILS ON THE I MPLEMENTATIONOF VLAD-B ASED VPR AND A CTION A SSIGNER
Revised: Under review as a conference paper at ICLR [Table] [Table] F M ORE D ETAILS ON V ISUAL P LACE R ECOGNITION F. D ETAILS ON THE I MPLEMENTATION

Original:  For VPR, we first extract SIFT[40] features from all the images captured during the exploration episode, and then compute VLAD feature for each image based on the clustered SIFT as described in [28].
Revised: AND A CTION SSIGNER For VPR, we first extract SIFT[50] features from all the images captured during the exploration episode, and then compute VLAD feature for each image based on the clustered SIFT as described in [34].

Original: This process involves running K-Means [38] on all the SIFT features aggregated.
Revised: This process involves running K-Means [48] on all the SIFT features aggregated.

Original: We used Scikit-Learn [51]’s K-Means implementation with n_clusters=16, n_init=1 .
Revised: We used Scikit-Learn [62]’s K-Means implementation with n_clusters=16, n_init=1 .

Original: We then store all the VLAD features into a ball tree [49, 37] with leaf_size=60 and L 2 as distance metric.
Revised: We then store all the VLAD features into a ball tree [59, 47] with leaf_size=60 and L 2 as distance metric.

Original: Finally, for every node, we use its image’s VLAD feature to query the ball tree for 20 nearest neighbors as candidates.
Revised: Finally, for every node, we use its image’s VLAD feature to query the ball tree for 20 nearest neighbors as candidates.

Original: All the hyperparameters remains unchanged during all the testing, and we have not done any hyper-parameter searching.
Revised: All the hyperparameters remains unchanged during all the testing, and we have not done any hyper-parameter searching.

Original: We show the VPR result in Fig.
Revised: We show the VPR result in Fig.

Original: IV where C. is the candidates picked by the VPR process and we can see VPR can successfully reflects image observations’ spatial closeness.
Revised: IV where C. is the candidates picked by the VPR process and we can see VPR can successfully reflects image observations’ spatial closeness.

Original: In practice, we again use Habitat-lab pathfinder to get the ground truth actions between any two VPR-connected image observations.
Revised: In practice, we again use Habitat-lab pathfinder to get the ground truth actions between any two VPR-connected image observations.

Original: If the obtained action list length is smaller than six, we append STOP action in the end.
Revised: If the obtained action list length is smaller than six, we append STOP action in the end.

Original: If the obtained action list length is larger than six, we remove the connection.
Revised: If the obtained action list length is larger than six, we remove the connection.

Original: In navigation task, the agent just stops once it sees the predicted STOP action from the inferred action sequence by the ActionAssigner .
Revised: In navigation task, the agent just stops once it sees the predicted STOP action from the inferred action sequence by the ActionAssigner .

Original: E. T IME E FFICIENCY OF VPR
Revised: F. T IME E FFICIENCY OF VPR

Original: [Table]
Revised: In Table.

Original: In this experiment, We implemented the retrieval network in the SPTM [48] to do classification between every two image on whether they should be connected.
Revised: III, we demonstrate the time efficiency superiority of our VPR system. The “Time Spent” indicates the amount of time each method takes to finish retrieving all node pairs that should be connected (to add edge) in a room. Specifically, we report the average time cost across all the 14 test rooms in the Gibson dataset (excluding ActionAssigner time cost). The resulting maps after spatial-adjacency connection created by the three methods are almost identical.

Revised: We can see that our proposed VPR-based method has the minimal time cost (or time complexity).

Revised: [Figure]

Original: The network architecture and hyper-parameters are the same as in the paper, with some small tweaks to the architecture’s dimension due to the fact the original paper is done in a low-resolution video game while our testing environment is a photo-realistic dataset.
Revised: Methods [Equation] In this experiment, We implemented retrieval network SPTM [58] they in the paper, with some small tweaks to the architecture’s dimension due to the fact the original paper is done in a low-resolution video game while our testing environment is a photo-realistic dataset.

Original: The memory graph in SPTM [48] is similar to the concept of topological graph in our setting, and SPTM locomotion network is similar to ourActionAssigner .
Revised: The memory graph in SPTM [58] is similar to the concept of topological graph in our setting, and SPTM locomotion network is similar to our

Original: The Brute-Force Geometric Verification method exhaustively apply geometric verification on all image pairs.
Revised: The Brute-Force Geometric Verification method exhaustively apply geometric verification on all image pairs.

Original: Its time cost can be potentially reduced by turning to parallelism computation.
Revised: Its time cost can be potentially reduced by turning to parallelism computation.

Original: All the experiments are done with 10 cores of a Intel Xeon Platinum 8268 (205W, 2.9GHz), 32GB RAM and a SSD.
Revised: All the experiments are done with 10 cores of a Intel Xeon Platinum 8268 (205W, 2.9GHz), 32GB RAM and a SSD.

Original: SPTM [48] additionally run on a Nvidia V100 GPU with a batch size of 160.
Revised: SPTM [58] additionally run on a Nvidia V100 GPU with a batch size of 160.

Original: F E XPLORATION E FFICIENCY D ISCUSSION
Revised: G E XPLORATION E FFICIENCY D ISCUSSION

Original: [Figure]

Original: e

Original: We can see that ATM inevitably leads to repetitive visiting of area.

Original: We argue

Original: “repetitive visiting” is mainly caused by most recently visited short image observation sequence to predict the action)

Original: ATM global global updated

Original: An ideal exploration should constantly keep reaching to new area, like the expert exploration does.
Revised: An ideal exploration should constantly keep reaching to new area, like the expert exploration does.

Original: We find our proposed ATM framework sometimes leads to inefficient exploration: repetitively visiting a local area.
Revised: We find our proposed ATM framework sometimes leads to inefficient exploration: repetitively visiting a local area.

Original: We show three examples of such inefficient exploration on Gibson Swormville environment scene in Fig.
Revised: We show three examples of such inefficient exploration on Gibson Swormville environment scene in Fig.

Original: V.
Revised: V.

Original: Our introduced random action perturbation strategy can reduce the “repetitive visiting” dilemma to some extent.
Revised: Our introduced random action perturbation strategy can reduce the “repetitive visiting” dilemma to some extent.

Original: According to our understanding, such “repetitive visiting” is mainly caused by ATM “short-memory” when predicting the next action to take, and involvement of no global environment scene information, including the whole historically visited memory (scene memory) and topdown map (telling the agent if the area to explore has already been visited), to supervised the whole exploration process.
Revised: According to our understanding, such “repetitive visiting” is mainly caused by ATM “short-memory” when predicting the next action to take, and involvement of no global environment scene information, including the whole historically visited memory (scene memory) and topdown map (telling the agent if the area to explore has already been visited), to supervised the whole exploration process.

Original: We do not involve such extra supervision because we want to keep our framework as simple and lightweight as possible.
Revised: We do not involve such extra supervision because we want to keep our framework as simple and lightweight as possible.

Original: It remains as a future research direction to explore how to involve extra information to achieve more efficient exploration.
Revised: It remains as a future research direction to explore how to involve extra information to achieve more efficient exploration.

Original: We also provide several video (in which left image shows the panoramic image observation and the right image shows the corresponding exploration trajectory update on the top-down map.) 
Revised: We also provide several video (in which left image shows the panoramic image observation and the right image shows the corresponding exploration trajectory update on the top-down map.) Under review as a conference paper at ICLR [Figure]

Original: G M ORE D ETAILS ON R EAL -W ORLD H ARDWARE E XPERIMENT G. H ARDWARE D ETAILS
Revised: ill e H M ORE D ETAILS ON R EAL -W ORLD H ARDWARE E XPERIMENT H. H ARDWARE D ETAILS

Original: See Fig.
Revised: See Fig.

Original: VI.
Revised: VI.

Original: for the physical setup of the robot we built.
Revised: for the physical setup of the robot we built.

Original: We added an aluminium frame on top of a iRobot Create 2 to host a Nvidia Jetson TX2 as the processor and an Insta 360 Pro2 to capture panoramic images.
Revised: We added an aluminium frame on top of a iRobot Create 2 to host a Nvidia Jetson TX2 as the processor and an Insta 360 Pro2 to capture panoramic images.

Original: The turning angle ( 10 deg ), step length(0.25m), camera height(1.5m) and other important parameters are designed to be as close to the configuration in the simulation as possible.
Revised: The turning angle ( 10 deg ), step length(0.25m), camera height(1.5m) and other important parameters are designed to be as close to the configuration in the simulation as possible.

Original: An example image captured by the camera is given in VIII.
Revised: An example image captured by the camera is given in VIII.

Original: G. R EAL -W ORLD E XPERIMENT D ETAILS
Revised: H. R EAL -W ORLD E XPERIMENT D ETAILS

Original: With the help of our ATM model (with no real-world fine-tuning), we conducted a preliminary experiment using this robot to explore an indoor environment.
Revised: With the help of our ATM model (with no real-world fine-tuning), we conducted a preliminary experiment using this robot to explore an indoor environment.

Original: The result is in VII The step budget is capped at 50 steps (vs. 1000 in simulation) due to limitations in battery capacity and prolonged execution time caused by the fact that the camera does real-time onboard images stitching to produce panoramic images.
Revised: The result is in VII The step budget is capped at 50 steps (vs. 1000 in simulation) due to limitations in battery capacity and prolonged execution time caused by the fact that the camera does real-time onboard images stitching to produce panoramic images.

Original: We are continuously improving the hardware system by finding better way to capture panoramic images and enhancing the stability of the camera by adding more support structures, and hope to deliver more hardware experiment results.
Revised: We are continuously improving the hardware system by finding better way to capture panoramic images and enhancing the stability of the camera by adding more support structures, and hope to deliver more hardware experiment results.