Active Topological Mapping by Metric-Free Exploration via Task and Motion Imitation
Authors: Yuhang He
Irving Fang
Yiming Li
Chen Feng
Venue: Submitted to ICLR 2023 (2023)
Keywords: Topological Mapping
Feature-Space Task and Motion Planning
Visual Navigation
Deeply-Supervised Learning

Abstract: Topological map is an effective environment representation for visual navigation. It is a graph of image nodes and spatial neighborhood edges without metric information such as global or relative agent poses. However, currently such a map construction relies on either less-efficient random exploration or more demanding training involving metric information. To overcome these issues, we propose active topological mapping (ATM), consisting of an active visual exploration and a topological mapping by visual place recognition. Our main novelty is the simple and lightweight active exploration policy that works entirely in the image feature space involving no metric information. More specifically, ATM's metric-free exploration is based on task and motion planning (TAMP). The task planner is a recurrent neural network using the latest local image observation sequence to hallucinate a feature as the next-step best exploration goal. The motion planner then fuses the current and the hallucinated feature to generate an action taking the agent towards the hallucinated feature goal. The two planners are jointly trained via deeply-supervised imitation learning from expert exploration demonstrations. Extensive experiments in both exploration and navigation tasks on the photo-realistic Gibson and MP3D datasets validate ATM's effectiveness and generalizability.

Under review as a conference paper at ICLR 2023
ACTIVE TOPOLOGICAL MAPPING BY METRIC-FREE EXPLO- RATION VIA TASK AND MOTION IMITATION
Anonymous authors Paper under double-blind review
ABSTRACT
Topological map is effective environment representation for visual navigation. It is graph of image nodes and spatial neighborhood edges without metric information such as global relative agent poses. However, currently such map construction relies on either less-efficient random exploration or more demanding training involving metric information. To overcome these issues, we propose active topological mapping (ATM), consisting of an active visual exploration and a topological mapping by visual place recognition. Our main novelty is the simple and lightweight active exploration policy that works entirely in the image feature space involving no metric information. More specifically, ATM's metric-free exploration is based on task and motion planning (TAMP). The task planner is recurrent neural network using the latest local image observation sequence to hallucinate a feature as the next-step best exploration goal. The motion planner then fuses the current and the hallucinated feature generate action taking the agent towards the hallucinated feature goal. The two planners are jointly trained via imitation learning from expert exploration demonstrations. Extensive experiments in both exploration and navigation tasks on the photo-realistic Gibson and MP3D datasets validate ATM's effectiveness and generalizability.
INTRODUCTION
Mobile agents often create maps to represent their surrounding environments [6]. Typically, such map is either topological or metrical (including hybrid ones). We consider a topological map to be metric-free free, which means it does not explicitly store global/relative position/orientation information with measurable geometrical accuracy [39, 38). Instead, it is graph that stores local sensor observations, such as RGB images, as graph nodes and the spatial neighborhood structure (and often navigation actions) as graph edges that connects observations taken from nearby locations. While metric maps are often reconstructed by optimizing geometric constraints between landmarks and sensor poses from classic simultaneous localization and mapping (SLAM), topological maps have recently attracted attention in visual navigation tasks due to the simplicity, flexibility, scalability, and interpretability [58, 13, 27, 40, 12].
A topological map used for visual navigation could be constructed in two ways. The first and simplest way is to let the agent explore the new environment through metric-free random walks, after which the map could be built by projecting the recorded observations into feature space and adding edges between nearby or sequentially obtained features [58]. However random walk is very inefficient especially in large or complex rooms, leading to repeated revisits of nearby locations in the same area. The other way is to design a navigation policy that controls the agent to more effectively explore the area while creating the map. It is known active SLAM and often involves some metric information either required input [42, 12] or intermediate estimations [13]. As shown in Fig. 1, could we combine the merits of the two ways by finding a metric-free (neither input nor estimates) exploration policy that discovers informative traversing trajectories in unknown environments for topological map construction after exploration?
To achieve this objective, we propose Active Topological Mapping (ATM) as shown in Fig. It contains two stages: active exploration through learned metric-free policy, and topological mapping through visual place recognition (VPR) [5]]. The first stage adopts the task and motion planning formalism (TAMP) [26, 55] and imitation learning [63] from expert demonstrations which could come from either oracle policy having full access to virtual environments, or simply a human expert in real world. Our main novelty is to design such an imitation at both the task and the motion levels with joint end-to-end training. Our task planner, a two-layer LSTM [31] network trained with deep supervision, conceives the next best goal feature to be

---

Under review as conference paper at ICLR 2023
Where to explore next?
BG
BGBB
Topological Strangy
Motrical
Stratogy
C
BG
Ap
Sp:
turn
forward
move
right
Soeward
Hallocanated
Metric Space
space
Peature
Figure I: Problem overview. We focus on the active mapping problem where mobile agent needs decide how to efficiently explore a novel environment. For planning and navigation, we embrace the topological feature space where each feature corresponds distance/pose information which is onerous to obtain accurately. Our main idea is to hallucinate goal features guide exploration actions, learned by imitating expert demonstrations.
an image observation, while the metric space involves
explored by hallucination from the current and historical image features. Our motion planner, a simple multi-layer perceptron, fuses the current and the hallucinated features and generates the best action that will move the agent to location whose feature is closer to the goal.
The second stage of ATM takes all observations recorded during the active exploration stage to create the topological map. This stage could be solved similar to [58], where nodes are first connected by the sequential visiting order, and then additional node connections are discovered by a binary classifier estimating the spatial adjacency between two nodes through their image similarity. Differently, we adopt VPR, classic technique SLAM for loop closure detection, to discover additional edges more effectively. We further train an action assigner assign each newly-added edge with corresponding actions that will move the agent between the two connected nodes. Finally, the topological map becomes our efficient environment representation for visual navigation in [58]-
We validate the efficacy of ATM on two tasks: exploration in which the goal is to maximize the explored area within a fixed step budget, and navigation in which the goal is to use ATM-constructed topological map navigate the agent to target image. In summary, our contributions are:
We propose simple and effective framework named as active topological mapping (ATM) for efficient and lightweight visual exploration. The topological map it constructs can be used for efficient visual navigation.
We develop joint trainable feature-space task and motion planning (TAMP) networks achieve metric-free and generalizable exploration.
We design a supervisd imitation learning strategy to train the feature-space TAMP networks with better data efficiency.
We validate our method on the photo-realistic Gibson [72] and MP3D [9] datasets in both visual exploration and navigation.
2 RELATED WORK
Topological map in exploration and navigation. Previous works tried to tackle navigation with end- to-end learning of sensorimotor control by directly mapping visual observations to the action space [57]. However, such purely reactive RL-based methods that have no explicit memory struggle navigate in complex scenarios [20, 74J. Newer methods that tackle this problem with scene memory [23, 29] often rely on metric information. An explicit metric map commonly used for localization and navigation in the literature [22, 21, 6]. but may face robustness and computation challenges, especially in dynamic and complex scenes, due the need for accurate geometric constraints during the map and pose optimization. Later, inspired by the animal and human psychology [70], researchers show that topological map may aid robot navigation [13, 15, 11, 29]. literature, many topological mapping solutions either uses a random walkthrough sequence [58], or incrementally constructs a topological graph during the navigation task [13, 40]. However, random exploration is inefficient in creating a comprehensive map given a limited exploration time. And the existing input or as intermediate estimation. Instead, we propose a two-stage solution that (1) learns an efficient and generalizable exploration policy to collect visual observations of novel environment, and (2) uses VPR to construct a topological map for future navigation. Similar cettoratiion-effo---naaaee
solutions still involve metric information either as required
pipelines include [5]
2

---

Under review as a conference paper at ICLR 2023
Regaor
Emage
Image
BesNet
Task Plaznor
Motions Planter
Action
Mag Inittalization
Map Completine. via VPS
Active Exploration In Teature Space
Topological Mapping
Figure 2: Workflow of ATM. ATM consists of two stages: (1) active exploration of a novel environment by task and motion planning in feature space; and (2) topological mapping composed of initialization via temporal adjacency and completion via visual place recognition (VPR).
and [54], yet they still need metric information to create the topological map and have lower data efficiency due to the use of RL.
Exploration in navigation. While various studies have focused on passive map-building and target-driven navigation policy learning, active SLAM has received less attention: how to actively control the camera for map construction [25]. Recent works [35, 14, 12] use learning- based methods to solve the problem of ea
active exploration. [12] learns a hierarchical policy for active exploration via both classic and RL methods. [16] learns policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards. [66] proposes a curiosity- RI. method which uses episodic memory to form the novelty bonus. Different from existing methods, propose task and motion imitation which learns metric-free, generalizable, and data-efficient exploration.
Hallucinating future latent feature. The idea of hallucinating future latent features has been discussed in other application domains. Previous work has utilized this idea of visual anticipation in video predic- tion/human action prediction [71, 73, 10, 24, 68), and researchers have applied similar ideas to robot motion and path planning [33, 37, 8, 60]. As stated in [71, 73, 68], visual features in the latent space provides an efficient way to encode semantic/high-leve information of scenes, allowing to do planning in the latent space, which is considered more computationally efficient when dealing with eigh-dimensional data as input [46, 32]. Different from previous robotics work, we take advantage of this efficient representation by adding deep supervision when anticipating the next visual feature, which was computationally intractable if we were to operate at the pixel level.
Learning has been extensively explored [41, 67, 44, 45] during the past several years. The main idea is to add extra supervisions to various intermediate layers of a deep neural network in order more effectively train deeper neural networks. In our work, we adopt similar idea to deeply supervise the training of feature hallucination and action generation
Task and motion planning. Task and motion planning (TAMP) divides robotic planning problem into: the high-level task allocation (task planning) and the low-level action for the task execution (motion planning). This framework is adopted in many robotic tasks such as manipulation [17, 53] exploration [7] and navigation [49, 69]. Such framework allows us to leverage high-level information about the scenes to tackle challenges in local control techniques [4]. In this work, to perform active topological mapping of novel environment, the agent firstly reasons at the highest level about the regions to navigate: hallucinate the next best feature point to visit. Afterward, the agent takes an action get to the target feature. The whole procedure is totally implemented in feature space without any metric information.
3 ACTIVE EXPLORATION BY IMITATION IN FEATURE SPACE
Our topological map is represented by graph G = AI, where the graph nodes denoted by I is a set of RGB panoramic image observations collected by the agent different locations I {4, I..... INI (where N denotes the number of nodes), and the edges denoted by A is composed of a set of actions a(r,,,)) E A which connects two spatially adjacent observations 7, and 1,. this work, each RGB panoramic image is of size 256 x 512, and the action space consists of three basic actions: move_ move_forwar ffr_forrrrd,.r turn_ _eef_. and turr_ surr_right.. Active exploration aims at maximizing the topological map coverage over an environment given certain step budget N. The coverage of the topological map denoted by C is defined as the total area in the map that is known to be traversable or non-traversable. Mathematically. let Ty denote the policy network parameterized by 6, a, denote the action taken at step t, and AC(a)) denote the gain in coverage introduced by taking action a, the following objective function is optimized to obtain the optimal exploration policy n²:
3

---

Under review as conference paper at ICLR 2023
Task Planner
Motion Planner
Inference
I-B
7
f
OT
f2-10
f
Ey
4-10
I
Figure 3: Training and inference for task and motion imitation. Feature extractor Ie takes image as input and generates the corresponding feature vector ft. TaskPlanner Te, is a recurrent neural network (RNN) consuming a sequence of features (fr-1o. ft} to hallucinate the next best feature visit fe+1. Motion- Planner Te consumes the concatenation (denoted by θ of fe and fi+1 and generates the action move the agent towards the hallucinated feature. During training, we supervise all the intermediate outputs including the intermediate hallucinated features {fr-9,-.. ... fi) and the intermediate actions {āz-1u, - in addition to the final output fi+1 and å, During inference, current observation 1₃ is firstly encoded and fed into Ter hallucinate f and then fe+1 combined with the Jr is fed into Ta, for motion planning. Lr is L2 loss and LM is cross entropy loss (the subscripts T and M denote Task and Motion respectively). he denotes the hidden state of RNN.
arg max TO BENNT
AC(a))
(1)
Learning from expert demonstrations. In literature, most works solve Eq. (1) by reinforcement learning maximize the reward [16, 43), such solutions are not only data-hungry, but also require complicated training involving metric information. Differently, adopt imitation learning [63] let our policy network Ta mimic the output of the expert policy स which could come from either an oracle policy having full access to virtual environments, or simply human expert in real world. Hence, our objective is to minimize the difference between our policy network and the expert policy:
arg min L( 7),
where L measures the discrepancy between two policies. We propose novel task and motion imitation in feature space to solve Eq. (2) which will be introduced in the following sections. We respectively introduce the feature extraction (3.1), the policy network ne composed of TaskPlanner denoted by Ter (3.2) as well as a Motion denoted by Tox (3.3), and the seeplysuueeriseed learning strategy (3.4).
3.1 FEATURE EXTRACTION
We firstly encode each visual observation I E I(t 1,2, N) with a feature extractor Iv parameterized by s which uses the ImageNet [64] pre-trained ResNet18 backbone [30]. The feature embedding ft Rd = 512) is obtained by ft = ge (T+), (see Fig. 3). Note that Φy jointly optimized with the task planner Toy, well as the MotionPlanner ne via imitation learning.
3.2 TASK PLANNER FOR NEXT BEST FEATURE HALLUCINATION
PranPer THr parameterized by Or takes the most recent m-step visual features F = {ft-m,-. }t} as input, and learns to hallucinate the next best feature to visit which is denoted by fe+1, see Fig. 3. In specific, Joy is a two-layer LST [31]:
fet1 = Toy (Jt-my' Se(Or)
(3)
4

---

Under review as a conference paper at ICLR 2023
To save computation, &er only takes the most recent m-step features as input and we empirically find that m = 10 achieves good performance. In other words, TaskPlanner is only equipped with short-term scene memory, and tries to extend the feature space as quickly as possible in order guide the agent to perform efficient exploration.
3.3 MOTION PLANNER FOR ACTION GENERATION
MotionPlanner N parameterized by dM takes the hallucinated feature fe+1 and the current feature ft as input, and outputs the action taking the agent towards the hallucinated goal (see Fig. 3). Specifically, Tox, a multe-layer-eerreptronn (MLP) taking the concatenation of two features input to classify the action:
à no (fet1,
3.4 DESPLY-SUPEERVIED IMITATION LEARNING STRATEGY
Our imitation pipeline is shown Fig. Given an expert exploration demonstration including a sequence of images and the corresponding expert actions & = {(4 ai }, {12, a2). {IN, }x we adopt the deeply-sun superssupee learning strategy [41] to jointly optimize the feature extractor gw, task planner Tey, and Motion Planner Tex-. Ultimately, objective in Eq. (2) becomes,
N-1
Σ (erfe + Car 0.05.0 Pr
min
(5)
where Lr is L2 loss to measure the discrepancy between two features, and Ls is cross entropy loss to make the model imitate the expert action category. The desired target feature fi+1 is obtained by Jr+1 = go 1))+ (I++1 is obtained from the expert demonstration E), the desired action a, is also read from E, the hallucinated feature fi+1 is calculated by Eq. (3), and the generated action âr is computed by Eq. (4). For each training iteration,
randomly clip m observations and the corresponding m actions from an expert exploration (m = 10 and N >> m). and feed them to Ié, To,, and Tex Note that we supervise all the intermediate output, as illustrated in 3ig.
Remark. Different from other works, we decompose the exploration policy into task planner TØ, as well as MotionPlanneer na and jointly learn them by imitation in feature space. TaskPlanner hallucinates high-level features and MotionPlannee generates low-level actions. We supervise all the intermediate output of the network including task-level features ft, thus our TAMP imitation strategy can be considered as supervide learning [41].
4 TOPOLOGICAL MAPPING VIA VISUAL PLACE RECOGNITION
The topological map G (I initialized by the active exploration experience in Sec. 3 is unidirectionally connected in the temporal domain. Each node (a panoramic RGB image observation) is just connected with its preceding node and next node, failing discover spatial adjacency. We propose to further complete the initial map G by adding edges to the node pairs if two nodes in a pair possess high visual similarity. Specifically, we adopt visual place recognition (VPR) [34, 3] method and geometric verification measure "'visual similarity" of node pair.
Given room of interest and all the nodes' images, for every node, first used non-learning, VLAD-based VPR system to pick out other images that are very similar to this node as candidates for further geometric verification. After this VPR process, we conduct geometric verification by inspecting if the number of matching SIFT keypoints between the node's 's image and a candidate's 's image is larger than a threshold (in our case 100). Passing the threshold means that we will add edges between these two nodes. Meanwhile, we train a model named ActionAssigner to assign action list to each new edge. The architecture of ActionAssigner is similar to MotionPlanner yet ActionAssigner predicts sequence of actions with two node features as input, while MotionPlannee is one-step action predictor (predict just one action).
We adopted VPR based method mainly due to its time efficiency compared to methods involving brute force, such as that in SPTM [58] brute-foree geometric verification. See Appendix F for more details the VPR process we used, its time efficiency, and our implementation of SPTM.
5

---

Under review as a conference paper at ICLR 2023
Our completed topological map can perform image-goal navigation tasks. Given the image observations for the start and goal positions, we localize them on topological map by retrieving the node image with the highest inlier number (via VPR). Once localized, we apply Dijkstra's s algorithm [19] to find the shortest path between the two nodes. We can then smoothly navigate the agent from the start position to the goal position without metric information.
5 EXPERIMENTS
We test ATM on two datasets: Gibson [72] and Matterport3D (MP3D) [9] dataset on Habitat-lab platform [52]. The two datasets are collected in real indoor spaces by 3D scanning and reconstruction methods. The agents can be equipped with mudti-modality sensors to perform various robotic tasks. The average room size of MP3D (100 m²) is much larger than that of Gibson ([14m², 85m²)..
We run experiments on two tasks: (1) autonomous exploration proposed by Chen et al. [16], in which the target is to maximize environment coverage within fixed step budget (1000-step budget following [12],, and (2) image-goal navigation where the agent uses the constructed topological map to navigate from current observation to target observation. Regarding the exploration,
employ two evaluation metrics:
(I) coverage ratio which is the percentage of covered area over all navigable area, and (2) absolute covered area (m²). We exactly follow the setting by ANS [12] that a point is covered by the agent if it lies within the agent's uid-of-vieww and is less then 3.2m away. Regarding the navigation, we adopt two evaluation metries: shortest path length (SPL.) and success rate (Suce. Rate) [2]. We again follow ANS [12] to train ATM on Gibson training dataset (72 scenes), and test ATM on Gibson validation dataset (14 scenes) and MP3D test dataset (18 scenes). Using the ATM trained on Gibson dataset to test on MP3D dataset helps to show ATM's generalizability.
5.1 EXPERIMENT CONFIGURATION
Exploration setup. In exploration, we independently explore each scene 71 times, each time assigning the agent a random starting point (created by random seed number). We keep track of all the random seed numbers for result reproduction. We use the Habitat-lab sliding function so that the agent does not stop when it collides with wall, but instead slides along it. In order to generate the initial 10 steps required by ATM, we constantly let the agent to execute move_ forwar rd action. Once it collides the wall, it randomly chooses action to continue to explore. Afterwards, we iteratively call TaskPlanner and MotionPlanmer to actively explore the environment. During ATM ATM-guided exploration, we allow the agent to actively detect its distance with surrounding obstacles or walls. When the agent's forward-looking distance to obstacles is less than 2-step distances and the ATM predicted action is still move move_forware rd, the agent will turn to a new direction (by randomly iteratively executing turn left and turn x right action) with the longer forward-looking distance to new obstacles.
turr_left or wurr_rigitt
We experiment with two locomotion setups: the first one is with step-size 0.25 m and turning angle 10% which follows the same setting established in [12] for comparing with baseline methods in the exploration task. The second one is with step-size 0.30 m and turn-angle 30. This helps us test ATM's generalization capability under different locomotion configurations.
Navigation setup. In navigation, we encourage the agent to visit enough positions for each room scene. Specifically, the agent has collected 2,000 images per room on Gibson and 5,000 images per room on MP3D 2,.000/5,000-steg ATM-guided exploration).
Expert demonstrations. To generate expert demonstrations, we place so-called anchor points relatively evenly over all navigable areas in each training room. The agent starts at a random anchor point and walks to the closest anchor point with help from Habitat's API. We repeat this process until all the anchor points are visited. During this process, we record all the actions the agent takes, along with all the panoramic photos at each time step. Note that such a demonstration is not a strong demonstration, as this process is not directly related to the task of maximizing exploration or efficient navigation. For more details on expert demonstration, please refer to Appendix D and Fig. 5
Training details. In our implementation, the local observation sequence length is 10 (m=10) because we
find it has achieved good performance-memorry trade-off. We experimentally tested m = 20 and got inferior performance. ATM network architecture is in Appendix E (parameter size is just 16 M). We train ATM with PyTorch [61]. The optimizer is Adam [36] with an initial learning rate 0.0005, but decays every 40 epochs with the decaying rate 0.5. In total, we train 70 epochs. We train all the ATM variants with the same hyperparameter setting for fair comparison.

---

Under review as a conference paper at ICLR 2023
5.2 COMPARISON METHODS
For exploration task, we compare ATM with six RL-based methods: 1. RL + 3LConv: An RL Policy with 3 layer convolutional network [52]; 2. RL + Res18: RL Policy initialized with ResNet18 [30] and followed by GRU [18]; 3. RL + Res18 + AuxDepth: adapted from [56] which uses depth map prediction as an auxiliary task. The network architecture is the same as ANS [43] with one extra deconvolution layer for depth prediction; 4. RL + Res18 + ProjDepth adapted from Chen el al. [16] who project the depth image in an egocentric top-down in addition to the RGB image as input to the RL policy. 5. ANS (Active Neural SLAM [12D) jointly learns a local and global policy network to guide the agent to explore; 6. OccAnt [65]; takes RGB, depth and camera as input to learn 2D top-down occupancy map to help exploration. For ablation studies, we have following ATM variants:
1. Random Walk The agent randomly choose an action to execute at each step. It serves as a baseline and helps us to know agent exploration capability without any active learning process. Please note that Random Walk is also the SPTM [58] exploration strategy.
2. Haleaallll We use the architecture of TaskPlanner to directly predict the next action. It discards task planning in feature space, but instead plans directly in action space. Its performance helps us understand if the hallucinated feature is truly necessary.
3. ATM_ withHistory. ATM is trained with only a short-memory (the latest m steps observations). To validate the influence of long-term memory, information: we evenly extract 10 observations among all historically explored observations excluding the latest m steps. After feeding them to ResNet18 [30] to get their embedding, we simply use average pooling to get one 51 512-dimensional vector and feed it to TaskPlanner LSTM as the hidden state input.
train new ATM variant by adding extra historical
WTM_LSTMActRegU TaskPlanner hallucinates a next-best feature at each time step to deeply super- vise (or regularize) the whole framework in feature space. As an alternative, we can predict action instead at each time step (regularize action). This ATM variant helps us to figure out whether supervising cach step of TaskPlanner LSTM in feature space is helpful.
5. ATM noDeepSup. ATM without decply-supervised learning. We remove LSTM per-frame feature supervision in TaskPlanner and neighboring frame action supervision in MotionPlanner. In other words, we just keep the feature prediction and action classification between the latest step and future step. It helps to test the necessity of involving deeply- supprvssupe learning strategy.
5.3 EVALUATION RESULTS ON EXPLORATION
The quantitative results on exploration task are shown in Table 1. We can observe from this table that ATM achieves comparable performance on both Gibson dataset with RL.-based methods, and best-performing result on MP3D dataset by outperforming all RL-based methods significantly (about 13% coverage ratio and 40m² area improvement). Since the comparing RL-based methods [1, 56, 16, 12] build the map in metric space and requires millions of training images, ATM is desirable because (1) it provides a metric-free free option for exploration, and (2) it is lightweight (in terms of parameter size 16 M) and requires much less training data (just about 0.45 million images, in contrast with 10 million images required by most RL-based methods). The room scenes in MP3D dataset are much more complex and larger than those in Gibson dataset. They contain various physical impediments (e.g. complex layout, furniture), and some rooms contain outdoor scenarios. Hence, ATM exhibits stronger generalizability in exploring novel scenes than RL-based methods. Moreover, performance gain more obvious on both Gibson and MP3D datasets when we change the agent to a different locomotion setup (from 0.25/10 to 0.30/30'), which also shows ATM is robust to different locomotion setups.
On Gibson dataset, ATM achieves lower coverage ratio than ANS [12] but higher average covered area. We find such performance difference mainly caused by ATM stronger capability in exploring large areas than RL-based methods. In most cases, ATM actively reaches to new area within limited steps. This result is also echoed by the coverage ratio curve between ATM and ANS [12] (Fig. III in Appendix C), from which we see ATM quickly increases the coverage ratio during the early exploration stage (higher curve slope). If we use a different locomotion setup (0.30/30*), ATM coverage ratio stays as highest along the exploration process (left-most sub-figure of Fig. III).
Comparison with random exploration. Random Walk serves as the baseline for our framework. It is also adopted by SPTM [58] to build topological map. It involves learning procedure and the agent randomly takes an action at each step to explore an environment. From Table 1, we can see that Random Walk
7

---

Under review as a conference paper at ICLR 2023
Table 1: Quantitative results of exploration task over 1000-step budget. Top three performances are highlighted by red, green, and blue color, respectively.
Domain Generalization
Gibson Val
MP3D Test
Method Description
Method
Sensor Used
e "Train Imga
3CCvv Cww. (m
Cuv, (m )
Non-learning Based
Kandom Walk by SPIM [58])
No
No
0.501
22.268
0.301
40.121
RL 3.Conv [I]
10M
0.737
22.838
0.332
47.758
RL-ResNet18
10
0.747
23.188
0.341
49.175
RI--ased
1e [56]
10
0.779
24.467
0.356
51.959
RL- PL-ReeN [16]
RGB, Depth, Pose
10 M
0.789
24.863
0.378
54.775
OccAnt [65]
1.5-2
0.935
31.712
0.500
ANS [12]
10 M
0.948
32,701
0.521
73,281
AIM SoDeppSu
0.768
26.671
0.292
37.163
ATM Variants
ATM_LSTMActo MMMLCSeMaiRega
0:914
35,238
0.610
101.734
AIM withHistory
0.917
35,331
0.618
102.302
NoFeatHallu
RGB
0.45 M
0.907
34.563
0.589
99,091
Deeply Supervised Imitation
ATM (0.25 m/I0")
0.918
35.274
0.642
109,077
(0.30 m/30%)
0.927
37.731
0.656
117.993
Gibson Musquilo
A. RandomWall
ATM nolcepSup
C. ATM LSTMActRegu
D.ATM
Gilson Riber
E. ATM: 500
ATM: 500 steps
G. ATM: 500 steps
H. ATM: 200 steps, 0.3m/30 degree
Figure 4: Exploration trajectories Visualization. Top row: various ATM variants exploration result (1000- step budget) on Gibson Mosquito scene. Bottom row: exploration with different start positions (E, F, G, 500-step budget, with agent step-size 0.25 m and turn-angle 10"). Agent with larger step-size and turn- angle (0.3 m/30%) achieves similar coverage ratio with much smaller steps (200 steps, F). The trajectory color evolving from cold (blue) to warm (yellow) indicates the exploration chronological order. More exploration results are given in Appendix A, Fig.
dramatically reduces the exploration performance in terms of both coverage ratio and average coverage area. The inferior performance of RandomWan Walk verifies the necessity of learning active exploration strategy in order to help agent to efficiently explore an environment. Figure 4 demonstrates the qualitative comparison between RandomWalk Walk and ATM exploration result.
Feature regularization and with history memory. If we replace feature regularization involved TaskPlan- with action regularization (ATM LSTMActRegu). we have observed more performance drop MP3D than on Gibson dataset (3% versus 0.2%), which shows adopting feature regularization improves the general- izability compared with action regularization. Moreover, introducing full history memory (ITM_FullHistory to TaskPlanner (used as LSTT hidden state input) produces very similar results on Gibson dataset, but significantly reduces the performance on MP3D dataset (more than 2% drop). It thus shows using historical memory tends to encourage ATM to overfit training data, so that its generalizability is inevitably reduced. We argue that such generalizability drop might lie in our over-simplified history memory modelling because we just evenly sample 10 nodes (image observations) from all historically visited nodes, which might be too simple to represent the whole history memory, or even confuses TaskPlanner if the agent has already explored many steps. More elegant long-term history memory model remains to be explored.
Deeply-supervieed learning and joint task and motion imitation. Removing deeply-supervised learn- ing ANTM_noDDephe
leads to obvious performance drop on both Gibson and MP3D dataset. In MP3D datast, it evens leads to worse performance than Random Walk. It thus shows the necessity of deep supervision in both feature space (TaskPlanner) and action space (MotionPllnmer), Meanwhile, ATM HMM_noFeatHllll leads to significant performance drop on both Gibson and MP3D datasets. It thus attests the advantage of our feature-space task and motion imitation strategy which jointly optimize TaskPlanner for high-level task allocation and MotionPlanner for low-level motion control.
8

---

Under review as a conference paper at ICLR 2023
Gibson Swormville
A. All Anchor Points (rod dot)
B. Expert Exploration Demonstration
ATM exploration trajectory
Figure 5: Expert Demonstration and ATM exploration visualization. A: all potential anchor points in Gibson Swormville scene. B: expert selectively traverses a subset of the anchor points (choose one anchor to traverse among a group of close anchors) to cover the whole area. C: ATM exploration trajectory with 1000-step budget, with the model learned from expert demonstration.
Table 2: Quantitative results of navigation tasks. Top three performances are in red, green, and blue color, respectively. We collect 2000 (Gibson/50000(P33)) images with agent setup (0.25m/10").
Gibson Val
Domain Gencralization on MP3D Testset
Method
Suce. Rate (f)
SPL
Suoe. Rate (f)
SP)
BandomWalk
0.027
0.021
0,010
0.010
RL + Blind
0.625
0.421
0.136
0.087
3LCovv GRU
0,550
0,406
0,102
0.080
RL + ResNeel8 + GRU
0,561
0.422
0.160
0.125
RL+ + ResNet18 GRU DexDe [56]
0.461
0.189
0.143
RL ResNet18- GRU ProjDepth
0,614
0.436
0.134
0.111
II ResNet18 + GRU
0,823
0.725
0,365
0.318
CMP [28]
0.827
0.730
0.320
0.270
OccAnt (RGB) [65]
0.882
0,712
N/A
N/A
ANS [12]
0,951
0.593
0.496
ATM
0,957
0.859
0,733
0.619
We also visualize the comparison between ATM hallucinated next-step future feature and truly observed feature in Appendix Fig. II (C). We see that the hallucinated feature more similar with the observed real feature when the agent is walking through a spacious area (in other words, the agent mostly takes Love_forwall action), than when the agent is walking along room corner, against the wall or through narrow pathway. This may be due to the learned TaskPlanner most likely hallucinates feature moving the agent forward if the temporary egocentric environment allows. This also matches expert exploration experience because experts (like humans) mostly prefer moving forward so as to explore as more area as possible.
5.4 EVALUATION RESULTS ON NAVIGATION
The navigation result is in Table 2. We compare ATM with most of methods compared in exploration task. CMP [28] builds up topdown belief map for joint planning and mapping. For OccAnt [65], we just report its result with model trained with RGB image (so as to be directly comparable with ATM). We can see that ATM outperforms all comparing methods on the two datasets, with the largest performance gain on MP3D dataset (about 14% Succ. Rate, 12% SPL improvement). Hence, we can conclude that our built topological map can be used effectively for image-goal navigation. More importantly, ATM exhibits satisfactory generalizability in navigation owing to both active exploration in feature space and topological mapping in VPR. In Fig. II (B), we can see VPR and ActionAssigner successfully add new edges (purple lines) to aajaaet RGB images (nodes), resulting in spatial-adjacency representative topological map. More detailed discussion on navigation is given in Appendix B.
6 CONCLUSION
We propose a novel active topological mapping (ATM) framework for visual navigation, which is lightweight and computationally efficient. Our ATM's active exploration is metric-free, and can be learned via deeply- supervised task and motion imitation learning with superior data efficiency and generalizability compared to other RL-based methods. We also provide some real-world experiments show our sim2real transfer- ablity (in Appendix H). Future works include designing elaborate historic memory modules and involving multi-modality sensors to further improve the performance of visual exploration and navigation.
9

---

Under review as a conference paper at ICLR 2023
7 REPRODUCIBILITY STATEMENT
Please refer to Section. 5.1. on how to reproduce our expert demonstration dateset. Appendix. E. provides details on how to implement all the neural networks used in our work, and Appendix. F. provides details on how to implement the VPR system used in the topologiaal map construction. All the code used in this paper are shared through Supplementary Material.
REFERENCES
[1] Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, and Amir R. Zamir. On evaluation of embodied navigation agents, 2018.
[2] Peter Anderson, Angel X. Chang. Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, and Amir Roshan Zamir. On evaluation of embodied navigation agents. CoRR, 2018.
[3] R. Arandjelović, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. NetVLDD: CNN architecture for weakly supervised place recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2017.
[4] Somil Bansal, Varun Tolani, Saurabh Gupta, Jitendra Malik, and Claire Tomlin. Combining optimal control and learning for visual navigation novel environments. 2019.
[5] Edward Beeching, Jilles Dibangoye, Olivier Simonin, and Christian Wolf. Learning to plan with uncertain topological maps. In European Conference on Computer Vision(ECCC 2020.
[6] Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J. Leonard. Past, present, and future of simultancous localization and mapping: Toward the robust-perception age. IEEE Transactions Robotics, 2016.
[7] Chao Cao, Hongbiao Zhu, Howie Choset, and Ji Zhang. TARE: A Hierarchical Framework for Efficiently Exploring Complex 3D Environments. In Proceedings of Robotics: Science and Systems, Virtual, July 2021. doi: 15607/5SS 202...22221.....
[8] Luca Carlone and Sertac Karaman. Attention and anticipation in fast visual-inertial navigation. Trans. Rob., 35(1):120,, feb 2019, ISSN 1552-3098.
doi: 10. 2019/TR..008..877244444 URL https:
//ooi.org////o 0. 09/ / TRO. 2018 2872402,
[9] Angel Chang, Angela Dai, Thomas Funkhouseer Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song. Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. International Conference on 31) Vision (3DV), 2017.
[10] Chien- Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, and Juan Carlos Niebles. Procedure planning in instructional videos. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision - ECCV 2020, pp. 334-350, Cham, 2020. Springer International Publishing. ISBN
[11] Matthew Chang, Arjun Gupta, and Saurabh Gupta. Semantic visual navigation by watching youtube videos. In Advances in Neural Information Processing Systems (NerulPS), 2020.
[12] Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning to explore using active neural slam. In International Conference on Learning Representa- tions(CCIan 2020.
[13] Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta. Neural topological slam for visual navigation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[14] Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, Jitendra Malik, and Ruslan Salakhutdinov. Seal: Self-supervised embodied active learning. In Advances in Neural Information Processing Sys- tems (NeurIPS), 2021.
[15] Kevin Chen, Juan Pablo de Vicente, Gabriel Sepulveda, Fei Xia, Alvaro Soto, Marynel Vázquez, and Silvio Savarese. A behavioral approach to visual navigation with graph localization networks. In Robotics: Science and Systems Conference (RSS), 2019.
10

---

Under review as a conference paper at ICLR 2023
[16] Tao Chen, Saurabh Gupta, and Abhinav Gupta. Learning exploration policies for navigation. In International Conference on Learning Representations (ICLR), 2019.
[17] Rohan Chitnis, Dylan Hadfield-Menell. Abhishek Gupta, Siddharth Srivastava, Edward Groshev, Christo- pher Lin, and Pieter Abbeel. Guided search for task and motion plans using learned heuristics. In IEEE International Conference on Robotics and Automation (ICRA), 2016.
[18] Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. In Eighth Workshop Syntax, Semantics and Structure in Statistical Translation, 2014.
[19] Edsger W Dijkstra. A note on two problems in connexion with graphs. Numerische mathematik, 1959.
[20] Alexey Dosovitskiy and Vladlen Koltun. Learning to act by predicting the future. In International Conference on Learning Representations (ICLR), 2016.
[21] Hugh Durrant-Whin and Tim Bailey. Simultaneous localization and mapping: part 1. IEEE Robotics & Automation Magazine, 2006.
[22] Alberto Elfes. Sonar-based real-world mapping and navigation. IEEE Journal on Robotics and Automation, 333):249-265, 1987.
[23] Kuan Fang, Alexander Toshev, Li Fei-Fei, and Silvio Savarese. Scene memory transformer for embodied agents in long- hori-horizon tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition ( (CVPR), 2019.
[24] B. Fernando and S. Herath. Anticipating human actions by correlating past with the future with jac- card similarity measures. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 13219-13228 Los Alamitos, CA, USA, jun 2021. IEEE Computer Society, URL https: doi wtweeeeeoootetssoceeeeo.... org
doi: 10.1 N0.100/CVPR444377 10. 1t.1100//VPR44333
.013022
[25] Jorge Sruentes-Pacheeee José Ruiz-Ascencio, and Juan Manuel Rendón-Mancha. Visual simultancous localization and mapping: survey. Artificial Intelligence Review, 2015.
[26] Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated task and motion planning. Annual Review of Control, Robotics, and Autonomous Systems, 2021.
[27] Sabine Gillner and Hanspeter A. Mallot. Navigation and acquisition of spatial knowledge in virtual maze. Journal of Cognitive Neuroscience, 1998.
[28] Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
[29] Meera Hahn, Devendra Singh Chaplot, Shubham Tulsiani, Mustafa Mukadam, James M Rehg, and Abhinav Gupta. No rl, no simulation: Learning to navigate without navigating. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[30] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[31] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, 1997.
[32]
Brian Ichter and Marco Pavone. Robot motion planning learned latent spaces. IEEE Roboties and Automation Letters, 4(3):2407-2414, 2019. doi: 10. 1 109/LA9 A0R.....11.... 100199920110000 19888
[33] Ashesh Jain, Avi Singh, Hema Swetha Koppula, Shane Soh, and Ashutosh Saxena. Recurrent neural networks for driver activity anticipation via sensory- fusion architecture. 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 31 3125, 2016.
[34] Hervé Jégou, Matthijs Douze, Cordelia Schmid, and Patrick Pérez. Aggregating local descriptors into a compact image representation. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3304-3311, 2010. doi: 10. 1w1.11/V...2200.5555000000
[35] Santhosh K. Ramakrishnan, Dinesh Jayaraman, and Kristen Grauman. An exploration of embodied visual exploration. In International Journal of Computer Vision (ICC),, 2021.
[36] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), (CCLR),2015, 2015.
11

---

Under review as a conference paper at ICLR 2023
[37] Hema S. Koppula and Ashutosh Saxena. Anticipating human activities using object affordances for reactive robotic response. IEEE Trans. Pattern Anal. Mach. Intell., 38(1):14 jan 2016. ISSN 0162-8828.
doi: 10. 10011/9AM...200052200000 2015. 011524430335.
URL BRLphttps:// /tttpp:://cc..oroo 1109/TPMMI
[38] David Kortenkamp and Terry Waymouth. Topological mapping for mobile robots using a combi- nation of sonar and vision sensing. In Proceedings of the twelfth national conference on Artificial Intelligence (AAAI). 1994.
[39] Benjamin Kuipers and Yung-Tal Tai Byun. A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Robotics and autonomous systems, 8(1-2):47-63, 1991.
[40] Obin Kwon, Nuri Kim, Yunho Choi, Hwiyeon Yoo, Jeongho Park, and Songhwai Oh. Visual graph memory with unsupervised representation for visual navigation. In IEEE International Conference on Computer Vision (ICCV), 2021.
[41] Chen- Yu Lee. Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Suppry-Sueer Nets. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2015.
[42] Cindy Leung, Shoudong Huang, and Gamini Dissanayake. Active slam using model predictive control and attractor based exploration. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026-5031. IEEE, 2006.
[43] Cindy Leung, Shoudong Huang, and Gamini Dissanayake. Active slam in structured environments. In IEEE International Conference on Robotics and Automation (ICRA), 2008
[44] Chi Li, M Zeeshan Zia, Quoe-Huy Tran, Xiang Yu, Gregory D Hager, and Manmohan Chandraker.
Deep supervision with shape concepts for occlusion-aware 3d object parsing. In Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5465-5474, 2017.
[45] Chi Li, M Zeeshan Zia, Quoc-Huy Tran, Xiang Yu. Gregory D Hager, and Manmohan Chandraker. Deep supervision with intermediate concepts. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2018.)
[46] Martina Lippi, Petra Poklukar, Michael C. Welle, Anastasiia Varava, Hang Yin, Alessandro Marino, and Danica Kragic. Latent space roadmap for visual action planning of deformable and rigid object manipulation. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5619-5626, 2020. doi: 10. 109/1000455574 202009944422
[47] Ting Liu, Andrew W. Moore, and Alexander Gray. New algorithms for efficient high-dimensional nonparametric classification. Journal of Machine Learning Research, 7(41):1135-11533 2006. URL http:/pappsss////ee /ww00 6a. .tmll.
http:// / /mmrr
[48] S. Lloyd. Least squares quantization in pem. IEEE Transactions on Information Theory, 2892)1129-137,
1982. doi: 19..100/TTT.992.000 1054448.
[49] Shih- Lo, Shiqi Zhang, and Peter Stone. Petlon: planning efficiently for task- navigation.
In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems,
pp. 220-228, 2018
[50] David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. International Journal on Computer Vision (IJCV), 2004.
[51] Stephanie Lowry, Niko Sünderhauf, Paul Newman, John J. Leonard, David Cox, Peter Corke, and Michael J. Milford. Visual place recognition: A survey. IEEE Transactions on Robotics, 2016.
[52] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A Platform for Embodied AI Research. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
[53] Michael James McDonald and Dylan Hadfield Guided imitation of task and motion planning. Conference on Robot Learning (CoRL), pp. 630-640. PMLR, 2022.
[54] Lina Mezghani, Sainbayar Sukhbaatar, Arthur Szlam, Armand Joulin, and Piotr Bojanowski. Learn- ing to visually navigate in photorealistic environments without any supervision. arXiv preprint arXiv:2004. 04954, 2020.
[55] Toki Migimatsu and Jeannette Bohg. Object-centric task and motion planning in dynamic environments. IEEE Robotics and Automation Letters, 2020.
12

---

Under review as a conference paper at ICLR 2023
[56] Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andy Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, Dharshan Kumaran, and Raia Hadsell. Learning to navigate in complex environments. In International Conference on Learning Representations (ICLR), 2017.
[57] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529-333, 2015.
[58]
Savinov Nikolay, Dosovitskiy Alexey, and Koltun Vladlen. remimparaettric topological memory for navigation. In International Conference on Learning Representations (ICLR), 2018.
[59] Stephen M. Omohundro. Five balltree construction algorithms. Technical report, ICSI Berkeley, 1989.
[60]
Hyun Soo Park. Jyh-Jing Hwang, Yedong Niu, and Jianbo Shi. Egocentric future localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4697-4705, 2016. doi:
[61] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vito,o, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Bus Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems (NeurIPS), 2019.
[62] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duches- nay. Scikit-learn Machine learning in Python. Journal of Muchine Leurning Research, 12:2825-2830, 2011.
[63] Stephane Ross and Drew Bagnell. Efficient reductions for imitation learning. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTA7S), 2010.
[64] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bemnstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (ICV), 2015.
[65] Ziad Al-Halah Santhosh Kumar Ramakrishnan and Kristen Grauman. Occupancy anticipation for efficient exploration and navigation. In Proceedings of the European Conference on Computer Vi-
sion (ECCV), 2020.
[66] Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Mare Pollefeys, Timothy Lillicrap, and Sylvain Gelly. Episodic curiosity through reachability. In International Conference on Learning Representations (ICLR), 2019.
[67] Dawei Sun Sun, Anbang Yao, Aojun Zhou Zhou, and Hao Zhao. Deeply-supervised knowledge synergy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[68] D D'idac Sur'is is, Ruoshi Liu, and Carl Vondrick. Learning the predictability of the future. 2021 IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pp. 12602-12612. 2021.
[69] Antony Thomas, Fulvio Mastrogiovanni, and Marco Baglietto. Mptp: Motion- waatnnn---nnnnnnnnnaeae aware task planning for navigation in belief space. Robotics and Autonomous Systems, 141:103786,, 2021.
[70] Edward C Tolman. Cognitive maps in rats and men. Psychological review, 55(4):189, 1948.
[71] C. Vondrick, H. Pirsiavash, and A. Torralba. Anticipating visual representations from unlabeled video. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 98-106. Los Alamitos, CA, USA, jun 2016. IEEE Computer Society. doi: 10.1109/CVP..0016 https: do leeecomp e www.eiesssttte.oeooo org./o/ggooooooo /00. 11 109 201109//VVSS CVPR. 2016.18.
URL
[72] Fei Xia, Amir R. Zamir, Zhi-Yang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: real-world perception for embodied agents. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
[73] K. Zeng, W. B. Shen, D. Huang, M. Sun, and J. Niebles. Visual forecasting by imitating dynamics in natural sequences. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3018-3027. Los Alamitos, CA. USA. oct 2017. IEEE Computer Society. doi: 10. 00.119//CC2.2017..222 URL https: /taps://doii aeeecoppterrocc ety. org / 1 0. 1109/ TCCV 2017. 3266
13

---

Under review as conference paper at ICLR 2023
[74] Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. Target-driven visual navigation in indoor scenes using deep reinforcement learning, In 2017 International Conference on Roboties and Automation (ICRA), 2017.
14

---

Under review as conference paper at ICLR 2023
APPENDIX
A
MORE RESULTS ON EXPLORATION
We show an extra exploration result on MP3D [9] q9v50 VnCiC room scene Fig. this figure, we show both the room scene 3D visualization in top row and multiple exploration results in the two bottom rows. Specifically, we show three efficient exploration results in the middle row, which the agent begins explore at different positions (marked by small red rectangle patch). We further show two less-effective exploration results the bottom row (last two sub-figures), which are mainly caused by repetitive visiting within local area (yellow-to-n) -o-red trajectory).
3DD Room Visualization
MP3D q9r q9vSo/
A. Efficient Exploration
B. Efficient Exploration
C. Efficient Exploration
D. Inefficient Exploration
E. Local Repetitive Exploration F. Local Repetitive Exploration
Figure ATM exploration result on MP3D [9] room scene q9vSolVncic. We show both efficient explo- ration result (middle row, sub-figure A. B, C). We also show relatively less-efficient exploration in sub-figure E and F, which are mainly due to local repetitive exploration. We further show an inefficient exploration example in sub-figure D. The room scene 3D visualization given in the top row. The agent exploration starting-position is marked by a red rectangle patch.
MORE RESULTS ON NAVIGATION
Visualizations of topological mapping. We show qualitative example of ATM in Fig. II. We from exploration trajectory that agent actively traverses all areas across multiple rooms in the environment scene, VPR and ActionAssigner sucessfully add new edges (purple lines in the middle figure) to spatial-adjacent nodes, resulting in complete and robust topological map.
C COVERAGE RATIO CURVE
The coverage ratio curve over 1000-step budget between ATM (two locomotion settings) and ANS [12] is shown in Fig. III, from which can ATM quickly increases the coverage ratio during the early exploration stage.
15

---

Under review as conference paper at ICLR 2023
140 2000
LPA OTTT
Exploration Trajectory
Topological Mapping Result
C. Error between Hallueinated Feat. and Real-Observed Feat.
Figure II: Feature Visualization A. The exploration trajectory (blue to yellow, step size 0.25 m and turn-angle 30") with 500-step budget, overlaid top of floor plan map. B. The spatially-adjacent panoramic images connected (purple lines) via VPR. C. The difference (Euclidean distance in 512-d feature space) between real-observed features and hallucinated features. The darker of the color, the lower of the difference.
Gibson Val Large
Gibson Val Small
Gibson Val Overall
1.0
1.0
0.8
0.8
0.8
Covorgg
0.6
0.6
0.6
0.4
0.4
ATM
ATM
ATM
ATM 30degree
ATM 3_3deeeeeeeeeee
ATM 30degree
ANS
ANS
ANS
0.0
200
400
600
800
1000
0.0
200
400
600
800
1000
0.0
200
400
800
1000
Step ength
Figure III: Comparison of ATM with ANS [12]. Coverage ratio curve over 1000-step budget over large 50m2, small < 50m2 and all (average) room size, respectively.
D MORE DETAILS ON EXPERT DEMONSTRATION
We sample set of anchor points across all navigable area in the each room scene, then call Habitat-lab Pathfinder API to let the agent traverse all anchor points to cover the whole area. Pathfindes API generates the most economic action sequence (following shortest-pale geodesic path) leading the agent from one anchor point to another. In practice, we use the provided episodes' start/end points in Gibson as the anchor points since they densely cover all the navigable area (see Fig. 5). We collected 450 training images and the associated actions 72 scenes from Gibson, which are used to train both ATM and ActionAssigner.
NEURAL NETWORK ARCHITECTURE
E. ATM NEURAL NETWORK ARCHITECTURE
Our ATM topological mapping neural network architecture is given in Table I, it consists of ResNet18 (as the image embedding module), LSTM layer (for task planner) and FC layer (for task planner and motion planner).
E.2 ACTIONASSIGNER NETWORK ARCHITECTURE
The ActionAssigner neural network is given in Table II. The ActionAssigner also uses ResNet18 as the image embedding layer. Then it uses sequence of FC layers to predict sequence of actions separately (sequence length 6). Bidirectional LSTM is applied to model actions mutual dependencies. The parameter size 13.5 M. We train ActionAssigner with the same parameter setting the famrwork of PlanPer and MotionPlanner (the network in Table I).
16

---

Under review as conference paper at ICLR 2023
Table I: ATM network architecture illustration. The network consists of basic 2D image convolution layers, such as ResNet18, LSTM and FC. The network is lightweight, the parameter size is just 16 M.
Layer Name
Filter Num
Ouiput Size
Image Embedding Layer
Input: [10, 3, 512]
Embedding Network: ReaNet18
Embedking Size: 110, 512)
Task Planner Network
LSTM
layers = 2, hidlensize 512
[10, 512]
Feat Prediction FC
Feal 512, Feut 512
110, 512)
Motion Planner Network
Input: Feat [10, 1024], Action: [10]
Feat Merge FC
fent 1024, feat 512
[10, 512]
Action Classification FC
in feat = 512, out 3 =
[10.3]
Table ActionAssigner neural network architecture. Given two panoramic image observations, we predict six sequential actions in total.
Layer
Filter
Outpui
Image Embedding Layer
Input: [2, 3, 256, 512]
Emhedding Network: ResNet18
Feat, Merge Layer
Concat Size: [1. [024]
1024,
[1. 512]
Action Predict Branch
branchl FC
in feul = 512, out feat 128
[1. 128]
branch2
out
128]
branch3 FC
in feat 512, out feat 128
[1. 128]
branch4
feat = 512, out feat 128
[1. 128]
brunch5 FC
feal 512, out feat 128
[1. 128]
branchó FC
in feat = 512, out feat 128
[1. 128)
Action Predict
Concal, Size: [1, 128]
BiLSTM
layers = 1, out leat 128
[1. 1281
Action Classily
leal 128, out
[,,
F MORE DETAILS ON VISUAL PLACE RECOGNITION
F. DETAILS ON THE IMPLEMENTATION OF VLAD-BASED VPR AND ACTION ASSIGNER
For VPR, we first extract SIFT[50] features from all the images captured during the exploration episode, and then compute VLAD feature for each image based on the clustered SIFT as described in [34]. This process involves rumning K-Means [48] all the SIFT features aggregated. We used Scikit-Learn [62]'s K- K-Means implementation with n fecchn_cuesseee clusters=11 6, n_init=1. We then store all the VLAD features into a ball tree [59, 47] with leaf si 60 and L2 as distance metric. Finally, for every node, VLAD feature to query the ball tree for 20 nearest neighbors as candidates. All the hyperparameters remains unchanged during all the testing, and have not done any hyper-parameter searching.
its image's s
We show the VPR result in Fig. IV where C. is the candidates picked by the VPR process and we can see VPR successfully reflects image observations' spatial closeness. In practice, we again use Habitat-lab path inder to get the ground truth actions between any two VPR-connecter image observations. If the obtained action list length smaller than six, we append STOP action in the end. If the obtained action list length is larger than six, we remove the connection. In navigation task, the agent just stops once it sees the predicted STOP action from the inferred action sequence by the ActionAssigner.
F.2 TIME EFFICIENCY OF VPR
In Table. III, we demonstrate the time efficiency superiority of our VPR system. The "TTme Spent" indicates the amount of time each method takes to finish retrieving all node pairs that should be connected (to add edge) in room. Specifically, we report the average time cost across all the 14 test rooms in the Gibson dataset (excluding ActionAssigner time cost). The resulting maps after ppatial-adjacenc connection created by the three methods almost identical. We that proposed VPR-based method has the minimal time cost (or time complexity).
17

---

Under review as conference paper at ICLR 2023
A. Non-similar Image Pair (15 steps) B. Semi-similar Image Pair (5 steps) C. Similar Image Pair (2 steps) Figure IV: VPR result visualization Gibson Cantwell environment scene. The colored lines indicate paired inlier points between two image. We use the the same reference figure (top row), and compute its VPR result with three images the agent takes at another location (different step location). Left: 15 steps difference with nearly dissimilar image pair, middle: steps difference with semi-similar image pair, right: 2 steps difference with the most similar image pair.
Methods
Time Spent
SPTM [58]
2.1
Brute-Force Geometric Verification
2 26 hr
VLAD-Based VPR (Ours)
= 0.2 hr
Table III: The average time spent on constructing topological map given all the images of
In this experiment, We implemented the retrieval network in the SPTM [58] to do classification between every image on whether they should be connected. The network architecture and hyper-parameters are the same as in the paper, with some small tweaks to the architecture's 's dimension due to the fact the original paper is done in low-resolution video game while our testing environment is a photo-realistic dataset. The memory graph in SPTM [58] is similar to the concept of topological graph in our setting, and SPTM locomotion network similar our iontsosssss gner.
The Brute-Force Geometric Verification method exhaustively apply geometric verification on all image pairs. Its time cost can potentially reduced by turning to parallelism computation.
All the experiments are done with 10 cores of Intel Xeon Platinum 8268 (205W, 2.9GHz), 32GB RAM and a SSD. SPTM [58] additionally run on Nvidia V100 GPU with a batch size of 160.
G EXPLORATION EFFICIENCY DISCUSSION
An ideal exploration should constantly keep reaching to new area, like the expert exploration does. We find our proposed ATM framework sometimes leads to inefficient exploration: repetitively visiting local area. We show three examples of such inefficient exploration on Gibson Swormville environment scene in Fig. V. Our introduced random action perturbation strategy can reduce the "repetitive visiting" dilemma to some extent. According to our understanding, such "repetitive visiting" is mainly caused by ATM "short- memory" when predicting the next action to take, and involvement of no global environment scene information, including the whole historically visited memory (scene memory) and topdown map (telling the agent if the area to explore has already been visited), to supervised the whole exploration process. We do not involve such extra supervision because we want to keep our framework as simple and lightweight as possible. It remains as future research direction to explore how to involve extra information to achieve more efficient exploration. We also provide several video (in which left image shows the panoramic image observation and the right image shows the corresponding exploration trajectory update on the top-down map.)
18

---

Under review as conference paper at ICLR 2023
Gibson Swornville
Trajectory on 500-step budget B. Trajectory on 500-step budget Figure V: Examples of inefficient exploration visualization on Gibson Swormville environment scene (with agent 0.3m/30 setup). We can that ATM inevitably leads repetitive visiting of local area. We argue that such such "repetitive visiting" is mainly caused by the "short memory" (just using the most recently visited short image observation sequence to predict the next action) of ATM, and exploitation of no global scene information, such as the assist of full history memory and global swiftly top-down map.
C. Trajectory on 700-step budget
H MORE DETAILS ON REAL-WORLD HARDWARE EXPERIMENT
H.1 HARDWARE DETAILS
See Fig. VI. for the physical setup of the robot built. We added aluminium frame top of iRobot Create to host a Nvidia Jetson TX2 as the processor and an Insta 360 Pro2 to capture panoramic images. The turning angle (10 deg), step length(0.25m), camera height(1. and other important parameters are designed to be as close to the configuration in the simulation as possible. An example image captured by the camera is given in VIII.
Figure VII: The hand-drawn trajectory of the 50-step exploration episode.
Figure VI: Two different angles the robot
H.2 REAL-WORLD EXPERIMENT DETAILS
With the help of our ATM model (with no real-world fine-tuning), we conducted preliminary experiment using this robot explore indoor environment. The result in VII The step budget is capped at 50 steps (vs. 1000 in simulation) due to limitations in battery capacity and prolonged execution time caused by the fact that the camera does real-time onboard images stitching to produce panoramic images. We continuously improving the hardware system by finding better way to capture panoramic images and enhancing the stability of the camera by adding more support structures, and hope to deliver more hardware experiment results.
19

---

Under review as conference paper at ICLR 2023
Figure VIII: One of the image captured by our Insta 360 Pro2 camera while in exploration
20


Reviews:
[{"review_id": "wZ_ns24CzV", "reviewer": "Reviewer_9Cpk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a framework of topological mapping, which is implemented as a pipeline of two main components including a planner to generate robot navigational actions and a visual place recognition (VPR) method to construct a topological map.", "summary_of_the_review": "The paper proposes to address an important problem of active mapping for agent navigation. However, novelty of the paper is a main concern: it is not clear why the topological mapping is \"active\" and the novelty of each component used in the pipeline is also low.", "strength_and_weaknesses": "Strength: \n\n+ Topological mapping is an important problem for agent navigation.\n+ The paper is well written; each method in the pipeline is well described.\n\nWeakness:\n\n- Novelty and contribution to address active mapping is a key concern. \n-- The paper seems to argue the contribution is on \"active\" mapping. However, it is not clear how exactly the approach allows an agent to \"actively\" map the environment. The agent simply uses the policy learns from a demonstrator, and no active actions are taken during the execution to map the environment.  \n-- Each component also uses or marginally extends existing methods, and the novelty of each component is not well justified. For example, the idea of using a network to learn representations of the environment and use the representations to generate agent actions is commonly used; the VPR component also uses the previous methods.\n\n- The term Task Planner in Section 3.1 is very confusing. The approach seems to simply predict the future feature vector in the next time step given a short sequence of the past features. What is the definition of tasks? Or what is the difference between task planning and feature prediction? \n\n- Although the related work section reviews task and motion planning mainly for manipulation, the paper does not review and compare with a large body of local planning and control methods for navigation in the robotics community, which is more relevant to the paper’s problem domain.\n\n- Why predicting the future feature vector in the next step is necessary? If the approach is able to predict a high-dimensional feature vector in the next step, then why the approach cannot directly predict the low-dimensional future action in the next step?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "v6NYfxMvFk_", "reviewer": "Reviewer_CpyF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose an approach for visual exploration using RGB cameras only. The approach is also able to build a topological map of the environment. To do the exploration, the authors use imitation learning to hypothesize the image feature in the next step and choose an action based on that feature (and current feature). For topological mapping, the authors use visual place recognition (VLAD and SIFT), which they claim is very efficient.", "summary_of_the_review": "The proposed approach outperforms the baselines for the exploration task, but I think the navigation task experiments could be more thorough, especially since the topological map is one of the main contributions of the paper and this is how the constructed topological map is used.", "strength_and_weaknesses": "## Strengths\n* The approach outperforms all baselines on the exploration and navigation tasks. The authors choose reasonable baselines for the exploration task.\n* The authors provide many details about their approach and architectures in the main paper and appendix.\n\n## Weaknesses\n* It seems strange that the locomotion setup has big impact on performance. Is this the case for the baseline approaches as well, or is the proposed approach particularly sensitive to this?\n* Missing baselines for navigation (Sec. 5.4). The experiments in this section can be more thorough. For example, [1], [2], [3], [4], [5] are all good baselines to compare against, as they build a topological memory from exploration trajectories and use them for navigation (e.g. image-goal).\n    * In appendix E.2 I find it surprising that SPTM takes so long to run. Is the topological map very dense?\n\n## Notes\n* Some additional related works on topological map construction and navigation are [1] and [2].\n    * [1] is also similar to Savinov et al. They construct a topological map and connect edges using reachability. They utilize an exploration trajectory and then build a topological map from that.\n    * [2] In particular, the topological map here is similar to the proposed approach where each node is a panorama. They utilize an exploration trajectory and then build a topological map from that.\n\n* [1] Meng et al. 2020. Scaling local control to large-scale topological navigation. \n* [2] Chen et al. 2021. Topological Planning with Transformers for Vision-and-Language Navigation. \n* [3] Chaplot et al. 2020. Neural topological SLAM.\n* [4] Savinov et al. 2018. Semi-parametric topological memory for navigation.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lcDEN25C3f", "reviewer": "Reviewer_1W7X", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents an imitation learning-based task and motion planning (TAMP) method for active exploration and topological mapping of unknown indoor environments. Being metric-free, topological mapping allows for greater computational efficiency with respect to metric-based approaches. The method is composed of two stages: 1) Learning to actively explore from expert demonstrations; and 2) Building a topological map of the environment given the data collected at the previous step.\n\n- Stage 1: Active exploration is achieved by joint deeply supervised learning of a panoramic scene feature extractor, a task planner, and a motion planner. In particular, the task planner (TP - a recurrent neural network) predicts/hallucinates the next image feature to be visited given a previous sequence. Then, the motion planner (MP - a multi-layer perceptron) predicts the action needed to reach/observe the hallucinated features from the current location/feature. TP is trained to minimize a regression error in feature space by minimizing an L2 loss. MP is a 3-class action classifier (turn left, turn right, move forward), trained by minimizing the cross-entropy loss. Supervision consists of demonstrations of expert explorations.\n\n- Stage 2: Topological mapping is carried out by connecting temporally adjacent nodes retrieved from the collected exploration data, and also those matched by a visual place recognition and an efficient geometric verification technique. The corresponding agent actions are then assigned to the newly-introduced edges by a recurrent classifier, which predicts a sequence of actions.\n\nThe learned model can be employed for coverage path (and motion) planning or navigation between a starting and a target point. Experiments demonstrate the performances of the method (trained on the Gibson environment using an automated expert demonstration policy) on both the Gibson and MP3D datasets, which comprise a large number of indoor environments. Topological mapping policies learned on Gibson are shown to be able to generalize to MP3D, suggesting good generalizability of the method.", "summary_of_the_review": "To my knowledge, the proposed TAMP learning method appears novel, timely, and technically sound. Experiments are convincing, especially since they include two relevant task and demonstrate significantly improved generalization (Gibson validation set) and domain generalization (MP3D test set) with respect to baselines. \n\nClaims of superior time efficiency with respect to RL-based baselines appear intuitively convincing, but are not supported by an exhaustive quantitative analysis. Still, the main contribution is a significant and well-supported methodological step forward.", "strength_and_weaknesses": "Strengths\n--------\n- The integrated TAMP method is shown to learn exploration policies which can be transfered to significantly different domains, including a Gibson validation set and the more challenging MP3D scenario. Performance improvement is significant in both coverage and navigation experiments (Tab. 1 & 2). It is possible that the high-level topological representation employed by the method may favon an effective domain generalization. (As a side note, it may also be interesting to probe the performance of the TP and MP modules separately in the target MP3D domain.)\n\nWeaknesses\n---------\n- Time efficiency is claimed to be one of the main advantages of the proposed method. This appears to be supported only by the number of collected images shown in Table 1 though. A comprehensive training and inference time complexity analysis and comparison with the other baselines does not seem to be reported. It would greatly strengthen the claim, ideally breaking down the individual phases, e.g., expert demos generation, pre-processing, exploration learning, and topological mapping.\n- The FullHistory variant is not completely convincing in my view, since it operates with a very limited frame memory of $m=10$ as far as I understood.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "iYLnd2dPVn", "reviewer": "Reviewer_bznr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method to explore an environment and collect data to then build a topological map of the considered scene. Such exploration policy is a simple lightweight recurrent baseline trained with imitation learning to mimic expert trajectories. The policy is composed of a feature extractor that encodes the observation, a task planner which is a 2-layer LSTM, and finally a motion planner that is an MLP outputting the action to take from the output of the motion planner. The main novelty in this work comes from an additional regularisation loss term that forces the output of the recurrent module (motion planner) to be as close as possible to the feature extracted from the next observation, i.e. enforcing the motion planner to output a next goal in feature space.\n\nThe exploration policy is compared with different baselines on the task of maximising floor coverage, and shows state-of-the-art performance on the challenging MP3D set of scenes. The data collected by the exploration policy is also used to build a topological map to perform the ImageGoal Navigation task.", "summary_of_the_review": "The core idea introduced in this paper, i.e. hallucinating goals directly in feature space, is very interesting and promising. The lack of other novel ideas could be a weakness of the paper. However, if the gain brought by this additional regularisation is clearly showcased empirically, it can be a relevant contribution. As already mentioned, I want to be sure the ablation about keeping the exact full exploration policy architecture along with the imitation learning action loss term (exactly as it is used in the proposed method) and removing the regularisation term is properly done. I also have a few other concerns (stated in the \"Weaknesses\" section). I thus tend to consider the work as marginally below the acceptance threshold but am looking forward to clarifications from the authors.", "strength_and_weaknesses": "Strengths:\n1. The core idea to hallucinate next goals directly in feature space is very interesting and promising.\n2. The method is compared against many baselines and beats previous state-of-the-art approaches, both in exploration and ImageGoal navigation from the topological map built from data collected by the exploration policy.\n\n\nWeaknesses:\n1. A potential weakness of this work is the lack of technical originality in the proposed method. The topological map, mentioned in the paper, is similar compared with what has already been done in previous work. I’m not sure authors should put emphasis on the topological mapping part, both in the title, the abstract and the introduction, as there is actually no contribution on this side. Even in the experiment part, only the short Section 5.4 is about using the topical map. It is actually not clear whether the gain on the navigation task is about the use of a topological map, or simply the fact that the underlying exploration policy covers more of the scene. The architecture of the exploration policy is also standard, as well as the imitation learning objective.  The only novelty could be the additional loss term to regularise the output of the motion planner. However, if the gain from this loss term is clear (see next remark), this could be a great contribution.\n2. Authors should make the description of the considered baselines in Section 5.2 clearer. In particular, is there a baseline that corresponds to the exact same architecture as the one proposed, but with only the action loss ($\\mathcal{L}_M$) and no regularisation term (no $\\mathcal{L}_T$)? This ablation is very important to showcase the importance of the contribution.\n3. Authors mention that the necessity to have access to metrics information as done in previous work is a constraint, which can be true. However, their method necessitates access to strong expert data, which can be ok as current simulators can allow this, but they cannot claim that this is a lighter constraint.\n4. In Section 5.1, authors mention \"During ATM-guided exploration, we allow the agent to actively detect its distance with surrounding obstacles and walls\". How is this implemented, and doesn't it make the comparison unfair with other SOTA methods ?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cdr8yRX4qpS", "reviewer": "Reviewer_ZEmd", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work follows task and motion planning (TAMP) to design a high-level task planner and a low-level motion planner for more efficient exploration of an indoor scene. The goal is to cover the scene as efficiently as possible to construct a topological map. After the exploration, an off-the-shelf visual place recognition (VPR) system is used to find more edges on the initially constructed map. Finally, the trained action assigner predicts the possible action list causing the transition on the newly detected edges. The main contributions in this work are three-fold:\n\n(1) Design an efficient exploration policy based on TAMP.\n\n(2) Predict a possible next hidden state (imitate expert trajectories) at the task planner and use it as input to the motion planner.\n\n(3) Further, leverage an off-the-shelf VPR method and a learned action assigner to establish the completed topological map reconstruction system.\n\nThe experiments on the exploration and the navigation tasks in the Gibson and MP3D environments demonstrate the effectiveness of the proposed framework. ", "summary_of_the_review": "This work proposes to learn an efficient exploration policy by way of TAMP. The main idea is to train a task planner to hallucinate the next state in a feature space so that the motion planner can predict the action accordingly. Although the idea is self-motivated, the studied problem is essential, and the paper is well-presented with good results, the technical contributions and overall novelty are incremental. In addition, there is a missing discussion on visual forecasting and its application to planning and control in the related work section. Therefore, I would suggest the authors revise the paper based on them.", "strength_and_weaknesses": "+) The idea of adopting TAMP to the exploration policy design is self-motivated. Moreover, the studied problem, efficient exploration over an unseen scene, is also a meaningful direction. Finally, the final experiments also demonstrate the effectiveness of the proposed framework.\n\n+) The paper is well-written and easy to follow. The teaser figure shows the studied problem clearly, and the following framework figure and model figure illustrate the implantation in detail. Several qualitative results and videos in the main paper and supplementary present a clear visualization.\n\no) In the “exploretraj_MP3D_q9vSo1VnCiC_1.mp4” video, could authors illustrate why the agent keeps circling at a local area for a while? Is the agent keep executing move_forward and colliding with a wall?\n\n-) The main idea of designing an efficient exploration policy is based on hallucinating features at the next step. However, this idea is similar to visual forecasting or anticipation, and there is no discussion regarding it in the related work. Many works focus on using visual input to predict the feature at the next step in a latent space, such as [1, 2, 3, 4, 5]. There are also many works utilizing future predictions for manipulation or navigation.\n\n-) Although the experimental results demonstrate that the overall framework performs well on the exploration and navigation tasks, the model design and the way to hallucinate features at the next step are well developed. Therefore, the technical contributions in this work are incremental. Nevertheless, it would be appreciated if the authors could explore or share more insights about how to utilize the hallucinate features for motion planning.\n\n[1] Carl Vondrick et al., “Anticipating Visual Representations from Unlabeled Video”, CVPR 2016.\n\n[2] Kuo-Hao Zeng et al., “Visual Forecasting by Imitating Dynamics in Natural Sequences”, ICCV 2017.\n\n[3] Chien-Yi Chang et al., “Procedure Planning in Instructional Videos”, ECCV 2020.\n\n[4] Basura Fernando et al., “Anticipating human actions by correlating past with the future with Jaccard similarity measures”, CVPR 2021.\n\n[5] Dídac Surís et al., “Learning the Predictability of the Future”, CVPR 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject