Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an imitation learning-based task and motion planning (TAMP) method for active exploration and topological mapping of unknown indoor environments. Being metric-free, topological mapping allows for greater computational efficiency with respect to metric-based approaches. The method is composed of two stages: 1) Learning to actively explore from expert demonstrations; and 2) Building a topological map of the environment given the data collected at the previous step.

- Stage 1: Active exploration is achieved by joint deeply supervised learning of a panoramic scene feature extractor, a task planner, and a motion planner. In particular, the task planner (TP - a recurrent neural network) predicts/hallucinates the next image feature to be visited given a previous sequence. Then, the motion planner (MP - a multi-layer perceptron) predicts the action needed to reach/observe the hallucinated features from the current location/feature. TP is trained to minimize a regression error in feature space by minimizing an L2 loss. MP is a 3-class action classifier (turn left, turn right, move forward), trained by minimizing the cross-entropy loss. Supervision consists of demonstrations of expert explorations.

- Stage 2: Topological mapping is carried out by connecting temporally adjacent nodes retrieved from the collected exploration data, and also those matched by a visual place recognition and an efficient geometric verification technique. The corresponding agent actions are then assigned to the newly-introduced edges by a recurrent classifier, which predicts a sequence of actions.

The learned model can be employed for coverage path (and motion) planning or navigation between a starting and a target point. Experiments demonstrate the performances of the method (trained on the Gibson environment using an automated expert demonstration policy) on both the Gibson and MP3D datasets, which comprise a large number of indoor environments. Topological mapping policies learned on Gibson are shown to be able to generalize to MP3D, suggesting good generalizability of the method.
strength_and_weaknesses: Strengths
--------
- The integrated TAMP method is shown to learn exploration policies which can be transfered to significantly different domains, including a Gibson validation set and the more challenging MP3D scenario. Performance improvement is significant in both coverage and navigation experiments (Tab. 1 & 2). It is possible that the high-level topological representation employed by the method may favon an effective domain generalization. (As a side note, it may also be interesting to probe the performance of the TP and MP modules separately in the target MP3D domain.)

Weaknesses
---------
- Time efficiency is claimed to be one of the main advantages of the proposed method. This appears to be supported only by the number of collected images shown in Table 1 though. A comprehensive training and inference time complexity analysis and comparison with the other baselines does not seem to be reported. It would greatly strengthen the claim, ideally breaking down the individual phases, e.g., expert demos generation, pre-processing, exploration learning, and topological mapping.
- The FullHistory variant is not completely convincing in my view, since it operates with a very limited frame memory of $m=10$ as far as I understood.
clarity,_quality,_novelty_and_reproducibility: Clarity
-------
- The paper is in general well-structured. It was a pleasant read. Figures and tables are indeed very helpful to understand the method. I found the text to be sometimes a bit repetitive on some points, while skipping others which would be helpful to expand on.
- From the main text (Sec. 4) it is not clear to me how the ActionAssigner works. Why does it predict sequences, how is it trained, and its role in the mapping phase are hard to decipher.
- Some background on the employed deeply-supervised learning technique would help the reader understand the central point of joint feature extractor, TP, and MP training.
- Are the example scenes in the main text chosen from the Gibson validation set? How would they compare with MP3D test scenes (e.g. Fig. 4 and 5)? Showing some exploration and navigation examples from the more challenging MP3D test set in the main text would be informative.
- A significant number of typos is present

Quality
-------
- The technical soundness of the approach appears high. The method is described in sufficient detail and the schemes in Figures 1 to 3 are particularly heplful. 
- The experimental setup is quite convincing, especially the transfer to a different domain not seen during training (MP3D)
- Testing on both coverage and navigation tasks strengthens the work
- A comprehensive quantitative evaluation and comparison of the method's overall efficiency is missing

Novelty
-------
- The work appears well-placed in the literature
- The question appears relevant and timely. Learning for TAMP is a very active field of investigation with good potential
- To my knowledge, the proposed method is novel (although I am not actively doing research in TAMP for mapping and navigation at the moment)

Reproducibility
---------
- The approach is described in sufficient detail for reproduction, with detailed material in the Appendix (only skimmed)
- Code is provided (although not checked)

Other comments
--------
- Notation-wise, in (1) and (2) it would be more correct to optimize with respect to $\pi_\theta$ rather than $\theta$
- $\theta_M$ and $\theta_T$ appear redundant and heavy in (3) and (4). Subscript $\pi_\theta$ may be enough.
summary_of_the_review: To my knowledge, the proposed TAMP learning method appears novel, timely, and technically sound. Experiments are convincing, especially since they include two relevant task and demonstrate significantly improved generalization (Gibson validation set) and domain generalization (MP3D test set) with respect to baselines. 

Claims of superior time efficiency with respect to RL-based baselines appear intuitively convincing, but are not supported by an exhaustive quantitative analysis. Still, the main contribution is a significant and well-supported methodological step forward.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a method to explore an environment and collect data to then build a topological map of the considered scene. Such exploration policy is a simple lightweight recurrent baseline trained with imitation learning to mimic expert trajectories. The policy is composed of a feature extractor that encodes the observation, a task planner which is a 2-layer LSTM, and finally a motion planner that is an MLP outputting the action to take from the output of the motion planner. The main novelty in this work comes from an additional regularisation loss term that forces the output of the recurrent module (motion planner) to be as close as possible to the feature extracted from the next observation, i.e. enforcing the motion planner to output a next goal in feature space.

The exploration policy is compared with different baselines on the task of maximising floor coverage, and shows state-of-the-art performance on the challenging MP3D set of scenes. The data collected by the exploration policy is also used to build a topological map to perform the ImageGoal Navigation task.
strength_and_weaknesses: Strengths:
1. The core idea to hallucinate next goals directly in feature space is very interesting and promising.
2. The method is compared against many baselines and beats previous state-of-the-art approaches, both in exploration and ImageGoal navigation from the topological map built from data collected by the exploration policy.


Weaknesses:
1. A potential weakness of this work is the lack of technical originality in the proposed method. The topological map, mentioned in the paper, is similar compared with what has already been done in previous work. I’m not sure authors should put emphasis on the topological mapping part, both in the title, the abstract and the introduction, as there is actually no contribution on this side. Even in the experiment part, only the short Section 5.4 is about using the topical map. It is actually not clear whether the gain on the navigation task is about the use of a topological map, or simply the fact that the underlying exploration policy covers more of the scene. The architecture of the exploration policy is also standard, as well as the imitation learning objective.  The only novelty could be the additional loss term to regularise the output of the motion planner. However, if the gain from this loss term is clear (see next remark), this could be a great contribution.
2. Authors should make the description of the considered baselines in Section 5.2 clearer. In particular, is there a baseline that corresponds to the exact same architecture as the one proposed, but with only the action loss ($\mathcal{L}_M$) and no regularisation term (no $\mathcal{L}_T$)? This ablation is very important to showcase the importance of the contribution.
3. Authors mention that the necessity to have access to metrics information as done in previous work is a constraint, which can be true. However, their method necessitates access to strong expert data, which can be ok as current simulators can allow this, but they cannot claim that this is a lighter constraint.
4. In Section 5.1, authors mention "During ATM-guided exploration, we allow the agent to actively detect its distance with surrounding obstacles and walls". How is this implemented, and doesn't it make the comparison unfair with other SOTA methods ?
clarity,_quality,_novelty_and_reproducibility: The paper is generally well written. As mentioned in the previous section, however, I would like authors to present more clearly the baselines they introduce. Originality of the work is limited, but the simple contribution can be considered as solid if properly evaluated.
summary_of_the_review: The core idea introduced in this paper, i.e. hallucinating goals directly in feature space, is very interesting and promising. The lack of other novel ideas could be a weakness of the paper. However, if the gain brought by this additional regularisation is clearly showcased empirically, it can be a relevant contribution. As already mentioned, I want to be sure the ablation about keeping the exact full exploration policy architecture along with the imitation learning action loss term (exactly as it is used in the proposed method) and removing the regularisation term is properly done. I also have a few other concerns (stated in the "Weaknesses" section). I thus tend to consider the work as marginally below the acceptance threshold but am looking forward to clarifications from the authors.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose an approach for visual exploration using RGB cameras only. The approach is also able to build a topological map of the environment. To do the exploration, the authors use imitation learning to hypothesize the image feature in the next step and choose an action based on that feature (and current feature). For topological mapping, the authors use visual place recognition (VLAD and SIFT), which they claim is very efficient.
strength_and_weaknesses: ## Strengths
* The approach outperforms all baselines on the exploration and navigation tasks. The authors choose reasonable baselines for the exploration task.
* The authors provide many details about their approach and architectures in the main paper and appendix.

## Weaknesses
* It seems strange that the locomotion setup has big impact on performance. Is this the case for the baseline approaches as well, or is the proposed approach particularly sensitive to this?
* Missing baselines for navigation (Sec. 5.4). The experiments in this section can be more thorough. For example, [1], [2], [3], [4], [5] are all good baselines to compare against, as they build a topological memory from exploration trajectories and use them for navigation (e.g. image-goal).
    * In appendix E.2 I find it surprising that SPTM takes so long to run. Is the topological map very dense?

## Notes
* Some additional related works on topological map construction and navigation are [1] and [2].
    * [1] is also similar to Savinov et al. They construct a topological map and connect edges using reachability. They utilize an exploration trajectory and then build a topological map from that.
    * [2] In particular, the topological map here is similar to the proposed approach where each node is a panorama. They utilize an exploration trajectory and then build a topological map from that.

* [1] Meng et al. 2020. Scaling local control to large-scale topological navigation. 
* [2] Chen et al. 2021. Topological Planning with Transformers for Vision-and-Language Navigation. 
* [3] Chaplot et al. 2020. Neural topological SLAM.
* [4] Savinov et al. 2018. Semi-parametric topological memory for navigation.
clarity,_quality,_novelty_and_reproducibility: The work is original and the writing is clear. While the authors included many details of their approach in their work, I did not see that they will release the code.
summary_of_the_review: The proposed approach outperforms the baselines for the exploration task, but I think the navigation task experiments could be more thorough, especially since the topological map is one of the main contributions of the paper and this is how the constructed topological map is used.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a framework of topological mapping, which is implemented as a pipeline of two main components including a planner to generate robot navigational actions and a visual place recognition (VPR) method to construct a topological map.
strength_and_weaknesses: Strength: 

+ Topological mapping is an important problem for agent navigation.
+ The paper is well written; each method in the pipeline is well described.

Weakness:

- Novelty and contribution to address active mapping is a key concern. 
-- The paper seems to argue the contribution is on "active" mapping. However, it is not clear how exactly the approach allows an agent to "actively" map the environment. The agent simply uses the policy learns from a demonstrator, and no active actions are taken during the execution to map the environment.  
-- Each component also uses or marginally extends existing methods, and the novelty of each component is not well justified. For example, the idea of using a network to learn representations of the environment and use the representations to generate agent actions is commonly used; the VPR component also uses the previous methods.

- The term Task Planner in Section 3.1 is very confusing. The approach seems to simply predict the future feature vector in the next time step given a short sequence of the past features. What is the definition of tasks? Or what is the difference between task planning and feature prediction? 

- Although the related work section reviews task and motion planning mainly for manipulation, the paper does not review and compare with a large body of local planning and control methods for navigation in the robotics community, which is more relevant to the paper’s problem domain.

- Why predicting the future feature vector in the next step is necessary? If the approach is able to predict a high-dimensional feature vector in the next step, then why the approach cannot directly predict the low-dimensional future action in the next step?

clarity,_quality,_novelty_and_reproducibility: The novelty and originality of the work are a concern since the proposed framework is generally based on a pipeline of existing techniques. On the other hand, the paper is well written, and each component in the pipeline is well described.
summary_of_the_review: The paper proposes to address an important problem of active mapping for agent navigation. However, novelty of the paper is a main concern: it is not clear why the topological mapping is "active" and the novelty of each component used in the pipeline is also low.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work follows task and motion planning (TAMP) to design a high-level task planner and a low-level motion planner for more efficient exploration of an indoor scene. The goal is to cover the scene as efficiently as possible to construct a topological map. After the exploration, an off-the-shelf visual place recognition (VPR) system is used to find more edges on the initially constructed map. Finally, the trained action assigner predicts the possible action list causing the transition on the newly detected edges. The main contributions in this work are three-fold:

(1) Design an efficient exploration policy based on TAMP.

(2) Predict a possible next hidden state (imitate expert trajectories) at the task planner and use it as input to the motion planner.

(3) Further, leverage an off-the-shelf VPR method and a learned action assigner to establish the completed topological map reconstruction system.

The experiments on the exploration and the navigation tasks in the Gibson and MP3D environments demonstrate the effectiveness of the proposed framework. 
strength_and_weaknesses: +) The idea of adopting TAMP to the exploration policy design is self-motivated. Moreover, the studied problem, efficient exploration over an unseen scene, is also a meaningful direction. Finally, the final experiments also demonstrate the effectiveness of the proposed framework.

+) The paper is well-written and easy to follow. The teaser figure shows the studied problem clearly, and the following framework figure and model figure illustrate the implantation in detail. Several qualitative results and videos in the main paper and supplementary present a clear visualization.

o) In the “exploretraj_MP3D_q9vSo1VnCiC_1.mp4” video, could authors illustrate why the agent keeps circling at a local area for a while? Is the agent keep executing move_forward and colliding with a wall?

-) The main idea of designing an efficient exploration policy is based on hallucinating features at the next step. However, this idea is similar to visual forecasting or anticipation, and there is no discussion regarding it in the related work. Many works focus on using visual input to predict the feature at the next step in a latent space, such as [1, 2, 3, 4, 5]. There are also many works utilizing future predictions for manipulation or navigation.

-) Although the experimental results demonstrate that the overall framework performs well on the exploration and navigation tasks, the model design and the way to hallucinate features at the next step are well developed. Therefore, the technical contributions in this work are incremental. Nevertheless, it would be appreciated if the authors could explore or share more insights about how to utilize the hallucinate features for motion planning.

[1] Carl Vondrick et al., “Anticipating Visual Representations from Unlabeled Video”, CVPR 2016.

[2] Kuo-Hao Zeng et al., “Visual Forecasting by Imitating Dynamics in Natural Sequences”, ICCV 2017.

[3] Chien-Yi Chang et al., “Procedure Planning in Instructional Videos”, ECCV 2020.

[4] Basura Fernando et al., “Anticipating human actions by correlating past with the future with Jaccard similarity measures”, CVPR 2021.

[5] Dídac Surís et al., “Learning the Predictability of the Future”, CVPR 2021.
clarity,_quality,_novelty_and_reproducibility: Although the proposed framework in this work is self-motivated and presented clearly, there are some missing details and confusing parts:

(1) In the navigation experiment, the number of images collected during the exploration stage would affect the navigation results significantly. Why is the agent allowed 2k/5k steps to collect 2k images per room on Gibson and 5k images per room on MP3D? Is it based on the coverage of the exploration area? Did authors conduct ablation studies on this hyperparameter?

(2) Regarding the ATM_noDeppSup baseline, after removing the feat regularization loss in the task planner and action regularization loss in the motion planner, how to train the model? Based on Equ (5), these two losses are the only training signal to train the model. Does the policy degenerate into a random policy?

(3) Since MP3D has a larger area in the scene, why not train the policy on the MP3D environment, then evaluate on the Gibson environment?
	
(4) What does the ActionAssigner perform on the validation or testing set? Can authors share some evolution numbers on it? Since the constructed topological map is metric-free, this ActionAssigner should play an essential role in the downstream task (e.g., navigation).
	
(5) In Figure II (C), the authors show the error between hallucinated and real-observed features. Can authors share some quantitative evaluation on the hallucinated feature? It is unclear how to interpret what a policy learns in a very high-dimension latent space. Can authors share some insights regarding it?
summary_of_the_review: This work proposes to learn an efficient exploration policy by way of TAMP. The main idea is to train a task planner to hallucinate the next state in a feature space so that the motion planner can predict the action accordingly. Although the idea is self-motivated, the studied problem is essential, and the paper is well-presented with good results, the technical contributions and overall novelty are incremental. In addition, there is a missing discussion on visual forecasting and its application to planning and control in the related work section. Therefore, I would suggest the authors revise the paper based on them.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper had received an unusual number of 5 reviews, which were quite mixed. The reviewers appreciated the idea to hallucinate next goals directly in feature space. However, several critical issues were raised, which the authors' response could not sufficiently answer. This concerned,
- positing the work wrt to prior work and claims of novelty, as in particular the the topological map (on which the paper heavily focuses on) cannot be concsidered as a novelty; Novelty has been assessed as incremental.
- Missing or incomplete ablations.
- Missing comparisons with prior work, in particular topological maps.
- Missing references and descriptions of relevant prior work.

These issues weighted heavily into the decision, the AC recommends rejection.

The AC would like to point out that this decision was not taken on the basis of disagreements on on the definition of terms, like the meaning of the word “active”  in vision/mapping, or others.
justification_for_why_not_higher_score: The Meta-review is self explanatory
justification_for_why_not_lower_score: -
Decision: Reject