Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a method to perform ObjectGoal navigation (i.e. goto the table) based on active semantic mapping. Specifically the proposed approach contains a semantic mapping module that hallucinates unseen areas. This mapping module is initially trained on a set of trajectories that are shortest paths between two points. Then additional trajectories are selected via active learning to maximize the learning signal.

ObjectGoal navigation is performed by training class specific map predictors and selecting goal locations via the upper confidence bound.

This method outperforms prior map-based works (SemExp) on the Habitat Challenge ObjectGoal navigation dataset.
main_review: ### Strengths
- The method is well motivated and all components perform well
- The uncertainty component is used both for active learning and to select goals for ObjectNav
- Ablations are done well and comprehensive
- Code is provided

### Weaknesses

- Computational requirements. Uncertainty is estimated via an ensemble, thus increasing the computation by N(=4). Where other uncertainty estimate methods tried/considered?
- Lack of test-std results. The Habitat Challenge also contains a public leaderboard with a held-out test set (https://eval.ai/web/challenges/challenge-page/802/leaderboard/2195). The authors should submit to this.

### Suggestions for improvement

While I don't expect the authors to compare with these works as they are concurrent (one wasn't even public before the ICLR deadline), I encourage the authors also include Ye et al. (ICCV 2021) and Maksymets et al. (ICCV 2021) in their table for completeness.

Maksymets: https://openaccess.thecvf.com/content/ICCV2021/papers/Maksymets_THDA_Treasure_Hunt_Data_Augmentation_for_Semantic_Navigation_ICCV_2021_paper.pdf
Ye: https://arxiv.org/abs/2104.04112
summary_of_the_review: This paper presents a strong contribution. My main concern is lack of test set results, however I consider this to be a significant issue as the test set is the standard for reporting on this task/dataset.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a novel framework that learns how to construct spatial + semantic maps for target-driven semantic (ObjectGoal) navigation. The scene maps being learned can also hallucinate beyond the immediate observed regions by exploiting semantic priors in the scene during the learning process. The key contribution of this work is in two different places of the navigation pipeline: (1) an active learning setting for training the mapper that optimizes selection of maximally informative regions of the environment’s state space (metric locations on the floor-plan) for the agent to navigate towards and (2) incorporating the uncertainty in the mapper’s predictions while selecting the target semantic object on the map learned so far. The novel insight that connects both these contributions is borrowed from literature on intrinsic rewards via disagreement — the variance between the predictions of an ensemble of multiple mappers serves as a proxy for the model’s uncertainty about a specific location / the informativeness of that location for training the mapper.
main_review: Firstly, thanks to the authors for their efforts. This is a very well written and high quality paper and I absolutely enjoyed reading it! In my opinion, the paper scores high on novelty, provides an interesting extension to the status quo on how mappers are generally used by embodied agents and nicely extends ideas from prior work in order to incorporate them into the task at hand. More concretely, I believe the following contributions are particularly the strengths of this work:
+ An active learning formulation for training the mapper by selection of informative waypoints where the mapper is expected to gain maximal amount of information
+ Using the variance of an ensemble of mappers as a proxy for taking into account the current mapper’s uncertainty for selecting goal targets in the map constructed so far: the formulation is elegant and nice leads to a controllable exploitation-exploration trade-off.

The insights presented in the paper are intuitive (learning from maximally informative regions is obviously a more efficient strategy than randomly sampling training trajectories for the mapper), the formulation is elegant (I liked how the same ideas related to variance of predictions applied to both mapper training and goal selection) and overall seem potentially useful for the Embodied AI community.

The paper also contains experiments pertaining to all the logical design choices for their approach and the empirical results largely support the claims made by the authors (refer to the question/suggestions section for some exceptions to this).

The only major complaint I have with the paper is the absence of a thorough comparison with state-of-the-art. The authors do compare with a couple of related contemporary approaches in Tab. 3. However, ObjectNav on MP3D/Habitat is a well established task with an active leaderboard (https://eval.ai/web/challenges/challenge-page/580/leaderboard/1634) and a comparison with the best performing models from the leaderboard seem to be missing.

I also had minor issues with some missing bits of information pertaining to how the overall system is working. I encourage the authors to clarify these points in the discussion phase. For instance,
+ There seem to be a couple of different, but related decision making processes going on: (a) the agent has to select informative waypoints to optimally train the semantic map prediction module and (b) the agent has to select the target semantic goal location on the map built so far while being mindful of the uncertainties of the current semantic predictions. Although both these decisions are made using the same fundamental principles at the core, it is not clear to me the order in which these decisions are made. Does the agent alternate between the two? If I understand correctly, both cannot be happening at the same time. Once either of the modules selects a waypoint, the agent has to move towards the same. Some clarification on how the overall system works would be helpful in this context.
+ The authors mention that the optimizations (eqn. 1-4), in practice, happen over the geo-centric map. Does it happen only for the set of map locations observed so far? That is, are there any constraints on the l_j’s in the geo-centric map?
+ It is not immediately clear what is causing the variance in predictions for the ensemble of mappers. Is it because of different initializations? Or, is the training for each being warm-started on a different subset of the dataset?

Given below are some minor questions/points of clarifications:
+ From the description in Sec. 3.3.2, the theory seems to suggest that using the LowerBound strategy for goal selection is more conservative towards exploration, as compared to the UpperBound strategy. One would expect a higher SPL as a consequence of that (the map predictor training strategy staying the same across both). Why isn’t that the case in Tab. 3 (L2M-Active-LowerBound=9.6 v/s L2M-Active-UpperBound=13.3. SPL)? Also related to this, the “LowerBound" numbers for “Offline" map predictor training is missing (L2M-Offline-LowerBound)
+ There are two rows for the same experiment in Tab.3 (L2M-Active-UpperBound). I suspect the reason is the special case for the SemExp baseline where the comparison happens only for a subset of 6 objects categories. Would be better to mention this explicitly in the text / table caption.
+ This is purely speculative from the reviewer (doesn't factor into the review), but I was wondering if the same ideas would work in the space of features, as opposed to logits, as in current work (that are being projected onto the top-down map). Prior work (https://arxiv.org/abs/2010.01191) has shown that projection of features, rather than projected labels from egocentric segmentations circumvents the "label splatter" issue. I'm curious to know the authors' thoughts on this -- do they think that the same framework on variance of ensembles would work equally well in the semantic feature space?
summary_of_the_review: As mentioned above, I feel that the paper has sufficient novel contributions to warrant an acceptance. I encourage the others to include a discussion on comparisons with SoTA from the leaderboard. Also, there are some minor issues that I hope get resolved as part of the discussions. But overall, keeping in mind the insights, novelty and usefulness of the ideas presented, my vote is to accept.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method for leveraging semantic predictions in unobserved areas to help agent navigation. The strategy is motivated by how humans leverage priors on house layouts to perform navigation and tasks without being the house before. The key idea is to train a semantic segmentation model which can predict plausible class assignments in unseen regions. Then use the uncertainty estimates during navigation to pick goals when trying to reach a particular target category. 
main_review: The paper explains the high level ideas fairly clearly. However, the proposed method has a few components. It would be easier to follow the details if there are more structured algorithm boxes that go with components in section 3. For instance, it is not very clear what the training loss for the segmentation model is does it account for class imbalance etc. Some of the details are present in the appendix but need a bit of hopping around to figure out. 

The motivation and high-level idea makes sense. Leveraging priors on typical semantic arrangement of objects for exploration seems natural. That said leveraging these priors as proposed in the paper requires very precise semantic segmentation maps which are fairly expensive to obtain. It seems like the proposed method in the paper can be applied to bounding boxes which are much cheaper to obtain as opposed to semantic segmentation. 

The uncertainty estimation relies on an ensemble of models trained for the semantic map prediction. This can be fairly expensive especially with semantic segmentation models. There are other strategies to estimate uncertainty using a single model (https://arxiv.org/pdf/1506.02142.pdf). Have the authors considered these alternatives?

The experimental evaluation focuses on comparison with baselines from chaplot et.al where the problem setup and assumptions are a bit different. However, the overall pipeline seems fairly similar and leverages similar high-level ideas (https://arxiv.org/pdf/2006.09367.pdf). It is unclear what the main distinctions from this line work the current paper is contributing. Moreover, semantic segmentation is typically less robust that instance segmentation and tends to ignore smaller objects. In the experimental results is there a breakdown of navigation queries to semantic categories which are small and are only represented by a small fraction of the training data. 

A simple baseline that seems to be missing is just using the projected semantic segmentation map without trying to guess the semantic layout in unseen regions to perform the navigation tasks. Are the navigation queries such that they cannot be solved with just what is observed is that ensured in the experiments?
summary_of_the_review: The high-level ideas the paper is based seem natural and interesting. However, fairly similar ideas have been proposed in prior work which is cited by the paper. It seems to me that the distinctions are mostly in the details a lot of which emerge from the slight differences in the problem setup. 

Post-discussion: The authors have addressed the main concerns about distinctions from prior work. Therefore, I am improving my rating.
review_title: null

Meta-review: This paper addresses the problem of goal navigation in unseen environments by learning to build a local, then a registered, global occupancy and semantic map of object categories from reprojected RGB+D observations, while extrapolating (hallucinating) unseen observations from contextual semantic priors (e.g., "tables are usually surrounded by chairs"). It then uses a measure of epistemic uncertainty on different estimations (realisations) of that map as a navigation goal selection policy to perform active exploration, and controls the agent using a local goal-driven policy; different information gain metrics are investigated. Essentially, the policy accumulates the predicted semantic maps and uses the uncertainty of the semantic mapping to select informative goals. The semantic map predictor is implemented as three U-nets for occupancy extrapolation from depth projection, semantic segmentation of RGB, and semantic map inference from the ground projection and extrapolated depth projection maps. The whole method is evaluated on the Matterport3D environment.

Reviewers praised the well-written and comprehensively-evaluated study, the active learning formulation and the idea of epistemic information gain as a measure of uncertainty for goal selection, and the code availability. Reviewers' major concerns included the computational cost of ensemble-based uncertainty estimation (Nmyk), and a missing submission to an active learderboard of the habitat-challenge (Nmyk, CF2f). Reviewers CF2f and qy8x had a longer list of issues that have been addressed in the rebuttal.

Reviewers engaged in a discussion with the authors, and the scores are 5, 6 (though not updated) and 8. I believe that the paper just meets the conference acceptance bar and would advocate for its inclusion in the conference.
Decision: Accept (Poster)