Learning to Map for Active Semantic Goal Navigation
Authors: Georgios Georgakis
Bernadette Bucher
Karl Schmeckpeper
Siddharth Singh
Kostas Daniilidis
Venue: ICLR 2022 Poster (2022)
Keywords: visual navigation
semantic map
uncertainty estimation

Abstract: We consider the problem of object goal navigation in unseen environments. Solving this problem requires learning of contextual semantic priors, a challenging endeavour given the spatial and semantic variability of indoor environments. Current methods learn to implicitly encode these priors through goal-oriented navigation policy functions operating on spatial representations that are limited to the agent's observable areas. In this work, we propose a novel framework that actively learns to generate semantic maps outside the field of view of the agent and leverages the uncertainty over the semantic classes in the unobserved areas to decide on long term goals. We demonstrate that through this spatial prediction strategy, we are able to learn semantic priors in scenes that can be leveraged in unknown environments. Additionally, we show how different objectives can be defined by balancing exploration with exploitation during searching for semantic targets. Our method is validated in the visually realistic environments of the Matterport3D dataset and show improved results on object goal navigation over competitive baselines.

Published as a conference paper at ICLR 2022
LEARNING TO MAP FOR ACTIVE SEMANTIC GOAL NAVIGATION
Georgios Georgakis*| Bernadette Bucher Karl Schmeckpeper', ,iddharthh Singh', Kostas Daniilidis'
University of Pennsylvania, AAmazon (ggeorgak, bucherb, karls @seas,. upenn edu, www.saaaataaaaaaaaa...aaaaaeeaaa amazon com, gostas@cis.upenas edu
ABSTRACT
We consider the problem of object goal navigation in unseen environments. Solving this problem requires learning of contextual semantic priors, challenging endeav- our given the spatial and semantic variability of indoor environments. Current methods learn implicitly encode these priors through goal-oriented navigation policy functions operating on spatial representations that are limited to the agent's s observable areas. this work, we propose novel framework that actively learns to generate semantic maps outside the field of view of the agent and leverages the uncertainty over the semantic classes in the unobserved areas to decide on long term goals. We demonstrate that through this spatial prediction strategy, we are able learn semantic priors in scenes that can be leveraged unknown environ- ments. Additionally, exploration with exploitation during searching for semantic targets. Our method is validated in the visually realistic environments of the Matterport3D dataset and show improved results on object goal navigation over competitive baselines.
show how different objectives can be defined by balancing
INTRODUCTION
What enables biological systems to successfully navigate semantic targets in novel environments? Consider the example of dog whose food tray at its own house is situated next to the fridge. Upon entering new house for the first time, the dog will look for its food tray next to the fridge, even though the new house can largely differ in appearance and layout. This is remarkable, as it suggests that the dog is able encode spatial associations between semantic entities that can be leveraged when trying to accomplish a navigation task. Humans exhibit the same skills in similar scenarios, albeit more nuanced, since given existing observations knowledge over the semantic structure of the world, or to continue exploring the environment. In other words, if we have a partial view of a room containing an oven, we can infer that a fridge most likely exists in the unobserved space. In addition, if we are trying to reach the sofa, then we can infer with high certainty that will be located in a different room. This implies that we have internal mechanisms for quantifying the uncertainty of inferred information from unobserved spaces, which guides our decision making process.
can consciously choose trust our prior
Inspired by these observations, in this work, we study the problem of object goal navigation for robotic agents in unseen environments and propose an active learning method for encoding semantic priors in indoor scenes. Our approach involves learning a mapping model than can predict (hallucinate) semantics in unobserved regions of the map containing both objects (c.g. chairs, beds) and structures (e.g. floor, wall), and during testing uses the uncertainty over these predictions to plan path towards the target. Contrary to traditional approaches for mapping and navigation (Cadena et al., 2016) (i.e. SLAM) where the focus is on building accurate 3D metric maps, our uncertainty formulation is designed to capture our lack of confidence about whether a certain object exists at a particular location. This results much more meaningful representation, suitable for target-driven tasks.
Recently, learned approaches to navigation have been gaining popularity, where initial efforts in addressing target-driven navigation focused on end-to-end reactive approaches that learn map pixels directly to actions (Zhu et al. 2017 Mousavian et al. 2019). These methods do not have an
Denotes equal contribution.

---

Published as a conference paper at ICLR 2022
explicit representation of the environment and tend suffer from poor generalization. To remedy this issue, most current methods learn a map representation that enables the encoding of prior information about the geometry and semantics of a scene, acting as episodic memory (Chaplot et al. 2020b at Gupta et al. 2017 Georgakis et al. 2019). However, maps created by these methods are restricted to contain information only from areas that the agent has directly observed, which led to the introduction of spatial prediction models that either anticipate occupancy (Santhosh Kumar Ramakrishnan & Grauman 2020) room layouts (Narasimhan et al. 2020) beyond the agent's 's field of view and demonstrated improved performance on navigation tasks. Our work differs from these methods in three principled ways: 1) We formulate an active training strategy for learning the semantic maps, 2) we exploit the uncertainty over the predictions in the planning process, and 3) in contrast to predicting occupancy, our model tackles a harder problem which requires learning semantic patterns (e.g. tables surrounded by chairs).
In this work we introduce Learning to Map (L.2M), novel framework for object-goal navigation consisting of two parts. First, we actively learn an ensemble of two-stage segmentation models by choosing training samples through an information gain objective. The models operate on top-down maps and predict both occupaney and semantic regions. Second, we estimate the model uncertainty through the disagreement between models in an ensemble from Pathak et al.] (2019); [Seung et al. (1992), and show its effectiveness in defining objectives in planners to actively select long-term goals for semantic navigation. In addition, we investigate different information gain objectives during active training and illustrate how the use of model uncertainty can balance exploration with exploitation in finding semantic targets. Our proposed approach demonstrates improved success rates on the object-goal navigation task over competitive baselines in the Matterport3D (Chang al. 2017 dataset using the Habitat (Savva et al. 2019) simulator.
2 RELATED WORK
Semantic SLAM. Classical approaches for navigation focus on building 3D representations of the environment before considering downstream tasks (Cadena et al. 2016). While these methods are typically geometric, several SLAM methods have attempted to associate semantic information to the reconstructed geometric map, mainly at the object-level (Salas- Moreno et al.] 2013] Yang & &Scherer 2019 McCormac al., 2018 Bowman et al. 2017 Kostavelis & Gasteratos (015)). For example, in McCormac et al. (2018 8) instance segmentations predicted by Mask R-CNN are incorporated to facilitate per-object reconstructions, while the work Bowman et al. (2017) proposes a probabilistic formulation to address uncertain object data association. However, SLAM systems rarely consider active exploration as they are not naturally compatible with task-driven learnable representations from deep learning architectures that can encode semantic information. Other recent works (Katsumata jet al.] 2020; Cartillier et al., (02T) have sought to build 2D semantic maps and focused either on semantic transfer of global scene assumed the environments were accessible before-hand. In contrast, our proposed approach tackles the object goal task in unknown environments by actively learning how to predict semantics in both observed and unobserved areas of the map around the agent.
Learning based navigation methods. There has been a recent surge of learning based methods (Zhu
ct al. 2017: Mousavian et al.] 22199; Gupta et al.) 2017: Chen et al. 2020 Chaplot et al.) 2020b
Fang et al. 2019; Yang et al. 2019: Ye et al. 2222
Zhang et al. 2021 Chattopadhyy et al. 2021
for indoor navigation tasks (Anderson et al. 2018 Batra al. 2020
Das et al. 2018), propelled
by the introduction of high quality simulators such as Gibson (Xia et al.] 2018). Habitat (Savaa
let al. 2019). and A12-THOR (Kolve et al. (017). Methods which use explicit task-dependent
map representations (Parisotto & Salakhutdinov 20188] Gupta et al. 2017 Chaplot et al., 2020b a Georgakis al. 2019 Gordon et al. 2018) Mishkin et al. 2019)) have shown to generalize better in unknown environments than end-to-end approaches with implicit world representations. For example, in/uuta et al. (2017) a differentiable mapper learns to predict top-down egocentric views of the scene from RGB images, followed by differentiable planner, while in Chaplot et al.) (2020a) Mask R-CNN is used to build a top-down semantic map of the scene used by a learned policy that predicts goal locations in the map. More conceptually similar to our method, are approaches that attempt encode semantic or layout priors by learning to predict outside the field-of-vied of the agent (Santhosh Kumar Ramakrishnan & Grauman 2020 Liang et al. 2021 Narasimhan al.] 2020 Georgakis et al. 2022). In contrast to all these works we formulate an active, turget-independemt strategy to predict semantic maps and define goal selection objectives.
2

---

Published as a conference paper at ICLR 2022
Semantic Map Prediction
Registration
Policy
Selection
RGB
Pa
Prediction
Giobal Map
Uncertainty
Local Policy
Action
Figure 1: Overview of our approach. We select goals based on the predicted probability of location containing our target and the uncertainty of the prediction. Note that in this particular example, the sofa to the right of the agent not visible, but is correctly predicted by our method.
Uncertainty Estimation. Many recent works estimate uncertainty of deep learning models (Gal 2016) Abdar et al., 202T). some cases, these uncertainty estimates can be used as objectives for active exploration (Sekar al. 2020 Pathak et al. 2019) since maximizing epistemic uncertainty is used as a proxy for maximizing information gain (Seung et al.) 1992. Pathak et al.) (019).. Uncertainty estimates of deep learning models able be used objectives for active exploration fall into two categories: Bayesian neural networks and ensembles. [akshminarayanan et al. 2017); Gawlikowski al. 202I) found the multiple passes required by Bayesian methods for every prediction can result in a higher computational cost than small ensembles. In addition, ensembles are simpler to implement with little hyperparameter tuning in comparison Bayesian methods, leading to focus our work on ensemble based approaches. We tested estimating epistemic uncertainty with entropy (Shannon 1948). an approximation of model information gain using predictive entropy (BALD) (Houlsby et al. 2011), D, and variance between model outputs (Seung et al. 1992). We found maximizing the variance yielded the best performance as an objective to actively fine-tune our map prediction models. This training procedure is similar to the active learning approaches leveraged during training presented by Bucher et al. (2021), Chaplot et al. (2020c): [Sener & Savarese (2018). We also use our epistemic uncertainty estimate construct confidence bounds for estimated probability distribution which we use to select goals for target-driven navigation at test time. Both lower (Galichet et al.] (2013) and upper (Auer al.) (2022) confidence bound strategies for balancing exploration, exploitation, and safety have been previously proposed in the multi-armed bandit literature and extended for use in MDPs Azar al. 2017) and reinforcement learning (Chen al. 2017
APPROACH
We present a new framework for object-goal navigation that uses a learned semantic map predictor to select informative goals. In contrast prior work (Santhosh Kumar Ramakrishnan & Grauman 2020; Chaplot et al.] 2020a), we leverage the predictions outside the field-of-view of the agent to formulate uncertainty-bant
goal selection policies. Furthermore, we actively collect data for training the map predictor and investigate different information gain objectives. Due to our goal selection policy formulation. our method does not need to be trained specifically predict goals for every target object, enabling target-ine ingeeeddndeeeneent learning of the semantic priors. Our method takes as input an RGB-D observation, and predicts semantics in the unobserved areas of the map. This is followed by goal selection based on the estimated uncertainty of the predictions. Finally local policy is responsible for reaching the goal. An overview of our pipeline can be seen in Figure]
3.1 SEMANTIC MAP PREDICTION
We describe a method for learning how to map by predicting the semantic information outside the field of view of the agent. We emphasize that this goes beyond traditional mapping (i.e. accumulating multiple views in an agent's path) as it relies on prior information encoded as spatial associations between semantic entities in order to hallucinate the missing information. Motivated by the past success of semantic segmentation models in learning contextual information (Zhang et al, 2018 Yuan et al. 2020), we formulate the semantic map prediction two-stage segmentation problem.
3

---

Published as a conference paper at ICLR 2022
Death Projection
fo
Depth
Pt
Projection
RGS
Semanci Segmentation
Figure 2: Overview of semantic map predictor approach for single time-step. The model predicts the top-down egocentric semantics of unobserved areas in two-step procedure. First, the occupancy is predicted, which is concatenated with ground-projected semantic segmentation of the RGB observation before producing the final output ri. Note that our model learns to predict semantically plausible maps (i.e. chairs surrounding a table) as shown in this example.
Our method takes as input an incomplete occupancy region Pt E C'Ixxx and a ground- projeeeed semantic segmentation & R|C||xxxw at time-step 4. The output a top-down semantic local region m, TR|C") [xlx") where C the set of occupancy classes containing unknown, occupied, and free, C is the set of semantic classes, and h, w are the dimensions of the local crop. To obtain Pt. we use the provided camera intrinsics and depth observation at time to first get a point cloud which is then discretized and ground-proo prounddpoojected similar to Santhosh Kumar Ramakrishnan & Grauman (2020). To estimate & we first train a UNet (Ronneberger et al. (0135) model to predict the semantic segmentation of the RGB observation at time t. All local regions egocentric, i.e., the agent is in the middle of the crop looking upwards. Each spatial location in our map (cell) has dimensions 10cm 10cm.
The proposed two-stage segmentation model predicts the hallucinated semantic region in two stages. First, estimate the missing values for the occupancy crop in the unobserved areas by learning to hallucinate unseen spatial configurations based on what is already observed. Second, given predicted occupancy, we predict the final semantic region my, These steps are realized as two UNet encoder-decoder models, f that predicts in occupancy space Po: = {p:; 8*), and f that predicts in semantic space mn (f: e &y; (*), where fe is the predicted local occupancy crop which includes unobserved regions, @ refers to the concatenation operation, and 0%, 0" are the randomly initialized weights of the occupancy and semantic networks respectively. The image segmentation model is trained independently and its ground projected output 8, conditions f on the egocentric single-view observation of the agent. The model is trained end-to-end using pixel-wise cross-entropy losses for both occupancy and semantic classes and predicts probability distribution over the classes for each map location. We assume that ground-truth semantic information is available such that can generate egocentric top-down occupancy and semantic examples. This combined objective incentivizes learning to predict plausible semantic regions by having the semantic loss backpropagating gradients affecting both f and fa. Also, performing this procedure enables the initial ballucination of unknown areas over small set of classes C before expanding to the more difficult task of predicting semantic categories C* which includes predicting the placement objects together with scene structures such as walls in the map. An overview of the semantic map predictor can be seen in Figure 2 During navigation episode, the local semantic region is registered to a global map which is used during planning. Since we predict probability distribution at each location over the set of classes, the local regions are registered using Bayes Theorem. The global map is initialized with uniform prior probability distribution across all classes.
3.2 UNCERTAINTY AS AN OBJECTIVE
key component of a robotic system is its capacity to model what does not know. This ability enables agent to identify failure cases and make decisions about whether to trust its predictions or continue exploring. In our semantic map prediction problem, estimating the uncertainty over the semantic predictions at each location of the map enables understanding what the model does not know. We considered two types of uncertainty we face in modeling vision problems with deep learning: aleatoric and epistemic uncertainty (Kendall & Gal 2017 Gal 2016). First, aleatoric uncertainty
4

---

Published as a conference paper at ICLR 2022
the system uncertainty. Suppose the true probability of sofa being at specific location given an observation is 30%. Consider a scenario where our target is sofa and our model estimates the true probability of 30% that sofa is at the specific location. Our model would be correct regardless of whether the sofa is present at that location. This uncertainty is reflected in the probability output of our model. We denote this model f (pt, 84; 0) my where I are the parameters of f.
Second, epistemic uncertainty captures the uncertainty over the model's parameters. In training. our objective is to improve the prediction model by identifying cases it underperforms. We use epistemic uncertainty to formulate this objective, as samples with high epistemic uncertainty are associated with increased information gain. We recall that f is a classifier trained with the cross- entropy loss, so the output of f is probability distribution. In order to estimate epistemic uncertainty, we consider the probabilistic interpretation P(melpt, 8t,0) 0) of our model f which defines likeli- hood function over the parameters 0. The parameters 0 are random variables sampled from the distribution (()). We construct f as an ensemble of two-stage segmentation models defined over the parameters {O,..... 3. Variance between models in the ensemble comes from different ran- dom weight initializations each network (Pathak et al.) 2019; /ekar et al. 2020p. Our model estimates the true probability distribution P(melpe, 6t) by averaging
sampled model weights, N. Ei= fpr, Se 0,) [lakshminarayanan et al. 2017 Gal et al. 2017). Then, following prior work (Seung et al. 1992 Pathak et al. 2019). the epistemic uncertainty can be approximated from the variance between the outputs of the models in the ensemble, Varf $x; 0). We use uncertainty estimation in two distinct ways in our method. First, during training of the semantic predictor we actively select locations of the map with high information gain (Section B.2.I). Second, during object- -oal navigation we actively choose long-term goals that encourage the agent to explore in search of the target object (Section 3.3)
P(me Pt. 8t) R Egwg(e) (pt, Se; šc; Ri
3.2.1 ACTIVE TRAINING
A typical procedure for training the semantic map predictor would be to sample observations along the shortest path between two randomly selected locations in scene. However, this results in collecting a large amount of observations from repetitive or generally uninteresting areas (e.g. spaces devoid of objects of interest). We use that naive strategy for pre-training (around 900K examples), and formulate a following step where we actively collect training samples using an information gain objective. We choose destinations for which the predictions of our pre-trained models maximize this objective. Since we can interpret our hallucinated map me as a prediction {$(+1...., $1TT} of the semantic segmentation of future observations over some window T, our greedy objective for information gain allows us to collect data in directions where we expect to make the most informative observations. The agent then moves using a local policy (described in Section 3.3.3). The models are then fine-tuned using the collected training examples (around 500K).
We evaluate which observations will be useful data for training by selecting (1r, v)-grid locations to maximize an approximation of I [m, 0 Pt, &t). [(m;; 0lp,, 6t) denotes the information gain with respect to the map my from an update of the model parameters 0 given the occupancy observation pe and semantic map crop 6,. For brevity, we specify grid location L E (4.....4 44) where k hw for an h x w map region over which our model f estimates rn. We select locations from the map which have the maximum epistemic uncertainty let al. 2019 Seung et al. 1992). To this end, we define the average epistemic uncertainty across all classes. We select locations ly from the map at time with the greedy policy
proxy for maximizing information gain (Pathak)
arg max [(m;; 8t) F arg max 1,
1
Σ
Varf 8):
C
In practice, these locations are selected from the accumulated uncertainty estimates in the global map. Alternatives to our chosen active training strategy include policies maximizing entropy (Shannon) 1948) or an approximation of model information gain using predictive entropy (BALD) (Houlsby ct al. 2011) We experimentally compare these alternatives to our approach in Section 4.1
3.3
GOAL NAVIGATION POLICY
We study the problem of target-driven navigation within novel environments, which can be formulated as partially observable Markov decision process (POMDP) (S, A, O, P(s'|s, s, a), R(s, a)). We
5

---

Published as a conference paper at ICLR 2022
interested in defining policy that outputs goal locations close as possible to target class The state space S consists of the agent's 's pose æ and the semantic predictions my accumulated over time in the global map. The action space A is comprised of the discrete set of locations h over the map. The observation space O are the RGB-D egocentric observations, and P(s'|s ((5'|s,0 a) are the transition probabilities. A common reward choice for a supervised policy would be R(s, a) D(s, ( which is the distance reduced between the agent and the target, where D(...) is distance on the shortest path. However, this results in target-dependent learned policy, which would require re- re-training when novel target is defined. Therefore, we formulate a policy which accumulates the predicted semantic crops m at every time-step. and leverages the predicted class probabilities along with uncertainty estimation over the map locations to select informative goals.
3.3.1 UPPER CONFIDENCE BOUND FOR GOAL SELECTION
We now use our uncertainty-drivee approach to exploration to explicitly propose an objective for goal selection. During task execution at test. time, I cannot gain information because we do not update the model online with agent observations. However, the agent gains information by accumulating observations and successive predictions in the global map. We construct a policy in order to select goals from unobserved map locations using this accumulated information. The idea behind our policy simple; if the agent is not very confident of where the target is situated, then it should prioritize exploration, otherwise it should focus more on exploiting its knowledge of candidate goal locations.
Since our task is target-driven, we can narrow our information gain objective to reduce uncertainty in areas of the map with the highest uncertainty about the target class. We denote Se as the function which only returns the values for a given target class c. For class c, our ensemble fe estimates Po (m( (Pe, 8z), the probability class e is at location i given observation pt and semantic segmenta- tion & for each map location. The target class uncertainty at test time is given by the variance over the target class predictions Varfe(Pt. Sc; 0).
We propose selecting goals using the upper confidence bound of our estimate of the true probability (m (Pt, $2) in order to select locations with high payoff's but also high potential for our model to gain new information. Upper confidence bounds have long been used to balance exploration and
exploitation in problems with planning under uncertainty (Auer et al. 2002 Azar et al.. 2017 Chen al. 2017). We denote (p 8,) / /Varfo(pt, 8z; 0) as the standard deviation of the target class probability, and we denote (pt, 8x) = Ei fe(pt, &e; 0z). Then, we observe the upper bound Po (p: &,) S pe(P,, &,) + (Pt, s,) holds with some fixed but unknown probability where a is a constant hyperparameter. Following [Cenn et al. (2017). we use this upper bound and select goals from any map region the agent has observed or hallucinated with the policy
max (He(pi, $t) + a1oc(pe,
(2)
4
In practice, this is evaluated over our predictions and uncertainty estimations accumulated over time.
3.3.2 ALTERNATIVE STRATEGIES
While we found our proposed upper bound strategy to choose the best candidate goal locations, we hypothesized several alternative policies which yield competitive performance. We consider the following interpretations of our uncertainty and probability estimates for object goal navigation.
High (Pt, $t), Low (e(P., $:): we are fairly certain we found the target object
High (e $t), High (pt, $t) we are uncertain we found the target object
Low (o(p,, 8t), High (pe, s,): : we are uncertain we did not find the target object
Low pe(pt, 8t), Low (p 6t): we are fairly certain we did not find the target object
From these interpretations see that our upper bound strategy risks choosing locations with high potential information gain (high variance) over locations where we are fairly certain we found the target object (high probability, low variance). To consider other exploration and exploitation tradeoffs, we also observe the lower bound Pa(m, [Pt. $.) 2 Pe (Pt, &t) a1 & holds with some fixed but unknown probability. Then, we can formulate the following alternative goal selection strategies.
Lower Bound Strategy. Lower bound strategies optimize safety (Galichet et al.) 2013). We can differ- entiate between multiple locations which have high probability of containing our target class by choos-

---

Published as a conference paper at ICLR 2022
Method
1%)
Soft SPI. (P))
Success (%)
DTS (m) 4
9.4....4
15.5 1.4.0.4 0.4
29.3
3.666 1 0.074
9.6 0.4
16.0 + 0.4
0.9
3.528 + 0.070
BALD-
9.6 0.4
15.7 1: 0.4
30.4
3,664 1 0.074
BEMM-Acixe-UpperBoa
13.3 + 0.5
19.1 + 0.4
34.3 + 0.9
3.495 0.077
Table 1: Comparison of different active training strategies on object-goal navigation.
ing the one with the lowest uncertainty with the objective: arg max,, (Me(Pr,)))) - -01 However, this strategy does not explore regions with high uncertainty to gain information.
Mixed Strategy. We can try to balance the pros and cons of the lower and upper bound strategies by switching between the two based on how high the probability is that we found the target object. We tune hyperparameter 0g to determine the cutoffs for "high" and "low" values of pe (p &t). We select goals with the objective: arg max,, (Me(Pt, &t) + sgn(a2 - /ee(Pe, (.)))11..(Pt 8z)), so that we choose a safe strategy via our lower bounds when the probability of the class at a location is high and an exploration strategy via our upper bounds when the probability of the class is not high.
Mean Strategy. To evaluate whether our uncertainty estimate is useful, we also consider the following objective which does not incorporate uncertainty; arg max,, (e &t).
3.3.3 LOCAL POLICY
Finally, in order to reach selected goal in the map, we employ the off-the-shelf deep reinforcement learning model DD-PPO (Wijmans et al. 2019) without re- re-training. This model is trained for the task of point-goal navigation and at each time-step receives the egocentric depth observation and the current goal, and outputs the next navigation action for our agent.
4 EXPERIMENTS
We perform experiments the Matterport3D (MP3D) (Chang et al. 2017) dataset using the Habitat (Savva et al. 2019) sim- ulator. MP3D contains reconstructions of real indoor scenes with large appearance and layout variation, and Habitat pro- vides continuous control for realistic agent movement. We use the standard train/val split as the test set is held-out for the online Habitat challenge, which contains 56 scenes for training and 11 for validation. We conduct three key experiments. First, we evaluate the performance of our semantic map predictor both in terms of navigation and map quality under different
Method
ToU loU(%%
Fl (%)
L2M-Oflline
20.1
30.5
L2M- Entropy
20.7
31.2
L2M-BAL BALD
21.2
31.8
L2M-Active
25.6
38.3
Table 2: Comparison of active train- ing methods in semantic map pre- diction.
active training strategies (sec. T. Second, we compare our method to other navigation strategies on reaching semantic targets and provide ablations over different goal-selection strategies (sec. (sec.44.2). Finally, we conduct an error analysis over the effect of the stop decision and local policy in the overall ensemble size N 4. Trained models and com wttp://owgggeegaaaaaaoooam L
performance (sec. 4.3 For all experiments
code can be found here: httpps:://..tbb...ooo/ whw..
We follow the definition of the object-goal navigation task as described in (Batra al. (2020). Given a semantic target (e.g. chair) the objective is to navigate to any instance of the target in the scene. The agent is spawned semantic map predictor training. The agent has access to RGB-D observations and pose provided by the simulator without noise. We note that estimating the pose from noisy sensor readings out of the scope of this work and can be addressed by incorporating off- off-the-shelf visual odometry (Zhao et al. 2021). The action space consists of MOVE_F FRRARD by 25cm, TURN_ LEFT and TURN_ RIGHT by 10 and STOP. An episode is successful if the agent selects the STOP action within a certain distance (Im) from the target and must be completed within a specific time-budget (500 steps). Unless otherwise stated, for our experiments we use representative set of 11 object goal categories present in MP3D: chair, sofa, bed, cushion, counter, table, plant, toilet, rv, cabinet, fireplace and report the following metrics: (1) Success: percentage of successful episodes. (2) SPL: Success weighted by path length, (3) Soft SPL: Unlike SPL which is 0 for failed episodes, this metric is distance covered
random location in novel scene which was not observed during the
generated 2480 test episodes across the validation scenes. To evaluate all methods
7

---

Published as a conference paper at ICLR 2022
Method
(%)
Sof. (%)
Success (%)
DTS 4
Random Walk
0.2 0.1
0.0
0.1
5,408 1 0.141
Segm. ANS [2] OracleStop
11.1
12.2
=
0.982
L2M-Offine- DOfline-FBE [1]
0.3
10.1
20.6 0.8
1.074
E2M-Ofline-Upper/i
9.6+0.4
0.4
30.1 -0,9
3,528
+
L2M-Actixe-Meuun
0.4
15.7 0.4
29.8
J2M-Active-Lowerie
9.6
1..7
30.4-099 0.9
1.003
-eM-Active-Mixed
0..4
16.8 0.4
30.8
3.620+0.007
L2M-Active- AMM-Accii--Upperrouudd
13.3 - 0.5
19.1 = 0.4
34.3 0.9
3.495 + 0.077
SemExp [3]
16.5 0.9
28.1
1.2
4,848 + 0.074
12M-vetive- Sirriive-UppBBBBund
14.8 + 0.7
20.0 + 0.6
34.8
1.3
3.669 + 0.114
Table 3: Comparison against baselines and variations of method. Baselines are adapted from
[1]] Yamauchii (1997). [2] Chaplot et al. (2020b), and [3] CCaplot et al. (2020a). Comparisons above the double horizontal line were carried out on our set of I1 objects while the comparison below the double line against SemExp was carried out on the 6 objects reported in [3]. Note that the discrepancy
between the results of SemExp reported here and those in [3] are due to different sets of test episodes.
towards the goal weighted by path length. (4) DTS: geodesic distance of agent to the goal at the end of the episode. Different variants of our method are evaluated in the following experiments. We follow the naming convention of L2M-X-Y where X, Y correspond to the map predictor training strategy and the goal-selection strategy respectively. For example, -u--eiii--Upp refers to our proposed method that uses the semantic map predictor after active training (Section 3.2.T) and Eq. 2 for goal selection. More details of specific variants are provided in the following sections.
4.1 EVALUATION OVER ACTIVE TRAINING METHODS
We evaluate the impact of our approach to actively training our semantic mapping predictor by comparing different ensemble- based active training strategies over the quality of the predicted maps and object-goal navigation. The semantic map prediction evaluated on the popular segmentation metrics Intersection over Union (IoU), and Fl score. We use nine classes: unknown, floor, wall, bed, chair, cushion, sofa, counter, table. 17900 test examples were collected in the 11 validation scenes which had not been observed during training. The evaluation is con- ducted on predicted map crops of size 64 64 that collected in sequences of length 10. The purpose of this experiment is to ascertain the quality of the predicted mapped area around the agent while it is performing a navigation task. Results are shown in Table [ (object-goal navigation) and Table 2 (map prediction). The variant Offline is our semantic map prediction model without fine-tuning with an active strategy, BALD adapts the BALD objective (Houlsby al.) 2011) for actively training our ensemble, and Entropy refers to our model fine-tuned with an entropy objective for active training (Shannon, 1948). ). For the navigation comparison all methods use the upper bound objective from Eq. While the baselines report similar per- formance to each other, our method gains 4.4% in loU, 6.5% FI, 3.9% Success, and 3.7% SPL from the second best method. This indicates that L2M-Active is more effective in targeting data with high epistemic uncertainty during training and validates our choice for the information gain objective presented in Section 4.1 of the main paper. Figure shows qualitative results of L2M-Active.
Proj.
Sem. Pred.
WAN
tale
chair
counter
cab met
Sreplace
door
Figure 3: Qualitative map predic- tion results.
4.2 COMPARISONS TO OTHER NAVIGATION METHOOSS
Here evaluate L2M against three competitive baselines:
L2M-Offline- We combine the classical Frontier-based exploration (FBE) from Yamauchi (197) for goal selection with map predictor facilitate the stop decision.
8

---

Published as a conference paper at ICLR 2022
Method
(
Soft SPI.(%)
Success (%)
DTS (m) 4
L2M
12.5 1: 0.7
21.1 0.7
32.7 1: 1.6
3.898 + 0.148
L2M + GtPath
17.7 - 0.8
20.6 0.8
50.4 + 1.7
3.545 = 0.175
L2M + OracleStop
33.1 +: 1.3
31.5
50.8 1: 1.7
3.585 1 0.134
1.2M + GtPath + OracleStop
52.3 + 1.3
41.6+ 1.0
80.5 + 1.4 2.620 + 0.150
Table 4: Ablations of our method that investigate the effect of stop decision and local policy.
Segm+ANS+OOradeSuup: This baseline uses Active Neural SLAM (ANS) as an exploration policy to traverse the map and image semantic segmentation detect objects. If the target object is detected, the agent navigates to that goal, and an oracle decides to stop the episode if the agent reaches the correct target. The agent does not have access semantic map.
SemExp: The method proposed in Chaplot et al. (2020a) which was the winner of the CVPR 2020 Habitat ObjectNav Challenge. Since the model that was used in the Habitat challenge is not publicly available, we compare against the six object categories reported in Chaplot et al. (2020a) using the variant of this method that used Mask R-CNN. Furthermore, since the MP3D evaluation episodes used in Chaplot et al. (2020a) are also not available, we ran SemExp on our set of evaluation episodes.
In addition, we evaluate variations of our method (LowerBound, Mixed, and Mean) as defined in section 3.3.2 with 01 0.1 and 022 = 0.75. Our results are presented in Table 3 We observe method outperformed all baselines in terms of success rate by
that our L2M-Active-Uppee SMM--Attie-UpperBune significant margin and is comparable to SemExp in terms of SPL. This result is not surprising since our upper bound strategy often chooses goals that maximize information gain over the map for effective exploration rather than choosing the shortest path to the goal. Interestingly, -ff-Offiine- FBE outperforms Segm. +ANS OraclettStto even though the latter has access to the stop oracle (which results in a high SPL. performance). This demonstrates the advantage of having access our map prediction module for the object-goal navigation task. Furthermore, any performance gains of our method towards 12M-Offflne-FFl L2M variations, the upper bound strategy performed best across all metrics. We note that we expect the mixed and upper bound strategies to have close performance results since by definition the mixed strategy executes the upper bound strategy whenever the probability of the target class at given location is less than (22 0.75.
are a direct result of our goal selection strategies. Regarding our
4.3 ERROR ANALYSIS
common source of failure in navigation tasks deciding to stop outside the success radius of the target. In this last experiment we investigate the effect of the stop decision in failure cases of our model by defining an oracle that provides our model with the stop decision within success distance of the target (OracleStop). In addition, we explore the contribution of the local policy in failure cases by replacing it with shortest paths estimated by the habitat simulator towards our selected goals (GiPuth). The rest of the components follow our proposed L2M-Active ActtieeUUpperouueee The evaluation for this experiment was carried out on a subset of 795 test episodes which are harder due to larger geodesic to euclidean distance ratio between the start position and the target and have larger mean geodesic illustrates our findings. We observe a significant
distance than the rest of our test episodes. Table 4 increase of performance for all baselines. In the case of L2M GtPath the performance gap suggests that the local policy has difficulties reaching the goals, while in the case of L2M + OracleSTOP it suggests that our model selects well positioned goals but our stop decision criteria fail to recognize a goal state. Finally, L2M + GtPath + OracleSTOP achieves mean success rate of 80%, thus advising further investigation of these components of our pipeline.
CONCLUSION
We presented Learning to Map (L2MM, a novel framework for object-goal navigation that leverages semantic predictions in unobserved areas of the map to define uncertainty-bo objectives for goal selection. In addition, the uncertainty of the model is used as an information gain objective to actively sample data and train the semantic predictor. We investigated different information gain objectives and found epistemic uncertainty to be the most effective for this problem. Furthermore, we proposed multiple goal-selection strategies and observed that balancing exploration and exploitation using upper confidence bounds of our predictions produced higher performance. Finally, our method outperformed competitive baselines on the Matterport3D dataset for object-goal navigation.

---

Published as a conference paper at ICLR 2022
Ethics Statement. Our work advances the capability of autonomous robots navigate in novel environments which can help create a positive social impact through technologies such as robot caregivers for medically underserved populations. However, which can yield negative social consequences. Our semantic hallucination method does not model -ittr---istritttion scenes and is trained on data from homes in North America and Europe. If utilized for safety critical tasks such as medical caregiving in hospitals instead of homes or in homes in different regions of the world, our method would act on biases driven by home structure in the training set with potentially harmful outcomes. Another technical limitation of our work is our inability to model 3D relationships. We ground project 3D information from depth 2D map representation for occupancy, thus losing 3D spatial context. This can be important for objects such as cushions which are often on top of other objects such as sofas. Losing this context potentially contributes to our lower success rate on specific objects.
approach has technical limitations
Reproducibility Statement. In Section I we include a footnote after the first sentence of the section with link to our GitHub repository. This repository includes the code for our models and instructions for reproducing our results. We provide both a Docker image with the dependencies for running our code and instructions to install the required dependencies without using Docker. We give instructions for generating initial training data from the Habitat simulator as well as instructions for collecting training data using our active policy. We have instructions for training and testing all of the model variants described in our work. We provide Google Drive links to the test episodes we used to evaluate our models as well as the MP3D scene point clouds which include the semantic category labels for the 3D point clouds. Each of our trained models is also shared via Google Drive links, and we link to the pre-trained DD-PPO model provided by the authors of that work which we leverage in our experiments. In addition, implementation details can be found in section AA. of the Appendix where we describe the hyperparameter values used during our experiments, the training procedure that we followed for the semantic map predictors, and we provide the pseudo-code algorithm for the execution of our method during a navigation episode.
ACKNOWLEDGMENTS
We would like to thank Samuel Xu for implementation support in evaluating baseline methods. Research was sponsored by the Honda Research Institute through the Curious Minded Machines project, by the Army Research Office under Grant Number W91 1NT-220001-00 and by the National Science Foundation under grants CPS 2038873, NSF IIS 1703319.
REFERENCES
Moloud Abdar. Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U Rajendra Acharya, al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 2021.
Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents. arXiv preprint arXiv: 1tX..v:180.007777 2018.
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2):235-25 256, 2002.
Mohammad Gheshlaghi Azar, Ian Osband, and Rémi Munos. Minimax regret bounds for reinforce- ment learning. In International Conference on Machine Learning (ICML), pp. 263-272. PMLR, 2017.
Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006. 13171, 2020.
Sean L Bowman, Nikolay Atanasov, Kostas Daniliiiis, and George J Pappas. Probabilistic data association for semantic slam. In 2017 IEEE international conference on robotics and automation (ICRA), pp. 1722-1729. IEFE, 2017.
10

---

Published as a conference paper at ICLR 2022
Bernadette Bucher, Karl Schmeckpeper, Nikolai Matni, and Kostas Daniilidis. An adversarial objective for scalable exploration. In 2021 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 2670-2677, 2021.
Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, lan Reid, and John Leonard. Past, present, and future of simultaneous localization and mapping: Toward the pebbuttperceptiin age. IEEE Transactions on robotics, 32(6):1309-1133 2016.
Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, and Dhruv Batra. Semantic mapnet: Building allocentric semantic maps and representations from egocentric views. AAAI, 2021.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. International Conference on 3/) Vision (3DV), 2017.
Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, and Ruslan Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. In In Neural Information Processing Systems (NeurIPS), 2020a.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning explore using active neural slam. ICLR, 2020b.
Devendra Singh Chaplot, Helen Jiang, Saurabh Gupta, and Abhinav Gupta. Semantic curiosity for active visual learning. In ECCV, 2020c.
Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, and Aniruddha Kembhavi. Robustnav:
Towards benchmarking robustness in embodied navigation. ICCV, 2021.
Richard Y Chen, Szymon Sidor, Pieter Abbeel, and John Schulman. UCB exploration via e-nsembles. arXiv preprint 1706.01770.00555000 2017.
Tao Chen, Saurabh Gupta, and Abhinav Gupta. Learning exploration policies for navigation. ICLR, 2020.
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-10, 2018.
Kuan Fang, Alexander Toshev, Li Fei-Fei, and Silvio Savarese. Scene memory transformer for embodied agents in horizooizon tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 538-547, 2019.
Yarin Gal. Uncertainty in Deep Learning. PhD thesis, University of Cambridge, 2016.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning (ICML), pp. 1183-1192. PMLR, 2017.
Nicolas Galichet, Michele Sebag, and Olivier Teytaud. Exploration vs exploitation vs safety: Risk- aware multi-armed bandits. In Asian Conference Machine Learning, pp. 245-260. PMLR, 2013.
Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, al. A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.0334. 2021.
Georgios Georgakis, Yimeng Li, and Jana Kosecka. Simultancous mapping and target driven navigation. arXiv preprint arXiv: 2019.
Georgios Georgakis, Bernadette Bucher, Anton Arapin, Karl Schmeckpeper, Nikolai Matni, and Kostas Danilidis.. Uncertainty-drine -divvn planner for exploration and navigation. International Conference in Robotics and Automation (ICRA), 2022.
Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali
Farhadi. Iqa: Visual question answering in interactive environments. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 4099-4098, 2018

---

Published as a conference paper at ICLR 2022
Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2616-2625, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: 2011.
Yuki Katsumata, Akira Taniguchi, Lotfi EI Hafi, Yoshinobu Hagiwara, and Tadahiro Taniguchi. Spcomapgan: Spatial concept formation-bass semantic mapping with generative adversarial networks. In 2020 JEEE/RS/ International Conference Intelligent Robots and Systems (IROS), pp. 7927-7934. IEEE, 2020.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, pp. 5574-5584, 2017.
Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual arXiv preprint /r 2017.
loannis Kostavelis and Antonios Gasteratos. Semantie mapping for mobile robotics tasks: A survey. Robotics and Autonomous Systems, 66:86-103, 2015.
Balaji Lakshminarayaman, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. NIPS, 2017.
Yiging Liang, Boyuan Chen, and Shuran Song, Sscnav: Confidence-aware semantic scene comple- tion for visual semantic navigation. Proc. of The International Conference in Robotics and Automation (ICRA), 2021.
John McCormac, Ronald Clark, Michael Bloesch, Andrew Davison, and Stefan Leutenegger. Fu- sion++; Volumetric object-level slam. In 2018 International Conference on 3D Vision (3DV), pp. 32-41. IEEE, 2018
Dmytro Mishkin, Alexey Dosovitskiy, and Vladlen Koltun. Benchmarking classic and learned navigation in complex 3d environments. arXiv preprint arXiv: 10915, 2019.
Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson. Visual representations for semantic target driven navigation. In 2019 International Conference on Robotics and Automation (ICRA), pp. 8846-8852. IEEE, 2019.
Medhini Narasimhan, Erik Wijmans, Xinlei Chen, Trevor Darrell, Dhruv Batra, Devi Parikh, and Amanpreet Singh. Seeing the un-scene: Learning amodal semantic maps for room navigation. ECCV, 2020.
Emilio Parisotto and Ruslan Salakhutdinov. Neural map: Structured memory for deep reinforcement learning. ICLR, 2018
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Detioo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. NIPS, 2017.
D. Pathak, D. Gandhi, and A. Gupta. Self-Supervised Exploration via Disagreement. ICML, 2019.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer. 2015.
Renato F Moreno, Richard A Newcombe, Hauke Strasdat, Paul HJ Kelly, and Andrew J Davison. Slam++: Simultancous localisation and mapping at the level of objects. In Proceedings of the IEFE conference computer vision and pattern recognition (CVPR), pp. 1352-1359, 2013.
12

---

Published as a conference paper at ICLR 2022
Ziad Al-Halah Santhosh Kumar Ramakrishnan and Kristen Grauman. Occupancy anticipation for efficient exploration and navigation. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE International Conference on Computer Vision (CVPR), pp. 9339-9347, 2019.
Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning to explore via self-supervised world models. ICML, 2020.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: core-set approach. ICLR, 2018.
H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. Query by committee, In Proceedings of the fifth annual workshop on Computational learning theory, pp. 287-294, 1992.
Claude E Shannon. A mathematical theory communication. The Bell system technical journal, 27 (3):3794423, 1948.
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. arXiv, pp. arXiv-1911, 2019.
Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: world perception for embodied agents. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9068-99899999 0109-9079,0000 2018
Brian Yamauchi. A frontier-basee approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA 197." "Towards New Computational Principles for Robotics and Automation", pp. 146-151. IEEE, 1997.
Shichao Yang and Sebastian Scherer. Cubeslam: Monocular 3-d object slam. IEEE Transactions on Robotics, 35(4))925-938, 2019.
Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, and Roozbeh Mottaghi. Visual semantic navigation using scene priors. ICLR, 2019.
Joel Yc. Dhruv Batra, Abhishek Das, and Erik Wijmans. Auxiliary tasks and exploration enable objectnav. arXiv preprint 14Xiv:2104.04112 2021.
Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic segmentation. ECCV, 2020.
Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal. Context encoding for semantic segmentation. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp. 7151-7160, 2018
Sixian Zhang, Xinhang Song, Yubing Bai, Weijie Li, Yakui Chu, and Shuqiang Jiang. Hierarchical object-to-zone graph for object navigation. ICCV, 2021.
Xiaoming Zhao, Harsh Agrawal, Dhruv Batra, and Alexander Schwing. The surprising effectiveness of visual odometry techniques for embodied pointgoal navigation. ICCV, 2021.
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 IEEE international conference robotics and automation (ICRA), pp. 3357-3364. IEEE, 2017.
13

---

Published as a conference paper at ICLR 2022
A APPENDIX
Here we provide the following additional material:
1. Implementation details.
2. Additional experimental results for semantic map prediction.
3. Per-object navigation results for the different active training strategies.
4. Per-object error analysis over stop decision and local policy.
5. Evaluation of our method on easy vs hard episodes.
6. Additional visualizations of semantic maps and navigation examples.
A.1 IMPLEMENTATION DETAILS
All (Roneehrgeerr et al. (2015) in our implemen- tation are combined with a backbone ResNet1 18 He at (2016) for providing initial encodings of the in- puts.
Algorithm 1: L2M for ObjectNav
Input:
Semantic target c;
Each UNet has four encoder and four decoder convolutional blocks with skip connections. The mod- els are trained the PyTorch Paszke et al. (2017) framework with Adam optimizer and learning rate of 0.0002. All experiments are conducted with an ensemble size tion we receive RGB and depth observations of size 256 256 and define crop and global map dimensions as h w = 64, H = W = 384 respectively. We use C 27 semantic classes that we selected from original 40 categories of MP3D [hang er al. (2017) We generate the ground-truth for semantic crops us- ing the 3D point clouds of the scenes which contain semantic labels. We executed training and testing on internal cluster RTX 2080 Ti GPUs. To train our final model, our image segmentation network first trained for 24 hours. Then, each model in our of- fline ensemble trained for 72 hours on separate GPUs. Finally, each model was fine-tuned on the actively collected data for 24 hours on separate GPUs.
Time = 0 with current position 24;
Stop decision probability threshold Spi
Stop decision distance threshold Sa; Replan interval R:
while t < max__teps__ppppppp wwtess_per_epiiiieeee pep_sdd do
4. For the semantic map predic-
Observe RGB I,, occupancy pe; Segment I and ground project to compute $,;
Hallucinate semantic map 144.. = Me(pt, &,); Estimate uncertainty:
if (Goal is reached) or
(1 mod R == 0) then
Compute Eq. 2 select goal I":
end
if tine > Sp at Lj on path I" then if ((),,, < Sy then
Stop decision;
end end
Regarding navigation, we use threshold of 0.75 on the prediction probabilities to determine the occur- rence of the target object in the map, and stop de- cision distance of 0.5m. Finally, we re-select goal every steps. Algorithm| [ shows additional details regarding the execution of our method during an object-goal navigation episode.
Navigate toward goal with DD-PPO;
1 t=t+1
end
A.2 SEMANTIC MAP PREDICTION
Here we provide additional results for our semantic map predictor, including evaluation over oc- cupancy predictions (unknown, occupied, free). The set-up for this experiment is the same in Section 4.T of the main paper. The purpose of this evaluation is to demonstrate the superiority of our method to possible non-prediction alternatives such as using directly the projected depth, or projecctd image segmentation. To this end we compare against the following baselines:
Depth Projection: Occupancy map estimated from a single depth observation.
Multi-view Depth Projection: Occupancy map accumulated over multiple views of depth observations.
Image Segmentation Projection: Our semantic segmentation model that operates on image observations, followed by projection of resulting labels.
14

---

Published as a conference paper at ICLR 2022
Occupancy Prediction
Method
Acc (%)
Ia0 (%)
Fl (%)
Depth Proj.
31.2
14.5
24.3
Multi-view Depth Proj.
48.5
31.0
47.0
1.2M-Offline
65.2
45.5
61.9
L2M-Active
68.1
48.8
Semantic Prediction
Image Segm. Proj.
13.9
5.9
10.2
Sem. Sensor Proj. Multi-view Sem. Sensor Proj.
16.9
9.1
16.0
27.6
18.9
31.4
L2M-Offline L-2--Active
31.2
20.1
30.5
33.5
25.6
38.3
Table 5: Comparison of our occupancy and semantic map predictions to non-prediction alternatives.
0.25
0.40
0.35
0.20
0.30
loU (%)
(%)
0.15
0.25
0.20
0.10
0.15
L2M-Offline
0.10
L2M-Active
0.05
L2M-BALD
0.05
L2M-Entropy
0.00
0.00
chaircounter bed cushion sofa
table
chaircounter bed cushion sofa table
Figure 4: Semantic mapping results per class over different active training methods.
Semantic Sensor Projection: Semantic map generated by single-view ground-truth seman- tic sensor images provided by the simulator.
Multi-view Semantic Sensor Projection: Same as the previous baseline, but with multiple views accumulated in the semantic map.
We present mean values for accuracy, intersection over union (IoU), and Fl score in Table Both our approaches outperform all baselines by significant margin. This suggests that our predictions of unobserved areas can provide more useful information to the agent than only relying on accumulated views from egomotion. In the case of semantic prediction our results are even more compelling as they are compared against the ground-truth semantic sensor of the Habitat simulator. Note that the Mulli-view Sensor baseline gets far from perfect score two reasons:
still contains unobserved areas, and 2) due to the pooling of the labels in the lower spatial dimensions of the top-down map (which affects all projections). contrast, our approach is unaffected by this problem as we learn to predict semantics from depth projected inputs.
A.3 PER OBJECT EVALUATION FOR ACTIVE TRAINING METHOOSS
In this section we show per object results over map prediction quality and chjeettggoal navigation for the different active training strategies (122MBBLLD L2M-Entropy, L.M--fflinee introduced in Section 4.1 the main paper against our proposed L2M-Active. We show results over a representative subset of six objects chair, bed, cushion, counter, sofa, table in Figure 4 (map prediction) and Figure cobject-goal navigation). In both cases we observe that the performance gap between L2M-Active and the baselines is larger more challenging target classes such cushion and counter. This suggests that our active training successfully selected examples with high information value as opposed to 12M-BAII and 122--Entoppy which do not show significant improvement over 1.2M-
15

---

Published as a conference paper at ICLR 2022
40
0.14
Success
30-
0.10
0.08
0.06
L2M Entropy UpperBound
ree-UperBBooni
0.04
L2M BALD UpperBound
0.02
chair
sofa
bed
cushioncounter
table
0.00
chair
50fa
bed
cushion table
Figure 5: Navigation results per class different active training methods.
Ours
gtPath
Curs+ccacleeToO
Surs-gtPPtt---rccessoo
0.5
Success
60
0.4
40
0.3
20
0
0.0
chair
sofa
bed
aushoonccunter table
chair
sofa
bed
cushion
table
Figure 6: Ablations of our method that investigate the effect of the stop decision and local policy,
A.4 PER OBJECT ERROR ANALYSIS
Here we present additional results the experiment in section 4.3 where we investigated the effect of the stop decision and local policy in failure cases of our model. To do so, we combined our 1.2M- Active method with an oracle that stops the agent within success distance of the target (OracleStop) and replaced the local policy with shortest paths estimated by the habitat simulator to reach selected goals (GtPath). Results are shown in Figure 6 The largest performance improvement (especially for SPL) is seen when the OracleStop is enabled, opposed to GtPath, suggesting that very common failure case is recognizing that we' ve reached the target. Perhaps unsurprisingly, this is also more pronounced in categories where our map predictor seems to be underperforming (see Figure 4) such as cushion, counter and table. This result also indicates that our method selects well positioned goals across all object categories, but often fail either reach the target due in the local policy or fail to recognize goal state.
A.5 EASY VS HARD EPISODES
Another way of evaluating the impact of our proposed method by analyzing its performance with respect to easy and hard episodes. We generated 1685 easy and 795 hard episodes, which combined make up our entire test set used Section of the main paper. The hard episodes are generated with larger geodesic to euclidean distance ratio between the starting position and the target (1.1 vs 1.05), which translates to more obstacles present in the path, and have larger mean geodesic distance (6.5m visualized in Figure 7 It is worth noting that the performance gap is higher
vs 4.5m).. The results in the case of hard. Our actively trained semantic mapping model seeks out difficult data during training, resulting in more consistently high performance both easy and hard episodes than the models trained only with offline data (.2M-Offfine or using different information gainn objectives (L.2M- 112---ntrppy, 12M-BAI.D).
16

---

Published as a conference paper at ICLR 2022
0.12
35-
30
Success
25
20
L2M-Offline-Mean
SPL
L2M-Offinne-owwrrB Bound
0.00
15
L2M-Offiine-Miser
L2M-Eni 22M-Entropy--o Entrrry-UppppeeBur d
0.04
10
L2M Offiin UpperBound
5-
L2M BALD- UpperBound
0.02
LZM-Acliv-UpperrBn
Easy
Hard
Easy
Hard
Figure 7: Our actively trained semantic mapping model shows consistently higher performance on hard episodes.
A.6 ADDITIONAL VISUALIZATIONS
Finally, we provide some additional visualizations. Example navigation episodes are shown in Figure A of semantic predictions are shown in Figure 9 while Figure [0 showeases predictions from the individual models in the ensemble, qualitatively demonstrating the variation within the ensemble over semantic predictions.
17

---

Published as a conference paper at ICLR 2022
Ejeceraff
Goocenbife
Egocentricc
time
floor
cushion
bed
sofa
bathtub
shower
wall
table
chair
counter
cabinet fireplace
door
Figure 8: Examples of successful navigation episodes where the targets are "chair" (top) and "bed" (bottom). For each example, first row shows egocentric RGB observations, second row are egocentric semantic predictions, third row the registered geocentric predictions, and the last row shows the model uncertainty of the target class over the geocentric predictions (brighter color signifies higher uncertainty). The agent shown blue dot, and the current goal as magenta.
18

---

Published as a conference paper at ICLR 2022
Proj
Sem. Pred.
GT
floor
custion
bed
sofa
bathtub
shower
other
wall
tablc
chair
counter
cabinct
fireplaze
door
Figure 9: Qualitative semantic prediction results using L2M-Active approach.
Ensemble Model Predictions
KBll
Table
charr
courter
cabinet
Figure 10: Qualitative semantic prediction from the individual models in the ensemble (first four columns) using our L2M-Active approach.
19


Reviews:
[{"review_id": "USFgnpe96U9", "reviewer": "Reviewer_Nmyk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n- The method is well motivated and all components perform well\n- The uncertainty component is used both for active learning and to select goals for ObjectNav\n- Ablations are done well and comprehensive\n- Code is provided\n\n### Weaknesses\n\n- Computational requirements. Uncertainty is estimated via an ensemble, thus increasing the computation by N(=4). Where other uncertainty estimate methods tried/considered?\n- Lack of test-std results. The Habitat Challenge also contains a public leaderboard with a held-out test set (https://eval.ai/web/challenges/challenge-page/802/leaderboard/2195). The authors should submit to this.\n\n### Suggestions for improvement\n\nWhile I don't expect the authors to compare with these works as they are concurrent (one wasn't even public before the ICLR deadline), I encourage the authors also include Ye et al. (ICCV 2021) and Maksymets et al. (ICCV 2021) in their table for completeness.\n\nMaksymets: https://openaccess.thecvf.com/content/ICCV2021/papers/Maksymets_THDA_Treasure_Hunt_Data_Augmentation_for_Semantic_Navigation_ICCV_2021_paper.pdf\nYe: https://arxiv.org/abs/2104.04112", "summary_of_the_paper": "This paper presents a method to perform ObjectGoal navigation (i.e. goto the table) based on active semantic mapping. Specifically the proposed approach contains a semantic mapping module that hallucinates unseen areas. This mapping module is initially trained on a set of trajectories that are shortest paths between two points. Then additional trajectories are selected via active learning to maximize the learning signal.\n\nObjectGoal navigation is performed by training class specific map predictors and selecting goal locations via the upper confidence bound.\n\nThis method outperforms prior map-based works (SemExp) on the Habitat Challenge ObjectGoal navigation dataset.", "summary_of_the_review": "This paper presents a strong contribution. My main concern is lack of test set results, however I consider this to be a significant issue as the test set is the standard for reporting on this task/dataset. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "IQgSOkDwu08", "reviewer": "Reviewer_qy8x", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper explains the high level ideas fairly clearly. However, the proposed method has a few components. It would be easier to follow the details if there are more structured algorithm boxes that go with components in section 3. For instance, it is not very clear what the training loss for the segmentation model is does it account for class imbalance etc. Some of the details are present in the appendix but need a bit of hopping around to figure out. \n\nThe motivation and high-level idea makes sense. Leveraging priors on typical semantic arrangement of objects for exploration seems natural. That said leveraging these priors as proposed in the paper requires very precise semantic segmentation maps which are fairly expensive to obtain. It seems like the proposed method in the paper can be applied to bounding boxes which are much cheaper to obtain as opposed to semantic segmentation. \n\nThe uncertainty estimation relies on an ensemble of models trained for the semantic map prediction. This can be fairly expensive especially with semantic segmentation models. There are other strategies to estimate uncertainty using a single model (https://arxiv.org/pdf/1506.02142.pdf). Have the authors considered these alternatives?\n\nThe experimental evaluation focuses on comparison with baselines from chaplot et.al where the problem setup and assumptions are a bit different. However, the overall pipeline seems fairly similar and leverages similar high-level ideas (https://arxiv.org/pdf/2006.09367.pdf). It is unclear what the main distinctions from this line work the current paper is contributing. Moreover, semantic segmentation is typically less robust that instance segmentation and tends to ignore smaller objects. In the experimental results is there a breakdown of navigation queries to semantic categories which are small and are only represented by a small fraction of the training data. \n\nA simple baseline that seems to be missing is just using the projected semantic segmentation map without trying to guess the semantic layout in unseen regions to perform the navigation tasks. Are the navigation queries such that they cannot be solved with just what is observed is that ensured in the experiments?", "summary_of_the_paper": "The paper proposes a method for leveraging semantic predictions in unobserved areas to help agent navigation. The strategy is motivated by how humans leverage priors on house layouts to perform navigation and tasks without being the house before. The key idea is to train a semantic segmentation model which can predict plausible class assignments in unseen regions. Then use the uncertainty estimates during navigation to pick goals when trying to reach a particular target category. ", "summary_of_the_review": "The high-level ideas the paper is based seem natural and interesting. However, fairly similar ideas have been proposed in prior work which is cited by the paper. It seems to me that the distinctions are mostly in the details a lot of which emerge from the slight differences in the problem setup. \n\nPost-discussion: The authors have addressed the main concerns about distinctions from prior work. Therefore, I am improving my rating. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CunddNGXe3N", "reviewer": "Reviewer_CF2f", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Firstly, thanks to the authors for their efforts. This is a very well written and high quality paper and I absolutely enjoyed reading it! In my opinion, the paper scores high on novelty, provides an interesting extension to the status quo on how mappers are generally used by embodied agents and nicely extends ideas from prior work in order to incorporate them into the task at hand. More concretely, I believe the following contributions are particularly the strengths of this work:\n+ An active learning formulation for training the mapper by selection of informative waypoints where the mapper is expected to gain maximal amount of information\n+ Using the variance of an ensemble of mappers as a proxy for taking into account the current mapper’s uncertainty for selecting goal targets in the map constructed so far: the formulation is elegant and nice leads to a controllable exploitation-exploration trade-off.\n\nThe insights presented in the paper are intuitive (learning from maximally informative regions is obviously a more efficient strategy than randomly sampling training trajectories for the mapper), the formulation is elegant (I liked how the same ideas related to variance of predictions applied to both mapper training and goal selection) and overall seem potentially useful for the Embodied AI community.\n\nThe paper also contains experiments pertaining to all the logical design choices for their approach and the empirical results largely support the claims made by the authors (refer to the question/suggestions section for some exceptions to this).\n\nThe only major complaint I have with the paper is the absence of a thorough comparison with state-of-the-art. The authors do compare with a couple of related contemporary approaches in Tab. 3. However, ObjectNav on MP3D/Habitat is a well established task with an active leaderboard (https://eval.ai/web/challenges/challenge-page/580/leaderboard/1634) and a comparison with the best performing models from the leaderboard seem to be missing.\n\nI also had minor issues with some missing bits of information pertaining to how the overall system is working. I encourage the authors to clarify these points in the discussion phase. For instance,\n+ There seem to be a couple of different, but related decision making processes going on: (a) the agent has to select informative waypoints to optimally train the semantic map prediction module and (b) the agent has to select the target semantic goal location on the map built so far while being mindful of the uncertainties of the current semantic predictions. Although both these decisions are made using the same fundamental principles at the core, it is not clear to me the order in which these decisions are made. Does the agent alternate between the two? If I understand correctly, both cannot be happening at the same time. Once either of the modules selects a waypoint, the agent has to move towards the same. Some clarification on how the overall system works would be helpful in this context.\n+ The authors mention that the optimizations (eqn. 1-4), in practice, happen over the geo-centric map. Does it happen only for the set of map locations observed so far? That is, are there any constraints on the l_j’s in the geo-centric map?\n+ It is not immediately clear what is causing the variance in predictions for the ensemble of mappers. Is it because of different initializations? Or, is the training for each being warm-started on a different subset of the dataset?\n\nGiven below are some minor questions/points of clarifications:\n+ From the description in Sec. 3.3.2, the theory seems to suggest that using the LowerBound strategy for goal selection is more conservative towards exploration, as compared to the UpperBound strategy. One would expect a higher SPL as a consequence of that (the map predictor training strategy staying the same across both). Why isn’t that the case in Tab. 3 (L2M-Active-LowerBound=9.6 v/s L2M-Active-UpperBound=13.3. SPL)? Also related to this, the “LowerBound\" numbers for “Offline\" map predictor training is missing (L2M-Offline-LowerBound)\n+ There are two rows for the same experiment in Tab.3 (L2M-Active-UpperBound). I suspect the reason is the special case for the SemExp baseline where the comparison happens only for a subset of 6 objects categories. Would be better to mention this explicitly in the text / table caption.\n+ This is purely speculative from the reviewer (doesn't factor into the review), but I was wondering if the same ideas would work in the space of features, as opposed to logits, as in current work (that are being projected onto the top-down map). Prior work (https://arxiv.org/abs/2010.01191) has shown that projection of features, rather than projected labels from egocentric segmentations circumvents the \"label splatter\" issue. I'm curious to know the authors' thoughts on this -- do they think that the same framework on variance of ensembles would work equally well in the semantic feature space?", "summary_of_the_paper": "The paper presents a novel framework that learns how to construct spatial + semantic maps for target-driven semantic (ObjectGoal) navigation. The scene maps being learned can also hallucinate beyond the immediate observed regions by exploiting semantic priors in the scene during the learning process. The key contribution of this work is in two different places of the navigation pipeline: (1) an active learning setting for training the mapper that optimizes selection of maximally informative regions of the environment’s state space (metric locations on the floor-plan) for the agent to navigate towards and (2) incorporating the uncertainty in the mapper’s predictions while selecting the target semantic object on the map learned so far. The novel insight that connects both these contributions is borrowed from literature on intrinsic rewards via disagreement — the variance between the predictions of an ensemble of multiple mappers serves as a proxy for the model’s uncertainty about a specific location / the informativeness of that location for training the mapper.", "summary_of_the_review": "As mentioned above, I feel that the paper has sufficient novel contributions to warrant an acceptance. I encourage the others to include a discussion on comparisons with SoTA from the leaderboard. Also, there are some minor issues that I hope get resolved as part of the discussions. But overall, keeping in mind the insights, novelty and usefulness of the ideas presented, my vote is to accept.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of goal navigation in unseen environments by learning to build a local, then a registered, global occupancy and semantic map of object categories from reprojected RGB+D observations, while extrapolating (hallucinating) unseen observations from contextual semantic priors (e.g., "tables are usually surrounded by chairs"). It then uses a measure of epistemic uncertainty on different estimations (realisations) of that map as a navigation goal selection policy to perform active exploration, and controls the agent using a local goal-driven policy; different information gain metrics are investigated. Essentially, the policy accumulates the predicted semantic maps and uses the uncertainty of the semantic mapping to select informative goals. The semantic map predictor is implemented as three U-nets for occupancy extrapolation from depth projection, semantic segmentation of RGB, and semantic map inference from the ground projection and extrapolated depth projection maps. The whole method is evaluated on the Matterport3D environment.

Reviewers praised the well-written and comprehensively-evaluated study, the active learning formulation and the idea of epistemic information gain as a measure of uncertainty for goal selection, and the code availability. Reviewers' major concerns included the computational cost of ensemble-based uncertainty estimation (Nmyk), and a missing submission to an active learderboard of the habitat-challenge (Nmyk, CF2f). Reviewers CF2f and qy8x had a longer list of issues that have been addressed in the rebuttal.

Reviewers engaged in a discussion with the authors, and the scores are 5, 6 (though not updated) and 8. I believe that the paper just meets the conference acceptance bar and would advocate for its inclusion in the conference.
Decision: Accept (Poster)