Learning to Map for Active Semantic Goal Navigation

1 INTRODUCTION . What enables biological systems to successfully navigate to semantic targets in novel environments ? Consider the example of a dog whose food tray at its own house is situated next to the fridge . Upon entering a new house for the first time , the dog will look for its food tray next to the fridge , even though the new house can largely differ in appearance and layout . This is remarkable , as it suggests that the dog is able to encode spatial associations between semantic entities that can be leveraged when trying to accomplish a navigation task . Humans exhibit the same skills in similar scenarios , albeit more nuanced , since given existing observations we can consciously choose to trust our prior knowledge over the semantic structure of the world , or to continue exploring the environment . In other words , if we have a partial view of a room containing an oven , we can infer that a fridge most likely exists in the unobserved space . In addition , if we are trying to reach the sofa , then we can infer with high certainty that it will be located in a different room . This implies that we have internal mechanisms for quantifying the uncertainty of inferred information from unobserved spaces , which guides our decision making process . Inspired by these observations , in this work , we study the problem of object goal navigation for robotic agents in unseen environments and propose an active learning method for encoding semantic priors in indoor scenes . Our approach involves learning a mapping model than can predict ( hallucinate ) semantics in unobserved regions of the map containing both objects ( e.g . chairs , beds ) and structures ( e.g . floor , wall ) , and during testing uses the uncertainty over these predictions to plan a path towards the target . Contrary to traditional approaches for mapping and navigation Cadena et al . ( 2016 ) ( i.e . SLAM ) where the focus is on building accurate 3D metric maps , our uncertainty formulation is designed to capture our lack of confidence about whether a certain object exists at a particular location . This results in a much more meaningful representation , suitable for target-driven tasks . Recently , learned approaches to navigation have been gaining popularity , where initial efforts in addressing target-driven navigation focused on end-to-end reactive approaches that learn to map pixels directly to actions Zhu et al . ( 2017 ) ; Mousavian et al . ( 2019 ) . These methods do not have an explicit representation of the environment and tend to suffer from poor generalization . To remedy this issue , most current methods learn a map representation that enables the encoding of prior information about the geometry and semantics of a scene , acting as an episodic memory Chaplot et al . ( 2020b ; a ) ; Gupta et al . ( 2017 ) ; Georgakis et al . ( 2019 ) . However , maps created by these methods are restricted to contain information only from areas that the agent has directly observed , which led to the introduction of spatial prediction models that either anticipate occupancy Ramakrishnan et al . ( 2020 ) or room layouts Narasimhan et al . ( 2020 ) beyond the agent ’ s field of view and demonstrated improved performance on navigation tasks . Our work differs from these methods in three principled ways : 1 ) We formulate an active training strategy for learning the semantic maps , 2 ) we exploit the uncertainty over the predictions in the planning process , and 3 ) in contrast to predicting occupancy , our model tackles a harder problem which requires learning semantic patterns ( e.g . tables are usually surrounded by chairs ) . In this work we introduce Learning to Map ( L2M ) , a novel framework for object-goal navigation consisting of two parts . First , we actively learn an ensemble of two-stage segmentation models by choosing training samples through an information gain objective . The models operate on top-down maps and predict both occupancy and semantic regions . Second , we estimate the model uncertainty through the disagreement formulation of ensemble models from Pathak et al . ( 2019 ) , and show its effectiveness in defining objectives in planners to actively select long-term goals for semantic navigation . In addition , we investigate different information gain objectives during active training and illustrate how the use of model uncertainty can balance exploration with exploitation in finding semantic targets . Our proposed approach demonstrates improved success rates on the object-goal navigation task over competitive baselines in the Matterport3D Chang et al . ( 2017 ) dataset using the Habitat Savva et al . ( 2019 ) simulator . 2 RELATED WORK . Semantic SLAM . Classical approaches for navigation focus on building 3D representations of the environment before considering downstream tasks Cadena et al . ( 2016 ) . While these methods are typically geometric , several SLAM methods have attempted to associate semantic information to the reconstructed geometric map , mainly at the object-level Salas-Moreno et al . ( 2013 ) ; Yang & Scherer ( 2019 ) ; McCormac et al . ( 2018 ) ; Bowman et al . ( 2017 ) ; Kostavelis & Gasteratos ( 2015 ) . For example , in McCormac et al . ( 2018 ) instance segmentations predicted by Mask R-CNN are incorporated to facilitate per-object reconstructions , while the work of Bowman et al . ( 2017 ) proposes a probabilistic formulation to address uncertain object data association . However , SLAM systems rarely consider active exploration as they are not naturally compatible with task-driven learnable representations from deep learning architectures that can encode semantic information . Other recent works Katsumata et al . ( 2020 ) ; Cartillier et al . ( 2020 ) have sought to build 2D semantic maps and focused either on semantic transfer of a global scene or assumed the environments were accessible before-hand . In contrast , our proposed approach tackles the object goal task in unknown environments by actively learning how to predict semantics in both observed and unobserved areas of the map around the agent . Learning based navigation methods . There has been a recent surge of learning based methods Zhu et al . ( 2017 ) ; Mousavian et al . ( 2019 ) ; Gupta et al . ( 2017 ) ; Chen et al . ( 2019 ) ; Chaplot et al . ( 2020b ) ; Fang et al . ( 2019 ) ; Yang et al . ( 2018 ) ; Ye et al . ( 2021 ) ; Zhang et al . ( 2021 ) ; Chattopadhyay et al . ( 2021 ) for indoor navigation tasks Anderson et al . ( 2018 ) ; Batra et al . ( 2020 ) ; Das et al . ( 2018 ) , propelled by the introduction of high quality simulators such as Gibson Xia et al . ( 2018 ) , Habitat Savva et al . ( 2019 ) , and AI2-THOR Kolve et al . ( 2017 ) . Methods which use explicit task-dependent map representations Parisotto & Salakhutdinov ( 2017 ) ; Gupta et al . ( 2017 ) ; Chaplot et al . ( 2020b ; a ) ; Georgakis et al . ( 2019 ) ; Gordon et al . ( 2018 ) ; Mishkin et al . ( 2019 ) have shown to generalize better in unknown environments than end-to-end approaches with implicit world representations . For example , in Gupta et al . ( 2017 ) a differentiable mapper learns to predict top-down egocentric views of the scene from RGB images , followed by a differentiable planner , while in Chaplot et al . ( 2020a ) Mask R-CNN is used to build a top-down semantic map of the scene used by a learned policy that predicts goal locations in the map . More conceptually similar to our method , are approaches that attempt to encode semantic or layout priors by learning to predict outside the field-of-view of the agent Ramakrishnan et al . ( 2020 ) ; Liang et al . ( 2020 ) ; Narasimhan et al . ( 2020 ) . In contrast to all these works we formulate an active , target-independent strategy to predict semantic maps and define goal selection objectives . Uncertainty Estimation . Many recent works estimate uncertainty of deep learning models Gal ( 2016 ) ; Abdar et al . ( 2021 ) . We leverage the approach first proposed by Seung et al . ( 1992 ) to estimate our model ( epistemic ) uncertainty with the variance between the output of an ensemble of models . This was first used for active exploration in vision-based reinforcement learning by Pathak et al . ( 2019 ) . Maximizing epistemic uncertainty is used as a proxy for maximizing information gain Seung et al . ( 1992 ) ; Pathak et al . ( 2019 ) . We use this uncertainty objective to actively fine-tune our models with a training procedure similar to the active learning approaches leveraged during training presented by Bucher et al . ( 2020 ) ; Chaplot et al . ( 2020c ) ; Sener & Savarese ( 2017 ) . In particular , Chaplot et al . ( 2020c ) uses this active training method to improve performance on a semantic segmentation model , leading us to initially hypothesize analogous results would be possible for our semantic hallucination task . We also use this epistemic uncertainty estimate to construct confidence bounds for our estimated probability distribution which we use to select goals for target-driven navigation at test time . Both lower Galichet et al . ( 2013 ) and upper Auer et al . ( 2002 ) confidence bound strategies for balancing exploration , exploitation , and safety have been previously proposed in the multi-armed bandit literature and extended for use in MDPs Azar et al . ( 2017 ) and reinforcement learning Chen et al . ( 2017 ) . 3 APPROACH . We present a new framework for object-goal navigation that uses a learned semantic map predictor to select informative goals . In contrast to prior work Ramakrishnan et al . ( 2020 ) ; Chaplot et al . ( 2020a ) , we leverage the predictions outside the field-of-view of the agent to formulate uncertainty-based goal selection policies . Furthermore , we actively collect data for training the map predictor and investigate different information gain objectives . Due to our goal selection policy formulation , our method does not need to be trained specifically to predict goals for every target object , enabling a target-independent learning of the semantic priors . Our method takes as input an RGB-D observation , and predicts semantics in the unobserved areas of the map . This is followed by goal selection based on the estimated uncertainty of the predictions . Finally a local policy is responsible for reaching the goal . An overview of our pipeline can be seen in Figure 1 . 3.1 SEMANTIC MAP PREDICTION . We describe a method for learning how to map by predicting the semantic information outside the field of view of the agent . We emphasize that this goes beyond traditional mapping ( i.e . accumulating multiple views in an agent ’ s path ) as it relies on prior information encoded as spatial associations between semantic entities in order to hallucinate the missing information . Motivated by the past success of semantic segmentation models in learning contextual information Zhang et al . ( 2018 ) ; Yuan et al . ( 2019 ) , we formulate the semantic map prediction as a two-stage segmentation problem . Our method takes as input an incomplete occupancy region pt ∈ R|C o|×h×w and a ground-projected semantic segmentation ŝt ∈ R|C s|×h×w at time-step t. The output is a top-down semantic local region m̂t ∈ R|C s|×h×w , where Co is the set of occupancy classes containing unknown , occupied , and free , Cs is the set of semantic object classes , and h , w are the dimensions of the local crop . To obtain pt we use the provided camera intrinsics and depth observation at time t to first get a point cloud which is then discretized and ground-projected similar to Ramakrishnan et al . ( 2020 ) . To estimate ŝt we first train a UNet Ronneberger et al . ( 2015 ) model to predict the semantic segmentation of the RGB observation at time t. All local regions are egocentric , i.e. , the agent is in the middle of the crop looking upwards . Each spatial location in our map ( cell ) has dimensions 10cm× 10cm . The proposed two-stage segmentation model predicts the hallucinated semantic region in two stages . First , we estimate the missing values for the occupancy crop in the unobserved areas by learning to hallucinate unseen spatial configurations based on what is already observed . Second , given predicted occupancy , we predict the final semantic region m̂t . These steps are realized as two UNet encoder-decoder models , fo that predicts in occupancy space p̂t = fo ( pt ; θo ) , and fs that predicts in semantic space m̂t = fs ( p̂t ⊕ ŝt ; θs ) , where p̂t is the predicted local occupancy crop which includes unobserved regions , ⊕ refers to the concatenation operation , and θo , θs are the randomly initialized weights of the occupancy and semantic networks respectively . The image segmentation model is trained independently and its ground projected output ŝt conditions fs on the egocentric single-view observation of the agent . The model is trained end-to-end using pixel-wise cross-entropy losses for both occupancy and semantic classes and predicts a probability distribution over the objects for each map location . We assume that ground-truth semantic information is available such that we can generate egocentric top-down occupancy and semantic examples . This combined objective incentivizes learning to predict plausible semantic regions by having the semantic loss backpropagating gradients affecting both fo and fs . Also , performing this procedure enables the initial hallucination of unknown areas over a small set of classes Co , before expanding to the more difficult task of predicting semantic object categories Cs . An overview of the semantic map predictor can be seen in Figure 2 . During a navigation episode , the local semantic region is registered to a global map which is used during planning . Since we predict a probability distribution at each location over the set of objects , the local regions are registered using Bayes Theorem . The global map is initialized with a uniform prior probability distribution across all classes .

Summary: The paper presents a novel framework that learns how to construct spatial + semantic maps for target-driven semantic (ObjectGoal) navigation. The scene maps being learned can also hallucinate beyond the immediate observed regions by exploiting semantic priors in the scene during the learning process. The key contribution of this work is in two different places of the navigation pipeline: (1) an active learning setting for training the mapper that optimizes selection of maximally informative regions of the environment’s state space (metric locations on the floor-plan) for the agent to navigate towards and (2) incorporating the uncertainty in the mapper’s predictions while selecting the target semantic object on the map learned so far. The novel insight that connects both these contributions is borrowed from literature on intrinsic rewards via disagreement — the variance between the predictions of an ensemble of multiple mappers serves as a proxy for the model’s uncertainty about a specific location / the informativeness of that location for training the mapper.