Visual Semantic Navigation using Scene Priors

1 INTRODUCTION . Consider the kitchen scene shown in Figure 1 ( a ) and the task of finding an object such as a mug . Even though we have never seen this particular kitchen before and no mug is visible in the scene , we can still infer the likely locations to find the mug and create an exploration plan accordingly . For example , in Figure 1 ( a ) , we can infer that since there is a coffee machine , the mug is most likely in the cabinet near the coffee machine . How do we do that ? We infer that mugs are usually used for coffee . And since there is a coffee machine , the mug is likely to be near the machine due to functional efficiency . We argue that humans use strong priors about the functional and semantic structure of the world to develop such efficient navigation strategies . And how do we learn such functional/semantic priors ? Our prior experience and exploration of tens of kitchens help us to learn these priors . But these priors are not just used for navigating to known objects but also to completely unknown and unseen objects . For example , let us assume you have never seen a mango before and someone gives you a task of finding a mango in a new kitchen you have never seen before ( let ’ s say Figure 1 ( b ) ) . How would you do it ? Assuming you have searched for fruits like apples and grapes before , and you know mango is also a fruit ; so a similar exploration strategy might apply . Therefore , in Figure 1 ( b ) , you are more likely to navigate to the fridge to search for a mango . Therefore , we use the semantic/functional priors to navigate to unseen objects as well . Inspired by these observations , in this paper , we explore how to exploit semantic priors for the task of semantic and goal-oriented navigation . In our visual navigation task , the environment , the path to the target , the target location , or the exact appearance of the target object can be unknown . The prior knowledge about the semantic/functional structure of the world helps to improve the navigation . We propose to use Graph Convolutional Networks ( GCNs ) ( Kipf & Welling , 2017 ) to incorporate the prior knowledge into a Deep Reinforcement Learning framework . The knowledge of the agent is encoded in a graph . GCNs allow arbitrary structured graphs to be encoded in an efficient way . The knowledge is updated according to the current observation of the agent , which is specific to the current environment , and the knowledge at the previous time step or the prior knowledge . The prior knowledge is obtained from large-scale datasets designed for scene understanding . Our model is based on the actor-critic model ( Mnih et al. , 2016 ) that is augmented by the knowledge graph and object visibility information . To evaluate our model , we use the AI2-THOR framework ( Kolve et al. , 2017 ) , which provides near photo-realistic customizable environments . The agent can take navigation actions in these environments and observe the changes as a result of those actions . AI2-THOR includes various objects that can be arranged in many different configurations . The agent location is randomized as well at each episode of training or testing . Our experiments show that the semantic prior improves the performance of the baseline RL models significantly . Furthermore , we show the results of the model on the challenging setting where the scene and/or the object are new to the agent . Our contributions are summarized as follows : ( 1 ) We integrate a deep reinforcement learning model with knowledge graphs . This allows the agent to encode any form of knowledge that can be represented by graph structures . ( 2 ) We show that semantic prior knowledge can significantly improve the navigation performance . ( 3 ) By considering the prior information and the semantics of the target objects , we improve generalization to unseen environments and novel target objects . 2 RELATED WORK . Semantic and goal-oriented navigation is one of the most prominent tasks that intelligent species perform in their daily life . There are several challenges involved in visual navigation . First , the environment might be unknown to the agent . In this situation , the agent requires to explore the environment to have a better understanding of that environment . The second challenge is about the visibility of target objects . The target object might not be visible when the agent starts the navigation or it might go out of the field of view during navigation . Hence , the agent needs to learn an efficient search strategy to find the target object . The third challenge is related to planning . The object might be visible but planning a reasonable path towards the object is another issue that the agent needs to deal with . There have been several efforts in the past to tackle these challenges which we describe below . Geometry-based navigation . Navigation methods can be divided into two main categories of geometry-based and learning-based . Most of the traditional navigation approaches fall into the former category , where it is assumed that either the map of the environment is known a priori , e.g. , Matthies & Shafer ( 1987 ) ; Borenstein & Koren ( 1991 ) ; Meng & Kak ( 1993 ) ; Kim & Nevatia ( 1999 ) or a map is built on the fly e.g. , Thrun ( 1998 ) ; Feder et al . ( 1999 ) ; Jones & Soatto ( 2011 ) ; Siagian et al . ( 2014 ) . Our work is different from these approaches since we do not rely on a map for our navigation and we leverage semantic prior knowledge to reduce the required exploration time . Learning-based navigation . Recent success of deep learning and reinforcement learning has made learning-based navigation approaches more popular . Zhu et al . ( 2017 ) propose a deep RL-based navigation approach , where they provide the picture of the target object . In contrast , we only provide semantic labels to the agent , so we can show generalization to unseen scenes . Gupta et al . ( 2017 ) propose a mapper and planner to output navigation actions . Mirowski et al . ( 2017 ) also propose a navigation framework that optimizes a loss for auxiliary tasks such as depth prediction and loop closure classification . Sadeghi & Levine ( 2017 ) propose an RL-based approach for collision avoidance . Brahmbhatt & Hays ( 2017 ) explore a CNN-based approach for navigating in cities using local observations of the streets . Wu & Tian ( 2017 ) combine deep RL with curriculum learning in a first-person shooting game setting . Savinov et al . ( 2018 ) introduce a topological landmark-based memory for navigation . Kahn et al . ( 2018 ) propose a method based on model-free and model-based RL to learn navigation policies using a few samples . Mousavian et al . ( 2018 ) use object detection and semantic segmentation to better navigate in unseen environments . Chen et al . ( 2015 ) directly map the input image to an action in an autonomous driving setting . There is also a large body of work that address visually grounded navigation instructions e.g. , Anderson et al . ( 2018b ) ; Chaplot et al . ( 2018 ) ; Hermann et al . ( 2017 ) ; Yu et al . ( 2018 ) ; Misra et al . ( 2017 ) ; Mei et al . ( 2016 ) . In contrast to all these approaches , we incorporate semantic and functional priors to improve navigation performance and better generalize to unseen scenes and objects . Context and scene prior . Contextual reasoning has been studied extensively in the computer vision literature ( Torralba et al. , 2003 ; Hoiem et al. , 2005 ; Rabinovich et al. , 2005 ; Divvala et al. , 2009 ; Desai et al. , 2009 ; Marszalek et al. , 2009 ; Malisiewicz & Efros , 2009 ; Mottaghi et al. , 2014 ; Zhu et al. , 2015 ; Shrivastava & Gupta , 2016 ) . However , contextual information is mainly used for static settings such as object detection , semantic segmentation or action recognition . We use contextual reasoning for an interactive navigation task , where the agent updates its belief based on the current observation and the prior knowledge as it moves in the environment . Object relationships have been used for tasks such as image retrieval ( Johnson et al. , 2015 ) , visual relation detection ( Zhang et al. , 2017 ) , referring expressions ( Nagaraja et al. , 2016 ; Hu et al. , 2017 ) , and visual question answering ( Johnson et al. , 2017 ) . Knowledge graphs . There are recent works that use knowledge graphs for computer vision problems . A knowledge graph is used by Marino et al . ( 2017 ) for image classification , by Li et al . ( 2017 ) for situation recognition and by Wang et al . ( 2018 ) for zero-shot recognition . We use knowledge graphs in an RL setting for the interactive task of visual navigation . Reasoning about unknown environments or objects . Various works have explored zero-shot reasoning in the context of reinforcement learning . Yu et al . ( 2018 ) address the problem of learning language in a 2D maze , where they can handle unseen word combinations or new sentences that contain unseen words . Harrison et al . ( 2017 ) ; Higgins et al . ( 2017 ) study zero-shot policy transfer in the scenarios that the dynamics or the states of the target domain is different from those of the source domain . Pathak et al . ( 2018 ) propose a zero-shot imitation learning approach where the expert demonstration for a particular task is never seen . Oh et al . ( 2017 ) address generalization of RL to unseen instructions and longer instructions . Our problem is different since we address navigation to novel objects or navigating in unseen scenes using scene priors . 3 VISUAL SEMANTIC NAVIGATION . In this section , we first define the task of visual semantic navigation . We then describe the formulation using deep reinforcement learning and the baseline model for the task . 3.1 TASK DEFINITION . Our goal is to navigate from a random starting location in a scene to a specified target object category given only egocentric RGB perception of the agent . The target object category is specified by a semantic label , thus we call our task visual semantic navigation . The task is considered successful if an instance of the target object category is visible . By “ visible ” , we mean the target object is in the field of view and within a threshold of distance . 3.2 THE BASELINE MODEL . We formulate the visual semantic navigation using a deep reinforcement learning framework . Given a semantic task objective g , the agent perceives a state st ( i.e. , the egocentric RGB image from the current location and orientation ) at the time step t and samples an action at from the set of possible actions A according to its policy π . We approximate the policy by a deep policy network π ( · ; θ ) : at ∼ π ( φ ( st ; u ) , ψ ( g ; v ) ; θ ) , ( 1 ) where u , v , and θ are the parameters for the network . Since the visual state and the semantic objective are from different modalities , we design two branches of subnetworks φ ( · ; u ) and ψ ( · ; v ) to map these two inputs into a joint visual-semantic feature embedding . Visual network . As illustrated in Figure 2 ( top ) , the visual network takes 224×224 RGB images as input and generates a 512-d feature vector as output . The backbone of the visual branch is ResNet50 ( He et al. , 2016 ) pre-trained on ImageNet . Specifically , we extract the 2048-d feature after the global average pooling of ResNet-50 . To account for the history of the actions taken by the agent , we concatenate the features of the current frame and three past observations , which results in a 8192-d feature vector . We then add a fully connected layer and a ReLU layer to map the concatenated image feature into the 512-d visual-semantic feature . Semantic network . The semantic task objective is described by an object category , e.g. , Microwave or Television . We use fastText ( Joulin et al. , 2016 ) to compute a 100-d embedding for each word . Then we map the word embedding into a 512-d feature by a fully connected layer and ReLU , as illustrated in Figure 2 ( middle ) . Actor-Critic policy network . We employ the Asynchronous Advantage Actor-Critic ( A3C ) ( Mnih et al. , 2016 ) model to predict the policy at each time step . The input of our A3C model is the joint representation of the current state and the semantic task objective , which is a 1024-d feature vector made by concatenating the outputs of the visual network and the semantic network . The A3C model generates two outputs , i.e. , the policy and the value . We sample the action from the predicted policy . Our implementation of the A3C model consists of three layers : the input , the hidden layer , and the outputs . The hidden layer is a fully connected layer followed by the ReLU activation layer which maps the fused input into a 512-d latent space . Then the |A| dimensional policy and the value are generated by two branches of network , as shown in Figure 2 . Unlike previous work ( e.g. , Zhu et al . ( 2017 ) ) which uses different policy networks for different scenes , we use a single policy network for different scene examples . This makes our model more compact and generalizable . Reward . We consider a reward to minimize the trajectory length to the targets : If any object instance from the target object category is reached within a certain number of steps , the agent receives a large positive reward 10.0 . Otherwise , we penalize each step with a small negative reward -0.01 . The design of the reward function is also affected by the types of actions A . In our experiments , we ablate two sets of actions A with or without the stop action . In the setting without the stop action , the agent will receive the positive reward if the environment notifies it when it reaches the target , which also ends an episode of training . In the setting with the stop action , the episode is terminated when the stop action is executed , and the positive reward will be provided only if the agent is within the threshold of distance from the target ( 1 meter in our experiments ) and facing the target . This makes the task much more challenging .

Summary: This paper tackles the problem of navigating scenes to find objects which are potentially not included in the training phase. To find an unseen object from a scene, the proposed model incorporates an external knowledge graph as an augmented input of the actor-critic model. To construct a knowledge graph, entities in a scene are identified by ResNet and then the link structure between entities are extracted from VIsual Genome dataset. Through the ablation study, it is shown that using the knowledge graph helps to track and identify unseen objects during training.