Embodied Multimodal Multitask Learning

1 INTRODUCTION . Humans learn language by interacting with a dynamic perceptual environment , grounding words into visual entities and motor actions ( Smith and Gasser , 2005 ; Barsalou , 2008 ) . In recent years , there has been an increased focus on training embodied agents capable of visually-grounded language learning . These include multimodal tasks involving one-way communication , such as mapping navigational instructions to actions ( MacMahon et al. , 2006 ; Chen and Mooney , 2011 ; Artzi and Zettlemoyer , 2013 ; Mei et al. , 2016 ; Misra et al. , 2018 ) ; and tasks involving two-way communication such as embodied question answering ( Gordon et al. , 2018 ; Das et al. , 2018 ) and embodied dialogue ( de Vries et al. , 2018 ) . Other studies have shown that grounded semantic goal navigation agents can be effective at exploiting the compositionality of language to generalize to unseen instructions with an unseen composition of semantic attributes ( Hermann et al. , 2017 ; Chaplot et al. , 2018 ) , or an unseen composition of steps in a multi-step instruction ( Oh et al. , 2017 ) . However , current grounded language learning models have certain limitations . Firstly , these models are typically trained only for a single multimodal task and lack the ability to transfer grounded knowledge of ‘ concepts ’ ∗ across tasks . For example , if an agent learns to follow the instruction ‘ Go to the red torch ’ and answer the question ‘ What color is the pillar ? ’ , then ideally it should also understand ‘ Go to the red pillar ’ and ‘ What color is the torch ? ’ without additional training . Training multitask grounded-language models can also improve training sample efficiency , as these multimodal tasks share many common learning challenges including perception , grounding , and navigation . The second limitation is the inability of trained models to quickly transfer to tasks involving unseen concepts . For example , consider a household instruction-following robot trained on an existing set of objects . We would like the robot to follow instructions involving a new object ‘ lamp ’ that has been added to the house . Existing models would need to be trained with the new object , which typically requires thousands of samples and can also lead to catastrophic forgetting of known objects . Even if the models were given some labeled samples to detect the new objects , they would require additional training to learn to combine existing grounded knowledge with the new concept ( e.g. , ‘ blue lamp ’ if ‘ blue ’ is already known ) . In this paper , we train a multimodal multitask learning model for two tasks : Semantic Goal Navigation , where the agent is given a language instruction to navigate to a goal location , and Embodied Question Answering , where the agent is asked a question and it can navigate in the environment to gather information to answer the question ( see Figure 5 ) . We make the following contributions in this paper : †See https : //sites.google.com/view/emml-iclr2020 for demo videos . ∗In this paper , we refer to the knowledge of a word and its grounding in the visual world as the knowledge of a concept ( for example , concept ‘ torch ’ involves word ‘ torch ’ and how torch looks visually ) . First , we define a cross-task knowledge transfer evaluation criterion to test the ability of multimodal multi-task models to transfer knowledge of concepts across tasks . We show that several prior single-task models , when trained on both tasks , fail to achieve cross-task knowledge transfer . This is because the visual grounding of words is often implicitly learned as a by-product of end-to-end training of the underlying task , which leads to the entanglement of knowledge of concepts in the learnt representations . We propose a novel Dual-Attention model which learns task-invariant disentangled visual and textual representations and explicitly aligns them with each other . We create datasets and simulation scenarios for testing crosstask knowledge transfer and show an absolute improvement of 43-61 % on instructions and 5-26 % for questions over baselines . Second , the disentanglement and explicit alignment of representations makes our model modular and interpretable . We show that this allows us to transfer the model to handle instructions involving unseen concepts by incorporating the output of object detectors . We also show that our model is able to combine the knowledge of existing concepts with a new concept without any additional policy training . Finally , we show that the modularity and interpretability of our model also allow us to use trainable neural modules ( Andreas et al. , 2016 ) to handle relational tasks involving negation and spatial relationships and also tackle relational instructions involving new concepts . 2 RELATED WORK . A lot of early work on visual instruction-following in the embodied space such as in robotics applications ( Tellex et al. , 2011 ; Matuszek et al. , 2012 ; Hemachandra et al. , 2015 ; Misra et al. , 2016 ) and on mapping natural language instructions to actions ( MacMahon et al. , 2006 ; Chen and Mooney , 2011 ; Artzi and Zettlemoyer , 2013 ; Mei et al. , 2016 ) required hand-designed symbolic representations . Recently , there have been efforts on learning to follow navigational instructions from raw visual observations ( Anderson et al. , 2018 ; Misra et al. , 2018 ; Chen et al. , 2019 ; Blukis et al. , 2018 ) . Oh et al . ( 2017 ) ; Chaplot et al . ( 2018 ) ; Hermann et al . ( 2017 ) study the language learning aspect of instruction-following in a more controlled setting , and show that grounded language learning agents are able to learn spatial and logical reasoning and exploit the compositionality of language to generalize to new instructions . Question Answering in the embodied space has been comparatively less-studied with recent work studying QA which requires exploration , navigation , and interaction with objects in the environment ( Gordon et al. , 2018 ; Das et al. , 2018 ) . In contrast to the prior work which tackles a single grounding task , we tackle both instruction-following and question answering in the embodied space and study the ability to transfer the knowledge of concepts across the tasks and tackle instructions with new concepts . In addition to the above , there is a large body of work on multimodal learning in static settings which do not involve navigation or reinforcement learning . Some relevant works which use attention mechanisms similar to the ones used in our proposed model include Perez et al . ( 2018 ) ; Fukui et al . ( 2016 ) ; Xu and Saenko ( 2016 ) ; Hudson and Manning ( 2018 ) ; Gupta et al . ( 2017 ) for Visual Question Answering and Zhao et al . ( 2018 ) for grounding audio to vision . 3 PROBLEM FORMULATION . Consider an autonomous agent interacting with an episodic environment as shown in Figure 1 . At the beginning of each episode , the agent receives a textual input T specifying a task . T could be an instruction to navigate to a target object or a question querying some visual detail of objects in the environment . At each time step t , the agent observes a state st = ( It , T ) where It is the first-person ( egocentric ) view of the environment , and takes an action at , which could be a navigational action or an answer action . The agent ’ s objective is to learn a policy π ( at|st ) which leads to successful completion of the task specified by T . Environments . We adapt the ViZDoom-based ( Kempka et al. , 2016 ) language grounding environment proposed by Chaplot et al . ( 2018 ) for embodied multitask learning . It consists of a single room with 5 objects . The objects are randomized in each episode based on the textual input . We use two difficulty settings for the Doom domain as shown in Figure 2 : Easy : The agent is spawned at a fixed location . The candidate objects are spawned at five fixed locations along a single horizontal line in the field of view of the agent . Hard : The candidate objects and the agent are spawned at random locations and the objects may or may not be in the agent ’ s field of view in the initial configuration . The agent must explore the map to view all objects . The agent can take 4 actions : 3 navigational actions ( forward , left , right ) and 1 answer action . When the agent takes the answer action , the answer with the maximum probability in the output answer distribution is used . Datasets . We use the set of 70 instructions from Chaplot et al . ( 2018 ) and create a dataset of 29 questions using the same set of objects and attributes . These datasets include instructions and questions about object types , colors , relative sizes ( tall/short ) and superlative sizes ( smallest/largest ) . We create train-test splits for both instructions and questions datasets to explicitly test a multitask model ’ s ability to transfer the knowledge of concepts across different tasks . Each instruction in the test set contains a word that is never seen in any instruction in the training set but is seen in some questions in the training set . Similarly , each question in the test set contains a word never seen in any training set question . Table 1 illustrates the train- test split of instructions and questions used in our experiments† . Note that for the EQA trainset , unseen words can be present in the answer . 4 PROPOSED METHOD . In this section , we describe our proposed architecture ( illustrated in Figure 3 ) . At the start of each episode , the agent receives a textual input T ( an instruction or a question ) specifying the task . At each time step t , the agent observes an egocentric image It which is passed through a convolutional neural network ( LeCun et al. , 1995 ) with ReLU activations ( Glorot et al. , 2011 ) to produce the image representation xI = f ( It ; θconv ) ∈ RV×H×W , where θconv denotes the parameters of the convolutional network , V is the number of feature maps in the convolutional network output which is by design set equal to the vocabulary size ( of the union of the instructions and questions training sets ) , and H and W are the height and width of each feature map . We use two representations for the textual input T : ( 1 ) the bag-of-words representation denoted by xBoW ∈ 0 , 1V and ( 2 ) a sentence representation xsent = f ( T ; θsent ) ∈ RV , which is computed by passing the words in T through a Gated Recurrent Unit ( GRU ) ( Cho et al. , 2014 ) network followed by a linear layer . Here , θsent denotes the parameters of the GRU network and the linear layer with ReLU activations . Next , the †More details about the datasets and the environments are deferred to the supplementary material . Dual-Attention unit fDA combines the image representation with the text representations to get the complete state representation xS and answer prediction xAns : xS , xAns = fDA ( xI , xBoW , xsent ) ( 1 ) Finally , xS and xAns , along with a time step embedding and a task indicator variable ( for whether the task is SGN or EQA ) , are passed to the policy module to produce an action .

Summary: The paper explores multi-task learning in embodied environments and proposes a Dual-Attention Model that disentangles the knowledge of words and visual attributes in the intermediate representations. It addresses two tasks, namely Semantic Goal Navigation (SGN) and Embodied Question Answering (EQA), using a simple synthetic environment. The paper compares against a few simple baselines and baselines adapted from models in each task.