Sub-Instruction Aware Vision-and-Language Navigation (2020)

Abstract: Vision-and-language navigation requires an agent to navigate through a real 3D environment following natural language instructions. Despite significant advances, few previous works are able to fully utilize the strong correspondence between the visual and textual sequences. Meanwhile, due to the lack of intermediate supervision, the agent’s performance at following each part of the instruction cannot be assessed during navigation. In this work, we focus on the granularity of the visual and language sequences as well as the traceability of agents through the completion of an instruction. We provide agents with fine-grained annotations during training and find that they are able to follow the instruction better and have a higher chance of reaching the target at test time. We enrich the benchmark dataset Room-to-Room (R2R) with sub-instructions and their corresponding paths. To make use of this data, we propose effective sub-instruction attention and shifting modules that select and attend to a single sub-instruction at each time-step. We implement our sub-instruction modules in four state-of-the-art agents, compare with their baseline models, and show that our proposed method improves the performance of all four agents. We release the Fine-Grained R2R dataset (FGR2R) and the code at https://github.com/YicongHong/Fine-Grained-R2R .

## introduction creating an agent that can navigate through an unknown environment following natural language instructions has been a dream of human-beings for many years. such an agent needs to possess the ability to perceive its environment, understand the instructions and learn the relationship between * authors contributed equally figure 1 : visual navigation with sub-instruction and sub-path pairs. we enrich the r2r dataset by providing fine-grained matching between sub-instructions and viewpoints along the ground-truth path. despite the significant progress made by recent approaches, there is little evidence that agents learn the correspondence between observations and instructions. @xcite found that a modified self-monitoring agent @xcite , could achieve similar performance with (success rate 40.5%) and without (success rate 39.7%) visual information. among other reasons, such as dataset bias, the result suggests that this agent gains little from having the two streams of information. we argue that one of the main reasons be-hind this is that current methods are not adequately teaching agents the relationship between perception-things that the robot is observingand parts of the instructions. since datasets do not provide such information agents can only use the ground-truth trajectory as a learning signal. moreover, given the lack of fine-grained annotation, current methods cannot evaluate the (perceptual or linguistic) grounding process at each step as there is no ground truth signal to indicate which part of the instruction has been completed. to address this problem, we enhance the r2r dataset @xcite to acquire intermediate supervision for the agents, providing a finegrained matching between sub-instructions and the agent's visual perception, as illustrated in figure 1 , to produce our fine-grained room-to-room dataset (fgr2r). we argue that the granularity of the navigation task should be at the level of these sub-instructions, rather than attempting to ground a specific part of the original long and complex instruction without any direct supervision or measure navigation progress at word level. our work aims to make the navigation process traceable and encourage the agent to run precisely on the described path rather than just focusing on reaching the target. we hypothesize that the agent should reach the target with higher success rate by following a detailed instruction with richer information, and in practice, the agent could complete some additional tasks on its way to the target. in light of this, we propose a novel subinstruction attention mechanism to better learn the correspondence between visual features and language features. our agent first segments the long and complicated instruction into short and easierto-understand sub-instructions using a heuristic method based on the grammatical relations provided by the stanford nlp parser @xcite . moreover, we propose a shifting module that infers whether the current sub-instruction has been completed. hence, only one sub-instruction is available to the agent at each time step for textual grounding. these modules can be easily applied to previous vln models. we conduct experiments to compare the performance of four state-of-the-art agents to evaluate with or without our sub-instruction module, for agents based on imitation learning @xcite @xcite and reinforcement learning @xcite . analyzing the results we find that the intermediate supervision and our proposed modules help the agents to better follow the instructions. furthermore, we demonstrate the traceability of the navigation process through qualitative and quantitative analysis. ## related work visual and textual grounding. visual grounding aims to infer the relationship between a text description and a spatial or temporal region in an image or video, respectively. it is an essential component for a variety of tasks in vision-and-language research such as visual question answering (vqa) @xcite @xcite , image captioning @xcite @xcite , video understanding @xcite @xcite and phrase localization @xcite . in the case of vision-and-language navigation, at each navigational step, the agent attends to the relevant part of the instruction according to visual clues to direct the future action. meanwhile, the agent attends the visual inputs at different directions as described by text to perceive the environment @xcite . vision and language navigation. @xcite formalized the vision-and-language navigation task in a photo-realistic environment, and proposed a benchmark room-to-room (r2r) dataset and a sequence-to-sequence agent as a baseline model. other datasets in real environments, such as r4r @xcite , which is an extended version of r2r with longer instruction-path pairs, and touchdown @xcite for navigation on streets have also been proposed for study. researchers have addressed the r2r task through a great variety of approaches. @xcite propose a look-ahead model that combines model-based and model-free reinforcement learning, predicts the agent's next state and reward during navigation. @xcite proposed the speaker-follower model which generates augmented samples for training and makes use of the panoramic action space to ground and navigate efficiently. later, @xcite introduced the self-monitoring agent which includes a vision and language co-grounding network and a progress monitor. the progress monitor estimates a normalized distance to the target and guides the transition of the textual attention. @xcite applied the reinforce algorithm @xcite to improve the agent's generalizability and proposed a self-supervised imitation learning (sil) method to facilitate lifelong learning in a new environment. the back translation agent @xcite applied the a2c algorithm @xcite and made use of a speaker module with environmental dropout for data augmentation. @xcite applied dynamic convolutional filters for image feature extraction for low-level grounding of visual inputs and @xcite grounded multiple modalities using a mixture-of-experts approach and applied joint training strategy. besides, the regretful agent @xcite and the tactical rewind agent @xcite are models which focus on path-scoring and backtracking methods. very recently, zhu et al. (2020a) introduces multiple auxiliary losses in training to help exploring the semantic meaning of visual features, huang et al. (2019) and hao et al. (2020) apply pre-trained encoders to generate generic visual and textual representations for the agent. in contrast to all previously mentioned methods that ground the complete instruction, we propose to divide the instruction into meaningful semantic sub-instructions, and teach the agent to complete each one at a time before reading the next subinstruction. in that spirit, our method is similar to the image captioning work by @xcite . they design a shifting gate over the image regions to control the visual features that feed into each time step of the caption module. we differ from their work in the modality that is attended. our method works in the language domain, and the shifting depends only on local context rather than looking over all the sub-instructions. babywalk @xcite ) is a concurrent work to ours, it uses sub-instructions for curriculum learning which trains the agent to complete shorter navigation tasks before trying to solve the longer ones. comparing the sub-instruction and sub-path pairs in fgr2r and babywalk, babywalk aligns the textual and visual sequences by solving a dynamic programming problem, whereas fgr2r employs human annotation, which is more fine-grained and accurate. ## sub-instruction aware vln in this section, we first introduce the vln problem and the general architecture of the agent. then, we discuss about the proposed chunking function for producing the sub-instructions and the novel sub-instruction module for enabling sub-instruction attention and transition. the vln task requires the agent to navigate through a real environment to a target location following a natural language instruction. formally, an instruction w is a sequence of words w 1 , w 2 , . . . , w l provided to the agent at the beginning of its navigation, where w i denotes the i-th word in the sequence. the environment is defined as set of viewpoints {p j } denoting all the navigable locations. at time step t, the agent at viewpoint p t receives a panoramic view v t composed of n single view images v t,1 , v t,2 , . . . , v t,n . using the given instruction w and the current observation v t , the agent needs to infer an action a t which triggers a transition signal from p t to p t+1 . the episode ends when the agent output a st op action or the maximum number of steps allowed is reached. ## the fgr2r dataset to acquire the matching between vision and language sub-sequences, we introduce a fine-grained room-to-room (fgr2r) dataset which enriches the benchmark room-to-room dataset by dividing the instructions into sub-instructions and pairing each of those with their corresponding viewpoints in the path. dataset collection. we first apply the chunking function introduced in section 3.2 to generate the sub-instructions automatically from the original r2r data. we demonstrate the quality of the generated sub-instructions by comparing the output sub-instructions against a manually annotated subset of 300 samples, obtaining a smoothed bleu-4 score of 0.84. then, we add annotations of subpath corresponding to each sub-instruction using the amazon mechanical turk (amt) 2 . we refer the readers to appendix a.1 for more information about the data collection interface and the qualification process of the annotators that we designed to ensure the quality of the collected data. dataset statistics. the original r2r possesses 21,567 navigation instructions and 7,189 paths in 91 real-world environments, where 3 or 4 different natural language instructions describe each path. the r2r data has been split for learning proposes, with 4,675 paths for training and 340 paths for seen validation in 61 scenes, 783 paths in 11 scenes for unseen validation and the remaining 1,391 paths in 18 scenes for testing foot_0 . based on the original r2r data, fgr2r divides the instructions for the training and validation set in an average of 3.6 subinstructions. each sub-instruction has 7.2 words on average. sub-instructions are paired on average 2.4 viewpoints, and with a minimum and maximum of 1 and 7 viewpoints, respectively. we refer the readers to appendix a.1 for more dataset statistics. ## results and analysis we compare the performance of the four agents on the r2r unseen validation set. we also present the traceability of the navigation process resulting from our fgr2r data. ## conclusion in this paper we introduce a novel sub-instruction module and the fine-grained r2r dataset to encourage the learning of correspondences between vision and language. the sub-instruction module enables the agent to attend to one particular sub-instruction at each time-step and decides whether the agent needs to proceed to the next subinstruction. our experiments show that by implementing the sub-instruction module in state-of-theart agents, most of the agents are able to follow the given instruction more closely and achieve better performance. we also show that, with the subinstruction annotations, the entire navigation trajectory is trackable. we believe that the idea of subinstruction module and a sub-instruction annotated dataset can benefit future studies in the vln task as well as other vision-and-language problems. wang zhu, hexiang hu, jiacheng chen, zhiwei deng, vihan jain, eugene ie, and fei sha. 2020b. baby-walk: going farther in vision-and-language navigation by taking baby steps. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 2539-2556. association for computational linguistics.