Masked Path Modeling for Vision-and-Language Navigation (2023)

Abstract: Vision-and-language navigation (VLN) agents are trained to navigate in real-world environments based on natural language instructions. A major challenge in VLN is the limited available training data, which hinders the models’ ability to generalize effectively. Previous approaches have attempted to alleviate this issue by using external tools to generate pseudo-labeled data or integrating web-scaled image-text pairs during training. However, these methods often rely on automatically-generated or out-of-domain data, leading to challenges such as suboptimal data quality and domain mismatch. In this paper, we introduce a masked path modeling (MPM) objective. MPM pretrains an agent using self-collected data for subsequent navigation tasks, eliminating the need for external tools. Specifically, our method allows the agent to explore navigation environments and record the paths it traverses alongside the corresponding agent actions. Subsequently, we train the agent on this collected data to reconstruct the original action sequence when given a randomly masked subsequence of the original path. This approach enables the agent to accumulate a diverse and substantial dataset, facilitating the connection between visual observations of paths and the agent’s actions, which is the foundation of the VLN task. Importantly, the collected data are in-domain, and the training process avoids synthetic data with uncertain quality, addressing previous issues. We conduct experiments on various VLN datasets and demonstrate the applications of MPM across different levels of instruction complexity. Our results exhibit significant improvements in success rates, with enhancements of 1.3%, 1.1%, and 1.2% on the val-unseen split of the Room-to-Room, Room-for-Room, and Room-across-Room datasets, respectively. Additionally, we underscore the adaptability of MPM as well as the potential for additional improvements when the agent is allowed to explore unseen environments prior to testing.

## introduction a vision-and-language navigation (vln) agent is trained to follow natural language instructions and navigate within an environment to achieve a specified goal. this task requires the agent to possess several sophisticated abilities, including understanding and grounding language phrases to visual objects, as well as planning and executing actions in a real-world setting. the pretraining-then-finetuning paradigm @xcite @xcite has proven to be effective in addressing these challenges in the field of vln. by utilizing various supervision signals and proposing pretraining objectives, significant improvements have been demonstrated across vln tasks. prior works have explored the use of internetscale image-text datasets to acquire grounded vision and language representations. notable contributions in this area include the works of @xcite and @xcite , who leverage web-scraped image-caption corpora to learn general vision-language representations and then finetune the models using in-domain vln data to adapt the representations specifically for vln tasks. similarly, @xcite and @xcite employ the clip vision encoder @xcite pretrained with image-text contrastive loss and showcase the application of such models in various embodied tasks. while the vision-language pretrained representations improve the alignment between vision and language modalities, it is important to note that there is a domain gap between the pretraining data and vln data. in addition, these models are unaware of how to connect the learned representations to actions because they are not explicitly trained for generating actions, which is a critical skill for vln tasks. to address the issue and integrate action generation into pretraining, prior works such as prevalent @xcite hamt @xcite ) use a single-step action prediction objective based on human-annotated and synthetic image-text-action triplets. however, the scalability of the pretraining objective is limited due to the scarcity of annotated instruction-action pairs. specifically, they require training on pairs of natural language instruction and action sequences which are costly to obtain in a scalable way. there is also a line of work that utilizes synthetic data for training, either obtaining pseudo-instructions from sampled paths @xcite @xcite or generating both the visual environments as well as the language instructions @xcite . however, the automatically-generated data cannot be perfect and the noise during data generation can impact the model performance. in this paper, we present an approach to pretrain vln models with masked path modeling (mpm), which leverages in-domain path data collected by an agent for self-supervised learning. the proposed objective targets addressing the two major limitations of previous work: as a result, our pretraining objective is scalable and well-suited for addressing the vln task. we evaluate the proposed method on various vln datasets with different types of instructions, including room-to-room @xcite , room-for-room @xcite , and roomacross-room @xcite . experimental results demonstrate that mpm can achieve significant improvements in both seen and unseen environments compared with strong baselines. for example, we achieve improvements of 1.32%, 1.05%, and 1.19% on the val-unseen split of the room-to-room, room-for-room, and room-across-room datasets, respectively. in addition, we demonstrate the mpm can be flexibly integrated into different types of models and achieve improvements. furthermore, an analysis reveals the potential for additional improvements when the agent is allowed to explore unseen environments prior to testing. ## methods in this section, we first introduce the basic settings and model architecture of vision-and-language navigation, then illustrate the details of each component of our proposed approach. ## experiments in this section, we present our experimental results with the proposed masked path modeling objective. ## conclusion in this paper, we propose masked path modeling, a pretraining objective designed for vision-andlanguage navigation. the objective can utilize scalable data explored from different environments to improve the agent's conditioned action generation ability. we incorporate the objective into a strong baseline and demonstrate improvements across different settings. an analysis further reveals the potential for scaling the objective to large-scale settings. future directions include designing better exploration strategies as well as investigating applications in more fields.