Learning to Stop: A Simple yet Effective Approach to Urban Vision-Language Navigation (2020)

Abstract: Vision-and-Language Navigation (VLN) is a natural language grounding task where an agent learns to follow language instructions and navigate to specified destinations in real-world environments. A key challenge is to recognize and stop at the correct location, especially for complicated outdoor environments. Existing methods treat the STOP action equally as other actions, which results in undesirable behaviors that the agent often fails to stop at the destination even though it might be on the right path. Therefore, we propose Learning to Stop (L2Stop), a simple yet effective policy module that differentiates STOP and other actions. Our approach achieves the new state of the art on a challenging urban VLN dataset Touchdown, outperforming the baseline by 6.89% (absolute improvement) on Success weighted by Edit Distance (SED).

## introduction vision-and-language navigation (vln) aims at training an agent to navigate in real environments by following natural language instructions. compared to indoor vln @xcite , navigation in urban environments @xcite is particularly challenging, since urban environments are often more diverse and complex. several research studies @xcite @xcite have been conducted to solve the problem. in this paper, we also focus on the urban vln task. as shown in fig. 1 , given a natural language instruction, the agent perceives local visual scene and chooses actions at every time step, learning to match the instruction with the produced trajectory and navigate to the destination. existing vln models @xcite @xcite @xcite seem to neglect the importance of the stop action and treat all actions equally. however, this can lead to undesirable behaviors, also noticed in @xcite ; @xcite , that the agent fails to stop at the target although it might be on the right path, because the stop action is severely underestimated. we argue that the stop action in the urban vln tasks is crucially important and deserves special treatment. first, in contrast to errors on other actions that are likely to be fixed later in the journey, the price of stopping at a wrong location is higher, because producing stop terminates the episode, and there will be no chance to fix a wrong stop. second, the statistical count of stop is much lower than other actions as it only appears once per episode. thus stop will receive less attention if we treat all actions equally and ignore the difference of occurrence frequency. moreover, stop and other actions need different understandings of the dynamics between the instruction and the visual scene. both require the alignment between trajectories and instructions, but stop would emphasize the completeness of the instruction and the matching between the inferred target and the sur-rounding scene, while choosing directions requires a planning ability to imagine the future trajectory. therefore, we introduce a learning to stop (l2stop) module to address the issues. l2stop is a simple and model-agnostic module, which can be easily plugged into vln models to improve their navigation performance. as we demonstrate in fig. 1 , the l2stop module consists of a stop indicator to determine whether to stop and a direction decider to choose directions when at key points. besides, we weigh stop action more than other actions in the loss function, forcing the agent to pay more attention to the stop action. we conduct experiments on a language-grounded street-view navigation dataset touchdown foot_0 @xcite . extensive results show that our proposed approach significantly improves the performance over the baseline model on all metrics and achieves the new state-of-the-art on the touchdown dataset. foot_1 ## approach fig. 2 illustrates the framework of our l2stop model. specifically, a text encoder and visual encoder are used to get the text and visual representations. then the trajectory encoder uses the representations to compute the hidden context state, which is the input of the policy module. unlike previous vln models, which use one branch policy module, we use our proposed l2stop module, a two-branch policy module that separates the policies for stop and other actions. we detail each component below. ## conclusion we investigate the importance of the stop action and study how to learn a policy that can not only make better decisions on where to go but also stop more accurately. we propose the l2stop module for the vision-language navigation task situated in urban environments. experiments illustrate that l2stop is modular and can be plugged into other vln models to further boost their performance in urban environments.