NDH -Full: Learning and Evaluating Navigational Agents on Full-Length Dialogue (2021)

Abstract: Communication between human and mobile agents is getting increasingly important as such agents are widely deployed in our daily lives. Vision-and-Dialogue Navigation is one of the tasks that evaluate the agent’s ability to interact with humans for assistance and navigate based on natural language responses. In this paper, we explore the Navigation from Dialogue History (NDH) task, which is based on the Cooperative Vision-and-Dialogue Navigation (CVDN) dataset, and present a state-of-the-art model which is built upon Vision-Language transformers. However, despite achieving competitive performance, we find that the agent in the NDH task is not evaluated appropriately by the primary metric – Goal Progress. By analyzing the performance mismatch between Goal Progress and other metrics (e.g., normalized Dynamic Time Warping) from our state-of-the-art model, we show that NDH’s sub-path based task setup (i.e., navigating partial trajectory based on its correspondent subset of the full dialogue) does not provide the agent with enough supervision signal towards the goal region. Therefore, we propose a new task setup called NDH-Full which takes the full dialogue and the whole navigation path as one instance. We present a strong baseline model and show initial results on this new task. We further describe several approaches that we try, in order to improve the model performance (based on curriculum learning, pre-training, and data-augmentation), suggesting potential useful training methods on this new NDH-Full task.

## introduction with the increased number of intelligent agents being deployed in our daily lives, effective communication between humans and agents is becoming more important. natural language is one of the 1 our code and dataset are publicly available at: https: //github.com/hyounghk/ndh-full should i stay on this floor or go down the stairs? yeah, go down the stairs. and then i think you turn right but i can't really tell. should i go left toward the door, or right around the corner? go to the left, through that door. target: picture most effective ways of communication due to its flexibility. therefore, many efforts have been devoted to exploring the potential of its application in several tasks. vision-and-language navigation (vln) is one of the tasks in which agents have to navigate to a goal location in the indoor or outdoor environment by following natural language instructions @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite . while most vln datasets only provide instructions from the oracle without considering the navigator's response, the useful cooperative vision-and-dialogue navigation (cvdn) @xcite dataset extends this one-way communication to two-way multi-turn dialogue (english) interaction between the oracle and the navigator. the dataset simulates a situation in which agents navigate through indoor environments towards a goal region by holding a conversation with humans for oracle guidance. figure 1 shows an example in the cvdn dataset. given the target in-formation "picture" only, the navigator is asked to explore the environment by intuition (green path). the navigator can ask the oracle for assistance during navigation and then make progress (blue and red path) based on the oracle's response. from this dataset, @xcite proposed the navigation from dialogue history (ndh) task, in which the agents are asked to navigate toward the goal region g given dialogue history and the current round of the dialogue. however, we find that this sub-path-based task setup does not provide enough supervision for the agent to reach the goal region g, and its primary evaluation metric -goal progress (gp) does not appropriately measure the agent's performance on the sub-path based task. in the example shown in figure 1 , one cvdn example is split into three navigation instances starting from p 0 , p 1 , p 2 and ends at p 1 , p 2 , g, respectively. one ndh instance only contains dialogue before the current navigation path (e.g., for navigation from p 1 to p 2 , the agent only knows the target "picture" and the first round of the dialogue, which is in the blue box), thus lacks supervision for how to navigate from p 2 to the goal region g. however, the agent is evaluated with gp -the distance made towards the goal region g from its starting point. this metric does not consider whether the agent follows the reference path. as a result, the agent could wander around to get a high gp score without following the path. hence, in this paper, we aim to redefine the ndh task via enhanced levels of supervision given to the agent, for better path fidelity while maintaining the advantage of learning from interactive dialogues. for this, we first build a strong state-of-the-art model based on vision-language transformers and pre-training, and illustrate that the current ndh task setup is not suitable for evaluating the agent's ability to follow natural language instructions. we show this by comparing the behaviors of the model on different evaluation metrics. specifically, we find that a model with a higher gp score has a lower ndtw (normalized dynamic time warping; ilharco et al. (2019) ) scores (see therefore, we next propose a new task setup called ndh-full. we combine the sub-paths from the ndh task into the full path with the corresponding full dialogue, allowing the full supervision for agents on the instruction-following navigation task setup. as shown in the example of figure 2 , the ndh-full instance requires the agent to navigate from p 0 to g with full dialogue instruction (i.e., target and multiple rounds of dialogues). in this setting, the agent has explicit supervision towards the goal region and is further faced with the challenge of understanding and grounding longer dialogues to navigate longer paths compared with the ndh task. we present a strong baseline model and several enhancement suggestions (based on curriculum learning, pre-training, and data-augmentation) for this task, and still leaves a large room for useful future work by the community on this challenging and realistic ndh-full task setup. our contributions are three-fold: (1) we first present a state-of-the-art model for the ndh task. ## dataset background and task setup in this section, we discuss the vision-and-dialogue navigation task (ndh). we first introduce the cvdn dataset, and then show the two main issues of ndh and propose a new setup, ndh-full. ## ndh and ndh-full models we present the ndh task model and ndh-full task model in this section. to be specific, the ndh task model is built based on the visionand-language transformer. similar to the previous works @xcite , we employ lxmert @xcite as the base architecture (figure 3 ). the ndh-full task model takes the same architecture and is dialogue encoder visual view encoder dialogue progressor cross-attention + self-attention … … proxy token [cls] towel [sep] [cls] towel [sep] back in the bathroom or in the bedroom? [sep] go back towards the bed … down the curvy wooden hallway [sep] [cls] towel [sep] up the mini stairs, bathroom or other room? [sep] go towards the mini stairs and … with a blue couch [sep] proxy token additionally equipped with the progressor module for moving through dialogue rounds. the ndh task model shows the state-of-the-art performance. however, by analyzing the behavior of the ndh task model on different metrics, we find the ndh task might not be suitable for evaluating the instructing-following navigation ability, thus, we propose the new ndh-full task and the baseline model (see sec. 6.1 and 6.2). pre-training model. pre-training is an effective approach to infuse prior knowledge in the vision-and-language navigation models (majumdar et al., 2020; @xcite . compared with the previous works, our work proposes a new objective for pre-training. instead of training the model with similarity score prediction @xcite or discrete action label @xcite , we train the model with the objective that is nearly identical to the main navigation task for more effective transfer to the main task. given a visual view sequence v @xmath0, v 2 , ..., v t } and a corresponding navigation dialogue d @xmath1, d i2 , ..., d i|d i | }, we train the model to select the next view to proceed among the candidates c @xmath2, c t2 , ..., c t|ct| }. additionally, we apply masked visual view prediction and masked language model loss as well. we employ resnet @xcite to get visual view features from panoramic images and use a multilayer transformer to encode dialogue features like in lxmert. the encoded features are fed to the lxmert-based transformer module, tf lxt . ## experimental setup metrics. we consider ndtw as the main metric of the new ndh-full task because ndtw reflects path fidelity better than other metrics @xcite . other than ndtw, we also present evaluation results on success rate (sr), success weighted by path length (spl), trajectory length (tl), and goal progress (gp) to allow evaluation from different perspectives. training details. for the pre-training model, we use 9 language and 5 cross-modal lxmert layers (but did not use their pre-trained weights), and use 768 as the hidden size. following @xcite , we use adam (kingma and ba, 2015) as the optimizer with the learning rate 1 × 10 -4 and linear decay as in @xcite . we use l2 loss for visual view prediction, and cross-entropy loss for masked language model and next view selection. we use cvdn @xcite , r2r @xcite , and a part of r2r's augmented data @xcite as the training data. for the ndh task model, we use adamw @xcite as the optimizer with the learning rate 1 × 10 -5 . only cvdn data is used for fine-tuning the model. in the ndh-full task, we do not apply pre-training for the full-dialogue model. we use resnet-152 feature and resnet50based clip feature. all the experiments are run using the nvidia titan xp / geforce gtx 1080 ti / geforce rtx 2080 ti gpus. we use py-torch @xcite to build all models. we use manual tuning (e.g, learning @xmath3, ..., 1 × 10 -6 }, and the layers of the transformer @xmath4, 9/5}) for selecting hyper-parameters. the number of trainable ## conclusion we explored the ndh task, which is built on the useful cooperative vision-and-dialogue navigation (cvdn) dataset, and found the mismatch between the task setup and evaluation by analyzing the scoring behaviors of our state-of-the-art model. therefore, we proposed a new task called ndh-full. we combined all split paths and dialogue rounds of ndh to create the full path and dialogue, resulting ndh-full has longer paths and dialogues than ndh and it makes ndh-full more challenging. we also presented a baseline model, resulting scores, and suggestions for promising research directions on the ndh-full task.