FOAM : A Follower-aware Speaker Model For Vision-and-Language Navigation (2022)

Abstract: The speaker-follower models have proven to be effective in vision-and-language navigation, where a speaker model is used to synthesize new instructions to augment the training data for a follower navigation model. However, in previous work, the speaker model is follower-agnostic and fails to take the state of the follower into consideration. In this paper, we present FOAM, a FOllower-Aware speaker Model that is constantly updated given the follower feedback, so that the generated instructions can be more suitable to the current learning state of the follower. Specifically, we optimize the speaker using a bi-level optimization framework and obtain its training signals by evaluating the follower on labeled data. Experimental results on the Room-to-Room and Room-across-Room datasets demonstrate that our methods can outperform strong baseline models across settings. Analyses also reveal that our generated instructions are of higher quality than the baselines.

## introduction the task of vision-and-language navigation (vln) requires an agent to navigate in a real-world environment given natural language instructions. in vln, one of the major challenges is the lack of training data. to alleviate the issue, speakerfollower models @xcite have been proposed. specifically, in the speaker-follower models, an instruction-follower agent is trained to follow a provided natural language instruction to complete a specified goal, and a speaker model learns to model how humans describe routes and synthesize new instructions so as to create more training data for the follower. while speaker-augmented data is widely used in vln (fried et al., 2018b; wang et al., 2019; ma et al., 2019; tan et al., 2019; zhu et al., 2020a; 1 code is available at https://github.com/ pluslabnlp/follower_aware_speaker . ## methods we first introduce the background before discussing the details of foam. ## related work we overview two lines of related work: vision-and-language navigation. training embodied navigation agents has been an increasingly active research area @xcite @xcite @xcite . @xcite propose to augment the training data with the speaker-follower models, which is improve by @xcite who add noise into the environments so that the speaker can generate more diverse instructions. @xcite propose methods to measure the quality of the generated instructions and filter noisy samples. @xcite propose to adversarially sample the most difficult paths for the follower and translate these paths into instructions using the speaker for data augmentation. while using the speaker-augmented data has been widely used in vln, most of the existing work has been focused on improving the follower navigation model @xcite @xcite . for example, the self-monitoring agent @xcite improves cross-modal alignment through a visual-text co-grounding module and a progress monitor; @xcite propose to utilize four self-supervised auxiliary tasks that can provide additional training signals for the agent. most similar to our work, @xcite build a speaker that reason about how the instructions may be interpreted; @xcite propose a generative approach where a speaker model is trained to model the probability of an instructions given actions, and the follower chooses actions that maximize this probability. bi-level optimization. bi-level optimization algorithms have been widely applied in various fields, such as learning initialization parameters @xcite , neural architecture search @xcite , re-weighting training data @xcite . our method takes inspirations from @xcite , which is applied in pseudo labeling and optimizes the teacher parameters given the student feedback. similar techniques have also been used in machine translation @xcite , where a meta-validation set is constructed to evaluate the model performance and provide feedback. ## conclusions in this paper, we propose the foam model where we improve the speaker-follower model in visionand-language navigation by constantly updating the speaker given the follower feedback during training. we frame the idea as a bi-level optimization problem and obtain the feedback signal based on the performance of the follower on labeled data. experimental results on room-to-room and roomacross-room datasets demonstrate that our method can outperform strong vln baselines in different settings. analyses also suggest that the quality of our speaker model is indeed improved during training. future directions include testing our method on more datasets and investigating more options on the feedback signals. fengda zhu, yi zhu, xiaojun chang, and xiaodan liang. 2020a. vision-language navigation with selfsupervised auxiliary reasoning tasks. in proceedings of the conference on computer vision and pattern recognition (cvpr). wang zhu, hexiang hu, jiacheng chen, zhiwei deng, vihan jain, eugene ie, and fei sha. 2020b. babywalk: going farther in vision-and-language navigation by taking baby steps. in proceedings of the annual meeting of the association for computational linguistics (acl).