  Vision-and-language navigation (VLN) is a multimodal task where an agent
follows natural language instructions and navigates in visual environments.
Multiple setups have been proposed, and researchers apply new model
architectures or training techniques to boost navigation performance. However,
recent studies witness a slow-down in the performance improvements in both
indoor and outdoor VLN tasks, and the agents' inner mechanisms for making
navigation decisions remain unclear. To the best of our knowledge, the way the
agents perceive the multimodal input is under-studied and clearly needs
investigations. In this work, we conduct a series of diagnostic experiments to
unveil agents' focus during navigation. Results show that indoor navigation
agents refer to both object tokens and direction tokens in the instruction when
making decisions. In contrast, outdoor navigation agents heavily rely on
direction tokens and have a poor understanding of the object tokens.
Furthermore, instead of merely staring at surrounding objects, indoor
navigation agents can set their sights on objects further from the current
viewpoint. When it comes to vision-and-language alignments, many models claim
that they are able to align object tokens with certain visual targets, but we
cast doubt on the reliability of such alignments.
