Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel
Authors: Zun Wang
Jialu Li
Yicong Hong
Songze Li
Kunchang Li
Shoubin Yu
Yi Wang
Yu Qiao
Yali Wang
Mohit Bansal
Limin Wang
Venue: ICLR 2025 Poster (2025)
Keywords: vision-and-language navigation
data flywheel
dataset curation
Primary area: applications to robotics, autonomy, planning

Abstract: Creating high-quality data for training robust language-instructed agents is a long-lasting challenge in embodied AI. In this paper, we introduce a Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale navigational instruction-trajectory pairs by iteratively refining the data pool through the collaboration between two models, the instruction generator and the navigator, without any human-in-the-loop annotation. 
Specifically, SRDF starts with using a base generator to create an initial data pool for training a base navigator, followed by applying the trained navigator to filter the data pool. This leads to higher-fidelity data to train a better generator, which can, in turn, produce higher-quality data for training the next-round navigator. Such a flywheel establishes a data self-refining process, yielding a continuously improved and highly effective dataset for large-scale language-guided navigation learning. Our experiments demonstrate that after several flywheel rounds, the navigator elevates the performance boundary from 70\% to 78\% SPL on the classic R2R test set, surpassing human performance (76\%) for the first time. 
Meanwhile, this process results in a superior generator, evidenced by a SPICE increase from 23.5 to 26.2, better than all previous VLN instruction generation methods. Finally, we demonstrate the scalability of our method through increasing environment and instruction diversity, and
the generalization ability of our pre-trained navigator across various downstream navigation tasks, surpassing state-of-the-art methods by a large margin in all cases.

Published as a conference paper at ICLR 2025
BOOTSTRAPPING LANGUAGE-GUIDED NAVIGATION LEARNING WITH SELF-REFINING DATA FLYWHEEL
Zun Wang 1.2c Jialu Li
Yicong Hong Songze Li
Kunchang Li'
Shoubin Yu?
Yi Wang!. 1.5 Yu Qiao'
Yali Wang' Mohit Bansal Limin Wang
Shanghai Al Laboratory
UNC Chapel Hill
3 Adobe Research
*Nanjing University Shanghai Innovation Institute
(zunwang, jialuli, mbansal] @cs. wes.uunc... unc. enu. @anglimi@@pii wwa.lorg............ooo..o...o org.en en
ABSTRACT
Creating high-quality data for training robust language-instructed agents is long- lasting challenge in embodied AI. In this paper, we introduce Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale navigational instruction-traaecton pairs by iteratively refining the data pool through the collab- oration between two models, the instruction generator and the navigator, without any human-in- in-the-loop annotation. Specifically, SRDF starts with using base generator to create an initial data pool for training a base navigator, followed by applying the trained navigator to filter the data pool. This leads to higher-fidele data to train a better generator, which can, in turn, produce higher-quality data for training the next-round navigator. Such flywheel establishes data self- refining process, yielding a continuously improved and highly effective dataset for large-scale language-guider navigation learning. Our experiments demon- strate that after several flywheel rounds, the navigator elevates the performance boundary from 70% to 78% SPL on the classic R2R test set, surpassing human performance (76%) for the first time. Meanwhile, this process results in a superior generator, evidenced by a SPICE increase from 23.5 26.2, better than all previ- ous VLN instruction generation methods. Finally, we demonstrate the scalability of our method through increasing environment and instruction diversity, and the generalization ability of our pre- trained navigator across various downstream nay- igation tasks, surpassing state-of ofttte-arttt methods by large margin all cases."
Generator improves Nevigator Data Refining
(Prreggaated VLA0
Previous
Human
Novigstor improves Generafor via Data Fillering
ScakeV.
Oure
(Long-cam MLNI
Iratrction Coramation)
VI.N F Data Pool
242
Generator
Navigator
712
CVLM Dislog) CAEN
Pasn fo-ffte--aoo
Generator
Navigator
28
Previous
High-leee lewe VUNI
1ontthupue Erw VL.0 (GFH-C2
Generator
Navigator
ougneniation
Traking
CVD0:
Roflow wDTW
TESCRE
Othwen: CP1
(a) Pipeline
(b) Comparisons
Figure 1: (a) Our Pipeline: After using the (instruction) generator to label paths for data augmenta- tion in navigator training, we leverage the trained navigator to filter high-quality data to train a better generator, and the improved generator refines the data pool to train stronger navigator, iteratively running on the flywheel. (b) Comparison with state-of-the-art (SoTA) methods: Our approach significantly outperforms SoTA on all tasks. It also surpasses human performance on R2R and approaches numan-leved results on RxR-Englin and CVDN (for other tasks, human performance is not reported in their paper). The R2R result from the test set, while others are from val unseen.
'Code and data are available https: hom.tttooooooooee.oooooooooo wz0 011 9-VIN- Intermed Shanghai AT Laboratory.

---

Published as a conference paper at ICLR 2025
1 INTRODUCTION
The lack of high-quality data is one of the main bottlenecks in training embodied agents complete Saahh-uualtyyaaaaae
real-world human activities. Unlike many other discriminative or generative learning problems, where the data itself naturally formulates a self-supervised learning objective (Devlin, 2018; He et al., 2022) or the data labeling can be facilitated by existing models (Ros et al., 2016; Tian et al., 2024), training embodied agents usually requires expensive human annotation on complex vision- linguistic contents and physical interactions. In the essential embodied Al problem of Vision-ana andd Language Navigation (VLN) (Zhang al., 2024b), one widely considered solution is to first train a senjectory-to-istruee a small amount of human-annotater data and then use the generator to caption large-scale trajec- tories sampled from interactive environments (Hao et al., 2020; Chen et al., 2022c; Wang et al., 2023d). Among them, Hao et al. (2020) demonstrates the effectiveness of scaling the synthetic cntructioo-traaector pairs in the existing training environments, while Chen et al. (2022c); Wang et al. (2023d); Li & Bansal (2023) emphasizes the importance of scaling training environments for better generalization ability of agents. However, the quality of the synthetic data, especially the lan- guage fidelity, is highly under-investigater We find that training solely with the large-scale synthetic data (352x larger in pnorruction-traecttnn pairs, and 13 more diverse in environments) yields worse results compared to training with small human-annotater annotated dataset (see Table 1), which indicates the need for high-quality instructions beside simply scaling the data and environments.
generator (Fried et al., 2018; Tan et al., 2019; Chen et al., 2022c) using
Though many methods have been proposed for improving data quality in multi-modal understand- ing and generation tasks (Fang et al., 2023; Li et al., 2024a; Betker et al.; Fang et al.. 2024; Li et al., 2024b; Nguyen et al., 2024b). improving synthetic instruction instruction-traaectory trajectory pairs for language- guided navigation has several distinct challenges. First, the most straightforward approach to build a strong instruction generator, but the limited amount of high-quality training data (e.g., R2R (Anderson et al., 2018b) train split has only 14K numan-labeled data) makes it challeng- ing to train robust generator capable of producing high- figh-tidelity instructions for diverse trajec- tories. Additionally, manually correcting instructions by humans is resource-intensive and costly. Moreover, the alignment of instruction- trajeccoor pairs in VLN is hard to evaluate, as the instructions not only contain semantic information (e.g., 'Walk past the ta- ble') but also have rich directional information (e.g.. 'Turn left') to match with the corresponding trajectory. Besides, the visual elements mentioned in the instruc- tions are also temporally aligned to panoramas in mul- tiple scenes. As result, traditional metrics like CLIP score (Radford et al., 2021) struggle to evaluate such multi-scene directional and semantic alignment, as they only capture single-scene semantic relationships.
Table 1: Performance (on R2R validation unseen split) on different datasets solely. Directly training with R2R yields the best SPL compared to training with other aug- mentation datasets.
Training Data
#data
#Env.
SRT
SPL
R2R
14K
61
65.9
55.9
Prevalent ScaleVIN
1.0M
60
67.1
54.8
4.9M
800
63.9
50.1
In this work, we propose a Self-Refining Data Flywheel, SRDF, that automatically evaluates and improves the quality of the generated instructions at scale, through an iterative collaboration be- tween the navigator and the instruction generator. As shown in Figure I (a), our first step is similar to ScaleVLN'ss process (Wang et al., 2023d), which trains an instruction generator using the original human-labeles data, then generates instructions for unlabeled paths sampled from 800 HM3D train- ing environments and trains a strong navigator using the generated data. Then, for evaluating the generated instructions, we propose to use the trained navigator's path-fidelity score (nDTW (Ilharco et al., 2019) and SPL (Anderson et al., 201 8a), measuring how closely the navigator's 's followed path matches the original trajectory) as the similarity score. Since the trained navigator is strong enough (achieves human-level performance in Wang et al. (2023d)) and has already learned to connect vi- sual landmarks and directional cues to actions effectively, its fidelity in following the instructions pairs are aligned. It also avoids manually
can naturally reflect how well the instruction-traaecti setting vague thresholds when using CLIP-scorelikee metrics for filtering. In our case, SPL=1 yields perfect trajectory match. After using the navigator the generated data, this subset of data will be used to train a better instruction generator in the next iteration. The improved instruction generator re- re-generates instructions for bad samples to produce better datasets, which is used to train a stronger navigator. The process iterates, with the navigator improving the instruction generator via data filtering, and the instruction generator improving the
filter to obtain a high-quality subset of
2

---

Published as a conference paper at ICLR 2025
navigator via data refining, ultimately producing both highly capable navigator and instruction generator, along with a substantially high-quality synthetic VLN dataset.
We build our flywheel upon the R2R dataset (Anderson al., 2018b) and provide detailed analy- sis. Empirically, we show that our navigator and instruction generator can iteratively improve each other with our SRDF. We also demonstrate the scalability of our method: the instruction genera- tor consistently improves with additional environments and data, and the navigator benefits more from increased instruction diversity when trained with our high-quality instructions compared to with low-quality datasets. On the R2R dataset, our approach surpasses previous state-of-the-art re- sults by a wide margin in both instruction following and generation, and notably, for the first time, we significantly surpass human performance (76% SPL) in instruction following, demonstrating the effectiveness of our SRDF to improve data quality. Furthermore, we evaluate the transferabil- ity of our pre-trained navigator across various downstream navigation tasks, including VLN with dialogue-ban based instructions (CVDN (Thomason et al., 2020)), long-term VLN (RxR-Englie (Ku et al., 2020), R4R (Zhu et al., 2020)), high-level VLN (SOON (Zhu et al., 2021). REVERIE (Qi et al., 2020)), and even VLN in continuous environments (R2R-CE (Krantz et al., 2020)). As shown in Figure 1 (b), we achieve state-of -ortte--tttttttt- performance on all the tasks, while approaching human performance for RxR-Englite and CVDN, underscoring the superior quality of our generated data and the robust transferability of our pre-trained navigator.
2 RELATED WORK
Vision-and- Lan-uuga Navigation. VLN requires an agent to navigate unseen environments based on natural language instructions. Numerous datasets have been proposed for this task (An- derson et al., 201 8b; Ku et al., 2020; Qi et al., 2020; Shridhar et al., 2020; Thomason et al., 2020; Padmakumar et al., 2022; Nguyen & Daumé III, 2019; Chen et al., 2019; Kim et al., 2021), span- ning both indoor and outdoor environments with varied levels of instruction detail. The limited availability of human-annotated data for training generalizable VLN agents to achieve near-human performance is a key challenge due to the high cost of collecting instruction--raaeccooyy pairs. To address this, various data augmentation approaches have been explored. Some focus scaling environments by editing existing ones (Li et al., 2022; Liu et al., 2021b) or generating new ones with text-to-imaae models (Li & Bansal, 2023). Others scale instruction data by training instruction generators to generate instructions for unannotated paths (Hao et al., 2020; Zhang & Kordjamshidi, 2023; Zhang et al., 2024a), or by leveraging large sets of rendered environments from simulators (e.g.. HM3D (Ramakrishnan et al., 2021), Gibson (Xia et al., 2018)) (Wang et al., 2023d; Kamath al., 2022: Chen et al., 2022c). While data scaling has been effective for VLN, the quality of the data, particularly the alignment between instructions and trajectories, remains under-explored. In this paper. we investigate the impact of data quality on VLN and propose a data flywheel that iteratively refines itself through collaboration between the navigator and the generator.
High-Quality Multimodal Dataset Curation. Many multimodal studies show that improving data quality can significantly enhance model performance, either through advanced dataset filter- ing (Fang et al., 2023; Li et al., 2024a; Gadre et al., 2024; Sun et al., 2023) or refining captions with strong models (Betker et al.; Fang et al., 2024: Li et al., 2024b; Nguyen et al., 2024b; Wang al., 2024c; Tan et al., 2024). Recently, Segment Anything (SAM) (Kirillov et al., 2023) demonstrated how data and models can improve each other through a data flywheel with a numan-in-the-loow process, evolving from model-assisted to fully automated annotation. Our data flywheel similarly integrates filtering and re-captioning data via navigator verification and generator refinement, oper- ating in data-model loop like SAM, but without any human intervention.
Self-Improving Language Models. Studies show that Large Language Models (LLMs) can im- prove themselves by training on their own generated outputs across tasks like programming (Halupt- zok et al., 2022), summarization (Patil et al., 2024), question -nnwoerin (Lee et al., 2024; Yu et al., 2024), reasoning (Prasad al., 2024), and others (Li et al., 2023a; Madaan et al., 2024; Zhou al., 2024c), where the quality of the self-generated data is ensured via human (Ouyang et al., 2022; Bai et al., 2022a), off-the-shelf verifiers/reward models (Ni et al., 2023; Wang et al., 2019; Dou et al., 2024; Bai et al., 2022b; Lee et al., 2023; Nguyen et al., 2024a) or model's self feedback (Yuan et al., 2024; Wu et al., 2024; Wang et al., 2024d). In our pipeline, the instruction generator iter- atively self-improves using its own generated data. Unlike prior work, our approach establishes fully automated, multi-round, two-model mutual improvement process, enabling the navigator, the generator, and the dataset evolve concurrently through continuous model-driven feedback.
3

---

Published as a conference paper at ICLR 2025
Round,
Round
Fittered Nav Training Fi
Navigator Training D)
Püttrred Nav Training 2D
Destruction] wwokk. Dnstruntinn go-
poth
Instruction):
Navigutor N
Lmv-quaiity equalityylo Bain Lof
dondor Sample
Frein
Low-quality Data LDC
Generator
Navigator
Generator 6111
Iustrutiion
X
< pashy intrastion pastty testraccion >
Greedl Infer All Patks
Trais
Trais
inseruction
Fituredd Data
Fitered Gen Training Data
Generatoc Training Data
Piitered Gen Training Data F
dnstructinn
V
patty hutrussird) pasily Instraccinn
Filter
Iustructionj
X
Low-quality Data
FD?
Df
Sin Single-roudd Pipeline
(b) Filter Pipeline
Figure 2: (a) Our (instruction) generator refines low-quality data filtered in the previous round via sampling train the next-round navigator, and greedily generates high-confidence instructions as candidates to train next-round generator. Then the navigator filters the data for further use. (h) In filtering, the navigator re-runs instructions compute the path-path similarity score (nDTW & SPL). High-fidelity data is kept for training while low-quality data will be refined.
METHODOLOGY
3.1 BACKGROUND
VLN data typically consists of instruction-traaectnn pairs, where trajectory represents path in 3D environment, and the corresponding instruction guides the agent to follow it. This data can be used for training either instruction-to-aune manually annotating trajectories is costly, a common approach to first train a generator on limited human-annotated data and then use it to generate instructions for paths in unlabeled environments, which are subsequently used to augment navigator training.
navigators
generators. Since
While this method helps increase the data amount, it poses challenges regarding the quality and fidelity of the generated instructions. These challenges imply two essential problems how to generate better data and evaluate the data quality, which aim address in the following sections.
3.2 SRDF: SELF-REFINING DATA FLYWHEEL
In this section, we introduce the Self-Refinigg Data Flywheel (SRDF) to tackle the challenge of evaluating and improving VLN data to bootstrap its learning. Broadly, our system comprises navigator, N, and an instruction generator, G, shown in Figure (a). We use N to assist G by optimizing G using the data filtered by N, while G enhances N by refining the low-quality data. This iterative process is repeated multiple times, consistently enhancing both G and N's performance.
Main Components. Our SRDF requires the following resources at the beginning: (1) seed data DSeed: typically human-annotated, for training a base G and N (2) unlabeled trajectory pool, Drrajs usually collected from large-scale environment datasets for generating new training data.
Training Base Instruction Generator. Most previous works (Fried al., 2018; Tan al., 2019; Dou & Peng. 2022) train instruction generators from scratch, which limits their ability to generalize text effectively. Some recent approaches leverage pretrained vision-and- language models but neglect critical directional information during trajectory encoding (Li & Bansal, 2023; Kong et al., 2024), leading to instructions with inconsistent or incorrect directional cues.
We hypothesize that an effective instruction generator should be capable of understanding both multi-image inputs and interleaved image-text inputs. Multi Muulti-iaaee understanding is crucial for ac- curately encoding multi naltr-pa trajectories and interleaved image-text comprehension helps code directional images within the raw text space, enabling simple yet effective visual prompting of trajectories. To achieve this, utilize Mantis (Jiang et al., 2024), interleaved multi-image mul-
4

---

Published as a conference paper at ICLR 2025
timodal large language model (MLLM), which includes SigLIP vision encoder (Zhai et al., 2023), a multimodal projector, and a LLaMA-3 language model backbone (Dubey et al., 2024), pretrained on curated corpus of interleaved image-tex: pairs. We use our designed template (see Appendix 3, 13 for details) to convert the instruction-traaectryy pairs from D Seed into supervised fine-tuning (SFT) data, and then LoRA-finettne the language backhone using this converted data. This results in robust instruction generator G1, which serves as the foundation for our data generation process.
Generating Base Training Data. Once the base instruction generator, G1. is trained. it is used to generate two types of data from the unlabeled trajectories. Drraf: one for training the navigator and another for improving the round-2 instruction generator. For data training the instruction generator, we use greedy decoding to generate the most reliable instruction for each trajectory to form D For data used to train the navigator, we prioritize diversity by generating multiple instructions via random sampling to form DI The data generation step aims to provide sufficient training data for both improving the navigator and the instruction generator in subsequent iterations.
Training Base Navigator. We employ the DUET model (Chen et al., 2022b) as our navigator. The model is pre-trained using Dj and subsequently fine-tuned on D Seed- This results in a highly capable navigator N1, which will be used to evaluate the quality of the generated data.
Evaluating and Filtering the Generated Data. We propose using the trained navigator N1 to self-evaluate the generated data DN Intuitively, if the navigator successfully follows the generated instruction and navigates along the original path, it suggests that the instruction is well-aligned with the trajectory and the strong performance of this navigator (Could achieve near-human performance in previous work (Wang et al., 2023d)) further ensures its reliability. Such self-consistency trans- forms the challenging task of computing instruction-traeecno similarity into the simpler problem of computing trajecoryy similarity.
We run the navigator on the generated instructions and compute suajectory-trajecton similarity scores. We filter high-quality data FDS for the round-2 instruction generator training by select- ing instances from DG with SPL== (indicating the navigator perfectly follows the path). For DN (navigator training data), we filter with nDTW20.9 (indicating very close alignment between tra- jectories) get F FD 22 to use in the round-2 training, shown in Figure 2. This filtered-out high- quality data will be kept in the follow-up round, and we only re-generate low-quality data LD, filtered by nDTW=0.9. This filtering step ensures that only reliable data is used for subsequent iterations of training, reducing data noise and increasing alignment quality.
Iterative Self-Refining. The core of SRDF is its iterative loop between the instruction generator and the navigator, where each model contributes improving the other by providing data feedback. Specifically, at each iteration f, we first train the generator G, using the filtered high-quality data FD Then we use G, to generate new navigation-training data, N ND, for previously bad samples, LDN, and new generator training data, D 1 for Diraj, following the same details in generating base training data. Then we Combine N ND with previously filtered navigation data, FDN to form the complete dataset, D.V, for training the navigator. Note that the whole data size won't I change as we are only refining base samples. After training the navigator Ne using D.N. we use Ne to filter high-quality data subsets, resulting in FD filtered from DEg and FND; D.N from N NDN. Finally, we combine the newly filtered navigation data, FNDN, to form FD for the next- round training. This looping mechanism ensures that the data quality continually improves through a self-refining process. Each iteration benefits from the enhanced quality of both the instruction generator and the navigator, ultimately yielding high-quality dataset and highly capable models. To summarize, the pseudocodes of the SRDF are detailed in Appendix Alg. 1.
3.3 FINAL DATASET
We build our flywheel upon the R2R dataset (Anderson et al., 2018b) as Dseed, containing 14.039 human-annotated training data, along with the 178,270 and 2,890,267 unlabelled trajectories from MP3D (Chang et al., 2017) and HM3D (Wang et al., 2023d) environments, respectively, as Drraj- We run the flywheel for three rounds to create the final dataset, named SRDF-20M. This dataset con- sists of 19.5 million pre-training examples D&Y, with 6 instructions generated for each HM3D path
5

---

Published as a conference paper at ICLR 2025
Table 2: Statistics of training data on different VLN datasets.
Datasct
Instruction
#Env.
#lnstr.
#Vocab.
Instr. Length
R2R (Anderson et al., 2018b)
61
14,039
3,063
26.33
RxR-en (Ku et al., 2020)
60
26,464
7,249
102.13 13
REVERIE (Qi al., 2020)
Manually
60
10,466
1,140
18.64
CVDN (Thomason al., 2020)
Labelled
57
4,742
2,068
53.21
SOON (Zhu et al., 2021)
34
2,780
735
44.09
R4R (Zhu al., 2020)
59
233,532
3,004
52.25
Prevalent (Hao et al., 2020)
60
1,069,620
993
24.23
Marky (Wang et al., 2022b)
60
333,777
2,231
99.45
AutoVLN (Chen et al., 2022c)
Generated
900
217.703
1,696
20.52
ScaleVLN (Wang et al., 2023d)
1289
4,941,710
172
21.61
SRDF-20M (Ours)
860
20,417,874
10,363
24.05
(via top-3 sampling) and 12 instructions (6 top-3 and 6 top-5 sampling) for each MP3D path, and 0.9M greedy-decoded filtered data FDG (which will be used to fine-tune the pretrained navigator and the third-round generator here). Overall, we generated 20M instructions across 860 environ- ments for navigator training.
Comparison to Previous VLN Datasets. Table 2 presents detailed statisties of our dataset and previous VLN datasets. A common issue in existing augmentation datasets is the lack of vocabulary diversity. For instance, prior datasets like Prevalent (Hao et al., 2020), despite being significantly larger than R2R, possess only 1/3 R2R's 's vocabulary size, even with 76 times more instructions. This issue is even more pronounced in ScaleVLN (Wang al., 2023d), which suffers from highly limited vocabulary. In contrast, our dataset contains over 10,000 unique words and 20 million in- structions- three times the vocabulary diversity and 1,454 times the instruction count compared to the original R2R dataset - surpassing all previous human-labeled and synthetic VLN datasets in both vocabulary richness and instruction quantity. Importantly, this substantial increase in size and diversity is achieved without compromising quality, due to the robustness of our instruction gener- ator and the gigh-precision filtering process guided by our strong navigator. We also provide some visualization examples of our generated instructions in Appendix Figure 4 and 5.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and Evaluation Metries. We establish data flywheel and perform ablation studies primarily on the R2R dataset, while also assessing the transferability of our pre-trained navigator on range of downstream tasks. These include fine-grained VLN (R2R, RxR-English), VLN with dialog history (CVDN), high-level VLN (REVERIE, SOON), long-term VLN (R4R), and VLN in continuous environments (R2R-CE). Each dataset is split into training, val_seen, and val_unseen sets, with R2R, CVDN, REVERIE, SOON, and R2R-CE also containing test splits. The statistics for the training splits are summarized in Table 2 (manually-labele datasets), and further details can be found in the Appendix.
We evaluate our navigator using the standard path-fidelity metrics including Success Rate (SR), Suc- cess Rate Weighted by Path Length (SPL) (Anderson et al., 201 (0),,), Goal Progress (GP) (Thomason et al., 2020), Navigation Error (NE), normalized Dynamic Time Warping (nDTW) (Ilharco et al., 2019) and Success Rate Weighted by Dynamic Time Warping (sDTW) (Ilharco et al., 2019). We leave the details of these metrics to Appendix.
Although REVERIE and SOON include object grounding tasks, we focus evaluating navigation performance, as our generated dataset specifically designed to enhance navigation tasks. We use SPL the primary metric for R2R, REVERIE, SOON, and R2R-CE, nDTW for RxR-English, sDTW for R4R, and GP for CVDN.
For evaluating our instruction generator, assess the linguistic quality of the generated instructions using standard text similarity metrics such as BLEU (Papineni et al., 2002), Meteor (Banerjee & Lavie, 2005), and Rouge (Lin,, 2004), along with commonly used image captioning metries, includ-

---

Published as a conference paper at ICLR 2025
Table 3: Navigator and instruction generator results different rounds in R2R val unseen split.
Instruction Following
Instruction Generation
Method
NEL
OSR1
SR
SPL1
SPICE1
SPICE-Dt
Bleu-11 | Bleu-41
CIDErt
Meteort
Rouget
Buscline
2.37
85,5
78.6
69.9
21.8
28.0
72.5
27.7
42.2
23.6
49.0
Ours (round 1)
1.95
87.1
82.4
75.9
23.7
28.4
71.4
29.5
46.5
23.1
50.2
Ours (round 2)
1.81
88.5
83.6
77.3
25.2
29.9
73.7
31.0
50.7
24.2
51.3
Ours (round 3)
1.76
89.6
84.4
77.6
25.7
30.4
74.5
30.8
49.7
24.5
51.3
ing SPICE (Anderson et al., 2016) and CIDEr (Vedantam et al., 2015). Additionally, we consider SPICE-D (Zeng et al., 2023), directional-aware variant of SPICE tailored for VLN instruction evaluation. We adopt SPICE as our primary metric.
Implementation. We utilize Intern (Chen et al., 2024), powerful ViT with 6B parameters, as the visual encoder in our instruction-oolloiin experiments unless otherwise specified. For contin- uous environments, we employ CLIP-B/16 (Radford et al., 2021) the visual backbone.
In our data flywheel, we pre-train the DUET navigator from scratch for 45,000 iterations using a batch size of 1024 and a learning rate of 5 x 10 on 8 NVIDIA Tesla A100 GPUs. Multiple checkpoints are fine-tuned to select the best pre-training model. The selected model is then fine- tuned for 6K iterations with batch size of 16 on single GPU using only the R2R dataset. For the instruction generator, we fine-tune Maatis-8B-SiigL to the query and value layers in each transformer block. Initially, both the navigator and instruction generator are trained from random weights, while subsequent rounds use previous-round weights.
(Jiang et al., 2024) using LoRA, applying
After final-round navigator pre- pre-training we fine-tune the navigator for various downstream tasks, ing PanoGen (Li & Bansal, 2023) for MP3D-level environment augmentation. For augmentation, we round-3 filtered data F D for R2R, Scale N(REVERII) (Wang et al., 2023d) for REVERIE, and marky-English (Wang et al., 2022b) for RxR- Rag-Eng and no augmentation for other tasks. For R2R-CE, we fine-tune our pre-trained navigator on ETPNav (An al., 2023b) with a waypoint pre- dictor (Hong et al., 2022: An et al., 2022). Modules not pre-trained, such as the object grounding module in REVERIE and the depth-image embedding module in ETPNav, are randomly initialized and trained from scratch. We also use F DD to fine-tune the 3rd-round generator to build the final instruction generator. This process can also viewed as the fourth round generator training.
4.2 FLYWHEEL RUNNING RESULTS
Table 3 presents the results for both the instruction generator and navigator across all rounds. We follow SLNleVLN (Wang et al., 2023d) to train the navigator with CcaleVLN-HMSS and Prevalent data for augmentation, while we use Intern (Chen et al., 2024) features for fair comparison. The instruction generator for SI.IeVI.N is EnyDrop (Tan et al., 2019). In round 1, our new instruc- tion generator, fine-tuned R2R with LoRA using the pre-trained Mantis, significantly surpasses EnvDrop. This results in a substantial SR boost for the navigator from 78.6% to 82.4%.
At each round, use the naviga- tor to filter high-quality data F DDC to re-train the instruction generator, and use the improved instruc- tion generator to refine low-quality instructions LDD, to re-train the navigator. Despite the strong performance of the round baseline, the generator is further improved by incorporating navigator- filtered data in round 2. The high-quality data filtered by the navigator leads to +1.5 SPICE and +4.2 CIDEr. This trend continues in round 3, where SPICE reaches 25.7, while other metrics remain stable, demonstrating the crucial role of the navigator enhancing the instruction generator via data filtering. For the navigator, the data-refining process leads to continuous improvements in naviga- tion performance with +1.2% SR in round 2, and +0.8% SR in round 3, underscoring the importance of the generator data refinement in enhancing the navigator, as well as the effectiveness of iterative navigator- collaboration to build an effective self-refining flywheel.
Navigator and Instruction Generator Improve Each Other.
4.3 ANALYSIS
Comparison of Different Scoring Functions. We analyzed classical filtering methods will likely fail to capture complex path-inse aath-instructton similarity the previous discussion. In Table 4, we further verify the importance of our navigator-filterng compared to other filtering baselines. Using our
7

---

Published as a conference paper at ICLR 2025
Table 5: Results of instruction di- versity (finsttr. per path) in naviga- tor training in val unseen split.
Table 6: Results of different additional augmentation data in instruction generator training in val unseen split.
Additional Data
SPICET
Blcu-41
CIDErt
Rouge1
Aug Data
NE1
SR
SPLT
23.7
29.5
46.5
50.2
Prev #larr1
3.21
71.86
61.04
SealeVLN Data
23.5
29.0
46.1
49.8
OurS #hsir1
2.97
73.86
63.58
Prevalent Data
23.6
29.3
46.7
50.1
Prev 3
3.12
72.67
62.53
100-HM3D-EBN Ours
23.9
29.8
47.8
50.0
Ours Mhorm3
2.81
75.21
64.56
200-HM3D-Fmm Ours
24.2
30.1
49.1
50.3
Prev #lner=
3.07
72.84
63.12
400-HM3D-Fav Ours
24.6
30.3
48.9
50.7
Ours Ahotrzh
2.55
76.93
66.89
800-HM3D-Eav Ours
25.2
31,0
50.7
51.3
Ours 4buir=12
2.59
77.05
66.53
800-HM3D-EEN Ours (Sample)
24.8
30.3
48.3
51.0
round instruction generator, we produce instructions for 783 trajectories from the validation unseen split, rank them based on various scoring functions, and filter the top 400 to assess their similarity to GT instructions. 'No filter' refers to the average score of all 783 instructions, and intuitively, more similar instructions should yield higher NLP metric scores.
We compare our navigator's nDTw-score with CLIP-Sim (Radford al., 2021) and Mantis score (Jiang et al., 2024). CLIP-Sim is computed by averaging image-instruction similarities across all observations in the trajectory, while Mantis-score is produced by inputting the path as interleaved image-text pairs and asking Mantis to provide a similarity score. Results show that the Mantis score fails to improve over the baseline, likely due to the trajectories is too complex to understand for the MILM. CLIP-Sim provides slight SPICE improvement, possibly because it can capture some landmark-level similarities between the trajectory images and the instructions, but does not improve SPICE-D as it lacks directional understanding. In contrast, our Navigator-nDTW similarity filter- ing method successfully identifies high-quality instructions, leading to a substantial improvement, demonstrating our navigator-nDTW captures aath-instruction similarities much better than others.
Effeet of Instruction Diversity in Navigator Training We assess the impact of instruction diversity by training the navigator with different numbers of instructions per path (#instr 1, 3, 6, 12) on the MP3D environments, as
Table 4: Effects of Scoring Functions.
Filter
SPICET
SPICE-D
CIDErt
shown in Table 5. We use CLIP-B/16 as the visual fea- ture establish well-known baseline (Li & Bansal, 2023; Wang et al., 2023d) with the Prevalent dataset, while "Our" uses instructions generated by our round instruction gen- erator. Compared to Prevalent, our instructions consistently achieve stronger downstream results each #instr level, emphasizing the importance of instruc- tion quality. Our navigator also benefits significantly when increasing #instr from I to 3 and 3 to 6, while Prevalent's performance saturates after 3, demonstrating that scaling instruction diversity will be more effective when instruction quality is higher. Increasing #instr to 12 yields similar results to 6, suggesting that #instr is an optimal balance for training.
No Filter
23.7
28.4
46.5
CLIP Sim
24.4
28.7
45.8
Mantis Score
23.6
28.2
48.3
Navigator-nDTWW
25,4
30.6
53,9
Effeet of Additional Data in Instruction Generator Training. In Table 6, we examine the im- portance of high-quality data in instruction generator training, and the potential scalability of our pipelines by evaluating the influence of environment numbers. The round-1 generator without sup- plementary data serves as the baseline. Adding the VLN dataset does not improve performance, likely due to its low diversity, which limits generalization in text generation tasks.
When training with the Prevalent dataset, which has greater diversity, performance gains remain minimal, possibly because of the data noise, as it also shows low quality in Table 1. In contrast, adding data from ours with increased environments (we split FDG by environments) consistently enhances performance. This improvement is likely due to both the diversity and quality of our data, which are carefully maintained throughout the process, boosting the SPICE from 23.7 to 25.2.
We also experimented with training using sampled versus greedy-decoded instructions. Sampled instructions resulted in slightly lower performance, suggesting they may introduce noise, whereas greedy-decoded instructions, produced with higher confidence, are more reliable. These results show the strong extensibility of our pipeline, and the critical role of high-quality data, both in in- struction generator training.
8

---

Published as a conference paper at ICLR 2025
Table 7: Comparison of single-run performance on R2R and R2R-CE datasets. Best results are marked in bold blue and second best in bold.
Room-to-Room Dataset
Room-to-Room- Dutuset
Methods
Validation- Unseen
Test-Unseen
Vulidation-
Test-Unseen
NEI OSR
SR
SPL1
NEI
OSR
SR1 SPL
NEL SR1
SPL1
NEL SRT
SPL1
Human
1.61
90
Sey2Seq (Anderson et ul., 2018b)
7.81
28
21
7.85
27
20
Speaker Follower (Fried al., 2018)
6.62
45
36
6.62
35
28
EnvDrop (Tan 2019)
5.22
52
5.23
$1
BERT (Hong et al., 2021)
3,93
63
4.09
70
HAMT (Chen et al., 2021)
2.29
66
61
3.93
72
65
60
HOP (Qiao al., 2022)
3,80
64
57
3.83
64
59
DUET (Chen al., 2022b)
3,31
3.65
BEVBert (An et al., 2023a)
2.81
75
3.13
4.57
59
50
4.70
50
V (Wang et al., 20234)
2.09
81
2.27
86
80
4.80
55
5.11
50
GridMM (Wang al., 2023c)
2.83
75
64
3.13
73
62
5.11
49
41
5.64
46
39
ETPNav (An al., 2023h)
4.71
57
49
5.12
48
(Wang et al., 2024e)
4.42
NaviLLM (Zheng et al., 2024)
3.51
67
3.71
NavGPT-2 (Zhou et al., 2024a)
2.84
84
74
61
3.33
80
72
60
VER (Liu al., 2024)
2.80
76
65
2.74
76
66
MAGIC-L (Wang al., 2024b)
2,22
2.75
GOAT (Wang et al., 2024a)
2,40
3.04
SRDF (Ours)
1.62
90
86
79
1.82
89
85
78
4.12
65
57
4.35
64
56
Table 8: Comparison with previous methods various downstream tasks. indicates the RxR-en
results are reproduced using their officially released checkpoints. means pre-exploration methods. Best results are marked in bold blue and second best in bold.
RxR english
R4R
CVDN
REVERIE
SOON
Methods
Vil unseen nDTWt
Val unseen
Val
Test
Val
Test
Val unseen SR1
Test
sDTW1
GP1
GP1
SR1
SPL.1
SR1
SPL,
SPL
SR
SPL,
HAMTY (Chen et al., 2021)
56.4
63.0
44.6
31.8
5.13
5.58
33.0
30.2
30.4
26.7
MARVAL (Kamath al., 2022)
64.7
70.4
DUET (Chen et al., 2022h)
47.0
33.7
52.5
36.1
36.3
22.6
33.4
21.4
AutoVLN (Chen et al., 2022c)
55.9
40.9
55.2
38.9
41.0
30.7
40.4
27.9
RREx-Bot" (Sigundsson
2023)
61.0
58.8
65.9
62.0
49.2
48.6
47.5
47.1
BEVBent (An et al., 2023a)
66.7
69.6
51.8
36.4
52.8
36.4
ScaleVLN (Wang 2023d)
6.12
6.97
57.0
41.9
56.1
39.5
M (Zheng
2024)
6.16
7.90
44.6
36.6
43.5
34.5
38.3
29.2
35.0 26.2
VER (L.uu al., 2024)
47.0
33.0
56.0
39.7
56.8
38.8
PRETY et 2024)
70.9
MAGIC-L (Wang 2024b)
68.1
Cupilot (Qiao 2024)
GOAT (Wang et al., 2024a)
66.8
28.1
SRDF (Ours)
74.4
64.4
44.6
7.67
8.19
60.4
45.4
61.4
47.7
50.3
41.7
46.6
4.4 COMPARISION WITH STATE OF THE ARTS
R2R and R2R-CE. Table 7 compares agent performance on the R2R and R2R-CE datasets. The DUET navigator, trained on our high-quality datasets, improves SoTA SPL (ScaleVLN (Wang al., 2023d)) by 8% on the R2R test set, demonstrating our data's strong instraction-traaectorr alignment that allows effective decision-me makinn learning. Additionally, The gap between oracle success and success is reduced to 4%, compared to the previous best of 6%, highlighting that our data provides stronger clues for the agent to Iearn when to stop. Notably, our data-centric approach yields greater improvements compared most model design modifications, highlighting the importance of build- ing high-quality data to boost model performance. For R2R-CE, despite ETPNav using in-domain pre-training with Habitat-rendered RGBD images (Savva et al., 2019), our model, using no-rendered images from MP3D and Scale HM3D without depth-iman pre-training, achieves an absolute gain of +8% in SR and SPL, demonstrating the strong transferability of our pre-trained navigator.
REVERIE and SOON. For high-level navigation tasks, as shown in Table 8, our method achieves notable improvements the val unseen split for REVERIE and SOON, with +3.5% SPL over

---

Published as a conference paper at ICLR 2025
Table 9: Performance of different instruction generators on R2R. i means reproduced results. Best results are marked in bold blue, second best in bold, and third best in underlined.
R2R Validation Unseen
Methods
SPICET
SPICE-DT
Bleu-1"
Bleu-47
CIDEr1
Meteorf
Rouge
B'T-spuuker (Fried al., 2018)
18.9
25.1
68.2
26.3
37.9
21.7
48.0
LandmarkSelect (Agurwal et al., 2019)
19.7
15.9
23.1
35.7
Env Drop (Tin et al., 2019)
21.8
28.0
72.3
27.1
41.7
23.6
49.0
OCC (Wang el al., 2022a)
21.4
27.8
70.8
27.2
46,1
23.1
47.7
FOА (Dou Peng. 2022)
21.7
28.1
72.5
27.3
42.4
23.4
49.2
KEFA (Zeng et al., 2023)
23.4
29.3
73.8
28.3
42.7
24.4
50.3
LANA (Wang ct al., 2023a)
22.6
73.6
28.9
45.7
23.7
49.8
LANA+ (Wang et al., 2023b)
22.8
73.2
29.5
46.0
24.1
49.6
C-Instructor (Kong al., 2024)
21.2
26.6
23.9
47.3
BEVInsructor (Fan et al., 2024)
20.8
69.9
26.4
44.9
23.0
46.7
SRDF (Ours, round 2)
25.2
29.9
73.7
31,0
50.7
24.2
51.3
SRDF (Ours, round
25.7
30.4
74.5
30.8
49.7
24.5
51.3
SRDF (Ours, round 3 fine-tuned w/ F
26.2
30.9
75.3
31.1
49.2
25.0
51.4
SLaleVLN and +10.0% SPL over AutoVLN (Chen al., 2022c), respectively. These gains are pecially impressive given that both AutoVLN and ScaleVLN (Wang et al., 2023d) used in-domain pre-training data, while comes from the out-of-domait R2R-style dataset. This demonstrates that our high-quality data not only improves fine- mee-grained instruction instruction-following following but also enhances goal-finding, likely due diverse stopping guidance. Surprisingly, our model achieves results com- parable to the pre-exploration agent RREX- --tt (Sigurdsson et al., 2023) on REVERIE (-0.6% SR), and even surpasses
SOON (+1.4% SR), showing model's robust goal-finding ability.
CVDN. Our method also improves the previous best the val unseen set of CVDN by large margin (+1.51 meters) in Table 8, showing that our model can be generalized to different instruction styles, likely due to learning strong landmark alignment ability, which can be shared across tasks.
RxR-English and R4R. For long-horizon (and fine (ne-grained) VLN tasks including RxR-en and R4R, our navigator also shows strong results on the val unseen split, surpassing previous SoTAS by large margin shown in Table 8. It's worth noting that results R4R provide a very strong improvement of +16.6% SR. This shows our highly aligned data facilitates learning step-by-step instruction following even with very long trajectories.
R2R Instruction Generation. We also compare our instruction generator with previous SoTAs for path-to-instruccion generation task on R2R val unseen split in Table 9. Thanks to our navigator- filtered high-quality data, our round 2 generator has already beat the previous SoTA significantly, with + 1.8 SPICE and + 4.6 CIDEr. The stronger data generated in the third round results in an additional +0.5 SPICE improvement while keeping other scores comparable better. This sub- stantial strong performance can even be further enhanced by incorporating FD₃ for fine-tuning, leading to a +0.5 improvement in SPICE and better results across five additional metries, indicating the importance of high-quality data in instruction generator training.
5 CONCLUSION
In this work, we introduce a fully automatic self-refining data flywheel to construct substantially high-quality VLN dataset for augmentation. We propose iteratively refine the data with the navi- gator and instruction generator working in tandem-the navigator filtering high-quality data to train a better instruction generator and the instruction generator regenerating better instructions to train better navigator, ultimately producing both a strong navigator, instruction generator and a high- quality VLN dataset. We thoroughly analyzed the impact of each component the flywheel, demonstrating that our approach significantly surpasses state-of-the-art methods across multiple VLN benchmarks, covering various instruction styles (R2R, CVDN, REVERIE, SOON), trajectory lengths (R4R, RxR-English), and control spaces (R2R-CE), as well instruction generation task on R2R. Our self-refining flywheel provides novel, scalable solution to the data bottleneck in VLN, highlighting the crucial role of instruction quality and alignment in training embodied agents. This method has the potential to drive future advancements embodied navigation models and paves the way for exploring more sophisticated tasks that rely high-quality, dynamic, and scalable data.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Sanyam Agarwal, Devi Parikh, Dhruv Batra, Peter Anderson, and Stefan Lee. Visual landmark selection for generating grounded and interpretable navigation instructions. In CVPR Workshop, volume 3, pp. 7, 2019.
Dong An, Yuankai Qi, Yan Huang, Qi Wu, Liang Wang, and Tieniu Tan. Neighbor-view enhanced model for vision and language navigation. In Proceedings of the 29th ACM International Confer- ence on Multimedia, pp. 5101-5109, 2021.
Dong An, Zun Wang, Yangguang Li, Yi Wang. Yicong Hong, Yan Huang, Liang Wang, and Jing Shao. 1st place solutions for rxr-habitat lsion-and-language navigation competition (cvpr 2022).
arXiv preprint arXiv:2206. /610, 2022.
Dong An, Yuankai Qi. Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, and Jing Shao. Bevbert: Multimodal map pre-training for language-guided navigation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2737-2748, 2023a.
Dong An, Hanqing Wang, Wenguan Wang. Zun Wang, Yan Huang, Keji He, and Liang Wang. Etp- nav: Evolving topological planning for vision-language navigation in continuous environments. arXiv preprint 2304:2300.03344, 2023b.
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propo- sitional image caption evaluation. In Computer Vision-ECCV 2016: 14th European Confer- ence, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pp. 382-398. Springer, 2016.
Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, al. On evaluation of embodied navigation agents. arXiv preprint arXiv: 18XXi0:8007.00 1807.06777, 2018a.
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and- lan-auuugee navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Con- ference on Computer Vision and Patlern Recognition, pp. 3674-3683, 2018b.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds. Danny Hemandez, Tristan Hume, Scott Johnston, Shauna Kravee, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.05862. 2022a.
Yuntao Bai, Sauray Kadavath, Sandipan Kundu, Amanda Askell, John Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, E Perez, Jamie Kerr, Jared Mueller, Jeff Ladish, Landau, Kamal Ndousse, Kamilé Lukovsiūté, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noem'i Mercado, Nova Das- sarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton. Tom Conerly, Tom Henighan, Tristan Hume. Sam Bowman, Zac Hatfield- Batfield-Dodds, Benjamin Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom B. Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedhack. ArXiv, abs/2212.0807z. 2022b.
Satanjeev Banerjee and Alon Lavie. Meteor:
An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. pp. 65-72, 2005.
James Betker, Gabriel Goh, Li Jing, TimBrooks, Jianfeng Wang, Linjie Li, LongOuyang, Jun- tangZhuang, JoyceLee, YufeiGuo, WesamManassra, PrafullaDhariwal, CaseyChu, YunxinJiao, and Aditya Ramesh. Improving image generation with better captions.

---

Published as a conference paper at ICLR 2025
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In 2017 International Conference on 3D Vision (3DV), pp. 667-676. IEEE, 2017.
Howard Chen, Alane Suhr. Dipendra Misra, Noah Snavely, and Yoav Artzi. Touchdown: Natural language navigation and spatial reasoning in visual street environments. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1p.122 12538-12547, 2019.
Jinyu Chen, Chen Gao, Erli Meng, Qiong Zhang, and Si Liu. Reinforced structured state-evolution for lision-angguage navigation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15450-15459, 2022a.
Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for vision-and-language navigation. Advances in Neural Information Processing Sys- tems, 34:5834 5844,4877744 2021.
Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act local: Dual-scale graph transformer for Assion-and-lanuuee navigation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16537- 16547, 2022b.
Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Learning from unlabeled 3d environments for saioo-and-langusee navigation. In European Conference on Springer, 2022c.
Computer Vision, pp.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference Computer 2024.
Vision and Pattern Recognition, pp.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.0480) 2018
Zi-Yi Dou and Nanyun Peng. Foam: A follower-aware speaker model for vision-and-languagg navigation. arXiv preprint arXiv:2206.0422 2022.
Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, and Nanyun Peng. Reflection-reinfrreed self-training for language agents. arXiv preprini 40X1 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXhv:2407.21783, 2024.
Sheng Fan, Rui Liu, Wenguan Wang, and Yi Yang. Navigation instruction generation with bev perception and large language models. arXiv preprint arXiv:2407. /5087, 2024.
Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks. arXiv preprint arXiv:2309. /7425, 2023.
Yunhao Fang, Ligeng Zhu, Yao Lu, Yan Wang, Pavlo Molchanov, Jang Hyun Cho. Marco Pavone, Song Han, and Hongxu Yin. Vila2: Vila augmented vila. arXiv preprint arXiv:2407.17453 2024.
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency. Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for rision-and-le and-landnnnn language navigation. In Advances in Neural Information Processing Sys- tems, pp. 3314-3325, 2018
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Published as a conference paper at ICLR 2025
Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. Airbert: In-domain pretraining for Assion-and-language navigation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pp. 1634-1643, 2021.
Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. Language models can teach them- selves to program better. arXiv preprint arXiv:2207.14502 2022.
Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning generic agent for vision-and-land language navigation via pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13137-13146, 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Keji He, Yan Huang, Qi Wu, Jianhua Yang, Dong An, Shuanglin Sima, and Liang Wang. Landmark- rxr: Solving vision-and-lane lan--aa navigation with fine-graind alignment supervision. In Confer- ence on Neural Information Processing Systems, volume 34, pp. 653-663, 2021.
Yicong Hong, Cristian Rodriguez, Yuankai Qi, Qi Wu, and Stephen Gould. Language and visual en- tity relationship graph for agent navigation. Advances in Neural Information Processing Systems, 33, 2020.
Yicong Hong. Qi Wu, Yuankai Qi, Cristian Patriiuuz-Opazze and Stephen Gould. recurrent Aision-and-languaal bert for navigation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR). pp. (643-1653,
June 2021.
Yicong Hong, Zun Wang. Qi Wu, and Stephen Gould. Bridging the gap between learning in dis- crete and continuous environments for vision-and-langee language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15439-15449, 2022.
Gabriel Ilharco, Vihan Jain, Alexander Ku, Eugene Ie, and Jason Baldridge. General eval- uation for instruction conditioned navigation using dynamic time warping. arXiv preprint 1rXiv:1907.05446. 2019.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max W.F. Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv2405.0148., 2024.
Aishwarya Kamath, Peter Anderson. Su Wang. Jing Yu Koh, Alexander Ku, Austin Waters, Yinfei Yang, Jason Baldridge, and Zarana Parekh. A new path: Scaling vision-and-language navigation with synthetic instructions and imitation learning. arXiv preprint arXiv:22/00031112 2022.
Hyounghun Kim, Jialu Li, and Mohit Bansal. NDH-full: Learning and evaluating navigational agents full-length dialogue. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6432-6442, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 18.11863/v1// 1ttp:////22...mmmmmaaa... mannpp-miin.11..
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CV International Conference on Computer Vision, pp. 4015-4026, 2023.
Xianghao Kong, Jinyu Chen, Wenguan Wang, Hang Su, Xiaolin Hu, Yi Yang. and Si Liu. Con- trollable navigation instruction generation with chain of thought prompting. arXiv preprint arXiv.2407.07433, 2024.
Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav- vav-graph: lnd-language navigation in continuous environments. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16, pp- 104-120. Springer, 2020.
13

---

Published as a conference paper at ICLR 2025
Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-actoss-romm Multilingual asioo-and-languae navigation with dense spatiotemporal grounding. Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4392 4412, 2020.
Harrison Lee. Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard. Colton Bishop, Vic- tor Carbune, and Abhinav Rastogi. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. International Conference on Machine Learning. 2023.
Nicholas Lee, Thanakul Wattanawong, Schoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipali, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. Llm2llm: Boosting Iims with novel iterative data enhancement. arXiv preprint arXiv:2403. 2024.
Jeffrey Li, Alex Fang, Georgios Smymis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Datacomp-Im: In search of the next generation of training sets for language models. arXiv preprint arXiv:2406.11794 2024a.
Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for Assio-and-langee navigation. arXiv preprint 2rX1v:2305.19195, 2023.
Jialu Li, Hao Tan, and Mohit Bansal. Envedit: Environment ediiron for iision-and-languaae naviga- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15407-15417, 2022.
Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06255) 2023a.
Xiangyang Li, Zihan Wang, Jiahao Yang, Yaowei Wang, and Shuqiang Jiang. Kerm: Knowledge enhanced reasoning for siioo-anddlanguage navigation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 2583 2592, 2023b.
Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, et al. What if we recaption billions web images with Ilama-3? arXiv preprint arXiv:2406.0844 08478, 2024b.
Chin-Yews Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pp. 74-81, 2004.
Kunyang Lin, Peihao Chen, Diwei Huang. Thomas H Li, Mingkui Tan, and Chuang Gan. Learning vision-anddlanguage navigation from youtube videos. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 9317-8326, 2023.
Chong Liu, Fengda Zhu, Xiaojun Chang, Xiaodan I Liang, Zongyuan Ge, and Yi-Dong Shen. Vision- language navigation with random environmental mixup. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pp. 1644-1654, 2021a.
Chong Liu, Fengda Zhu, Xiaojun Chang. Xiaodan Liang, Zongyuan Ge, and Yi-Dong Shen. Vision- language navigation with random environmental mixup. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pp. 1644-1654, 2021b.
Rui Liu, Xiaohan Wang, Wenguan Wang, and Yi Yang. Bird's seeye-vew graph for vision- language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10968-10980, 2023.
Rui Liu, Wenguan Wang, and Yi Yang. Volumetric environment representation for rision-language navigation. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recog- nition, pp. 16317-16328, 2024.
Renjie Lu, Jingke Meng, and Wei-Shi Zheng. Pret: Planning with directed fidelity trajectory for vision and language navigation. arXiv preprint arXiv:2407.1 1487, 2024.
14

---

Published as a conference paper at ICLR 2025
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2024.
Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Laser: Learning to adaptively select reward models with multi-armed bandits. arXiv preprint arXiv:2410.01133 2024a.
Khanh Nguyen and Hal Daumé III. Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning. arXiv preprint vrXiv:/90.00187) 2019.
Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Sewoong Oh, and Ludwig Schmidt. Improving multimodal datasets with image captioning. Advances in Neural Information Processing Systems, 36, 2024b.
Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang. and Xi Victoria Lin. Lever: Learning to verify language-tt tinguage-to-oode to-code generation with execution. In International Conference on Machine Learning, pp. 26106 2612222118. PMLR, 2023.
Long Ouyang. Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke F. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.021555 2022.
Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan- Spandana Gella, Robinson Piramuthu, Gokhan Tur, and Dilek Hakkani-Tur. Teach: Task-driven embodied agents that chat. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 2017-2025 2022.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Vaidehi Patil, Leonardo Ribeiro, Mengwen Liu, Mohit Bansal, and Markus Dreyer. Refinesumm: Self-refining milm for generating multimodal summarization dataset. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 13773-13786, 2024.
Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang, Jing Xu, Maryam Fazel-Zarandi, Mohit Bansal, Sainbayar Sukhbaatar, Jason Weston, and Jane Yu. Self-consistency preference opti- mization. arXiv preprint arXiv:2411.04109 2024.
Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environ- ments. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9982-9991, 2020.
Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu. Hop: History-and- order aware pre-training for vision-ande -ad---aa-- navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15418-15427, 2022.
Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu. Hop+: History- enhanced and order-aware pre-training for lision-add-language navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023a.
Yanyuan Qiao, Yuankai Qi, Zheng Yu, Jing Liu, and Qi Wu. March in chat: Interactive prompting for remote embodied referring expression. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 15758-15767. 2023b.
Yanyuan Qiao, Qianyi Liu, Jiajun Liu, Jing Liu, and Qi Wu. Lim as copilot for coarse coarse-grained vision- and-language navigation. In European Conference on Computer Vision, pp. 459.476, 2024.
15

---

Published as a conference paper at ICLR 2025
Wenda Qin, Teruhisa Misu, and Derry Wijaya. Explore the potential performance of vision-and- language navigation model: a snapshot ensemble method. arXiv preprint arXiv:2111.14267, 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-888888 8763. PMLR, 2021.
Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg. John M Turner, Eric Undersander. Wojeiech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3dy: 1000 large-scale 3d environments for embodied ai. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M Lopez. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3234- 3243, 2016.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp-
9339-9347, 2019.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10740-10749, 2020.
Gunnar A Sigurdsson, Jesse Thomason, Gaurav S Sukhatme, and Robinson Piramuthu. Rrex-bot: Remote referring expressions with a bag of tricks. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5203-5210. IEEE, 2023.
Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Her- rmann, Sjoerd van Steenkiste, Ranjay Krishna, al. Dreamsyne: Aligning text-to-image gen- eration with image understanding feedback. In Synthetic Data for Computer Vision Workshop@ CVPR 2024, 2023.
Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from trans- formers. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing, 2019.
Hao Tan, Licheng Yu, and Mohit Bansal. Learning to navigate unseen environments: Back transla- tion with environmental dropout. In Proceedings of NAACL-HLT, pp. 2610-2621, 2019.
Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Man- sooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. Large language models for data annotation and synthesis: A survey. arXiv preprint arXiv:2402. / 13446, 2024.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog navi- gation. In Conference on Robot Learning. pp. 406, 2020.
Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. Stablerep: Synthetic images from text-to-image models make strong visual representation learners. Advances in Neural Information Processing Systems, 36, 2024.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566-4575, 2015.
Hanqing Wang, Wenguan Wang, Wei Liang, Caiming Xiong, and Jianbing Shen. Structured scene memory for vision- language navigation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 8464, 2021.
16

---

Published as a conference paper at ICLR 2025
Hanqing Wang, Wei Liang, Jianbing Shen, Luc Van Gool, and Wenguan Wang. Counterfactual cycle-consistent learning for instruction following and generation vision lainunla navigation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 15471-15481, 2022a.
Liuyi Wang, Zongtao He, Ronghao Dang, Mengjiao Shen, Chengju Liu, and Qijun Chen. Vision- and-language navigation via causal learning. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024a.
Liuyi Wang. Zongtao He, Mengjia Shen, Jingwei Yang, Chengju Liu, and Qijun Chen. Magic: Meta-ability guided interactive chain-of-distilation for effective-and-efficient vision-and- language navigation. arXiv preprint arXiv:2406. 17960, 2024b.
Su Wang, Ceslee Montgomery, Jordi Orbay, Vighnesh Birodkar, Aleksandra Faust, Izzeddin Gur, Natasha Jaques, Austin Waters, Jason Baldridge, and Peter Anderson. Less is more: Generating grounded navigation instructions from landmarks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15428-15438, 2022b.
Xiaohan Wang, Wenguan Wang, Jiayi Shao, and Yi Yang. Lana: A language-capable navigator for instruction following and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19048-19053 2023a.
Xiaohan Wang. Wenguan Wang, Jiayi Shao, and Yi Yang. Learning to follow and generate in- structions for language-capable navigation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2023b.
Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang. and Lei Zhang. Reinforeed cross-mode matching and sulf--sue imita- tion learning for vision-language navigation. In Pruceedings of the IEEE Conference Computer Vision and Puttern Recognition. pp. 6629-6638, 2019.
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, et al. Internvide02: Scaling video foundation models for multimodal video understanding. arXiv preprint arXiv:2403. 15377, 2024c.
Zhaoyang Wang, Weilei He, Zhiyuan Liang. Xuchao Zhang, Chetan Bansal, Ying Wei. Weitong Zhang, and Huaxiu Yao. Cream: Consistency regularized self-rewarding language models. arXiv preprint arxiv:2410. 22..22 2024d.
Zihan Wang, Xiangyang Li, Jiahao Yang, Yeqi Liu, and Shuqiang Jiang. Gridmm: Grid memory map for uision-and-langure navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15625-15636. 2023c.
Zihan Wang, Xiangyang Li, Jiahao Yang, Yeqi Liu, Junjie Hu, Ming Jiang, and Shuqiang Jiang. Lookahead exploration with neural radiance representation for continuous vision-language navi- gation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13753-13762, 2024e.
Zun Wang, Jialu Li, Yicong Hong. Yi Wang, Qi Wu, Mohit Bansal, Stephen Gould, Hao Tan, and Yu Qiao. Scaling data generation in vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12009-12020, October 2023d.
Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Meta-rewarding language models: Self-improving alignment with llm-as-a-metta-judee arXiv preprint arXiv:2407. (tt1:2407.119944, 2024.
Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world perception for embodied agents. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9068-9079, 2018.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint 2304 /4/78, 2023.
17

---

Published as a conference paper at ICLR 2025
Shoubin Yu. Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image- langgal model for video localization and question answering. Advances in Neural Information Processing Systems, 36, 2024.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models. arXiv preprini urXiv:2401.10022 2024.
Haitian Zeng, Xiaohan Wang, Wenguan Wang, and Yi Yang. Kefa: A knowledge enhanced and fine- grained aligned speaker for navigation instruction generation. arXiv preprint arXiv:2307. 13368, 2023.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 11975-11988 2023.
Yue Zhang and Parisa Kordjamshidi. Explicit object relation alignment for vision and language navigation. In Proceedings of the 60th Annual Meeting of the Association for Computational Student Research Workshop, pp. 322-331, 2022a.
Linguistics:
Yue Zhang and Parisa Kordjamshidi. Lovis: Learning orientation and visual signals for vision and language navigation. In Proceedings of the 29th International Conference Computational Linguistics, pp. 5745-5754, 2022b.
Yue Zhang and Parisa Kordjamshidi. VIn-trans: Translator for the vision and language navigation agent. arXiv preprint arXiv:2302.092:0. 2023.
Yue Zhang, Quan Guo, and Parisa Kordjamshidi. Navhint: Vision and language navigation agent with a hint generator. arXiv preprint arXiv:2402.02559 2024a.
Yue Zhang, Ziqiao Ma, Jialu Li, Yanyuan Qiao, Zun Wang, Joyce Chai, Qi Wu, Mohit Bansal, and Parisa Kordjamshidi. Vision-and- langugge navigation today and tomorrow: survey in the era of foundation models. arXiv preprint arXiv:2407.07033 2024b.
Yusheng Zhao, Jinyu Chen, Chen Gao, Wenguan Wang. Lirong Yang, Haibing Ren, Huaxia Xia, and Si Liu. Target-driven structured transformer planner for vision-language navigation. In Pro- ceedings of the 30th ACM International Conference on Multimedia, pp. 4194-4203, 2022.
Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. Towards learning a generalist model for embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 13624-13634, 2024.
Gengze Zhou, Yicong Hong. Zun Wang. Xin Eric Wang, and Qi Wu. Navgpt-2: Unleashing naviga- tional reasoning capability for large vision-language models. In European Conference Com- puter Vision, pp. 260-278, 2024a.
Gengze Zhou, Yicong Hong, Zun Wang, Chongyang Zhao, Mohit Bansal, and Qi Wu. Same: Learning generic aanuugge-uideet visual navigation with state-adaptive mixture of experts. arXiv preprint arXiv:24/2.0555: 2024b.
Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405.1 2tt.:2405..462., 2024c.
Fengda Zhu, Xiwen Liang, Yi Zhu, Qizhi Yu, Xiaojun Chang, and Xiaodan Liang. Soon: Sce- nario oriented object navigation with graph-based exploration. In Proceedings of the JEFF/CVF Conference on Computer Vision and Pattern Recognition. pp. 12689-12699, 2021.
Wang Zhu, Hexiang Hu, Jiacheng Chen, Zhiwei Deng, Vihan Jain, Eugene Ie, and Fei Sha. Baby- walk: Going farther in vision-and-languagee navigation by taking baby steps. arXiv preprint arXiv:2005.044 04625, 2020.
18

---

Published as a conference paper at ICLR 2025
A APPENDIX
We first present additional implementation details of our experiments in Section A.1, including specifics of the generator, navigator model, and training procedures. Section A.2 illustrates the SRDF pipeline with pseudo code, while Section A.4 provides further details and experiments re- garding trajectory-encodin
template design. Details of datasets and evaluation metrics provided in Section A.5, with more comprehensive downstream results shown in Section A.6. Fi- nally, Section A.7 presents detailed examples of our generated instructions in comparison with other baselines.
A.1 ADDITIONAL. IMPLEMENTATION DETAILS
Navigator. We use DUET model (Chen et al., 2022b) as navigator, which integrates global and local information through dual-scale graph transformer architecture. This architecture processes both high-level scene representations well detailed local features simultaneously, enhancing the model's 's capability to interpret language instructions in complex visual contexts. By constructing a topological map in the meanwhile, DUET extends the navigational action space from the current viewpoint to all navigable directions encountered, thus improving planning and error correction. The model employs attention mechanisms to balance global scene contexts with local observations, significantly improving navigation accuracy towards targets based on natural language commands.
Generator. We use Mantis (Jiang et al., 2024) as our base model for instruction generation. Mantis comprises SigLIP vision encoder (Zhai et al., 2023), multimodal projector, and LLaMA-3- 8B-Instruct language model backbone (Dubey et al.. 2024). It is first pre-trained on multimodal projector data using LLaVa pre-training, followed by fine-tuning on the multi-image interleaved Mantis-Instree dataset.For For our task, we initialize the instruction- -unded model, Mantis-8B-sighhe Ilama3, and apply updates only the added LoRA layers in the language backbone.
Training Details. The navigator is trained using similar objectives as in (Wang al., 2023d), including Masked Language Modeling and Single Action Prediction. We initialize our model with LXMERT (Tan Bansal, 2019) for the first and second rounds, and with our round-2 pre-trained model for the third round. During generator training, all parameters except the injected LoRA layers are frozen, and only the LoRA layers are fine-tuned. For navigator training, we initialize the instruction generator with Mantis in the first and second rounds and use the round-2 2 trained generator directly for round 3. Additionally, for downstream task supervisions, we encourage the model to go back to the viewpoint on the GT path yielding best nDTW to the current progress for R4R and RxR- English as their trajectories are not poortest-path. while we use shortest-pai path supervision as teacher action for other tasks.
A.2 PSEUDO CODE OF SRDF
Alg. provides detailed pseudo code for our Self-Refining Data Flywheel (SRDF), illustrating the process described in Section 3.2. Specifically, the pipeline starts with training an initial instruction generator using seed data. The generator creates training data for the navigator, which is then used to train a base navigator. The navigator, in turn, filters high-quality data to further improve both itself and the generator subsequent rounds. This iterative process continues for T' iterations, refining data quality and improving model robustness with each cycle.
A.3 SFT DATA TEMPLATE
We use the template shown in Figure 3 to construct our SFT data for fine-tuning the instruction generator, which encodes trajectory into an interleave image-action sequence. At each viewpoint, we include key views, such as the view when arriving at the viewpoint and the view when leaving it. For each view, also append the corresponding action in raw text after the image tokens, creating a multi--mage interleaved format to effectively encode the trajectories. The output is the corresponding instruction of the input trajectory.
19

---

Published as a conference paper at ICLR 2025
[Model Input]
This is navigation trajectory consists of several image sequences:
(Viewpoint I:
Action: left (135.71 degree) and up (0.00 degree)),
(Viewpoint 1: Imager<image>, Action: forward),
(Viewpoint 2: Image:<image>, Action: left (33.14 14 degree) and up (0.00 degree)),
(Viewpoint 5: Image:<image>, Action: forward),
(Viewpoint 6: Image:< Action: stop). Could you give me its corresponding navigation instruction in details?
[Model Output]
Start by exiting the bedroom into the hallway. Turn right and proceed down the hall until you reach the first door on your left. Enter this room, which is the library. Once inside, walk towards the large bookshelf directly across from the doorway. Stop there and face towards the window on the right side of the room.
Figure 3: Interleaved trajectory prompt and expected response for training our instruction generator. The LLM's response is the corresponding instruction of this path.
A.4 ADDTIONAL EXPERIMENTS
Fourth-Round Generator- Training. Our method demonstrates the potential for continued per- formance improvement with additional refinement rounds. To illustrate this, we conducted a fourth- round generator training experiment (which is equivalent to the "round 3 w/ FD. results in Ta- ble Following similar procedure, we generated FD trained the generator using this data, and observed consistent performance gains over the third round. Results in Table 10 show new state- of-the-art (SoTA) SPICE score of 26.2, marking an improvement of +2.8 over the previous SoTA (as shown in Table 9). These results highlight the scalability and effectiveness of our approach, indicating that with sufficient computational resources and time, further rounds of improvement can be sustained.
Effect of Different Scoring Metries in Generator Training. We conducted additional experi- ments to demonstrate that classical methods for language model self-improvement face limitations in the VLN context without reliable feedback from the navigator. Specifically, we evaluated two ap- proaches: (1) self-score, a self-rewarding method where the language model scores its own outputs, and (2) CLIP-score, which uses an external tool (CLIP) to provide similarity scores. In these exper-
Algorithm Pipeline of Self- Refi-Reeinnng Data Flywheel (SRDF)
Require: Seed data Seed (Human- annotated),, Unlabelled trajectories Drraj, Total iterations I'.
1: Train base instruction generator G, with D
2: Use Gi generate nav-training data DN and gen-training data D for Drraj-
3: / DN is generated via random sampling while Dî via greedy decoding
4: Train base navigator N₁ with D
5: Use Ni to filter high-quality subsets FDf from Dr and FDN,, from DÑ
6: for each iteration (1 < & T) do
7: /* Note: Seed data Dseed is used in training stages of both Ge and N₂ but omitted for simplicity
8: Train generator G. with F FD
9: Use G2 to generate nav-training data NDN for LDN and gen-training data D for Drvej-
10:
1e NDN is generated via random sampling while DG via greedy decoding
11:
Combine NDN and FDN to form D.N.
12:
Train navigator N₂ with D.."
13:
Use N, to filter high-quality subsets FDG,, from D and FNDN from NDN.
14:
Combine EN DN and F N. to form F F
15:
end for
20

---

Published as a conference paper at ICLR 2025
iments, conducted during round 1, instructions were scored and the top 300K instructions filtered using either (1) or (2) were used to train the instruction generator round 2. The results in Table 11 showed that neither self-scores nor CLIP scores yielded significant improvement baseline. In contrast, our navigator- filterinn method using nDTW demonstrated substantial gains, highlighting the challenges of providing effective feedback and emphasizing the effectiveness of our approach.
the round
Table 10: Generator results in the additional fourth round.
Method
SPICET
Bleu-1f
Bleu-4f
CIDErt
Meteorf
Rouget
Baseline
21.8
72.5
27.7
42.2
23.6
49.0
Ours (round
23.7
71.4
29.5
46.5
23.1
50.2
Ours (round 2)
25.2
73.7
31.0
50.7
24.2
51.3
Ours (round 3)
25.7
74.5
30.8
49.7
24.5
51.3
Ours (round 4)
26.2
75.3
31.1
49.2
25.0
51.4
Table 11: Second-round generator performance with different scorers.
round scorer
SPICE1
Bleu- 1
Bleu-41
CIDErt
Meteort
Rouget
round
23.7
71.4
29.5
46.5
23.1
50.2
round
Self-score
23.6
71.3
29.4
46.4
23.5
50.3
round Cl.P--core
23.9
70.6
30.0
48.6
23.1
50.4
round
navigator-DDTW
25.2
73.7
31.0
50.7
24.2
51.3
Table 12: Two-round generator performance with mPLUG-Owl
Method
SPICET
Bleu-11
Bleu-47
CIDErT
Meteort
Rouget
Baseline
21.8
72.5
27.7
42,2
23.6
49.0
mPLUG-owl (round 1)
22.7
70.3
28.0
44.4
23.0
49.1
mPLUG-owl (round 2)
72.2
29.1
45.2
23.7
50.0
Effeet Different MLLM To demonstrate that our model-boosting process is not reliant on Mantis (Jiang et al., 2024), we conducted additional experiments with weaker multimodal large language model (MLLM), mPLUG-Owl-7B (Ye et al., 2023). Using the same methodology, we applied the flywheel process and completed the first two rounds of generator training. In Table 12, we observed a significant improvement in the round generator's performance when trained on its data filtered by the navigator, highlighting the navigator's critical role in enhancing the generator. Given that the reciprocal improvement the navigator by the generator has been validated in prior work using the Speaker-Follower framework, these results strongly support our assertion that the model-boosting process is robust and not tied to a specific MLLM.
Effeet of Different Encoding Formats. In our previous discussion, we hypothesized that the interleaved image-text understanding ability is crucial for training instructor generator based on pre-trained MLLMs. Some prior works (Li & Bansal, 2023; Kong et al.. 2024) use only image information to build an image sequence for fine-tuning the VLM, but we argue that this approach loses important directional clues. Additionally, if action information is added without an interleaved format (i.e., an action sequence followed by an image sequence), the model may struggle to reason effectively between the two sequences.
We verify this hypothesis in Table 13, using two baselines: (I) an image-only sequence (Figure without action descriptions) and (2) action sequence followed by an image sequence (Figure with actions listed after the image sequence). Our results show that using only an image sequence leads much lower performance, primarily because the model finds it difficult infer actions between key frames. For the image-sequence + action-sequence format, it still underperforms com- pared to our interleaved image-text sequence template, likely due to challenges in reasoning across
21

---

Published as a conference paper at ICLR 2025
Table 13: Effect of different trajectory-encodiing templates.
R2R Validation Unseen
Templates
SPICE!
SPICE-D"
Bleu-11
Bleu-4
CIDErt
Meteort
Rouge1
Image Seq.
21.8
24.4
68.7
25.7
21.9
48.3
Image Seq. + Action Seq.
22.9
27.1
70.4
29.1
46.5
22.9
50.2
Interleave Image-Action Seq.
23.7
28.4
71.4
29.5
46.5
23.1
50.2
separate sequences. In contrast, the Interleaved Image-Action Sequence performs best, demonstrat- ing its effectiveness in trajectory encoding, which is used in our experiments.
A.5
DETAILS OF DATASETS AND EVALUATION METRICS
Datasets.
We conduct our downstream experiments on 7 datasets listed below.
Consists of 22k human-annotated navigational instructions, each describing a trajectory that traverses multiple rooms in MP3D. On average, an instruction contains 32 words, and each ground- -ruth path is formed by seven nodes with total length of 10 meters.
R2R:
REVERIE: Inherits the trajectories in R2R but provides high-level instructions that describe target object. The task an agent first to find the object and then localize it in the observation. Provides instructions describing target rooms and objects. The average length of instruc- tions is 47 words. SOON does not provide object bounding boxes and requires the agent to predict object center locations in the panorama. We use automatic object detector to obtain candidate object boxes. The length of expert paths ranges from 2 to 21 steps, with an average of 9.5 steps. CVDN: Provides dialogues between navigator who tries to find target by asking for guid- ance and an oracle with a privileged view of the best next step. The agent must find the way by interpreting the dialogue history.
SOON:
R2R-CE: Transfers the discrete trajectories in R2R to continuous 3D scans rendered by the Habi- tat simulator, where an agent can freely travel in the open space and interact with obstacles. The dataset contains 16k instruction ttaaecctrrt pairs after removing non non-transferable paths.
RxR: An extension of R2R that addresses shortest path biases and includes more object references. We use the English segment of RxR, which consists of 42,002 instructions, averaging 108 words per instruction.
R4R: Created by concatenating adjacent paths in the -oom-to-Room dataset. The ground-truth path is not the shortest path, encouraging the agent to follow the instructions to reach the target rather than exploit environment bias to navigate the shortest route.
Detailed Evaluation Metrics, (1) Success Rate (SR), which measures whether the agent stops within meters the target; (2) Success Rate Weighted by Path Length (SPL), which penalizes inefficient, longer paths; (3) Goal Progress (GP), which calculates the agent's 's progress toward the target; (4) Navigation Error (NE), which the average distance between the agent's final position and the target in meters; (5) normalized Dynamic Time Warping (nDTW), which measures step- wise alignment between the ground truth and the agent-predicted path; (6) Success Rate Weighted by Dynamic Time Warping (sDTW); (7) Coverage weighted by Length Score (CLS); (8) Remote Grounding Success (RGS), the proportion of successfully exccuted instructions; and (9) RGS Pe- nalized by Path Length (RGSPL). Metries (7) to (9) are used only in the detailed results provided in the appendix, with (8) and (9) specifically evaluating object grounding in REVERIE and SOON.
A.6 DETAILED RESULTS
In this section, we present detailed results for our downstream navigators, across multiple datasets: R2R (Table 18), (4R(eable 14), RxR-English (Table 15), REVERIE (Table 17), and SOON datasets (Table 16). Specifically, on the R2R dataset, our model not only demonstrates improved generaliz- ability to unseen environments but also achieves higher success rates and SPL in seen environments, surpassing previous state-of-the- method achieved more than a 10% improvement in both SR and sDTW the R4R dataset. Besides, on the RxR English dataset, our approach significantly enhances SPL and sDTW addition to SR and nDW, elevating the state-of-the-art SPI. to 69.2% and sDTW to 66.3%. Lastly, REVERIE
(SoTA) approaches by over 3% in SR and 4% in SPL. Our
22

---

Published as a conference paper at ICLR 2025
and SOON datasets, navigator not only enhances the agents' navigation performance signifi- cantly but also substantially improves their grounding capabilities, improving RGSPL by 0.4% on REVERIE test set, and 1.7% on SOON test set compared with previous SoTA approaches. Notably, our navigator relies solely on pretraining with Masked Language Modeling (MLM), and Single Ac- R2R datasets and the augmentation dataset collected with
tion Prediction (SAP) objectives data flywheel. This is in contrast to other approaches that additionally employ an Object Grounding (OG) objective. The superior performance indicates the strong instruction-trajectorr alignment in our high-quality data, which is crucial for effectively learning object grounding from scratch during fine-tuning.
Table 14: Comparison of single-run performance on R4R dataset.
Validation Unseen
Methods
NEL
SRT
CLSSt
nDTWT sDTW
HAMT (Chen al., 2021)
6.09
44.6
57.7
50.3
31.8
LOVIS (Zhang & Kordjamshidi, 2022b)
6.07
45.0
45.0
43.0
23.0
VL.. Trans (Zhang & Kordjamshidi, 2023)
5.87
46.0
45.0
25.0
PanoGen (Li & Bansul, 2023)
6.02
47.8
BSG (Liu al., 2023)
6.12 47.0
59.0
53.0
34.0
NavHint (Zhang et al., 2024a)
6.04
46.0
45.0
25.0
VER (Liu 2024)
6.10
47.0
61.0
54.0
33.0
SRDF (Ours)
4.21
64.4
61.0
56.5
44.6
Table 15: Comparison of single-run performance on RxR English dataset. f indicates the results are reproduced using their officially released checkpoints.
Validation Seen
Validation Unseen
Methods
NEG
SR1
SPL,
sDTW
nDTW]
NE
SR1
SPL
sDTW:
nDTW
Landmark-RxR (He ct al., 2021)
64.1
53.1
63.4
40.1
27.5
40.3
HAMTf (Chen ul., 2021)
59.4
50.9
65.3
56.5
48.3
63.1
MARVAL (Kamath ul., 2022)
3.31
74.0
64.7
PRETI (Lu ct al., 2024)
2.68
77.1
71.8
67.1
77.5
3,36
71.0
63.6
63.5
70.9
MAGIC-L (Wang
2024b)
81.3
77.5
69.2
76.6
72.9
65.4
58.7
68.1
BEVBertf (An al., 2023a)
61.1
GOAT (Wang al., 2024a)
74.1
68.1
61.4
71.0
68.2
61.7
56.6
67.1
SRDF (Ours)
1.95
82.9
77.7
75.6
83.4
2.61
78.8
69.2
66.3
74.4
Table 16: Comparison of single-run performance SOON dataset.
Validation Unseen
Test
Methods
Navigation SR
Grounding RGSPL
Navigation
Grounding
OSR1
SPL1
OSR1
SR1
SPL1
RGSPL1
DUET (Chen al., 2022h)
50.9
36.3
22.6
3.8
43.0
33.4
21.4
4.2
AutoVIN (Chen al., 2022c)
53.2
41.0
30.7
4.1
48.7
40.4
27.8
5.1
et al., 20236)
38.1
23.2
NaviLLM (Zheng et al., 2024)
38.3
29.2
35.0
26.3
GOAT (Wang et al., 2024a)
54.7
40.4
28.1
5.9
50.6
40.5
25.2
6.1
SRDF (Ours)
59.6
50.3
41.7
5.1
51.6
46.6
37.9
8.4
23

---

Published as a conference paper at ICLR 2025
Table 17: Comparison of single-run performance on REVERIE datasets.
Validation Unseen
Test Unseen
Methods
Navigation
Grounding
Navigation
Grounding
OSR
SR1
SPL
RGST
RGSPL
OSR
SR1
SPL1
RGS1
RGSPL1
HAMT (Chen al., 2021)
36.8
33.0
30.2
18.9
17.3
33.4
30.4
26.7
14,9
13.1
DUET (Chen ul., 20226)
47.0
33.7
23.0
BEVBert (An et al., 2023a)
56.4
51.8
36.4
34.7
24.4
57.3
52.8
36.4
32.1
22.1
AutoVLN (Chen et al., 2022c)
62.1
55.9
40.9
36,6
26.8
KERM (Li et al., 2023b)
55.2
50.4
35.4
34.5
24.5
52.4
39.2
32.4
23.6
BSG (Liu et al., 2023)
52.1
35.6
24.2
33.2
22.3
ScaleVLN (Wang et ul., 2023d)
63.9
57.0
41.8
62.7
56.1
39.5
MIC (Qian et al., 2023b)
62.4
57.0
43.6
37.5
28.7
62.4
55.7
42.0
35.3
26.2
NaviLLMM (Zheng et al., 2024)
53.7
44.6
43.5
VER (1.iu et al., 2024)
61.1
56.0
39.7
33.7
23.7
62.2
56.8
38.8
33.9
23.2
GOAT (Wang et al., 2024a)
53.4
36.7
38.4
26.1
57.7
40.5
38.3
26.7
WI.N-Colipo (Qian al., 2024)
62.6
57.4
43.6
38.9
29.8
63.3
57.8
42.3
36.6
26.6
SRDF (Ours)
72.2
60.4
45.4
37.3
27.8
66.2
61.4
47.7
35.6
27.1
Table 18: Comparison of single-run performance on R2R dataset.
Validation Seen
Validation Unseen
Test Unseen
Methods
OSR
SRI
SPL
NEL
OSR
SR
SPL
NEI
OSR1
SRT
SPL
Human
1.61
90
86
76
Sey2Seq (Anderson 201
6.01
53
39
7,81
28
21
7.85
27
20
Speuker Follower (Fried al. 2018)
3.36
74
66
6.62
45
36
6.62
35
RCM (Wang et ul., 2019)
3.53
75
67
6.09
50
43
6.12
50
43
38
EnvDrop (Tan et al., 2019)
3.99
62
59
5.22
52
48
5.23
59
51
47
PREVALENT (Hao al., 2020)
3.67
69
65
4.71
58
53
5.30
61
54
EntityGraph (Hong al., 2020)
3,47
67
65
4.73
S7
4.75
55
NvEM (An et al., 2021)
3,44
69
4.27
60
55
4.37
66
58
SSM (Wang ct al., 2021)
3.10
80
71
62
4.32
73
62
45
4.57
70
61
46
AirHert (Guhur et al., 2021)
2.68
75
70
4,10
62
56
4.13
62
57
BERT (Hong
2021)
2.90
72
68
3.93
63
4.09
70
63
HAMT (Chen al., 2021)
76
72
66
3.93
72
EnvMix (Liu al., 2021a)
2.48
75
72
3.89
64
3.87
72
65
SnapEnsemble (Qin et al., 2021)
3.63
67
60
3.82
65
60
EXOR (Zhang & Kordjamshidi, 2022a)
60
58
52
49
49
46
SEvol (Chen 2022a)
3,56
67
3.99
62
4.13
62
MARVAL (Kumath et al., 2022)
2.99
73
69
4.06
65
4.18
67
62
LOVIS (Zhang & Kordjamshidi, 2022b)
2,40
77
72
3.71
65
$9
4.07
63
58
HOP+ (Qian et al., 2023a)
2.33
78
73
3.49
67
61
3.71
66
60
TD-STP (Zhao al., 2022)
2.34
83
77
73
3.22
76
70
63
3.73
72
67
DUET (Chen et al., 20226)
2.28
86
79
73
3.31
72
3.65
76
ScaleVLN (Wang et al., 2023d)
2.12
87
81
75
2.09
88
81
70
2.27
86
80
BEVBert (An et al., 2023a)
2.17
88
81
74
2.81
84
75
64
3.13
81
73
62
VLNN-Trans (Zhang & Kondjamshidi, 2023)
2.45
77
72
3.34
69
63
3.94
66
60
Lily (Lin 2023)
2.90
74
62
3.44
72
NaviLLM (Zheng ul., 2024)
3.51
67
3.71
68
HavHint (Zhang et al., 2024a)
3.23
69
4.00
65
60
NavGPT-2 (Zhou et al., 2024a)
2.84
83
74
63
2.84
84
74
61
3.33
80
72
60
SAME (Zhou al., 2024b)
2.73
76
66
3.03
74
64
VER (Liu 2024)
2,80
76
2.74
76
MAGIC-L (Wang et al., 2024b)
1.73
84
79
2.22
86
79
70
2.75
82
77
GOAT (Wang et al., 2024a)
1.79
89
84
79
2.40
85
78
3.04
80
75
SRDF (Ours)
1,54
91
87
83
1.62
90
86
79
1.82
89
85
78
24

---

Published as a conference paper at ICLR 2025
A.7 QUALITATIVE CASE STUDY OF GENERATED INSTRUCTIONS
In Figure 4 and 5, we visualized some examples of our generated instructions, and compare them with Prevalent (Hao et al., 2020) and ScaleVLN (Wang al., 2023d) baselines. All the example trajectories in Figure 4, and Figure 5 (a), (b) are collected using recovered environment images from VLN (Wang et al., 2023d), while Figure (c), (d) are from MP3D environments.
Recognition Ability. Figure 4 demonstrates the strong image-text un- derstanding capability of our instruction generator. Specifically, our generator can recognize rare objects, such dentist chair/room grandfather clock, thanks to interleaved image-action trajectory rajeciory-encoding design. This design preserves the original abilities of the pretrained MLLM while effectively encoding trajectories to generate instructions with rich and accurate landmarks. In contrast, the baseline instruction generator fails to capture these rare concepts due to its from-scratch training paradigm. Instead, it only generates some general landmarks with weak clues.
Ability, Figures 4 (c) and (d) illustrate that our instruction generator can describe key objects along the path with greater detail. For instance, while the baseline mentions the bar and the painting successfully, our generator provides more specifics, such as brown leather bar stool and large painting on the wall. Such detailed descriptions are crucial for helping the navigator learn richer visual cues. Additionally, in Figure 4 (d), our generator performs slight spatial reasoning between objects, resulting more precise stopping guidance the blue and white throw pillows on the right side of the couch.
Detailed
Generalization to Outdoor Environments. In Figure 5 (a), (b), we demonstrate the ability of our generator to produce some useful instructions for outdoor environments, even though the model is training using instructions from indoor environments. For instance, in (a), our generated instruction identifies the glass doors leading outside, which is more distinct the ScaleVLN's table still a gen- eral landmark without strong viewpoint-specine clue. In (b), the generator successfully identifies outdoor landmarks including the cur and the buches, while the baseline only knows walkway.
OCR Ability. Surprisingly, instruction generator demonstrates interesting OCR capabilities, as shown in Figures (c) and (d). In example (c), the generator successfully identifies the words cape and plug the wall, while in example (d), it even identifies full Lettenee-eef start to redefine how work gets done. This OCR ability is likely inherited from the pre-trained MLLM, and our fine-tuning approach effectively retains this capability, resulting in highly detailed and accurate guidance in the generated instructions.
Idiomatic Expressions. Our generator sometimes uses idiomatic expressions in its instructions. An example is shown in Figure 5 (c), Sample 2, where the generator says, go past the desk then stop at the end of the rope. The phrase at the end of the rope usually means that someone has reached the limit of their patience endurance. In this context, however, it refers reaching the farthest point that the navigator can proceed-likes the wall. This ability adds diversity to the instructing style, making the generated instructions more varied and engaging.
25

---

Published as a conference paper at ICLR 2025
SealeVL. N-RavDrop:
ScnleVLN-Eooe E.e.------
lurn around and walk through the doorway. left walk past table and chairs, turn right and walk into the room, wait near the sink.
walk through the kitchen into dining room walk past the dining table and front of the stairs.
walk through the hallway and into the dining room. walk past the table and chairs and into the living room. walk past the rocking chair stop grandfather
walk through the door on the left and inso the dentist room. walk past deneal chair. and stop front window the closer.
(a) Rare room identification
(b) Rare landmark identification
SeuleVL-NEEvDDrou NroonDrrpp: walk the dining room table and chairs and turn right. walk past dining room chairs turn right stop couch.
ScaleVLN LLVV-L---vDDipp
past the dining table and bar and turn left. stop in front the painting
left. walk past
Ours:
Ours:
down hallway past dining table coctinue into the living room and walk past the couch. stop front of blue and white throw pillows the right side of the couch.
walk past the dining sable and chairs, walk past kitchen island. tum and walk past the brown leather har stools, turn and stop front of large painting wall.
(c) Object relationships
(d) Detailed descriptions
Figure 4: Visualization of generated instructions.
26

---

Published as a conference paper at ICLR 2025
ScaleVLN. Eny Drop:
ScaleVLN-Eav-Dee walk down walloway turn right. walk down the walkway and stop at the end of the walkoay..
down the hallway and right. walk the stop in front of the table.
Ours:
hallway, go through the hallway, walk past dining room table and chairs, walk past the couch and in front the dooes leading outsick.
and walk
Ours:
post tum busites.
by
(a) Ourdoor environment identification
Ourdoor environment generalization
ND
Prevalent:
tam around, go foeward, right when you get into the center room and stop.
Prevalent:
(Sample
office right go down luall stop
go down hallway and past the desk, turn right the end of hallway and stop fromt of the wall with the word "cape" on it and "plug" wall.
doorway straight in ballroom.
Ours (Sample
walk around the sable and chairs, and turn into hallway to the right with sign the door that read "Let's redefined how week gets done". down the hullway you see the on that read "O1".
go siniight, Ihen lurn and go around pillar, then turn right and straight. passed the desk then the end of the rope.
(c) OCR ability Idiomatic expressions
(d) OCR ability
Figure 5: Visualization of generated instructions.
27


Reviews:
[{"review_id": "WtdTq6kz5U", "reviewer": "Reviewer_FkSF", "summary": "This paper introduces the Self-Refining Data Flywheel pipeline within the context of Vision-and-Language Navigation (VLN). This pipeline iteratively improves itself by generating navigation instructions and refining navigation paths through repeated cycles of self-improvement. The goal is to enhance VLN performance by creating a feedback loop that iteratively optimizes both instruction generation and navigation, facilitating autonomous performance improvements over time.", "questions": "1. How does performance change when using a different baseline from R2R? Could the choice of base data impact the outcomes, and if so, how?\n\n2. SPL and nDTW are used as metrics for improvement. Could you explain why these were chosen specifically, and how would the results differ with alternative metrics?\n\n3. Considering research outside of VLN, what novel aspects of this framework make it distinct within the scope of self-improving models?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Simplicity and Effectiveness: The concept of iteratively refining instructions and navigation steps is straightforward, yet the experimental results validate its utility in enhancing VLN task performance.\n\n- Clarity and Structure: The paper is well-written and clear, with a structured presentation that helps readers understand the pipeline. Additionally, it effectively contextualizes the work within existing VLN research, making its contributions easy to follow.\n\n- SoTA performance: While I'm not an expert in the fields of VLN, it seems the paper achieved the good performance compared to the prior SoTA>", "weaknesses": "- Terminology and Scope: The term \"Flywheel\" might be overly strong for what is essentially a straightforward iterative self-improvement process, and it could be perceived as a bit exaggerated.\n\n- Limited Dataset Scope: The method is only tested on the R2R as a base dataset, and the lack of the discussion on its broader applicability.\n\n- Limited Analysis of Performance Boundaries: While performance improvements are reported in up to three rounds (Table 3), little discussion exists on why the improvement plateaus and whether there are theoretical or practical limits to the approach’s performance gains.\n\n- Positioning within Broader Self-Improving Frameworks: The novelty of this approach needs to be clarified when considering related research beyond VLN. The \"Self-Improving Language Models\" section lacks a strong position on how this framework differentiates itself within the wider landscape of self-improving models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SLu24q6E4W", "reviewer": "Reviewer_H4hM", "summary": "This paper proposes a framework in which the instruction model and navigation model bootstrap each other through an iterative process. It begins with the pre-trained instruction model labeling sampled trajectories, which are then used to fine-tune the navigator. The updated navigator subsequently samples additional trajectories to further refine the instruction model, and the cycle continues. The authors conduct experiments on various VLN benchmarks, including R2R, CVDN, REVERIE, SOON, R4R, and R2R-CE. The experiment results show that the proposed method is able to", "questions": "1. Why is the performance of the speaker-follower model much lower than the results presented in the original paper?\n2. Which dataset and split is used in the experiment reported in Table 3?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well written and easy to follow.\n2. The authors have conducted experiments on diverse downstream VLN tasks to validate the effectiveness of the method.", "weaknesses": "1. My major concern is the novelty of this paper. The proposed method highly resembles the Speaker Follower model (Fried et al., 2018), and the self-training paradigm has been widely studied in previous studies [1], [2]. \n2. It’s unclear whether the performance improvements primarily result from the bootstrapping process between the instruction model and navigation model, or from the advantage of initializing the instructor with a pre-trained MLLM (Mantis) that has been trained on a large-scale dataset. From Table 3 we can see that the model improves by a visible margin after round 1, but subsequent rounds yield only marginal gains. Prior studies [3] also indicate that generally, the self-training paradigm does not improve model's capabilities on out of distribution cases by much. It will be interesting to see how model's generalizability improved without relying on the pre-trained MLLM.\n3. Some of the notations are not defined clearly. For example, what is D^N_1 and what is the data source of it? It is also unclear to me what is the exact dataset use to produce D_seed.\n4. The author mentioned that they use DUET as the baseline navigator, could the author explain the performance discrepancy between the baseline in Table 3 (SR=78) and the DUET model in Table 7 (SR=72)?\n\nIn short, I think my major concern is the novelty of the paper. Also, I'm not sure whether the model is benefited by the powerful pre-trained MLLM or the bootstrapping process itself. \n\n[1] Reinforced Self-Training (ReST) for Language Modeling, Gulcehre et al.\n\n[2] Re-ReST: Reflection-Reinforced Self-Training for Language Agents, Dou et al.\n\n[3] The Entropy Enigma: Success and Failure of Entropy Minimization, Press et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hZXdXDohzi", "reviewer": "Reviewer_sXMN", "summary": "This paper proposes an iterative instruction-generation and navigation pipeline to increase the quality of machine-labeled data for Vision Language Navigation tasks.  By using a navigation model to identify which generated instructions are achievable, the navigation model can improve the instruction generator's data.  This in turn creates a better instruction generator, which can expand the set of data used to train the navigation model.  The paper shows that by repeating this filtering/training cycle, the resulting navigator can perform comparably (or better than) humans on some benchmarks, and also that the instructions generated by the model are better than other baselines.", "questions": "- The paper mentions several times that the navigator model being trained is already very good.  What happens in the case where the navigator is not as good?  Do you expect several more rounds of the flywheel to be required?\n- Why does the paper stop at 3 rounds for evaluation?  I expect diminishing returns, but is that a reflection of the lack of high-quality data remaining for the model to label, or something else?\n- When discussing label diversity, the paper mentions that the vocabulary size of existing datasets is small.  Can you say more about the effect of vocabulary size?  If, for example, the dataset were relabeled by asking a LLM to rephrase the instructions, would that be sufficient to get better performance?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The presented method generates impressive empirical results in the navigation task and in the instruction generation task as well.  The evaluations show the importance of carefully selecting (or generating) data to train VLN models.  Although this is not a new observation generally, the thorough and careful application of filtering and data labeling to the VLN problem is a solid contribution with state of the art results.\n- Comparison to related work is thorough, both in evaluations and in the text.\n- Presentation is clear and highlights strong results without grand claims.  The diagrams are helpful in understanding the approach.  Detailed description of implementation.", "weaknesses": "I found few substantial weaknesses in this paper.  Here are some minor suggestions:\n- Radar plot in Fig 1. is difficult to read, the Human results are difficult to distinguish from ScaleVLN.\n- Line 234 says \"navigator $G_1$\" which I believe should be $N_1$", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DMnSzbWzfF", "reviewer": "Reviewer_JHxr", "summary": "This paper presents a self-refining data flywheel (SRDF) approach to generate the high quality the instruction data for vision-language navigation task. It proposes a generator and navigator iteratively tuned to train each other. First, a navigator and a generator are trained on the base corpus, then the navigator serves as a filter for the generator to collect high-quality synthetic data, then refine the navigator with synthetic data iteratively. Empirical experiments on several standard vision-language navigation datasets (R2R, CVDN, R4R, REVERIE etc.) show very promising results.", "questions": "Some questions -\n\n1. For the table 5, the number of instructions, did you use multiple instructions encoders (with one path trajectory) as input? for example, N=3, do you treat (3 instructions, one trajectory) as one instance? or (3 instructions, one trajectory) as three (1 instructions, one trajectory) instance? This is quite different.  If your input is multiple instruction encoders (or shared encoder for multiple instructions as input), this is quite different with one instruction input. \n\nCould you clarify the exact input and encoding process for this part?\n\n2. And also for the main table 7, how many instructions per path trajectory in your setting?\n\n3. Did you try this method on a different environment? rather than on R2R environments. Nearly all the tasks of this work are built on the R2R environments.  One question here is the generalization of the proposed method, if you could try out-of-domain environments, for example, ALFRED task?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. An interesting paper to show that iteratively tuning the generator and navigator works in the vision-language navigation, and also demonstrates the state-of-the-art results on several tasks. The empirical results are impressive.", "weaknesses": "1. It looks this work (proposing iteratively tuning generator and navigator), is a upgraded version of Speaker-Follower model, which you can think it is only the first iteration of the model. Could you discuss in more details about the relevance and difference with Speaker-Follower in the paper?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)