Curriculum Learning for Vision-and-Language Navigation
Authors: Jiwen Zhang
zhongyu wei
Jianqing Fan
Jiajie Peng
Venue: NeurIPS 2021 Poster (2021)
Keywords: Vision-and-language navigation
curriculum learning
multimodal learning

Abstract: Vision-and-Language Navigation (VLN) is a task where an agent navigates in an embodied indoor environment under human instructions. Previous works ignore the distribution of sample difficulty and we argue that this potentially degrade their agent performance. To tackle this issue, we propose a novel curriculum- based training paradigm for VLN tasks that can balance human prior knowledge and agent learning progress about training samples. We develop the principle of curriculum design and re-arrange the benchmark Room-to-Room (R2R) dataset to make it suitable for curriculum training. Experiments show that our method is model-agnostic and can significantly improve the performance, the generalizability, and the training efficiency of current state-of-the-art navigation agents without increasing model complexity.

Curriculum Learning for Vision-and-Language Navigation
Jiwen Zhang', Zhongyu Weil.2+, Jianqing Fan Jiajie Peng?
School of Data Science, Fudan University, China
2 Research Institute of Intelligent and Complex Systems, Fudan University, China
Department of Operations Research and Financial Engineering, Princeton University, USA edu, jiajiepengênypu.cm edu.
{ jjzhang16, zywei wdw.deeedddduuu.e.odoommmom en,
Abstract
Vision-and and-Lanuuage Navigation (VLN) is task where an agent navigates in an embodied indoor environment under human instructions. Previous works ignore the distribution of sample difficulty and we argue that this potentially degrade tackle this issue, we propose novel curriculum- based training paradigm for VI.N tasks that can balance human prior knowledge and agent learning progress about training samples. We develop the principle of curriculum design and re-arrange the benchmark to make it suitable for curriculum training. Experiments show that our method is model-agnostic and can significantly improve the performance, the generalizability, and the training efficiency of current state-of-the th--ttrrrtttt navigation agents without increasing model complexity.
their agent performance.
Coum-to-Roon (R2R) dataset
Introduction
Vision-and -oae--nnd-Lannnn Lan-LLaguage (VLN) navigation task proposed recently by (Anderson et al., 2018) is step towards building smart robots. It requires the agent to perceive the environment, understand human language instructions and finally unify the multi-modal information to make actions. Many state- of-the-art methods have been proposed. Some of them focus on the alignment between visual and textual inputs by improving model structure (Ma et al., 2019) or proposing novel auxiliary losses (Zhu et al., 2020) whereas others put their attention on the data augmentation (Fried et al., 201 8; Tan al., 2019; Hong et al., 2020; Ku al., 2020). Large scale pre-training has also been employed to better generalize the downstream VLN tasks (Li et al., 2019; Majumdar et al., 2020; Hao et al., 2020; Huo et al., 2021).
Despite great progress previous works have made, very few of them care about how much the agent learns from the dataset, i.e. is the agent good student? computer vision, (Hlynsson al., 2019) tries to answer this question by measuring the data efficiency performance as function of training set size of deep learning methods. In vision-an and-aand-aggee language navigation, (Huang et al., 2019) develops discriminator that can filter the low quality instruction-par -attioon--aphhhha pairs to boost the learning efficiency. In this work, we focus on another aspect: can a VLN agent be further educated without model structure change and data modification?
We observe that many works neglect the internal distribution of the sample difficulty. For example, navigation task within a single room should be considered easier than one that needs to travel two or more rooms. Current training methods simply flush in the data and do distinguish difficulty levels among the training samples. As shown in Figure (a)), navigation agents trained by such learning process do not perform well on those "casy" tasks even on the previously seen environments. We
"Corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

3.0
KZ7
2
other
2002
2.5
2992
2.0
1.5
1.0
25
50
100
125
150
175
200
total
epoch
(a) Success rate during training (without curriculum).
(b) Ratio of different errors.
Figure 1: (a) The success rate of Soff-Monitoring agent(Ma et al., 2019) validation seen split of R2R dataset during its training process. Bars with different colors represent the success rate of different sample groups. Numbers in the legend represent the difficulty level of samples, i.e. the navigation task should be completed within how many rooms. (b) The ratio of different types of the first error an agent made during navigation. There three error types where the agent fails to make the right choice. Type "in" represents the correct next viewpoint is within the current room, type "cross" represents the correct next viewpoint is in another room, and type "others" represents that the agent predicts the ground truth trajectory but it fails to stop at the right place.
monitor the first error agent makes during the navigation and demonstrate the ratio of different types of these errors in Figure (b). We find that when the navigation agent fails, about 50% errors caused by the agent wrongly predicts the next in-room direction. The ratio of this kind of error decreases as the navigation task spans more rooms, but still remains a relatively high level. These phenomenon indicates that the navigation agent is limited by its ability to navigate inside room and cross two rooms.
The poor performance of agents those easy cases inspires to borrow the idea from curriculum learning (Bengio et al., 2009) and propose a curriculum- of curriculum learning is to start small, learn easier aspects of the task and then gradually increase the difficulty level. The definition of "difficulty" is therefore worthwhile. We hypothesise that the difficulty of navigation task is closely related the rooms the agent needs to pass by towards the destination. On top of this, we introduce the curriculum design for VLN and re-arrange the benchmark Room-to-Room (R2R) dataset (Anderson al., 2018) to make suitable for curriculum learning (Section 3). We incorporate human prior knowledge about training samples into the training process of navigation agent self-paced curriculum learning (SPCL) (Jiang et al., 2015), We adapt the traditional SPCL algorithm to efficiently train deep learning models (Section 4). Experiments show that our method can consistently improve both the navigation performance and the training efficiency of navigation agents (Section 5).
training paradigm. The basic idea
In summary, main contributions are:
We propose to explicitly incorporate human prior knowledge about training samples into navigation
agent training process.
We design the curriculum for VLN and put forward the training paradigm of a navigation agent by curriculum learning without increasing the complexity the model.
We empirically validate that the role of curriculum learning is to smooth loss landscape and hence find better local optima.
2
Related Work
Tision-and-Langee Language Navigation. Vision- Assion-and-langee lnn-lan Navigation (VLN) is a task where an agent navigates to a goal location in photo-realistic 3D environment under human instructions. (Anderson et al., 2018) formalized this task, proposed the benchmark Roon-to-Room (R2R) dataset and set up an attention-base sequence-to- sequencee baseline model. Other related VLN datasets include Touchdown dataset (Chen et al., 2019), which is the first large-scale outdoor VLN dataset,

---

and CVDN dataset (Thomason et al., 2019) that emphasizes on the robot-human dialogues during navigation.
Embodied navigation tasks suffer from a limited size of training data, which results in several different research focuses. For data augmentation, (Fried al., 2018) developed speaker-follower model where the speaker model is usually used as a tool for instruction generation, and (Tan et al., 2019) proposed to apply a effective environmental dropout layer mimic unseen environments during training. For better generalization, (Wang et al., 2018) 8) introduced a hybrid model that integrated the model-based and model-free reinforeement learning whereas (Wang et al., 2019) proposed a Reinforced Moossl (RCM) agent and a self-supervised imitation learning method to explore the unseen environment. Besides, (Li et al., 2019; Huo et al., 2021; Hao et al., 2020) applied pre-training skills towards better instruction understanding and generalization.
Another resolution the paucity of training data is rather straightforward: generating more training data and annotations. For example, (Hong et al., 2020) split the instructions in R2R dataset into sub-instructions and annotated the corresponding sub-paths. (Ku et al., 2020) proposed Room Across Room (RxR) dataset, a multilingual VLN dataset with dense spatiotemporal grounding. Furthermore, like (Wang et al., 2020) one can use multitask learning framework to grasp common knowledge from other homologous datasets so as to better the agent's 's performance on the current task.
These works are worthwhile but we observe that most of them ignore how much the model can learn from the dataset. In this paper, we focus to improve training methods to better exploit data. A better training paradigm can benefit both previous and future works VLN tasks.
Curriculum Learning. Curriculum Learning (Bengio et al., 2009) learning paradigm that mimic the learning principle underlying the cognitive process of humans and animals, where a model is learned by gradually including from easy to complex samples in training. (Jiang et al., 2015) bridged the gap between curriculum learning and self-paced learning by proposing novel self- self-paced curriculum learning method. (Graves et al., 2017) proposed an automated curriculum learning method that can automatically select the curriculum syllabus. Later, (Matiisen et al., 2017) introduced teacher-student framework for automatic curriculum learning, where the student tries to learn a complex task and the teacher automatically chooses the task for student train on.
Curriculum learning has been applied to several natural language processing tasks, such as question answering (Sachan and Xing, 2016; Liu et al., 2018), machine translation (Platanios et al., 2019) and so on. An empirical work done by (Hacohen and Weinshall, 2019) conclude that curriculum learning can effectively modify the optimization landscape and under mild conditions it does not change the corresponding global minimum of the optimization function. These works encourage us to apply the curriculum learning into the VLN tasks as it does not need any modification towards the navigation agent and is capable of improving the agent performance. To the best of our knowledge, this is the first work that introduces curriculum learning purely as a training paradigm for VLN tasks.
Babywalk (Zhu et al., 2020) is another work that applies the curriculum learning idea. It aims to enhance the transfer ability of navigation agents. Therefore, Babywalk uses dynamic programming to decompose the long instructions into shorter ones and then applies these sub-instructions for curriculum-bart based reinforcement learning. It does not pay much attention to how an agent performs within a dataset. Babywalk is a carefully designed VLN agent with complex model structure whereas our method is a easy, extendable model-agnostic training method. Our method will change neither the difficulty of training data nor the model complexity.
Curriculum Design for VLN
We observe that different samples in the dataset have different navigation difficulties. Our intuition is that, for human beings it is easy to find an object or place within small range. After exploration, it is natural for us to exploit the knowledge about the environment and complete the harder cross-room tasks. Hence, we hypothese that the number of rooms path could cover (namely room length, R(p)) dominates the difficulty level of a navigation task. We therefore propose to re-arrange the R2R dataset based on R(p). Two examples with various R(p) shown in Figure 2.
We firstly investigate the distributions of rooms that can be covered by a R2R path. Approximately 17% of the paths only pass through less than two rooms whereas about 15% of the paths span 5 rooms
3

---

Instruction: bed, Soot the middlle dircled
through
Step
Bedrovm
Dourway
Roarway
Figure 2: Navigation samples in validation seen split with different room length. Self-Monitoring agent (Ma al., 2019) succeed to complete the in-room navigation task with trajectory length 6.03 (above) and the cross-room navigation task with trajectory length 10.17m (below).
Table Basic statistics of CLR2R dataset. Start coverage defined as the ratio of rooms where the ground truth paths start to all room in the train scans. Room coverage is the ratio of rooms covered by the ground truth paths to all room in the train scans.
Average
Average Instruction
Start Coverage (%)
Room Coverage (%)
Trajectory
Train Set
Paths
Instructions
Length (m)
Length (words)
R2R
4675
14039
9.91
29.41
89.5
96.1
CLR2R
Round
345
1037
8.86
24.31
12.0
12.8
-Round2 2
471
1415
8.25
24.78
20.2
38.3
RRound 1632
4897
9.19
28.08
59.5
79.1
Round 1530
4593
10.42
31.16
59.5
83.0
Round 697
2097
12.12
34.33
33.5
62.2
or more. We then propose to split the train set of R2R dataset into 5 mutually exclusive subsets and each subset contains paths whose R(p) are constrained. We believe that the learning from simple to difficult very similar to play arcade games, so the subsets are named round to 5 according its difficulty. To be specific, round (i = 1, 2, 3, 4) subset contains samples whose ground truth path covers rooms. Round subset contains the left samples.
The basic statistics of the re-arranged R2R dataset is summarized Table 1. As the construction of such dataset incorporate human prior knowledge and such dataset will be futher used for curriculum learning, we call Room-to-Room for Curriculum Learning (CLR2R) dataset.
It should be noted that the definition of "difficulty" is open and task-specific, we throw possibility here and encourage other attempts. Similar standards are easy to find in other datasets, for example, the number of dialogue rounds and the richness of dialogue phenomenon difficulty indicators on CVDN set of R2R dataset unchanged so to have comparable experimental results.
both be taken as et al., 2019). Finally, we leave validation and test
4 Methods
There are multiple settings of curriculum learning. When series of tasks is presented, we can apply automated currieulum learning (Graves et al., 2017) where agents are trained by tasks adapted their capacities. While for single task setting, appropriate if the task dataset has samples with various difficulty levels. Our observations better fit the latter mode, hence adopt self-paced curriculum learning as the training method.
curriculum learning (Jiang et al., 2015) is more
4.1 General Setup
Self-paced curriculum learning (SPCL) an "instructor-student collaborative" learning method considers both prior knowledge known before training and information learning during training in

---

Table 2: Different forms of self- celf-paced functions and the corresponding closed form solutions towards
Equation 2 when the curriculum region is [0, 1 1() is the indicator function.
Function Form
Closed Form Optimal Solution
Binary Scheme
g(w; A)
->
w (((ss))) 4
Linear Scheme
g(w; A)
A 2
(w] 2ws)
w ((ess)) (1-4)
1=1
unified framework. To be specific, the objective loss function of SPCL is defined as
min w wE[0,1]" ,008
E(w, 1, =
W.L (y/s, (2) 0)) + g(w; A)
s.t.
WE
where f (m; 0) denotes the navigation agent, ur [u1, w., v,, E [0. 1J" is the weight variable reflecting the sample importance. g is called the self- self-paced function which controls the learning scheme, and À is the information predetermined curriculum, defined below.
that limits the learning pace. is a feasible region that encodes
Definition 4.1 For training samples X = {ar)"=>> given a curriculum Ty defined on it, the feasible region, defined by,
① == fw a" w sc}
curriculum region of y if it holds: 1)
[0, is nonempty; 2) a, aj for all
(xx) < (2,) for all 7 (x;) = 7 (xz)).
where vector a E R that parameterizes linear space is a function of curriculum It is not hard to see that, prior human knowledge influence the feasible region 4 which constrains the weight parameter w. Hence, the update of w is subject to the learning progress L, self-paced function g and feasible region 4. Based on our curriculum design, samples within each round should be given the same curriculum rank. Therefore, 5 scalars is enough to define good vector
Compared with self- learning (SPL) (Kumar et al., 2010), SPCL is more general by introducing self-paced function as a flexible regularization term. Self-paced functions are defined to be convex, which ensures that we can find good solutions of w within the linear curriculum region. Following (Jiang et al., 2014)'s work, (Jiang et al., 2015) further discussed several examples of self-paced functions. In this paper, we focus on the simplest two: (1) Binary scheme and (2) Linear scheme. As summarized in Table 2, the convex optimization problem
w = argmin Σ wily + g(w; A)
have the closed form solution optimization problem, we can simply apply a projection gradient descent method to obtain the optimal weight u²
[0, 1]". Therefore, to solve the linear constrained convex
4.2
Algorithm
(Jiang et al., 2015) proposed an alternative convex search (ACS) (M. Bazaraa and Shetty, 1993) algorithm to solve Equation 1. The main problem of the original algorithm is at step 4, where the optimal model parameter 0" is learned with most recent weight vector w" fixed. In the training of navigation agents, is impossible to compute the exact optimum of EEw"" 0, X, M due to both the time consumption and the lack of global convergence guarantees. We propose not to compute the exact minimum but replace step 4 with several gradient descent update steps. Doing so benefits the algorithm as the training is much faster then before.
According to (Gorski et al., 2007), the original SPCL algorithm proposed by (Jiang et al., 2015) is guaranteed to converge globally as the objective function is monotonically decreasing and bounded below. Our algorithm does not have the monotonically decreasing property as the stochastic gradient descent (SGD) update not assure the continuous decrease of the function value. Alternatively,
5

---

our algorithm can be viewed as naive version of mini- randomized block coordinate descent (MRBCD) method. (Zhao et al., 2014) claimed that MRBCD using semi-stochastic optimization scheme can attain linear rates of convergence, while the convergence of naive MRBCD is still left to be discussed. Further experiments show that our algorithm empirically converged, which encourages the theoretical analysis in this direction.
Algorithm takes the input of predetermined curriculum y, an instantiated self-paced function g. stepsize parameter μ and an update interval T;; it outputs an optimal model parameter 0.
Algorithm 1 Self-paced Curriculum Learning
Input: Input dataset D, predetermined curriculum y, function g, stepsize and update interval T.
Output: Model paramater 0.
1: Derive the curriculum region & from 7,
2: Initialize w' and À in the curriculum region.
3: while not converged do
4: fort = 0, , --- 1 do
5: Update 0.+1 = SGD(E(w", 02; D))
6: end for
7: Record 0" = Or
8: Update w" argmin,,. E(w, 0"; ; +)
9: Á is small, then increase X by stepsize μ.
10: end while
11: return 0"
Experiment Results and Analysis
5.1 Experiment Setup
Navigation agents. We experiment with three state-of-the-art VLN agents using different training paradigms and compare their performance on the original R2R validation set. We choose several sfate-of-the-art navigation agents including the Speaker-Follower agent (Fried et al., 2018) which applies the panoramic action space, the Self-Monitoring agent (Ma et al., 2019) which enforces cross- model alignment and EnvDrop with Back Translation (Tan et al., 2019) that applies reinforcement learning. We reproduce these agents in a unified code framework and replicate the experimental results presented in original papers. We do not use any data augmentation tricks and all three agents are trained within the R2R dataset range.
Implementation details. We We experiment with three training paradigms. They include the traditional machine learning (ML) strategy that training by uniformly sample mini-batches from R2R dataset, naive curriculum learning (NCL) strategy that the agent is firstly trained on CLR2R round split, then round 1-2 splits, and gradually trained on the whole CLR2R train set, and previously introduced self-paced curriculum learning (SPCL) strategy. As discussed Section 4, we set G₂ = for samples in CLR2R round split. The constant e chosen within range [0.95 ||a] 1, ||a ].] For self-paced functions, we choose the binary and linear scheme. For EnvDrop agent (Tan et al., 2019) which is trained by mixed loss of imitation learning and reinforcement learning, we only use the ground truth trajectory- based loss to update the weight variable. More information is available in supplementary materials.
Evaluation metries. We follow the standard metrics that (Fried et al., 2018) employed for evaluat- ing the agent's performance, including average Navigation Error (NE) which measures the distance between the end location and the target location, Success Rate (SR) which is the percentage of predicted end location within 3m of the target location, and Oracle Success Rate (OSR) which is the percentage of trajectory the shortest distance between which and the target location within 3m. We also adopt success weighted by path length (SPI.) as recommended by (Anderson et al., 201 8). Furthermore, we consider the coverage weighted by length score (CI.S) (Jain et al., 2019) which measures the overall correspondence between the predicted and ground truth trajectories. Actually,

---

Table Comparison results on validation unseen and unseen split using different training paradigm. Bolded agent name indicates that it is trained with traditional machine learning method. The plus sign
(+) represents the specific curriculum learning method used train this agent. Evaluation metrics are higher the better except for the navigation error (NE).
Validation Seen
Validation Unseen
Model
NE
SR
OSR
SPL
NE 4
SR
OSR
SPL
CLS
CLS
(m)
(%)
(%)
(%)
(m)
(%)
(%)
(%)
Follower
4.85
52.3
65.3
44.3
58.2
7.12
28.6
40.9
20.3
35.0
+ Naïve CL
5.03
48.6
62.0
40.4
55.9
7.13
31.1
42.8
21.2
34.3
Self-Paced CL
4.23
58.7
69.2
51.1
63.3
6.69
32.2
44.2
24.5
38.9
Self-Monitoring
4.27
58.4
67.0
51.9
64.1
6.42
38.4
48.1
28.3
41.5
Naïve CL
4.08
61.0
69.8
54.6
64.9
6.30
40.0
51.7
28.6
41.1
Sclf-Paced CL
4.19
58.8
68.2
53.3
65.4
5.98
41.0
52.4
30.8
43.9
EnyDrop
4.55
57.7
65.6
54.4
67.2
5.92
45.7
54.2
41.8
57.0
Naïve CL
4.49
57.8
63.1
54.8
67.2
5.93
44.3
50.5
41.3
57.6
+ Self-Paced CL
4.42
58.1
65.6
54.8
67.4
5,48
47.6
54.3
44.1
59.1
Follower
Self-Monitoring
EnvDrop
0.40
0.30
0.45
0.25
SPCL
0.35
SPCL
175
100
125
150
175
200
100
125
150
175
200
0.45
w
0.55
OSR
0.40
ML
SPCL
0.45
ML
SPCL
0.50
ML
SPCL
0.35
100
125
150
175
200
100
125
150
175
200
100
125
150
175
200
epoch
Figure 3: Success rate (SR) and oracle success rate (OSR) of navigation agents trained by machine learning (ML) and self-paced curriculum learning (SPCL) on validation unseen split.
nDTW and SDTW proposed by (Magalhães et al., 2019) are both useful metrics that capture path fidelity, hence we supplement full-metric version of main results in supplementary materials.
5.2 Overall Performance
We first compare the performance of the different training paradigms on the CLR2R(R2R) validation set. Table shows overall results. Experiments show that naive curriculum learning is surprisingly effective. It improves the navigation outcomes of Follower and Self-Monitoring agent on validation unseen split. In contrast machine learning where the whole training set is available, naïve curriculum learning exposes the training samples sequentially from easy to hard. Hence, follower agent trained by NCL is underperformed on the validation seen split due to its limited learning ability. For self-paces curriculum learning, initialize the algorithm to pay splits, then allow actively assigns higher weight for examples which the agent learns better. During training, the weight for each sample can finally converge nearly to one. Experiments show that agents trained by SPCL consistently achieve good performance on both validation seen and unseen split.
attention
round I and
Furthermore, we compare the learning speed of agents trained by different training paradigms. As shown in Figure 3, there obviously a gap between agents trained by machine learning and self- paced curriculum learning. Agents trained by curriculum learning can learn fast and achieve better performance after the same number of iterations. This indicates that self-paced curriculum learning can not only improve the performance, but also optimize the training efficiency of the agents.
7

---

Follower
Self-Monitoring
EnvDrop
Validation Seen
36
54
50
12
0.00
0.50
0.75
RG
1.00
Pg
0.50
1.00
Validation Unseen
30
39
SR
46
5R
28
35
44
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
3
0.00
0.25
3.50
0.75
1.00
Figure 4: Success rate (SR) on C R2R equivalently) validation set with different SPCL hyper- parameter settings. The self-paced function fixed as the linear scheme. Red dashed line represents the result from agents trained by machine learning. The figure is drawn by cubic interpolation.
Fallower
Self- Monitoring
EnvDrop
20
10-
ML
ML
SPCL
SPCL
SPCL
Loss per Iterattonn
16-
16
0
50
100
150
200
50
100
150
200
100
150
200
Epoch
Epech
Epoch
Figure 5: Loss landscape of normal machine learning (ML) and self-paced curriculum learning (SPCL) trained agents. At particular training epoch, we record the maximum and minimum iteration loss, i.e. average loss of batch samples. We plot the distances between the two the shadowed area between lines.
5.3 Further Analysis
We also investigate the hyperparameter robustness of self-paced curriculum learning. Moreover, we present the transfer learning outcomes of curriculum learning method on CLR2R and RxR dataset.
Hyperparameter robustness. To understand how the weight initialization and stepsize choice influence the training paradigm, we grid search these two hyperparameters and the result is shown in Figure 4. Following the core idea of curriculum learning by starting small, we enforce the initial weight for round I and samples as one. Then wi represents the initial weight for samples in round 3-5 splits. As shown in Figure 4, SPCL favors smaller wy. This validates the principle of curriculum learning. Besides. SPCL is rather robust to weight initialization and the choice of stepsize as in most cases, the results in validation unseen split are better than the machine learning bascline. This also indicates a better generalizability of agents trained by SPCL. For validation seen split, the performance is usually no worse than the baseline. The overall outcomes prove that curriculum learning can improve the data exploitation efficiency and enhance the generalization ability.
Loss landscape. Following (Santurkar et al., 2018), we investigate the loss landscape during training by computing the distance between maximum and minimum loss. As shown in Figure SPCL narrows the loss gap for Follower and Self- Senf-Mooit agent. For EnvDrop agent, SPCL differs little from MI. We attribute this to the mixed learning strategy of EnvDrop, whose loss weighted sum of imitation lcarning(0.2) and reinforcement learning(0.8). Hence, the power of SPCL is not as
8

---

Table 4: Basic statistics of R2R and RxR dataset.
Average Path
Average
Average Instruction
Dataset
Length (m)
Edges
Length (words)
R2R (Anderson et al., 2018)
9.4
5
29
RxR (Ku et al., 2020)
14.9
8
78
Table 5: Transfer learning results of Self-Monitoring agent validation unseen split. Character F and Line (4) represents "first" and "second" respectively. RxR is short for RxR-en subset RxR datset in this table.
Validation Unseen
Train Set
NE(m)
SR(%)
OSR(%
%6)
SPL(%)
nDTW
R2R
RxR
R2R
RxR
R2R
RxR
R2R
RxR
R2R
RxR
R2R
RxR
6.42
10.39
38.4
16.0
48.1
29.9
28.3
9.1
43.7
28.0
2)
7.72
10.12
16.7
18.7
30.4
32.8
5.9
6.9
18.6
18.2
6.66
10.17
32.7
20.5
45.8
33.7
22.4
10.9
36.4
23.9
(4)
F
S
6.96
9.91
26.4
22.1
38.6
34.8
18.0
14.3
36.8
29.8
Naïve CL.
6.46
9.90
36.3
21.6
44.4
33.4
27.4
13.5
43.0
29.3
significant as in other agents. In general, our experimental results correspond to the theoretical result (Hacohen and Weinshall, 2019) that curriculum learning can effectively smooth the optimization landscape.
Transfer learning. To explore whether curriculum learning helps the agent to obtain better transfer- ring ability, we conduct the following experiment as summarized in Table 5. As mentioned in Section 2, RxR dataset (Ku et al., 2020) is a large-scale multilingual VLN dataset built upon Matterport3D simulator (Chang al., 2017). We select the RxR dataset as the target dataset of transfer learning since it shares the same environment with R2R dataset but is much harder in many aspects (see Table 4). As RxR dataset multilingual, we employ its english subset, RxR-en, to avoid the language bias. For fairly comparison, the number of training iterations is fixed at 80,000 for line (3)-(5) in table. Specifically, for line (3) the agent is trained on a integrated R2R+RxR-ea dataset for 80,000 iterations; for line (4), the agent is firstly trained on R2R trainset for 40,000 iterations and then transferred to RxR-en dataset for another 40,000 iterations; for line (5), the agent firstly trained on R2R trainset and then trained on the integrated R2R+RxR-en dataset for another 40,000 iterations.
Line (I) and (2) indicates that transfer learning (R2R-RXXB and sice-versa) performance is much weaker than the in-domain results. For line (3), the agent trained on an integrated dataset performs better on RxR-en validation split, but its performance on R2R validation split not as good as line (1). The difficulty gap between samples from two data source confuses the agent. Line (3) and (4) indicates that starting from simple improves the agents performance on RxR dataset. Compared with line (4), curriculum learning in line (5) helps the agent to preserve its navigation ability on R2R dataset. Actually, line (4) can also be taken as curriculum learning experiment whose target distribution is RxR dataset instead of R2R and RxR dataset.
This exploration experiment suggests that transfer from an easier dataset towards harder one, curriculum learning paradigm helps the agent to improve the performance on the harder dataset without degrading the performance on the easier dataset.
Combination with Pre-training. Pre-training methods are proved strong in many areas. We believe that navigation agents trained by our proposed curriculum training paradigm can also benefit from pre-training skills. That is, curriculum learning does not conflict with pre-training. To validate this, we combine agents trained by curriculum based methods with VLN-BERT (Majumdar et al., 2020), a visiolinguistic transformer-based model. VLN-BERR is scoring function that evaluates the compatibility between path-ins inth-instruction pairs, hence it can be easily applied to assist our navigation agents. Since VLN-BERT has to used under beam search settings we implement beam search only here. In (Majumdar et al., 2020), authors use beam size as 30 to include much path candidates as possible. While in this experiment, we aim to evaluate the usefulnees of combining curriculum and pre-training. Therefore, we restrict the beam size as 5 and purely use VLN-BERT model to scoring

---

Success Rate
Coverage weighted by Length Score
SINGLE-AUN
BEAM-1
50.6
25.0
52.9
18.5
49.3
49.2
34.5
32.3
31.1
30
40-
38.0
34.3
20
30-
ML
Nalve CL
Self-pacced CL
ML
Naive CL
Selt-paced CL
Figure 6: Success rate and coverage weighted by length score of Follower model trained by normal machine learning (ML), naive curriculum learning (Naive CL., self- curriculum learning (Self- paced CL. on validation unseen split under three different modes. SINGLE-RUN means always using argmax in action selection. BEAM-1 means beam search with beam size 1. VLN-BERT means the path-instruction pairs generated by beam search with size as are scored purely by VLN-BERT model.
and selecting the path-instruction pairs. We experiment with Follower agent under three different test modes, i.c single-run, beam search with size I, and vin-bert. Results are shown in Figure Beam search and VLN-BERT scorer both can improve the agent performance. Navigation agents trained by curriculum-bent methods obtain more improvements.
6 Conclusion and Future Works
We propose to apply curriculum learning as a useful training paradigm of VLN agents. We adapt the traditional self-paced curriculum learning and design the first curriculum for VLN task based R2R dataset. Experiments illustrate that our method model-agnostic and can improve both the performance and learning efficiency of the navigation agents. We verify the power of curriculum learning comes from its ability to smooth the loss landscape. Our further exploration indicates that curriculum learning is suitable for transfer learning and combination with pre-training methods.
In the future, we would like to explore the VLN task two directions. Along the path of this paper, would like to explore curriculum learning for vision-language navigation in different scenario of city traveling where the agent needs to perform multi-scale navigation, i.e. city-level and building-level. Alternatively, inspired by the ampleness of VLN tasks and datasets, i.e. R2R (Anderson et al., 2018), CVDN (Thomason et al., 2019), HANNA (Nguyen and Daumé, 2019), TOUCHDOWN (Chen al., 2019), REVERIE (Qi al., 2020) and so on, it is possible to apply meta-learning on top of these datasets and get an adaptable navigation agent.
Acknowledgements
This work is partially supported by Natural Science Foundation of China (No. 119917,1,,,, N0.6217020551,, Science and Technology Commission of Shanghai Municipality Grant (No.20dz1200600, 21 21QA1400600) and Zhejiang Lab (No. 2019KD0AD01).
References
Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Süiderrauff, I. Reid, S. Gould, A. V. D. Hen- gel, Vision-and- -ad--aann--ggegee Navigation: Interpreting Visually- -rouud- Navigation Instructions in Real Enviromments, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 3674-3683.
C.-Y. Ma, Lu, Z. Wu, Al-Regib, Kira, Socher, Xiong, Self-monitoring navigation agent via auxiliary progress estimation, ArXiv abs/1901.03035 (2019).
10

---

F. Zhu, Y. Zhu, X. Chang. X. Liang, Vision- Vision-language navigation with self-supervised auxiliary reasoning tasks, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) 10009-10019.
D. Fried, R. Hu, V. Cirik, A. Rohrbach, J. Andreas, L.-P. Morency, T. Berg- Kirkpatrick, K. Saenko,
D. Klein, T. Darrell, Speaker-follower models for vision-and sssion-and-langee ann-langu navigation, in: NeurIPS, 2018.
H. Tan, L. Yu, M. Bansal, Learning navigate unseen environments: Back translation with environmental dropout, ArXiv abs/ 1ss/1990004195 (2019).
Y. Hong, C. Rodriguez-OOazz, Q. Wu, S. Gould, Sub-instruction aware vision-and-language naviga- tion, in: EMNLP, 2020.
A. Ku, P. Anderson, R. Patel, E. le, J. Baldridge, Room-across-room Multilingual vision-and- language navigation with dense spatiotemporal grounding, in: EMNLP, 2020.
X. Li, C. Li. Q. Xia, Y. Bisk. A. Çelikyilmaz, Gao, N. A. Smith, Y. Choi, Robust navigation with language pretraining and stochastic sampling, in: EMNLP, 2019.
A. Majumdar, A. Shrivastava, S. Lee, P. Anderson, D. Parikh, D. Batra, Improving vision-and- language navigation with image-text pairs from the web, ArXiv abs/2000.14973 (2020).
W. Hao, C. Li, X. Li, L. Carin, J. Gao, Towards learning a generic agent for Visionaand-lanuune nav- igation via pre-training, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) 13134-13143.
Y. Huo, M. Zhang, G. Liu, H. Lu, Y. Gao, G. xing Yang. J. Wen, H. Zhang, B. Xu, W. Zheng, Z. Xi, Y. Yang, A. Hu, Zhao, R. Li, Y. Zhao, L. Zhang, Y. Song, X. Hong, W. Cui, D. Hou, Y. Li, J. Li, P. Liu, Z. Gong, C. Jin, Y. Sun, S. Chen, Z. Lu, Z. Dou, Q Jin, Y. Lan, W. X. Zhao, R. Song, Wen, Wenlan: Bridging vision and language by large-scale multi-modal pre-training, ArXiv abs/2103.00661 (2021).
H. D. Hlynsson, A. N. Escalante, L. Wiskott, Measuring the data efficiency of deep learning methods, in: ICPRAM, 2019.
H. Huang, V. Jain, H. Mehta, J. Baldridge, E. Ie, Multi-modal discriminative model for vision-and- language navigation, ArXiv abs/1905. (2019).
Y. Bengio, J. Louradour, R. Collobert, J. Weston, Curriculum learning, in: ICML '09, 2009.
L. Jiang, D. Meng, Q. Zhao, S. Shan, A. Hauptmann, Self-paced curriculum learning, in: AAAI, 2015.
H. Chen, A. Suhr, D. K. Misra, N. Snavely, Artzi, Touchdown: Natural language navigation and spatial reasoning in visual street environments, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 12530-12539.
J. Thomason, M. Murray, M. Cakmak, L. Zettlemoyer, Vision-and-dialog navigation, in: CoRL, 2019.
X. E. Wang, W. Xiong, H. Wang, W. Y. Wang, Look before you leap: Bridging model-free and basel reinforcement learning for planned- ahead 2018.
navigation, in: ECCV,
X. E. Wang, Q. Huang, A. Çelikyilmaz, J. Gao, D. Shen, Y. Wang. W. Y. Wang, L. Zhang, Reinforced cross-modal matching and self-supervised imitation learning for vision-languag navigation, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 66311
X. Wang, V. Jain, E. Ic. W. Y. Wang, Z. Kozareva, S. Ravi, Environment-agnosiee multitask learning for natural language grounded navigation, ArXiv abs/2003.00444 (2020).
A. Graves, Bellemare, Menick, R. Munos, K. Kavukcuoglu, Automated curriculum learning for neural networks, ArXiv abs/1704.03003 (2017).

---

T. Matiisen, A. Oliver, T. Cohen, J. Schulman, Teacher-student curriculum learning. CoRR abs/1707.00183 (2017).
M. Sachan, E. Xing, Easy questions first? case study on curriculum learning for question answering, in: ACL, 2016.
C. Liu, S. He, K. Liu, J. Zhao, Curriculum learning for natural answer generation, in: IJCAL, 2018.
E. A. Platanios, O. Streteu, G. Neubig, B. Poezos, T. M. Mitchell, Competence-based curriculum learning for neural machine translation, arXiv preprint arXiv:1903.0990 (2019).
G. Hacohen, D. Weinshall, On the power of curriculum learning in training deep networks, ArXiv abs/1904.00 036626 (2019).
W. Zhu, H. Hu, J. Chen, Z. Deng, V. Jain, E. le, Sha, Babywalk: Going farther in vision-and- language navigation by taking baby steps, in: ACL, 2020.
M. Kumar, B. Packer, D. Koller, Self-paced learning for latent variable models, in: NIPS, 2010.
L. Jiang, D. Meng, T. Mitamura, A. Hauptmann, Easy samples first: Self-paced reranking for zero-example multimedia search, Proceedings of the 22nd ACM international conference on Multimedia (2014).
H. M. Bazaraa, C. Shetty, Nonlinear programming: Theory and algorithms, in: John Wiley and Sons, Inc., 1993.
J. Gorski, F. Pfeuffer, K. Klamroth, Biconvex sets and optimization with biconvex functions: survey and extensions, Mathematical Methods Operations Research 66 (2007) 373-407.
Zhao, M. Yu, Y. Wang. R. Arora, Liu, Accelerated mini-batch randomized block coordinate descent method, Advances in neural information processing systems 27 (2014) 5614.
Anderson, A. X. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V. Koltun, J. Kosecka, Ma- lik, R. Mottaghi, M. Savva, A. Zamir, On evaluation of embodied navigation agents, ArXiv hts//807.06657 (2018).
V. Jain, G. Magalhães, A. Ku, A. Vaswani, E. Ie, Baldridge, Stay on the path: Instruction fidelity in vision-and-languaage navigation, in: ACL, 2019.
G. Magalhães, V. Jain, A. Ku, E. Ie, J. Baldridge, Effective and general evaluation for instruction conditioned navigation using dynamic time warping, ArXiv (2019).
S. Santurkar, D. Tsipras, A. Ilyas, A. Madry, How does batch normalization help optimization?, in: NeurlPS, 2018.
A. X. Chang, A. Dai, T. Funkhouser, M. Halber, M. Nießner, M. Savva, S. Sonog A. Zeng, Y. Zhang, Matterport3d: Learning from rgb-d data in indoor environments, 2017 International Conference on 3D Vision (3DV) (2017) 667-676.
K. Nguyen, H. Daumé, Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning, in: ViGIL LiGIl&@eeet @NeurIPS 2019.
Y. Qi, Q. Wu, P. Anderson, X. Wang, W. Wang, C. Shen, A. V. Hengel, Reverie: Remote embodied visual referring expression in real indoor environments, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) 9979-9988.
12


Reviews:
[{"review_id": "SGa2yooKoil", "reviewer": "Reviewer_mN5T", "summary": "This paper applies self-paced curriculum learning [1] to VLN. It presents two approaches: a naive approach where the agent is trained on data with increasing coverage over the environment, and a learning-based approach that dynamically adapts the weights of learning examples according to the agent's learning progress. On three types of VLN agents, these approaches yield moderate improvement `(~2-3% improvement in success rate on validation unseen environments). ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**: the technique is primarily based on self-paced curriculum learning by [1] with minor adaptions to a mini-batch setting. The novelty is mostly in the empirical results and analyses. The paper demonstrates potential benefits (in both success rate and learning efficiency) of enforcing a curriculum in training VLN agents. Another interesting result is that pre-training an agent with the (easier) R2R tasks can help with learning the (more difficult) RxR tasks. \n\n[1] Self-paced curriculum learning. https://www.cs.cmu.edu/~lujiang/camera_ready_papers/AAAI_SPCL_2015.pdf\n\n**Quality**: \n\nThe SPCL technique is based on a well-established prior work and thus is solid. However, I have a few concerns about the experimental procedure. \n\nFirst of all, I notice that only validation results are reported. If the validation sets are used to tune the agents, then these results do not indicate the generalizability of the agents, and thus are not sufficient to demonstrate the effectiveness of the proposed technique. I request the authors to include results on the test sets. \n\nSecond, because there are randomness in various parts of the proposed technique (due to the mini-batch setting), the authors should also repeat the experiments with multiple random seeds and report error bars of the results. Results as those in Figure 3 may not hold under a different initialization configuration. \n\nIn addition, the messages conveyed by Figure 1 are not very clear to me. In Figure 1a, to illustrate the benefits of training with a curriculum, there needs to be another contrasting Figure showing the same plot but with curriculum learning. Figure 2b shows that the agent \"is limited by its ability to navigate inside one room\" but I am not why that motivates curriculum learning. It makes more sense to me to plot the distribution of errors over the course of training. If we see that at the beginning, in-room errors dominate (in percentage) then it is reasonable to prioritizing training examples that can fix these errors. \n\nAs discussed in the paper, we should expect that the self-paced curriculum increasingly includes more examples over time. It would be nice to verify that (e.g. by visualizing the norm of w over time). \n\n**Clarity**: \n\nThe paper is straightforward to read. However, the descriptions of the SPCL leaves out a few important details and intuitions. In line 144 and even in the appendix, the *a* vector is not clearly defined. The update of lambda in algorithm 1 should be more detailed. When is lambda \"small\"? and what is the formulation for updating the lambda? This sentence seems to be copied and pasted from the SPCL's original paper. In this paper, you need to provide sufficient detail so that other researchers can reproduce the results. \n\nI also suggest the authors give a more detailed descriptions of the agents, specifically about their architectural differences. More importantly, are the architectural differences the only differences? Are the agents trained under the same data in each experiment? For example, the EnvDrop agent is trained with environment dropout in its paper; and the Follower agent is trained with additional back-translated data in its paper. Do you also include those data in training them in this paper?\n\nIn figure 1, you may want add to the caption that the results are obtained without curriculum learning.\n\nA minor thing is to put up and down arrows next to the metrics in the result tables so that readers can quickly identify which direction is better. \n\n**Significance**: overall, the gain the validation data is moderate. This paper represents an incremental step towards solving VLN. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The curriculum is designed based on the path structure only. It would be nice to measure and include the complexity of the language in the instructions in designing the curriculum. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "Osle2UYERSN", "reviewer": "Reviewer_uKaH", "summary": "The paper proposes a curriculum learning (CL) based approach for the task of Vision-Language Navigation (VLN). The dataset chosen for experimentation is R2R which involves an agent to navigate from room to room to follow the provided instruction. The proposed method creates a curriculum for the learning agent based on the number of rooms a given path covers. The empirical results are reported using two methods -- (1) naive CL where the agent is trained on gradually harder tasks, (2) self paced curriculum learning (SPCL).\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n-- The criteria for creating a curriculum is easy and straightforward for the given dataset.\n\n-- The main empirical results show clear gains due to the SPCL method.\n\n-- The paper is clearly written and the technical contributions are neatly presented. Fig 3,4,5 have good visualizations.\n\nWeaknesses:\n\n-- The contributions of this paper are limited to empirical studies. The concept of CL has been studied in this domain and shown to have value in Zhu et al. The methodology (SPCL) is the same as Jiang et al.\n\n-- From the results in Table 3, there are negligible gains due to CL on EnvDrop method. Since EnvDrop method obtains the best results (out of the 3 studied methods) on R2R dataset, it is unclear if CL is contributing to the success of other two methods or whether the other two methods benefit due to a side effect of using some simple ordering of example visitation. For instance, if an experiment is conducted where the examples are fed using number of rooms in the path as criteria but the feeding order is random instead of monotonically increasing from 1 to 5. Will a random order perform similar to the curriculum learning order?\n\n-- The criteria for creating a curriculum seems highly tailored for the given R2R dataset. Such criteria won't be extendible to other VLN datasets like CVDN or Touchdown.\n\n======= Post author response updates ==========\n\nIncreasing my score from 5 to 6. See my response to authors' response for rationale.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "KcPAmFiN6RX", "reviewer": "Reviewer_EhTe", "summary": "This paper proposes a curriculum-based training paradigm for Vision-and-Language Navigation tasks. They focus on the aspect of whether the performance of a VLN agent can be improved without model structure change and data modification. They define the difficulty of the episode based on how many rooms it needs to cross and use this to divide the dataset into 5 parts. These parts are then used to form a curriculum learning training schedule. They use Self-Paced Curriculum Learning, and show that their method improves both the navigation performance and the training efficiency for the VLN task (R2R).  ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality and Significance:\n-\tThis is the first paper to develop a model-agnostic curriculum learning approach for VLN.  \n-\tTheir training curriculum improves the model performance without introducing any new data or increasing the model complexity.\n-\tIn one of the experiments, they show that the curriculum learning paradigm helps the agent to improve the performance on the harder dataset without degrading the performance on the easier dataset.\n\nQuality and Clarity:\n-\tThe paper is well-written and easy to follow.\n-\tThey have shown that their curriculum-learning approach consistently improves performance for 3 VLN-based models, Follower from Speaker-Follower paper, Self-Monitoring agent, and EnvDrop agent. \n-\tThey have detailed their training algorithm and provided the hyperparameters to reproduce its results.\n\n\nTypos:\n\nL67, L181: argumentation -> augmentation\n\nFigure 2: Bottom-left image: Longue -> Lounge\n\nL91: ans -> and\n\nL256: hrader -> harder\n\nTable 5, caption: fisrt -> first\n\n\nMinor Comment:\n\nIn figure 1, all images seem to have a blue-ish tint. Based on my experience working with the Matterport3D images, it looks like the images were outputted in BGR instead of RGB, please do check.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They have explained limitations in the conclusions section.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "9SKzTtBLx9", "reviewer": "Reviewer_5Cjr", "summary": "This paper introduces a new curriculum learning scheme for vision-and-language navigation (more particularly, the R2R dataset), where the navigation agent learns from easier to harder paths based on the number of traversed rooms. The curriculum learning scheme improves the performance of multiple baseline models on the R2R dataset. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper introduces a new curriculum learning scheme for vision-and-language navigation (more particularly, the R2R dataset), where the navigation agent learns from easier to harder paths based on the number of traversed rooms. The curriculum learning scheme improves the performance of multiple baseline models on the R2R dataset. \n\nStrengths:\n- The proposed curriculum learning (CL) scheme is intuitive and effective. \n- It brings improvements to multiple baseline models on the R2R dataset.  \n- The paper is easy to read.\n\nWeaknesses:\n- Curriculum learning is not as novel as the authors claimed (certainly not the first CL method for VLN). [1] proposed a CL method for VLN already and [2] is also closely related. More discussions and comparison should be included, and the authors need to avoid overclaiming.\n- The baseline navigation models used in the paper are sort of out-dated. How about some recent advanced VLN models such as VLN-BERT and Transformers? Would the CL work for them or not?\n- Given that there are multiple datasets for VLN, new methods are typically expected to be validated on them. Particularly, the authors could have easily tested the CL approach on R4R. Why not?\n- The CLS metric is not used in the paper. \n- Regarding the ablation study on transfer learning in Table 5, [3] did some analysis on R2R and CVDN and showed that a simple yet effective multitask learning method can improve both datasets/tasks, which I think the authors should discuss.   \n\n[1] BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps\n[2] Sub-Instruction Aware Vision-and-Language Navigation\n[3] Environment-agnostic Multitask Learning for Natural Language Grounded Navigation\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work introduces a simple and intuitive approach to VLN based on the compositional nature of instructions, focusing therefore on single room, before multiform, etc.  The approach is then tested on multiple baselines for both R2R and RxR and shows consistent improvements.  The results are based on validation performance (not unseen test) and focus on older models which are substantially weaker than the current SotA on these domains.  This raises a natural question about the applicability of this approach to more contemporary models.
Decision: Accept (Poster)