Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a curriculum-based training paradigm for Vision-and-Language Navigation tasks. They focus on the aspect of whether the performance of a VLN agent can be improved without model structure change and data modification. They define the difficulty of the episode based on how many rooms it needs to cross and use this to divide the dataset into 5 parts. These parts are then used to form a curriculum learning training schedule. They use Self-Paced Curriculum Learning, and show that their method improves both the navigation performance and the training efficiency for the VLN task (R2R).  
main_review: Originality and Significance:
-	This is the first paper to develop a model-agnostic curriculum learning approach for VLN.  
-	Their training curriculum improves the model performance without introducing any new data or increasing the model complexity.
-	In one of the experiments, they show that the curriculum learning paradigm helps the agent to improve the performance on the harder dataset without degrading the performance on the easier dataset.

Quality and Clarity:
-	The paper is well-written and easy to follow.
-	They have shown that their curriculum-learning approach consistently improves performance for 3 VLN-based models, Follower from Speaker-Follower paper, Self-Monitoring agent, and EnvDrop agent. 
-	They have detailed their training algorithm and provided the hyperparameters to reproduce its results.


Typos:

L67, L181: argumentation -> augmentation

Figure 2: Bottom-left image: Longue -> Lounge

L91: ans -> and

L256: hrader -> harder

Table 5, caption: fisrt -> first


Minor Comment:

In figure 1, all images seem to have a blue-ish tint. Based on my experience working with the Matterport3D images, it looks like the images were outputted in BGR instead of RGB, please do check.


limitations_and_societal_impact: They have explained limitations in the conclusions section.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper introduces a new curriculum learning scheme for vision-and-language navigation (more particularly, the R2R dataset), where the navigation agent learns from easier to harder paths based on the number of traversed rooms. The curriculum learning scheme improves the performance of multiple baseline models on the R2R dataset. 
main_review: This paper introduces a new curriculum learning scheme for vision-and-language navigation (more particularly, the R2R dataset), where the navigation agent learns from easier to harder paths based on the number of traversed rooms. The curriculum learning scheme improves the performance of multiple baseline models on the R2R dataset. 

Strengths:
- The proposed curriculum learning (CL) scheme is intuitive and effective. 
- It brings improvements to multiple baseline models on the R2R dataset.  
- The paper is easy to read.

Weaknesses:
- Curriculum learning is not as novel as the authors claimed (certainly not the first CL method for VLN). [1] proposed a CL method for VLN already and [2] is also closely related. More discussions and comparison should be included, and the authors need to avoid overclaiming.
- The baseline navigation models used in the paper are sort of out-dated. How about some recent advanced VLN models such as VLN-BERT and Transformers? Would the CL work for them or not?
- Given that there are multiple datasets for VLN, new methods are typically expected to be validated on them. Particularly, the authors could have easily tested the CL approach on R4R. Why not?
- The CLS metric is not used in the paper. 
- Regarding the ablation study on transfer learning in Table 5, [3] did some analysis on R2R and CVDN and showed that a simple yet effective multitask learning method can improve both datasets/tasks, which I think the authors should discuss.   

[1] BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
[2] Sub-Instruction Aware Vision-and-Language Navigation
[3] Environment-agnostic Multitask Learning for Natural Language Grounded Navigation



limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a curriculum learning (CL) based approach for the task of Vision-Language Navigation (VLN). The dataset chosen for experimentation is R2R which involves an agent to navigate from room to room to follow the provided instruction. The proposed method creates a curriculum for the learning agent based on the number of rooms a given path covers. The empirical results are reported using two methods -- (1) naive CL where the agent is trained on gradually harder tasks, (2) self paced curriculum learning (SPCL).


main_review: Strengths:

-- The criteria for creating a curriculum is easy and straightforward for the given dataset.

-- The main empirical results show clear gains due to the SPCL method.

-- The paper is clearly written and the technical contributions are neatly presented. Fig 3,4,5 have good visualizations.

Weaknesses:

-- The contributions of this paper are limited to empirical studies. The concept of CL has been studied in this domain and shown to have value in Zhu et al. The methodology (SPCL) is the same as Jiang et al.

-- From the results in Table 3, there are negligible gains due to CL on EnvDrop method. Since EnvDrop method obtains the best results (out of the 3 studied methods) on R2R dataset, it is unclear if CL is contributing to the success of other two methods or whether the other two methods benefit due to a side effect of using some simple ordering of example visitation. For instance, if an experiment is conducted where the examples are fed using number of rooms in the path as criteria but the feeding order is random instead of monotonically increasing from 1 to 5. Will a random order perform similar to the curriculum learning order?

-- The criteria for creating a curriculum seems highly tailored for the given R2R dataset. Such criteria won't be extendible to other VLN datasets like CVDN or Touchdown.

======= Post author response updates ==========

Increasing my score from 5 to 6. See my response to authors' response for rationale.
limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper applies self-paced curriculum learning [1] to VLN. It presents two approaches: a naive approach where the agent is trained on data with increasing coverage over the environment, and a learning-based approach that dynamically adapts the weights of learning examples according to the agent's learning progress. On three types of VLN agents, these approaches yield moderate improvement `(~2-3% improvement in success rate on validation unseen environments). 
main_review: **Originality**: the technique is primarily based on self-paced curriculum learning by [1] with minor adaptions to a mini-batch setting. The novelty is mostly in the empirical results and analyses. The paper demonstrates potential benefits (in both success rate and learning efficiency) of enforcing a curriculum in training VLN agents. Another interesting result is that pre-training an agent with the (easier) R2R tasks can help with learning the (more difficult) RxR tasks. 

[1] Self-paced curriculum learning. https://www.cs.cmu.edu/~lujiang/camera_ready_papers/AAAI_SPCL_2015.pdf

**Quality**: 

The SPCL technique is based on a well-established prior work and thus is solid. However, I have a few concerns about the experimental procedure. 

First of all, I notice that only validation results are reported. If the validation sets are used to tune the agents, then these results do not indicate the generalizability of the agents, and thus are not sufficient to demonstrate the effectiveness of the proposed technique. I request the authors to include results on the test sets. 

Second, because there are randomness in various parts of the proposed technique (due to the mini-batch setting), the authors should also repeat the experiments with multiple random seeds and report error bars of the results. Results as those in Figure 3 may not hold under a different initialization configuration. 

In addition, the messages conveyed by Figure 1 are not very clear to me. In Figure 1a, to illustrate the benefits of training with a curriculum, there needs to be another contrasting Figure showing the same plot but with curriculum learning. Figure 2b shows that the agent "is limited by its ability to navigate inside one room" but I am not why that motivates curriculum learning. It makes more sense to me to plot the distribution of errors over the course of training. If we see that at the beginning, in-room errors dominate (in percentage) then it is reasonable to prioritizing training examples that can fix these errors. 

As discussed in the paper, we should expect that the self-paced curriculum increasingly includes more examples over time. It would be nice to verify that (e.g. by visualizing the norm of w over time). 

**Clarity**: 

The paper is straightforward to read. However, the descriptions of the SPCL leaves out a few important details and intuitions. In line 144 and even in the appendix, the *a* vector is not clearly defined. The update of lambda in algorithm 1 should be more detailed. When is lambda "small"? and what is the formulation for updating the lambda? This sentence seems to be copied and pasted from the SPCL's original paper. In this paper, you need to provide sufficient detail so that other researchers can reproduce the results. 

I also suggest the authors give a more detailed descriptions of the agents, specifically about their architectural differences. More importantly, are the architectural differences the only differences? Are the agents trained under the same data in each experiment? For example, the EnvDrop agent is trained with environment dropout in its paper; and the Follower agent is trained with additional back-translated data in its paper. Do you also include those data in training them in this paper?

In figure 1, you may want add to the caption that the results are obtained without curriculum learning.

A minor thing is to put up and down arrows next to the metrics in the result tables so that readers can quickly identify which direction is better. 

**Significance**: overall, the gain the validation data is moderate. This paper represents an incremental step towards solving VLN. 

limitations_and_societal_impact: The curriculum is designed based on the path structure only. It would be nice to measure and include the complexity of the language in the instructions in designing the curriculum.
review_title: null

Meta-review: This work introduces a simple and intuitive approach to VLN based on the compositional nature of instructions, focusing therefore on single room, before multiform, etc.  The approach is then tested on multiple baselines for both R2R and RxR and shows consistent improvements.  The results are based on validation performance (not unseen test) and focus on older models which are substantially weaker than the current SotA on these domains.  This raises a natural question about the applicability of this approach to more contemporary models.
Decision: Accept (Poster)