Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary:

There have been two recent related tasks proposed in vision-langauge settings: vision-langauge navigation (VLN) where natural language turn-by-turn instructions must be decoded by an agent in an indoor environment to reach the goal location and Navigation from Dialog History (NDH) where dialog between two humans trying to reach a goal is input to an agent to try to reach a goal location. This paper uses these two tasks' data in a multi-task manner to try to generalize better between indoor environments especially unseen environments which are not in the training set of the agent. 

Another proposed innovation is to use an auxiliary task of environment classification but via a gradient reversal layer such that the learnt latent representation input to the classifier should not overfit to foibles of the environment but should (hopefully) learn a representation that captures the intrinsics of the environment necessary for generalization. 

Comments:

- The paper is well-written and easy to understand! Experiments are thorough and has all the ablations one would ask for. Thanks!

- Overall I like the paper but have a number of comments:

1. Why not try more sophisticated methods of multi-task learning like 'MetaLearning' by Finn et al 2017 (MAML). It is common knowledge that straight up averaging across tasks is not as effective as doing the bilevel optimization in MAML. 

2. Why do RL at all? Already the authors are doing BC (naive form of imitation learning) but they could just do robust imitation learning like DAgger, AggreVateD, etc. The setting is already such that one has a natural oracle (which the authors are already using via BC in the objective) which is the shortest path planner during training time. Then combined with MAML one can do Meta-IL as in 'One-Shot Imitation Learning via Meta-Learning' Finn et al 2017. Note that imitation learning is exponenially more sample efficient than RL and removes all the reward-shaping complications. 

3. In Section 2 for error correction "Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention" by Nguyen et al. CVPR 2019 is directly relevant. 

4. Also for generalizaton performance these papers are directly relevant:

"Building Generalizable Agents with a Realistic and Rich 3D Environment" Wu et al ICLR 2018

"Learning and Planning with a Semantic Model" Wu et al 





review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper aims to apply the model of Wang 2019 to the new NDH task of Thomason '19.  Both of these datasets are built on the same room-to-room environment and both are for natural language instruction following.  Thomason's work extends the R2R paradigm to include a dialogue history which is collapsed into a single instruction.  The contribution of this paper is to build a single model which alternatingly samples trajectories from each of the two datasets to train a more general actor and the authors also believe that the presence of an environment classifier assists in generalization.

The claims of the paper focus on being "environment-agnostic" and notions of generality.  As hinted by the authors in their future work, to properly show this would probably require two different environments or tasks (e.g. Touchdown), not training on two tasks that use the same environments and pre-computed visual features.  Am I incorrect that the only difference between the NDH and VLN formulation is the structure of the sentences?  

Figure 3 is the most compelling component of the paper.  However, I am still not convinced it will generalize and all other components of the paper are largely minor tweaks to existing work.  

Figure 2 mostly leads me to believe that we have a simple data-augmentation situation which makes the bump in performance somewhat predictable.  Minor: Is there any reason why in Table 1 we can't simply run the VLN models on NDH and NDH on VLN?  

I commend the authors for putting this all together in the two months between the release of CVDN and the ICLR deadline.
I think training a joint model on these two datasets is a completely natural experiment that many of us wanted to see, and so I appreciate the effort of the authors and the benefit to the community of having these numbers, but I'm not convinced there is that much meat otherwise in this paper.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper addresses some challenges of following natural language instructions for navigating in visual environments. The main challenge in such tasks is the scarcity of available training data, which results in generalization problems where the agent has difficulty navigating in unseen environments. Therefore, the authors propose two key ideas to tackle this issue and incorporate them in the reinforced cross-modal matching (RCM) model (Wang et al, 2019). First, they use a generalized multitask learning model to transfer knowledge across two different tasks: Vision-Language Navigation (VLN) and Navigation from Dialog History (NDH). This results in learning features that explain both tasks simultaneously and hence generalize better. Moreover, by training on both tasks the effective size of training data is increased significantly. Second, they propose an environment-agnostic learning technique in order to learn invariant representations that are still efficient for navigation. This prevents overfitting to specific visual features of the environment and therefore helps improving generalization. The contribution of this paper is combining and incorporating these two key ideas in the RCM framework and verifying it on VLN and NDH tasks. This approach is novel compared to prior results in tackling the VLN task. Their experimental results show that the two proposed techniques improve generalization in a complementary fashion, measured by decreased performance gap between seen and unseen environments.  They demonstrate that their technique outperforms state-of-the-art methods on unseen data on some evaluation metrics.

Even though the two key ideas proposed in the paper has been explored in the literature in other contexts, this paper contributes to natural language guided navigation by incorporating these ideas in a unified framework and demonstrates promising results on two datasets. Therefore, I would recommend accepting this paper if some issues in delivery and clarity are addressed.

In particular, Section 3, where the authors introduce the novelty of the paper in more detail, could be better explained and a cleaner line should be drawn between prior results from other papers and novel results proposed by this paper. In general, the new ideas would require more emphasis, since they are somewhat lost between adaptations from prior work. Most importantly, Eq. (3) is stated without sufficient motivation and would require a more detailed explanation.

In addition to these points, I would like to disclose some recommendations that might improve the paper but are not strictly part of my decision. In some cases the notation is not clear and some variables are not defined or explained. For instance, after Eq. (8) Attention(.) is used without citation or definition, in Eq. (9) Wc and Wu are not defined and some of the notation in Eq. (6)-(7) are not defined. Moreover, reading the decrease in performance gap from the presented table format is inconvenient and a better visual representation might help demonstrating the improvement in generalization better.  Lastly, I noticed that the navigation error for shared decoder is slightly higher than for separate encoders in Table 3, even though it outperforms the separate encoders in every other measures. Is there a particular explanation for this?
review_title: Official Blind Review #3

Meta-review: The paper proposes a multitask navigation model that can be trained on both vision-language navigation (VLN) and navigation from dialog history (NDH) tasks. The authors provide experiments that demonstrate that their model can outperformance single-task baseline models.

The paper received borderline scores with two weak accept and one weak reject.  Overall, the reviewers found the paper to be well-written and easy to understand, with thorough experiments.

The reviewers had minor concerns about the following:
1. The generalizability of the work.  No results are reported on the test set, only on val.
2. The gains for val unseen are pretty small and there are other models (e.g. Ke et al, Tan et al) that have better results.  Would the proposed environment-agnostic multitask learning be able to improve those models as well?  Or is the gains limited to having a weak baseline?
3. It's unclear if the gains are due to the multitasking or just having more data available to train on.
4. There are some minor issues with the misspellings/typos.  Some examples are given:
Page 1: "Manolis Savva* et al" --> "Savva et al"
Page 5: "x_1, x2, ..., x_3" --> Should the x_3 be something like x_k where k is the length of the utterance?

The AC agrees with the reviewers that the paper is interesting and is mostly solid work.  The AC also feels that there are some valid concerns about the generalizability of the work and that the paper would benefit from a more careful consideration of the issues raised by the reviewers.  The authors are encouraged to refine the work and resubmit.
Decision: Reject