Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper introduces the generative modeling of the task of vision and language navigation. At each timestep, for each action, the generation of the instruction is scored. This modeling achieves the-state-of-the-art or competitive results on standard benchmarks Room-to-room (R2R) and Room-for-room (R4R). The proposed generative modeling also allows us to interpret how the model process the instruction. To do so, token-wise prediction entropy (1-TENT) is introduced. The core idea is that if a token is critical for the trajectory, the entropy of scoring the token for all actions will below. Thus by visualizing the 1-TENT for each timestep we can analyze how and when a model fails or what kind of capabilities are missing in the model. Below I list my questions (Q) and suggestions (S):

S1 Introduction 4th paragraph: It was rather hard to understand what's the difference between Fried et. al. and this work. Please try to simplify or add a figure or give examples

S2 Introduction penultimate paragraph: Please clarify the concept of 'richer learning signal'. This paragraph is rather more cryptical compared to the rest of the paper.

S3 Figure2: please add a sentence or two to clarify how this is generated and the punchline of the caption. Also, add the legend for x axis.

S4 Section 5.3: I believe you are referring to Figure 2 not Figure 5 which is in the appendix.

S5 Figure 3: You are trying to achieve a lot with two analyses. Please move one of them to the appendix -- which will give more space to panorama images. Also, the figure is already hard to interpret without the guidance for the level of 1-TENT scores for tokens. 

Q1: Section1: The assumption that the action probabilities are uniform given the state makes sense. However, in reality, it's not true. Have you done any experiments on this or any ideas on how to address that?

Q2: Section 4: This might be a lot to ask but is it possible to have results on TouchDown or any other outdoor navigation datasets? Problems are structurally the same however the way people give instructions in outdoor scenes will be different. It would be great to see the generalization of the generative approach to outdoor scenes.


review_title: adds the missing piece in VLN literature

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper presents a generative speaker model that selects actions at each timestep that facilitate the generation of the instruction, and demonstrates that a combination of discriminative and generative action prediction models outperform either alone, with the generative model primarily facilitating better generalization to unseen environments.
It would help to have more details in the paper rather than relegated to the appendix, maybe at the expense of the long TENT analysis? I didn't get too much out of that, but would have liked to see more details of the main algorithm in the paper (e.g., details of how the discriminative/generative combo was done).

Improvements:

- Assumption that p(a_t|h_t) is uniform is reasonable, but might be improved by using priors from the training data since there are strong action-conditioned biases in VLN R2R [ https://arxiv.org/abs/1811.00613 Figure 2 ]; some geometric form of this likely holds for the panoramic setting as well (especially for what were formerly "forward" actions -> continuing along headings close to previous heading).

- The (A) and (B) notation in Table 1 was super confusing and it took me a while to figure out it was just an alias for Disc + Aug versus Gen + Aug since that isn't written explicitly anywhere.

- That the combination of discriminative and generative policies gives best results is a major point in the paper, but the combined policy description is relegated to the appendix which feels weird.

Nits:
- typo Introduction Chang et al. is \citet but should be \cite
- Possible typo after Eq (3), "penalizes all the actions" should this be "penalizes all actions except a_t", or is a_t penalized as well (e.g., global optimum is zero)?
- 5.3 typo "Figure 5" -> "Figure 2"
- Typo 5.5 "as good as" -> "as well as"
- Typo 6 "parametrization"

review_title: A solid technical contribution that is concisely presented and analyzed.

final_score: rating: 4: Ok but not good enough - rejection
review_content: **Paper Summary**

The paper addresses the problem of vision-and-language navigation (Anderson et al., 2018). The idea of the paper is to use a generative policy where a distribution over all instruction tokens given the previous actions is computed. The agent takes the action that maximizes the probability of the current instruction. The paper reports the results on R2R and R4R datasets.

**Paper Strengths**

- The paper shows that the proposed generative model outperforms the discriminative formulation of the approach.

- The results have been compared with a number of strong baselines.

**Paper Weaknesses**

- The novelty is limited compared to the speaker model of Fried et al. (2018). It is mentioned in the introduction section that Fried et al. use the entire sequence of actions while this paper uses local information at each step. I do not consider this as a large change compared to the previous work.

- In section 4.3, it is mentioned that the method is initially trained using the entire trajectory similar to Fried et al. (2018). This makes it even more similar to the method of Fried et al. (2018) since it does not just use the local information as mentioned in the introduction.

- Figure 3 and its description in Section 5.4 are very confusing. What are different colors in Figure3? Why is the vertical axis labelled timestep? Isn't it supposed to be 1-TENT? The caption does not make it more clear either. It would be good to clarify.

- Two qualitative examples in Figure 3 are not sufficient. Quantitative analysis over the entire dataset should be provided.

- The paper does not respect the page limit. Section 5.3 is about a figure that appears in the appendix.

**Score Justification**

The idea of the paper is not novel. The proposed method does not provide strong results either (it is outperformed by some other methods). Therefore, I cannot justify acceptance for this paper.

**Post-rebuttal comments**

I read the rebuttal and other reviews. I am still not convinced that there is a big difference between this method and speaker-follower of Fried et al. The speaker model cannot be used for navigation by itself but the learned distribution is used to adjust the actions taken by the follower. The only difference between these two methods is that one uses the entire trajectory and the other one uses partial trajectories up to the current time step (pre-trained on the entire trajectory though). Also, as mentioned by another reviewer, some analysis should be provided about why the generative model works better. Due to these issues, I keep my original rating. 


review_title: Lack of novelty; Mediocre results

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper focuses on learning a navigation policy for a vision-and-language navigation problem. In this problem, the agent are given a language instruction and are asked to follow the instruction to navigation in a simulated 3D room. Unlike baselines which maximize the probability of selecting an action given an instruction, the authors proposed to apply the Bayes rule to maximize the probability of generating the instruction given an action. The authors claim that this gives better generalization in unseen environments.

Pros:
While the idea of using a critic model (to compute the probability of the instruction given the trajectory) for VLN tasks [1,2] is not new, existing works takes the whole trajectory as input to the critic model, yet this paper takes the trajectory up until time t as input. The idea is new.

Cons:
The authors didn't explain well why the proposed generative model is better than the discriminate model. The empirical results shows so, but there aren't enough discussion to explain why.
Experiments are somewhat less satisfying, especially when the results in Table 2 are Gen.+Disc. with backtracking instead of just Gen.+Disc.

Detailed Comments:
It is intuitive that Gen. + Disc. works well than Gen. or Disc. alone. But why Gen. performs better than Disc.? The statement that Gen. works better than Disc. is the main statement and main contribution of this paper. However, there is very little discussion on why this is true methodologically (not empirically). Also, I'd assume that it is extremely hard to learn p(X | a_t, h_t) when t is small (at the beginning of the trajectory). I don't see analysis on how well the instructions are predicted.

Table 1 shows the performance of both Gen. + Disc. and Gen. + Disc. + backtracking. However, In Table 2, the performance of Gen. + Disc. on test (unseen) is not shown. This is less satisfying. Meanwhile, in test (unseen), with backtracking, Gen. + Disc. achieves the same success rate compared with FAST (which also do backtracking), so the performance gain seems small.

The paper misses implementation details. For example, how p(X|a_t, h_t) is modeled is not shown. The authors say "We use the network architecture of the speaker from (Fried et al., 2018) to implement generative policies which include a language model p(X|at, ht)." But the paper is not self-contained. It is better to at least describe the model in appendix. 

The paper say "We finetune each policy using imitation learning". However, imitation learning is a very board term, not necessarily mean the student-forcing implied by the paper. It is better to use a more concrete terms.


[1] Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
[2] Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks

review_title: Reasons of why the proposed method improves the results is not well explained. 

Meta-review: The authors propose to take a token-level generative approach to the task of vision-language navigation (R2R/R4R). The reviewers raise a number of concerns which should be noted in the final version of this work.  The primary concern revolves around generality.  How will this approach generalize to more sophisticated generative and discriminative models?  To what extent is the model relying on the short instruction/action sequences to succeed and would not perform well on longer instructions, longer trajectories, or more abstract language.  Finally, the discussion of the uninformed prior is interesting because while "clean", reviewers note there is no realistic grounded language scenario in which an uninformative prior makes sense.
Decision: Accept (Poster)