Paper:
SELF-MONITORING NAVIGATION AGENT VIA AUXIL- IARY PROGRESS ESTIMATION
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments. This challenging task demands that the agent be aware of which instruction was completed, which instruction is needed next, which way to go, and its navigation progress towards the goal. In this paper, we introduce a self-monitoring agent with two complementary components: (1) visual-textual co-grounding module to locate the instruction completed in the past, the instruction required for the next action, and the next moving direction from surrounding images and (2) progress monitor to ensure the grounded instruction correctly reflects the navigation progress. We test our selfmonitoring agent on a standard benchmark and analyze our proposed approach through a series of ablation studies that elucidate the contributions of the primary components. Using our proposed method, we set the new state of the art by a significant margin (8% absolute increase in success rate on the unseen test set). Code is available at https://github.com/chihyaoma/selfmonitoring-agent.

Review:
This paper describes a model for vision-and-language navigation. The proposed
model adds two components to the baseline model proposed by Fried et al. (2018):
- a panoramic visual attention (referred to in this paper as "visual--textual
  co-grounding"), in which the full scene around the agent's current position is
  attended to prior to selecting a direction to follow
- an auxiliary "progress monitoring" loss which encourages the agent to to
  produce textual attentions from which the distance to the goal can be directly
  inferred
The two components combine to give state-of-the-art results on the Room2Room
dataset: small improvements over existing approaches on the "-seen" evaluation
set and larger improvements on the "-unseen" evaluation sets. These improvements
also stack with the data-augmentation approach of Fried et al.
I think this is a reasonable submission and should probably be accepted. However, I
have some concerns about presentation and a number of specific questions about
model implementation and evaluation.
PRESENTATION AND NAMING
First off: I implore the authors to find some descriptor other than "self-aware"
for the proposed model. "Self-aware" is an imprecise description of the agent in
this paper---the agent is specifically "aware" of its visual surroundings and
its distance from the goal, neither of which is meaningfully an aspect of
"self". Moreover, self-awareness means something quite different in adjacent
areas of cognitive science and philosophy; overloading the term in the specific
(and comparatively mundane) way used here creates confusion. See section 3.4 of
https://arxiv.org/abs/1807.03341 for broader discussion. Perhaps something
like "visual / temporal context-sensitivity" to describe what's new here? A bit
clunky, but I think it makes the contributions of this work much clearer.
As suggested in the summary above, I also think "visual--textual co-attention"
is also an unhelpfully vague description of this aspect of the contribution. The
textual attention mechanism used in this paper is the same as in all previous
work on the task. Representations of language don't even interact with the
visual attention mechanism except by way of the hidden state, and the salient
new feature of the visual attention is the fact that it considers the full
panoramic context before choosing a direction.
MODELING QUESTIONS
- p4: $y_t^{pm}$ is defined as the "normalized distance from the current
  viewpoint to the goal". Is this distance in units of length (as defined by the
  simulator) or units of time (i.e. the number of discrete "steps" needed to
  reach the goal)?
  The authors have already clarified on OpenReview that the progress monitor
  objective uses an MSE loss rather than a likelihood loss. Do I understand
  correctly that ground-truth distances are in [0, 1] but model predictions are
  in [-1, 1]? Why not use a sigmoid? Also, how does scoring beam-search
  candidates as $p_t^{pm} \times p_{k,t}$ work if $p_t^{pm}$ can flip the sign?
- The input to the progress monitor is formed by concatenating the attention
  vector $\alpha_t$ to a vector of state features, and then multiplying by a
  fixed weight matrix. How is this possible? The size of $\alpha_t$ varies
  depending on the length of the instruction sequence. Are attentions padded out
  to the length of the longest instruction in the training set? If so, how can
  the model learn when it's reached the end of a short instruction sequence?
  What would happen if the agent encountered a sequence that was too long?
EVALUATION QUESTIONS
- The progress monitor is used both as an auxiliary training objective and as a
  beam search heuristic. Is it possible to disentangle these two contributions?
  (E.g. by ignoring the scores during beam search, or by doing augmented beam
  search in a model that was trained without the auxiliary objective.)
- Not critical, but it would be nice to know if the contributions here stack
  with the pragmatic inference procedure in Fried et al.
- While, as pointed out on OpenReview, it is not required to include SPL
  evaluations, I think it would be informative to do so---the preliminary
  results with no beam search look good!
MISCELLANEOUS
p1: "without a map" If you can do beam search, you effectively have a map.
p1: "...smoothly" What does "smoothly" mean in this context?
p2: "the position of grounded instruction can follow past and future
    instructions". Is the claim here that if instructions are of the form "ACB"
    and the agent is supposed to do "ABC", that the proposed model will execute
    these instructions successfully and the baseline will not? This claim does
    not appear to be evaluated anywhere in the body of the paper.
p4: "intelligently prunes" "Intelligently" is unnecessary.
p4: "for empirical reasons" What does this mean?
p5: "Intuitively, an instruction-following agent is required..." The existence
    of non-attentive models that do reasonably well at these
    instruction-following tasks suggest that this is not actually a requirement.