Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary 
The paper proposes Visual Transformer Network which encodes the relationship between all detected object instances in a frame and uses it for navigation. The paper uses DETR for object detection and learn an association between local descriptors (from the object detector) with global descriptors (ResNet18) using the proposed VT model. They show that using VT improves performance on the object navigation task in AI2-THOR simulator compared to existing methods. 

Strengths 
- The paper proposed a novel transformer architecture that learns an association between local object descriptors with global image region features so that actions can be grounded to visual regions in the image. 

- Different from prior work, the paper uses all the objects detected for a label instead of just the most confident detection. 


Weaknesses 

- The paper doesn't fully address why DETR performs better than FasterRCNN features. Appearance features from FasterRCNN have been widely used for several downstream tasks in Vision and Language Navigation[1], Vision and Language tasks[2]. From the experiments, it's not clear why DETR is doing better than Faster-RCNN especially when the detection accuracy of DETR is also better than Faster RCNN.

- Additionally, I didn't fully follow how authors obtain the appearance features from Faster RCNN based method. The authors mention that object appearance features are extracted from different layers of a backbone network. How is it different from the approach taken by Bottom-Up, Top-Down[3] paper in which 2048-dim appearance features are extracted for each visual region?   

- The experimental setup isn't fully reflective of the object goal navigation task. The experiments are conducted in AI2 thor scenes which only contain one room. It's not clear, how this method will perform when evaluated on significantly more complicated environments like Matterport / Gibson [4]. Specifically, I am interested in how will the proposed architecture perform when the goal object is not in the same room as the agent. 

- The navigation task is also made simpler by discretizing into a grid. Single room environments and discrete grids simplify a lot of navigation-related challenges and the authors don't discuss how the proposed architecture will generalize to more complex object navigation tasks. 

- The use of spatial embeddings as well as appearance embedding isn't all that surprising. Existing work including Du et al. uses bounding box coordinates to help learn spatial associations between objects. 

Other questions: 
- Instead of pre-training without employing the navigation policy, did the authors try using shortest-path based demonstrations to help learn the navigation policy as well? In the first stage, the navigation policy learns using imitation learning and then finetuned with A3C? 

- What is the step size of the agent for the forward step? What are the turn angles for Turn-left, Turn-right actions? What are the tilt angles for look-up and look-down actions? 
 
- What's the reason for improvement over ORG (in absence of TPN). Is it superior visual representations (Faster RCNN vs DETR) or the fact ORG only chooses objects with the highest confidence while VT uses all the detected objects? 

- How does the agent learn long-term associations between objects across multiple frames. In my opinion, the proposed architecture puts all the burden of learning these long-term object relationships across multiple frames on the LSTM policy since the VT only learns association within a single frame. 

[1] Improving Vision-and-Language Navigation with Image-Text Pairs from the Web; Majumdar et al. 

[2] Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks; Li et al. 

[3] Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering; Anderson et al. 

review_title: Novel approach for learning visual representations for navigation; but weak experiments and explanations.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper demonstrates a model that uses the Transformer to encode the visual features that appeared in the visual input image during navigation. The model is firstly pre-trained under imitation learning objective with self-generated shortest-path trajectories. The empirical results show that the model used in the paper outperforms previous methods on AI2-THOR environment. The authors also show some studies on the contributions of each component in the model.

Paper strengths:

+ The proposed method further show that the Transformer is a powerful model for feature extraction

+ The authors demonstrate one method to make the training of Transformer work, i.e. pre-training transformers using shortest-path trajectories

+ Empirical result support the authors' claims.

+ A thorough ablation study and discussions are provided.

Cons:

- The paper adopts the Transformer and adapted it into the navigation problem. No new architecture/model is proposed.

- It seems that a similar usage of Transformer already appeared in the vision-and-language navigation task [1]. The paper also shows that pre-training of navigation tasks using Transformers can help to boost the performance. 

Minor: Two missing citations [2,3] that are potentially relevant. 

[1] Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training

[2] Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation

[3] Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation

--

I've read the authors' response and would like to maintain my original score
review_title: An empirical study of usefulness of pre-training for navigation with transformers

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### summary
This paper introduces transformer network to visual navigation, specifically, object-goal navigation. It also develops several new feature descriptors as the input of the transformer encoder and decoder. To properly train the whole model, a supervised pre-training stage is used to warm-up the transformer model. Great performance has been achieved on AI-THOR benchmark.

### pros
1. Lots of people must have thought to use transformer to replace the RNN/LSTM in lots of visual navigation framework. This paper provides a good example. Most importantly, this paper focus on the representation learning part of the whole pipeline, which isn't that straightforward of how to use a transformer.
2. The writing is mostly clear with clear motivation and background discussion. 
3. The performance boost, especially SPL, is relatively significant compared to previous SOTA, and the ablation studies have verified most of the design choices.

### cons
There are a couple of things which are not clear to me, or confused me when I was reading the paper:
1.  The writing in the approach section isn't very clear. First, it would be much better to define clear notations for all the features/descriptors, and use such notations in the figure. The current writing uses "instance feature", "global feature", "positional-global", "spatial feature", "spatial-enhanced"..., which are a little bit confusing to me. Second, I think most details are properly ignored in Fig.2. It becomes not as informative as the detailed version (Fig.4) in Appendix. Note that these two figures are not consistent that the "add" symbol for positional enhancement is missing is Fig.4. I also suggest that the positional embedding blob not crossing the arrow of global feature, they are just added together.Third, Sec.4.2 writes "we first reduce the channel dimension of a high-level activation map from D to a smaller dimension d", how the reduction is done exactly? From appendix it seems like a 256-dim vector is transformed into 249-dim. Fourth, $h$ and "w" are abused. In figure, they are annotated  on the long side of the tensor, in eq (1) they seem to be the output of positional embedding, and in Sec.4.2 description they are the resolution of 7. Similarly, $L$ is abused as it means input of encoder in Sec.4.1 but output of encoder in Sec.4.3. Let me stop here, but these things make the approach not super clear to me. 
2. In Sec.4.1, I'm not fully convinced of the statement of faster rcnn even thought the experiments empirically verified it. Faster RCNN w/o FPN only output features after "conv4+ROI-pooling" (ResNet-101-C4 variant). Why is it blamed for scale-sensitive? Actually, what does scale-sensitive mean here? Why DETR doesn't suffer from it? Honestly I don't think that's the reason why Faster RCNN performs worse.
3. Also, I'm not fully convinced of the statement of the "early stopping" in Sec.4.4. The penalties are the same for different model in RL,  why this transformer based representation learner suffers from "early stopping"? Is there a plausible explanation? It's fine that you cannot conclude something for sure because transformers are always hard to train, but the statement in paper reads not super convincing to me.
4. Sec.5.1 SPL formulation seems to be wrong? The success indicator seems missing? The current equation is simply a ratio between any episode length over the optimal length regardless whether it's an success episode or not.
5. Why not also adding global features into the transformer encoder? For example, reshape and concat with the input. Is the encoder supposed to be local?

### misc
1. The best results of VTNet in Tab.1 used TPN. It might be better to introduce TPN in Appendix for completeness. 
2. Variance is not reported in Tab.1, which is uncommon for RL/control paper. 
3. Because transformer has attention module and the relationship can be easily visualized. I was expecting more interpretation/visualization like Fig.1 right to show the proposed methods actually attend to proper areas. The numbers are hard to tell what do each modules do exactly.

### questions
1. Just to make sure I understand correctly, the instance feature (100x249) and spatial feature (100x7) are fed into a MLP for fusion? Can you describe the archi?
2. Local spatial feature contains the normalized bounding box, confidence and top-rated semantic label. Is the semantic label the class index (1,2,...,C)? why not use a one-hot embedding or something?
3. Is AI2-THOR the most popular benchmark for object-goal nav? I have seen lots of prior paper running on habitat. What's the specific reasons of using AI2-Thor over habitat? 

Please address my questions. I'm looking forward to discussing with the authors and the peer reviewers. 

review_title: applied transformer on visual navigation tasks; good performance achieved on AI2-THOR

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Paper summary**

The paper addresses the problem of navigation towards objects (ObjectNav) in a virtual environment. The idea of the paper is to incorporate spatial information of objects using a transformer-based framework called Visual Transformer Network. The paper compares the results with a number of state-of-the-art ObjectNav models and provides an ablation study. The results have been reported on the AI2-THOR framework.

**Paper strengths**

- The idea of incorporating object information using transformers for a navigation agent is new.

- The proposed method outperforms a number of strong baselines.

- The ablation studies show that the introduced components are effective.

**Paper weaknesses**

- It is hard to understand some parts of the paper. For example, the introduction discusses details such the difference between DETR and Faster RCNN or difficulty of training the transformers. It is difficult to understand these details without knowing the proposed method. The introduction should provide a high-level overview of the paper instead of these types of details. Also, the paper requires proof reading. There are several sentences with grammar issues.

- It is a bit strange that nothing is learned without the imitation pre-training. It would be good to dig deeper and provide a better explanation for why this happens.

- Equation 1 is not clear. A brief explanation would help.

- I recommend running the method on some other frameworks which include slightly larger scenes to see if the method generalizes to those as well. RoboTHOR (https://github.com/allenai/robothor-challenge) is very close to the framework used in this paper so it might be a good choice for these experiments.

**Justification of rating**

Overall, I am leaning towards accepting this paper since it introduces a new way for incorporating object information and it outperforms strong object navigation baselines. Writing is the main issue of this paper. 

**Post-rebuttal**

I read the rebuttal and the other reviews. The rebuttal addresses my concerns to some extent (writing has improved in the revised version, but it still has some issues). So I am going to keep my rating. 

review_title: Effective method, good results, but writing requires improvement

Meta-review: This paper addresses the problem of visual object navigation by defining a novel visual transformer architecture, where an encoder consisting of a pretrained object detector extracts objects (i.e. their visual features, position, semantic label, confidence) that will serve as keys in an attention-based retrieval mechanism, and a decoder computes global visual features and positional descriptors as a coarse feature map. The visual transformer is first pretrained (using imitation learning) on simple tasks consisting in moving the state-less agent / camera towards the target object. Then an RL agent is defined by adding an LSTM to the VTNet and training it end-to-end on the single-room subset of the AI2-Thor environment where it achieves state-of-the-art performance.

After rebuttal, all four reviewers converged on a score of 6. The reviewers praised the novelty of the method, extensive evaluation with ablation studies, and the SOTA results. Main points of criticism were about clarity of writing and some explanations (which the authors improved), using DETR vs. Faster R-CNN, and the relative simplicity of the task (single room and discrete action space). There were also minor questions, a request for more recent transformer-based VLN bibliography, and a request for a new evaluation on RoboThor. One area of discussion -- where I empathise with the authors -- was regarding the difficulty of pure RL training of transformer-based agents and the necessity to pre-train the representations.

Taking all this into account, I suggest this paper gets accepted.

Decision: Accept (Poster)