SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation
Authors: Abhinav Moudgil
Arjun Majumdar
Harsh Agrawal
Stefan Lee
Dhruv Batra
Venue: NeurIPS 2021 Poster (2021)
Keywords: Vision-and-Language Navigation
Multimodal Transformers

Abstract: Natural language instructions for visual navigation often use scene descriptions (e.g., bedroom) and object references (e.g., green chairs) to provide a breadcrumb trail to a goal location. This work presents a transformer-based vision-and-language navigation (VLN) agent that uses two different visual encoders -- a scene classification network and an object detector -- which produce features that match these two distinct types of visual cues. In our method, scene features contribute high-level contextual information that supports object-level processing. With this design, our model is able to use vision-and-language pretraining (i.e., learning the alignment between images and text from large-scale web data) to substantially improve performance on the Room-to-Room (R2R) and Room-Across-Room (RxR) benchmarks. Specifically, our approach leads to improvements of 1.8% absolute in SPL on R2R and 3.7% absolute in SR on RxR. Our analysis reveals even larger gains for navigation instructions that contain six or more object references, which further suggests that our approach is better able to use object features and align them to references in the instructions.

SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation
Abhinav Moudgil's * Arjun Majumdar', Harsh Agrawal', Stefan Lee', Dhruv Batra 1 Georgia Institute of Technology. Oregon State University
Abstract
Natural language instructions for visual navigation often use scene descriptions (e.g., "bedroom") ') and object references (e.g., 'green chairs') to provide a bread- crumb trail to goal location. This work presents a transformer-based vision-and- language navigation (VLN) agent that uses two different visual encoders - a scene classification network and an object detector which produce features that match these two distinct types of visual cues. In our method, scene features contribute high-level contextual information that supports object-level processing. With this design, our model is able to use vision-and-languaae alignment between images and text from large-scale web data) to substantially improve performance on the Room-to-Room (R2R) I and Room-Across-Be -Rom approach leads to improvements of 1.8%
pretraining (i.e., learning the
(RxR) [2] benchmarks. Specifically, absolute in SPL on R2R and 3.7% absolute in SR on RxR. Our analysis reveals even larger gains for navigation instructions that contain six more object references, which further suggests that our approach is better able to use object features and align them references in the instructions.
Introduction
The vision-and- lnn--angu navigation (VLN) task [ requires an agent to follow a path through an environment that is specified by natural language navigation instructions. A central component of this task associating (or grounding) the instruction to visual landmarks in the environment. Figure [ provides an illustrative example from the Room-to-Room (R2R) dataset []] 'Exit the bedroom and turn left. Continue down the hall and into the room straight ahead and stop before the desk with two green chairs.' The visual landmarks in this instruction include and 'hall") and specific object references (c.g., 'desk' and 'two green chairs'), )) To be successful, VLN agent should be able to (a) recognize and (b) ground both types of visual cues.
descriptions (e.g., 'bedroom'
Learning the appropriate grounding between referring expressions in an instruction and the corre- sponding visual regions is difficult in VLN due to the limited visual diversity example, the Room-te to-Room (R2R) [ and Room-Across- Across-Room (RxR) [2] datasets only use 61 unique training environments, models simply cannot learn about the long-tail of visual cues that appear in new testing (or validation) scenes. To address this challenge. recent work has shown the promise of transferring visual grounding with multimodal representation models that pretrained on a large amount of image-text web data before finetuning on the embodied VLN task 3 In this work, build on this general approach.
in training. For
For visual recognition, most VLN methods [| 4|44 first encode observations with a convolutional network that was trained to solve image-level classification task either using ImageNet [15] or the Places [6] scene recognition dataset. While ImageNet features may identify objects mentioned in the instructions and Places features might match the scene descriptions, neither solution was
Conrespondence 1bbinavmoudgg1188@maa....
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

scene descriptions
object references
model
bedroom
hall
desk
green chairs
actions
Multimodal Transformer
instructions observations
Navigation Instructions: "Exit the bedroom and turn left. Continue down the hall and into the room straight shead and stop before the desk with two green chairs."
Figure I: Illustrative VLN Instruction. Visual landmarks in the instruction include scene descriptions (e.g., 'bedroom' and 'hall") and object references (c.g., 'desk' and "green chairs') ) Unlike most previous methods,
model uses both scene (Á) and object features (O) match the visual references in the instructions. We employ a novel attention mechanism and view aggregation strategy to select the most relevant features for action prediction.
explicitly trained to recognize both types of visual cues, which is a limitation that we address in this paper. Furthermore, pretrained multimodal representation models have typically been trained with features from object detection networks rather than the image-level representations commonly used in VLN. Despite this, prior work leveraging these models [4] has continued to use the standard set of image-level features - leading to significant domain shift between pretraining and VLN finetuning.
In this work, we address these issues 1) by using multiple visual encoders to explicitly encode the inductive bias that the world is composed of objects and scenes and 2) by using object-level features from a detection network that match the features used pretrained multimodal representation models. The starting point for our work is a VLN BERT 4 agent that processes scene-level features from Places []6] CNN with transformer-based [1 multimodal representation model that is modified with a recurrence mechanism designed for the VLN task. Our work extends this approach by including object features as an additional input to the multinodal processing. However, we find that simply adding object features to VLN BERT does not improve (but rather slightly reduces) performance. Thus, we propose architectural changes that allow the model to take better advantage of these two distinct types of visual information. Specifically, we change the attention pattern within the the transformer to effectively freeze the scene representations and focus the processing on the object-level inputs. The result is new VLN agent that produces contextualized object representations by using scene features as high-level contextual cues.
We experiment with our proposed approach on the Room-to-Room (R2R) I and Room-Across- (RxR) [2] datasets. Empirically, we find that our model substantially improves VLN performance over our VLN BERT baseline on R2R and outperforms sate-of-thee-a methods on English language instructions RxR. Specifically, our proposed approach improves success weighted by path length (SPL) on the unseen validation split in R2R by 1.8 absolute percentage points. On RxR a more challenging dataset due to indirect paths and greater variations in path length we see even larger improvements. Success rate (SR) improves by 3.7 absolute percentage points, alongside gain of 2.4 absolute percentage points on the normalized dynamic time warping (NDTW) metric. Through ablation experiments we find that (consistent with the observations in (3] vision-and-languagg pretraining is vital to our approach, which suggests that strong visual grounding is key for using object- level features in VLN. Additionally,
RxR instructions that include six more object references (i.e., object-heavy instructions), our method has even larger improvements over VLNØBER of 7.9 absolute percentage points in SR.
To summarize, we make the following contributions:
We propose a scene- and object-aware transformer (SOAT) model vision-and-languagg navigation that uses both scene-level and object-level features explicitly encoding the inductive bias that the world is composed of objects and scenes. Our model uses a novel at- tention masking technique and view aggregation strategy, which both improve performance.

---

We demonstrate that our approach outperforms strong baseline approach by 1.8 absolute percentage points SPL on R2R and by 3.7 absolute percentage points SR on RxR.
We show that our method has significantly stronger performance on instructions that mention six or more objects (7.9 absolute percentage points of improvement in SR), which further suggests that our model is better able to recognize and ground object references.
Related Work
Ussionaand-Languaaa Navigation. The Room-to-Room (R2R) and Room-Across-Room (RxR) [2] datasets both situate the VLN task within Matterport3D [8] indoor environments. Since the release of R2R there has been steady improvement in VLN task performance [4H4]. Some of the key innovations include using instruction-generatii via 'speaker' models for data augmenta-
tion [5] TO), combining imitation and reinforcement learning [6], using auxiliary losses 13 and
different pretraining strategies 4 12 [4]. All of these methods have one thing in common they process visual observations with single convolutional network pretrained to solve an classification task (using either the ImageNet [I3] or Places [16] datasets). In contrast, this work explores using combination of features from visual encoders pretrained for scene classification and object detection.
Object Detectors in VLN. Intuitively, object detections should naturally match the object cues (e.g., 'green chairs') mentioned VLN instructions. Indeed, several recent studies K 19 [9 have demonstrated the utility of using object detectors for VLN. In labels from an object detector are encoded using a GLoVe [22] embedding. Similarly, 21 convert detections into feature vector using the classification label, object area, and detector confidence. Unlike these methods, our model directly uses object features produced by a detector, which provide richer, high-dimensional representation of each region. Additionally, our approach takes advantage of assion-and-languag pretraining, which eases the burden of learning the grounding between natural language and object representations from scratch using only VLN data, as is done in these prior methods. In 3 object features are used in a model that solves a path selection task in VLN, which requires pre-exploring an environment before executing the navigation task. By comparison, this work focuses on navigating without pre-exploration.
[99 20) object classification
3 Preliminaries: VLN, Visual Encoders, Multimodal Transformers
This section reviews the vision-and- lnnoo-an-- navigation (VLN) task and describes how multimodal transformers are used in the recently proposed VERT 4 model. Our approach builds on VLNÓ BERT and described in Section 4
3.1 Vision-and-Languuag Navigation
In VLN, agents are placed in photo-realistic 3D environment and must navigate goal location that is specified through natural language navigational instructions I (illustrated in Figure [ At each timestep the agent receives set of panoramic observations O, { composed of RGB images from 36 viewing angles (12 headings x elevations). We follow the VLN with known navigation graph setting [I 2] agents have access graph that specifies set of navigable locations from each viewpoint in the environment. We agree with the limitations of this setting discussed in (33. However, we report results for the nav-graph setting to be consistent with the long line of prior work in this area. We plan on generalizing to continuous environments in the future. Using the nav- nav-graph, agents select an action from the set A₁ { consisting of N, navigable locations and the stop action. The agent successful if it calls stop within 3m of the goal location.
136
3.2 Multimodal Transformers
Here we provide brief overview of multimodal transformers (e.g., OSCAR 24
)), which provide a basis for the VLN BERT architecture. Multimodal transformers are an extension transformer- based 17 language models such BERT (55 that process image-text pairs. As in BERT, the text input tokenized, encoded with learned embedding, and then combined with positional information (i.c.. word order). Commonly, the visual input (i.e., the image) is first preprocessed by object
3

---

(a) VLNÓ BERT
(b) Proposed Approach
prodiction
state pdo
Multimodal Transformer
stree spdece
Multimodal Transformer
Vestuclose
canckilete sinw iepreseetattinnn
state
Vesmussice
canckiatt sina reponceetiiii (SCATe and offecr Seaturee)
(ccone frenurer onty
StAte Eosler
cokarne
Scaee feature
piyect feetines
Figure 2: Model architectures. (a) VLN BERT uses multimodal transformer to encode state history, instruction tokens and scene features from candidate views. The candidate view corresponding to the scene features with the highest attention score is chosen as the next action. (b) Our approach includes object featurey as additional input. In view aggregation, the visual feature (either object or scene) with the maximum attention score is selected represent each candidate view. For action prediction, the model chooses the candidate view with the highest attention score as the next action.
detector (e.g.. a Faster R-CNN 26 model trained on Visual Genome (7 to produce a set of region features that combined with spatial information (e.g., the offset to the bounding box in the image). To summarize, the multimodal input consisting of L word tokens [Wi,..., wL) and M image regions fri,..., can be written as
[CLS] W1, wi...., WL, [SEP], Pi....,,.....
where [CLS] is special token used as global representation of the input and the [SEP] token separates modalities. This set of multimodal inputs are fed to a series of transformer encoder layers [17 to produce contextualized representations for each input using attention- based processing.
3.3 Multimodal Transformers for VIN
The input interface for multimodal transformers be easily adapted to process the language instructions and panoramic image from each timestep in VLN. However, VLN requires sequentially following navigation instructions (e.g.. 'Exit the bedroom...' then 'Continue down the hall..' then ssopp before the desk with two green chairs.). Accordingly, maintaining a history of the agent's state is helpful for understanding which sub-instruction to follow each timestep. Traditional VLN agents use recurrent neural networks (c.g., LSTM (28) to model state history. In contrast, ILNOBERT [4) introduces generic recurrence mechanism that can, in principle, be added to any multimodal transformer model to refashion it for the VLN task. In this work we extend the OSCAR 24 instantiation of BERT.
As shown in Figure 2. VLN BERT 44] uses a multimodal transformer to process word tokens from the navigation instructions and scene features for a set of "candidate" views from panoramic observation. There is one candidate view for each navigable action in A,, corresponding with the views that best align with each navigable direction (there are -4 such views average). Accordingly, the output representation for each view is used to compute the probability of moving in that direction. The recurrence mechanism in VLN BERT operationalized using a state token 8t, which is fed as an input to the multimodal transformer and is updated at each timestep.
Initialization. The state token 8, is initialized by passing word tokens from the navigation instruction along with special [CLS] and [SEP] tokens through the multimodal transformer. The output representation for the [CLS] token is used to set so and the outputs for the word tokens (denoted as (I)) are used as the language representation during navigation.
Visual Inputs. For each RGB image in the N₂ candidate views, VIN BERT runs convolutional network trained on Places [16] to extract high-level scene features. To facilitate the stop action an all-zeros feature vector is added to the set of visual inputs. We denoted this set of visual inputs as V

---

%
V
D0O AAA 0000
V
Transformer Layer
A scene features
instruction
cbject fealunea
attention
Figure 3: Selective attention: during navigation the instruction and scene features are kept frozen ie., they are not updated in the transformer layers and just serve as keys and values. The state token and object features updated through self-attention by attending to all input tokens.
Navigation. At each timestep, the input to the multimodal transformer is composed of the state token 8,, the encoded instruction ei( ), and the visual inputs V.. To reduce computation, the instruction representation is not updated during navigation; the encoded word tokens simply serve as keys and values when processed by the multimodal transformer.
Actions are predicted using "state-conde contte-codiiioned" attention scores for the scene features. These scores are produced the final layer of the multimodal transformer. Concretely, let the final layer outputs for the state token 8, and scene features , be denoted (bs)) e 12 and vi( V)) [(N++1)xe xd where N₂ is the number of navigable locations and d is the model's hidden dimension size. State-conditioned attention scores calculate using scaled dot-product
a Vi) = 1/( (b(8))
(1)
Vd
These scores ar V2) are normalized using the softmax function ā( V(V)) sof (oox(ax Vt)) (Ve))) the probability of moving to one of the N, navigable locations or stopping. Finally, the state history is maintained by constructing the next state token 8t+1 with the output representation for the state token (s;) using 'refinement" procedure detailed in the Appendix.
and used
Approach
Motivated by the observation that VLN instructions often scene descriptions (e.g., 'bedroom" ") and object references (e.g., 'green chairs') ") to provide visual cues. our goal is to design a model that effectively uses scene and object features for VLN. One straightforward idea is add an object feature input to I [4]. However, we find that this simple solution does not improve VLN performance (as we show in Table 3 row 2), which is perhaps why object features were not used for navigation in 4 Thus. we redesigned the VLN BERT architecture to create scene- and object-aware transformer (SOAT) model that benefits from access to both scene and object features.
First, we modify the processing in the model to focus on object features by designing the attention mask shown in Figure 3 and described in Section 4.2 The mask only allows the state token and object features to be updated by the multimodal transformer. Second, as illustrated in Figure 2 the output from our model includes scene and object representations for each candidate view. Thus, we design a simple approach (discussed in Section 4.3 to aggregate this information into a single representation for each view, which is used for action prediction.
4.1 Visual Inputs
To generate scene and object features we use two different visual encoders to process RGB images from the panoramic observations. Matching VINÓ BERT, we use a CNN trained on the Places [16] dataset extract scene features for each candidate view, and use all-zeros feature vector for the stop action to produce the set of scene features V.. For object features, we use the same object
Object features used in 4 to select objects for referring expressions task 29 but not for navigation.
5

---

detector that was used to pretrain the multimodal transformer (i.e., OSCAR (24) that we use to initialize our model - avoiding any additional domain gap between pretraining and fine-tuning. Specifically, we use Faster R-CNN [26] detector trained on Visual Genome 27 using the training procedure from B0) We denote the set of object features for detections from all of the candidate views as G, Note that for given image the detector may or may not detect any objects. As result, in our model, each candidate view is represented with one scene feature and zero or more object feature vectors. As illustrated in Figure the full set of inputs to our model include the state token 8z, encoded instructions y (f), and the visual inputs consisting of scene V. and object G, features.
4.2 Selective Object Attention
Simply adding object features as an additional input to the model does not improve performance. To overcome this challenge, we adjust the processing to focus on the object feature inputs. This is accomplished using the attention mask show in Figure 3 (right). Like VLNØ with this atten- tion mask, the encoded navigation instructions s5(I) are not updated in the multimodal transformer. However, in our approach the scene features V. are also kept frozen. As a result, these inputs only serve as keys and values during the attention based processing. The state token 8; and object features models.
G. operate as queries, keys and values, which is standard in transformer-bass
Intuitively, this attention mask allows scene features provide contextual information support refining the object representations. Note that the (unaltered) scene features are still used for action prediction (described in Section 4.3), which provides flexibility in how the model uses both types of visual representations. We hypothesize that this design leads to more effective transfer learn- ing (demonstrated in Section [ because the object-centric processing more closely matches the pretraining setup in which the model only operates on language and object features.
4.3
View Aggregation
At each timestep of navigation, the multimodal transformer uses selective object attention (described above) to produce contextualized representations of the state (s,) and object features ei (G,). As illustrated in Figure Pp. the object representations (G,) alongside the unaltered scene features Ve are used for view aggregation and action prediction. Specifically, state-conditioned attention scores calculated for the scene and object features using scaled not-product (as in Equation (I)
V Vd
3 1 (G.)²
Vi
(G.))
vd
where d is the model's hidden dimension size. As result, each candidate view is represented with one scene feature attention score and zero or more object feature attention scores.
For view aggregation, we select the maximum attention score for all of the scene and object features in a given view, and use the corresponding visual input to represent that view. Intuitively, this approach allows the model to either select relevant object (which may be mentioned in the instructions such as 'green chairs") or the full scene (which might match scene descriptions such as 'hallway') to represent each navigable viewpoint (i.e., candidate view). Finally, for action prediction, take the softmax over the selected attention scores and use these normalized scores to represent the probability of moving to each navigable location stopping.
5 Experiments
Datasets. We evaluate our method on the Room-to-Room (R2R) T and Room-Across-Ronn Room (RxR) [2] datasets. R2R is built using Matterport3D (MP3D) 18 indoor scenes and contains 21,567 path-instruction pairs, which are divided into four splits: training (14,025), val-seen (1,020), val-unseen (2,349) and test-unseen (4,173). Val-seen uses environments from the training split but the path-instruction pairs are novel. Val-unseen and test-unseen use new environments and new path-instruction pairs to evaluate generalization performance. We augment the R2R training data with IM instructions generated by the speaker model from [4]
RxR 2 is recently introduced multi-lingual VLN dataset that also uses MP3D scenes. It contains 126K path-instruction pairs in 3 languages (Hindi, English and Telugu). English instructions are collected from two regions: India (en and US (en-US). Since our model is pretrained on English

---

Table 1: R2R and RxR results on val-unseen. Rows 1 and 2 indicate performance of the random and human baselines. Rows 3-7 are results from prior state-of thrr methods. Rows 8 and 9 provide results of BERT 4 and our method in the same setting. f indicates reproduced results.
Pretraining
R2R
RxR
Methods
Web VLN
TL
NE
SR 1
SPL 1
NE
SR
SDTW 1
NDTW
Random
9.77
9.23
9.5
3.8
27.6
Human
1.32
90.4
74.3
77.7
RxR baseline
32
10.1
25.6
20.3
41.3
EnvDrop 10
10.70
5.22
52
48
PREVALENT
10.19
4.71
58
53
VLNØ BERT
(init. OSCAR)
11.86
4.29
59
53
7
VLN BERT
(init. PREVALENT)
12.01
3.93
63
57
8 BERT
(init. OSCAR)
12.16
4.40
58
51
7.31
40.5
33.0
52.4
Ours
12.15
4.28
59
53
6.72
44.2
36.4
54.8
language data, we focus on the English language subset of RxR (both en-IN and en-US), which includes 26,464 path-instruction pairs for training and 4,551 pairs in the val-unseen split.
Evaluation. We follow the standard evaluation protocols for R2R and RxR as shown in Table When applicable, use
indicate higher is better and J to indicate lower is better. On R2R, we report: Trajectory Length (TL), Navigation Error (NE L) average distance between the target and the agent's final position, Success Rate (SR 1) percentage of trajectories in which agent stopped within 3 meters of the target, and Success weighted by normalized inverse of Path Length (SPL t). On RxR dataset, in addition the NE and SR metrics, we also report the Normalized Dynamic Time Warping (NDTW t) and Success weighted by normalized Dynamic Time Warping (SDTW 1) metrics, which explicitly measure path adherence. Additional details included in the Appendix.
Baselines. In Table I we compare our approach (Row 9) with recent state-of-the-an methods. EnvDrop [10] (Row 4) trains an encoder-decoder model [5] on augmented data (in addition to R2R training data) generated with back- translation using "dropped out" scenes in order to generalize well unseen environments. PREVALENT [14] (Row 5) builds EnvDrop by pretraining multimodal transformer model on augmented data. After pretraining. PREVALENT feeds contextualized word embeddings from this pretrained transformer into the EnvDrop [0O encoder to fine-tune on R2R dataset. VLN Ó BERT (Row 6-8) extends PREVALENT by directly fine-tuning the pretrained multimodal transformer for navigation. VLN uses two initializations OSCAR (Row 6) and PREVALENT (Row 7). For fair comparison, we also report results reproduced with the released implementation of VLNÓ BERT by training from the OSCAR initialization (Row 8). Importantly, all these methods use either scene or object features as visual inputs for navigation.
Implementation Details. We implemented our model in Py 31 and trained on single Nvidia TitanX GPU. Consistent with VLNÓ BERT [4), we initialize our model with a pretrained OSCAR model 24 For both R2R and RxR, we fine-tune with behaviour cloning and reinforcement learning (RL) objectives adapted from prior work [4] 10 14. As in 4. 50% of each batch consists of behaviour cloning rollouts and 50% from RL (policy gradient). We train with constant learning rate of le-5 using the AdamW V optimizer with a batch size of 16 for 300k iterations. Like VLN O BERT, extract scene features with ResNet- 152 model (32 pretrained
the Places dataset (16). For object features, we extract detections from a pretrained bottom-up attention model (30], and adopt the filtering procedure from (3) discard redundant detections. In all of our experiments we use the same hyperparameters for the VLNO BERT bascline and our approach.
5.1 Main Results
Performance R2R and RxR. Rows 8 and 9 of Table I provide comparison with the VLNÓ BERT baseline on R2R (left) and RxR (right). RxR is a more challenging dataset because there is more variation in path lengths and agents often need follow indirect paths goal. On RxR, our approach (row 9), which uses both scene and object features, beats the scene features only VLNÓ BERT baseline (row 8) by 3.7% on SR, 3.4% SDTW and 2.4% on NDTW. On R2R, our approach (row 9) provides a 1.8% % gain in SPL over the VLNO BERT baseline (row 8). The larger
VER-BERT
7

---

Table 2: R2R test results. indicates reproduced results.
Pretraining
R2R Test Unseen
Mcthods
Web
VLN
TL
NE
SPL
1 Random
9.89
9.79
1.61
3 EnvDrop 10
11.66
5.23
51
47
PREVAL (init, OSCAR)
5.30
VLNÓ BERT
4
12.34
4.59
53
VLNÇ BERT
4
(init. PREVALENT)
12.35
4.09
63
57
7 VLNO BERT
4
(init. OSCAR) f
12.78
4.55
58
52
8 Ours
12.26
4,49
58
53
gains on RxR may result from a greater number of object references in RxR instructions (which is suggested by the linguistic analysis in (2), which are required describe the more complex paths. This hypothesis is consistent with our analysis in section 5.2 that shows larger gains from our method on instructions that contain the benefit of using object features for challenging datasets like RxR.
more object references. These significant improvements highlight
Table 3-7 show the results for ofute-of-the-ar methods R2R and RxR. On RxR, our method outperforms the previous state-of-the-ar established by the RxR baseline (row 3) by 13.5 absolute on NDTW and 18.6% absolute SR. On R2R, our approach is competitive with the previous state- of-the-art across all metrics. Table T results on the R2R test-unseen split, which again shows that approach is competitive with prior work. In both tables VLN BERT with PREVALENT initialization outperforms our approach. However, we note that PREVALENT [4]] uses an alternative pretraining setup with scene features and augmented VLN data, which orthogonal to our goals of using object features and transferring visual grounding for VLN. Combining these two different pretraining strategies is an interesting direction for future work.
5.2 Ablations and Analysis
Table 3: Ablation Study on R2R. i indicates reproduced results.
Object Features
View Aggrogation
Selective Attention
Val Unseen
Models
SR 1
SPL
Buseline
57.90
51.43
57.26
50.96
57.85
51.73
4
57.73
52.46
57.60
52.06
Ours
58.71
53.24
Do object features help without the proposed architectural changes? Table 3 presents the results of ablation experiment R2R that demonstrates the importance of each design choice in our model. In row 2 object features are used as an additional input to VINÓ BERT with no other architectural changes (c.g., selective attention or view aggregation). In this ablation, the visual inputs (scene and object fcatures) all get refined with self-attention and action probabilities are calculated in similar fashion VLN BERT 4. We that merely using object features an additional input leads to slightly reduced performance using object features with either view aggregation (Section 4.3) or selective attention (Section 4.2) reverses this trend. Furthermore, row 6 demonstrates that the largest gains are seen when all three modifications to the VLNĆ baseline are used together resulting in a 1.8 gain in SPL. Additionally, row 5 shows that using selective attention without object features does not out perform our full approach. More details are provided the appendix. To summarize, using object features only leads to improved performance with the architectural changes proposed in this work.
val-unseen (rows vs. 2). Rows 3 and 4 demonstrate that

---

Table 4: Results on R2R without vision-andd and-language pretraining.
R2R Val Unseen
Models
TL
NE |
SR
SPL1
VLNÓ BERT
10.31
5.21
49.21
45.53
Ours
12.00
4.99
50.36
44.69
Do object features help without -siin-and-langgugee pretraining? Table + reports results of the VLNÓ BERT baseline and our approach without vision-and-languus pretraining on the R2R val-unseen split. As evident from Table both the approaches perform similarly across all metrics. This demonstrates that transferring visual grounding is crucial for our approach to work; proach improves performance compared to the I baseline (Table [ Row 7 vs Row 8) predominantly because it more effectively utilizes the large-scale saiion-nd-languuaa pretraining.
ap-
Table 5: Results on a subset of RxR instructions divided by the number of object references.
Object Heavy Instructions
Not Object Heavy Instructions
Models
NE
SR f
SDTW
NDTW 1
NE 1
SR 1
SDTW
NDTW
VENT 4
8.46
33.89
26.36
46.95
4.07
58.21
50.33
67.75
2
Ours
7.27
41.80
32.92
52.46
3.77
61.64
53.87
69.37
How does our method perform instructions that heavily mention objects? The RxR dataset contains long instructions that frequently mention objects and scenes. Specifically, RxR contains entity references per instruction average, which nearly double the amount of entities referenced in R2R (3.7) 2]. Here, we evaluate our model on a subset of RxR instructions that heavily contain object references. To find this subset, we first randomly pick 500 instructions from val-unseen split in RxR. Then, we extract noun entities from the instructions using a standard NLP parser [33]. Next, manually inspect these nouns and remove direction, scene, or other non-object references. This results in a list that only contains object references. Finally, we split the randomly selected 500 instructions into sets containing or more object object references (i.e., object heavy instructions) or less than 6 object references (i.e., not object heavy instructions). The threshold of 6 was selected match the average number of entity references in RxR instructions (2). With this process 354 of the 500 instructions were selected for object heavy set.
Results of our method and VLN BERT baseline these subsets reported in Table 5 On the object heavy subset, our approach gives even stronger performance improvements over VLN BERT baseline than seen on the full RxR val-unseen split. Specifically, we obtain an absolute improvement of 7.91 % for Success Rate (SR), 6.56% for SDTW score and 5.51 % for NDTW. These gains are approximately twice the improvement on val-unseen split and larger that the ~3% improvements on the complementary subset containing 146 instructions with less that object references (Table right). These results suggest that our approach able to exploit object features and appropriately align them with object references to follow visually grounded navigation instructions.
6 Conclusion
In this work, we propose novel scene- and object-aware transformer (SOAT) model, which uses scene and object features for vision-and-language navigation. We introduce a selective attention pattern in the transformer for processing these two distinct types of visual inputs such that scene tokens only provide contextual information for object-centric processing. As a result, our approach improves overall performance over strong baseline on the R2R and RxR benchmarks. Our analysis shows that aarton--nd-langgaaa pretraining is crucial for exploiting the newly added object features. We obtain even larger performance improvement on instructions that heavily mention objects, which suggests that our approach is able to effectively ground object references in the instructions to visual object features.

---

7 Acknowledgements
The Georgia Tech effort was supported part by NSF, ONR YIPs, ARO PECASE, Amazon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government, any sponsor.
References
[1] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauff Ian Reid. Stephen Gould, and Anton van den Hengel. Vision-and- -nn-aaugge navigation: Interpreting risually-grunnder navigation instructions in real environments. CVPR, 2018.
[2] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-Across- Room: Multilingual vision-and-languaal navigation with dense spatiotemporal grounding. In EMNLP, 2020.
[3] Arjun Majumdar, Ayush Shrivastava, Stefan Loe, Peter Anderson, Devi Parikh, and Dhruv Batra. Improving vision-and-languaae navigation with image-text pairs from the web. In ECCV, 2020.
[4] Yicong Hong. Qi Wu, Yuankai Qi, Cristian Rodriguez- Opazo, and Stephen Gould. A recurrent vision-an and- language bert for navigation, 2020.
[5] Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berk-Kirkp,, Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for vision-and-language navigation. In NewrlPS, 2018.
[6] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang. William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. CVPR, 2019.
[7] Chih- Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, and Caiming Xiong. Self monitoring navigation agent via auxiliary progress estimation. In /CLR, 2019.
[8] Liyiming Ke, Xiujun Li, Yonatan Bisk. Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, and Siddhartha Srinivasa. Tactical rewind: Self-correction via backtracking in "ssio-anddllnguage navigation. In CVPR, 2019.
[9] Chih-Yao Ma, Zuxuan Wu, Ghassan AlRegib, Caiming Xiong, and Zsolt Kira. The regretful agent: Heuristic-aided navigation through progress estimation. In CVPR, 2019.
[10] Hao Tan, Licheng Yu, and Mohit Bansal. Learning navigate unseen environments: Back translation with environmental dropout. In NAACL, 2019.
[11] Peter Anderson, Ayush Shrivastava, Devi Parikh, Dhruv Batra, and Stefan Lee. Chasing ghosts: Instruction following as bayesian state tracking. In NeurlPS, 2019.
[12] Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith, and Yejin Choi. Robust navigation with language pretraining and stochastic sampling. In EMNLP, 2019.
[13] Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. Vision-language navigation with self-supervised auxiliary reasoning tasks. CVPR, 2020.
[14] Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning generic agent for iisionaand-language navigation via pre-training. In CVPR, 2020.
[15] Deng. Dong, R. Socher, L.-J. Li, Li, and Fei-Fei. ImageNet: Large-Scale Hierarchical Image Database, CVPR, 2009.
[16] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: 10 million image database for scene recognition. PAMI, 2017.
[17] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkorcit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NewrlPS, 2017.
[18] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song. Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D D data in indoor environments. In 3DV, 2017.
10

---

[19] Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, and Kate Saenko. Are you looking? grounding multiple modalities in vision-and-language navigation. In ACL, 2019.
[20] Yicong Hong, Cristian Rodriguez, Yuankai Qi, Qi Wu, and Stephen Gould, Language and visual entity relationship graph for agent navigation. In NewrIPS, 2020.
[21] Yubo Zhang, Hao Tan, and Mohit Bansal. Diagnosing the environment bias in Asioo-anddlanguaae navigation. IJCAI, 2020.
[22] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Representation. In EMNLP, 2014.
Global Vectors for Word
[23] Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-language navigation in continuous environments, In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, ECCV, 2020.
[24] Xiujun Li, Xi Yin, C. Li, X. Hu, Pengchuan Zhang, Lei Zhang, Longguang Wang. H. Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. Oscar: Object-semantics aligned pre-training vision-language tasks. In ECCV, 2020.
[25] Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, In NAACL-HLT, 2019.
[26] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NeurIPS, 2015.
[27] Ranjay Krishna, Yuke Zhu Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJC,, 2017.
[28] Sepp Hochreiter and Jürgen Schmidhuher: Long Short- Term Memory. Neural Computation, 1997.
[29] Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environments. In CVPR, 2020.
[30] Peter Anderson, Xiaodong He, Chris Buchler, Damien Tency, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR, 2018.
[31] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison. Andreas Kopf. Edward Yang. Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner. Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high sesh-performance deep learning library. In NewrIPS. 2019.
[32] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
[33] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strenggh Natural Language Processing in Python, 2020.


Reviews:
[{"review_id": "e0RgV7gAYYc", "reviewer": "Reviewer_7N3z", "summary": "This paper proposes a transformer based VLN model based on VLNBERT, which leverages both object-level features (from a Visual Genome pre-trained object detector) and scene-level features (from a Places pre-trained detector). The intuition is that natural language instructions found in the VLN task will contain both references of specific objects in the scene (e.g. chair, sofa), as well as higher level references to entire rooms (e.g. kitchen, bedroom), and that using image features of both types will better the agent’s language grounding. They additionally propose a selective attention mechanism and a view-aggregation strategy. The selective attention only modifies the object-level features through the self-attention layers, keeping the scene-level and instruction features fixed, used for contextualizing the object-level features. The view-aggregation strategy, in my understanding, is a max-pool operation which uses the maximally scoring (compatibility score with the current state) object or scene feature from each view as the view’s representative for action selection. The paper presents a series of experiments comparing the proposed method to prior state-of-the-art in the R2R and RxR datasets, generally beating the presented baselines in the R2R validation-unseen and test splits (with the exception of VLNBERT pre-trained on VLN data) as well as the RxR validation-unseen split. A series of ablations are presented showing that the improved performance is due to the unison of the 3 proposed components rather than any single one alone, and that it rests on vision-language pretraining. Finally, they show that the performance improvement is even larger when evaluating on instances with more than 6 object references. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall: I think the proposed method and motivation behind it are intuitive and will be of interest to the research community. Aside from some minor concerns (detailed below), I think the experiments appear to validate the paper’s claims. I am in favor of acceptance. \n\nOriginality: To my understanding, the proposed method is novel and is an extension to already established methods. \n\nQuality: The provided experiments appear to be thorough and clearly show improvements over baselines, with the provided ablations justifying the need for all three proposed components of the method. The only baseline which appears to beat the method is one pre-trained on VLN data, which the model presented does not pre-train on. \nMy only concerns regarding the experiments are as follows: \n* It appears that VLNBERT pre-trained with PREVALENT is the only baseline which beats the proposed method. The Ours condition appears to have been initialized with OSCAR, so I’m wondering if any experiments were run evaluating the performance of the proposed method initialized with PREVALENT? L278-280 comment on this, but I feel like I still wasn’t 100% clear on this. Could the authors please elaborate on this point? More exposition in the paper about why this is orthogonal could make it clearer to readers. \n* Were the experiments run with only a single seed? I understand it may not be possible to compare standard deviations to other works which haven’t reported performance across seeds, but it would still be nice to see how stable the presented results are across random seeds. \n\nClairity: The paper is generally clearly written and easy to follow. Some presentation suggestions: \n* L26-27: “The VLN training set” appears to be referencing either R2R or RxR, whereas VLN generally implies the type of task to me. I would suggest referencing the specific dataset here to make this sentence clearer. \n* It’s not entirely clear to me what the different sections in the tables mean (e.g. rows 3-7 vs. 8-9 in Table 1), specifying this explicitly in the captions may make the distinction clearer to readers. \n* Number bolding appears a little inconsistent. For example, why are some TL numbers bolded and others not? Also got a little confused with multiple numbers being bolded in the same columns (e.g. Table 1), although this seems related to my confusion in the point above about table sections. \n\n* Significance: The proposed method is intuitive and appears to be easy to integrate into existing pipelines -- I believe this will be of value to the research community. \n\n\n# Post-rebuttal Edit\nThe response has addressed my concerns, it's also nice to see that performance is consistent across seeds. Following this and reading the other reviews / responses I have decided to keep my score at a 7.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "My only concern with limitations is listed above (about PREVALENT pretraining). I don’t believe the work raises any concerns for potential negative societal impact. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "468iOkT8Iu6", "reviewer": "Reviewer_2Xzi", "summary": "This paper presents a method for the vision-and-language navigation task, where an agent needs to navigate through a set of rooms to a goal location as specified by an instruction. What this paper focuses on is making use of object-features in addition to scene features in the visual input. Having both explicit in the architecture covers objects and scenes that are often used in the instructions and allows the method to leverage pre-training of these representations. The paper finds that simply adding the object features to the input alone does not improve performance, which instead requires additional changes to the architecture, i.e. selective attention and view aggregation. With these changes in place, the paper shows improved performance, especially for instructions that heavily refer to objects.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Originality\n\nThe paper builds on existing approaches to the vision-and-language navigation task based on multimodal transformers and extends them to take objects into account. It is clear how this paper differs from prior work and credit to earlier work is adequately given. The paper does an excellent job in clearly separating novel contributions from prior work. Well done! \n\nThe paper is somewhat incremental but I do not see this as a negative point. The proposed approach is an instantiation of a logical next step and provides valuable insights that others can build on in turn.\n\n# Quality\n\nThe submission seems technically sound. All claims are well supported and the used methods and baselines are appropriate. The work seems complete and the comparison to SOTA seems fair. \n\nThe main weakness I see in this work is that I do not feel confident in having understood the relative importance or the interaction of the three main contributions (object features, view aggregation, and selective attention). The ablation analysis (Table 3) progressively adds the three contributions to the baseline and evaluates these versions, but based on these results we can really only be confident that the last of the additions contributes to improved performance, as the other versions (lines 2-3) are not significantly better than the baseline. To get a better understanding of the influence of the other components, we would need all 8 combinations of using or not using a component (only 2x the experiments that are already done) – and if that is not possible, removing the components one at a time would be preferable over the chosen setup. In addition to verifying that each contribution is valuable, it would be important to show that they interact and that the view aggregation and the selective attention enable the use of objects as claimed by the authors. With the current results, one explanation for all the datapoints would be that the object input generally hurts performance (except in the heavy object case) but that the other contributions independently lead to performance improvements so that the overall performance is better than the baseline.\n\nThe differences between the seen and unseen environments in Table 3 also point to the fact that what is happening here might mainly be a question of proper regularization. If that is the case it might be less important which modes (instruction, scene, objects) are selected in the selective attention but rather the fact that attention is selective in some way. To verify whether this is the case, it would be good to try out other versions of selective attention as well as other approaches that would regularize the model.\n\n# Clarity\n\nThe submission is very clearly written and well organized. The background section (Section 3) provides enough context to make this work very accessible. With the clear description and the provided code, it should be straightforward to reproduce the work. Well done! \n\n# Significance\n\nThe paper presents a significant step in the vision-and-language navigation task combining the use of both the scene and objects. If all proposed components are properly analyzed (see above), this paper should enable follow-up work on this topic. Unfortunately, the current evaluation does not fully convince me that all proposed components are actually necessary or that they interact in a way that the performance improvement is mainly driven by object-awareness as claimed by the paper. This experimental analysis needs to be improved before I can recommend accepting this paper.\n\n# Post-Rebuttal EDIT\n\nMy main concerns have been addressed by improving the analysis of the proposed components.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No issues here.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "-i7LPvu6ksJ", "reviewer": "Reviewer_peFD", "summary": "This paper proposes an approach for a VLN system based on the Transformer architecture. Building off of existing work, this paper proposes to use both scene and object features of candidate actions (neighboring nodes in the graph). Finding that this doesn't result in improved performance on R2R, the proposed method aggregates the outputs of the Transformer for each candidate action across scene and object representations. The experiments include evaluation on R2R and RxR. There is also analysis of model performance on instructions which include a relatively large number of objects, and it is found that the proposed method has even larger gains on this data.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: Although the method is new, it seems to be a relatively minor extension of prior work (combining features that prior work only used independently, and a small architecture modification on top of it). The analysis focused on object-heavy instructions is interesting.\n\nQuality: The experiments compare against baselines on two datasets. I would have liked to see experiments on other VLN datasets that also require identifying many objects and scenes, e.g., Touchdown (Chen et al. 2019). Does the proposed approach extend to other environments? \n\nWere experiments ran for multiple trials? I would liked to see significance results on some of the tables. The performance improvements on RxR seem somewhat clear, but it's difficult to tell if the other results are significant. Also, I am not sure why PREVALENT was not experimented with: I would like to see if this method also holds with other pretrained settings, and is not just a result specific to OSCAR (otherwise, it would seem a very specific result with limited applicability). Ablating scene and object features independently would be interesting to see. \n\nClarity: The paper was easy to read and the method was clearly explained.\n\nSignificance: Because this method was evaluated only on one environment (two datasets), and seems to rely heavily on pretraining setup, it is a bit unclear to me how extensible it is to other VLN datasets. I am not sure what the general takeaway from this paper is besides the fact that using multiple sources of input which prior work did not combine before improves model performance if the architecture is slightly modified.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "In comparison to prior work there don't seem to be additional limitations, although the relatively limited experiments make me wonder about the approach's applicability to other environments. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for vision and language navigation based on transformers and leveraging information from object-level processing.

The paper received favorable reviews and a consensus emerged very quickly on acceptance, in particular taking into account a convincing rebuttal.

The AC concurs.
Decision: Accept (Poster)