Landmark-RxR: Solving Vision-and-Language Navigation with Fine-Grained Alignment Supervision
Authors: Keji He
Yan Huang
Qi Wu
Jianhua Yang
Dong An
Shuanglin Sima
Liang Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Vision-and-Language Navigation
Fine-grained dataset
Cross-modal alignment
Reward Shaping
Re-initialization mechanism

Abstract: In Vision-and-Language Navigation (VLN) task, an agent is asked to navigate inside 3D indoor environments following given instructions. Cross-modal alignment is one of the most critical challenges in VLN because the predicted trajectory needs to match the given instruction accurately. In this paper, we address the cross-modal alignment challenge from the perspective of fine-grain. Firstly, to alleviate weak cross-modal alignment supervision from coarse-grained data, we introduce a human-annotated fine-grained VLN dataset, namely Landmark-RxR. Secondly, to further enhance local cross-modal alignment under fine-grained supervision, we investigate the focal-oriented rewards with soft and hard forms, by focusing on the critical points sampled from fine-grained Landmark-RxR. Moreover, to fully evaluate the navigation process, we also propose a re-initialization mechanism that makes metrics insensitive to difficult points, which can cause the agent to deviate from the correct trajectories. Experimental results show that our agent has superior navigation performance on Landmark-RxR, en-RxR and R2R. Our dataset and code are available at https://github.com/hekj/Landmark-RxR.

Landmark-RxR: Solving Vision-and-Language Navigation with Fine-Grained Alignment Supervision
Keji Hel.2
Yan Huang!.2
Qi Wu
Jianhua Yang
Dong An
Shuanglin Sima 1.2
Liang Wang
Center for Research on Intelligent Perception and Computing National Laboratory of Pattern Recognition
Institute of Automation, Chinese Academy Sciences
School Artificial Intelligence, University Chinese Academy Sciences SScoool of Computer Science, University of Adclaide
"School of Future Technology, University of Chinese Academy of Sciences
School of Artificial Intelligence, Beijing University of Posts and Telecommunications
6 "Center Excellence Brain Science Intelligence Technology (CEBSIT)
"Chinese Academy of Sciences, Artificial Intelligence Research (CAS-AIR) (keji. he, dong an, shuanglin. lnn..nnn... (ima))0rrtt ia.ac.c...oo (yhuang, vangliangl @nlpr. ia. ac
wu01 Cadelaide. edu youngjianhuaebupt.ced
Abstract
Assionandd-aagusee Navigation (VLN) task, an agent is asked to navigate inside 3D indoor environments following given instructions. Cross-modal align- ment one of the most critical challenges in VLN because the predicted trajec- tory needs to match the given instruction accurately. In this paper, address the cross- alignment challenge from the perspective of fine-grain, Firstly, to alleviate weak modal alignment supervision from coarse-grained data, fine-grained VLN dataset, namely Landmark-RxR.
introduce a human-annoter Secondly, to further enhance local cross-modal alignment under fine-grained su- pervision, we investigate the focal-oriented rewards with soft and hard forms, by focusing the critical points sampled from fine-grained Landmark-RxRR More- over, to fully evaluate the navigation process, we also propose re- re-initialization mechanism that makes metries insensitive difficult points, which can cause the agent to deviate from the correct trajectories. Experimental results show that our agent has superior navigation performance on Landmark-Ran RxxR, en-RxR and R2R. haa.a//aoo/eeillaaaaaaaaaaa
Our dataset and code are available at
Introduction
Vision-and-language navigation (VLN) is important task about cross-modal intelligence and can be applied to service and rescue robots. Different from other cross-modal tasks like image/video captioning (II, 2 and visual question answering [3], where the agent only needs to understand fixed images or videos, VLN agent has to learn and reason by dynamically interacting with the real envi- ronment guided by human instructions. Since Anderson et al. [+]] first introduced the VLN task with the coarse-grained dataset Room-to-Room (R2R), great progresses in this direction have been made, ranging from the ceepuecce-t-sequeeee [5--] to Transformer- models [7 [-2
models [3]] combined with cross-modal grounding modules
Cross-modal alignment is one of the most critical challenges in VLN the predicted trajectory needs to match the given instruction accurately. However, it is difficult for the agent to learn very
Coorresponding author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

accurate visual and textual modality alignment from coarse- coarse-grained data, i.e., only coarse-level cor- respondences between global instructions and trajectories are annotated, without finer ones between sub-instructions and sub-trajectories. Works mentioned above [H-8. [T-12) tried attending on spe- cific parts of the instructions during navigation to help the agent better align salient visual con- tents with the textual modality. But only coarse-grained data like R2R are not capable of providing enough supervising signals of accurate cross-modal alignment. Some recent works such as Fine- Grained R2R [MY]] and BabyWalk [[=]] also found this same issue and proposed fine-grained datasets based on R2R. However, their datasets are generated by heuristic rules and are not precise enough, which inevitably limit the navigation performance. Although Room-across aecross-Roon (RxR) dataset [[3]] includes word-level aligmment between each word and point in the trajectory, it lacks segmenta- tion tags to split an instruction into uubiinstructions with independent meanings and to figure out the landmarks in a trajectory, namely the marked end points of uub-trajectories. Since words in the same sub-instruction are highly correlated, these important correlations are lost because no sub-instruction exists in RxR. It is the same for the sub-trajectories. As result, there is also an absence of the corre- lation between the words in a sub- cobbiinstructin and the points in corresponding sub-trajectories, which should be an important cross-modal alignment supervision signal.
In addition to the dataset annotation, designing better rewards during reinforcement learning is an- other important issue on cross- modal alignment learning. Currently, reward shaping has been well investigated in VLN based on coarse-graine -griinegr data to align instructions and trajectories globally, but pays little attention to enhance local tross-modke aligmment. The goal-oriented reward simply [6] uses the arrival signal and the reduced distance to the goal point the reward, guiding agent to find the glohal goal points. The fidelity-oriented rewards IIG [7] help agent improve the similarity be- tween the predicted trajectory and demonstration trajectory globally by taking intermediate points as external supervision. However, some of the points in the trajectory have more detailed descriptions and are more helpful to navigation process, i.e., they can assist agent in knowing the visual scenes around these specific points in the trajectories could be better aligned with certain parts of the in- structions than the trivial points, where the alignment is in a local to local manner. These points are called critical points in this paper. Thus, an emphasis on these critical points in the reward shaping for better local cross mrosl alignment essential.
In this paper, we address the challenges above from the perspective of fine-grain. Firstly, based on the English Guide part of RxR (en-RxR), introduce fine-grained dataset Landmark-Rxle which is annontand,,tted,, landmark-based, fine-grained and currently the largest scale. With the groundtruth fine-grained data, experiments demonstrate that agent generalizes better to unseen en- vironments and instructions with domain gap. This indicates that the fine-grained data help agent to align textual and visual modalities better. Secondly, we propose two kinds of focal-oriented re- wards that encourage local alignment between instructions and critical points. Since the landmarks in Landmark- RxR naturally meet the requirements of critical points, we just sampled critical points from the landmark set in Landmark- RxR. The focal-oriented rewards outperform the commonly used goal-oriented reward and fidelity-oriented reward. We also propose the re-initialization mech- anism to fully evaluate the navigation process in a way that is insensitive to difficult points, which can cause the deviation from the correct trajectory. With the fine-grained data and focal-oriented rewards, our agent shows superior navigation performance on Landmark- en-RxR and R2R.
2 Related Work
Languugge Navigation Anderson et al. [4] first introduced the simulated, photo- realistic VLN task with benchmark Room-to-Room (R2R) and several sequence-to- to-suuueee base- lines. Fried et al. [5]] utilized the speaker-driven data and panoramic action space to improve naviga- tion performance. Wang et al. [ proposed a planned-ahead hybrid reinforcement learning model with model-free and model-based reinforcement learning for better generalization. Tan et al. [09] first trained agent with mixed imitation learning and reinforcement learning and used unseen triplets generated by environment dropout method for fine-tuning then. Jain et al. [[G]] and Ilharco et al. [IZZ] proposed the fidelity-oriented rewards to ensure the similarity between predicted trajectories and demonstration trajectories. Kurita et al. [20] combined generative and discriminative policy to enhance the navigation ability from different aspects. An et al. [2I] fused neighbor visual contexts
both globally and locally to get visual features with richer semanties. Huang et al. [D2] defined
two in-domain auxiliary tasks for representation learning which benefit the downstream navigation
task. Li et al. [ leveraged the large-scale pretrained language models BERT [3] and GPT [4]]

---

for powerful text representation. Hao et al. [00]]] pretrained and finetuned generic transformer- based model VLN environment and the pretrained model generalizes well other two navigation tasks. Majumdar et al. [[ demonstrated that pretrained on image-text pairs from the web, the transformer- based model can perform on VI.N task well after fine-tuning. Hong et al. [[22] proposed a Recurrent Language BERT to process time-dependent tasks like VLN efficiently.
Vision-and-I Vsion-and-Languagee
Wang et al. [6]] designed a cross-modal grounding module to infer which part of the instructions focus on. Ma al. [8] introduced self-monitoring agent with visual-textual co ro-grounding module to locate the required part in instructions and progress monitor to reflect navigation progress. Qi et al. [25] decoupled instructions into object-aware and action aware parts for more accurate action prediction. Zhu ef al. [[ adopted two swo-stage method for navigation. The given instruction is first split into several sub-instructions named baby-steps according to heuristic rules and then the baby-steps which are much easier to navigate are fed to the agent one by one during validation. Similar to [[=]], Hong et al. [[3]] first segmented instruction into sub-instructions using heuristic method. Then the agent navigates following these easy sub-instructions sequentially with a shifting module inferring whether current sub-instruction has been completed. Ku et al. [TS] explored utilizing spatially-temporallyy aligned annotations to supervise the textual and visual attention weights but got very few performance improvements. Different from these works, we propose uuman-annotated fine-gu grained dataset Landmark-Ran
Fine-Grained VLN
alleviate the weak crosll alignment supervision from the coarse- -oarneed data. Our experiments is designed to demonstrate the supervision from fine-gra mne-graindd and coarse- -griined data complement each other improve the cross-modal alignment ability of the model itself. In addition, two kinds of focal-oriented rewards using fine-ge fineggriiedd supervision signals proposed to enhance the local cross-modal alignment by paying more attention to critical points which benefit the navigation process more.
3 Landmark-RxR Dataset
Our Landmark-RxR is built based on the English Guide split of RxR (en-RxR). It contains sub- instruction and sub-trajectory pairs (sub pairs) split from instructions in en-RxR. To facilitate the collection of sub pairs, we develop a 3D web-based collection tool. It allows the annotator to mark the landmarks, namely the end points of sub-trajectories, in trajectories and split out the correspond- ing sub-instructions. During annotation, the annotators can move between the discrete points or change heading and evaluation continuously by mouse click, like what the agent will do in the VLN task. In the following, we will describe the data collection and dataset analysis of Landmark-Rsy
3.1 Data Collection
The annotators are asked read the complete instructions and then explore the 3D environments following corresponding trajectories. If landmarks are found, annotators need to mark them and split out the corresponding sub-instructions by interacting with the collection tool. As illustrated in Figure 0 (a). the complete instruction and complete trajectory pair (complete pair) are sampled from en-RxR. In Figure (b), three landmarks in the complete trajectory are marked and the cor- responding sub-instructions are also split out by annotators. These three sub pairs will be included in our Landmark-Ren RxR dataset. Five principles are proposed to ensure the annotation process more standardized.
Matching Principle: The sub-trajectory should match the sub-instruction accurately.
Independent Meaning Principle: Each sub-instruction expresses an independent meaning.
Position Change Principle: The end point of each sub-trajectory should differ from the start point.
Clear End Point Principle: The end point should be described clearly in sub-instruction. For example, the sub-instruction go loward the door has a clear description of end point door, but no end point is described in the sub-instruction go forward.
Minimum Granularity Principle: The sub-instruction and sub-trajectory pair should be the mini- mum granularity which means it can not be split anymore.
After an annotator finishes the annotations, another annotator will verify the annotations again and modify the inaccurate part to ensure the annotations satisfy the five principles. For some mistakes in RxR, we also correct them Landmark- RxR. Totally 30 annotators participated in the annotation task, contributing about 2,700 hours.
3

---

(0)
facing towards bed, slightly right go forward the carpet. Go forward and exit the room passing through the openod door: Slightly turn left and go forward, slightly tum right and go downstairs until you reach the last steg stairs.
(b)
Landmarke1
Landmark
Landmaric43
You are facing towards bed, slighdy right go forward on the carpet.
Go forward and exit mom passing through the opened door.
Slightly tum left and go forwand, slightly right and go downstairs until you reach the last step of the stairs,
Figure 1: Three landmarks are marked in the complete trajectory and the corresponding sub- instructions are split out from the complete instruction. and complete trajectory pair in en-RxR. (b) An example of three sub-instruction and sub-trajectory pairs split out from the complete instruction and complete trajectory pair. Each sub-instruction and sub-trajectory pair will be included in Landmark-RxR.
(a) An example of complete instruction
3.2
Dataset Analysis
Table I: Statistics on R2R, RxR, en-RxR and our Landmark-RxR, Edge means the number of edges each trajectory contains on average.
Instruction
Dataset
Edge
Trajectory
Train
Validation Seen
Validation Unseen
Test
Total
R2R
|
14,025
1,020
2,349
4,173
21,567
7,189
RxR
79,467
8,813
13.625
24,164|
126,069
16,522
en-RxR [-_]]
26,464
2,939
4.551
33,954
11,321
Landmark RRx
1.6
133,602
13,591
19,547
166,740
46,645
Table 11 gives statistics R2R, RxR, en-RxR and Landmark-RxR. The total number of sub- instructions from Landmark-RxR is 166,740, which contains 133,602 sub-instructions in train split, 13,591 sub-instructions in validation seen split, and 19,547 sub-instructions in validation unseen split. The average number of edges in sub-trajectory is 1.6 with 21 words in the corresponding sub-instruction on average. The number of sub-trajectories contained in Landmark-Ran -xR is 46.645. Landmark- RxR has the largest scale of instructions and trajectories and the minimum granularity among current -nnotttdd datasets. The fine-grained annotations are essential to effective cross-modal alignment learning and more accurate evaluation. In addition, because of the minimum granularity principle, the dataset can easily expanded a larger scale with different granularities by recombining the sub pairs.
4 Re-initialization Mechanism and Evaluation Metrics
4.1 Re-initialization Mechanism
agent needs navigate following the instructions strictly. So it is essential to evaluate the fidelity between predicted trajectories and the instructions exactly. Current metrics like Coverage weighted by Length Score (CLS) [[G]] and the normalized Dynamic Time Warping (nDTW) [MZ]] try to obtain the fidelity by computing the similarity between predicted trajectory and the demonstration
Note that the number of public instructions in RxR test split 12,469.

---

CCB
Figure 2: Agent deviates from the demonstration trajectory after encountering the difficult point. The green points indicate points in the predicted trajectory. The light blue points indicate points in the demonstration trajectory. The red triangle represents difficult point the demonstration trajectory.
(a) Initialize
(b) Fail
Fails
Suc.it
PaiLet
(c) Re-initialize
(d) Finish
Suc.92
Suc.1
SucA1
Suc
Figure 3: Re-initialization Mechanism. The green points mean the points in the predicted trajec- tory. The light green points mean the predicted trajectory before the last re-initialization and dark green points mean the predicted trajectory after the last re-initialization. The light blue points mean points in the demonstration trajectory. The orange points mean the landmarks in the demonstration trajectory. The red point represents the failure point where to re-initialize the agent.
trajectory. However, the nDTW and CLS scores are sensitive to the positions of difficult points, and have a deviation as a result when evaluating the navigation ability. As illustrated in Figure 2 the agent is initialized point A and ready to follow the given instruction navigating point B. Then during navigation, the agent deviates from demonstration trajectory seriously after encountering the difficult point and only get a low fidelity score. The difficult point indicates the point where next action hard to choose because the difficult visual scene, egg., cluttered and similar objects in Figure 2 ambiguous expressions in an instruction. However, for the from point to point A, it is easy to finish the majority of the trajectory and get higher fidelity score because the difficult point is just near the end. The distributions of difficult points in trajectory significantly impact current fidelity metrics, especially for the long trajectories.
agent navigating
Based on the landmark annotations in Landmark-Rxt we introduce the re-initialization mechanism to alleviate the influence difficult points sensitivity of the fidelity metrics. We define trajectory as T = [ST ST2 ..., STi.. where ST, [P1, Pr: ..., L.). and corresponding instruction is defined [S],, S12, ..... S(nn) P.j means the jth point in the 4th sub-trajectory ST. Le means the ith landmark the trajectory, other words, the end point of the 7t sub-trajectory and S ST, means the tt sub-instruction.
5

---

As illustrated in Figure EL if the agent reaches within 3 meters, which is a fixed threshold for success metric defined [+]], of the landmark L; (é [T]), or the agent can stop within 3 meters of the landmark Liz in (ı is set to 10 in this paper) steps from the last landmark, the navigation to landmark La is successful. The failure is defined that the agent fails to get within meters of the landmark I,, (i < (T]) in steps from the last landmark, or stops 3 meters away from the landmark Lrr-- If the agent fails in navigating to landmark L; (i < (T)), we re-initialize the agent at landmark Lg as the new start point, and reset the given instruction as rrestt [SI;-::.... S[
Since the re-initialization mechanism can correct serious deviations in time, for a complete trajectory. the averaged fidelity score of its sub-trajectories will be much less insensitive the difficult points and can better reflect the navigation ability no matter where the difficult points locate.
4.2 Evaluation Metrics
An agent should have high navigation accuracy and robustness, which means its trajectories should match the instructions accurately and the agent should make as few mistakes as possible during nav- igation. Based on the re-initialization mechanism which could effectively alleviate the influence of difficult points when using fidelity metrics, Sub-Trajectory Accuracy (SA) and Success weighted by Sub-Trajectory Accuracy (SSA) are defined to measure the navigation accuracy, and Loss Number (LN) is defined to measure navigation robustness.
Sub- Tubbrrrjettory Accuracy This metric is defined as normalized Dynamic Time Warping (nDTW) [17]] of the predicted sub-trajectory.
Success weighted by Sub- Sub-Trajectory Accuracy This metric is defined as normalized Dynamic Time Warping (SDTW) [[ of the predicted sub-trajectory.
Loss Number Every time agent fails navigate the next landmark, Loss Number will add one. The total numbers of landmarks in validation seen split and validation unseen split of Landmark- -xxR are 13,591 and 19,547 respectively.
Our Method
We implement our baseline with an encoder-decoder architecture similar to the RCM [6] imple- mented in [14]]. Please refer supplemental material for more details about the baseline.
To enhance the local alignment between instructions and corresponding critical points, we propose two kinds of focal-oriented rewards benefiting from the fine fine-grained supervision in Landmark-Rxk The focal-oriented rewards emphasize the critical points in trajectories, which are more helpful for an agent identify navigation directions because of their more detailed descriptions in instructions. Existing rewards are commonly based on either fidelity metric nDTW B or success metric SR. To fully demonstrate the key role of critical points in reward shaping, choose modify these two metrics to the focal-oriented rewards, namely soft focal-oriented reward and hard focal-oriented reward respectively. Since the landmarks in Landmark-Ran naturally meet the requirements of critical points, we sample the critical points for each trajectory from its landmark set annotated in our fine-grained Landmark-RxR.
Soft focal-oriented reward In focal-oriented reward, all points in demonstration trajectories taken into consideration. We base soft focal-oriented reward on the nDTW metric, which is used to identify the optimal alignment between predicted and demonstration trajectories in time order with minimized cumulative distance. Then we add an importance factor À to modify the nDTW metric. The importance factor À gives higher weight to the distances related to critical points, so it can bias the predicted trajectories in a soft manner, making them closer to the demonstration trajectories at positions of critical points especially.
Roott (R, Q) = =erp(- miny Eeind 95.) din
(1)
[R| den
A
= 1.
{
10, The E SR
others
CLS is less used than nDTW because is and not ideal in some scenarios described in [B]]

---

where W is warping of the predicted trajectory and demonstration trajectory defined in [α7]] is a function that calculates distance between the two given points, R (ri.72: "2.."...... M[nj means the demonstration trajectory consisting of point r', = {q1 92 .... q Qi means the predicted trajectory consisting of point di and Sa {C1, C2,..., C15R | means the set of sampled critical point C2 in demonstration trajectory. By contrast, the À is fixed to 1 in fidelity-orietted reward. So in the soft focal-oriented reward, the agent is penalized more once fails reach the critical points.
Hard focal-oriented reward In hard focal-oriented reward, only the sampled critical points considered in the final reward. If the agent succeeds in reaching the sampled critical points, the reward is positive to encourage series of decisions during navigation. Otherwise, the reward is set as a negative value for penalization. The hard focal-oriented reward function is defined as:
oooa (R, Q) D-Eesx (mingegg C) < dsn))
(3)
x is True
T(x)
11
is False
The hard focal-oriented reward encourages the agent to reach critical points during navigation and gives the agent more flexibility in exploring the trajectories between the critical points.
Policy gradient
We get loss from the non-differentiable reward as Equation B
T(0)) = - Io,goppaa, sc. 0) (Re(at, 8t) REavg)
(5)
where a, is the chosen action, 8; is the visual and textual state, 0 is the parameters the network, p is the probability of choosing the action de, Re is the reward and Reang is the averaged reward. Then the gradients to network parameters are computed as:
VoI(0)= EZ.(Re(a,, 8t) RR Vologp(aelse.
(6)
6 Experiment
6.1
Setup
Data The training data involves four parts: the sub-instruction and sub-trajectory pairs (sub pairs) from Landmark-RRx the synthesized instruction and synthesized trajectory pairs (synthesized pairs) which are augmented data obtained by concatenating several continuous sub pairs like [[GG], the complete instruction and complete trajectory pairs (complete pairs) from en-RxR, and the instruction and trajectory pairs from R2R. In validation phase, for en-RxR and R2R, the agent is validated based on their validation splits directly. For validation complete instructions which is same to en-RxR when initialized in the beginning and the instructions synthesized by unfinished iub-instructions after ce-initialization.
Landmark-RxR the agent navigates based on the
Training policy We warm the agent up by imitation learning with student-forcing strategy. Then we switch to reinforcement learning to learn more generalizable policy to unseen environments. During the reinforcement learning phase, we adopt the focal-oriented rewards and policy gradient [26] to update parameters in the network. Model training consumes about 1,600 minutes at the stage of imitation learning and 3,400 minutes the stage of reinforcement learning on single GTX3090 GPU.
Hyper parameters We use the visual feature from the ResNet [27]]] trained on ImageNet [2X] and adopt panoramic action space like [5]. The word embedding initialized by GloVe300 [29]] The importance factor À in soft focal-oriented reward is set to 10. For the focal-oriented reward, we sample the same number of critical points in each trajectory regularize the range of the reward value for each episode. Ro and Rherd both sample 2 critical points from landmark set in each trajectory, which has the best trade-off between SR and Loss Number metrics empirically. The maximum navigation step allowed for each sub-trajectory is set to 10. The batch size is set to 100 and learning rate is 1 1 The total iterations are 100,000 for imitation learning and 20,000 for reinforcement learning.
7

---

Table 2:
Multitask [0.5] results on R2R validation split. sub means the sub pairs from Landmark-
RxR. co means the complete pairs from en-RxR.
Validation Seen
Validation Unseen
R2R
sub
00
nDTW1
SDTW1
SRT
nDTWt
SDTW1
SRT
56.7
44.0
53.9
31.2
19.1
33.0
44.6
23.6
31.1
35.1
15.0
21.6
49.7
34.0
43.7
32.4
15.8
23.6
V
54.5
63.0
34.3
22.4
33.1
V
55.6
27.0
63.4
53.1
64.1
40.3
27.5
40.1
Evaluation metrics To fully demonstrate the effectiveness of fine- tine-granne data and the focal- oriented rewards, for R2R and en-RxR datasets, we adopt Success Rate (SR), the percentage of stop- ping within 3 meters of the final goal, normalized Dynamic Time Warping (nDTW), measuring the fidelity between predicted trajectory and the demonstration trajectory, Success weighted by normal- ized Dynamic Time Warping (SDTW), measuring the success rate and trajectory fidelity meanwhile. For Landmark-Ral RxR dataset, we adopt Sub- uub-Trajcctory Accuracy (SA) and Success weighted by Sub- Trajectory Accuracy (SSA), difficult points insensitive metrics measuring the navigation accuracy, and Loss Number (LN), a difficult points insensitive metric measuring the navigation robustness.
6.2 Results and Analysis
Fine-grained data contribute to instruction domain generalization Table shows the perfor- mance of model#44 which is trained on R2R and the complete pairs of en-RxR is equivalent to 1odel#1 which is only trained on R2R in unseen environments on SR metric. This indicates the complete pairs in en-RxR have little help for the navigation task in R2R because of the instruction domain gap, which firstly found in [[S]]. Once we replace the complete pairs of en-RxR to sub pairs of Landmark-Rxe as training data, the SR of model#6 improves 7.1% in unseen environments compared to 1odel#1. The better instruction domain generalization indicates that the supervision from the fine-grained data do well with the domain of R2R. Since our Landmark-Rat can be ex- tended to different granularities easily due to the minimum granularity principle, can also benefit the generalization different instruction domains.
Fine-grained data contribute to unseen environments generalization As shown in Table EL trained on synthesized pairs, model#8 outperforms model#7, model#9 and even model#12 in un- seen environments all metrics. It implies that fine-grained data with suitable granularity boost the performance unseen environments most.
Table @ and Table g show that, when we couple the complete pairs and sub pairs training data (model#11 1), we see that SR improves 5.5%, and LN reduces 700 times with SSA improving 3.5% meanwhile compared to model#9. Trained the joint granularity of sub pairs, synthetic pairs and complete pairs, model#13 outperforms models#7-12 on all metries in unseen environments. These indicate cross-modal alignment supervision from coarse-grained data is not enough but the supervision from fine Aine-grained data and coarse- grained data can complement each other to enhance the unseen environments generalization ability.
Focal-oriented rewards facilitate local cross-modal alignment We incorporate different re- wards in the reinforcement learning phase, with results on en-RxR and Landmark-Rat recorded in Table E and Table E respectively. Although model# 14 which takes nDTW reward has the high- est fidelity scores in unseen environments, gets the lowest SR and LN scores among rewards. This indicates that only considering the global alignment between instructions and trajectories make agent just concern about the trajectories similarities but not the locations that instructions really concern during navigation.
Compared to model#14 14, 44 model#17 (soft focal-oriented reward) also bases on nDTW but modifies it with more concern to the critical points. As illustrated in Table B and Table g. the model#17 17 outperforms the model#11 by a significant margin with SR increasing from 29.3% 33.7%, SSA increasing from 52.6 to 53.3 and LN decreasing from 5500 to 5173. In addition, 1odel#17 also

---

Table 3: Results on en-RxR validation seen and validation unseen splits. syn means the synthesized pairs augmented from Landmark-Ree -
IL Train
RL Train
Validation Seen
Validation Unsecn
#
sub
Renfa
7
RAIITW
Rsa
nDTWT
SDTWT
SR1
nDTWf
SDTW;
SR1
7
26.5
12.8
20.4
21.8
10.2
18.6
8
41.7
29.0
42.2
24.6
13.8
24.2
44.1
29.9
24.3
12.4
20.7
10
42.1
30.1
24.9
14.9
26.1
11
45.8
34.2
48.2
25.1
15.5
26.2
12
47.6
35.3
48.6
23.9
13.6
22.8
13
46.9
36.0
50.6
26.7
17.3
28.4
14
55.0
32.8
40.7
43.2
22.6
29.3
15
50.7
36.3
38.2
32.7
16
54.4
39.3
49.9
41.0
25.0
33.6
17
55.1
40.7
51.4
41.0
25.2
33.7
18
48.5
37.0
50.7
36.6
24.2
34.9
Table 4: Results on Landmark-Rar RxR validation seen and validation unseen splits. syn means the synthesized pairs augmented from Landmark-Ren -RxR.
IL Train
RL Train
Validation Seen
Validation Unseen
sub
SYM
Reott
RAIZTW
RsR
SAT
SSA
LN
SAT
SSAt
LN
7
56.6
51.7
4337 (31.9)
49.4
44.1
7849 (40.2)
8
59.1
56.5
3267 (24.0)
49.7
46.2
7029 (36.0)
(26.2)
48.0
7452 (38.1)
10
58.5
(21.3)
50.4
6677 (34.2)
11
60.9
58.6
2899 (21.3)
50.5
47.1
6752 (34.5)
12
60.6
58.2
3039 (22.4)
49.0
45.3
7302 (37.4)
13
61.8
59.9
2589 (19.0)
51.7
48.9
(32.0)
14
64.3
59.8
2764 (20.3)
58.4
52.6
5500 (28.1)
15
(19.6)
55.5
(27.5)
16
63.0
59.5
2622
57.2
$2.5
5309 (27.2)
17
63.8
60.7
2423 (17.8)
57.6
53,3
5173 (26.5)
18
60.7
58.6
2360 (17.4)
54.7
51.8
4951 (25.3)
keeps slight advantage over model# (fidelity-orietted reward with SSA increasing from 52.5 to 53.3 and LN down from 5309 to 5173, about 0.7% drop of the total sub-trajectories number 19,547.
The soft focal-oriented reward is replaced by the hard form in model# 18 (hard focal-oriented reward). Compared with the model#15 15 (goal-oriented reward) which also uses SR reward signal but only on the final goal point, SR of model#188 increases from 32.7% to 34.9% and LN of model#1 18 de- creases 416 times meanwhile, 2.2% of the total number. Compared with model#16 16, model#18 has 1.3% increased in SR and 358 decreased in LN during navigation, 1.9% drop of the total number 19,547. As illustrated Figure E model#18 gives more attention critical point related words like close door, and chooses the correct action, which indicates that model#1 performs local cross- modal alignment better. The above experiments suggest that our focal-oriented rewards which focus on local cross modal alignment help the agent better understand what instructions concern and make fewer mistakes during navigation.
Comparison with SoTA As shown in Table S on the en-RxR benchmark, The model Ou which trained on the data with joint granularities and equipped with soft focal-oriented reward outperforms RCM on validation unseen split", with 4.9 and 8.1% improvement on SDTW and SR metries respectively. In addition, the model Ourghard which trained on the data with joint
We formulate the fidelity-orienter reward based nDTW fidelity metric SR', because experiment empir- ically show that performs better on Loss Number metric than using the gain nDTW score after taking an action as the reward signal like [[]]]
Since trajectories RxR not included the coal-oriented reward here.
necessary the global shortest to the goal. the reduced distance related part
Note that monolingual agent is not allowed to test on RxR competition platform. We compare our methods to the state-of of oftte-of-thhaee the-art method only validation unseen split of en-RxR.

---

(a) lideeit--orenneed reward
(b) hard local-orierteed roward
destmatioo
standing
moving
Rerword
aighhty
winndow
<BOS>
Trom
place
and
keep
till
yon
ths
smd
near
the tabll
turs
Tigh aad
will
<608S
dertinaiion
well
Window
stad
right
tight
now
donr
cluudd
the
roach
all
moving forward
keep
and
standing
are
you
the place
left
tum
Figure 4: Compare agents trained on fidelity-orientee reward and hard focal-orientes reward. Agent trained fidelity-oriented reward fails when choosing candidate action to close door, but agent trained on hard focal-oriented reward succeed to align the instruction to the right candidate action.
Table Comparison of performance with the state-of-the-an methods on en-RxR. RCM the RCM[K] model trained on en-RxR implemented in [05]]. Ours and Ours hard are the same models to model#11 and model# respectively.
Validation Unseen
Agent
nDTW?
SDTWt
SR1
RC [
41.3
20.3
25.6
ooft
41.0
25.2
33.7
Ourghord
36.6
24.2
34.9
granularities and equipped with hard with 34.9% on validation unseen split, 9.3% definite improvement compared to RCM
reward achieves the best result SR metric
7 Conclusion
Strong cross-modal alignment ability is the guarantee of successful navigation for VLN agent. In this paper, we first proposed a human-annotated, fine-grained dataset Landmark-Ras RxR. Our I Landmark-RxR provides fine-grained supervision which complement the weak cross-modal alignment supervision from coarse-grained data. Then benefiting from the fine-grained annotations, two kinds of focal-oriented rewards proposed focusing the local cross-modal alignment leam- ing. In addition, the re-initialization mechanism is proposed for difficult points insensitive evaluation on Landmark-RxR, Experiments show that our fine-grained data and focal-oriented rewards help the agent gain superior navigation performance. However, this work is only a preliminary exploration based on the fine-grained Landmark-RxF dataset. We believe that there is much room for further investigation based on the Landmakr-Ree RxR, like data augmentation, curriculum reinforcement learn- ing and pretraining. In addition, our fine grain related work like focal-oriented rewards can not only enlightens for VLN task, but also for research on other tasks that can be modeled as Markov decision processes.
8 Acknowledgements
This work was jointly supported by National Key Research and Development Program of China Grant No. 2018AAA0100400, National Natural Science Foundation China (61633021, 61721004, 61806194, U1803261, and 61976132), Beijing Nova Program (Z201 100006820079), Shandong Provincial Key Research and Development Program (20191ZZY010119), Key Research Program of Frontier Sciences CAS Grant No. No.DDDBS-L-JSCC02, and CAS-AIB
10

---

References
[1] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3156-3164, 2015.
[2] Subhashini Venugopalan, Marcus Rohrbach, Jeffrey Donahue, Raymond Mooney, Trevor Darrell, and Kate Saenko. Sequence to sequence-video to text. In International Conference on Computer Vision, pages 1534-4542 2015.
[3] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu,, Margaret Mitchell, Dhruv Batra, Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In International Conference on Computer Vision, pages 2425 2433, 2015.
[4] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-langun navigation: Interpreting visually-grone grounded navigation instructions in real environments. In JEEE Conference Computer Vision and Pattern Recog- nition, 2018.
[5] Daniel Fried, Ronghang Hu. Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis Phiuiipp Morency, Tay- lor Berg- Kergpaikka Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for vision- and-language navigation. Advances in Neural Information Processing Systems, 2018.
[6] Xin Wang. Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fau Fang Wang. William Yang Wang, and Lei Zhang. Reinforced cross-modal matching and self-supervised imitation learning for vision- language navigation. IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 6629-6638, 2019.
[7] Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, and Yejin Choi. Robust navigation with language pretraining and stochastic sampling. In Empirical Methods in Natural Language Processing, 2019.
[8] Chih- Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, and Caiming Xiong. Self- beff-monitoring navigation agent via auxiliary progress estimation. International Conference on Learning Representations, 2019.
[9] Ashish Vaswani, Noam Shazcer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. 2017.
[10] Weituo Hao, Chumyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning a generic agent for vision-and-language navigation via pre-training. In IEEE Conference Computer Vision and Pattern Recognition, pages 11137-13166.. 2020.
[11] Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, and Dhruv Batra. Im- proving anddon-and-oo language navigation with image-text pairs from the web. In European Conference on Computer Vision, pages 259-274. Springer, 2020.
[12] Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, and Stephen Gould. A recurrent vision- and-language bert for navigation. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[13] Yicong Hong, Cristian Rodriguez=Opazo, Qi Wu, and Stephen Gould, Sub-instruction aware vision-and- language navigation. Empirical Methods in Natural Language Processing, 2020.
[14] Wang Zhu, Hexiang Hu, Jiacheng Chen, Zhiwei Deng, Vihan Jain, Eugene Ie, and Fei Sha. Babywalk: Going farther in vision-and aainn-an--language navigation by taking baby steps. Association for Computational Linguistics, 2020.
[15] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across-room Mul- tilingual rision-add-lanuaag navigation with dense spatiotemporal grounding. Empirical Methods in Natural Language Processing, 2020.
[16] Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Fugene Ie, and Jason Baldridge. Stay on the path: Instruction fidelity in vision-and-languaae navigation. Association for Computational Linguis- tics, 2019.
[17] Gabriel Ilharco. Vihan Jain, Alexander Ku, Eugene le, and Jason Baldridge. General evaluation for instruction conditioned navigation using dynamic time warping. NeurIPS Visually Grounded Interaction and Language Workshop, 2019.
11

---

[18] Xin Wang, Wenhan Xiong, Hongmin Wang, and William Yang Wang. Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead ussio-andd-anguuge navigation. In European Conference on Computer Vision, pages 37-53, 2018.
[19] Hao Tan, Licheng Yu, and Mohit Bansal. Learning to navigate unseen environments: Back translation with environmental dropout, North American Chapter of the Association for Computational Linguistics. 2019.
[20] Shuhei Kurita and Kyunghyun Cho. Generative gation with bayes' rule. In International Conference on Learning Representations, 2021.
policy vision and -anguag navi-
[21] Dong An, Yuankai Qi. Yan Huang. Qi Wu, Liang Wang, Tieniu Tan. Neighbor-view enhanced model for vision and language navigation. In ACM International Conference on Multimedia, pages 5101-5109, 2021.
[22] Haoshuo Huang. Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Magalhaes, Jason Baldridge, and Eugene le. Transferable representation learning in and-llnguage navigation. In International Conference on Computer Vision, pages 7404-7413, 2019.
[23] Jacob Devlin, Ming- Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019.
[24] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018.
[25] Yuankai Qi. Zizheng Pan, Shengping Zhang, Anton den Hengel, and Qi Wu. Object-and-action aware model visual language navigation. In European Conference on Computer Vision, pages 303- 317. Springer, 2020.
[26] Ronald, J., and Williams. Simple statistical learning. Machine Learning, 1992.
algorithms for counectionist reinforcement
[27] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[28] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, drej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 155(3)211-252) 2015.
[29] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word repre- sentation. In Empirical Methods in Natural Language Processing, pages 1532-1543, 1543, 2014.
12


Reviews:
[{"review_id": "u1N-jZxfL_4", "reviewer": "Reviewer_gPfu", "summary": "The paper tackles the problem of Vision and Language Navigation (VLN). The main contribution of the paper is a dataset called Landmark RxR. The instructions in the RxR dataset have been broken down into smaller segments by human annotators, where each segment corresponds to navigation to a landmark. A landmark is a point at which it is difficult to make a decision (due to ambiguity, clutter, etc). The paper proposes a \"re-initialization\" strategy to better evaluate VLN agents. Two types of reward functions based on the deviation from the landmarks are defined. The paper reports results on R2R, en-RxR and Landmark RxR datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n- The fine-grained annotation of the RxR dataset can be useful in certain navigation scenarios.\n\n- The paper shows state-of-the-art results on the en-RxR dataset.\n\n- One of the proposed reward functions (Soft focal-oriented reward) provides significant improvements compared to standard goal-oriented or fidelity-oriented reward functions.\n\n**Weaknesses**\n\nOne major issue is that it is not clear if the improvements are provided by the landmark annotations or just by breaking the long trajectories. Two baseline experiments should be added to the experiment section: (1) divide the instructions into equal size segments (2) divide the instructions into segments of random size (ending at a period). Without these two experiments, it is hard to judge if the provided dataset is a useful contribution or not.\n\nAlso, there are several critical points that are not clear:\n\n- Is the re-initialization mechanism used only for computing SA and SSA metrics? or is it used for computing the other metrics as well?\n\n- The performance drops when complete instructions are used in Table 2 (model #5 vs model #6), but using complete instructions provides improvement in Table 3 (line 266). What is the explanation for that?\n\n- Why does the method use only two points among the landmarks (line 250)?\n\n- Regarding Table 2, which dataset has been used for rows that do not have a tick for R2R?\n\nI will adjust my rating based on the rebuttal. There are also numerous typos.  \n\n**Post-rebuttal comments**\n\nThe rebuttal addresses my concerns (the baseline experiments and the response to the questions) so I retain my initial rating. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No, the paper does not address the limitations or the potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "oAuXH51O-fN", "reviewer": "Reviewer_HCah", "summary": "This paper proposes augmenting the en-RxR dataset by splitting the trajectories into finer grained sub-trajectories (by human annotators). It also introduces soft and hard focal reward formulations for their policy gradient agent which seem to do well empirically. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Observing that cross-modal alignment is an import aspect leading to higher performance in VLN metrics, this paper makes meaningful contributions both in terms of adding more fine-grained data to RxR dataset and exploiting this additional signal to improve performance on the original task. I have a few suggestions below:\n\nSuggestions\n\n- Can you split Table 3 and report numbers for en-RxR and Landmark-RxR separately?\n\n- Presentation\n  - Remove all occurrences of \"obviously\"\n  - There are several typos throughout the manuscript, please correct those\n\n- Related work section missing relevant citations:\n\n  - Transferable representation learning in vision and-language navigation. Huang et al\n  - Multi-modal discriminative model for vision-and-language navigation. Huang et al\n  - Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments. Chen et al\n  - Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View. Mehta et al", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No negative societal impact of this work", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "_8FI5oU8Kom", "reviewer": "Reviewer_dQiv", "summary": "The paper proposes to improve vision and language navigation by augmenting the RxR dataset with corresponded subinstructions and subtrajectories.  Using the notion of critical points (points that are more important/helpful in the navigation process), two kinds of focal-oriented rewards (soft and hard) are proposed to encourage the predicted trajectory to be close to the demonstration trajectories at critical points.  The authors conduct experiments on R2R and RxR and show that using the collected subinstructions and subtrajectories and proposed losses, the trained agent is able to outperform the baseline RCM model.  For the RxR dataset, the training data consisted both of original RxR data as well as an augmented dataset consisting of concatenating the collected subinstruction and subtrajectories together.\n\nMain contributions\n- Landmark-RxR: Additional annotations on top of the RxR dataset providing sub-instructions paired with sub-trajectories\n- Proposed to use modified rewards for improved cross-modal alignment\n- Ablation experiments of proposed method on the R2R and RxR dataset", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: Limited.  Much of what is done in this work has been proposed in prior work (having paired sub-instruction and sub-trajectory data, concatenating simpler instructions/trajectories together, the focus-oriented reward is also similar to rewards from prior work).  The main novelty lies in the application sub-instructions to the RxR dataset, using the sub-instructions to generate more novel combinations, and having the reward to restricted to a set of critical points. \n\nQuality: There are missing discussion and comparisons wrt prior work.  There is no discussion on the limitations and weaknesses of the work.\n\nClarity: The writing is hard to understand at places.  The paper is also missing some important details (see \"Other comments and questions\" for details)\n\nSignificance: The Landmark-RxR dataset will be useful for the community.  The significance of the proposed method is unclear.\n\nStrengths:\n- The collected Landmark-RxR dataset would be valuable\n- Experiments show that using the subinstructions and the proposed focal losses improves over the baseline RCM model\n\nWeaknesses: \n- Missing detailed discussion and comparison against related work (Jain et al, 2019, Ilharco et al, 2019 both used fidelity-oriented rewards with RCM).  There should be more detailed description of how this work relates and clear experiments comparing against the prior work. \n- Weak experimental setup.\n  - Missing comparisons against relevant prior work.  This includes the close works of RCM with fidelity oriented reward [Jain et al, 2019, Ilharco et al, 2019], as well as other prior work using sub-instructions (such as BabyWalk [Zhu et al, 2020], [Hong et al 2020]).\n  - Proposed method appears to considerably underperform prior work with RCM (Wang et al, 2019, Jain et al, 2019, Ilharco et al, 2019)\n  - For Table 4 (RxR against SOTA), results are reported only for on set (not all sets of validation seen, unseen, testset).  It's also not clear which set Table 4 numbers corresponds to (Table 4 indicates validation seen, but the test (lines 295) imply validation unseen - actual numbers appear to match val-unseen).  \n- There is no discussion of training variance\n- Contribution of the method is unclear\n- The writing is also poor, making it difficult to understand at places.  There are also terminology/wording that are not properly introduced and thus unclear. \n\nOther comments and questions:\n\n- What is the relationship between critical points and landmarks?\n\n- When is the re-initialization mechanism used?  Is it only for evaluation?  Or is it used during training? inference?\n\n- Is there a reason why the loss number metric is not normalized against the number of landmarks?  The lack of normalization makes it challenging to intuitively determine how good the agent is without some understanding of the total number of landmarks.\n\n- It's unclear how the sub-trajectory accuracy is computed.  Is this an average of the nDTW of all sub-trajectories?  If it is a nDTW, then the name \"sub-trajectory accuracy\" does not seem accurate (wouldn't \"sub-trajectory nDTW\" be better?)\n\n- How does the proposed soft and hard focal-oriented reward relate to the fidelity-based reward from Jain et al, 2019?\n\n- Line 201: What specifically is the distance function used? The L2 Euclidean distance?\n- Line 204: How are the critical points sampled? Is a random set of the landmarks uniformly sampled?\n\n- In the experiments, the R_nDTW and R_SR rewards are not fully explained.  How do these rewards correspond to what was introduced in prior work?  The combination of R_nDTW and R_SR seem to work well (comparable with the proposed method).\n\n- The subinstructions/subtrajectories are concatenated together for data augmentation.  Was there experiments comparing it against using data augmentation with the speaker-follower model?\n\n- Line 254: The claim that performance of model#1 and model#4 are equivalent is inaccurate.  While they have similar SR for val-unseen, the performance of the two methods are clearly different for other metrics\n\n- Overall the writing quality was poor, making it confusing and hard to understand.  See below for details:\n  - From the introduction, it was hard to understand what exactly the dataset consisted of and what specifically was meant by various terms such as \"fine-grained perspective\" and \"landmarks\".  Overall, I found the term \"landmark\" to be very confusing as I would expect a \"landmark\" (for a indoor scene) to be say a \"bed\" or a \"painting on the wall\" and correspond to either visual landmarks or landmarks identified by text.  But what it seems to refer to is a location on the trajectory (defined in the paper as \"marked end points of sub-trajectories\").  The term \"critical point\" was also not precisely defined.   Would all landmarks qualify to be a \"critical point\"?\n\n  - Line 57-58: \"english Guide part\" - the details of the RxR dataset has not been introduced so it is hard to understand what \"english Guide\" refers to. It's also unclear what \"fine-grained\" and \"currently the largest scale\" means.\n\n  - The set of annotation principles should be described in the main paper, not the supplemental.  Overall, the description in the main paper was not very informative.  The information in supplemental is clearer and more informative.  In addition to the annotation principles, the supplemental also explains how it differs from prior fine-grained dataset in a clear way, which was lacking from the main paper.  Table 1 from the supplement should be in the main paper (Table 1 in the main paper is better suited to be in the supplement)\n\n  - Line 245: It would make more sense if the \"Hyper parameters\" were described after the \"Training policy\" and before the \"Evaluation metrics\"\n\n  - Some other example of phrases with poor wording that are difficult to understand:\n    - Line 38: \"no sub-instruction can we get from RxR\" \n    - Line 85: \"to navigate in recurrent mechanism like LSTM efficiently\"\n    - Line 95: \"spatially-temporally aligned priori\"\n    - Line 96: \"a plain result\"\n    - Line 183: \"in reverse order originally\"\n    - Line 230: \"The agent reasons in one stage and end-to-end manner\"\n\n  - Some examples of wording / typos\n    - Line 3, 24, and others: \"prediction trajectory\" => \"predicted trajectory\"\n    - Line 98: \"Besides,\" => \"In addition,\"\n    - Line 105: \"supports\" => \"allows\"\n    - Line 106-107: \"During the annotation progress\" => \"During annotation\"\n    - Line 111: \"firstly\" => \"\"\n    - Figure 2 caption: \"mean\" => \"indicate\"\n    - Line 157: \"succeeds to reach within 3 meters\" => \"reaches within 3 meters\"\n    - Line 142: \"firstly\" => \"\"\n    - Line 219: \"the data contain two parts\" => \"the training data consists of two parts\"\n    - Table 2 caption: \"have little help\" => \"provides little improvement\"\n\n   - References are not alphabetically sorted, references not also not capitalized correctly\n\nPost-Rebuttal:\n- I increased my rating from 4 to 5.  The clarification by the authors has mitigated some of my concerns.  However, I still believe that the exposition lacking in clarity and detail, with terminology that is not clearly defined.   Due to the contribution of the dataset and the authors clarifications, I increase my score to 5.   I believe the current submission is not clear enough and a revision is needed.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There was no discussion of the limitations of the work or the any negative social impacts.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "KrVgmaeEl6W", "reviewer": "Reviewer_JseM", "summary": "This paper introduces a new fine-grained VLN dataset called Landmark-RxR which builds on top of RxR. Using the fine-grained annotations, they propose two kinds of focal-oriented rewards for improving local cross-modal alignment. They also propose a re-initialization mechanism to evaluate the navigation such that it is insensitive to difficult points.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality and Significance:\n-\tPrevious papers in VLN space have tried to split the instructions to sub-instructions using heuristics, but it is the first paper to create sub-goal level human annotations for sub-instructions for the instructions in RxR. This dataset will provide more fine-grained supervision for cross-modal alignment training.\n-\tThrough their experiments, they support the claims that they make. \n    -\tFine-grained data contribute to instruction domain generalization: They show a 7.1% increase in performance when using R2R data and sub-instructions from Landmark-RxR as opposed to just using R2R instructions or R2R+RxR instructions.\n    -\tComparison with SoTA: Their model with soft focal-oriented rewards outperforms the RCM model by 4.9 in sDTW and 8.1% on SR.\n\n\n### Quality and Clarity:\n\nThe paper is well-written and easy to follow. \n\n### Typos:\n-\tTable 4 Column heading should be “Validation Unseen”. It is currently “Validation Seen”.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work introduces a new resource for the English fold of RxR and shows how these annotations lead to significant gains in the original related domains/tasks.  The authors have provided substantial clarifications and new results during the discussion which are critical to understanding their work.  These need to be included in the final revision.  Note, in particular, many of the concerns raised by dQiv and the results for gPfu.  Some discussion of whether the resource can provide utility to the full RxR task (not just English) would also be appreciated.
Decision: Accept (Poster)