SlowGuess/ACL_2020_OCR huggingface dataset:
Here  $\mathbf{1}\left[\pmb{o}_m\in \mathcal{O}\right]$  represents the one-hot encoding of the landmarks that exists in the BABY-STEP  $x_{m}$  and  $|\mathcal{O}_{\pmb{x}_m}|$  is the total number of existed landmarks. We then apply dynamic programming (DP) to solve the trajectory segmentation specified by the following Bellman equation (in a recursive form).
$$
\Phi \left(t, m\right) = \left\{ \begin{array}{l l} \Psi (t, m), & \text {i f} t = 1 \\ \Psi (t, m) + & \\ \max  _ {i \in \{1, \ldots , t - 1 \}} \left\{\Phi (i, m - 1) \right\}, & \text {o t h e r w i s e} \end{array} \right.
$$
Here,  $\Phi(t, m)$  represents the maximum potential of choosing the state  $s_t$  as the end point of the BABY-STEP instruction  $\boldsymbol{x}_m$ . Solving this DP leads to a set of correspondingly segmented trajectories  $\mathrm{Y} = \{\boldsymbol{y}_m, m = 1, \dots, \mathrm{M}\}$ , with  $\boldsymbol{y}_m$  being the  $m$ -th BABY-STEP sub-trajectory.
B.1 Navigation Agent Configurations
Figure 7 gives an overview of the unrolled version of our full navigation agent.
Panoramic State-Action Space (Fried et al., 2018) We set up the states  $s_t$  as the stacked visual feature of agent-centric panoramic views in 12 headings  $\times 3$  elevations with 30 degree intervals. The visual feature of each view is a concatenation of the ResNet-152 feature vector of size 2048 and the orientation feature vector of size 128 (The 4-dimensional orientation feature  $[\sin(\phi); \cos(\phi); \sin(\omega); \cos(\omega)]$  are tiled 32 times). We use similar single-view visual feature of size 2176 as our action embeddings.
Figure 7: Our network architecture at the  $m$ -th BABY-STEP sub-task. Red line represents the procedure of encoding context variable  $z_{m}$  via summarizing the BABY-STEP trajectory  $f_{\text{SUMMARY}}(v(\hat{y}_{1}), \ldots, v(\hat{y}_{m-1}))$  and the corresponding (micro)instruction  $f_{\text{SUMMARY}}(u(x_{1}), \ldots, u(x_{m-1}))$  in the memory buffer. Blue line represents the procedure of encoding the (micro)instruction  $u(x_{m})$  of the current BABY-STEP. Purple line represents the detailed decision making process of our BABYWALK policy ( $\mathbf{A}_{s_{t}}$  is denoted as the set of navigable directions at  $s_{t}$  as defined by Fried et al. (2018))
Encoders Instruction encoder  $u(\cdot)$  for the instructions is a single directional LSTM with hidden size 512 and a word embedding layer of size 300 (initialized with GloVE embedding (Pennington et al., 2014)). We use the same encoder for encoding the past experienced and the current executing instruction. Trajectory encoder  $v(\cdot)$  contains two separate bidirectional LSTMs (Bi-LSTM), both with hidden size 512. The first Bi-LSTM encodes  $\boldsymbol{a}_{t_i}$  and outputs a hidden state for each time step  $t_i$ . Then we attend the hidden state to the panoramic view  $s_{t_i}$  to get a state feature of size 2176 for each time step. The second Bi-LSTM encoders the state feature. We use the trajectory encoder just for encoding the past experienced trajectories.
BABYWALK Policy The BABYWALK policy network consists of one LSTM with two attention layers and an action predictor. First we attend the hidden state to the panoramic view  $s_t$  to get state feature of size 2176. The state feature is concatenated with the previous action embedding as a variable to update the hidden state using a LSTM with hidden size 512. The updated hidden state is then attended to the context variables (output of  $u(\cdot)$ ). For the action predictor module, we concatenate the output of text attention layer with the summarized past context  $\hat{z}_m$  in order to get an action prediction variable. We then get the action prediction variable through a 2-layer MLP and make a dot product with the navigable action embeddings to retrieve
the probability of the next action.
Model Inference During the inference time, the BABYWALK policy only requires running the heuristic BABY-STEP identification on the test-time instruction. No need for oracle BABY-STEP trajectory during this time as the BABYWALK agent is going to roll out for each BABY-STEP by itself.
B.2 Details of Reward Shaping for RL
As mentioned in the main text, we learn policy via optimizing the Fidelity-oriented reward (Jain et al., 2019). Now we give the complete details of this reward function. Suppose the total number of roll out steps is  $\mathrm{T} = \sum_{i=1}^{\mathrm{M}} |\hat{\mathbf{y}}_i|$ , we would have the following form of reward function:
$$
r(\boldsymbol{s}_{t},\boldsymbol{a}_{t}) = \left\{ \begin{array}{ll}0, & \text{if} t <   T\\ \operatorname {SR}(Y,\hat{Y}) + \operatorname {CLS}(Y,\hat{Y}), & \text{if} t = T \end{array} \right.
$$
Here,  $\hat{\mathrm{Y}} = \hat{\mathbf{y}}_1\oplus \dots \oplus \hat{\mathbf{y}}_{\mathrm{M}}$  represents the concatenation of BABY-STEP trajectories produced by the navigation agent (and we note  $\oplus$  as the concatenation operation).
B.3 Optimization Hyper-parameters
For each BABY-STEP task, we set the maximal number of steps to be 10, and truncate the corresponding BABY-STEP instruction length to be 100. During both the imitation learning and the curriculum reinforcement learning procedures, we fix the learning rate to be 1e-4. In the imitation
learning, the mini-batch size is set to be 100. In the curriculum learning, we reduce the mini-batch size as curriculum increases to save memory consumption. For the 1st, 2nd, 3rd and 4th curriculum, the mini-batch size is set to be 50, 32, 20, and 20 respectively. During the learning, we pre-train our BABYWALK model for 50000 iterations using the imitation learning as a warm-up stage. Next, in each lecture (up to 4) of the reinforcement learning (RL), we train the BABYWALK agent for an additional 10000 iterations, and select the best performing model in terms of SDTW to resume the next lecture. For executing each instruction during the RL, we sample 8 navigation episodes before performing any back-propagation. For each learning stage, we use separate Adam optimizers to optimize for all the parameters. Meanwhile, we use the L2 weight decay as the regularizer with its coefficient set to be 0.0005. In the reinforcement learning, the discounted factor  $\gamma$  is set to be 0.95.
In this section, we describe a comprehensive set of evaluation metrics and then show transfer results of models trained on each dataset, with all metrics. We provide additional analysis studying the effectiveness of template based BABY-STEP identification. Finally we present additional qualitative results.
Complete set of Evaluation Metrics. We adopt the following set of metrics:
- Path Length (PL) is the length of the agent's navigation path.
- Navigation Error (NE) measures the distance between the goal location and final location of the agent's path.
- Success Rate (SR) that measures the average rate of the agent stopping within a specified distance near the goal location (Anderson et al., 2018)
- Success weighted by Path Length (SPL) (Anderson et al., 2018) measures the success rate weighted by the inverse trajectory length, to penalize very long successful trajectory.
- Coverage weighted by Length Score (CLS) (Jain et al., 2019) that measures the fidelity of the agent's path to the reference, weighted by the length score, and the newly proposed
- Normalized Dynamic Time Warping (NDTW) that measures in more fine-grained details, the spatiotemporal similarity of the paths by the agent and the human expert (Magalhaes et al., 2019).
- Success rate weighted normalized Dynamic Time Warping (SDTW) that further measures the spatiotemporal similarity of the paths weighted by the success rate (Magalhaes et al., 2019). CLS, NDTW and SDTW measure explicitly the agent's ability to follow instructions and in particular, it was shown that SDTW corresponds to human preferences the most.
C.1 Sanity Check between Prior Methods and Our Re-implementation
Data Splits
Perf. Measures	R2R Validation	Unseen
PL	NE↓	SR↑	SPL
Reported Results
SEQ2SEQ (Fried et al., 2018)	-	7.07	31.2	-
SF+(Fried et al., 2018)	-	6.62	35.5	-
RCM+(Wang et al., 2019)	14.84	5.88	42.5	-
REGRETFUL++ (Ma et al., 2019b)	-	5.32	50.0	41.0
FAST++ (Ke et al., 2019)	21.17	4.97	56.0	43.0
Re-implemented Version
SEQ2SEQ	15.76	6.71	33.6	25.5
SF+	15.55	6.52	35.8	27.6
RCM+	11.15	6.18	42.4	38.6
REGRETFUL++	13.74	5.38	48.7	39.7
FAST++	20.45	4.97	56.6	43.7
Table 6: Sanity check of model trained on R2R and evaluated on its validation unseen split ( $^+$ : pre-trained with data augmentation;  $\star$ :reimplemented or readapted from the original authors' released code).
As mentioned in the main text, we compare our re-implementation and originally reported results of baseline methods on the R2R datasets, as Table 6. We found that the results are mostly very similar, indicating that our re-implementation are reliable.
C.2 Complete Curriculum Learning Results
We present the curriculum learning results with all evaluation metrics in Table 7.
C.3 Results of BABY-STEP Identification
We present an additional analysis comparing different BABY-STEP identification methods. We compare our template-based BABY-STEP identification with a simple method that treat each sentence as an BABY-STEP (referred as sentence-wise), both using the complete BABYWALK model with the same training routine. The results are shown in the
Datasets	Metrics	IL	IL+ CRL W/ LECTURE #
1s	2nd	3rd	4th
R2R	PL	22.4	12.0	11.6	13.2	10.6	9.6
NE↓	6.8	7.1	6.8	6.8	6.7	6.6
SR↑	28.1	29.8	29.9	33.2	32.2	34.1
SPL↑	15.7	24.3	24.9	26.6	27.5	30.2
CLS↑	28.9	46.2	46.6	47.2	48.1	50.4
NDTW↑	30.6	43.8	42.5	41.0	47.7	50.0
SDTW↑	16.5	23.2	23.1	24.3	25.7	27.8
R4R	PL	43.4	22.8	23.9	25.5	21.4	19.0
NE↓	8.4	8.6	8.5	8.4	8.0	8.2
SR↑	24.7	25.0	24.1	26.7	27.9	27.3
SPL↑	8.2	11.2	11.0	12.3	13.7	14.7
CLS↑	27.9	45.5	44.8	45.9	47.4	49.4
NDTW↑	24.3	34.4	32.8	33.7	38.4	39.6
SDTW↑	11.1	13.6	13.5	15.2	17.0	17.3
R6R	PL	68.8	35.3	37.0	40.6	33.2	28.7
NE↓	9.4	9.5	9.4	9.4	8.9	9.2
SR↑	22.7	23.7	21.9	23.4	24.7	25.5
SPL↑	4.2	7.2	6.4	6.8	8.1	9.2
CLS↑	24.4	43.0	41.8	42.3	44.2	47.2
NDTW↑	17.8	28.1	26.0	26.9	30.9	32.7
SDTW↑	7.7	10.8	9.7	11.0	12.7	13.6
R8R	PL	93.1	47.5	50.0	55.3	45.2	39.9
NE↓	10.0	10.2	10.2	10.1	9.3	10.1
SR↑	21.9	21.4	20.4	22.1	23.1	23.1
SPL↑	4.3	6.1	5.5	6.1	6.8	7.4
CLS↑	24.1	42.1	41.0	41.5	43.9	46.0
NDTW↑	15.5	24.6	22.9	23.8	27.7	28.2
SDTW↑	6.4	8.3	7.9	9.2	10.5	11.1
Average	PL	51.8	26.8	27.9	30.6	25.1	22.1
NE↓	8.5	8.7	8.5	8.5	8.1	8.3
SR↑	24.7	25.5	24.6	27.0	27.5	28.1
SPL↑	8.6	13.1	12.9	13.9	15.1	16.5
CLS↑	26.6	44.5	43.9	44.6	46.2	48.6
NDTW↑	23.0	33.9	32.2	32.4	37.4	39.0
SDTW↑	11.0	14.8	14.4	15.7	17.3	18.4
Table 8. Generally speaking, the template based BABY-STEP identification provides a better performance.
C.4 In-domain Results of Models Trained on Instructions with Different lengths
As mentioned in the main text, we display all the indomain results of navigation agents trained on R2R, R4R, R6R, R8R, respectively. The complete results of all different metrics are included in the Table 9. We note that our BABYWALK agent consistently outperforms baseline methods on each dataset. It is worth noting that on R4R, R6R and R8R datasets,  $\mathrm{RCM(GOAL)^+}$  achieves better results in SPL. This is due to the aforementioned fact that they often
Table 7: Ablation on BABYWALK after each learning stage (trained on R4R).
Datasets	Metrics	Sentence-wise	Template based
R2R	PL	10.3	9.6
NE↓	6.8	6.6
SR↑	28.7	34.1
SPL↑	24.9	30.2
CLS↑	48.3	50.4
NDTW↑	43.6	50.0
SDTW↑	22.4	27.8
R4R	PL	20.9	19.0
NE↓	8.2	8.2
SR↑	26.3	27.3
SPL↑	12.7	14.7
CLS↑	46.4	49.4
NDTW↑	35.5	39.6
SDTW↑	15.9	17.3
R6R	PL	32.1	28.7
NE↓	9.0	9.2
SR↑	22.5	25.5
SPL↑	7.5	9.2
CLS↑	44.2	47.2
NDTW↑	29.3	32.7
SDTW↑	11.1	13.6
R8R	PL	42.9	39.9
NE↓	9.8	10.1
SR↑	21.2	23.1
SPL↑	6.3	7.4
CLS↑	43.2	46.0
NDTW↑	25.5	28.2
SDTW↑	9.3	11.1
Average	PL	24.2	22.1
NE↓	8.3	8.3
SR↑	25.2	28.1
SPL↑	13.8	16.5
CLS↑	45.9	48.6
NDTW↑	34.6	39.0
SDTW↑	15.4	18.4
Table 8: BABYWALK Agent performances between different segmentation rules (trained on R4R). Refer to text for more details.
take short-cuts to directly reach the goal, with a significantly short trajectory. As a consequence, the success rate weighted by inverse path length is high.
C.5 Transfer Results of Models Trained on Instructions with Different lengths
For completeness, we also include all the transfer results of navigation agents trained on R2R, R4R, R6R, R8R, respectfully. The complete results of all different metrics are included in the Table 10. According to this table, we note that models trained on R8R can achieve the best overall transfer learning performances. This could because of the fact that R8R trained model only needs to deal with interpo
Datasets	Metrics	SEQ2SEQ	RCM(GOAL)†	RCM(FIDELITY)‡	BABYWALK	BABYWALK
R2R↑R2R	PL	15.8	15.6	11.1	10.2	10.7	10.2
NE↓	6.7	6.5	6.2	6.2	6.2	5.9
SR↑	33.6	35.8	42.4	42.1	42.6	43.8
SPL↑	25.5	27.6	38.6	38.6	38.3	39.6
CLS↑	38.5	39.8	52.7	52.6	52.9	54.4
NDTW↑	39.2	41.0	51.0	50.8	53.4	55.3
SDTW↑	24.9	27.2	33.5	34.4	35.7	36.9
R4R↑R4R	PL	28.5	26.1	12.3	26.4	23.8	19.0
NE↓	8.5	8.3	7.9	8.4	7.9	8.2
SR↑	25.7	24.9	28.7	24.7	29.6	27.3
SPL↑	14.1	16.0	22.1	11.6	14.0	14.7
CLS↑	20.7	23.6	36.3	39.2	47.8	49.4
NDTW↑	20.6	22.7	31.3	31.3	38.1	39.6
SDTW↑	9.0	9.2	13.2	13.7	18.1	17.3
R6R↑R6R	PL	34.1	43.4	11.8	28.0	28.4	27.2
NE↓	9.5	9.6	9.2	9.4	9.4	9.3
SR↑	18.1	17.8	18.2	20.5	21.7	22.0
SPL↑	9.6	7.9	14.8	7.4	7.8	8.1
CLS↑	23.4	20.3	31.6	39.0	47.1	47.4
NDTW↑	19.3	17.8	25.9	25.8	32.6	33.4
SDTW↑	6.5	5.9	7.6	9.5	11.5	11.8
R8R↑R8R	PL	40.0	53.0	12.4	42.3	35.6	39.1
NE↓	9.9	10.1	10.2	10.7	9.6	9.9
SR↑	20.2	18.6	19.7	18.2	22.3	22.0
SPL↑	12.4	9.8	15.4	5.3	7.3	7.0
CLS↑	19.8	16.3	25.7	37.2	46.4	46.4
NDTW↑	15.8	13.5	19.4	21.6	29.6	28.3
SDTW↑	5.1	4.4	5.8	7.6	10.4	10.1
Table 9: Indomain results. Each model is trained on the training set of R2R, R4R, R6R and R8R datasets, and evaluated on the corresponding unseen validation set  $(^{+}$ : pre-trained with data augmentation).
lating to shorter ones, rather than extrapolating to longer instructions, which is intuitively an easier direction.
C.6 Additional Qualitative Results
We present more qualitative result of various VLN agents as Fig 8. It seems that BABYWALK can produce trajectories that align better with the human expert trajectories.
Datasets	Metrics	SEQ2Seq	RCM(GOAL)++	RCM(FIDELITY)++	REGRETFUL++	FAST++	BABYWALK	BABYWALK+
R2R→R4R	PL	28.6	28.9	13.2	14.1	15.5	29.7	19.5	17.9
NE↓	9.1	9.0	9.2	9.3	8.4	9.1	8.9	8.9
SR↑	18.3	16.7	14.7	15.2	19.2	13.3	22.5	21.4
SPL↑	7.9	7.4	8.9	8.9	10.1	7.7	12.6	11.9
CLS↑	29.8	30.0	42.5	41.2	46.4	41.8	50.3	51.0
NDTW↑	25.1	25.3	33.3	32.4	31.6	33.5	38.9	40.3
SDTW↑	7.1	6.7	7.3	7.2	9.8	7.2	14.5	13.8
R2R→RGR	PL	39.4	41.4	14.2	15.7	15.9	32.0	29.1	25.9
NE↓	9.6	9.8	9.7	9.8	8.8	9.0	10.1	9.8
SR↑	20.7	17.9	22.4	22.7	24.2	26.0	21.4	21.7
SPL↑	11.0	9.1	17.7	18.3	16.6	16.5	7.9	8.8
CLS↑	25.9	26.2	37.1	36.4	40.9	37.7	48.4	49.0
NDTW↑	20.5	20.8	26.6	26.1	16.2	21.9	30.8	32.6
SDTW↑	7.7	7.2	8.2	8.4	6.8	8.5	11.2	11.2
R2R→RGR	PL	52.3	52.2	15.3	16.9	16.6	34.9	38.3	34.0
NE↓	10.5	10.5	11.0	11.1	10.0	10.6	11.1	10.5
SR↑	16.9	13.8	12.4	12.6	16.3	11.1	19.6	20.7
SPL↑	6.1	5.6	7.4	7.5	7.7	6.2	6.9	7.8
CLS↑	22.5	24.1	32.4	30.9	35.3	33.7	48.1	48.7
NDTW↑	17.1	18.2	23.9	23.3	8.1	14.5	26.7	29.1
SDTW↑	4.1	3.8	4.3	4.3	2.4	2.4	9.4	9.8
Average	PL	40.1	40.8	14.2	15.6	16.0	32.2	29.0	25.9
NE↓	9.7	9.8	10.0	10.1	9.1	9.6	10.0	9.7
SR↑	18.6	16.1	16.5	16.8	19.9	16.8	21.2	21.3
SPL↑	8.3	7.4	11.3	11.6	11.5	10.1	9.1	9.5
CLS↑	26.1	26.8	37.3	36.2	40.9	37.7	48.9	49.6
NDTW↑	20.9	21.4	27.9	27.3	18.6	23.3	32.1	34.0
SDTW↑	6.3	5.9	6.6	6.6	6.3	6.0	11.7	11.6
(a) R2R trained model
Dataset	MeNics	SEQ-SEQ	RCM(GOAL)++	RCM(FIDELITY)++	BABYWALK
PL	14.5	19.4	8.1	15.5	9.4	9.2
R6R→R2R	NE↓	7.7	7.1	7.6	7.5	6.8	6.8
SR↑	19.3	21.9	19.6	22.6	31.3	30.6
SPL↑	13.3	11.6	17.2	14.1	28.3	27.8
CLS↑	32.1	26.2	43.2	34.3	49.9	50.0
NDTW↑	31.9	30.8	39.7	32.4	49.5	49.4
SDTW↑	13.1	13.3	15.3	14.3	25.9	25.4
R6R→R4R	PL	25.2	33.0	11.6	25.7	18.1	17.7
NE↓	8.7	8.6	8.5	8.4	8.4	8.2
SR↑	24.2	22.4	23.6	25.4	24.3	24.3
SPL↑	13.7	9.3	17.5	10.6	12.8	12.9
CLS↑	25.8	21.4	35.8	34.8	48.6	48.6
NDTW↑	22.9	20.6	29.8	26.5	39.0	39.4
SDTW↑	9.3	7.5	10.8	11.1	15.1	15.1
R6R→R8R	PL	43.0	52.8	14.2	29.9	38.3	36.8
NE↓	9.9	9.9	9.6	9.7	10.2	10.0
SR↑	20.1	20.3	20.3	22.4	20.8	21.0
SPL↑	11.2	9.4	14.9	8.1	6.6	6.8
CLS↑	20.6	18.3	27.7	38.9	45.9	46.3
NDTW↑	16.3	15.2	21.9	22.2	28.4	29.3
SDTW↑	5.6	5.0	6.4	6.8	9.6	9.9
Average	PL	27.6	35.1	11.3	23.7	21.9	21.2
NE↓	8.8	8.5	8.6	8.5	8.5	8.3
SR↑	21.2	21.5	21.2	23.5	25.5	25.3
SPL↑	12.7	10.1	16.5	10.9	15.9	15.8
CLS↑	26.2	22.0	35.6	36.0	48.1	48.3
NDTW↑	23.7	22.2	30.5	27.0	39.0	39.4
SDTW↑	9.3	8.6	10.8	10.7	16.9	16.8
Table 10: Transfer results of R2R, R4R, R6R, R8R trained model evaluated on their complementary unseen validation datasets ( $^+$ : pre-trained with data augmentation;  $\star$ : reimplemented or readapted from the original authors' released code).
(c) R6R trained model
Datasets	Medics	SEQ2SEQ	\( {SR}^{ + } \)	RCM(GOAL) \( {}^{ + } \)	RCM(FEDLITY) \( {}^{ + } \)	REGRETFUL \( {}^{ + } \)	FAST++	BABYWALK	BABYWALK
\( {R4R} \rightarrow {R2R} \)	PL	16.2	17.4	10.2	17.7	20.0	26.5	12.1	9.6
NE↓	7.8	7.3	7.1	6.7	7.5	7.2	6.6	6.6
SR↑	16.3	22.5	25.9	29.1	22.8	25.1	35.2	34.1
SPL↑	9.9	14.1	22.5	18.2	14.0	16.3	28.3	30.2
CLS↑	27.1	29.5	44.2	34.3	32.6	33.9	48.5	50.4
NDTW↑	29.3	31.8	41.1	33.5	28.5	27.9	46.5	50.0
SDTW↑	10.6	14.8	20.2	18.3	13.4	14.2	27.2	27.8
\( {R4R} \rightarrow {RGR} \)	PL	40.8	38.5	12.8	33.0	19.9	26.6	37.0	28.7
NE↓	9.9	9.5	9.2	9.3	9.5	8.9	8.8	9.2
SR↑	14.4	15.5	19.3	20.5	18.0	22.1	26.4	25.5
SPL↑	6.8	8.4	15.2	8.5	10.6	13.7	8.1	9.2
CLS↑	17.7	20.4	31.8	38.3	31.7	31.5	44.9	47.2
NDTW↑	16.4	18.3	23.5	23.7	23.5	23.0	30.1	32.7
SDTW↑	4.6	5.2	7.3	7.9	7.5	7.7	13.1	13.6
\( {R4R} \rightarrow {R8R} \)	PL	56.4	50.8	13.9	38.7	20.7	28.2	50.0	39.9
NE↓	10.1	9.5	9.5	9.9	9.5	9.1	9.3	10.1
SR↑	20.7	21.6	22.8	20.9	18.7	27.7	26.3	23.1
SPL↑	10.4	11.8	16.9	9.0	9.2	13.7	7.2	7.4
CLS↑	15.0	17.2	27.6	34.6	29.3	29.6	44.7	46.0
NDTW↑	13.4	15.1	19.5	21.7	19.0	17.7	27.1	28.2
SDTW↑	4.7	5.0	5.1	6.1	5.6	6.9	11.5	11.1
Average	PL	37.8	35.6	12.3	29.8	20.2	27.1	33.0	26.1
NE↓	9.3	8.8	8.6	8.6	8.8	8.4	8.2	8.6
SR↑	17.1	19.9	22.7	23.5	19.8	25.0	29.3	27.6
SPL↑	9.0	11.4	18.2	11.9	11.3	14.6	14.5	15.6
CLS↑	19.9	22.4	34.5	35.7	31.2	31.7	46.0	47.9
NDTW↑	19.7	21.7	28.0	26.3	23.7	22.9	34.6	37.0
SDTW↑	6.6	8.3	10.9	10.8	8.8	9.6	17.3	17.5
(b) R4R trained model
Datasets	Metrics	SEQ2SEQ	\( {SR}^{ + } \)	\( {RCM}\left( {GOAL}\right) {}^{ + } \)	\( {RCM}\left( {FIDELITY}\right) {}^{ + } \)	\( {BABYWALK} \)	\( {BABYWALK}{}^{ + } \)
\( {R8R} \rightarrow {R2R} \)	PL	13.7	19.3	7.8	17.8	9.1	9.8
NE↓	7.6	7.3	8.0	8.2	6.8	6.7
SR↑	18.7	23.4	14.8	19.2	30.0	32.1
SPL↑	13.3	12.9	12.9	10.6	27.0	28.2
CLS↑	32.7	26.6	37.9	28.9	49.5	49.3
NDTW↑	32.4	29.9	34.9	25.9	48.9	48.9
SDTW↑	12.7	14.5	11.1	10.5	24.6	26.2
\( {R8R} \rightarrow {R4R} \)	PL	23.1	31.7	11.1	32.5	17.4	19.0
NE↓	8.7	8.8	8.7	9.2	8.2	8.5
SR↑	23.6	21.8	23.2	21.7	24.4	24.4
SPL↑	15.1	10.5	18.2	7.4	12.6	12.5
CLS↑	24.9	20.8	32.3	29.4	48.1	48.5
NDTW↑	22.3	19.7	26.4	20.6	39.1	38.5
SDTW↑	8.8	7.7	9.3	8.4	14.9	15.2
\( {R8R} \rightarrow {R6R} \)	PL	30.9	42.2	11.9	39.9	26.6	29.2
NE↓	9.7	9.9	9.9	10.1	9.0	9.3
SR↑	15.4	14.7	14.8	20.0	22.9	22.9
SPL↑	8.6	6.7	11.6	5.3	8.4	7.9
CLS↑	22.2	18.5	29.1	33.5	46.9	46.6
NDTW↑	18.5	15.9	22.5	20.1	33.3	31.8
SDTW↑	5.5	4.7	6.0	7.8	12.1	11.8
Average	PL	22.6	31.1	10.3	30.1	17.7	19.3
NE↓	8.7	8.7	8.9	9.2	8.0	8.2
SR↑	19.2	20.0	17.6	20.3	25.8	26.5
SPL↑	12.3	10.0	14.2	7.8	16.0	16.2
CLS↑	26.6	22.0	33.1	30.6	48.2	48.1
NDTW↑	24.4	21.8	27.9	22.2	40.4	39.7
SDTW↑	9.0	9.0	8.8	8.9	17.2	17.7
(d) R8R trained model
Figure 8: Additional trajectories by human experts and VLN agents on two navigation tasks.