SlowGuess/ACL_2020_OCR huggingface dataset:
Encoders and Embeddings The encoder  $u(\cdot)$  for the (micro)instructions is a LSTM. The encoder for the trajectory  $y$  contains two separate Bi-LSTMs, one for the state  $s_t$  and the other for the action  $a_t$ . The outputs of the two Bi-LSTMs are then concatenated to form the embedding function  $v(\cdot)$ . The details of the neural network architectures (i.e. configurations as well as an illustrative figure), optimization hyper-parameters, etc. are included in the Appendix.
Learning Policy with Reinforcement Learning In the second phase of learning, BABYWALK uses RL to learn a policy that maximizes the fidelity-oriented rewards (CLS) proposed by Jain et al. (2019). We use policy gradient as the optimizer (Sutton et al., 2000). Meanwhile, we set the maximum number of lectures in curriculum RL to be 4, which is studied in Section 5.3.
We describe the experimental setup (§ 5.1), followed by the main results in § 5.2 where we show the proposed BABYWALK agent attains competitive results on both the in-domain dataset but also generalizing to out-of-the-domain datasets with varying lengths of navigation tasks. We report results from various ablation studies in § 5.3. While we primarily focus on the ROOM4ROOM dataset, we re-analyze the original ROOM2ROOM dataset in § 5.4 and were surprised to find out the agents trained on it can generalize.
5.1 Experimental Setups.
Datasets We conduct empirical studies on the existing datasets ROOM2ROOM and ROOM4ROOM (Anderson et al., 2018; Jain et al., 2019), and the two newly created benchmark datasets ROOM6ROOM and ROOM8ROOM, described in § 4.3. Table 1 and Fig. 4 contrast their differences.
Available at https://github.com/Sha-Lab/babywalk
Setting Metrics	In-domain	Generalization to other datasets
R4R → R4R	R4R → R2R	R4R → R6R	R4R → R8R	Average
SR↑	CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑
SEQ2SEQ	25.7	20.7	9.0	16.3	27.1	10.6	14.4	17.7	4.6	20.7	15.0	4.7	17.1	19.9	6.6
SF+	24.9	23.6	9.2	22.5	29.5	14.8	15.5	20.4	5.2	21.6	17.2	5.0	19.9	22.4	8.3
RCM(GOAL)+	28.7	36.3	13.2	25.9	44.2	20.2	19.3	31.8	7.3	22.8	27.6	5.1	22.7	34.5	10.9
RCM(FIDELITY)+	24.7	39.2	13.7	29.1	34.3	18.3	20.5	38.3	7.9	20.9	34.6	6.1	23.5	35.7	10.8
REGRETFUL++	30.1	34.1	13.5	22.8	32.6	13.4	18.0	31.7	7.5	18.7	29.3	5.6	19.8	31.2	8.8
FAST++	36.2	34.0	15.5	25.1	33.9	14.2	22.1	31.5	7.7	27.7	29.6	6.3	25.0	31.7	9.4
BABYWALK	29.6	47.8	18.1	35.2	48.5	27.2	26.4	44.9	13.1	26.3	44.7	11.5	29.3	46.0	17.3
BABYWALK+	27.3	49.4	17.3	34.1	50.4	27.8	25.5	47.2	13.6	23.1	46.0	11.1	27.6	47.9	17.5
Table 2: VLN agents trained on the R4R dataset and evaluated on the unseen portion of the R4R (in-domain) and the other 3 out-of-the-domain datasets: R2R, R6R and R8R with different distributions in instruction length. The Appendix has more comparisons.  $(^{+}$ : pre-trained with data augmentation.  $*$ : reimplemented or adapted from the original authors' public codes).
Evaluation Metrics We adopt the following metrics: Success Rate (SR) that measures the average rate of the agent stopping within a specified distance near the goal location (Anderson et al., 2018), Coverage weighted by Length Score (CLS) (Jain et al., 2019) that measures the fidelity of the agent's path to the reference, weighted by the length score, and the newly proposed Success rate weighted normalized Dynamic Time Warping (SDTW) that measures in more fine-grained details, the spatiotemporal similarity of the paths by the agent and the human expert, weighted by the success rate (Magalhaes et al., 2019). Both CLS and SDTW measure explicitly the agent's ability to follow instructions and in particular, it was shown that SDTW corresponds to human preferences the most. We report results in other metrics in the Appendix.
Agents to Compare to Whenever possible, for all agents we compare to, we either re-run, reimplement or adapt publicly available codes from their corresponding authors with their provided instructions to ensure a fair comparison. We also "sanity check" by ensuring the results from our implementation and adaptation replicate and are comparable to the reported ones in the literature.
We compare our BABYWALK to the following: (1) the SEQ2SEQ agent (Anderson et al., 2018), being adapted to the panoramic state and action space used in this work; (2) the Speaker Follower (SF) agent (Fried et al., 2018); (3) the Reinforced Cross-Modal Agent (RCM) (Wang et al., 2019) that refines the SF agent using reinforcement learning with either goal-oriented reward (RCM(GOAL)) or fidelity-oriented reward (RCM(FIDELITY)); (4) the Regretful Agent (REGRETFUL) (Ma et al., 2019b)
that uses a progress monitor that records visited path and a regret module that performs backtracking; (5) the Frontier Aware Search with Backtracking agent (FAST) (Ke et al., 2019) that incorporates global and local knowledge to compare partial trajectories in different lengths.
The last 3 agents are reported having state-of-the-art results on the benchmark datasets. Except the SEQ2SEQ agent, all other agents depend on an additional pre-training stage with data augmentation (Fried et al., 2018), which improves crossboard. Thus, we train two BABYWALK agents: one with and the other without the data augmentation.
In-domain Generalization This is the standard evaluation scenario where a trained agent is assessed on the unseen split from the same dataset as the training data. The leftmost columns in Table 2 reports the results where the training data is from R4R. The BABYWALK agents outperform all other agents when evaluated on CLS and SDTW.
When evaluated on SR, FAST performs the best and the BABYWALK agents do not stand out. This is expected: agents which are trained to reach goal do not necessarily lead to better instruction-following. Note that RCM(FIDELITY) performs well in path-following.
Out-of-domain Generalization While our primary goal is to train agents to generalize well to longer navigation tasks, we are also curious how the agents perform on shorter navigation tasks too. The right columns in Table 2 report the comparison. The BABYWALK agents outperform all other agents in all metrics except SR. In particular, on
Figure 5: Performance by various agents on navigation tasks in different lengths. See texts for details.
Setting Metrics	R4R → R4R	R4R → others
SR↑	CLS↑	SDTW ↑	SR↑	CLS↑	SDTW ↑
fSUMMARY = NULL	18.9	43.1	9.9	17.1	42.3	9.6
LSTM(·)	25.8	44.0	14.4	25.7	42.1	14.3
fSUMMARY = ∑i=1m-1αi·(·), i.e., eqs. (2,3)
γ = 5	27.5	46.8	15.8	26.7	44.4	14.9
γ = 0.5	27.3	49.4	17.3	27.6	47.9	17.5
γ = 0.05	27.5	47.7	16.2	26.0	45.5	15.2
γ = 0	26.1	46.6	15.1	25.1	44.3	14.4
SDTW, the generalization to R6R and R8R is especially encouraging, resulting almost twice those of the second-best agent FAST. Moreover, recalling from Fig. 1, BABYWALK's generalization to R6R and R8R attain even better performance than the RCM agents that are trained in-domain.
Fig. 5 provides additional evidence on the success of BABYWALK, where we have contrasted to its performance to other agents' on following instructions in different lengths across all datasets. Clearly, the BABYWALK agent is able to improve very noticeably on longer instructions.
Qualitative Results Fig. 6 contrasts visually several agents in executing two (long) navigation tasks. BABYWALK's trajectories are similar to what human experts provide, while other agents' are not.
Memory Buffer is Beneficial Table 3 illustrates the importance of having a memory buffer to summarize the agent's past experiences. Without the memory (NULL), generalization to longer tasks is significantly worse. Using LSTM to summarize is worse than using forgetting to summarize (eqs. (2,3)). Meanwhile, ablating  $\gamma$  of the forgetting
Table 3: The memory buffer is beneficial to generalizing to different tasks from on which the agent is trained.
Setting Metrics	R4R → R4R	R4R → others
SR↑	CLS↑	SDTW ↑	SR↑	CLS↑	SDTW ↑
IL	24.7	27.9	11.1	24.2	25.8	10.2
IL+RL	25.0	45.5	13.6	25.0	43.8	14.1
IL+ CRL w/ LECTURE #
1st	24.1	44.8	13.5	24.1	43.1	13.6
2nd	26.7	45.9	15.2	26.2	43.7	14.8
3rd	27.9	47.4	17.0	26.7	45.4	16.3
4th	27.3	49.4	17.3	27.6	47.9	17.5
Table 4: BABYWALK's performances with curriculum-based reinforcement learning (CRL), which improves imitation learning without or with reinforcement learning  $(\mathrm{IL} + \mathrm{RL})$
Eval Training	SR↑	→R6R	→R8R
CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑
R2R	21.7	49.0	11.2	20.7	48.7	9.8
R4R	25.5	47.2	13.6	23.1	46.0	11.1
Eval Training	SR↑	→R2R	→R4R
CLS↑	SDTW↑	SR↑	CLS↑	SDTW↑
R2R	43.8	54.4	36.9	21.4	51.0	13.8
R4R	34.1	50.4	27.8	27.3	49.4	17.3
Table 5: (Top) BABYWALK trained on R2R is nearly as effective as the agent trained on R4R when generalizing to longer tasks. (Bottom) BABYWALK trained on R2R adapts to R4R better than the agent trained in the reverse direction.
mechanism concludes that  $\gamma = 0.5$  is the optimal to our hyperparameter search. Note that when  $\gamma = 0$ , this mechanism degenerates to taking average of the memory buffer, and leads to inferior results.
Curriculum-based RL (CRL) is Important Table 4 establishes the value of CRL. While imitation learning (IL) provides a good warm-up for SR, significant improvement on other two metrics come from the subsequent RL  $(\mathrm{IL} + \mathrm{RL})$  . Furthermore, CRL (with 4 "lectures") provides clear improvements over direct RL on the entire instruction (i.e., learning to execute all BABY-STEPS at once). Each lecture improves over the previous one, especially in terms of the SDTW metric.
5.4 Revisiting ROOM2ROOM
Our experimental study has been focusing on using R4R as the training dataset as it was established that as opposed to R2R, R4R distinguishes well an agent who just learns to reach the goal from an agent who learns to follow instructions.
Given the encouraging results of generalizing to longer tasks, a natural question to ask, how well
Figure 6: Trajectories by human experts and VLN agents on two navigation tasks. More are in the Appendix.
can an agent trained on R2R generalize?
Results in Table 5 are interesting. Shown in the top panel, the difference in the averaged performance of generalizing to R6R and R8R is not significant. The agent trained on R4R has a small win on R6R presumably because R4R is closer to R6R than R2R does. But for even longer tasks in R8R, the win is similar.
In the bottom panel, however, it seems that R2R  $\rightarrow$  R4R is stronger (incurring less loss in performance when compared to the in-domain setting R4R  $\rightarrow$  R4R) than the reverse direction (i.e., comparing R4R  $\rightarrow$  R2R to the in-domain R2R  $\rightarrow$  R2R). This might have been caused by the noisier segmentation of long instructions into BABY-STEPS in R4R. (While R4R is composed of two navigation paths in R2R, the segmentation algorithm is not aware of the "natural" boundaries between the two paths.)
There are a few future directions to pursue. First, despite the significant improvement, the gap between short and long tasks is still large and needs to be further reduced. Secondly, richer and more complicated variations between the learning setting and the real physical world need to be tackled. For instance, developing agents that are robust to variations in both visual appearance and instruction descriptions is an important next step.
Acknowledgments We appreciate the feedback from the reviewers. This work is partially supported by NSF Awards IIS-1513966/1632803/1833137, CCF-1139148,
DARPA Award#: FA8750-18-2-0117, DARPA-D3M - Award UCB-00009528, Google Research Awards, gifts from Facebook and Netflix, and ARO# W911NF-12-1-0241 and W911NF-15-1-0484.
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sunderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In CVPR.
Jacob Andreas, Dan Klein, and Sergey Levine. 2017. Modular multitask reinforcement learning with policy sketches. In ICML.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In ICML.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D data in indoor environments. In 3DV.
David L Chen and Raymond J Mooney. 2011. Learning to interpret natural language navigation instructions from observations. In AAAI.
Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, and Yoav Artzi. 2019. Touchdown: Natural language navigation and spatial reasoning in visual street environments. In CVPR.
Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel. 2017. Reverse curriculum generation for reinforcement learning. In CoRL.
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-follower models for vision-and-language navigation. In NeurIPS.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR.
Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear.
Hexiang Hu, Liyu Chen, Boqing Gong, and Fei Sha. 2018. Synthesized policies for transfer and adaptation across tasks and environments. In NeurIPS.
Haoshuo Huang, Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Magalhaes, Jason Baldridge, and Eugene Ie. 2019. Transferable representation learning in vision-and-language navigation. In ICCV.
Vihan Jain, Gabriel Magalhaes, Alex Ku, Ashish Vaswani, Eugene Ie, and Jason Baldridge. 2019. Stay on the path: Instruction fidelity in vision-and-language navigation. In EMNLP.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of gans for improved quality, stability, and variation. In ICLR.
Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, and Siddhartha Srinivasa. 2019. Tactical rewind: Self-correction via backtracking in vision-and-language navigation. In CVPR.
Joohyun Kim and Raymond Mooney. 2013. Adapting discriminative reranking to grounded language learning. In ACL.
Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan Al-Regib, Zsolt Kira, Richard Socher, and Caiming Xiong. 2019a. Self-monitoring navigation agent via auxiliary progress estimation. In ICLR.
Chih-Yao Ma, Zuxuan Wu, Ghassan AlRegib, Caiming Xiong, and Zsolt Kira. 2019b. The regretful agent: Heuristic-aided navigation through progress estimation. In CVPR.
Gabriel Magalhaes, Vihan Jain, Alexander Ku, Eugene Ie, and Jason Baldridge. 2019. Effective and general evaluation for instruction conditioned navigation using dynamic time warping. In NeurIPS ViGIL Workshop.
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. 2019. The neurosymbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In ICLR.
Hongyuan Mei, Mohit Bansal, and Matthew R Walter. 2016. Listen, attend, and walk: Neural mapping of navigational instructions to action sequences. In AAAI.
Khanh Nguyen and Hal Daumé III. 2019. Help, anna! visual navigation with natural multimodal assistance via retrospective curiosity-encouraging imitation learning. In EMNLP.
Junhyuk Oh, Satinder Singh, Honglak Lee, and Pushmeet Kohli. 2017. Zero-shot task generalization with multi-task deep reinforcement learning. In ICML.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Sunrgyull Sohn, Junhyuk Oh, and Honglak Lee. 2018. Hierarchical reinforcement learning for zero-shot generalization with subtask dependencies. In NeurIPS.
Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In NeurlPS.
Hao Tan, Licheng Yu, and Mohit Bansal. 2019. Learning to navigate unseen environments: Back translation with environmental dropout. In EMNLP.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. 2019. Vision-and-dialog navigation. In CoRL.
Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. 2019. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. In CVPR.
Yuke Zhu, Daniel Gordon, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, and Ali Farhadi. 2017a. Visual semantic planning using deep successor representations. In ICCV.
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. 2017b. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In ICRA.
In this supplementary material, we provide details omitted in the main text. The content is organized as what follows:
- Section A. Details on identifying BABY-STEP instructions and aligning BABY-STEPs with expert trajectories. (§ 4.3 and § 4.4 of the main text)
- Section B. Implementation details of the navigation agent, reward function used in RL and optimization hyper-parameters. (§ 4.4 of the main text)
- Section C. Additional experimental results, including in-domain & transfer results of different dataset trained models, sanity check of our implementation, and extra analysis of BABYWALK. (\$ 5.1 and § 5.2 of the main text)
A Details on BABY-STEP Identification and Trajectory Alignments
In this section, we describe the details of how BABY-STEPs are identified in the annotated natural language instructions and how expert trajectory data are segmented to align with BABY-STEP instructions.
A.1 Identify BABY-STEPs
We identify the navigable BABY-STEPS from the natural language instructions of R2R, R4R, R6R and R8R, based on the following 6 steps:
1. Split sentence and chunk phrases. We split the instructions by periods. For each sentence, we perform POS tagging using the SpaCy (Honnibal and Montani, 2017) package to locate and chunk all plausible noun phrases and verb phrases.
2. Curate noun phrases. We curate noun phrases by removing the stop words (i.e., the, for, from etc.) and isolated punctuations among them and lemmatizing each word of them. The purpose is to collect a concentrated set of semantic noun phrases that contain potential visual objects.
3. Identify "landmark words". Next, given the set of candidate visual object words, we filter out a blacklist of words that either do not correspond to any visual counterpart or are misclassified by the SpaCy package. The word blacklist includes:
end, 18 inch, head, inside, forward, position, ground, home, face, walk, feet, way, walking, bit, veer, 've, next, stop, towards, right, direction, thing, facing, side, turn, middle, one, out, piece, left, destination, straight, enter, wait, don't, stand, back, round
We use the remaining noun phrases as the "landmark words" of the sentences. Note that this step identifies the "landmark words" for the later procedure which aligns BABY-STEPs and expert trajectories.
4. Identifying verb phrases. Similarly, we use a verb blacklist to filter out verbs that require no navigational actions of the agent. The blacklist includes: make, turn, face, facing, veer.
5. Merge non-actionable sentences. We merge the sentence without landmarks and verbs into the next sentence, as it is likely not actionable.
6. Merge stop sentences. There are sentences that only describe the stop condition of a navigation action, which include verb-noun compositions indicating the stop condition. We detect the sentences starting with wait, stop, there, remain, you will see as the sentences that only describe the stop condition and merge them to the previous sentence. Similarly, we detect sentences starting with with, facing and merge them to the next sentence.
After applying the above 6 heuristic rules to the language instruction, we obtain chunks of sentences that describes the navigable BABY-STEPS of the whole task (i.e., a sequence of navigational sub-goals.).
A.2 Align Expert Trajectories with identified BABY-STEPS
In the previous section, we describe the algorithm for identifying BABY-STEP instructions from the original natural language instructions of the dataset. Now we are going to describe the procedure of aligning BABY-STEPs with the expert trajectories, which segments the expert trajectories according to the BABY-STEPs to create the training data for the learning pipeline of our BABYWALK agent. Note
that during the training, our BABYWALK does not rely on the existence of ground-truth alignments between the (micro)instructions and BABY-STEPS trajectories.
Main Idea The main idea here is to: 1) perform visual landmark classification to produce confidence scores of landmarks for each visual state  $s$  along expert trajectories; 2) use the predicted landmark scores and the "landmark words" in BABY-STEPs to guide the alignment between the expert trajectory and BABY-STEPs. To achieve this, we train a visual landmark classifier with weak supervision — trajectory-wise existence of landmark objects. Next, based on the predicted landmark confidence scores, we use dynamic programming (DP) to chunk the expert trajectory into segments and assign the segments to the BABY-STEPs.
Weakly Supervised Learning of the Landmark Classifier Given the pairs of aligned instruction and trajectories (X, Y) from the original dataset, we train a landmark classifier to detect landmarks mentioned in the instructions. We formulate it as a multi-label classification problem that asks a classifier  $f_{\mathrm{LDMK}}(s_t; \mathcal{O})$  to predict all the landmarks  $\mathcal{O}_{\mathrm{X}}$  of the instruction X given the corresponding trajectory Y. Here, we denote all possible landmarks from the entire dataset to be  $\mathcal{O}$ , and the landmarks of a specific instruction X to be  $\mathcal{O}_{\mathrm{X}}$ . Concretely, we first train a convolutional neural network (CNN) based on the visual state features  $s_t$  to independently predict the existence of landmarks at every time step, then we aggregate the predictions across all time steps to get trajectory-wise logits  $\psi$  via max-pooling over all states of the trajectory.
$$
\boldsymbol {\psi} = \max  \left\{f _ {\mathrm {L D M K}} \left(\boldsymbol {s} _ {t}; \mathcal {O}\right) \mid t = 1, \dots , | \mathrm {Y} | \right\}
$$
Here  $f_{\mathrm{LDMK}}$  denotes the independent state-wise landmark classifier, and  $\psi$  is the logits before normalization for computing the landmark probability. For the specific details of  $f_{\mathrm{LDMK}}$ , we input the  $6 \times 6$  panorama visual feature (i.e. ResNet-152 feature) into a two-layer CNN (with kernel size of 3, hidden dimension of 128 and ReLU as non-linearity layer) to produce feature activation with spatial extents, followed by a global averaging operator over spatial dimensions and a multi-layer perceptron (2-layer with hidden dimension of 512 and ReLU as non-linearity layer) that outputs the state-wise logits for all visual landmarks  $\mathcal{O}$ . We then max pool all the state-wise logits along the trajectory
and compute the loss using a trajectory-wise binary cross-entropy between the ground-truth landmark label (of existence) and the prediction.
Aligning BABY-STEPs and Trajectories with Visual Landmarks Now, suppose we have a sequence of BABY-STEP instructions  $\mathrm{X} = \{x_{m}, m = 1, \dots, M\}$ , and its expert trajectory  $\mathrm{Y} = \{s_{t}, t = 1, \dots, |\mathrm{Y}|\}$ , we can compute the averaged landmark score for the landmarks  $\mathcal{O}_{\boldsymbol{x}_m}$  that exists in this sub-task instruction  $x_{m}$  on a single state  $s_t$ :
$$
\Psi \left(t, m\right) = \frac {\mathbf {1} \left[ \boldsymbol {o} _ {m} \in \mathcal {O} _ {\boldsymbol {x} _ {m}} \right] ^ {\top} f _ {\mathrm {L D M K}} \left(\boldsymbol {s} _ {t} ; \mathcal {O}\right)}{\left| \mathcal {O} _ {\boldsymbol {x} _ {m}} \right|}
$$