LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following
Authors: Cheng-Fu Yang
Yen-Chun Chen
Jianwei Yang
Xiyang Dai
Lu Yuan
Yu-Chiang Frank Wang
Kai-Wei Chang
Venue: EMNLP 2023 Main (2023)
Keywords: Embodied AI
Vision and Language Navigation

Abstract: End-to-end Transformers have demonstrated an impressive success rate for Embodied Instruction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environment. This lack of generalizability is due to
the agent’s insensitivity to subtle changes in natural language instructions. To mitigate this issue, we propose explicitly aligning the agent’s hidden states with the instructions via contrastive learning. Nevertheless, the semantic gap between high-level language instructions and the agent’s low-level action space remains an obstacle. Therefore, we further introduce a novel concept of meta-actions to bridge the gap. Meta-actions are ubiquitous action patterns that can be parsed from the original action sequence. These patterns represent higher-level semantics that are intuitively aligned closer to the instructions. When meta-actions are applied as additional training signals, the agent generalizes better to unseen environments. Compared to a strong multi-modal Transformer baseline, we achieve a significant 4.5% absolute gain in success rate in unseen environments of ALFRED Embodied Instruction Following. Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents.

LACMA: Language-Aligningg Contrastive Learning with Meta-Actions for Embodied Instruction Following
Cheng-Fu Yang", Yen-Chun Chen', Jianwei Yang", Xiyang Dai",
Lu Yuan,, Yu-Chiang Frank Wang24,
Kai-Wei Chang'
UCLA
Microsoff NNational Taiwan University
"Nvidia
{cfyang, kwchang)êcs, wwaess.uccl.eee.. edu (yen-chun, jianwei. yang, xiyang.dai
frankwangenvidia.comm
Abstract
Camera Observation:
Instruction:
Take half step left and walk until you
End-to-end Transformers have demonstrated an impressive success rate for Embodied In- struction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environ- ment. This lack of generalizability is due to the agent's 's insensitivity to subtle changes in natural language instructions. To mitigate this issue, we propose explicitly aligning the agent's hidden states with the instructions via con- trastive learning. Nevertheless, the semantic gap between high-level language instructions and the agent's low-level action space remains an obstacle. Therefore, we further introduce novel concept of meta-action Meta-actions are ubiquitous action patterns that can be parsed from the original action sequence. These patterns represent higher-level semantics that are intuitively aligned closer to the instruc- tions. When meta-actions applied ad- ditional training signals, the agent generalizes better to unseen environments. Compared to strong multi-modal Transformer baseline, achieve significant 4.5% absolute gain in suc- cess rate unseen environments ALFRED Embodied Instruction Following. Additional analysis shows that the contrastive objective and meta-actions are complementary in achiev- ing the best results, and the resulting agent bet- tor aligns its states with corresponding instruc- tions, making it more suitable for real-world embodied agents.
are about two feet from the fireplace
1=0
Inferred Meta-action:
Sidestop
Move Forward
Low-level action:
Porstelat
Mowsttwad
Aeowwdight
Mowwlnes
Moreineed
Mosstbasd
Moveimees 1S
Mosainead
Bird-eye view:
Retetelight
Sturing point
bridge the gap.
Fieploon
RotetsLart
Figure I: An embodied agent takes camera observations and instructions and then execute actions to fulfill goal. The large semantic gap between the instruction "Take half step left" and the action sequence "RotateLeft. MoveAhead, RotateRight igtt, MoveAhead" may cause the agent learn shortcuts noring the language instruction. We propose semantic meaningful meta-actions to bridge this gap.
camera observation, ig-
manner. In a typical EIF simulator, the agent's sole perception of the environment is through its ego- centric view from visual camera. To complete all the sub-tasks in this challenging setting, variants of Transformer (Vaswani et al., 2017) have been employed collectively encode the long sequence of multi-modal inputs, which include language structions, camera observations, and past actions. Subsequently, the models are trained end-to-end to imitate the ground-truth action sequences, i.e. expert trajectories, from the dataset.
1 Introduction
Embodied Instruction Following (EIF) necessitates an embodied Al agent to interpret and follow natu- ral language instructions, executing multiple sub- tasks to achieve final goal. Agents instructed sequentially navigate locations while localiz- ing and interacting with objects in iin-grained
Significant progress has been made in this field (Pashevich et al., 2021: Suglia et al., 2021: Zhang and Chai, 2021). Nevertheless, our obser-
"work partially done as research intern Microsoft
'The code is available at: gtthub.cmmjooyy:588/A.C.
"We use action sequences and trajectories interchangeably.

---

vations suggest that existing approaches might not learn follow instructions effectively. Specifically,, our analysis shows that existing models can achieve a high success rate even without providing any lan- guage instructions when the environment in the test time is the same as in training. However, perfor- mance drops significantly when they are deployed into unseen environment even when instructions are provided. This implies that the models learn memorize visual observations for predicting ac- tion sequences rather than learning to follow the instructions. We hypothesize that this overfitting of the visual observations is the root cause of the sig- nificant performance drop in unseen environments. Motivated by this observation, we raise research question: Can we build EIF agent that reliably follows instructions step-by-ste -teep?
its visual observations. This concept draws inspi- ration from recent studies that improve EIF agents with human-defined reusable skills (Brohan et al., 2022; Ahn et al., 2022). An illustrative example is shown in Figure 1.
More specifically, our agent first predicts meta- actions, and then predicts low-level actions condi- tioning on the MAs. However, an LA sequence may be parsed into multiple valid MA sequences. To determine the optimal MA sequences, we parse the trajectories following the minimum descrip- tion length principle (MDL; Grünwald, 2007). The MDL states that the shortest description of the data yields the best model. In our case, the optimal parse of a trajectory corresponds the shortest MA sequence length. Instead of an exhaustive search, optimal MAs can be generated via dynamic programming.
To address the above question, we aim to im- prove the alignment between the language instruc- tion and the internal state representation of an EIF agent. Sharma al. (2021) suggest leveraging language as intermediate representations of trajec- tories. Jiang et al. (2022) demonstrate that iden- tifying patterns within trajectories aids models in adapting to unseen environments. Inspired by their observations, we conjecture two critical directions: I) language may be used as pivot, and 2) common language patterns across trajectories could be lever- aged. We propose Language-Aligningg Contrastive Learning with Meta-Actions (LACMA), a method aimed at enhancing Embodied Instruction Follow- ing. Specifically, we explicitly align the agent's hidden states, which are employed in predicting the next action, with the corresponding sub-task in- struction via contrastive learning. Through the pro- posed contrastive training, hidden states more effectively aligned with the language instruction.
To evaluate the effectiveness of LACMA, we con- duct experiments on the ALFRED dataset (Shrid- har et al., 2020). Even with a modest set of meta- actions consisting of merely 10 classes, our agent significantly outperforms in navigating unseen envi- ronments, improving the task success rate by 1.7% and 4.5% on the unseen validation and testing en- vironments, respectively, while remaining competi- tive the seen environments. Additional analysis reveals the complementary nature of the contrastive objective and meta-actions: Learning from meta- actions effectively reduces the semantic gap be- tween low-leval actions and language instructions, while the contrastive objective enforces alignment to the instructions, preventing the memorization of meta- vettaaactoon sequences in environments. Our contributions can be summarized as follows:
We propose a contrastive objective to align the agent's 's state representations with the task's 's natural language instructions.
Nevertheless, a significant semantic gap per- sists between high-level language instructions. e.g., "take step left and then walk to the fire- place" and the agent's low-level action space, e.g., MoveForward, RotateRight, etc. To further nar- row this gap. we introduce the concept of meta- actions (MA), a set of action patterns each repre- senting higher-level semantics, and can be sequen- tially composed exccute any sub-task. For clarity, we will henceforth refer to the original agent's ac- tions low-level actions (LA) for the remainder of the paper. The elevated semantics of meta-actions may serve as a more robust learning signal, prevent- ing the model from resorting to shortcuts based on
We introduce the concept of meta-actions to bridge the semantic gap between natural lan- guage instructions and low-level actions. We also present a dynamic programming algo- rithm to efficiently parse trajectories into meta- actions.
By integrating the proposed language-aligned meta-actions and state representations, we en- hance the EIF agents' ability to faithfully fol- low instructions.

---

Contrastive
Left
Move
Hight
Hight
Move
...
Move
Predicted Action 01:T
Learning
Sidestep Left
Step Back
Move Forward
Inferred Meta-Action 1::.
Action Prediction Module
Subgoal Instruction
Representation XV
State Representation
21:N
1:-
Multi-modal Encoder
[SEP]
Walk.. [SEP]
RotanoL Moveshcadd
$1
$2
Language Instructions W1:L
Observation U1:T
Actions 01:T
Figure 2: LACMA takes instructions WI.L. camera observations V1.T, and actions dir imputs, and then output state representations 2.r- We additionally extract the output features corresponding to the [SEP] tokens as the representations the sub-goal instructions zi.N- zं- and =i.N are used for contrastive learning, while =f.r- are further utilized to predict the meta-action sequences M1.3 and the low-level action sequences â1:7-
2 Method
2.2 Contrastive State-Instruction Alignment
this section, we first define settings and notations the embodied instruction following (EIF) tasks in Section 2. Then, in Section 2.2, we introduce the language-ing induucd contrastive objective used to extract commonalities from instructions. Finally, in Section 2.3, we will explain how generate the labels for meta-actions and how they lever- aged bridge the gap between instructions and the corresponding action sequences.
In Section 1, we put forth the hypothesis that tasks with similar objectives or navigation goals exhibit shared language patterns. Extracting such common- alities can effectively facilitate the alignment be- tween language instructions and action sequences, further enhancing the generalizability of acquired skills. This alignment is further reinforced through the utilization of contrastive objective during training. In this subsection, we first describe the process of obtaining the model's state representa- tion, which encapsulates the relevant information necessary for the agent to make decisions and take actions. We then explain how we associate such representations with linguistic features to construct both positive and negative pairs for contrastive learning.
2.1 Settings and Notations
Given a natural language task goal G which con- sists of N sub-goals, each corresponding to sub- goal instruction
$1.N. The agent is trained to predict a sequence executable low-level ac- tions ar.7 to accomplish the task. During training time, the ground-truth trajectories of the task are represented by the tuple (WILL, V1.T, a1.r). where T denotes the length of the trajectories. W1:1 rep- resents the concatenation of the task description G and all the subgoal instructions ST,N, with each instruction appended by a special token [SEP]. stands for the total number of tokens in the concate- nated sentence. V1.r denotes the camera observa- tions of the agent over 7 steps, with each camera frame 24 being RGB image with a spatial size of W x H, denoted e I Wxx3333 The action sequences 01.r denote the ground-truth actions. At each timestep i, the navigation agent, parameter- ized by 0, trained to optimize the output distri- bution Po (atl WT.L, Ulit, Qt-1). 1). An overview of the framework illustrated in Figure
State and Instruction Representations Follow- ing prior studies (Pashevich et al., 2021; Sug- lia al., 2021: Zhang and Chai, 2021), we use a Transformer encoder to process all input in- formation, which includes the input instructions, camera observations and previously executed ac- tions (WI.L, Ul.t. 0t-1). As shown in Figure 3, our model generates the state representation 2 at each timestep t, which captures the current state of the agent and the environment. To extract representa- tions for each sub-goal, we take the output features of the [SEP] tokens appended after each instruc- tion, resulting in features 2T.N-- Please refer to the appendix for more details.

---

Meta-Actions
Regular Expressions
Example
1o ettnact
State representation
Action Sequences
repel
Linguistic representation
Step Left
3)r"
1, m. m.
Positivo Spoxt))
1
1
0
0
Move Forward
"m(1.3"
Intra-task Negativee
0
0
1
0
Step Back
"(11Ir))((11(17)) 1,1,m,r
(1oN))ox(()
Inter-task Negative
Table 1: Examples of meta-actions. "T" "m", and "r" represent RotateLeft, MoveAhead, and NotateRight, respectively. Due the page limit, leave the full meta-action list int Table 8 in the appendix.
Figure 3: We contrasts single positive "pos(t) (the corresponding language instruction) for each state repre- sentation 2 against set of intra-task negatives (other instructions from the same task G) and inter-task nega- tives (instructions from other tasks).
Optimal Meta-Actions We draw an analogy between the minimum description length princi- ple (MDL; Grünwald, 2007) and finding the opti- mal MA sequences for given LA trajectory. Both approaches share common goal: finding the most concise representation the data. The MDL prin- ciple suggests that the best model is the one with the shortest description of the data. Similarly, we aim to find MA sequences that are compact yet lossless representations of LA trajectories. There- fore, define the optimal meta-action sequence as the one that uses the minimal number of MAs represent the given low-level action trajectory.
Constructing Positive and Negative Pairs As illustrated in Fig. 3, our contrastive loss function compares specific positive pair, consisting of state representation 2 and the feature of its cor- responding subgoal instruction "pos())= with a col- lection of negative pairs. pos(t) is the index of the instruction features corresponding state This mapping ensures that each frame at timestamp is correctly aligned with its corresponding language instructions. The negative pairs include other sub- goal instructions from the same task (intra-task negatives) as well as instructions from different tasks (inter-task negatives). We denote these in- structions as z "(:s)))ps(()' The contrastive objective takes the following form:
MA Identification via Dynamic Programming We formulate the process of finding the minimal number of MAs to represent given LA trajec- tory dynamic programming (DP) problem. The high level idea to iteratively solve the sub- problem of the optimal MA sequence up to each LA step. To achieve this, we first convert the LA sequence into a sequence of letters and then string match the regular expression form of the given MA set. Table 1 showcases some of the conver- sions. For instance, the LA sequence "MoveAhead, MoveAhead, MoveAhead" is written "m, m, m", and the MA "Move Forward" is represented as "m[1, 3" (m appears one or more times con- secutively). Next, we sequentially solve the sub- problem for each time step, finding the optimal MA sequence to represent the LA trajectory up until the current time step. Further details of the algorithm, pseudo code, pre-defined meta-actions, regular expressions, and example low-level action sequences provided in the appendix A.3 and A.2. By formulating the meta-action identification as a DP problem, we can efficiently extract the op- timal meta-action sequence m1.M with length to represent any low-level action sequence.
T
expl (/T)))
LCL = Σ
log EN exp(tet(=H)/T)
(1)
where (, denotes the inner product and is the temperature parameter. By contrasting the positive pair with these negative pairs, our contrastive loss LcL encourages the model to better distinguish and align state representations with the language instructions, which allows our model to transfer the learned knowledge from seen environments to unseen environments more effectively.
2.3 Learning with Meta-Actions (MA)
To bridge the semantic gap between natural lan- guage instructions and navigation skills, we pro- pose the concept of meta-action (MA), repre- senting higher-level combinations of low-level ac- tions (LAs), depicted in Fig. 1. In this subsection, we first introduce how determine the optimal meta-action sequence given low-level action jectory and set of pre-defined meta-actions. Next, we detail our training paradigm, which involves both generated MAs and the ground-true truch LAs.

---

Pre-training
Fine-tuning LA
et al., 2016; Misra al., 2018: Ku et al., 2020; Krantz et al.. 2020; Das et al., 2018: Prabhude- sai et al., 2020; Padmakumar et al., 2022; Gao et al., 2022) and enviromments (Ramakrishnan et al., 2021; Kolve et al., 2017: Li et al., 2023; Savva et al., 2019) have been proposed to study embodied intel- ligent agents, Among them, vision-and- aonoo-and--anguagee navigation (VLN) is the most comparable task to our setting. Various models have demonstrated im- pressive performance on the task of VLN (Ke al., 2019; Chen al., 2021; Jain et al., 2019; Tan al., 2019; Zhu et al., 2020; Li et al., 2019; Zhu et al.,
LM
Action Predictor
Sombel-Stttman
Meta-Action Predlcctor
Meta-Action Predictor
representation
11-T
Multi-modal Encoder
Language Instructions
Camera Observatione
Previous Actions
2021; Schumann and Riezler, 2022). In addition, Liang et al. (2022) contrasted data within the same modality to improve robustness on variations of in- structions and visual scenes. We specifically focus the ALFRED dataset (Shridhar al., 2020)
Figure 4: Two-stage training of LACMA. In the pre- training stage, our model optimized with the DP- labeled meta-actions (MAs). In fine-tuning, the ground-truth low-level actions (LAs) as supervision, and the model predicts LAs from its own prediction of MAs. To jointly optimize the MA predictor, we apply Gumbel-softmal to allow gradients to flow through the sampling process of MAs.
only involves longer episodes of navigation but also requires models to understand complex instructions, perform fine- ane-grained grounding, and interact with objects.
Training Strategies As shown in Fig. 4, adopt the pretrain-finetus paradigm train our model. Specifically, in the initial pre-training stage, we optimize the model using DP-labeled meta- action sequences together with Eqn. (1), the con- trastive objective. The objective function of op- timizing MA is the standard classification loss: LM CrossEEtropy
Neural EIF Agents In recent years, two lines of works have been developed to tackle embodied instruction following tasks: modular and end-to- end methods. Modular methods (Min et al., 2022; Blukis et al., 2022; Inoue and Ohashi, 2022) em- ploy multiple modules trained with specific sub- tasks and direct supervision decompose the EIF tasks. While our work focuses on aligning the state representations with language instructions for improved generalization, modular methods do not produce state representations for task plan- ning. Therefore, we focus specifically on end-to- end methods (Shridhar et al., 2020; Suglia et al., 2021; Zhang and Chai, 2021; Pashevich et al., 2021; Nguyen al., 2021) to address these limitations. These methods generally utilize a single neural network directly predict low-level actions from input observations. However, these methods gener- ally suffer from limited interpretability and general- ization (Bysenbach et al., 2022). On the other hand, LACMA aligns the decision making process with language instructions, simultancously enhancing interpretability and generalization.
mL:A), thus the pre- training loss can be written L, = LCL + LM- In the fine-tuning stage, utilize ground-truth LA sequences as supervision. Similarly, the fine-tuning loss can be written as: L; = LOL + Ca. where C. = Erosssnnrrpyy .1:r)) denotes the loss function of LA prediction. The use of LCL in both stage enforces our model to align the learned nav- igation skills with the language instructions, pre- venting it from associating specific visual patterns objects with certain actions.
However, conditioning LA prediction on DP- labeled MAs might cause exposure bias (Ranzato et al., 2016). This can result in diminished perfor- mance in testing, where MAS are predicted rather than being explicitly labeled. To mitigate this train- test mismatch, we employ Gumbel-softmax (Jang et al., 2017), allowing model to condition predicted MAs for LA prediction during training.
Skill Learning Learning skills from demonstra- tions has been an active research area in the field of machine learning and robotics. Several approaches have been proposed acquire skills, including the use of latent variable models to partition the expe- rience into different skills (Kim et al., 2019; Jiang
3 Related Works
Embodied Instruction Following (EIF) Vari- ous benchmarks (Anderson et al., 2018b; Pejsa

---

Seen
Unseen
Method
Val
Test
Val
Test
SR
GC
SR
GC
SR
GC
SR
GC
SEQ2SEQ (Shridhar al., 2020)
3.1
10.0
4.0
9.4
0.0
6.9
0.4
7.0
MOCA (Singh et al., 2021)
25.9
34.9
22.1
28.3
5.4
16.2
5.3
14.3
EmBERT (Suglia et al., 2021)
37.4
44.6
31.8
39.2
5.7
15.9
7.5
16.3
E.T. (Pashevich et al., 2021)
34.7
42.0
28.9
36.3
3.5
13.6
4.7
14.9
LACMA
36.9
42.8
32.4
40.5
8.2
18.0
9.2
20.1
Table 2: Results on ALFRED. SR and GC denote the task success rate and the goal condition success rate, respectively. For path-length-weeighted scores, please see Table 10. (t: exclude data from unseen environments.)
et al., 2022; Ajay et al., 2021; Tanneberg et al., 2021). Other works focus on learning skills from language supervision (Ahn et al., 2022; Pashevich et al., 2021: Andreas et al., 2018; Sharma et al.. 2021; Fried et al., 2018). However, there remains challenge in bridging the gap between the learned latent skills and natural language. To close the gap, we introduce the concept of meta-actions, which are higher-level actions that captures the semantic meaning of actions in relation to instructions.
the ALFRED dataset and then use the ResNet backbone to extract image features. These inputs from different modalities are then fused by a multi- modal Transformer encoder. More training details can be found in the appendix A.I.
Baselines As discussed in Sec. 3, LACMA fo- cuses on aligning state representations with lan- guage instructions. For fair comparisons, we specifically choose end-to-end methods that do not incorporate explicit planner, including SEQ2SEQ (Shridhar al., 2020), MOCA (Singh et al., 2021), EmBERT (Suglia et al., 2021), and Episodic Transformer (E.T.; Pashevich et al., 2021). Note that the original E.T. was trained using addi- tional trajectories from the unseen environments. which violates our assumption. Therefore, we re- produce the model using only the data from the original training set.
4 Experiments
4.1 Experimental Settings
Dataset The ALFRED dataset (Shridhar et al., 2020) comprises demonstrations where agent completes household tasks based on goals specified in natural language. ALFRED consists of 21, 21,023 train, 1,641 validation (820 seen 821 unseen), and 3,062 test (1,533 seen 1,529 unseen) episodes.
Evaluation Metrics Following Shridhar et al. (2020), we report the task success rate (SR) and the goal condition success rate (GC). SR measures the percentage tasks where the agent success- fully accomplishes all the subgoals, while GC is the ratio of subgoals fulfilled at the end of the task. For example, the task "put a hot potato slice on the counter" consists of four goal-conditions: slicing the potato, heating the potato slice, placing it the counter, and ensuring it is both heated and on the counter. task considered success only if all the goal-conditions are successful.
4.2 Quantitative Results
The results on ALFRED are shown in Table 2. We can see that LACMA performed favorably against the best end-to-end models across different metrics. LACMA substantially improves the task success rates (SR) and goal condition success rates (GC), especially in the unseen environments. On the validation split, our method outperforms the base- line (F.T.) by 4.7% in SR, and on the test split by 4.5%. This verifies our design in aligning the learned skills with language instructions and using meta-actions to bridge the semantic gap between instructions and low-level actions. Moreover, in the seen environments, LACMA only exhibits improvements compared the baseline, but is also comparable to EmBERT. Note that EmBERT con- siders a 360-degree view, while our method only perceives a narrower 90-degree front view.
Implementation Details Our method was built upon Episodic Transformer (E.T.; Pashevich et al., 2021). Specifically, BERT (Devlin et al., 2019) is used to extract features from the language in- structions. For visual observations, we pre-train a Re Faster R-CNN (Girshick, 2015) on

---

MA
Seen
Unseen
Seen
Unseen
LeL
Method
DP Gumbel
SR
GC
SR
GC
SR
A(1)
SR
A(4)
34.7
42.0
3.5
13.6
E.T.
34.7
3.5
35.4
42,4
5.0
15.2
w/o instructions
22.0
-12.7
0.8
-2.7
31.5
38.9
2.7
12.0
LACMA
36.9
8.2
24.7
35.2
1.6
11.2
w/o instructions
0.0
-36.9
0.0
-8.2
36.9
42.8
8.2
18.0
Table 4: Model's performance on validation split when removing sub-goal instructions from input inference. denotes the SR gap after the removal. Smaller gap indicates model being less sensitive to language instruc- tions when predicting actions.
Table 3: Analyses the contrastive loss LeL and the use meta-actions (MA). We find that these two de- signs are complementary, combining them leads to the best performance. Note that during fine-tuning, the ac- tion prediction conditioned on either the DP-labeled meta-actions the Gumbel sampled meta-actions.
Seen
Unseen
E.T. LACMA
48.2
47.7
4.3 Ablation Studies
79.7
79.1
Following the same evaluation procedures in Sec. 4.2, we discuss each individual contribution of the contrastive loss LCL and the use of meta- actions. We present the results in Table 3. The findings demonstrate the mutual benefit of these design choices, leading to the best performance.
Table 5: Results from Instruction Perturbation. We assess how effectively the model alters its output in response to instruction perturbations.
tributed to a phenomenon akin to what we clabo- rated upon in Section 4.4. In the absence of the con- trastive objective, the model tends to overly depend on visual cues to predict meta actions. Importantly, as the action prediction process is conditioned on meta actions, any inaccuracies originating from the over-dependence on visuals can propagate through the system, resulting an undesired reduction performance.
Contrastive Objective Regarding the contrastive objective (Ccr.), we observe a slight improvement in model performance when using it alone, as shown in the second row of the table. The results confirm our motivation that aligning actions to in- structions can enhance the agent's generalizability in the unseen environments. Furthermore, the re- sults in the third row demonstrate that without Ler, model would become overly reliant on the meta- actions as they are highly correlated to the action sequences. Model may learn a degenerate solution which rely solely on the meta-action for predicting actions, ignoring other relevant information.
4.4 Instruction-Sensitive EIF Agents
To confirm our hypothesis that current models lack sensitivity to changes in instructions, we performed experiments where models were given only the task goal description G while excluding all sub-goal instructions S1.v- The results are presented in Ta- ble 4. In addition, we evaluate how well our models alters its output in response instruction perturba- tions and report the results in Table 5. The combing results suggest that the proposed LACMA is more sensitive to language input. This reinforces our aim of aligning instructions with actions, thereby mit- igating model's 's over-reliance on visual input and enhancing the trained agents' generalization.
Meta-Actions In Table 3 show that the use of meta-actions can further improved the perfor- mance with proper regularization from Ler. We hypothesize that this is because the proposed meta actions encapsulate higher-level semantics that ex- hibit an intuitive alignment with the instructions. The notable improvements observed in the unseen domain further validate our hypothesis that this aligning nature facilitates better comprehension of language within model. As result, our model demonstrates more effective action predic- tion when comes to generalizing across diverse environments. The observed performance degra- dation when using meta actions alone can be at-
4.5 Language Aligned State Representations In order assess the alignment between the learned state representations and language instruc- tions, we conducted a probing experiment using

---

Seen
Unseen
Seen
Unseen
Method
Method
R@1
R@5
R@1
R@5
PC
LS
CLS
PC
LS
CLS
Retrieve from 100 instructions
E.T.
90.1
66.7
60
82,4
50.1
41.3
E.T.
91.5
99.9
87.9
99.8
LACMA
92.3
70.8
65.4
85.4
52.3
45.3
LACMA 96.8
100.0
91.2
99.9
Table 7: Fidelity of the generated trajectories. evalu- path coverage (PC), length scores (LS), and coverage weighted by length score (CL.) on the ALFRED vali- dation set.
Retrieve from Ik instructions
E.T.
62.3
97.8
48.0
94.6
LACMA 81.3
99.7
60.0
98.1
Retrieve from Sk instructions
E.T.
33.0
81.8
28.2
77.3
one that our approach consistently outper- forms E.T. across all categories. This suggests that LACMA excels following instructions and demonstrates a stronger grasp of language nuances compared to E.T.
LACMA 53.9
97.3
46.4
94.3
Table 6: Results of probing the learned state representa- tions on validation split. We probe the models using a inauuction-rttrevar task. R@] and R@s refer to Recall at and Recall at 5, respectively.
These results highlight the capability of our ap- proach accurately retrieve the associated instruc- tions based the learned state representations. By introducing the contrastive objective, our model demonstrates significant improvements in the re- trieval task, showcasing its ability to effectively incorporate language instructions into the state rep- resentation.
retrieval task. The purpose of this experiment evaluate the model's capability to retrieve the appropriate sub-goal instructions based on its state representations. To accomplish this, we fol- lowed the process outlined in Section 2.2, which extracts the state representations z and pairing them with the corresponding instruction represen- tations "pos(t)- Subsequently, we trained single fully-connected network to retrieve the paired in- struction, with training duration of 20 epochs and batch size of 128.
4.6 Qualitative Results
We visualize the learned meta-actions and the re- trieved instructions in Fig. 5. LACMA demonstrates high interpretability since we can use the state rep- resentation to retrieve the currently executing sub- goal. It is worth noting that while our model may produce a meta-action sequence that differs from the an-anttteeed one, the generated sequence re- mains valid and demonstrates a higher alignment with the retrieved instruction. This behavior stems from our training approach, where the model is not directly supervised with labeled meta-actions. Instead, we optimize the meta-action predictors through joint optimization of the contrastive ob- jective Lei, and the action loss L Consequently, our model learns meta-actions that facilitate cor- rect action generation while also aligning with the provided language instructions. Due to page limit, we visualize more trajectories in appendix A.6.
During the testing phase, we progressively in- creased the difficulty of the retrieval tasks by vary- ing the number of instructions to retrieve from: 100, 1,000, and 5,000. The obtained results are pre- sented in Table 6. Notably, our method achieved superior performance across both seen and unseen environments compared to the baseline approach. Specifically, at the retrieval from 5000 instructions, our model surpasses E.T.s recall 1 by 20.9% and 18.2% on seen and unseen split, respectively. For the retrieval tasks involving 100 and 1,000 instruc- tions, our method consistently outperforms E.T., demonstrating its effectiveness in aligning state representations with language instructions.
In addition, we also provide a holistic evaluation of instruction fidelity using the metrics proposed in Jain al. (2019). Specifically,, we caleulate how well does the predict trajectories cover the ground- truth path, and report path coverage (PC), length scores (LS), and Coverage weighted by Length Score (CLS) in Table 7. From the provided table
Conclusion
In this paper, we propose LACMA, a novel approach that addresses the semantic gap between high-level
language instructions and low-level action space in Embodied Instruction Following. Our key contribu- tions include the introduction of contrastive learn-

---

its performance and robustness in situations where navigation plans fail or lead incorrect actions.
Lewcland
Action
Porsis.at Moveiinedd
Moveiinad Moreisead Macoeltes
Labetod NA
Turn Let
Nove Forvors
Predicted
Turn Lart
Noes Fonwort
Ethics Statement
Rettered
Our research work does not raise any significant ethical concerns. In terms of dataset characteristics, we provide detailed descriptions to ensure readers understand the target speaker populations for which our technology is expected to work effectively. The claims made in our paper align with the experi- mental results, providing a realistic understanding of the generalization capabilities. We thoroughly evaluate the dataset's quality and describe the steps taken to ensure its reliability.
Leaelend
Actore
Petsseiight
Pucckjuc
Perssniiggi Mevelinwdd
Roesslant
Labeted NA
Sy
Inserctioon
Step Right
Prediceed
Inseuctioon
Turn Right
Cocaact
Move
Rettened
Leaclend Actort
Dtap Persestet
Moveshaae
Moweltead RecesRight
Labeted MA
Byy
Intoracioo
Lort
Produced
Intaraction
Step Left
Rettleadd
lelt
Figure 5: Visualization of the learned meta-actions and the retrieved low-level instructions. Segments with dif- ferent color indicating different types of meta-actions,
Acknowledgements
We thank anony reviewers, Po-Nien Kung, Te- Lin Wu, Zi-Yi Dou and other members of UCLA- NLP+ group for their helpful comments. This work was partially supported by Amazon AWS cred- its, ONR grant N00014-22 1-00014-----7800 and a DARPA ANSR program FA8750-23-2-0008 23-2-0004. The views and conclusions are those of the authors and should not reflect the official policy or position of DARPA or the U.S. Government.
ing align the agent's hidden states with instruc- tions and the incorporation of meta-actions, which capture higher-level semanties from the action se- quence. Through these innovations, we achieve a significant 4.5% absolute gain in success rate on unseen environments. Our results demonstrate the effectiveness of LACMA in improving alignment between instructions and actions, paving the way for more robust embodied agents.
References
Limitations
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. 2022. Do i can, not as say: Ground- ing language in robotic affordances. arXiv preprint arXiv:2204.0166 2,
Despite the effectiveness of meta-actions and the contrastive objective in our approach, there are sev- eral limitations to consider. One key limitation is the use of a ResNet-50 encoder to extract a single feature for each frame. By pooling the entire im- age into single vector, there is potential loss of fine-grained information. To address this limita- tion, incorporating object-aware object-centric features could enhance the model's performance. By considering the specific objects present in the environment, the model may gain a more nuanced understanding of the scene and improve its ability to generate accurate and contextually relevant ac- tions. Another limitation is that our model does not employ any error escaping technique like back- tracking (Zhang and Chai, 2021; Ke al., 2019) replanning (Min et al., 2022). These techniques have shown promise in improving the model's abil- ity to recover from errors and navigate challenging environments. By incorporating an error recovery mechanism, our model could potentially enhance
Anurag Ajay, Aviral Kumar, Pulkit Agrawal, Sergey Levine, and Ofir Nachum. 2021. Opal: Offline primi- tive discovery for accelerating offline reinforcement learning. In ICIR. 6
Peter Anderson, Angel Chang, Devendra Singh Chap- lot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mot- taghi, Manolis Savva, et al. 20.8a. On evalua- tion of embodied navigation agents. arXiv preprint 1807 807.0067777.. 14
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. 2018b. Vision- and-language navigation: Interpreting visually- grounded navigation instructions real environ- ments. In CVPR. 5
Jacob Andreas, Dan Klein, and Sergey Levine. 2018 Learning with latent language. In NAACL-HLT.

---

Valts Blukis, Chris Paxton, Dieter Fox, Animesh Garg, and Yoav Artzi. 2022. A persistent spatial semantic representation for high-level natural language instruc- tion exccution. In CoRL.
Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman. Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, and Siddhartha Srinivasa. 2019. Tactical rewind: Self- correction via backtracking in vision-and-languuge navigation. In CVPR. 5, 9
Anthony Brohan, Noah Brown, Justice Carbajal, Yev- gen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jas- mine Hsu, et al. 2022. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint 1ixiv:2212.06817. 2
Taesup Kim, Sungjin Ahn, and Yoshua Bengio. 2019. Variational temporal abstraction. In NeurlPS. 5
Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli Van- derBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. 2017. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprial 17X1v:1712.05474.
Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. 2021. History aware multimodal transformer for vision-and -anguage navigation. In NeurIPS. 5
Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. 2020. Beyond the nav-graph: Vision-and- Vision-and-language language navigation in continuous envi- ronments. In ECCV.
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. 2018. Embodied question answering. In CVPR. 5
Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. 2020. Room-across-room Mul- tilingual vision-and-language navigation with dense spatiotemporal grounding. In EMNLP.
Jacob Devlin, Ming-Wii Wei Chang, Kenton Lee, and Kristina Toutanova. 2019, Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT. 6, 12
Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gok- men, Sanjana Srivastava, Roberto Martín-Martin, Chen Wang. Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. 2023. Behavior- lk: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In CoRT. 5
Benjamin Eysenbach, Soumith Udatha, Russ Salakhutdinov, and Sergey Levine. 2022. Imitating past successes can be very suboptimal. In NeurlPS. 5
Daniel Fried. Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis- Philippe Morency, Taylor King-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-follower models for lision-and-language navigation. In NeurIPS. 6
Xiujun Li, Chunyuan Li, Qiaolin Xia. Yonatan Bisk. Asli Celikyilmaz, Jianfeng Gao, Noah Smith, and Yejin Choi. 2019. Robust navigation with language pretraining and stochastic sampling. In EMNLP.
Xiwen Liang, Fengda Zhu, Yi Zhu Bingqian Lin, Bing Wang, and Xiaodan Liang. 2022. Contrastive instruction-trajecttory learning for vision-language navigation. In AAAI.
Xiaofeng Gao, Qiaozi Gao, Ran Gong, Kaixiang Lin, Govind Thattai, and Gaurav S Sukhatme. 2022. Di- alfred: Dialogue-enabled agents for embodied in-
struction following. IEFE Robotics and Automation Letters, 7(4):10049-1000 10056. 5
So Yeon Min. Devendra Singh Chaplot, Pradeep Raviku- mar, Yonatan Bisk, and Ruslan Salakhutdinov. 2022. Film: Following instructions in language with modu- methods. In ICLR, 5,
Ross Girshick. 2015, Fast r-enn. In ICCV. 12
Peter D Grünwald. 2007. The minimum description length principle. MIT press. 2,
Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi. 2018. Mapping instructions to actions in 3d environments with visual goal prediction. In EMNLP. 5
Yuki Inoue and Hiroki Ohashi. 2022. Prompter: Utiliz- ing large language model prompting for a data effi- cient embodied instruction following, arXiv preprint arXiv:2211.033 5
Van Van-Quang Nguyen, Masanori Suganuma, and Takayuki Okatani. 2021. Look wide and interpret twice: Improving performance interactive instruction- following tasks. In 1
Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Eugene le, and Jason Baldridge. 2019. Stay on the path: Instruction fidelity in vision-and- language navigation. In ACL. 5,8
Aishwarya Padmakumar, Jesse Thomason, Ayush Shri- vastava, Patrick Lange, Anjali Narayan-Chen, Span- dana Gella, Robinson Piramuthu, Gokhan Tur, and Dilek Hakkani- Tur. 2022. Teach: Task-driven em- bodied agents that chat. In AAAI. 5
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Cate- gorical reparameterization with gumbel-softmax. In ICLR.
Yiding Jiang, Evan Liu, Benjamin Eysenbach, J Zico Kolter, and Chelsea Finn. 2022. Learning options via compression, In NewrIPS. 2,5
Alexander Pashevich, Cordelia Schmid, and Chen Sun. 2021. Episodic transformer for vision-and-languaee navigation. In ICCV. 1, 3, 5, 6, 12, 14

---

Tomislav Pejsa, Julian Kantor, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. 2016. Room2room: En- abling life-size telepresence in projected augmented reality environment. Proceedings of the 19th ACM conference on computer-supportee cooperative work & social computing. 5
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurlPS.
Yichi Zhang and Joyce Chai. 2021. Hierarchical task learning from language instructions with unified transformers and self-monitoring. In Findings of ACL. 5,9
Mihir Prabhudesai, Hsiao-Yo Yu Fish Tung. Syed Ashar Javed, Maximilian Sieb, Adam W Harley, and Kate- rina Fragkiadaki. 2020. Embodied language ground- ing with 3d visual feature representations. In CVPR. 5
Wang Zhu, Hexiang Hu, Jiacheng Chen, Zhiwei Deng.
Vihan Jain, Eugene Ic, and Fei Sha. 2020. Babywalk: Going farther in vision-and-language navigation by taking baby steps. In ACL. 5
Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg, John M Turner, Eric Undersander, Wojeiech Galuba, Andrew Westbury, Angel X Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. 2021. Habitat-matter catttat-matterport 3d dataset (HM3d): 1000 large-scale 3d environ- ments for embodied AI. In NeurlPS (Datasets and Benchmarks Track).
Yi Zhu, Yue Weng, Fengda Zhu, Xiaodan Liang, Qix- iang Ye, Yutong Lu, and Jianbin Jiao. 2021. Self- motivated communication agent for real-world vision- dialog navigation. In ICCV.
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level train- ing with recurrent neural networks. In ICLR.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. 2019. Habitat: platform for embodied research. ICCV. 5
Raphael Schumann and Stefan Riezler. 2022. Analyzing generalization of vision and language navigation to unseen outdoor areas. In ACL.
Pratyusha Sharma, Antonio Torralba, and Jacob dreas. 2021. Skill induction and planning with latent language. In ACL. 2,
Mohit Shridhar, Jesse Thomason. Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020, Alfred: A bench- mark for interpreting grounded instructions for every- day tasks. CVPR. 5, 6,
Kunal Pratap Singh, Suvaansh Bhambri, Byeonghwi Kim, Roozbeh Mottaghi, and Jonghyun Choi. 2021. Factorizing perception and policy for interactive in- struction following. ICCV. 14
Alessandro Suglia, Qiaozi Gao, Jesse Thomason, Govind Thattai, and Gaurav Sukhatme. 2021. Em- bodied bert: A transformer model for embodied, angguage-guider visual task completion. arXiv preprint 1rXiv:2108.0492: 1, 3, 6,
Hao Tan, Licheng Yu, and Mohit Bansal. 2019. Learn- ing to navigate unseen euvironments: Back transla- tion with environmental dropout. In NAACL-HLT. 5
Daniel Tanneberg, Kai Ploeger, Elmar Rueckert, and Jan Peters. 2021. Skid raw: Skill discovery from raw trajectories. IEFE Robotics and Automation Letters, 6(3):469644703..

---

Meta-Actions
Regular Expressions
Example Action Sequences
Low-level Actions
Letter Expression
MoveAhead
I
RotateRight
Step Right
"rm(, 3311"
1,1
RotateLeft
Step Left
"Im[ 3)r"
LookUp
Move Forward
"m{1,3" 3
m, m,
LookDown
d
Step Back
"(1IIrr)=((u1rr))
1, r.
PickupObject
Left
PutObject
Turn Right
"r1"
Toggle0bjecton
Turn Around
1,,,
Toggle0bjectoff
Look Up
"u(1,3"
u, U.
CloseObject ect
Look Down
"d(1,3"
d, d.
OpenObject
Interaction
"i"
i,
Sl SliceObject
Table 8: Full list of meta-actions. actions throughout our experiments.
We use 10 meta-
Table 9: The letter expression of low-level actions, we translate the entire action sequence into the string based on the rule presented in the table.
Appendix
A.1 Implementation Details
to prevent them from seeing the future frames and actions. In similar spirit, we apply the same mask- ing strategy to the action input.
Model Architecture We use Episodic Trans- back-
former (E.T.; Pashevich al., 2021) bone. We first extract input from different modal- ities using modality-specific encoders, followed by a multi-modal Transformer encoder to fuse and reason over the multi-modal input. Specifically, we use a BERT-base (Devlin et al., 2019) encoder to extract features from the language instructions. For visual observations, pre-train ResNet-50 Faster R-CNN (Girshick, 2015) the ALFRED dataset and use the ResNet backbone extract image features. Note that we do not update the visual backbone during our training, instead, we use 2 convolution by layers, followed by a fully- connected (FC) layer, to project the features from ResNet into an embedding of the size 768. To han- dle actions, we train lookup table that maps discrete action to 768-dineen dineennnnnio embedding. The multi-modal encoder comprises transformer encoder layers, each with 12 self-attention heads, and hidden size of 768, will take the aforemen- tioned features from each modality, and produce the final state representations. We then use two separate FC layers for action and meta-action pre- diction. Following Pashevich et al. (2021), we use three different kinds of masking strategies for input from different modalities.
Training parameters We train our model for 20 epochs in both pre-training and fine-tuning phases. The learning rate for both phases starts at 0 and linearly warms up
10 for the first 1000 steps, and drops to 1 x 10-5 after 10 epochs. The effective batch size is 32, and we utilize NVIDIA 1080Ti GPUs for training.
A.2 Full List of Meta-Actions
We provide the full list of meta-actions in Table 8. We use letters to represent low-level actions, and we present the translate rule in Table 9, The av- erage length of the original low-level action tra- jectories is around 50. While the average length of the meta-action sequences after translation is around 10, which effectively reduce the complexity of solution space. The average branching factor low-level actions is 1250 ~ 1053 (50 average steps for 12 low-level actions), while for meta-action it is 10
A.3 Dynamic Programming for Meta-Action Identification
Here present the details of using dynamic- programming identify the optimal meta-action sequences. To perform DP, begin by determin- ing the valid interval for each meta-action. Algo- rithm outlines the pseudo-code for this process. We initialize table to store the intervals associ-
Masking Strategy Specifically, the language in- put can only attend to ourselves, it has no access to the image and action input. The visual input can attend to all text features, but we use causal masks

---

end end for end for Algorithm Finding Valid Interval For Meta-Actions
1: procedure CREATEMETAACTIONTTAR
2: A low-level action sequences
3: M < of possible meta-actions
4: MATable <- table of size (IM1,AAI11) initialized with
5: for + 1 to M do
6: interval < re.finditer(M[[i..
7: for < to interval] do
8:
start, end + intervallj]
9:
if M = moveahead then
10:
[MTable]][start|]]]]
end] + 1
else
MATable [ start]|end] t-1
13:
end if end for
14:
15:
end for
16:
return MATable
17:
end procedure
Algorithm Dynamic Programming for Meta-Action Identification
1: procedure META
2: A f low-level action sequences
3: M <- set of possible meta-actions
4:
DP array of size |A] initialized with co
5: DP(0] <
MATable CRESTEMETTAATTIONAAELLEA. M)
7: MetaActions <- array of size |A| initialized with
8: for - to AA| do for j - to JM| do for + 1 to |M| do if MATable [b] == then
9:
10:
11:
12:
if DP + 11 + 1] then DP +1] DP
MetaActions(j + +1] MetaActions(jj.copy
MetaActions(j + .p.ppeed((k)
16:
end
17:
18:
19:
20:
end for
21: return MetaActiion[ :]
end procedure
ated with each meta-action. Using regular expres- sions, we identify all matching intervals for the meta-actions. If [ā]]ble(i]]]]k{ is equal to 1, it indicates that the i-th meta-action is valid from the j itth action to the k-th action.
tailed in Algorithm 2. gramming table with the length of the transformed action sequence. Each cell in the table represents the optimal meta-action sequence up to that point, We iterate through the table, starting from the first cell, and update each cell by considering all possi- ble meta-actions that match the corresponding sub-
We initialize a dynamic pro-
Once we have the MATable, we can perform DP to find the optimal meta-action sequences, as de-

---

Seen
Unseen
Method
Val
Test
Val
Test
SR
GC
SR
GC
SR
GC
SR
GC
SEQ2SEQ (Shridhar al., 2020)
2.1
7.0
2.0
6.3
0.0
5.1
0.1
4.3
MOCA (Singh et al., 2021)
19
26.4
19.5
26.3
3.2
10.4
4.2
11.2
EmBERT (Suglia et al., 2021)
28.8
36.4
23.4
31.3
3.1
9.3
3.6
10.4
E.T. (Pashevich et al., 2021)
24.6
31.0
20.1
27.8
1.8
8.0
2.6
8.3
LACMA
27.5
33.8
24.1
31.7
5.1
12,2
5,8
13.5
Table 10: Path-Length Weighted (PLW) results on ALFRED. Note that SR and GC denote task success rate and goal condition success rate, respectively. (f: trained without using data from unseen environments.)
Seen
Unseen
A.5
Preliminary Investigation on Backtracking
Method
SR
GC
SR
GC
Since proposed contrastive learning and meta actions are orthogonal to backtracking, we believe that incorporating backtracking would further im- prove the performance of our LACMA. From Ta- ble we can see that LACMA can be extended to incorporate backtracking and further improve the success rate. Specifically, we first use E.T. to predict a sequence of subgoals and input them into LACMA. If the interaction subgoal fails, we revert the preceding navigation subgoal. We be- lieve further study on more sophisticated BT meth- ods can be interesting future works.
LACMA
36.9
42.8
8.2
18.0
LACMA backtrack
37.1
43.8
10.2
20.6
Table 1I:
Results of applying naive backtracking tech- nique to the proposed LACMA.
string of the transformed action sequence. Among these meta-actions, we select the one that will lead to the minimal use of meta-actions so far, and up- date the current cell with this optimal meta-action sequence, along with the number of meta-actions used. We gradually fill the dynamic programming table until we reach the end of the sequence. Fi- nally, the DP algorithm traces back through the table retrieve the optimal meta-action sequence.
A.6
More Qualitative Results of the learned Meta-Actions
We provide more results in Fig. 6. The visualization further illustrates the learned meta-actions and their alignment with the retrieved instructions. Despite potential variations from the annotated meta-action sequences, the generated meta-action sequences remain valid and demonstrate a strong correspon- dence the language instructions. These supple- mentary visualizations provide a comprehensive view of the effectiveness and robustness of our ap- proach.
A.4 Path-Length Weighted (PLW) Scores on ALFRED
Path-length weighted (PLW) scores for vision-and- language navigation are proposed in Anderson et al. (2018a). The path-weighted score 8p defined as:
L
Sp $ X
(2)
max L, D)
where L denotes the path length of the ground- truth trajectories, and L represents the length of the predicted paths.
From Table 10 can observe consistent perfor- mance trends reported in the Table 2, where our model substantially improves the task performance in the unseen environments terms of success rate (SR) and goal condition success rate (GC).

---

Low-level
RotateRight
RotateRlight
MoveAhead
RotateLeft
MoveAhead
Action:
Labeled MA
Turn Right
Step Right
Move
(By DP)
Forward
Predicted MA
Turn Around
Move
Turn Left
Move
Forward
Forward
Retrieved Instruction
Turn around and take step forward, then turn left and
walk over to the oven
Low-level Action:
RotateLeft
MoveAhead
RotateRight
MoveAhead
RotateLeft
Labeled MA (By DP)
Step
Turn
Predicted MA
Turn Left
Step Right
Retrieved Instruction
Carry the tomato around the table to the microwave
above the stove
Low-level
RotateLeft
RotateLeft
MoveAhead
MoveAhead
MoveAhead
Action:
Labeled MA (By DP)
Turn Around
Move Forward
Predicted
Turn Around
Move Forward
Retrieved Instruction
Turn around and walk to the sink by the window
Figure 6: Visualization of the learned meta- meta-actions and the retrieved low-level instructions. Segments with different color indicating different kinds of meta-actions


Reviews:
[{"review_id": "5yTLjn7rdr", "reviewer": "Reviewer_jyH4", "summary": "", "questions": "See above", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer jyH4's valuable suggestions and their recognition of our work's value on *disentangled learning for better understanding of EIF agents*. Please see our responses below for each raised issue.\n\n\n> **Q1.** The baseline models used for comparison are much earlier models for the ALRED challenge. This is hardly impressive if that’s the focus (check the leaderboard).\n\n**A1.** We acknowledge the advancements been made in the field and discussed them in related works. However, we would like to clarify that achieving top performance on ALFRED is *not* our focus. LACMA's core objective is an embodied agent that *faithfully follows instructions* (L62). Top ALFRED leaderboard models, on the other hand, optimizes the task success rate (SR) only, by planner-based approaches that explore the environment *regardless of the instruction*.  More specifically, planner-based methods such as FILM [1] and Prompter [2] replace the natural language instruction with predefined goal templates and then conduct extensive exploration to find the goal, ignoring the instructions that usually describe how to approach the goal. Although this achieved high SR, the agents *do not follow instructions*. \n\nThis limitation becomes evident when examining the maximum achievable instruction fidelity score (CLS). For instance, in the case of FILM, its CLS is capped at 0.4 due to a path-length-weight (PLW) of 0.4, in both seen and unseen data splits. In contrast, our method achieves PLW values of 0.74 and 0.62, respectively. Consequently, the resulting CLS scores are 0.65 and 0.45 for these respective subsets.\n\nFurthermore, our claim of *better instruction following* has experimental support: LACMA successfully encodes the instruction in its internal state (Sec. 4.5), which leads to more faithful instruction following trajectories. Please see the additional faithfulness evaluation in **A2**.\n \n> **Q2**. In fact, the current set up in ALFRED may not serve to address the fidelity of instruction following, especially evaluation metrics. In VLN, researchers have looked at similar problems. Jain et., 2019 cited by the paper looked at the fidelity of following language instructions. A similar set up on fidelity might be helpful here.\n\n**A2.** Thanks for suggesting the fidelity metrics! We evaluate path coverage (PC), length scores (LS), and converage weighted by length score (CLS) on the ALFRED validation set for E.T. and LACMA. The results are summarized below.\n\n|                       | **Seen** |          |          | **Unseen** |          |          |\n|-----------------------|----------|----------|----------|------------|----------|----------|\n|                       | **PC**   | **LS**   | **CLS**  | **PC**     | **LS**   | **CLS**  |\n| E.T.                  |     90.1 |     66.7 |       60 |       82.4 |     50.1 |     41.3 |\n| LACMA                 | **92.3** | **70.8** | **65.4** |   **85.4** | **52.3** | **45.3** |\n\nFrom the table, it's evident that our approach consistently outperforms E.T. across all fidelity metrics. This suggests that our proposed method improves the instruction following fidelity over E.T.\n\n\n\n> **Q3.** Meta actions proposed in the paper concern navigation actions. Its use might be limited to manipulation actions especially if the patterns will need to be defined before training. Just for the navigation task, how much does it cover the ALFRED data? Any errors during inference time are attributed to insufficient coverage of patterns?\n\n**A3.** We'd like to clarify that the proposed meta actions achieve *full coverage* for navigation actions since we already include the atomic action (RotateLeft/RotateRight/MoveAhead) in the meta action set. The set of meta actions used during training are presented in Table 6 in Appendix A.2. Given any trajectories in ALFRED's navigation action space, our dynamic programming algorithm can find the optimal (shortest) meta actions to represent the trajectories (L249). As a result, these atomic actions won’t be used unless it can yield the shortest sequence. We also provide the pseudo code in Appendix A.3.\n\n> **Q4.** Any reason the path length weighted metrics are not reported in the paper?\n\n**A4.** We did not include them at submission for simplicity. Here we provide the corresponding path weighted version for each metric in parentheses.\n\n| **Method** |      **Seen**       |                     |                     |                     |    **Unseen**     |                   |                   |             |\n| ---------- |:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-----------------:|:-----------------:|:-----------------:|:-----------:|\n|            |       **Val**       |                     |      **Test**       |                     |      **Val**      |                   |     **Test**      |             |\n|            |       **SR**        |       **GC**        |       **SR**        |       **GC**        |      **SR**       |      **GC**       |      **SR**       |   **GC**    |\n| SEQ2SEQ    |      3.1 (2.1)      |       10 (7)        |        4 (2)        |      9.4 (6.3)      |       0 (0)       |     6.9 (5.1)     |     0.4 (0.1)     |   7 (4.3)   |\n| MOCA       |      25.9 (19)      |     34.9 (26.4)     |     22.1 (19.5)     |     28.3 (26.3)     |     5.4 (3.2)     |    16.2 (10.4)    |     5.3 (4.2)     | 14.3 (11.2) |\n| EmBERT     | **37.4** (**28.8**) | **44.6** (**36.4**) |     31.8 (23.4)     |     39.2 (31.3)     |     5.7 (3.1)     |    15.9 (9.3)     |     7.5 (3.6)     | 16.3 (10.4) |\n| E.T.       |     34.7 (24.6)     |       42 (31)       |     28.9 (20.1)     |     36.3 (27.8)     |     3.5 (1.8)     |     13.6 (8)      |     4.7 (2.6)     | 14.9 (8.3)  |\n| LACMA      |     36.9 (27.5)     |     42.8 (33.8)     | **32.4** (**24.1**) | **40.5** (**31.7**) | **8.2** (**5.1**) | **18** (**12.2**) | **9.2** (**5.8**) | **20.1** (**13.5**) |\n\nWe observe consistent performance trends as reported in the original manuscript's Table 2. We will update this table in the revision.\n\n[1] FILM: Following Instructions in Language with Modular Methods. Min et al., ICLR 2022.\n[2] Prompter: Utilizing Large Language Model Prompting for a Data Efficient Embodied Instruction Following. Inoue et al., arXiv preprint 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "-\tDisentangling learning to *follow language instructions” from learning to “perform tasks” by following language instructions can provide a better understanding of agents’ abilities.", "reasons_to_reject": "-\tThe baseline models used for comparison are much earlier models for the ALRED challenge. In the last two years, many models have been developed that have pushed the results far beyond the reported results by this work.  One of the conclusions “Compared to a strong multi-modal Transformer baseline, we achieve a significant 4.5% absolute gain in success rate at the unseen environments of ALFRED Embodied Instruction Following”. This is hardly impressive if that’s the focus (check the leaderboard). \n-\tIn fact, the current set up in ALFRED may not serve to address the fidelity of instruction following, especially evaluation metrics. In VLN, researchers have looked at similar problems.  Jain et., 2019 cited by the paper looked at the fidelity of following language instructions. A similar set up on fidelity might be helpful here. \n-\tMeta actions proposed in the paper concern navigation actions. Its use might be limited to manipulation actions especially if the patterns will need to be defined before training.  Just for the navigation task, how much does it cover the ALFRED data? Any errors during inference time are attributed to insufficient coverage of patterns? \n-\tAny reason the path length weighted metrics are not reported in the paper?", "questions_to_authors": "See above", "paper_topic_and_main_contributions": "This paper incorporates contrastive learning and meta actions (mid level action semantics) in a neuro-symbolic architecture for embodied instruction following. The main objective is to enable agents to follow instructions faithfully. The model is trained and evaluated on the ALFRED benchmark.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kjs1j7JBWC", "reviewer": "Reviewer_UNnF", "summary": "", "questions": "a. The method on how to achieve meta actions could be explained further. Is there a fixed set of meta actions for the model to classify during training? What if two similar instructions share most of the same low level instructions but differ by one or two small steps? How scalable is the added meta-action learning step?\n\nb. The same high level instruction \"take a step left\" could be applied in various environment. If the method forces the alignment between instructions (z^w) and visual action embeddings (z^v), wouldn't the method overfit the training visual data?\n\nc. Table 3: why is the performance of (Gumbel only) and (contrastive learning + DP)  worse than no training objective at all (first line)? Ln 113, if DP reduces the performance, what is the purpose of this statement? \n\nd. According the the qualitative examples, the \"high level\" instructions actually offers a lot more details than natural human instructions. For example, instead of saying \"take a half step left and walk until you are about two feet from the fireplace\", human would probably just say \"get close to the fireplace\". How would this method be able to generalize to unspecified levels of detailed instructions?\n\ne. A lot of the time, a high level language instruction can be translated to a set of low level instructions, differing by minor details that are environment specific. Would be helpful to include an ablation study to demonstrate the importance of the visual input. \n\nQuestions on details:\n\nf. Does the input camera observation history include all the way right before the action or after the action?\n\ng. During contrastive learning, how are different timestamped & length instructions mapped to camera and action since L is not equal to T?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer UNnF for the positive comments and suggestive remarks. We are excited to see that the reviewer finds our work (1) is a new method, (2) bridge the gap between instructions and actions, (3) strengthened alignment between instructions and and visual action embeddings, and (4) thorough experiments and improved performance. Please see our responses below for each raised issue.\n\n> **Q1.** The method on how to achieve meta actions could be explained further. Is there a fixed set of meta actions for the model to classify during training? What if two similar instructions share most of the same low level instructions but differ by one or two small steps? How scalable is the added meta-action learning step?\n\n**A1.** We are happy to clarify this issue. Yes, during the pre-training stage, we directly optimize the model using the DP-labeled meta action sequences (L282). In scenarios where two instructions exhibit similarity but vary slightly by one or two steps, we don't have to modify the set of meta actions, as we already include the atomic action (RotateLeft/RotateRight/MoveAhead) in the meta action set. The set of meta actions used during training are presented in Table 6 in Appendix A.2. Since DP will generate meta action sequences with minimal length, these atomic actions won't be used unless it can yield the shortest sequence.\n\nAs a result, our proposed method can efficiently scale with a diverse range of trajectories. Referencing L254-L278, given a set of meta-actions, the designed dynamic programming algorithm can generate the optimal meta-action sequences to represent the trajectories. The detailed pseudo code and algorithmic breakdown can be found in Appendix A.3.\n\n> **Q2.** The same high level instruction \"take a step left\" could be applied in various environment. If the method forces the alignment between instructions (z^w) and visual action embeddings (z^v), wouldn't the method overfit the training visual data?\n\n**A2.** To clarify, the state representation $z^v_t$ is more than just a visual encoding --- it encompasses the comprehensive context of navigation by conditioning on full instructions $w_{1:L}$, observations up to the current time step $v_{1:t}$, and the past actions $a_{1:t-1}$ (L195-L197).\n\nBy design, this objective mitigates the risk of overfitting within the training domain, leading to enhanced generalization across diverse environments. The outcome is evidenced by the significant 4.5% absolute gain in success rate within unseen environments, as shown in Table 2.\n\n> **Q3.** Table 3: why is the performance of (Gumbel only) and (contrastive learning + DP) worse than no training objective at all (first line)? Ln 113, if DP reduces the performance, what is the purpose of this statement?\n\n**A3.** We understand the particular concern of the degraded performance when meta action is used alone, and we are glad to further clarify this issue. \n\nWe would like to first note that DP and Gumbel indicate that during fine-tuning stage, the action prediction is conditioned on the DP-labeled meta actions or the Gumbel sampled meta actions. Both methods are pre-trained using DP-labeled meta actions, as described in L279-L287. You can also find the depicted training process in Fig. 4. We now detail the reason why gumbel only model and contrastive + DP model perform worse than the baseline method.\n\nIn L297-L310 of the original manuscript, we mentioned that conditioning on the DP-labeled meta actions for action prediction can lead to serious train-test mismatch, where meta actions are predicted rather than labeled. By leveraging Gumbel-Softmax, our model is able to condition on the predicted meta actions for action prediction during fine-tuning, reducing the train-test disparities, and also enable the meta action predictor to be jointly updated.\n\nThe observed performance degradation when using only meta actions can be attributed to a phenomenon akin to what we elaborated upon in Section 4.4. In the absence of the contrastive objective, the model tends to overly depend on visual cues to predict meta actions. Importantly, as the action prediction process is conditioned on meta actions, any inaccuracies originating from the over-dependence on visuals can propagate through the system, resulting in an undesired reduction in performance.\n\nHowever, in Table 3 we show that with proper regularization (the contrastive objective), the model's performance can be further improved comparing to using the contrastive objective alone. For the sake of clarity, we present the results from Table 3 below for discussions.\n\n| Contrastive objective |  Meta Actions  |   Seen   |          | Unseen  |        |\n|:---------------------:|:--------------:|:--------:|:--------:|:-------:|:------:|\n|                       |                |  **SR**  |  **GC**  | **SR**  | **GC** |\n|           x           |       x        |   34.7   |   42.0   |   3.5   |  13.6  |\n|           v           |       x        |   35.4   |   42.4   |   5.0   |  15.2  |\n|           v           | gumbel-sampled | **36.9** | **42.8** | **8.2** | **18.0** |\n\nThis is because the proposed meta actions encapsulate higher-level semantics that exhibit an intuitive alignment with the instructions. The notable improvements observed in the unseen domain further validate our hypothesis that this aligning nature facilitates a better comprehension of language within our model. As a result, our model demonstrates more effective action prediction when it comes to generalizing across diverse environments.\n\n> **Q4.** According the the qualitative examples, the \"high level\" instructions actually offers a lot more details than natural human instructions. For example, instead of saying \"take a half step left and walk until you are about two feet from the fireplace\", human would probably just say \"get close to the fireplace\". How would this method be able to generalize to unspecified levels of detailed instructions?\n\n**A4.** In fact, ALFRED training data comprises instruction of varying levels of specification, ranging from detailed step-by-step instructions to those with more general phrasing. Here are some example of general instructions: *\"Move away from the counter\"*,  *\"go through the kitchen\"*, which fall into the category of less specific directions. LACMA is able to generalize to unspecified levels of details due to the fact that each action trajectory is independently annotated by 3 annotators in ALFRED. The model is therefore trained to relate potentially different levels of instructions to the same visual input. This process equips the model to effectively navigate scenarios with both generic and detailed instructions.\n\n> **Q5.** A lot of the time, a high level language instruction can be translated to a set of low level instructions, differing by minor details that are environment specific. Would be helpful to include an ablation study to demonstrate the importance of the visual input.\n\n**A5.** Thanks for suggesting an additional experiment! We present the results in the table below.\n\n|                  |  Seen  |        | Unseen |        |\n| ---------------- |:------:|:------:|:------:|:------:|\n|                  | **SR** | **GC** | **SR** | **GC** |\n| LACMA w/o vision |   0.0  |  2.9   |  0.0   |   3.5  |\n| LACMA            |  36.9  |  42.8  |  8.2   |  18.0  |\n\nNote that the ALFRED environment requires the agent to predict an \"interaction mask\" based on the camera observation before it can interact with objects. Therefore, the above experiment only removes the visual input while agent navigates, i.e., not interacting with any object.\n\nOne can see that the agent can barely success without visual input. After a closer examination, we found the only succesful instances all require only few navigation steps. The results demonstrate the importance of the visual information.\n\n> **Q6.** Does the input camera observation history include all the way right before the action or after the action?\n\n**A6.** Yes, the input camera observation history indeed includes all frames right up to the moment before the action is taken. During training, while we employ teacher forcing and input all frames at once, we also incorporate a causal attention mask. Similar to autoregressive language models, this mask ensures that the model does not have access to information from future frames.\n\n> **Q7.** During contrastive learning, how are different timestamped & length instructions mapped to camera and action since L is not equal to T?\n\n**A7.** Thanks for raising a question to one important implementation detail! During training, we establish and maintain a mapping denoted as $pos(t)$ (L209). This mapping ensures that each frame at timestamp $t$ is correctly aligned with its corresponding language instructions. Note that this alignment can be derived from the official ALFRED annotations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper introduced a new method for the embodied instruction following tasks hoping to alleviate the seen environment overfitting problem by adding a meta-action prediction. The paper designed two meta-action prediction methods to bridge the gap between high level natural language instructions and low lever instructions. The paper also proposed a contrastive learning scheme to strengthen the alignment between instructions and visual action embeddings. The paper conducted thorough experiments on the ALFRED dataset and demonstrated improved performance compared to baseline works.", "reasons_to_reject": "Several questions below.", "questions_to_authors": "a. The method on how to achieve meta actions could be explained further. Is there a fixed set of meta actions for the model to classify during training? What if two similar instructions share most of the same low level instructions but differ by one or two small steps? How scalable is the added meta-action learning step?\n\nb. The same high level instruction \"take a step left\" could be applied in various environment. If the method forces the alignment between instructions (z^w) and visual action embeddings (z^v), wouldn't the method overfit the training visual data?\n\nc. Table 3: why is the performance of (Gumbel only) and (contrastive learning + DP)  worse than no training objective at all (first line)? Ln 113, if DP reduces the performance, what is the purpose of this statement? \n\nd. According the the qualitative examples, the \"high level\" instructions actually offers a lot more details than natural human instructions. For example, instead of saying \"take a half step left and walk until you are about two feet from the fireplace\", human would probably just say \"get close to the fireplace\". How would this method be able to generalize to unspecified levels of detailed instructions?\n\ne. A lot of the time, a high level language instruction can be translated to a set of low level instructions, differing by minor details that are environment specific. Would be helpful to include an ablation study to demonstrate the importance of the visual input. \n\nQuestions on details:\n\nf. Does the input camera observation history include all the way right before the action or after the action?\n\ng. During contrastive learning, how are different timestamped & length instructions mapped to camera and action since L is not equal to T?", "paper_topic_and_main_contributions": "The paper proposed a new method for instruction following in a simulated embodied environment. It leverages a contrastive learning between action and visual input history, versus the instruction embedding. It further proposed a two-stage training by predicting meta actions as well as low level actions. The meta actions were computed either through dynamic programming or Gumbel softmax. The paper experimented the methods on the ALFRED dataset and demonstrated improved performance compared to existing works.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QeA9sQN9G3", "reviewer": "Reviewer_3Xca", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer 3Xca's valuable suggestions. We are motivated to see that the reviewer finds our work (1) easy to understand, (2) intuitive, sample, and effective, and (3) could benefit other navigation methods by introducing the proposed contrastive objective. \n\n\n> **Q1.** The effectiveness of meta-actions (MA) cannot be justified by the experiments and analysis. Table 3 shows some conflicting messages: adding MA to Episodic Transformer (ET) decreases the performance significantly (under both DP and Gumbel settings), but when the contrastive loss is present, MA slightly increases the performance. The discussion in L470-484 does not analyze the root cause of this nor gives any hypothesis, making one doubt if MA is useful at all.\n\n**A1.** We thank the reviewer for giving us the chance to clarify this issue. \n\nIn Table 3 we show that with proper regularization (the contrastive objective), the model's performance can be further improved comparing to using the contrastive objective alone. For the sake of clarity, we present the results from Table 3 below for discussions.\n\n| Contrastive objective |  Meta Actions  |   Seen   |          | Unseen  |        |\n|:---------------------:|:--------------:|:--------:|:--------:|:-------:|:------:|\n|                       |                |  **SR**  |  **GC**  | **SR**  | **GC** |\n|           x           |       x        |   34.7   |   42.0   |   3.5   |  13.6  |\n|           v           |       x        |   35.4   |   42.4   |   5.0   |  15.2  |\n|           v           | gumbel-sampled | **36.9** | **42.8** | **8.2** | **18.0** |\n\nWe hypothesize that this is because the proposed meta actions encapsulate higher-level semantics that exhibit an intuitive alignment with the instructions. The notable improvements observed in the unseen domain further validate our hypothesis that this aligning nature facilitates a better comprehension of language within our model. As a result, our model demonstrates more effective action prediction when it comes to generalizing across diverse environments.\n\nThe observed performance degradation when using meta actions alone can be attributed to a phenomenon akin to what we elaborated upon in Section 4.4. In the absence of the contrastive objective, the model tends to overly depend on visual cues to predict meta actions. Importantly, as the action prediction process is conditioned on meta actions, any inaccuracies originating from the over-dependence on visuals can propagate through the system, resulting in an undesired reduction in performance.\n\n\n> **Q2.** There are some logical flaws in the Ablation Studies section (Sec 4.3). An alternative ablation study proposal could help prove that LACMA understand language better.\n\n**A2** We are grateful for the reviewer's detailed examination of the ablation studies and the thoughtful suggestions. We'll address each concern in the following:\n\n> **Q2.1** Logic gap in table 4: Authors suggest that because there is a smaller gap for LACMA (this paper's method) before and after dropping language instruction from input compared to E.T., E.T. is thus \"not incorporating language instructions when predicting actions\" (L489). I'm afraid I am not convinced by this.\n\n**A2.1** We apologize if our language led to a misunderstanding. We didn't mean to imply that E.T. disregards language completely. Instead, the observation we aimed to convey was that \"the smaller gap in E.T.'s performance indicates potential issue that E.T. mainly relies on visual cues to predict actions rather than considering both language and vision.\" This will be updated in next paper revision.\n\n> **Q2.2** Logic flaw in Table 5: better retrieval only indicates the latent space is closer, but it is a leap to claim that the action prediction would benefit from these closer-to-language latent vectors than the vectors used by E.T.\n\n**A2.2** With respect, we clarify that we do not claim \"action prediction would benefit from these closer-to-language latent vectors\" in Sec. 4.5. Making this claim would have been a logically flaw. The intention of this experiment was to demonstrate that our approach indeed results in \"closer-to-language latent vectors than the vectors used by E.T.\", providing additional insights about LACMA beyond conventional metrics.\n\nIf specifc line is found to suggest this wrong implication, we are happy to update the manuscript in revision.\n\n> **Q2.3** Here is an alternative ablation study proposal that I think could help prove that LACMA \"understands\"/\"follows\" language instruction: for a given language instruction, apply some perturbation (change \"left\" to \"right\", \"forward\" to \"backward\"), does the model's predicted action change accordingly? Maybe ET would be less sensitive and LACMA is more sensitive? If so I think there is a stronger evidence that LACMA can understand language better.\n\n**A2.3** Regarding the proposed ablation study, we find it to be a valuable suggestion to test the model's sensitivity to language perturbations. We present this additional study in the table below.\n\n|       | Seen | Unseen |\n| ----- |:----:|:------:|\n| E.T.  | 48.2 |  47.7  |\n| LACMA | 79.7 |  79.1  |\n\nIt is evident that LACMA exhibits a notable ability to adapt its actions based on modified language instructions, which verifies our objective of building an embodied agents that can follow instructions faithfully.\n\nAs for the experimental design details, we select 100 trajectories with instructions containing phrases like \"Turn Left/Right/Around\" from the validation split. Subsequently, we will substitute the terms \"Left/Right/Around\" with alternative choices. If the model's predicted actions align with the modified instructions, we will consider the prediction accurate.\n\nIn addition, we also provide a hollistic evaluation of instruction fidelity using the metrics proposed in [1]. Specifically, we report path coverage (PC), length scores (LS), and Coverage weighted by Length Score (CLS) in the table below.\n\n|                       | **Seen** |          |          | **Unseen** |          |          |\n|-----------------------|----------|----------|----------|------------|----------|----------|\n|                       | **PC**   | **LS**   | **CLS**  | **PC**     | **LS**   | **CLS**  |\n| E.T.                  |     90.1 |     66.7 |       60 |       82.4 |     50.1 |     41.3 |\n| LACMA | **92.3** | **70.8** | **65.4** |   **85.4** | **52.3** | **45.3** |\n\nFrom the provided table one can see that our approach consistently outperforms E.T. across all categories. This suggests that LACMA excels in following instructions and demonstrates a stronger grasp of language nuances compared to E.T.\n\n> **Q3.** (not a major weakness): Agent not following instructions faithfully has been studied before. This has been termed \"Instruction Fidelity\". See: [1] and [2]. I suggest delete the claim at L131 \"We are the first ...\".\n\n**A3.** We thank the reviewer for the suggestive comment. We concur with the suggestion to remove the claim at L131. Furthermore, in response to this suggestion, we plan to incorporate experiments related to the instruction fidelity metrics proposed in [1] (see the table above in **A2.3**) within the revised version of the paper.\n\n> **Q4.** (not a major weakness): Contrastive learning between trajectory and language instruction has been studied before. See [3].\n\n**A4.** Thanks for referencing to a relevant study involving contrastive learning in the vision language navigation domain. However, we point out there are significant differences in both the approach and outcome. Regarding the approach, Liang et al. [3] contrasted data *within the same modality* (vision-to-vision, langauge-to-language), while we constrast *between instruction and state representations*. As to the outcome, Liang et al. [3] results in improved *robustness on variations* of instructions and visual scenes. In contrast, LACMA's contrastive objective enables the agent to *faithfully follow the instructions*. We will cite their paper and include this discussion in revision.\n\n> **Q5.** HiTUT [4] is also an end-to-end solution with hierarchical modeling of subgoals and actions and therefore is very relevant to the topic of this paper. I'd like to see more discussions and comparisons with it.\n\n**A5.** Thanks for your interest in seeing more comparisons. For simplicity, we did not include detailed comparisons methods that utilize backtracking / error-recovering techniques in our original submission (L588), because it might distract the readers from our focus of instruction following.\nHiTUT explicitly models subgoal to enable backtracking (BT) during navigation, and its strong performance largely came from BT (see HiTUT's Table 4).\nWe believe our proposed contrastive learning and meta actions are orthogonal to backtracking, since instruction fidelity is a distinct topic from error recovering.\nTo demonstrate that, we conduct additional experiments showing that LACMA can also incorporate backtracking.\n|                       |   Seen   |          |  Unseen  |          |\n| --------------------- |:--------:|:--------:|:--------:|:--------:|\n|                       |  **SR**  |  **GC**  |  **SR**  |  **GC**  |\n| LACMA                 |   36.9   |   42.8   |    8.2   |   18.0   |\n| LACMA w/ backtracking | **37.5** | **43.8** | **10.2** | **20.6** |\n\nThis illustrates LACMA can be extended to incorporate BT and further improve the success rate. We will include this study in revision. \nRegarding the implementation details, we first use E.T. to predict a sequence of subgoals and input them into LACMA. If the interaction subgoal fails, we revert to the preceding navigation subgoal. We believe further study on more sophisticated BT methods can be interesting future works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1: Contrastive learning between language and action is intuitive, simple to implement, and effective. Other systems, modular or end-to-end, could benefit from having this objective in their pipelines.\n\nS2: Clear presentation of the method. Easy to understand.", "reasons_to_reject": "W1: The effectiveness of meta-actions (MA) cannot be justified by the experiments and analysis. Table 3 shows some conflicting messages: adding MA to Episodic Transformer (ET) decreases the performance significantly (under both DP and Gumbel settings), but when the contrastive loss is present, MA slightly increases the performance. The discussion in L470-484 does not analyze the root cause of this nor gives any hypothesis, making one doubt if MA is useful at all.\n\nW2: There are some logical flaws in the Ablation Studies section (Sec 4.3).\n- Logic gap in table 4: Authors suggest that because there is a smaller gap for LACMA (this paper's method) before and after dropping language instruction from input compared to E.T., E.T. is thus \"not incorporating language instructions when predicting actions\" (L489). I'm afraid I am not convinced by this. LACMA's performance drops to zero while E.T. doesn't only indicate that the lack of language destroys LACMA's latent space more than it destroys E.T.'s latent space, which makes sense, b/c LACMA has an explicit object to align the latent space to the language input, but from this, we cannot deduce that E.T. doesn't incorporate language instructions; in fact, I feel it kind of proves the opposite of the what the author claims: E.T's performance is somewhat sensitive to the language input and that's a sign of it learned some information from the language. \n- Logic flaw in Table 5: better retrieval only indicates the latent space is closer, but it is a leap to claim that the action prediction would benefit from these closer-to-language latent vectors than the vectors used by E.T.\n\nHere is an alternative ablation study proposal that I think could help prove that LACMA \"understands\"/\"follows\" language instruction: for a given language instruction, apply some perturbation (change \"left\" to \"right\", \"forward\" to \"backward\"), does the model's predicted action change accordingly? Maybe ET would be less sensitive and LACMA is more sensitive? If so I think there is a stronger evidence that LACMA can understand language better.\n\nW3 (not a major weakness): Agent not following instructions faithfully has been studied before. This has been termed \"Instruction Fidelity\". See: [1] and [2]. I suggest delete the claim at L131 \"We are the first ...\".\n\nW4 (not a major weakness): Contrastive learning between trajectory and language instruction has been studied before. See [3].\n\nW5: HiTUT [4] is also an end-to-end solution with hierarchical modeling of subgoals and actions and therefore is very relevant to the topic of this paper. I'd like to see more discussions and comparisons with it.\n\n[1] \"Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation\" (https://arxiv.org/abs/1905.12255)\n\n[2] \"General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping\" (https://arxiv.org/abs/1907.05446)\n\n[3] \"Contrastive Instruction-Trajectory Learning for Vision-Language Navigation\" (https://arxiv.org/abs/2112.04138)\n\n[4] \"Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring\" (https://arxiv.org/abs/2106.03427)", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper tackles the Embodied Instruction Following (EIF) challenge. The authors propose to improve Episodic Transformer (ET), an end-to-end architecture for EIF, by introducing two components: 1. apply contrastive learning between the latent state representation used to predict action against the latent representation of the language instruction; 2. make action prediction in ET hierarchical by predicting high-level \"meta actions\" first: meta actions are commonly-appeared action sequence patterns summarized from training trajectories.\n\nThe first contrastive learning point is motivated by the observation that agents sometimes don't faithfully follow the language instruction but can still complete the task by memorizing patterns / taking shortcuts. This problem has been spotted before and studied before (contrary to the authors' claim it is a newly identified problem, see \"Weakness 3\" below), but its simplicity and effectiveness could still be of interest to the readers.\n\nThe second point on hierarchical modeling of action, however, has been extensively studied by methods on EIF in both end-to-end and modular solutions and has proven to be useful. But in this paper, the results do not seem to justify the effectiveness of the proposed \"meta actions\" (see more discussion below).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "See above.", "Typos_Grammar_Style_and_Presentation_Improvements": "L54: \"a existing\" -> \"an existing\""}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a framework combining contrastive learning and high-level meta actions computed in different ways for embodied instruction following in the ALFRED benchmark. They show improved performance on the benchmark with their method and evaluate the ablated components of their system to assess the difference in performance. The authors amended and revised some flaws in the ablation studies pointed out by reviewers which has made and I urge them to consider other points raised as well, but overall this paper is a good contribution and would raise interesting discussions in the field.
Decision: Accept-Main