LACMA : Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following (2023)

Abstract: End-to-end Transformers have demonstrated an impressive success rate for Embodied Instruction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environment. This lack of generalizability is due to the agent’s insensitivity to subtle changes in natural language instructions. To mitigate this issue, we propose explicitly aligning the agent’s hidden states with the instructions via contrastive learning. Nevertheless, the semantic gap between high-level language instructions and the agent’s low-level action space remains an obstacle. Therefore, we further introduce a novel concept of meta-actions to bridge the gap. Meta-actions are ubiquitous action patterns that can be parsed from the original action sequence. These patterns represent higher-level semantics that are intuitively aligned closer to the instructions. When meta-actions are applied as additional training signals, the agent generalizes better to unseen environments. Compared to a strong multi-modal Transformer baseline, we achieve a significant 4.5% absolute gain in success rate in unseen environments of ALFRED Embodied Instruction Following. Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents.

## introduction embodied instruction following (eif) necessitates an embodied ai agent to interpret and follow natural language instructions, executing multiple subtasks to achieve a final goal. agents are instructed to sequentially navigate to locations while localizing and interacting with objects in a fine-grained * work partially done as a research intern at microsoft 1 the code is available at: github.com/joeyy5588/lacma. the large semantic gap between the instruction "take a half step left" and the action sequence "rotateleft, moveahead, rotateright, moveahead" may cause the agent to learn shortcuts to the camera observation, ignoring the language instruction. we propose semantic meaningful meta-actions to bridge this gap. to address the above question, we aim to improve the alignment between the language instruction and the internal state representation of an eif agent. @xcite suggest leveraging language as intermediate representations of trajectories. @xcite demonstrate that identifying patterns within trajectories aids models in adapting to unseen environments. inspired by their observations, we conjecture two critical directions: 1) language may be used as a pivot, and 2) common language patterns across trajectories could be leveraged. we propose language-aligning contrastive learning with meta-actions (lacma), a method aimed at enhancing embodied instruction following. specifically, we explicitly align the agent's hidden states, which are employed in predicting the next action, with the corresponding sub-task instruction via contrastive learning. through the proposed contrastive training, hidden states are more effectively aligned with the language instruction. nevertheless, a significant semantic gap persists between high-level language instructions, e.g., "take a step left and then walk to the fireplace" and the agent's low-level action space, e.g., moveforward, rotateright, etc. to further narrow this gap, we introduce the concept of metaactions (ma), a set of action patterns each representing higher-level semantics, and can be sequentially composed to execute any sub-task. for clarity, we will henceforth refer to the original agent's actions as low-level actions (la) for the remainder of the paper. the elevated semantics of meta-actions may serve as a more robust learning signal, preventing the model from resorting to shortcuts based on its visual observations. this concept draws inspiration from recent studies that improve eif agents with human-defined reusable skills @xcite . an illustrative example is shown in figure 1 . more specifically, our agent first predicts metaactions, and then predicts low-level actions conditioning on the mas. however, an la sequence may be parsed into multiple valid ma sequences. to determine the optimal ma sequences, we parse the trajectories following the minimum description length principle (mdl; @xcite . the mdl states that the shortest description of the data yields the best model. in our case, the optimal parse of a trajectory corresponds to the shortest ma sequence in length. instead of an exhaustive search, optimal mas can be generated via dynamic programming. to evaluate the effectiveness of lacma, we conduct experiments on the alfred dataset @xcite . even with a modest set of metaactions consisting of merely 10 classes, our agent significantly outperforms in navigating unseen environments, improving the task success rate by 4.7% and 4.5% on the unseen validation and testing environments, respectively, while remaining competitive in the seen environments. additional analysis reveals the complementary nature of the contrastive objective and meta-actions: learning from metaactions effectively reduces the semantic gap between low-leval actions and language instructions, while the contrastive objective enforces alignment to the instructions, preventing the memorization of meta-action sequences in seen environments. our contributions can be summarized as follows: figure 2 : lacma takes instructions w 1:l , camera observations v 1:t , and actions a 1:t as inputs, and then output state representations z v 1:t . we additionally extract the output features corresponding to the [sep] tokens as the representations for the sub-goal instructions z w 1:n . z v 1:t and z w 1:n are used for contrastive learning, while z v 1:t are further utilized to predict the meta-action sequences m1:m and the low-level action sequences â1:t . ## method in this section, we first define settings and notations of the embodied instruction following (eif) tasks in section 2.1. then, in section 2.2, we introduce the language-induced contrastive objective used to extract commonalities from instructions. finally, in section 2.3, we will explain how we generate the labels for meta-actions and how they are leveraged to bridge the gap between instructions and the corresponding action sequences. ## conclusion in this paper, we propose lacma, a novel approach that addresses the semantic gap between high-level language instructions and low-level action space in embodied instruction following. our key contributions include the introduction of contrastive learn-low-level action: labeled ma (by dp) predicted ma rotateleft moveahead moveahead moveahead moveahead turn left move forward turn left move forward retrieved instruction turn left and walk to the counter to the left of the sink. rotateright putobject rotateright moveahead rotateleft turn right interaction step right turn left… place the pot… turn right, then face the sink to your left turn right interaction turn right move forward turn left pickupobject rotateleft moveahead moveahead rotateright interaction step left take the apple… step left to where the pot now is interaction step left low-level action: labeled ma (by dp) predicted ma retrieved instruction low-level action: labeled ma (by dp) predicted ma retrieved instruction ing to align the agent's hidden states with instructions and the incorporation of meta-actions, which capture higher-level semantics from the action sequence. through these innovations, we achieve a significant 4.5% absolute gain in success rate on unseen environments. our results demonstrate the effectiveness of lacma in improving alignment between instructions and actions, paving the way for more robust embodied agents.