Finding Sub-task Structure with Natural Language Instruction (2022)

Abstract: When mapping a natural language instruction to a sequence of actions, it is often useful toidentify sub-tasks in the instruction. Such sub-task segmentation, however, is not necessarily provided in the training data. We present the A2LCTC (Action-to-Language Connectionist Temporal Classification) algorithm to automatically discover a sub-task segmentation of an action sequence.A2LCTC does not require annotations of correct sub-task segments and learns to find them from pairs of instruction and action sequence in a weakly-supervised manner. We experiment with the ALFRED dataset and show that A2LCTC accurately finds the sub-task structures. With the discovered sub-tasks segments, we also train agents that work on the downstream task and empirically show that our algorithm improves the performance.

## introduction building computational agents that execute actions given natural language instruction has a great deal of potential in real-world applications. one common approach is to cast the problem as mapping from instruction text into action sequence, and train an agent with supervised learning @xcite . a challenge on effective machine learning stems from a long horizon of the tasks. typical navigation tasks often involve more than a paragraph of instructions @xcite @xcite . in such cases, many existing approaches exploit the task hierarchy, e.g., decompose one episode of the task into sub-tasks and treat them as separate training instances @xcite , incorporate the hierarchical information into the model @xcite , or aid the learning process with progress monitoring @xcite . however, annotations for such a decomposition are not necessarily available in training data. in this case, previous work has attempted to perform sub-task segmentation by augmenting data through crowdsourcing @xcite , or developing a heuristic algorithm @xcite . in this paper, we present a2lctc (action-to-language connectionist temporal classification), an unsupervised algorithm that automatically discovers a sub-task segmentation of an action sequence. given pairs of a natural language instruction and action sequence, a2lctc maps each action to a fine-grained instruction (figure 1 ). we formulate the problem of sub-task segmentation as classification of each action into a finegrained instruction. inspired by the connectionist temporal classification algorithm @xcite , we consider an objective function that maximizes the log-likelihood of the coarsely aligned data. this formulation allows us to learn the classification in a weakly-supervised manner without any need of the ground truth mapping. we experiment with the alfred dataset which involves navigating a robot to perform household task @xcite . a2lctc successfully discovers the sub-task information in the unsegmented training data ( §3), which are shown to be useful for the downstream task ( §4). ## 2 learning sub-task segmentation given a training instance of a navigation task which consists of language instruction x and action sequence @xmath0, ..., a t ], we aim at finding a decomposition of the instance (x, a) = [(x 1 , a 1 ), ..., (x l , a l )], which is semantically plausible and useful for learning the task. our approach starts with dividing the instruction into fine-grained segments. in the experiment, the segment x i corresponds to a verbal phrase (e.g., "go to the desk") extracted from the instruction using a simple rule-based algorithm (appendix a.1). ## experiments we test a2lctc with the alfred dataset (shridhar et al., 2020) 2 , where the agent performs household tasks in a simulated indoor environment given english instruction. we choose this dataset because (1) the data involves relatively long sequence of actions (over 100); (2) it offers ground-truth sub-task segmentation data which allows the evaluation of our algorithm. although the alfred task itself involves understanding visual inputs, which is currently beyond the scope of a2lctc, the action set defined in the dataset is semantically rich enough for a2lctc to solve the segmentation problem. ## evaluation with the downstream task we evaluate the effectiveness of sub-task segments induced by a2lctc on the downstream alfred task. models. our baseline agent (baseline) is based on the cnn-lstm sequence-to-sequence architecture in @xcite , which takes the whole instruction and current state as input and then predicts an action at each time step. unless specified, we use the same hyperparameters as the original implementation. to incorporate the sub-task information, we extend the baseline with the progress monitoring module @xcite . we use two progress monitoring schemes from @xcite , which estimate the current time step and the completed sub-tasks. specifically, the modules are trained to predict the proportion of elapsed steps or completed sub-tasks to the total numbers. we evaluate the segmentation of a2lctc as well as the two baseline methods (uniform and bpe). those algorithms are applied on the training data and the agents are trained with the progress monitoring according to the segmentation. metric. we evaluate the agents with the subgoal sequence accuracy. the agent predicts the next valid_seen valid_unseen baseline 57.6 ± 2.0 29.6 ± 1.5 uniform 63.6 ± 2.5 38.8 ± 1.4 bpe 66.0 ± 2.0 39.1 ± 0.4 a2lctc 69.7 ± 1.4 41.4 ± 0.8 ## conclusion we presented a2lctc, which finds a hierarchical structure of an action sequence by mapping each action to fine-grained natural language instructions without ground-truth mapping data. we demonstrated that a2lctc successfully learns meaningful segments and training the alfred agents with these segments leads to improved performance. a2lctc currently relies only on semantic correspondence between actions and text. applying a2lctc to the tasks with low-level actions is an important extension, e.g., actions specifying the direction to move. furthermore, the instruction may often describe the visual input whose information is not encoded in the actions. another important future direction is to incorporate visual or additional information to tackle a broader range of domains. embedding size 50 window size 1 # of iterations 15 # of negative samples 5