Can Sequence-to-Sequence Transformers Naturally Understand Sequential Instructions? (2023)

Abstract: While many real-life tasks require reasoning over multi-step sequential instructions, collecting fine-grained annotations for each intermediate step can be prohibitively expensive. In this work, we study how general pretrained sequence-to-sequence transformers perform under varying types of annotation for sequential instruction understanding. We conduct experiments using T5 (Raffel et al., 2020) on a commonly-used multi-step instruction understanding dataset SCONE (Long et al., 2016) that includes three sub-tasks. First, we show that with only gold supervision for the final step of a multi-step instruction sequence, depending on the sequential properties of different tasks, transformers may exhibit extremely bad performance on intermediate steps, in stark contrast with their performance on the final step. Next, we explore two directions to relieve this problem. We show that with the same limited annotation budget, using supervision uniformly distributed across different steps (instead of only final-step supervision), we can greatly improve the performance on intermediate steps with a drop in final-step performance. Further, we explore a contrastive learning approach to provide training signals on intermediate steps with zero intermediate gold supervision. This, however, achieves mixed results. It significantly improves the model’s bad intermediate-step performance on one subtask, but also shows decreased performance on another subtask.

## introduction transformer-based sequence-to-sequence models @xcite have shown remarkable performance on many natural language understanding tasks including semantic parsing @xcite , dialog state tracking @xcite , procedure text understanding @xcite etc. however, much of this success relies on fine-grained annotations. for example, many instruction-following datasets @xcite contain the corresponding parse or label for every single instruction showing their immediate effects. however, such data can be hard to collect in practice because even seemingly simple and straightforward tasks can involve multiple steps, foot_0 making the collection of detailed annotations expensive and time-consuming. for these scenarios, many earlier works applied task-specific methods to provide additional inductive biases about the sequential nature of these instructions @xcite . these methods need substantial prior knowledge and can be harder to generalize to new domains. 2 in this work, first, we provide a case study to explore whether transformer-based seq2seq models trained only using end-step supervision (i.e., gold supervision is given only at the very end of the entire sequence) can naturally handle these sequential instructions without task-aware specific architecture changes. we conduct experiments on the scone dataset @xcite including three different subtasks. the input of each example contains a sequence of instructions. during training, the model only observes the final state (label) after executing all the instructions, while for evaluation, the model needs to predict both the final states and all the intermediate states. we use t5 @xcite as our baseline model. interestingly, we observe mixed trends on the three different subtasks of scone depending on their different sequential properties. on two out of three tasks (scene and tangrams), t5 models demonstrate good performance on the intermediate steps. on alchemy, however, the performance on intermediate steps is extremely bad, in stark contrast with their decent performance on final steps. such behavior reveals that the model does not learn to understand the instructions sequentially and is not maintaining a correct intermediate state. therefore, while these models may do well on instructions similar to the training examples, they can also fail miserably on instructions shorter or longer than the instructions they are trained on. hence, we next explore two potential mitigations to this problem. we first study an alternative labeling schema. we find that if the same amount of labels are uniformly sampled across multiple steps instead of only coming from the last step, the model can have substantially better performance at intermediate steps, despite a drop of the performance on the final step. this can be a favorable behavior if the target application has more focus on intermediate steps. however, re-collecting labels may not always be practical. therefore, for scenarios where only final-step labels are accessible, we also explore a contrastive learning based approach to improve the intermediate-state performance without additional gold labels. specifically, we use a contrastive learning loss to encourage an alignment between the change in the predicted states and the most recent instruction, and provide useful training signals on the intermediate steps. however, we see mixed results from this approach. while it can significantly improve the low intermediate-step performance on alchemy, it decreases performance on scene and does not further improve other models that already have decent performance. finally, we discuss the limitation of this approach and point out that the lack of precise regularization to capture the fine-grained state differences may be the reason behind the mixed results, which makes it hard to further improve strong baselines already showing sequential understanding abilities (as in scene). dataset. we use the scone dataset as it contains three different subtasks: alchemy, scene and tangrams @xcite , and covers a diverse set of different states and instructions. for every example in these three subtasks, the instruction contains 5 steps. see appendix a for examples and a detailed dataset introduction. ## intermediate state prediction with uniformly sampled annotations one of the major reasons behind the poor performance in sec. 2 is that all the gold labels are at the final step, so for the intermediate steps, there is no strong supervision to ensure a desirable behavior. while for many applications, final-step labels are indeed more natural to collect, in this section, we explore if a better annotation strategy can improve the performance with the same amount of labeling budget. specifically, we replace the final-step-only supervision with the same amount of supervision for the instruction vector, we directly feed the latest instruction inst i+1 to an off-the-shelf sentence-t5 model @xcite . 5 with these two set of embeddings, we compute a contrastive matching loss @xcite : results. the results for our contrastive learning method are in the "+cl" row of ## conclusion we study seq2seq transformers for sequential instruction following. depending on data properties, if only final-supervision is given, transformers may naturally perform well on intermediate steps, but can also fail miserably. we explore two potential remedies, one with uniformly sampled supervision, and the other with contrastive learning.