Generalization in Instruction Following Systems (2021)

Abstract: Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence. In this paper, we focus on instruction understanding in the blocks world domain and investigate the language understanding abilities of two top-performing systems for the task. We aim to understand if the test performance of these models indicates an understanding of the spatial domain and of the natural language instructions relative to it, or whether they merely over-fit spurious signals in the dataset. We formulate a set of expectations one might have from an instruction following model and concretely characterize the different dimensions of robustness such a model should possess. Despite decent test performance, we find that state-of-the-art models fall short of these expectations and are extremely brittle. We then propose a learning strategy that involves data augmentation and show through extensive experiments that the proposed learning strategy yields models that are competitive on the original test set while satisfying our expectations much better.

## introduction building agents that can understand and execute natural language instructions in a grounded environment is a hallmark of artificial intelligence @xcite . there is wide applicability of this technology in navigation @xcite @xcite , collaborative building @xcite , and several others areas @xcite . the key challenge underlying these and many other applications is the need to understand the natural language instruction (to the extent that it is possible) and ground relevant parts of it in the environment. while the use of deep networks has led to significant progress on several 1 our code is publicly available at: http://cogcomp.org/page/publication_view/936 benchmarks @xcite an investigation into the instruction understanding capabilities of such systems remains lacking. we do not know the extent to which these models truly understand the spatial relations in the environment, nor their robustness to variability in the environment or in the instructions. this understanding is also important from the viewpoint of safety critical applications , where robustness to variability is essential. while robustness to input perturbations at test-time has been studied in computer vision @xcite and in certain natural language tasks @xcite @xcite , it remains relatively elusive in the instruction following task in a grounded environment. this can be attributed to the difficulty in characterizing the different expectations of robustness in this setting, due to the multiple channels of input involved, which semantically constrain each other. the blocks world domain is an ideal platform to study the abilities of a system to understand instructions @xcite @xcite @xcite @xcite . despite being seemingly simple, it presents key reasoning challenges, including compositional language understanding and spatial understanding, that need to be addressed in any instructional domain. in @xcite , the environment consists of a number of blocks placed on a flat board. the model is provided with the current configuration of blocks in the environment along with an instruction, and is tasked with executing the instruction by manipulating appropriate blocks. in this work, we follow the more challenging setting in @xcite where the blocks are unlabeled, necessitating the use of involved referential expressions in the instructions. fig. 1 shows that the instruction and block configuration are semantically dependent, jointly determining the outcome. despite the success of top performing models @xcite on the test set for this task, we question if the models are able to reason about the complex language and spatial concepts of this task and generalize or are merely over fitting the test set. to investigate these questions we formulate the following expectations one should have from an instruction following model: (1) identity invariance expectation: the performance of the model on an input should not degrade on slightly perturbing the input. our expectations complement existing work in three dimensions: (1) is related to adversarial perturbations @xcite and ( 2 ) is related to equivariance of cnns explored in computer vision @xcite . it is also related to contrast sets @xcite and counterfactual data augmentation @xcite . here, we extend the investigation to this new task of instruction following involving both natural language and an environment, discrete and continuous perturbations and both regression and classification tasks. contrast (3) is related to @xcite where vulnerability to length in a toy sequence-sequence task was demonstrated. here we show that length-based vulnerability exists in another modality-the number of blocks present on the board, for this much more complicated task. while these form only a subset of the expectations one might have from an instruction following model, it already allows us to formally characterize some of the dimensions of robustness an instruction following agent must have. as an example, a tiny shift in the location of each block should not affect the model prediction (identity invariance). in sec. 2, we formulate concrete perturbations to test whether a given model satisfies these expectations. the space of perturbations that we consider have the following attributes: (a) semantic preserving or semantic altering. (b) linguistic or geometric. (c) discrete or continuous. we find that both models studied suffer a large performance drop under each of the perturbations, and fall short of satisfying our expectations. we then present a data augmentation scheme designed to better address our expectations from such models. our extensive experiments in sec. 2.3 indicate that our learning strategy results in more robust models that perform much better on the perturbed test set while maintaining similar performance on the original test set. target location to move it to. while the target output is always a location @xmath0, for the source task the model can either predict a particular block @xmath1, 2, ..., 20} (tan and bansal, 2018) or a particular source location @xmath2. let p denote a perturbation space and (i , w ) be the perturbed version of (i, w ) under p. note that (i , w ) can be chosen randomly or adversarially as the perturbation which maximizes the loss : here ℓ denotes a loss function and o denotes the gold source/target location. if the perturbation space is discrete and finite we can simple search over all candidate (i , w ) to find the one with the maximum loss. if it is continuous and infinite, we can use a first order method (eg: first order gradient signed method fgsm @xcite ) to find the adversarial (i , w ). now we characterize p. broadly, we have the following two types of perturbations: ## conclusion in this paper we formulated the performance expectations for an instruction following system. based on these expectations, we created several categories of perturbations and showed that existing models fail spectacularly on them. we then demonstrated the benefits of adversarial data augmentation on each perturbation category.