Compositional Generalization in Grounded Language Learning via Induced Model Sparsity (2022)

Abstract: We provide a study of how induced model sparsity can help achieve compositional generalization and better sample efficiency in grounded language learning problems. We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We show that standard neural architectures do not always yield compositional generalization. To address this, we design an agent that contains a goal identification module that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. The output of the goal identification module is the input to a value iteration network planner. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations. We examine the internal representations of our agent and find the correct correspondences between words in its dictionary and attributes in the environment.

## introduction ideally, when training an agent that acts upon natural language instructions, we want the agent to understand the meaning of the words, rather than overfitting to the training instructions. we expect that when an agent encounters an unfamiliar instruction made up of familiar terms, it should be able to complete the task. in this sense, the agent learns to leverage both groundedness of language; for example in english, tokens in the language map to observed attributes of objects or phenomena in its environment, as well as its compositionality; which enables the description of potentially infinite numbers of new phenomena from known components @xcite . using language to express goals is potentially a way to approach task distribution shift and sample efficiency, key problems in 1 github.com/aalto-ai/sparse-compgen reinforcement learning @xcite . however, compositional generalization does not come automatically with standard architectures when using language combined with multi-modal inputs, as indicated by the mixed results of generalization performance in @xcite ; @xcite . concurrently with @xcite , we show that the transformer architecture can demonstrate generalization, but requires large amounts of data for training. in this work, we tackle sample inefficiency and retain generalization. our contributions are as follows. we propose a model and a training method that utilizes the inductive biases of sparse interactions and factor compositionality when finding relationships between words and disentangled attributes. we hypothesize that such sparsity in the interactions between object attributes and words (as opposed to just their representations) leads to a correct identification of what attributes the words actually correspond to, instead of what they are merely correlated with. we show in both quantitative and qualitative experiments that such sparsity and factor compositionality enable compositional generalization. to improve sample efficiency, we decouple the goal identification task (which requires language understanding) from the planning process (implemented with an extension of value iteration networks). ## experimental setup we study the performance of our proposed approach on the gotolocal task of the babyai environment. a detailed description of the environment is given in appendix a. the environment can be seen as a goal-conditioned markov decision process, (formally defined in @xcite ). each episode is generated by a seed i and has an initial state s (i) 0 . to obtain a reward during an episode, the agent must successfully complete the languageencoded instruction (denoted g) that it is given. the language is simple and generated by the use of a templating system. gotolocal consists only of statements "go to (a|the) (color) (object)". each state is a fully observable 8-by-8 grid world and each cell (denoted c ij ) may contain an object, the agent, or nothing. the information in each cell is disentangled; the object's color is in a separate channel to the object's type. we work with disentangled observations because they have been shown to improve the performance and sample-efficiency of attentionbased models (see, e.g., @xcite . this disentanglement is preserved by embedding each component separately as factored embeddings q a . the environment also comes with an expert agent which can produce an optimal trajectory for a given initial state and goal τ (i) |s 0 , g. the key performance metric is success rate. a success happens if the agent completes the instruction within 64 steps. we study compositional generalization and sample efficiency. by compositional generalization we mean maintaining performance when navigating to objects with attribute combinations not seen during training. to study this, we separate goals into g id and g ood following the principle of leaving one attribute combination out (shown in by sample efficiency we mean achieving a high level of performance given a smaller number of samples than conventional methods might require. we denote n as the number of trajectories per goal that an agent has access to and study performance at different levels of n . we train various models using d train and describe the training methodology and results in section 5.1. ## designing a learning method we now design a learning agent with section 2 in mind. to complete an instruction, the agent needs to identify the goal and plan actions to reach it. the learning problem is decomposed into separate modules with separate training processes. subsections 4.1 and 4.2 describe a sparse vision-language architecture and training process for identifying goal cells (s(s, g) ∈ r h×w ). subsection 4.3 shows how to plan given that identification π(a t |s(s, g)). ## conclusion we studied the problem of compositional generalization and sample efficient grounded language learning for a vision-language navigation agent. we showed that even under strong assumptions on environment conditions such as full observability and disentanglement of inputs, compositional generalization and sample efficiency do not arise auto-matically with standard learning approaches. we demonstrate how such conditions can be leveraged by our sparse factored attention model presented in section 4.1. we demonstrate a method to learn goal identification without labels in section 4.2 and planning section 4.3 using a small number of offline trajectories. we further showed superior sample efficiency and generalization performance in section 5.1 and perform a model analysis and ablation study in section 5.2 to show how our proposed approach works the way we intended.