Defending Compositionality in Emergent Languages (2022)

Abstract: Compositionality has traditionally been understood as a major factor in productivity of language and, more broadly, human cognition. Yet, recently some research started to question its status showing that artificial neural networks are good at generalization even without noticeable compositional behavior. We argue some of these conclusions are too strong and/or incomplete. In the context of a two-agent communication game, we show that compositionality indeed seems essential for successful generalization when the evaluation is done on a suitable dataset.

## introduction compositionality is a property of language that describes its specific hierarchical structure. multiple atomic units (e.g., words) can be combined to produce more complex units (e.g., sentences), while the meaning of the larger units can be inferred from the simpler parts and the way they are combined. of course, there are inherently noncompositional structures in language, idioms being a prime example. if someone is making waves, it usually means that the person is causing trouble and no water is implied. still, many reputable researchers @xcite @xcite have seen compositionality as a key ingredient that enables language to be used productively, i.e., in an infinite number of novel situations. often, this productivity is juxtaposed with the learning of artificial neural networks (anns), whose performance is known to suffer when tested in new scenarios. recently, a strand in the literature has been making waves (sic) by calling into question the proposed benefits of compositionality. various papers have shown that anns can generalize well to unseen contexts (be productive) even if they work with internal representations that are noncompositional @xcite @xcite @xcite . the goal of this paper is to counterbalance these claims. 1 more precisely, we would like to relativize some of the stronger conclusions that are provided and show (by running modified experiments) that the reported experimental results can be harmonized with the view that compositionality is necessary for successful knowledge transfer. we will do this by first providing an overview of the related research in section 2. then in section 3, we give our arguments against some of the presented assumptions and/or conclusions. these arguments are further supported by experiments that are described in section 4. we conclude the paper with a short discussion in section 5. ## emergent languages generalize without compositionality much of the critical research comes from experimentation with languages that emerge during a communication game. here, two agents (anns) are trained to communicate in order to perform a certain task. the first agent (sender/speaker) encodes the input into a message, a sequence of discrete symbols. the second agent (receiver/listener) does not have direct access to the original input, but only sees it as represented by the message. the receiver's goal is to transform the message into a desired output. the output can take many forms depending on the task: reconstructing the input fully @xcite or partially @xcite or reconstructing the input after going through some deterministic transformation @xcite . inputs can be conceptualized as representations of objects by means of independent nominal at-tributes, e.g., blue circle, red square, orange triangle for two-dimensional input vectors (color and shape). a uniform random sample of all such objects is then held out for testing, the rest of the data (or its part) is used for training. @xcite and @xcite develop custom metrics to measure the compositionality of messages passed between the sender and the receiver. the metrics compare each message to the corresponding input and try to assess to what extent each part of the input (attribute value) can be isolated in the message regardless of the context (i.e., other attribute values). they report runs in which models communicate through messages with low compositionality scores, but still achieve good generalization on unseen data. @xcite replace training the first agent (sender) by hand-coding the messages. this gives them the advantage of having direct control over the emergent language. they find that good generalization can sometimes be achieved using a non-compositional language (sometimes leading to even better results than using a compositional language). ## allowing compositionality to have an effect our main argument is that the process of selecting the test data plays a crucial role in evaluating the effects of compositionality. sampling examples with uniform probability is a mainstay in machine learning, and algorithms have been shown many times that they can perform very well under these conditions. however, once we move away from the static world of i.i.d. data samples into the dynamic world of ever-changing distributions, the limitations of such models become obvious. we argue that this is where compositionality is supposed to be helpful. analyzing the world (or data points to keep the discussion down to earth) through a hierarchy of parts and their relations enables inferring a 'rule-based algebraic system', which is 'an extremely powerful generalization mechanism' @xcite . systematic compositionality exploited by human learners enables them to be sample efficient, i.e., quickly learn a new task seeing just one or a limited number of training examples @xcite . therefore, we conclude that showing compositionality not being correlated with generalization on in-domain held-out data is not very informative. instead, it is preferable to control the exposition of certain patterns in the training and testing data as illustrated by, e.g., the scan benchmark @xcite . we also point out that @xcite and @xcite implement both agents (sender and receiver) as relatively standard encoder/decoder architectures with recurrent (lstm, gru) layers. these architectures are not necessarily known for their ability to produce or utilize compositional sequences. the assumption seems to be that when running an experiment many times (with different random initialization of the models), some runs will be successful in the sense that the agents will develop a more or less compositional language by chance. moreover, the degree of compositionality would have to be large enough to influence the generalization of the models (should such an effect be real). we do not think this assumption is justified. @xcite avoid a part of the above problem by creating compositional messages manually. their main conclusion is that one can devise different tasks, and in some of them a compositional representation of the input data might even prove disadvantageous. indeed, it is possible to create, for example, an arbitrary bijective function from the input space to the output space and train a model to learn such a mapping. not surprisingly, such a model will fail at the test time. however, if we first encode the input with the same arbitrary transformation (thus creating a noncompositional representation of the input data), the model is then asked to learn the identity function, which it might achieve quite well. therefore, we agree with the conclusion that 'in isolation from the target task, there is nothing special about a language being . . . compositional.' yet, as mentioned above, compositionality is often discussed in conjunction with natural language or human cognitive abilities more generally. in the lived experience of biological species (to be less human-centric), it is reasonable to expect that the 'underlying factors of variation' or 'explanatory factors' @xcite behind the input data a) repeat in many different situations and b) are directly relevant for different tasks. for example, it is reasonable to expect that many languages have a word for water rather than a word for water and the wind is blowing, simply because the first is more useful. therefore, we believe that the em-model train in-domain test out-of-domain test sender (ours) ## experiments we follow the communication game experiments of @xcite . we create a set of instances, each of which is represented by i att attributes. each attribute has n val possible values. the messages passed between the agents are limited by the maximum length (c len ) and the size of the vocabulary (c voc ). the receiver's goal is to reconstruct the input. as messages are sequences of discrete symbols, which prevents gradients from passing through, the sender must be trained with the reinforce algorithm @xcite . the receiver is trained using backpropagation. we use the egg toolkit @xcite to implement the experiments. 2 we focus mainly on the setting of (i @xmath1, n @xmath2, c @xmath3, c @xmath4. in this case, the dataset contains isntances such as (12, 34), (0, 99), (99, 0) etc. we create three splits. the out-of-domain (ood) test set contains all pairs where 0 appears, apart from three examples: (0, 0), (0, 1), and (1, 0). the training set contains these three zero examples together with 90% of the remaining (nonzero) examples (random sample). the rest of the data constitute in-domain (ind) test set. in other words, we designate a special symbol (0), which appears only in a limited number of contexts in training. we then separately evaluate how the models perform on unseen examples with ordinary symbols (ind test set) and on unseen examples with the special symbol (ood test set). given the absence of any incentive for the models to develop compositional messages during training, we opt for architectural biases in our experiments. we use the models that have been proven to be successful in ood generalization in the scan benchmark @xcite @xcite . each agent is implemented as a separate seq2seq encoderdecoder architecture with recurrent layers and a modified attention mechanism. details are provided in appendix a ## discussion there are many questions that remain for further analysis. the distinction between the in-domain and out-of-domain data is not clear-cut. one might object that seeing just one or two examples of a given symbol in the training data is too little for anns to learn its embedding and reliably map it close to other 'similar' symbols in the semantic space @xcite . this is actually the issue as it seems that human learners unlike anns are able to succeed in such a scenario and work with limited data or, in other words, 'not-yet-converged embeddings'. see @xcite for a more thorough discussion. the goal of this paper was to show that some conclusions in the literature on compositionality are too strong or incomplete. however, there are other arguments that remain untackled. @xcite gives examples of neural networks that generalize (partially) well to out-of-domain data. for instance, @xcite show that a simple convolutional network is enough to improve accuracy from 1.2% to 60% in a difficult task from the scan benchmark. @xcite demonstrate that a language model is capable of preferring grammatical nonsense sentences (certainly not seen in training) to ungrammatical ones. in general, the practical success of anns in many applications can serve as a proof of their strong generalization abilities @xcite . in response, we would like to point out that such success often coincides with new developments in neural architectures (convolutional nns in vision, attention in nlp). these developments might actually point in the direction of compositionality. a trained convolutional nn actually detects primitive shapes (at least by the filters in the lower layers) and combines these into composit representations. similarly, a trained attention-based encoder-decoder language model represents each input as a sequence of contextualized embeddings of the original units. some of these embeddings might primarily represent the corresponding input units, and others might represent their collections. we also acknowledge that for practical applications, especially in the short term, focusing on compositionality is not guaranteed to help @xcite . the most practical way so far has been to enable training on as much data as possible. however, it is likely that such an approach will eventually result in diminishing returns. there are many potential aspects that favor compositional behavior: inductive architectural biases (e.g., attention); limited channel capacity relative to the input space @xcite ; ease of transmission within the population @xcite ; generalization pressure (contrary to @xcite ), if such a pressure is allowed to have some effect (e.g., by meta-learning); . . . . it is also possible that systematic compositionality might emerge foot_2 only as a result of multiple such factors.