The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study (2022)

Abstract: Obtaining human-like performance in NLP is often argued to require compositional generalisation. Whether neural networks exhibit this ability is usually studied by training models on highly compositional synthetic data. However, compositionality in natural language is much more complex than the rigid, arithmetic-like version such data adheres to, and artificial compositionality tests thus do not allow us to determine how neural models deal with more realistic forms of compositionality. In this work, we re-instantiate three compositionality tests from the literature and reformulate them for neural machine translation (NMT).Our results highlight that: i) unfavourably, models trained on more data are more compositional; ii) models are sometimes less compositional than expected, but sometimes more, exemplifying that different levels of compositionality are required, and models are not always able to modulate between them correctly; iii) some of the non-compositional behaviours are mistakes, whereas others reflect the natural variation in data. Apart from an empirical study, our work is a call to action: we should rethink the evaluation of compositionality in neural networks and develop benchmarks using real data to evaluate compositionality on natural language, where composing meaning is not as straightforward as doing the math.

## introduction although the successes of deep neural networks in natural language processing (nlp) are astounding and undeniable, they are still regularly criticised for lacking the powerful generalisation capacities that characterise human intelligence. a frequently mentioned concept in such critiques is compositionality: the ability to build up the meaning of a complex expression by combining the meanings of its parts (e.g. @xcite . compositionality is assumed to play an essential role in how humans understand language, but whether neural networks also exhibit this property has since long been a topic of vivid debate (e.g. @xcite @xcite . studies about the compositional abilities of neural networks consider almost exclusively models trained on synthetic datasets, in which compositionality can be ensured and isolated (e.g. @xcite . 2 in such tests, the interpretation of expressions is computed completely locally: every subpart is evaluated independently -without taking into account any external context -and the meaning of the whole expression is then formed by combining the meanings of its parts in a bottom-up fashion. this protocol matches the type of compositionality observed in arithmetic: the meaning of (3 + 5) is always 8, independent of the context it occurs in. however, as exemplified by the sub-par performance of symbolic models that allow only strict, local protocols, compositionality in natural domains is far more intricate than this rigid, arithmeticlike variant of compositionality. natural language seems very compositional, but at the same time, it is riddled with cases that are difficult to interpret with a strictly local interpretation of compositionality. sometimes, the meaning of an expression does not derive from its parts (e.g. for idioms), but the parts themselves are used compositionally in other contexts. in other cases, the meaning of an expression does depend on its parts in a compositional way, but arriving at this meaning requires a more global approach because the meanings of the parts need to be disambiguated by information from elsewhere. for instance, consider the meaning of homonyms ("these dates are perfect for our dish/wedding"), potentially idiomatic expressions ("the child kicked the bucket off the pavement"), or scope ambiguities ("every human likes a cat"). this paradoxical tension between local and global forms of compositionality inspired many debates on the compositionality of natural language. likewise, it impacts the evaluation of compositionality in nlp models. on the one hand, local compositionality seems necessary for robust and reliable generalisation. yet, at the same time, global compositionality is needed to appropriately address the full complexity of language, which makes evaluating compositionality of state-of-the-art models 'in the wild' a complicated endeavour. in this work, we face this challenge head-on. we concentrate on the domain of neural machine translation (nmt), which is paradigmatically close to the tasks typically considered for compositionality tests, where the target represents the 'meaning' of the input. 3 furthermore, mt is an important domain of nlp, for which compositional generalisation is important to produce more robust translations and train adequate models for low-resource languages (see, e.g. @xcite . as an added advantage, compositionality is traditionally well studied and motivated for mt @xcite @xcite . we reformulate three theoretically grounded tests from @xcite : systematicity, substitutivity and overgeneralisation. since accuracycommonly used in artificial compositionality testsis not a suitable evaluation metric for mt, we base our evaluations on the extent to which models behave consistently, rather than correctly. in our tests for systematicity and substitutivity, we consider whether processing is local; in our overgeneralisation test, we consider how models treat idioms that are assumed to require global processing. our results indicate that models often do not behave compositionally under the local interpretation, but exhibit behaviour that is too local in other cases. in other words, models have the ability to process phrases both locally and globally but do not always correctly modulate between them. we further show that some inconsistencies reflect variation in natural language, whereas others are true compositional mistakes, exemplifying the need for both local and global compositionality as well as illustrating the need for tests that encompass them both. with our study, we contribute to ongoing questions about the compositional abilities of neural networks, and we provide nuance to the nature of this question when natural language is concerned: how local should the compositionality of models for natural language actually be? aside from an empirical study, our work is also a call to action: we should rethink the evaluation of compositionality in neural networks and develop benchmarks using real data to evaluate compositionality on natural language, where composing meaning is not as straightforward as doing the math. ## local and global compositionality tests for compositional generalisation in neural networks typically assume an arithmetic-like version of compositionality, in which meaning can be computed bottom up. the compositions require only local information -they are context independent and unambiguous: "walk twice after jump thrice" (a fragment from scan by @xcite is evaluated similarly to (2 + 1) × (4 -5). in mt, this type of compositionality would imply that a change in a word or phrase should affect only the translation of that word or phrase, or at most the smallest constituent it is a part of. for instance, the translation of "the girl" should not change depending on the verb phrase that follows it, and in the translation of a conjunction of two sentences, making a change in the first conjunct should not change the translation of the second. while translating in such a local way seems robust and productive, it is not always realistic -e.g. consider the translation of "dates" in "she hated bananas and she liked dates". in linguistics and philosophy of language, the level of compositionality has been widely discussed, which led to a variety of definitions. one of the most well-known ones is from @xcite : "the meaning of a compound expression is a function of the meanings of its parts and of the way they are syntactically combined." 4 this definition hardly places restrictions on the relationship between expressions and their parts. the type of function that relates them is unspecified and could take into account the global syntactic structure or external arguments, and the parts' meanings can depend on global information. partee's definition is therefore called weak, global, or open compositionality @xcite . when, instead, the meaning of a compound depends only on the meanings of its largest parts, regardless of their internal structure (similar to arithmetic), that is strong, local or closed n template ## experiments and results in our experiments, we consider systematicity ( §4.1) and substitutivity ( §4.2), to test for local compositionality, and idiom translation to probe for a more global type of processing ( §4.3). ## manual analysis our systematicity and substitutivity results demonstrate that models are not behaving compositional according to a strict definition of compositionality. however, we ourselves have argued that strict compositionality is not always appropriate to handle natural language. a reasonable question to ask is thus: are the inconsistencies we marked as noncompositional actually incorrect? annotation setup to address this question, we perform a manual analysis. we annotate 900 inconsistent translation pairs of the systematicity and substitutivity tests to establish whether the inconsistencies are benign or concerning. we consider four different types of changes: 1. cases of rephrasing, where both translations are equally (in)correct; 2. changes reflecting different interpretations of source ambiguities; 3. cases in which one of the two translations contains an error; 4. formatting (mostly punctuation) changes. for substitutivity samples, we also annotate whether the changes are related to the translation of the synonym, where we distinguish cases where i. one of the synonym translations is incorrect; ii. both are incorrect but in a different manner; iii. both are correct but translated differently; iv. one synonym remains untranslated. we annotate all changes observed per pair and report the relative frequency per class. we summarise the results, aggregated over different training set sizes and the three data types, in figure 5 . for a more elaborate analysis and a breakdown per model and data type, we refer to appendix f. ## related work in previous work, a variety of artificial tasks have been proposed to evaluate compositional generalisation using non-i.i.d. test sets that are designed to assess a specific characteristic of compositional behaviour. examples are systematicity @xcite @xcite , substitutivity @xcite , localism @xcite , productivity (lake and baroni, 2018) or overgeneralisation @xcite @xcite . generally, neural models struggle to generalise in such evaluation setups. there are also studies that consider compositional generalisation on more natural data. such studies typically focus on either mt @xcite @xcite or semantic parsing @xcite @xcite . most of these studies consider small and highly controlled subsets of natural language. instead, we focus on models trained on fully natural mt datasets, which we believe to be the setup for compositionality evaluation that does most justice to the complexity of natural language: contrary to semantic parsing, where the outputs are structures created by expert annotators, in translation both inputs and outputs are fully-fledged natural language sentences. to the best of our knowledge, the only attempt to explicitly measure compositional generalisation of nmt models trained on large natural mt corpora is the study presented by @xcite . they measure productivity -generalisation to longer sentence lengthsof an lstm-based nmt model trained on a fullsize, natural mt dataset. other studies using nmt, instead, consider toy datasets generated via templating @xcite or focus on short sentences excluding more complex constructions that contribute to the complexity of natural language for compositional generalisation, such as polysemous words or metaphors @xcite . ## discussion whether neural networks can generalise compositionally is often studied using artificial tasks that assume strictly local interpretations of compositionality. we argued that such interpretations exclude large parts of language and that to move towards human-like productive usage of language, tests are needed that assess how compositional models trained on natural data are. 11 we laid out reformulations of three compositional generalisation testssystematicity, substitutivity and overgeneralisation -for nmt models trained on natural corpora, and assessed models trained on different amounts of data. our work provides an empirical contribution but also highlights vital hurdles to overcome when considering what it means for models of natural language to be compositional. below, we reflect on these hurdles and our results. the proxy-to-meaning problem compositionality is a property of the mapping between the form and meaning of an expression. since translation is a meaning-preserving mapping from form in one language to form in another, it is an attractive task to evaluate compositionality: the translation of its sentence can be seen as a proxy to its meaning. however, while expressions are assumed to have only one meaning, translation is a many-to-many mapping: the same sentence can have multiple correct translations. this does not only complicate evaluation -mt systems are typically evaluated with bleu because accuracy is not a suitable option -it also raises questions about how compositional the desired behaviour of an mt model should be. on the one hand, one could argue that for optimal generalisation, robustness, and accountability, we like models to behave systematically and consistently: we expect the translations of expressions to be independent of unrelated contextual changes that do not affect their meaning (e.g. swapping out a synonym in a nearby sentence). additionally, model performance could be improved if small changes do not introduce errors in unrelated parts of the translation. on the other hand, non-compositional behaviour is not always incorrect -it is one of the main arguments in our plead to test compositionality 'in the wild' -and we observe that indeed, not all noncompositional changes alter the correctness of the resulting translations. changing a translation from "atleet" ("athlete") to "sporter" ("sportsman") based on an unrelated word somewhat far away may not be (locally) compositional, but is it a problem? and how do we separate such 'harmful' mistakes from helpful ones? the locality problem inextricably linked to the proxy-to-meaning problem is the locality problem. in our tests we see that small, local source changes elicit global changes in translations. for instance, in our systematicity tests, changing one noun in a sentence elicited changes in the translation of a sentence that it was conjoined with. in our substitutivity test, even synonyms that merely differed in spelling (e.g. "doughnut" and "donut") elicited changes to the remainder of the sentence. this counters the idea of compositionality as a means of productively reusing language: if a phrase's translation depends on (unrelated) context that is not in its direct vicinity, this suggests that more evidence is required to acquire the translation of this phrase. tests involving synthetic data present the models with sentences in which maximally local behaviour is possible, and we argue that it is, therefore, also desirable. our experiments show that even in such setups, models do not translate in a local fashion: with varying degrees of correctness, they frequently change their translation when we slightly adapt the input. on the one hand, this well-known volatility (see also @xcite ) might be essential for coping with ambiguities for which meanings are context-dependent. on the other hand, our manual analysis shows that the observed noncompositional behaviour does not reflect the incorporation of necessary contextual information and that oftentimes it is even altering the correctness of the translations. furthermore, this erratic behaviour highlights a lack of default reasoning, which can, in some cases, be problematic or even harmful, especially if faithfulness @xcite or consistency is important. in linguistics, it has been discussed how to extend the syntax and semantics such that 'problem cases' can be a part of a compositional language @xcite . in such formalisations, global information is used to disambiguate the problem cases, while other parts of the language are still treated locally. in our models, global behaviour appears in situations where a local treatment would be perfectly suitable and where there is no clear evidence for ambiguity. we follow @xcite in suggesting that we should learn from strategies employed by humans, who can assign compositional interpretations to expressions but can for some inputs also derive noncompositional meanings. for human-like linguistic generalisation, it is vital to investigate how models can represent both these types of processing, providing a locally compositional treatment when possible and deviating from that when needed.