Probing Linguistic Systematicity (2020)

Abstract: Recently, there has been much interest in the question of whether deep natural language understanding (NLU) models exhibit systematicity, generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear. There is accumulating evidence that neural models do not learn systematically. We examine the notion of systematicity from a linguistic perspective, defining a set of probing tasks and a set of metrics to measure systematic behaviour. We also identify ways in which network architectures can generalize non-systematically, and discuss why such forms of generalization may be unsatisfying. As a case study, we perform a series of experiments in the setting of natural language inference (NLI). We provide evidence that current state-of-the-art NLU systems do not generalize systematically, despite overall high performance.

## introduction language allows us to express and comprehend a vast variety of novel thoughts and ideas. this creativity is made possible by compositionalitythe linguistic system builds utterances by combining an inventory of primitive units such as morphemes, words, or idioms (the lexicon), using a small set of structure-building operations (the grammar; @xcite @xcite @xcite @xcite . one property of compositional systems, widely studied in the cognitive sciences, is the phenomenon of systematicity. systematicity refers to the fact that lexical units such as words make consistent contributions to the meaning of the sentences in which they appear. @xcite provided a famous example: if a competent speaker of english knows the meaning of the sentence john loves the girl, they also know the meaning of the girl loves john. this is because for speakers of english knowing the meaning of the first sentence implies knowing the meaning of the individual words the, loves, girl, and john as well as grammatical principles such as how transitive verbs take their arguments. but knowing these words and principles of grammar implies knowing how to compose the meaning of the second sentence. deep learning systems now regularly exhibit very high performance on a large variety of natural language tasks, including machine translation @xcite , question answering @xcite , visual question answering @xcite , and natural language inference @xcite . recently, however, researchers have asked whether such systems generalize systematically (see §4). systematicity is the property whereby words have consistent contributions to composed meaning; the alternative is the situation where words have a high degree of contextually conditioned meaning variation. in such cases, generalization may be based on local heuristics @xcite , variegated similarity @xcite , or local approximations @xcite , where the contribution of individual units to the meaning of the sentence can vary greatly across sentences, interacting with other units in highly inconsistent and complex ways. this paper introduces several novel probes for testing systematic generalization. we employ an artificial language to have control over systematicity and contextual meaning variation. applying our probes to this language in an nli setting reveals that some deep learning systems which achieve very high accuracy on standard holdout evaluations do so in ways which are non-systematic: the networks do not consistently capture the basic notion that certain classes of words have meanings which are consistent across the contexts in which they appear. the rest of the paper is organized as follows. §2 discusses degrees of systematicity and contextually conditioned variation; §3 introduces the distinction between open-and closed-class words, which we use in our probes. §5 introduces the nli task and describes the artificial language we use; §6 discusses the models that we tested and the details of our training setup; §7 introduces our probes of systematicity and results are presented in §8. foot_0 ## systematicity and contextual conditioning compositionality is often stated as the principle that the meaning of an utterance is determined by the meanings of its parts and the way those parts are combined (see, e.g., @xcite . systematicity, the property that words mean the same thing in different contexts, is closely related to compositionality; nevertheless, compositional systems can vary in their degree of systematicity. at one end of the spectrum are systems in which primitive units contribute exactly one identical meaning across all contexts. this high degree of systematicity is approached by artificial formal systems including programming languages and logics, though even these systems don't fully achieve this ideal @xcite . the opposite of systematicity is the phenomenon of contextually conditioned variation in meaning where the contribution of individual words varies according to the sentential contexts in which they appear. natural languages exhibit such context dependence in phenomena like homophony, polysemy, multi-word idioms, and co-compositionality. nevertheless, there are many words in natural language-especially closed-class words like quantifiers (see below)-which exhibit very little variability in meaning across sentences. at the other end of the spectrum from programming languages and logics are systems where many or most meanings are highly context dependent. the logical extreme-a system where each word has a different and unrelated meaning every time it occurs-is clearly of limited usefulness since it would make generalization impossible. nevertheless, learners with sufficient memory capacity and flexibility of representation, such as deep learning models, can learn systems with very high degrees of contextual conditioning-in particular, higher than human language learners. an important goal for building systems that learn and generalize like people is to engineer systems with inductive biases for the right degree of systematicity. in §8, we give evidence that some neural systems are likely too biased toward allowing contextually conditioned meaning variability for words, such as quantifiers, which do not vary greatly in natural language. ## compositional structure in natural language natural language distinguishes between content or open-class lexical units and function or closedclass lexical units. the former refers to categories, such a nouns and verbs, which carry the majority of contentful meaning in a sentence and which permit new coinages. closed-class units, by contrast, carry most of the grammatical structure of the sentence and consist of things like inflectional morphemes (like pluralizing -s in english) and words like determiners, quantifiers, and negation (e.g., all, some, the in english). these are mostly fixed; adult speakers do not coin new quantifiers, for example, the way that they coin new nouns. leveraging this distinction gives rise to the possibility of constructing probes based on jabberwockytype sentences. this term references the poem jabberwocky by lewis carroll, which combines nonsense open-class words with familiar closedclass words in a way that allows speakers to recognize the expression as well formed. for example, english speakers identify a contradiction in the sentence all jabberwocks flug, but some jabberwocks don't flug, without a meaning for jabberwock and flug. this is possible because we expect the words all, some, but, and don't to contribute the same meaning as they do when combined with familiar words, like all pigs sleep, but some pigs don't sleep. using jabberwocky-type sentences, we tested the generalizability of certain closed-class word representations learned by neural networks. giving the networks many examples of each construction with a large variety of different content wordsthat is, large amounts of highly varied evidence about the meaning of the closed-class words-we asked during the test phase how fragile this knowledge is when transferred to new open-class words. that is, our probes combine novel open-class words with familiar closed-class words, to test whether the closed-class words are treated systematically by the network. for example, we might train the networks to identify contradictions in pairs like all pigs sleep; some pigs don't sleep, and test whether the network can identify the contradiction in a pair like all jabberwocks flug; some jabberwocks don't flug. a systematic learner would reliably identify the contradiction, whereas a non-systematic learner may allow the closed-class words (all, some, don't) to take on contextually conditioned meanings that depend on the novel context words. ## related work there has been much interest in the problem of systematic generalization in recent years @xcite @xcite @xcite @xcite @xcite . in contrast to our approach (testing novel words in familiar combinations), many of these studies probe systematicity by testing familiar words in novel combinations. @xcite adopt this approach in semantic parsing with an artificial language known as scan. @xcite introduce a naturalistic nli dataset, with test items that shuffle the argument structure of natural language utterances. in the in the inductive logic programming domain, @xcite introduced the cluttr relational-reasoning benchmark. the novel-combinations-of-familiar-words approach was formalized in the cfq dataset and associated distribution metric of @xcite . @xcite introduced a semantic-rolelabeling and negation-scope labeling dataset, which tests compositional generalization with novel combinations of familiar words and makes use of syntactic constructions like relative clauses. finally, @xcite explore pre-training schemes' abilities to learn prepositions and wh-words with syntactic transformations (two kinds of closedclass words which our work does not address). a different type of systematicity analysis directly investigates learned representations, rather than developing probes of model behavior. this is done either through visualization @xcite , training a second network to approximate learned representations using a symbolic structure @xcite or as a diagnostic classifier @xcite , or reconstructing the semantic space through similarity measurements over representations @xcite . ## simulations 6.1 models we analyze performance on four simple baseline models known to perform well on standard nli tasks, such as the stanford natural language inference datasets, @xcite . following @xcite , the hypothesis u and premise v are individually encoded by neural sequence encoders such as a long short-term memory (lstm; @xcite or gated recurrent unit (gru; @xcite . these vectors, together with their element-wise product u * v and element-wise difference u -v are fed into a fully connected multilayer perceptron layer to predict the relation. the encodings u and v are produced from an input sentence of m words, w 1 , . . . , w m , using a recurrent neural network, which produces a set of a set of m hidden representations h 1 , . . . , h t , where h @xmath0, . . . , w m ). the sequence encoding is represented by its last hidden vector h t . the simplest of four models sets f to be a bidirectional gated recurrent unit (bgru). this model concatenates the last hidden state of a gru run forwards over the sequence and the last hidden state of gru run backwards over the sequence, for example, @xmath1, -→ h m ]. our second embedding system is the infersent model reported by @xcite , a bidirectional lstm with max pooling (infs). this is a model where f is an lstm. each word is represented by the concatenation of a forward and backward representation: h @xmath2, -→ h t ]. we constructed a fixed vector representation of the sequence h t by selecting the maximum value over each dimension of the hidden units of the words in the sentence. our third model is a self-attentive sentence encoder (satt) which uses an attention mechanism over the hidden states of a bilstm to generate the sentence representation @xcite . this attention mechanism is a weighted linear combination of the word representations, denoted by @xmath3, where the weights are calculated as follows: finally, we test the hierarchical convolution-alnetwork (conv) architecture from @xcite which is itself inspired from the model adasent @xcite . this model has four convolution layers; at each layer the intermediate representation u i is computed by a max-pooling operation over feature maps. the final representation is a concatenation @xmath4, ..., u l ] where l is the number of layers. ## probing systematicity in this section, we study the systematicity of the models described in §6.1. recall that systematicity refers to the degree to which words have consistent meaning across different contexts, and is contrasted with contextually conditioned variation in meaning. we describe three novel probes of systematicity which we call the known word perturbation probe, the identical open-class words probe, and the consistency probe. all probes take advantage of the distinction between closed-class and open-class words reflected in the design of our artificial language, and are performed on sentence pairs with novel open-class words (jabberwocky-type sentences; see §5.5 ). we now describe the logic of each probe. ## results in this section, we report the results of two control analyses, and that of our three systematicity probes described above. ## discussion and conclusion systematicity refers to the property of natural language representations whereby words (and other units or grammatical operations) have consistent meanings across different contexts. our probes test whether deep learning systems learn to represent linguistic units systematically in the natural language inference task. our results indicate that despite their high overall performance, these models tend to generalize in ways that allow the meanings of individual words to vary in different contexts, even in an artificial language where a totally systematic solution is available. this suggests the networks lack a sufficient inductive bias to learn systematic representations of words like quantifiers, which even in natural language exhibit very little meaning variation. our analyses contain two ideas that may be useful for future studies of systematicity. first, two of our probes (known word perturbation and consistency) are based on the idea of starting from a test item that is classified correctly, and applying a transformation that should result in a classifiable item (for a model that represents word meaning systematically). second, our analyses made critical use of differential sensitivity (i.e., variance) of the models across test blocks with different novel words but otherwise identical information content. we believe these are a novel ideas that can be employed in future studies.