What determines the order of adjectives in E nglish? Comparing efficiency-based theories using dependency treebanks (2020)

Abstract: We take up the scientific question of what determines the preferred order of adjectives in English, in phrases such as big blue box where multiple adjectives modify a following noun. We implement and test four quantitative theories, all of which are theoretically motivated in terms of efficiency in human language production and comprehension. The four theories we test are subjectivity (Scontras et al., 2017), information locality (Futrell, 2019), integration cost (Dyer, 2017), and information gain, which we introduce. We evaluate theories based on their ability to predict orders of unseen adjectives in hand-parsed and automatically-parsed dependency treebanks. We find that subjectivity, information locality, and information gain are all strong predictors, with some evidence for a two-factor account, where subjectivity and information gain reflect a factor involving semantics, and information locality reflects collocational preferences.

## introduction across languages, there exist strong and stable constraints on the order of adjectives when more than one is used to modify a noun @xcite . for example, in english, big blue box sounds natural and appears relatively frequently in corpora, while blue big box sounds less natural and occurs less frequently @xcite . in this paper, we take up the scientific question of what explains these constraints in natural language. to do so, we implement quantitative models that have been proposed in previous literature as explanations for these constraints, and compare their accuracy in predicting adjective ordering data in parsed corpora of english foot_0 . in the last few years, adjective order has become a crucial testing ground for quantitative theories of syntax. these theories provide mathematical models that can describe the distribution of words in sentences and the way those words combine to yield the meaning of a sentence, in a way that captures the fine-grained quantitative patterns observable in large text datasets @xcite @xcite . quantitative syntactic theories are often efficiency-based, meaning that they model word distributions as the result of a process that tries to maximize information transfer while minimizing some measure of cognitive cost; as a result, they often use the mathematical language of information theory. such theories promise not only to describe distributions of words, but also to explain why they take the shape they do, by viewing human language as an efficient code subject to appropriate constraints. this work informs nlp by providing a theory of language structure that integrates with data-driven, optimization-based machine learning models. adjective order is a fruitful empirical target for quantitative theories of syntax because it is an area where the traditional discrete and symbolic theories become highly complex, and a quantitative approach becomes more attractive. for example, in the formal syntax literature, a standard explanation for adjective order constraints is that each adjective belongs to a certain semantic class (e.g., color or size) and that there exists a universal total order on these semantic classes (e.g., @xmath0, which determines the order of adjectives in any given instance @xcite . such discrete theories of adjective order become complex rapidly as the number of semantic classes to be posited becomes large (upwards of twelve in @xcite and more fine-grained (see @xcite for discussion of the learning problem posed by such classifications). in contrast, quantitative syntax theories typically identify a single construct that grounds out in real-valued numerical scores given to adjectives, which determine their ordering preferences. these scores can be estimated based on large-scale corpus data or based on human ratings. in what follows, we test the predictions of four such theories: the subjectivity hypothesis @xcite @xcite @xcite , the information locality hypothesis @xcite @xcite , the integration cost hypothesis @xcite , and the information gain hypothesis, which we introduce. we begin with a presentation of the details of each theory, then implement the theories and test their predictions against large-scale naturalistic data from english. in addition to comparing the predictors in terms of accuracy, we also perform a number of analyses to determine the important similarities and differences among their predictions. the paper concludes with a discussion of what our results tell us about adjective order and related issues, and a look towards future work. subsequent work has attempted to explain the role of subjectivity in adjective ordering by appealing to the communicative benefit afforded by ordering adjectives with respect to decreasing subjectivity. for example, @xcite use simulated reference games to demonstrate that, given a set of independently-motivated assump-tions concerning the composition of meaning in multi-adjective strings, subjectivity-based orderings lead to a greater probability of successful reference resolution; the authors thus offer an evolutionary explanation for the role of subjectivity in adjective ordering (see also @xcite @xcite . ## related work previous corpus studies of adjective order include @xcite , who examined methods for ordering adjectives in a natural language generation context, and @xcite , who examined effects of phonological length, syntactic category ambiguity, semantic closeness, adjective frequency, and a measure similar to pmi called noun specificity. our work differs from this previous work by focusing on recently-introduced predictors that have theoretical motivations grounded in efficiency and information theory. the theories we test here (except information gain) have been tested in previous corpus studies, but never compared against each other. @xcite validate that subjectivity is a good predictor of adjective order in corpora, and hahn et al. (2018) and futrell et al. (2019) evaluate both information locality and subjectivity. dyer (2018) uses integration cost to model the order of same-side sibling dependents cross-linguistically and across all syntactic categories. ## methods our task is to find predictors of adjective order based solely on data about individual adjectives and nouns. more formally, the goal is to find a scoring function s(a, n ) applying to an adjective a and a noun n , such that the order of two adjectives modifying a noun a 1 a 2 n can be predicted accurately by comparing s(a 1 , n ) and s(a 2 , n ). furthermore, the scoring function s should not include information about relative order in observed sequences of the form a 1 a 2 nthe scoring function should be based only on corpus data about co-occurrences of a and n , or on human ratings about a and/or n . we apply this restriction because our goal is to evaluate scientific theories of why adjectives are ordered the way they are, rather than to achieve maximal raw accuracy. ## results ## conclusion we examined a number of theoretically-motivated predictors of adjective order in dependency treebank corpora of english. we found that the predictors have comparable accuracy, but that it is possible to identify two broad factors: a semantic factor variously captured by subjectivity scores and information gain based on word clusters, and a wordform-based factor captured by pmi. this study provides a framework for evaluating further theories of adjective order, and for evaluating the theories given here against new data from dependency treebanks. generalizing to larger datasets of english is straightforward. more excitingly, we now have the opportunity to bring new languages into the fold. the vast majority of research on adjective ordering, and all the corpus work to our knowledge, has been done on english, where adjectives almost always come before the noun. studying other typologically-distinct languages provides an opportunity to disentangle the theories that we studied here in a way that cannot be done in english. the available behavioral evidence suggests that mirror-image preferences (e.g., "box blue big") may be the norm in post-nominal adjective languages @xcite . information locality, subjectivity, and integration cost make precisely that prediction, though none addresses mixed-type languages in which adjectives can precede or follow nouns. it is an open question how to implement ig for these postor mixed-placement adjectives; one possibility is to measure the information gained when the set of adjectives associated to a noun a n is partitioned by an adjective a. in that case, the predictions about post-nominal order could differ substantially from the predictions about pre-nominal order. our dependency-treebank-based methods can be applied to any other corpus of any language, provided it has enough data in the form of adjective-noun pairs to get reliable estimates of the information-theoretic predictors. such studies will be crucial to achieve a complete computational understanding of natural language syntax.