Corpus evidence for word order freezing in R ussian and G erman (2020)

Abstract: We use Universal Dependencies treebanks to test whether a well-known typological trade-off between word order freedom and richness of morphological marking of core arguments holds within individual languages. Using Russian and German treebank data, we show that the following phenomenon (sometimes dubbed word order freezing) does occur: those sentences where core arguments cannot be distinguished by morphological means (due to case syncretism or other kinds of ambiguity) have more rigid order of subject, verb and object than those where unambiguous morphological marking is present. In ambiguous clauses, word order is more often equal to the one which is default or dominant (most frequent) in the language. While Russian and German differ with respect to how exactly they mark core arguments, the effect of morphological ambiguity is significant in both languages. It is, however, small, suggesting that languages do adapt to the evolutionary pressure on communicative efficiency and avoidance of redundancy, but that the pressure is weak in this particular respect.

## introduction languages are optimized for communicative efficiency, at least in some aspects of their structure @xcite . one manifestation of evolutionary pressures for optimization is the trade-off in core argument marking: languages tend to have either rigid order of subject (s), verb (v) and object (o) or rich morphological marking that shows which argument is the subject and which is the object, rarely both, and never neither @xcite . this tendency, long recognized in qualitative linguistic literature @xcite , has recently received strong quantitative support, both from grammar-based @xcite , corpus-based @xcite @xcite and experimental @xcite studies. the explanation of the trade-off is quite straightforward. for successful communication, it is convenient to have the semantic distinction between s and o overtly coded. however, one means of coding (either syntactic or morphological) is enough, two are redundant, and languages optimize also by eliminating redundancy @xcite . to consider an example: english is a language with a very rigid word order (svo) and almost no morphological marking, while russian is exactly the opposite: there is rich morphological marking (through nominal cases and verbal agreement), and all six word order permutations are grammatically possible. it should be highlighted that both word order freedom and presence of morphological marking are gradual phenomena, not binary. english does have morphological marking for pronominal arguments (both cases and verbal agreement), while russian word order is not entirely free, as it usually conveys pragmatic and/or stylistic information. a corpus-based approach allows us to adequately quantify these phenomena. all quantitative typological studies cited above focused on the correlation across languages. we test whether an equivalent tendency can be observed within a language. in other words, in the previous studies, datapoints were languages, while in ours, datapoints are clauses. this work is licensed under a creative commons attribution 4.0 international license. license details: http:// creativecommons.org/licenses/by/4.0/ . more specifically, we test the following hypothesis: is it true that if a language has relatively free word order, then those clauses where s and o cannot be distinguished by morphological means (due to case syncretism or other kinds of ambiguity) will have more rigid order of s, v and o than those clauses where unambiguous morphological marking is available? this phenomenon has sometimes been dubbed word order freezing in the literature (see section 2). we also make a stronger prediction: in ambiguous clauses, word order will be more often equal to the one which is basic or dominant (most frequent) in the language (for instance, svo for russian). note that the second prediction entails the first, but not vice versa, at least in principle. ## word order freezing one of the earliest explicit wordings of the word order freezing hypothesis belongs to @xcite : "if in a language like russian the nominal subject and object are not distinguished by morphological means, the relative order so is compulsory". since then, word order freezing has been claimed to exist in russian @xcite , korean, bulgarian, papuan languages @xcite , japanese @xcite , german @xcite , hindi and urdu @xcite . there is, however, a dearth of quantitative corpus-based studies. @xcite demonstrate word order freezing for dutch, while @xcite does the same for norwegian. note, however, that both dutch and norwegian have relatively rigid word order (and no case marking on nouns), and virtually the only deviation from the default word order (svo) that is possible is ovs (which results from object fronting). it is thus unclear whether the results of these studies can be generalized to typologically different languages. in addition, @xcite focuses on a somewhat different facet of word order freezing hypothesis ("word order freezes when properties of the arguments are maximally marked in some sense") and does not directly test the role of morphological ambiguity. we are interested in directly testing the "morphological ambiguity freezes the word order" hypothesis on two languages, russian and german, using corpus data from universal dependencies (ud) 2.6 (zeman et al., 2020). the choice of languages is driven by the following considerations: a) languages must have relatively free word order; b) there must be a substantial proportion of nominative-accusative syncretism in the nominal paradigm; c) large treebanks must be available (a pilot study on russian data suggests that the effect is not visible on smaller datasets). given the current state of the ud collection, that basically leaves germanic and slavic languages, of which we choose one language per group. we will call a clause containing s, v and o "unambiguous" if s and o can be distinguished from morphological marking and "ambiguous" if they cannot. we will ignore semantic and pragmatic information. in the vast majority of clauses, s and o can be distinguished using context, background knowledge or common sense. for instance, in a german sentence like die zahlung wickelt die deutsche bank ab 'deutsche bank is processing the payment', it is clear that die zahlung 'payment' is the object and die deutsche bank 'deutsche bank' is the subject, because banks normally process payments, and not vice versa. it is, however, extremely difficult to formalize and quantify these factors (besides, they are orthogonal to our hypothesis about formal marking). likewise, we ignore other factors that might influence word order, including, for instance, information structure, dependency locality @xcite , predictability @xcite , the length (weight) of arguments @xcite etc. unlike @xcite and @xcite , we will not use animacy and definiteness as predictors. first, the availability of this information varies: german treebanks are not annotated for animacy, while russian treebanks are not annotated for definiteness. second, animacy and definiteness strongly overlap with availability of overt morphological marking of syntactic role (in russian, for instance, animate arguments will almost always be marked), which makes it difficult to reliably estimate their contribution. ## materials and methods for russian, we concatenate all available ud treebanks, which yields 73k sentences with 1.3m tokens. for german, we concatenate hdt and gsd (excluding the smaller treebanks, pud and lit, that lack the necessary annotation foot_0 ), which yields 206k sentences with 3.7m tokens. we extract all clauses that contain a verb that has a nominal subject (related via nsubj) and a nominal object (obj). we include both proper and common nouns, but exclude arguments that belong to any other part of speech (pronouns etc.) to avoid adding potential confounds. if there are several coordinated arguments, we include only those that satisfy these criteria. see below for other, language-specific filters. to establish whether a clause is ambiguous we check whether at least one of the arguments and/or the verb mark which argument is the subject and which is the object. for every argument, all available conjuncts (that pass the filter) are checked, and if at least one of them is marked, then the argument is considered marked. we treat morphological ambiguity/markedness as a binary feature, not trying to quantify degree of markedness. to achieve high accuracy, we do it using a set of language-specific handcrafted rules (see below). we manually check the accuracy of the algorithms on subsample of 100 clauses, randomly drawn from all suitable clauses. ## results our final dataset contains 8,575 clauses for russian and 53,373 for german (remember that for german we have main clauses only, see section 3.2). the strategies of morphological marking are clearly different for the two languages. for russian, the distribution is more uniform. the most frequent strategy is marking subject and verb, but not the object. it is followed by the following strategies: marking subject only; marking verb only; not marking anything; marking everything. the remaining strategies are less common, marking object only is the least frequent one. for german, the two most frequent strategies (marking verb only; not marking anything) account for 71% of all cases, followed by marking object only, the rest are less common, marking everything is virtually non-existent (it happens when both arguments are masculine singular and have a modifier, and in addition one and only one of them is coordinated with other nouns and thus behaves as if in plural). most importantly for the current study, the proportion of ambiguous sentences is 13% for russian and 35% for german. the proportions of different word orders in ambiguous and unambiguous clauses are provided in to quantify word order freedom, we calculate shannon entropy (in bits) over the six possible word orders, see to test whether the observed differences for the proportion of the basic word order are significant (i.e. whether our second prediction holds), we apply the same kind of test: p-values are 0.011 for russian and 0 for german. the bootstrap tests are visualized on figure 1 . it can be argued that potential word-order preferences of individual verbs should be controlled for in order to ensure more rigorous testing. to do that, we fit a mixed-effects logistic regression model with word order as the dependent variable. to make the model simpler and more interpretable, we treat word order as a binary variable: so (reference level) or os. the independent variable is whether the clause is unambiguous (reference level), with by-verb random intercept and random slope. in lme4 @xcite notation, the model looks as follows: we performed the calculations in r (r core team, 2020), using the lmertest package to calculate p-values @xcite . the summary of the model is presented in for both languages, the coefficient for ambiguity is negative, meaning that in ambiguous clauses the probability of os word order is lower (and the probability of the basic word order, so, is higher). for russian, the significance of the coefficient is slightly higher than the 0.05 threshold, for german, it is much lower. ## discussion both in russian and german, both our predictions are confirmed: clauses where subject and object are not marked morphologically have lower word-order entropy and higher proportion of basic word order. according to the bootstrap test, the differences are significant, but according to a mixed-effect logistic regression model with by-verb random slope and intercept the effect of morphological ambiguity is significant only for german, but not for russian. interestingly, russian and german exhibit considerable differences with respect to how exactly the ambiguity is resolved (by marking subject, object, verb or several words at once) and to the distribution of word orders. nonetheless, the effect is clearly present in both languages, and in both languages it is small. this implies that the pressure for word-order freezing is relatively weak (and probably mitigated by other factors). pace @xcite , the trade-off is not absolute: morphologically ambiguous clauses where word order is different from the basic one do occur in both languages. we inspected such counterexamples in order to see whether any factors that could explain why these clauses do not have any formal marking of core arguments, but did not discover any patterns. absence of formal marking, however, is not surprising, given that in most cases the clauses can be disambiguated using semantic and pragmatic information, as well as context and background knowledge. further work in this direction might include psycholinguistic experiments (do human beings rely more on word order when interpreting morphologically ambiguous sentences?), diachronic studies (do morphological changes in russian, german, or other relevant languages that increase or decrease the degree of syncretism affect word order) and machine-learning experiments designed to determine to what extent state-of-the-art parsers rely on morphology and word order (using e.g. ablation techniques à la berdicevskis and eckhoff ( 2016 )) the scripts for morphological analysis and statistical tests, as well as detailed results are openly available foot_1 .