Revisiting Dependency Length and Intervener Complexity Minimisation on a Parallel Corpus in 35 Languages (2023)

Abstract: In this replication study of previous research into dependency length minimisation (DLM), we pilot a new parallel multilingual parsed corpus to examine whether previous findings are upheld when controlling for variation in domain and sentence content between languages. We follow the approach of previous research in comparing the dependency lengths of observed sentences in a multilingual corpus to a variety of baselines: permutations of the sentences, either random or according to some fixed schema. We go on to compare DLM with intervener complexity measure (ICM), an alternative measure of syntactic complexity. Our findings uphold both dependency length and intervener complexity minimisation in all languages under investigation. We also find a markedly lesser extent of dependency length minimisation in verb-final languages, and the same for intervener complexity measure. We conclude that dependency length and intervener complexity minimisation as universals are upheld when controlling for domain and content variation, but that further research is needed into the asymmetry between verb-final and other languages in this regard.

## introduction efficiency in language production and processing is widely held as a universal, underpinning various aspects of human language evolution and use. @xcite . within syntax, an expression of this is found in the theory of dependency locality @xcite : the principle that syntactically related information should appear in close proximity in a sentence, so as to minimise the memory load required to parse it. its observable effect is dependency length minimisation (dlm): the ordering of a sentence such that the sum distance of dependencies in sentences is minimised @xcite . this effect has been well studied crosslingually @xcite , and is widely held to be a universal of syntax, with the study of @xcite finding evidence of the effect in all languages in a sample of 37 languages in universal dependencies @xcite , among other such cross-lingual studies. 1 there remain, however, inconsistencies and asymmetries in how and where dlm is applied, within languages and within sentence structures. for example, a common finding is that dlm is less pronounced or even absent in head-final languages when controlling for various factors such as sentence type, and when looking only at lexical tokens. @xcite find a negative association between head-finality and dependency length when controlling for harmony and considering only lexical dependencies, an effect that they find to be robust against multiple random baselines. @xcite also finds mixed evidence for the correlation between dependency length and ordering choices for pre-verbal arguments in head-final languages; whereas argument ordering choice is more clearly associated with dependency length in languages with post-verbal arguments. these findings point to a more nuanced picture of dlm, where the effect is asymmetric in terms of word order, more clearly pronounced in head-initial languages @xcite . another question is the extent to which dlm exists as an independent effect, as opposed to being a function of other constraints. @xcite propose the alternative measure of sentence complexity, intervener complexity measure (icm), which measures not the number of tokens between dependants and their heads, but the number of syntactic heads between them, suggesting optimisation for icm underlies the observed dlm effect. as a first step in investigating these questions, our replication study revisits the work of @xcite to broadly replicate this study on a new corpus, with some additions in light of subsequent research. we seek to reevaluate the following questions: the use of parallel corpora is beneficial in making language data more comparable between languages, controlling for domain differences and the natural variation of communicative intent in sentences @xcite . however, most currently available parallel corpora suffer either from limited size and language coverage (e.g. parallel universal dependencies), or from being drawn from highly specific lects that do not reflect common language use (e.g. parallel bible corpora, un declaration of human rights). a related problem in parallel corpora is the phenomenon of translationese @xcite : the effect whereby translated texts are identifiable by certain characteristics that are atypical in the target language, caused by language-specific or universal effects of the translation process @xcite . fictional and non-fictional prose are not immune to the effects of translationese @xcite . nevertheless, since the goal of translated prose is entertainment rather than exactitude, we expect that translators will use stylistic translations that may be closer to the conventions of the target language, thus mitigating this concern. 3 the books that we use are large, containing thousands of sentences. and, though we do not escape the bias of translations mostly being available in a small set of languages, we nevertheless manage a decent coverage of 35 languages, with at least 20,000 sentences in each. our use of this corpus addresses a potentially confounding issue in @xcite and other corpus-based studies: the variation in domain coverage across the ud corpora. with a parallel corpus, we once again put these findings to the test. ## background word ordering with respect to phrase heaviness has long been a topic of interest in constituencybased syntax @xcite , and has been adapted to a dependency grammar framework as dependency length @xcite . since the inception of universal dependencies @xcite and other consistently annotated multilingual corpora, more multilingual studies of dlm have been carried out. @xcite compare the sum dependency lengths of observed sentences in 37 languages in universal dependencies to random baselines of sentences permuted to random orders. they find that in all 37 languages, dependency length as a function of sentence length shows a consistently slower increase than would be expected in random word order baselines, whether free or fixed. @xcite extend this study to probe the impacts of canonical word order constraints versus variability on dlm. building on the setup of @xcite , they use randomly permuted baselines with same valency (i.e. all heads in the permuted sentence must have the same number of dependants on each side) and same side (i.e. dependants must be on the same side of their head) constraints, and find that each baseline shows a reduction in dependency length, and that atypical orderings in a language usually contribute to this. in several studies, @xcite @xcite probes the dlm effect with regard to ordering flexibility and pre-and postverbal argument domains. among her findings is that while dependency length minimisation is well-correlated with phrase ordering choices in postverbal languages (e.g. english, bulgarian, dutch), this effect is much weaker or nonexistent in preverbal languages (e.g. japanese, persian), suggesting that the relevance of dlm depends greatly on word ordering constraints, among other pressures. intervener complexity measure is introduced by @xcite . they operationalise the complexity of intervening information in long dependencies as intervener complexity measure, which counts the number of syntactic heads between a dependant and its head. by comparing random permutations of trees alternately matched for dependency length or intervener complexity, they find that random linear arrangements matched for dependency length tend to have very close icm to the original sentence, but that the inverse effect is not as strong. @xcite perform their experiments using several languages in surface universal dependencies @xcite , accounting for language as a random effect, we are unaware of any multilingual study so far that has directly measured the extent of intervener complexity minimisation per language. most prior large cross-lingual studies of dependency length minimisation have used universal dependencies or surface universal dependencies corpora, or other dependency corpora pre-dating ud @xcite , without control for domain and sentence variation. however, there are some that have used parallel corpora. for example, @xcite compare effects of sentence length and dependency direction in a parallel english-chinese corpus; and ferrer-i cancho (2017) use the parallel universal dependencies (pud) corpora. we are unaware of any previous work with parallel corpora of the same size as ciep+. ## method in our investigation, we broadly replicate the experimental setup of @xcite . the dependency length of a token in a sentence is defined as the number of tokens between it and its head in the linear surface order, including itself (i.e. a minimum of 1). the dependency length of a sentence is then the sum of dependency lengths for each token, excluding the root. we compare the dependency lengths of observed sentences to a set of random baselines: reorderings of the sentences in the corpora with the same underlying tree structure but a different linear surface order of tokens. these baselines are: after creating permutations of each sentence in each book in each language, we use a linear mixedeffects model to estimate the rate at which dependency lengths increase as a function of sentence length. the response variable of the model is sentence dependency length, while the fixed variables are the interaction between sentence length (in number of tokens) and permutation mode: the baseline that produced the sentence (including the unaltered original sentence). we use sentence id as a random effect in the model. sentence id is shared across all permutations of a sentence, and including it accounts for the effect of the variance in sentence structure. this random effect is simplified compared to futrell and gibson's, which groups permutations by sentence id. we found that doing this caused singular fits in the model. performing this separately for each language in the corpus, we use the coefficient of the model fit as the measure of a language's rate of dependency length increase. the higher the coefficient, the greater the dependency lengths we can expect to see as sentence length increases. the model gives us a separate fit for each of the baselines, and so we are able to compare the true rate of increase to what we could expect to see in each of the baseline conditions. if the true rate of increase is not lower from the random baseline, for example, then we do not see dlm in the language. we use the same approach to measure intervener complexity minimisation. the intervener complexity of a token is defined by @xcite as the number of syntactic heads that come between it and its own head; including the token's head itself, meaning that for each token the minimum intervener complexity is one. the intervener complexity measure of a sentence is then the sum of tokenwise intervener complexities in the sentence. fig. 1 shows an example of intervener complexity measure for a sentence in contrast to dependency length. man wearing a red hat with glasses waves 7 1 2 1 3 1 6 ø 4 1 1 1 1 1 3 ø figure 1 : a demonstration of the difference between dependency length and intervener complexity. the top layer of numbers is dependency length; the bottom layer is intervener complexity. for example. there are seven tokens between waves and its syntactic head man, but only three heads between them (wearing, hat and glasses). the icm of this sentence is 12, compared to 21 for dependency length. ## discussion overall, our results seem to uphold dlm as a universal, though with the ame asymmetry between verb-final and verb-initial or -medial languages. we also find this same asymmetry in intervener complexity, with the same languages showing a faster rate of increase in icm, showing that the antilocality effect extends to this measure as well. the next step is to turn our attention to explanations of this tendency for reduced dlm in sov/verb-final languages. there is already work underway to explain these tendencies @xcite . the use of a parallel corpus has supported the results of previous research in this area. in other words, we do not see a very different picture when using a parallel corpus. an interpretation of this that dependency length and intervener complexity minimisation effects are strong enough that they show through the noise of domain and sentence variation. however, we still maintain that parallel corpora should be used wherever possible in such studies. our study has applied to languages as a whole, using the full range of sentences each language in the corpus. on the other hand, we hypothesise that the more focused the linguistic structures under investigation -for example, verb phrases with single object and oblique arguments @xcite , or verb phrases with two oblique arguments @xcite -the more the noise of differing domains and sentence content will affect the results. it is particularly these kinds of studies that we believe will benefit from large parallel corpora. a meta-study of dependency length and related experiments using both universal dependencies and parallel corpora would be useful to measure the extent to which such noise affects different kinds of experiments. we leave this for future research. there are also some improvements that could be made to this study in particular. we would like to find an algorithm for finding the linear ordering that truly optimises intervener complexity measure, so that we can properly assess how close observed orderings are to this baseline. we are unaware of such an algorithm as of yet, and gildea and temperley's algorithm is an imperfect stand-in. this would be particularly valuable because of the tentative evidence we find for observed word order reflecting optimised icm. some previous studies have used surface uni-versal dependencies (sud) annotated corpora @xcite instead of universal dependencies. while we do not expect vastly different results, there is some contention that sud is more appropriate for modelling syntactic difficulty and cognitive demand @xcite , and it would be beneficial to compare experiments on corpora using each of the two formalisms. finally, as more languages are added to ciep+, we hope to be able to expand our analyses to more languages, particularly non-indo-european languages. ## conclusion our replication of a keystone study on dependency length minimisation as a language universal on a much larger, parallel parsed corpus has corroborated previous findings that show evidence of systematic dependency length minimisation in a variety of the world's languages, controlling for the effect of sentence and domain variation. we find a similar effect for intervener complexity measure. we make available our code for permuting parsed corpora according to different permutation baselines, and for analysing them in terms of dependency length, intervener complexity and other properties. 7 we plan to use this corpus in further replications and original studies on syntactic complexity and word order constraints. among our topics of interest are research into why dependency length minimisation is less of a pressure in verb-final languages; and the extent to which other constraints such as information locality @xcite and memory-surprisal tradeoff @xcite subsume dependency length as an explanatory factor for word order.