Typological Word Order Correlations with Logistic Brownian Motion (2022)

Abstract: In this study we address the question to what extent syntactic word-order traits of different languages have evolved under correlation and whether such dependencies can be found universally across all languages or restricted to specific language families. To do so, we use logistic Brownian Motion under a Bayesian framework to model the trait evolution for 768 languages from 34 language families. We test for trait correlations both in single families and universally over all families. Separate models reveal no universal correlation patterns and Bayes Factor analysis of models over all covered families also strongly indicate lineage specific correlation patters instead of universal dependencies.

## introduction over the long time of their history, humans have come to develop a wide variety of languages. overall, these languages share basic structural similarities. the nature and extent of these similarities have been subject of many theories. as language structure is mostly described in terms of syntax trees, these are often used as explanatory models for structural confines of possibly observable word orders in different languages. the explanations for such confines differ from chomskian innate universal grammar @xcite , to more general aspects of computational ease of for the brain @xcite . on the other hand, increasing data about languages and computational means have prompted examinations of which structural universalities can be found empirically @xcite @xcite . more recent representatives of this start also to account for the historical evolution and relationships of the researched languages @xcite . by treating the evolution of languages analogous to biological evolution, they apply a method from bioinformatics @xcite to model historical transition rates between word-order traits. @xcite the focus lay on four language families: austronesian, bantu, indo-european and uto-aztecan. across those they found widely differing trait correlations, proposing that correlations arise only specific to lineages and not from cognitive factors universally determining language evolution, but instead due to more local cultural evolution. jäger (2018) applied the models on a set of 34 families including models which cover all families at once comparing universal with lineage-specific correlations. this comparison resulted in a group of word-order traits being correlated universally. this suggests that phylogenetic models for individual language families cannot fully capture the universal correlations between different families. in this work, we attempted to test this assumption with a different phylogenetic model. we used brownian motion to describe the evolutionary process of the trait change over time. this type of application is common in a biological phylogenetic context. it is described by, e.g., @xcite . the brownian motion part of the model below is also based on this description. we then applied logistic linkage to model the observed categorical features of word order with a binomial distribution. the paper is structured as follows. section 2 describes the brownian motion models used. section 3 will give a short overview of the data. the experimental setup is described in section 4. finally, we present the results in section 5, followed by the conclusions in section 6. ## brownian motion model a brownian motion model is used in this study. it describes the process of evolutionary change of specified traits through time as a multivariate normal distribution. we denote it here as with means a and variance-covariance matrix v , describing the distribution of the observed traits x. let v be the variance-covariance matrix which is computed of two matrices c and r. c encodes the degree of relation t between species such that languages with a longer shared history in a phylogenetic tree have a stronger covariance. we then define r as the correlation matrix. it is composed of the traits' evolutionary rates σ 2 n and their correlations σ 12 : v is computed by the kronecker product as shown below on the example of a family with two traits and two species: the means a and the correlation r are parameters to be estimated. the traits x and phylogenetic matrix c are observed data as described in the following section. finally, logistic linking is introduced to model the categorical word-order traits x as a binomial distribution with probabilities p. their logits, logistic(p) are modelled as brownian motion: ## word-order traits & language families the data used in this paper are of two kinds. the first are language features that are to be modeled. the second are phylogenetic data used as basis for the evolutionary model. combined, these provide data for 768 languages from 34 language families as described in more detail by wichman et al. @xcite . all data were kindly provided by gerhard @xcite . we provide the data as part of this work foot_0 . we considered the same eight word-order traits ( the phylogenetic data represent the languages' historical relations in the form of binary trees as for example in figure 1 . the trees' edge lengths represent the time since the last common ancestor split up. they have been estimated by @xcite and range from 10 to 1000 samples per family. ## experiments in our experiments, we compared variations of this model with different base assumptions: in addition, we used the information criteria waic @xcite & looic @xcite . these were added to test for consistency across different metrics. they are both based on pointwise log-likelihoods and can become unreliable, when dealing with strongly dependent data points @xcite . both express model comparisons primarily in differences where the value of the difference is not directly amenable to assess how strongly one model is favoured over another. to address this, we utilized the rethinking package by @xcite . this offers a convenient function to compare models via waic including the assignment of weights ranging from 0 to 1 to give an accessible overview over the relative quality of the models. the models were implemented in the stan modeling language (2020) which uses the nuts sampler @xcite for parameter estimation. the code is available here 2 . based on the suggestion of @xcite the models are 2 https://github.com/hartunka/ typologicalwordordercorrelations run for 21,000 iterations after warm-up, to achieve reliable bayes factors. distributed on 14 chains, this becomes 1.500 iterations per chain, in addition to 1,500 warm-up iterations each. the sampling was done with the default control parameters for stan's sampler. with these parameters the lineagespecific models of the nd-nnum trait pair and the independent models of the ng-vo & an-vo trait pairs did return a too small effective sample size for the bridgesampling to be effective. so these models were rerun with max_treedepth raised from 10 to 15. ## results for the single-family models, the comparisons did not yield bayes factors above 5 for 15 of the 34 families. therefore, no conclusions about preferred models are possible in those cases. those trait correlations that did come up, only appeared in a minority of the covered families, not providing any pattern for universal correlations. figure 2 shows for each trait pair in how many families it was estimated to be correlated. the information criteria at face value find up to 22 families for which the same trait-pair is correlated. we filtered cases close to equivalence with waic weights to test how many cases have a robust preference. we chose the cutoff at 0.6. filtering such cases results in less cross-family similarities, with no more than 3 families sharing the same trait pair as correlated. cross-metric agreement: the percentage of model comparisons where bayes factors & waic weights favoured the same model or both don't favour any model at all is 85.5% with a looic & waic agreement of 99.8%. the model comparison between universal and lineage specific models strongly favour lineage specificity, with bayes factors favouring lineagespecific correlation for each trait pair (figure 3 ). in contrast to single family models, bayes factors and information criteria strongly disagree. both waic & looic strongly favour universal correlation models for each trait pair. waic weights are fully assigned to the universal models. comparing lineage specificity with universal independence shows similar results. bayes factors strongly favour lineage-specificity but information criteria favour independence. universal trait pair correlations, based on the dependent-independent model comparisons center around the adposition-noun order (figure 4 ). ## conclusions we applied logistic brownian motion under a bayesian framework to model the trait evolution for 768 languages from 34 language families. the models for single language families and those across all families indicate no universal word-order trait correlations across language families when compared using bayes factors. this result matches with the observations made by @xcite , but contradicts the results over all families presented by @xcite . thus, representing all families in one model did not add much information to single family models. it is noteworthy, that the results from the information criteria contradict those from the bayes factors regarding the universal models. although this could be attributed to some unreliability of the information criteria given the data and the nature of the model, it is still worth further investigation.