On the Evolution of Word Order (2021)

Abstract: Most natural languages have a predominant or fixed word order. For example in English the word order is usually Subject-Verb-Object. This work attempts to explain this phenomenon as well as other typological findings regarding word order from a functional perspective. In particular, we examine whether fixed word order provides a functional advantage, explaining why these languages are prevalent. To this end, we consider an evolutionary model of language and demonstrate, both theoretically and using genetic algorithms, that a language with a fixed word order is optimal. We also show that adding information to the sentence, such as case markers and noun-verb distinction, reduces the need for fixed word order, in accordance with the typological findings.

## introduction word order is a linguistic concept that refers to how syntactic elements are arranged in a sentence in different languages. specifically, it deals with the relative position of the three prominent syntactic roles in a sentence: subject (s), verb (v), and object (o). the three syntactic roles can be ordered in six different arrangements: svo, sov, vso, vos, ovs, osv. apriori, there is no reason to assume that one order is superior to others. however, typological findings show that most natural languages have one preferred or fixed word order, with the vast majority of them belonging to the first two families: svo or sov @xcite . interestingly, languages that have developed spontaneously and did not contact other languages also have a distinct preference for particular word order, such as the al-sayyid bedouin sign language @xcite . another typological finding is related to case markers, which are morphological indicators (usually prefixes or suffixes) that indicate the syntactic role of a word in a sentence. for example, the word for the boy in arabic is al-walad-u if it is the subject and al-walad-a if it is the object. apparently, in languages with a rich system of case markers, such as arabic, latin and sanskrit, the word order is more flexible @xcite . these findings raise several research questions. first, why is there a preferred or fixed order in the majority of natural languages? second, why do most languages have svo or sov as their preferred order? third, how can we explain the relationship between case markers and flexible word order? this work addresses the aforementioned questions within an evolutionary perspective @xcite . as in the evolution theory of the animal world, if a particular feature appears in many different species, it is assumed to give a functional advantage to the organisms carrying it, thus helping them survive natural selection. similarly, linguistic phenomena that are common to all or most languages are assumed to provide a functional advantage and facilitate better communication. within this perspective, a plethora of works suggested to use genetic algorithms (ga), an optimization method inspired by evolutionary principles, in order to investigate typological findings @xcite @xcite . in this work, we follow this framework and use ga to study the phenomenon of word order, yet not investigated to the best of our knowledge. in particular, we attempt to answer whether a fixed word order gives a functional advantage, which may explain its emergence in most natural languages. we also provide a theoretical analysis to prove that the optimal grammar, with respect to our functional assumptions, is a grammar with fixed word order. although it would be interesting to explore why a specific word order is preferred, for example svo or sov, this work addresses the fundamental question about why languages have fixed word orders at all, rather than specific ones. ## method following @xcite , we view language as a form of communication designed to transmit a message between a speaker and a hearer. thus, if we are to speak of an "optimal" language, it is the one that facilitates precise delivery of a message between the speaker and the listener. in this view, the hearer observes only the sentence received from the speaker, which does not contain information regarding the relationship between the words and their syntactic roles. intuitively, word order might impact the way the hearer understands the sentence. for example, compare the english sentence "we ship a man" with "we man a ship". in this case, as the words "ship" and "man" may function as both a noun and a verb, and due to the absence of case markers, the meaning is determined solely by the order of words in the sentence. under these assumptions, we use ga to simulate a language's evolution, starting with no preferred word order, and test whether communication constraints enforce a fixation of particular word order. ## theoretical analysis as mentioned, optimal communication occurs when the distance between the input and output is minimal. in this section, we prove that under this definition of optimality, and assuming the hearer always identifies the words in the lexicon correctly despite the noise (as indeed happens with a high probability), the following holds theorem 1. any optimal grammar has exactly one fixed word order. proof. let x 1 . . . x n and y 1 . . . y n be random variables of the occurrence of possible word orders, associated with the speaker and the hearer, respectively. we define let d ij be the distance between x i and y j , defined as the number of syntactic roles with different positions between x i and y j . then, the expected distance is given by since the hearer has no additional information other than the observed sentence, he picks the most probable order from his grammar, independently of the speaker. accordingly, p (x i , y j ) = p i p j , and therefore the expected value of the distance is given by note that p i d ij p @xmath0, ∀i, j, since p i , p j are probabilities, and d ij is a distance. since d @xmath1, to get the minimal expected distance 0 it should hold ∀@xmath2. therefore, p i can be greater than zero in exactly one entry. for other words, the only probability distributions that give an expected distance of zero have probability 1 for a particular word order x i and zero for the rest. accordingly, the optimal grammar must have fixed word order. ## experiments we tested the model in four different scenarios. ## results for every scenario, we present the average distance between the input and the output among all grammars in each generation (figure 1 ). it can be observed that the base scenario starts with the highest distance among the scenarios, as the sentences carry no information regarding the syntactic roles. a similar discussion could be held for the n-v scenario, in which the hearer can identify the verb in the sentence but cannot distinguish between the subject and object. in both scenarios, we observe a drop in the average distance, meaning that at some point, the hearers successfully parsed the sentences with only a small error. in the two scenarios with the case markers, we see that the distance is very close to zero even in the first generation, meaning that the hearers parsed the sentences correctly from the first generation. to examine if the grammars converged to fixed word order, we present the grammars' average entropy in every generation (figure 2 ). the entropy is defined by @xmath3. it can be observed that the average entropy of grammars begins at 1 (absolute randomness) in all scenarios, as the probability for every word order is equal. in the base scenario, we observe a fast decline in entropy until it converges to zero, i.e., a fixed word order. a decline is also observed in the n-v scenario. however, it is slower and does not converge to zero even after 1000 generations. as for the scenarios with the case markers, in both cases, we observe that the entropy remains high even after 1000 generations, meaning that no particular word order was fixed. we also present the grammar that had the smallest distance after 1000 generations (figure 3 ). in the base and n-v scenarios, we observe that the grammar converged to an (almost) fixed word order, vos in the base scenario, and vso in the n-v scenario. in contrast, in both scenarios with case markers, the word order remained flexible even after 1000 generations. ## discussion in the base scenario, we observed a fast fixation of single word order. therefore, in line with the theoretical result, we conclude that given no additional information regarding the sentence's syntactic relations, fixed word order is necessary to allow successful communication. we also observed a fixation in the n-v scenario, meaning that a distinction between nouns and verbs does not provide sufficient information for communication, and thus fixed word order is also necessary in this case. in contrast, both scenarios with case markers did not converge to preferred word order, meaning that the case markers by themselves are enough to allow communication. this finding is consistent with the result of artificial language learning experiments conducted on human learners @xcite . note that since the case markers we used were just one letter, they could have been flipped by the random noise, leading to a wrong parsing by the hearer. however, the chances that two case markers in the same sentence will be flipped are small, and thus the hearer can parse the sentence correctly with high probability. we note that additional experiments performed with a more significant number of syntactic roles yielded similar trends. hence we conclude that these findings do not depend on the number of syntactic roles. the reason why the base scenario converged to vos and the n-v scenario converged to vso is arbitrary, due to random drift, since there is no functional advantage to a particular word order over the others. ## conclusions this work presents a simplistic functional model of language that demonstrates how a fixed word order in languages may result from a lack of information about syntactic relations. empirically, we used genetic algorithms (ga) in order to investigate whether additional information would render the fixed word order unnecessary. we showed that a distinction between nouns and verbs provides additional information to the hearer, but it is not sufficient. in contrast, case markers provide enough information to enable a flexible word order. these results are consistent with the typological findings. regarding the reason why most natural languages tend exhibit svo or sov orderings, the model used in this work does not seem to provide an explanation. it has been demonstrated that even where a single word order has been fixed, we have received various possible orders that are not necessarily one of these two. this is because we did not consider the functional advantages of certain orders over others, so the final order is entirely determined by random chance. additional psycho-linguistic considerations are needed to explain the typological finding, as done in other studies @xcite .