Analysis of Corpus-based Word-Order Typological Methods (2023)

Abstract: This article presents a comparative analysis of four different syntactic typological approaches applied to 20 different languages. We compared three specific quantitative methods, using parallel CoNLL-U corpora, to the classification obtained via syntactic features provided by a typological database (lang2vec). First, we analyzed the Marsagram linear approach which consists of extracting the frequency word-order patterns regarding the position of components inside syntactic nodes. The second approach considers the relative position of heads and dependents, and the third is based simply on the relative position of verbs and objects. From the results, it was possible to observe that each method provides different language clusters which can be compared to the classic genealogical classification (the lang2vec and the head and dependent methods being the closest). As different word-order phenomena are considered in these specific typological strategies, each one provides a different angle of analysis to be applied according to the precise needs of the researchers.

## introduction typology is usually described as language classification regarding structural types. its scope can be defined as the quest for answers about how languages differ from each other, and about the explanation for the attested differences and similarities. in terms of syntactic typology, one possible linguistic aspect that is analyzed concerns word-order patterns. these phenomena are commonly used to define sets of typological universals in terms of implications, correlations, and universals. most studies in this field rely on the identification of the most frequent word-order phenomena in different languages. although based on attested syntactic constructions, what is extracted from the available linguistic data concerns only the most common syntactic structures. thus, possible word-order patterns which are not the standard ones are usually ignored in these analyses. it is the case of the syntactic information provided by standard typological databases such as wals @xcite . although limited, these databases provide valuable information for theoretical typological analyses and can be used to improve the effectiveness of natural language processing (nlp) tools, as shown by @xcite . on the other hand, corpus-based typological studies can provide a more precise description in terms of possible syntactic phenomena, thus, allowing languages to be compared in a more detailed way, as presented by @xcite . quantitative methods can be used in the analysis of numerous linguistic phenomena, and, even though they can present some bias regarding the corpora selection and annotation, they provide new insights that can challenge and/or complement classic theoretical approaches. the aim of this article is to propose three different corpus-based quantitative methods concerning word-order typology and compare the obtained language classifications to the one provided by the comparison of the syntactic features provided from a typological database. the objective is to show that different approaches provide valuable but diverse contributions in terms of word-order structures attested in annotated corpora. the paper is composed as follows: section 2 presents an overview of the related work to this topic. section 3 describes the campaign design: the language and data-set selection and the syntactic typological approaches; section 4 present the obtained results which are discussed in section 5. in section 6 we provide conclusions and possible future directions for the research. ## related work according to @xcite , the wals database is one of the most used typological resources in nlp studies as it contains phonolog-ical, morphosyntactic, and lexical information for a large number of languages. besides that, the uriel typological compendium is a metarepository composed of several databases (wals included) and is the base of the lang2vec tool @xcite . this specific resource provides typological information about languages in the format of feature and value pairs. thus languages can be represented by vectors which are composed of the selected linguistic information required by the user (e.g.: genealogical, phonological, syntactic, etc). one problem usually observed in these databases is the fact that they suffer from discrepancies that are caused by their variety of sources. therefore, comparisons can only be made if the selected languages have values for the ensemble of chosen features. furthermore, there are many gaps as not all languages have the same amount of descriptive literature. moreover, as previously mentioned, most databases fail to illustrate the variations that can occur within a single language (i.e.: only the most frequent phenomena are reported, and not all possible ones). on the other hand, quantitative methods, such as the ones proposed in this article, provide precise information regarding the frequency of all attested word-order phenomena inside the analyzed corpora. an extended survey of corpora-based typological studies was provided by @xcite . while certain authors quantitively analyzed specific word-order patterns (e.g.: subject, verb, and object position @xcite , and verb and locative phrases @xcite ), other authors have focused on quantitative analyses regarding language complexity (e.g.: @xcite and @xcite ). with the aim of examining diachronic syntactic changes that characterize the evolution from latin to romance languages, @xcite proposed a quantitative approach to analyze the distributions of dependency directions. in total, 15 modern languages (8 romance languages and 7 from other families) and 2 ancient ones (latin and ancient greek) were scrutinized by the extraction of syntactic information from annotated corpora. the attested dependency syntactic networks for each language were analyzed with the calculation of certain syntactic parameters extracted from each corpus (i.e.: the mean sentential length, the percentage of the head-final dependencies, the head-initial dependencies, the dependencies between adjacent words, and of dependencies between non-adjacent words, the mean distance of all head-final dependencies, and the mean distance of all head-initial dependencies). it has been shown that the dependency syntactic networks arising from the selected data-sets reflect the degree of inflectional variation of each language. the adopted clustering approach also allowed romance languages to be differentiated from latin diachronically and between each other synchronically. however, the authors used data from the shared tasks of @xcite and conll 2007 @xcite , however the dependency annotation schemes differed substantially from each other, so any studies based on those treebanks were problematic. another method concerning the extraction and comparison of syntactic information from treebanks was proposed by @xcite . they developed the marsagram tool, a resource that allows typological syntactic information (together with its statistics) to be obtained by inferring context-free grammars from syntactic structures inside annotated corpora. in terms of word-order, this tool allows the extraction of linear patterns (i.e.: if a specific part-of-speech precedes another one inside the same node of the syntactic tree governed by a determined head). the authors conducted a cluster analysis comparing 10 different languages and showed the potential in terms of typological analysis of this resource. however, the results were only compared to the genealogical classification of the selected languages and did not provide any comparison to other quantitative methods. thus, one of the corpus-based typological approaches to be examined and compared in this article concerns the linear patterns provided by marsagram tool. the concept of typometrics was introduced by @xcite . the authors extracted rich details for testing typological implicational universals and explored new kinds of universals, named quantitative universals. in their study, different word-order phenomena were analyzed quantitatively (i.e.: the distribution of their occurrences in annotated corpora) to identify universals (i.e.: present in all or most languages). our approach differs from theirs as our aim is not to identify these implications or correlations but to compare languages (i.e.: language vectors) using all syntactic structures identified in the corpora to obtain a more general syntactic overview of the elements in our language set. what is possible to observe in many studies regarding corpus-based typology is that usually a method is presented without a specific comparison to the existing approaches or to the classic one concerning the typological databases. moreover, usually, the selected corpora are not completely homogeneous in terms of size or genre. thus, in this study, the idea is to compare 20 different languages by using parallel corpora. @xcite showed the benefit of using this type of data, as the bias regarding size and content is avoided. especially in this case, where syntactic patterns are the center of the analysis, the usage of parallel sentences allows the focus to be on the syntactic strategies that are used by each language to express the same meaning. our objective is not to determine which is the best corpus-based approach, but to show how data can be explored from different angles, allowing typological nuances to be analyzed in detail. ## campaign design in this section, a brief overview of the selected datasets is provided, followed by a complete description of the syntactic typological approaches which were selected to conduct the corpus-based word-order analyses. ## results as explained previously, each one of the presented typological methods generates a cluster dendrogram which is displayed in this section (figures 1 to 4 ). starting with the lang2vec dendrogram (1), it is possible to notice that the central cluster is divided into two sub-groups (one composed of chinese, english, and swedish, and the other of finnish, german, icelandic, and the slavic pud languages). arabic is classified in the same sub-group as indonesian and romance languages. it is also noticeable that hindi, korean, japanese, and turkish form an isolated cluster. moreover, germanic languages are split into two sub-clusters, one formed by english and swedish, together with chinese, and the other composed of german and icelandic (grouped with slavic languages). regarding this specific genus, although polish and russian are closer in both dendrograms, czech is positioned closer to finnish. furthermore, as previously mentioned, when considering only lang2vec syntactic features, thai and arabic are classified as closer to romance languages when compared to the others in the pud collection. if we consider the 3 main clusters provided by this dendrogram, it is possible to analyze which syntactic lang2vec features are shared by its languages. the isolated cluster formed by hindi, japanese, korean and turkish is composed of sov languages with postpositions and with adjectives before nouns. the middle cluster (i.e.: slavic and germanic languages, plus chinese and finnish) has svo languages with adjectives before nouns. and, finally, the cluster on the right side of the dendrograms is composed of vo (but not necessarily svo) languages with prepositions and adjectives after the noun. moreover, this cluster differs from the one located on the extreme left side of the dendrograms by ordering the negative word before the verb. the analysis of the dendrogram concerning marsagram linear patterns (2) shows icelandic as an isolated language inside pud collection. japanese is also quite isolated from the other languages, however with lower distance values than icelandic. chinese, turkish, finnish, and hindi form one small central cluster, as well as italian and spanish, and the other languages are grouped altogether. for this specific representation, languages from the same family or genus are not always clustered together (e.g.: portuguese and spanish, which formed a sub-cluster in lang2vec dendrograms in this case). the large purple cluster is composed of romance, slavic, and germanic languages (except icelandic), but it also includes indonesian, arabic, thai and korean. subject, verb, and object positions are not relevant criteria in this type of language classification. marsagram extracts wordorder patterns between elements that are part of the same syntactic node, thus, these components are not necessarily syntactically related. when considering the classification provided by the dendrogram obtained with the head and dependent patterns (3), we observe that the romance languages form one single isolated cluster positioned on the left side of the figure. on the other hand, the germanic sub-group is closer to the slavic one (with icelandic being positioned inside the slavic cluster and not with the other germanic pud languages). it is also noticeable that thai, arabic, and indonesian form a specific sub-group closer to the germanic and slavic languages. japanese is clustered with hindi and closer to other ov languages (i.e.: turkish and korean). the large cluster containing all ov languages from pud also includes finnish and chinese which are not ov. when compared to the genealogical classification of pud languages, it is possible to see that the proximity between spanish and portuguese and their relation to french and italian is also present when the head and dependent orderings are examined. icelandic is genealogically closer to swedish, however, in terms of head directionality it is closer to slavic languages, this classification is closer to the one proposed by @xcite : icelandic, czech, and russian are all considered as type 10. nevertheless, still according to @xcite , indonesian and thai are from the same language type as romance languages (type 9), but in these dendrograms, although these two languages are grouped together, they are not classed among romance ones. moreover, although not genealogically related, the syntactic proximity between finnish and turkish is similarly attested with the head directionality analysis. as expected, when vo and ov patterns are used to generate a dendrogram (4), there is a clear split of pud languages into two clusters: one contains all ov languages and german (with no dominant order, according to wals database), and the other, all the vo languages. when analyzing vo languages in detail, it is noticeable that french and czech are closer in the verb and object relative position dendrogram. finnish is placed together with germanic languages (except for german) and indonesian. slavic languages (except for czech) are clustered with romance languages (except for french) in a sub-group that also contains arabic. the thai language forms a small sub-cluster with chinese. as not only nominal objects are considered for the construction of this dendrogram, it also provides also insights into how other types of objects are ordered (e.g.: pronominal). thus, this classification cannot be compared to the one provided by @xcite where only nominal objects were analyzed. the overall analysis of all obtained dendrograms shows that both lang2vec and head and dependent position figures have more similarities to the classical genealogical classification of languages. marsagram dendrogram clearly presents a specific typological classification that considers word-order phenomena not contemplated by the other analysis. the verb and object classification provides a particular typological overview that can be interesting for studying focusing on how these two elements are positioned. in comparison with the language types proposed by @xcite , the typological classifications presented in this article present the advantage of allowing languages to be compared in terms of a larger number of word-order structures, thus, being more precise for nlp applications where the objective is to find the closest languages. for example, as previously mentioned, indonesian and thai are classified as type 9 by @xcite , the same group as the pud romance languages. however, using the described quantitative methods it is possible to determine how close these two languages are to the romance ones in a more detailed way. ## conclusion and perspectives in this paper, we presented three new typological approaches regarding word-order phenomena applied to 20 different languages using parallel corpora. the new methods were compared to the standard one which considers syntactic features provided by a typological database (lang2vec). each approach provided a syntactic typological classification of languages in the format of a dendrogram which was obtained via dissimilarity matrices composed of euclidean distances between language vectors. we showed that each different approach has its own particularities. the aim of this study was not to state which is the best typological method but to show in which way they provide different angles for typological analysis. however, it is possible to notice that the lang2vec and the head and dependent relative position dendrograms are more coherent with the genealogical classification of languages. the marsagram approach provides interesting aspects regarding specific word-order phenomena of elements that are not syntactically related, while the verb and object relative position approach provides a specific analysis of all attested phenomena regarding these elements. the usability of each method depends on which particular syntactic features are of interest and the purpose of further linguistic processing. preliminary experiments showed that the language distances obtained using the described quantitative typological methods present moderate or strong correlations with the improvement of dependency parsing results when different languages are combined to train deep-learning models. thus, in the future, we aim to analyze precisely how each method provides valuable information concerning the improvement of the dependency parsing results to determine the best corpus-based typological strategy for this aim.