Corpus-based Syntactic Typological Methods for Dependency Parsing Improvement (2023)

Abstract: This article presents a comparative analysis of four different syntactic typological approaches applied to 20 different languages to determine the most effective one to be used for the improvement of dependency parsing results via corpora combination. We evaluated these strategies by calculating the correlation between the language distances and the empirical LAS results obtained when languages were combined in pairs. From the results, it was possible to observe that the best method is based on the extraction of word order patterns which happen inside subtrees of the syntactic structure of the sentences.

## introduction dependency parsing is a natural processing processing (nlp) task that concerns the process of determining the grammatical structure of a sentence by examining the syntactic relations between its linguistic units. in other words, it consists of the identification of heads and dependents as well as the type of relationship between them @xcite . from 2015 onward, the usage of deep learning techniques has been dominant in studies regarding the dependency parsing task. although it has provided a great improvement in overall results even for under-resourced languages @xcite , it requires a large amount of annotated data which can be problematic, particularly in terms of cost @xcite . to overcome the problem of lack of data, crosslingual parsing strategies using typological methods have been proposed to determine which languages can be combined for effective improvement of dependency parsing results @xcite . most of these studies rely on the usage of information provided by typological databases such as wals @xcite sometimes combined with n-grams analysis extracted from corpora. on the other hand, the usage of corpusbased typology for this aim is still incipient. moreover, most studies focus on the obtained improvement, without analyzing the existence of a proper correlation between the typological features involved in the process with the overall synergy regarding the impact on the dependency parsing results. therefore, our aim in this paper is to propose an examination of several corpus-based typological methods in terms of correlation between language distances and dependency parsing scores. the paper is composed as follows: section 2 presents an overview of the related work to this topic. in section 3, we describe the campaign design: language and data-sets selection, corpus-based typological characterization, dependency parsing experiments, and correlation measures; section 4 presents the obtained results which are discussed in section 5. in section 6 we provide conclusions and possible future directions for research. ## related work the wals database is one of the most used typological resources in nlp studies @xcite . it contains phylogenetic, phonological, morphosyntactic, and lexical information for a large number of languages that can be used for a large variety of linguistic studies @xcite . along with that, the uriel typological compendium was conceived as a metarepository that is composed of numerous databases (wals included) and is the base of the lang2vec tool @xcite . this tool is a powerful resource that allows languages to be characterized as vectors composed of typological features associated with specific values. users can choose the type of features (i.e.: genealogical, phonological, syntactic, etc) according to their precise needs. while proposing an effective way to compare languages typologically, this tool does not characterize all lan-guages homogeneously as it depends on the availability of linguistic descriptions provided by its sources. thus, low-resourced languages usually have less information. for example, it is not possible to compare all 24 european union languages as there are no common features with valid values for all of them. furthermore, typological databases usually fail to illustrate the variations that can occur within a single language (i.e.: in general, only the most frequent phenomena are reported in the literature, and not all attested ones). in terms of corpus-based typological studies, a broad survey was provided by @xcite . the author showed that while several authors quantitively analyzed specific word-order patterns (e.g.: subject, verb, and object position @xcite , and verb and locative phrases @xcite ), other researchers have focused on quantitative analyses regarding language complexity (e.g.: @xcite and @xcite ). on the other hand, the concept of typometrics was introduced by @xcite . the focus of their research was to extract rich details from corpora for testing typological implicational universals and explored new kinds of universals, named quantitative ones. thus, different word-order phenomena were analyzed quantitatively (i.e.: the distribution of their occurrences in annotated corpora) to identify the ones present in all or most languages. thus, it is possible to notice that most studies regarding quantitative typology focus either on the analysis of specific linguistic phenomena or on the identification of universals. our approach differs from theirs as our aim is to compare languages (i.e.: language vectors) using quantitative information concerning all syntactic structures extracted from corpora to obtain a more general syntactic overview of the elements in our language set and use the results as strategies to improve dependency parsing results. an interesting method concerning the extraction and comparison of syntactic information from tree-banks was developed by @xcite . the marsagram tool is a resource that allows syntactic information (together with its statistics) to be extracted from annotated corpora by inferring context-free grammars from the syntactic structures. marsagram allows the extraction of linear patterns (i.e.: if a specific part-of-speech precedes another one inside the same subtree ruled by a determined head). the authors conducted a clus-ter analysis comparing 10 different languages and showed the potential in terms of typological analysis of this resource. however, the results were only compared to the genealogical classification of the selected languages and did not provide any comparison to other corpus-based methods. moreover, the authors did not use the obtained classification with the perspective of improvement of dependency parsing systems via corpora-combination. one example of effective usage of typological features (from uriel database) to improve results of nlp methods was presented by @xcite . the authors developed the udapter tool that uses a mix of automatically curated and predicted typological features as direct input to a neural parser. the results showed that this method allows the improvement of the dependency parsing accuracy for low-resourced languages. a similar study, using a different deep-learning architecture was conducted by @xcite , however, in both cases, there is no detailed analysis of which features were the most relevant. furthermore, @xcite proposed a study concerning the irish language using delexicalized corpora. the authors performed a series of crosslingual direct transfer parsing for the irish language and the best results were achieved with a model trained with the indonesian corpus, a language from the austronesian language family. the authors proposed some analysis considering similarities between the treebanks of both languages in terms of dependency parsing labels, however, a detailed statistical analysis of corpora and a complete comparison of specific typological features were not carried out. while some papers focus on genealogical features, others consider syntactic ones. for example, @xcite presented a study whose aim was to identify cross-lingual quantitative trends in the distribution of dependency relations in annotated corpora from distinct languages by using an algorithm (lisca -linguistically-driven selection of correct arcs) @xcite which detects patterns of syntactic structures in tree-banks. however, only four indo-european languages were scrutinized but some interesting insights concerning language peculiarities were observed. thus, studies regarding corpus-based typology and dependency parsing are usually presented without a specific comparison to other existing ap-proaches or to the classic one concerning typological databases. that is why in this article the idea is to analyse possible quantitative typological methods in terms of correlation with the improvement obtained regarding dependency parsing results when corpora from different languages are combined. ## campaign design in this section, a brief overview of the selected data-sets is provided, followed by a description of selected the corpus-based syntactic typological approaches. moreover, we detail the dependency parsing experiments and the correlation measures that were chosen for the analysis of the results. ## results in the following subsections, we present the baseline results regarding the dependency parsing experiments together with an overview of the las values obtained when languages were associated. then, the correlation analyses are displayed. ## discussion the results displayed in moreover, from tables 5 and 6, it is possible to notice that the method with the highest number of inverse correlations is the marsagram linear one with language distances calculated with the cosine measure. the scores were either moderate or strong for half of the languages in the pud collection. this specific corpus-based approach seems to be more effective than the state-of-the-art one (i.e.: using features from the lang2vec tool). moreover, since the highest values were obtained with pearson's correlations, it is possible to say that what is observed is a linear inverse correlation between the distances and the deltas. however, even though the marsagram linear (cosine) strategy provides the most optimized results, it fails to explain the las values for 10 pud languages. for icelandic, indonesian, and turkish, the pearson's correlation coefficient of this strategy is lower than -0.2, which indicates, at least, a low correlation, while for italian, this coefficient is lower than -0.10 but higher than -0.20. on the other hand, for chinese, japanese, german, and russian, this coefficient is very close to 0.00 (i.e.: no correlation). and, for korean and hindi, values are positive. with the values from the dissimilarity matrix obtained using the marsagram linear method, it is possible to generate a dendrogram with the hclust() function using r. the classification in clusters is presented in the annex (figure 6 ). it is possible to notice some similarities with the languages' genealogy (e.g.: romance languages in the same cluster) and with other typological classifications (e.g. ov languages on the same side of the dendrogram), however not all languages are classed following these expected configurations. ## conclusion and perspectives in this paper, we presented four corpus-based typological approaches and evaluated them in comparison with the state-of-the-art method consisting of using syntactic information from databases. first, we described these new strategies followed by the results of the dependency parsing experiments via corpora association. we showed that the combination of corpora is an effective way to improve las results in lowresourced scenarios and that the typological approach concerning the order of elements inside subtrees (marsagram linear) is the one with the highest number of moderate and strong correlations for the languages in the pud collection. in the future, we aim to analyze in detail the languages for which this method was not effective. moreover, we intend to increase the number of languages to have a more homogeneous language-set in terms of the number of languages per linguistic family as well as conduct tests with non-parallel corpora. another perspective for future work is to optimize marsagram linear method defining weights for the features as the extracted patterns may influence the results differently. hd vo vo l2v l2v all all lin. lin. euc. cos euc. cos euc. cos euc. cos euc. cos arb -0.11 -0.52 -0.03 -0.57 -0.54 -0.65 -0.59 -0.47 -0.59 -0.55 cmn 0.19 -0.11 -0.26 0.00 0.25 0.15 -0.06 -0.21 0.01 -0.03 ces -0.25 -0.60 -0.28 -0.57 -0.65 -0.67 -0.57 -0.57 -0.36 -0.28 eng -0.34 -0.53 -0.21 -0.59 -0.41 -0.49 -0.35 -0.16 -0.36 -0.41 fin -0.16 -0.52 -0.46 -0.63 -0.46 -0.44 -0.71 -0.72 -0.10 -0.01 fra -0.50 -0.51 -0.52 -0.62 -0.62 -0.59 -0.38 -0.31 -0.50 -0.47 deu -0.48 -0.11 -0.22 -0.03 -0.23 -0.22 0.03 0.46 -0.03 -0.02 hin -0.36 -0.27 0.05 0.40 0.12 0.41 0.56 0.50 0.44 0.46 isl 0.18 -0.19 -0.26 -0.36 -0.12 -0.31 -0.49 -0.44 -0.40 -0.42 ind 0.23 -0.30 0.20 -0.21 0.12 0.05 0.00 0.05 -0.21 -0.11 ita -0.21 -0.23 -0.02 -0.13 -0.14 -0.17 -0.30 -0.16 -0.10 -0.17 jpn -0.18 0.06 -0.15 -0.05 0.38 0.35 0.02 0.07 0.40 0.50 kor 0.30 0.29 0.08 0.38 0.42 0.49 0.41 0.47 0.43 0.37 pol -0.23 -0.37 -0.50 -0.62 -0.13 -0.34 -0.51 -0.40 -0.37 -0.34 por -0.64 -0.52 -0.39 -0.61 -0.64 -0.53 -0.45 -0.40 -0.57 -0.50 rus -0.16 -0.08 0.17 0.03 -0.27 -0.24 -0.46 -0.28 -0.15 -0.17 spa -0.59 -0.45 -0.57 -0.51 -0.53 -0.50 -0.43 -0.38 -0.60 -0.55 swe -0.48 -0.59 -0.31 -0.64 -0.58 -0.63 -0.59 -0.49 -0.70 -0.68 tha 0.26 -0.59 -0.22 -0.62 -0.64 -0.88 -0.60 -0.80 -0.76 -0.81 tur -0.09 0.10 -0.34 -0.25 -0.45 -0.53 -0.42 -0.56 -0.61 -0.60 euc. cos arb -0.05 -0.33 -0.09 -0.53 -0.55 -0.66 -0.65 -0.52 -0.70 -0.69 cmn 0.24 -0.15 -0.12 -0.08 0.36 0.18 0.03 -0.12 -0.02 -0.03 ces -0.14 -0.54 -0.31 -0.49 -0.51 -0.48 -0.52 -0.57 -0.31 -0.31 eng -0.38 -0.59 -0.27 -0.48 -0.49 -0.52 -0.46 -0.02 -0.37 -0.38 fin -0.20 -0.48 -0.41 -0.60 -0.35 -0.44 -0.74 -0.66 -0.09 -0.06 fra -0.50 -0.48 -0.55 -0.59 -0.57 -0.56 -0.47 -0.26 -0.50 -0.53 deu -0.52 -0.28 -0.22 -0.03 -0.30 -0.29 0.05 0.44 -0.09 -0.08 hin -0.31 -0.23 0.06 0.32 -0.05 0.34 0.68 0.60 0.43 0.44 isl 0.24 -0.19 -0.21 -0.46 -0.03 -0.20 -0.50 -0.26 -0.43 -0.44 ind 0.13 -0.27 0.02 -0.22 0.04 0.01 -0.16 -0.24 -0.29 -0.23 ita -0.23 -0.31 -0.02 -0.11 -0.16 -0.15 -0.24 0.12 -0.20 -0.20 jpn 0.08 0.16 -0.01 -0.26 0.45 0.52 -0.10 -0.16 0.50 0.49 kor 0.52 0.34 0.13 0.52 0.18 0.53 0.22 0.17 0.24 0.27 pol -0.29 -0.44 -0.67 -0.62 -0.23 -0.42 -0.55 -0.48 -0.31 -0.31 por -0.42 -0.29 -0.23 -0.37 -0.41 -0.42 -0.49 -0.47 -0.48 -0.47 rus -0.01 -0.14 0.16 0.07 -0.08 -0.09 -0.46 -0.06 -0.08 -0.06 spa -0.51 -0.45 -0.55 -0.55 -0.56 -0.53 -0.50 -0.55 -0.67 -0.66 swe -0.46 -0.73 -0.38 -0.68 -0.70 -0.74 -0.80 -0.40 -0.64 -0.63 tha 0.25 -0.49 -0.19 -0.62 -0.51 -0.81 -0.36 -0.69 -0.68 -0.70 tur 0.09 -0.15 -0.26 -0.18 -0.59 -0.69 -0.15 -0.31 -0.59 -0.57