C ompara T ree: A Multi-Level Comparative Treebank Analysis Tool (2025)

Abstract: ComparaTree is a tool for comparative treebank analysis that combines various methods of quantitative linguistic analysis to provide a general overview of the differences and similarities between two treebanks. The comparison tool covers a range of subfields of linguistic analysis, providing a summary of the differences and similarities in terms of the lexical diversity, n-gram diversity, part-of-speech and dependency relation proportions, syntactic complexity, and syntactic diversity. We explain the various quantitative analyses performed on every level along with the generation of graphical visualizations, which add value by enabling user-friendly comparisons at a glance. We exemplify the comparison process by presenting the results produced by the tool when comparing two treebanks from the Universal Dependencies collection.

## introduction the universal dependencies initiative (de @xcite has produced a large repertoire of treebanks featuring a consistent, crosslinguistically applicable grammatical annotation format. as of the latest 2.15 release of ud, the collection includes almost 300 treebanks in over 150 languages @xcite , while at the same time boasting considerable diversity in terms of the various text genres included in the treebanks @xcite , containing also different language modalities such as spoken language @xcite . given this high degree of diversity, the ud collection is ideal for conducting both intra-linguistic as well as cross-linguistic comparisons, with cross-linguistic studies using ud treebanks becoming especially common (e.g., @xcite , @xcite , @xcite ). although comparative studies based on universal dependencies are becoming increasingly common, there is still a lack of general-purpose tools that facilitate such analyses in a systematic way, as only a handful of specialized tools currently support comparative work. the quansyn python package @xcite supports the analysis of syntactic properties, such as the distribution of parts of speech and dependency relations, within and across treebanks. the stark tool @xcite enables the extraction of dependency (sub)trees from parsed corpora and supports frequency-based comparisons between datasets. the conllu-diff utility foot_0 generates statistical summaries of differences between conll-u files, but is limited to individual token-level labels. beyond these, various processing tools and programming packages are listed on the official ud website, foot_1 but they are typically not optimized for direct comparative analysis. while each of the tools mentioned above offers valuable functionality, they are typically limited in scope-focusing on a single linguistic level, requiring programming expertise, or lacking support for user-friendly side-by-side comparison. to address this gap, the present paper introduces comparatree, a user-friendly tool for comparative treebank analysis that combines multiple methods of quantitative linguistic analysis. it supports comparisons across lexical diversity, n-gram diversity, part-of-speech and dependency relation distributions, syntactic complexity, and syntactic diversity. comparatree also generates visualizations in the form of graphs and diagrams, providing a clear visual overview of the similarities and differences between two treebanks. in the present paper we first describe the different levels of linguistic analysis for which com-paratree generates a comparison in section 2. in section 3 we exemplify the usage of the tool and present the format of the results by performing an analysis using two ud treebanks. finally, in section 4 we conclude with a discussion of the possible future improvements and extensions. ## treebank comparison comparatree is a tool written in the python programming language that takes two treebanks in the conll-u format foot_2 as input and calculates values for various linguistic measures. although the tool was designed to be used with ud treebanks, in principle any dependency grammar formalism is supported, as long as the treebank used conforms to the standard conll-u format. the source code of the tool is publicly available and can be accessed via a dedicated github repository along with the documentation for its use. 4 the calculated linguistic measures pertain to five different levels of linguistic analysis: lexical diversity, n-gram diversity, part-of-speech and dependency relation proportions, syntactic complexity, and syntactic diversity. for every level of comparison the tool also outputs a visualization of the results. in the following, we first describe a special process of segment-based averaging in section 2.1 that is performed for several of the analysis levels. next we describe the various measures calculated on each level in section 2.2, while section 2.3 introduces the various types of resulting visualizations. ## example comparison: ssj-ud vs sst-ud in this section we present an example comparison performed using the comparatree tool. the pair of compared treebanks consists of the slovenian ssj ud treebank @xcite , which represents a balanced sample of written slovenian, and the slovenian sst ud treebank @xcite , which represents a balanced sample of spoken slovenian. both treebanks were provided to comparatree as input in the conll-u file format and all the default levels of lingusitic analysis were included. the default segment length of 1000 tokens was used, while for the n-gram analysis only 3-grams were analyzed during this comparison session. in appendix a, on the basic level, ## conclusion in this article we introduced comparatree, a tool for comparative linguistic analysis which produces a multi-level comparison of two treebanks. we presented the various levels of linguistic analysis that comparatree offers and exemplified its use and output using two treebanks included in the universal dependencies treebank collection. many functionalities still remain to be added to comparatree, which will improve its analysis capabilities. presently only the ud label proportion analysis is equipped with a statistical significance test that establishes the statistical significance of the observed patterns. in the future, various methods of statistical significance testing along with effect size calculations should be added to other analysis dimensions as well. although the current presentation of results offers a good glimpse into the tendencies that can be observed in the data, rigorous statistical methods are required to give additional weight to the findings made using com-paratree. additionally, the tool presently only supports pairwise comparisons of two treebanks. important insights could be gained from comparing more than two treebanks simultaneously, so support for multiple comparisons should be implemented in the future. such an expansion should also be accompanied by more advanced visualization techniques, which would shed light on different tendencies present in the data and complement the current assortment of histograms and barcharts. there is also much room to expand the current inventory of measures calculated and range of analyses performed at each level (also in line with new methods that have recently been proposed, such as in čibej (upcoming)) as well as the potential to expand into other dimensions of linguistic analysis, such as semantics, discourse analysis, etc. future improvements to the tool in this regard should be dictated by the demand presented by the target users and the broader computational linguistics community. comparatree for the comparison between the ssj and sst ud treebanks. the ud label proportions subdivision presents the four labels for which the proportion difference between the two treebanks is the greatest, thus presenting the labels that are most typical of one treebank with respect to the other. the absolute values of the differences are provided next to the label names.