On the Flatness, Non-linearity, and Branching Direction of Natural Language and Random Constituency Trees: Analyzing Structural Variation within and across Languages (2025)

Abstract: Natural languages exhibit remarkable diversity in their syntactic structures. Previous research has investigated the cross-lingual differences in local structural features such as word order or dependency relations. However, considering structural variation within individual language, it remains unclear how such features influence the variation in the overall constituency tree structure and hence the structural variation across languages. To this end, we focus on the shape of constituency trees, analyzing the cross-lingual overlap in the distributions of flatness, non-linearity, and branching direction. While acknowledging that the findings may be influenced by the potential annotation idiosyncrasies across treebanks, the experiments quantitatively suggest that flatness and branching direction vary significantly across languages. As for non-linearity, the cross-lingual difference was relatively small, and the distributions tend to skew towards linear structures. Furthermore, comparison with randomly generated trees suggests that while phrase category and frequency information is crucial for reproducing the branching direction found in natural languages, non-linearity can be replicated reasonably well even without such information.

## introduction uncovering the universals and variations in syntactic structures across natural languages is a central challenge in computational linguistics and natural language processing. in the context of linguistic typology, differences and universal properties among languages have been extensively discussed from perspectives such as word order @xcite @xcite , dependency relations @xcite , morphology @xcite @xcite , and phonology @xcite . however, capturing structural variation within language remains a challenge for existing typological studies @xcite . when considering within-language variations, it is not trivial how local structural features, such as word order and dependency relations, influence the variation in the overall structure of constituency trees and, consequently, relate to cross-lingual structural differences and universals. to address this issue, we propose an approach based on the shape of constituency trees. specifically, we quantify three features characterizing tree shape: flatness, non-linearity, and branching direction (figure 1 ). 1 we then analyze the distribution of these shape features within each language and their distributional overlap across languages, using treebanks from diverse languages. a key advantage of our approach, using tree shape features, is its potential to enable comparisons beyond natural languages. while traditional linguistic typology relies mainly on features derived from linguistic theories and often limits its scope to comparisons among natural languages @xcite , our approach allows us to investigate the statistical properties of natural language constituency trees within the broader space of all theoretically possible tree structures. furthermore, in addition to comparing tree shape distributions across natural languages, we conduct comparisons with randomly generated tree structures. this comparative analysis aims to shed light on the fundamental question of what information (e.g., grammatical category information) is essential for characterizing the structure of natural language trees. our analyses are conducted on constituency treebanks from 11 diverse languages. experimental results quantitatively show that while flatness and branching direction exhibit significant crosslingual variation with minimal distributional overlap, the distributions of non-linearity are skewed towards the linear region, resulting in a certain degree of overlap across languages. moreover, comparisons with random trees suggest that category information is crucial for reproducing branching direction, whereas non-linearity can be relatively well replicated even without such information. however, a key limitation is that the treebanks used in the experiments are not harmonized, meaning that the findings may be influenced by differences in the annotation schemes. disentangling the genuine linguistic differences from potential annotation artifacts is a crucial direction for future work. the implementations of the experiments are available at https://github.com/ mynlp/tree-shape-distribution.git . ## background cross-lingual analysis based on the shape features of dependency trees has been actively conducted. for example, (directed) dependency distance is used to investigate typological differences @xcite and universal tendencies like dependency distance minimization @xcite . previous research has also examined structural properties like clause/center embedding depth across languages @xcite and statistical patterns such as menzerath's law @xcite @xcite , sometimes including comparisons with ran-dom trees @xcite . in contrast, cross-lingual analysis based on the shape of constituency trees is relatively sparse compared to those on dependency trees. for example, while there are studies on the relationship between the center embedding depth and human reading time in english @xcite , and a comparison of branching direction in english and chinese @xcite , these studies are often limited to a small number of languages. tanaka- @xcite conducted an extensive analysis on various natural languages and random trees, but their work is limited to the strahler number @xcite , which measures the lower bounds on memory requirements for processing constituency trees. this study aims to conduct a systematic comparative analysis specifically for constituency trees, using 3 tree shape features -flatness, non-linearity, and branching direction-across a diverse set of languages. ## tree shape measures this section defines the tree shape measures used in this study. following @xcite , we analyze the tree shape of delexicalized constituency trees, where leaves are pos tags and internal nodes represent phrases. while some parsing research assumes only binary trees @xcite , we consider general n-ary tree structures after removing unary nodes and ignore phrase category labels. we specifically examine three features: flatness, non-linearity, and branching direction ( ## experiments and discussion datasets. in this study, we use treebanks from 11 languages: english (penn treebank @xcite ), chinese (chinese treebank @xcite ), japanese (npcmj), french, german, korean, basque, hebrew, hungarian, polish, and swedish (spmrl @xcite ). 6 it should be noted that these treebanks are not harmonized and thus annotation schemes are not identical. following chan et al. (2010), we focus on delexicalized constituency trees. for preprocessing, we apply the following steps to the annotated tree structures in each treebank: (1) remove null elements, (2) strip functional tags from category labels, (3) remove word tokens and treat pos tags as the new leaf nodes, and finally (4) remove unary nonterminals by concatenating their category labels, similar to @xcite . 7 note that we do not remove punctuation. furthermore, to analyze the distributions of overall tree shape, we use only sentences with lengths of 10 or more in the experiment. the tree shape distributions for random tree models are also calculated from this subset. this is because shorter sentences have a limited number of possible tree structures, which makes it difficult to analyze crosslingual differences. additionally, this length-based filtering provides a simple way to exclude typically short non-sentential fragments, e.g., "(frag (pu () (vv 完) (pu )))". 8 evaluation. for all evaluation metrics, we compute distributions as normalized histograms with 100 bins. to quantify cross-lingual differences in tree shape distributions, we use histogram intersection (hi). hi measures the proportion of overlap among a set of distributions, yielding a score in [0, 1], where 0 indicates no overlap and 1 indicates identical distributions. this direct measure of overlap makes the score highly intuitive to interpret. furthermore, unlike pairwise metrics such as kullback-leibler divergence, hi has the advantage of being applicable to multiple distributions simultaneously, which allows us to compute a single overall score across all languages. while hi quantifies the overall overlap, to further understand the characteristics of each distribution, we also analyze its shape using standard deviation, skewness, and kurtosis. skewness measures how a distribution is biased towards left or right; positive/negative skewness implies large part of the distribution is on the left/right-side. kurtosis describes the sharpness of a peak of distribution and the weight of its tails compared to a normal ar nce cc ewc rj hi 0.03 0.33 0.03 0.02 0.02 ## conclusion we investigated structural variation of constituency trees within and across 11 languages, focusing on flatness, non-linearity, and branching direction. analysis of the cross-lingual distributional overlap revealed that flatness and branching direction vary significantly across languages, indicating that cross-lingual differences emerge even when considering the structural variation within each language. meanwhile, the distributions of non-linearity showed smaller cross-lingual difference and tend to skew towards linear trees. comparison with 6 random tree models based on the yule model and pcfg showed that category information, accompanied by frequency statistics, are crucial for reproducing the branching direction patterns in natural language. in contrast, non-linearity was reasonably replicated even by relatively simple yule models that lack such information, suggesting that non-linearity may be governed by more universal mechanisms independent of fine-grained grammatical details. while this work focused on the shape of constituency trees, a key future direction is to analyze the joint distribution of overall tree shape features and local structural features, such as word order or dependency relations used in traditional linguistic typology. such an analysis could lead to a deeper understanding of cross-lingual variations and universality in syntactic structures.