Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions (2020)

Abstract: A grammatical gender system divides a lexicon into a small number of relatively fixed grammatical categories. How similar are these gender systems across languages? To quantify the similarity, we define gender systems extensionally, thereby reducing the problem of comparisons between languages’ gender systems to cluster evaluation. We borrow a rich inventory of statistical tools for cluster evaluation from the field of community detection (Driver and Kroeber, 1932; Cattell, 1945), that enable us to craft novel information theoretic metrics for measuring similarity between gender systems. We first validate our metrics, then use them to measure gender system similarity in 20 languages. We then ask whether our gender system similarities alone are sufficient to reconstruct historical relationships between languages. Towards this end, we make phylogenetic predictions on the popular, but thorny, problem from historical linguistics of inducing a phylogenetic tree over extant Indo-European languages. Of particular interest, languages on the same branch of our phylogenetic tree are notably similar, whereas languages from separate branches are no more similar than chance.

## introduction as many as half the world's languages carve nouns up into classes @xcite . in these languages, nouns are subdivided into gender categories, which together comprise the language's grammatical gender system. a gender system tends to use a small, fixed number of categories with fixed usage across speakers. such categories, like 'feminine', can be defined extensionally, 1 and are reflected by agreement with other words within the noun phrase (i.e., concord). gender exhaustively divides up the language's nouns; that is, the union of gender categories is the entire nominal lexicon. taken this way, a gender system can be viewed as a partition of the lexicon into communities of same-gendered nouns. given this, a lexical typologist might naturally wish to ask: how similar are two languages' gender systems? using modern statistical and informationtheoretic tools from the community detection literature, we offer the first cluster evaluation @xcite perspective on grammatical gender, and quantify the overlap of gender systems. we can compare the pairwise overlap of partitions of gender systems using a rich literature of measures, such as mutual information and several variants @xcite @xcite , which we survey and contrast. individual partitions of lexicons can also be framed as members of distributions over partitions-for instance, the distribution consisting of all partitions of n items, or of all partitions of n items into k gender clusters, as in figure 1 . for example, spanish is bi-gendered (with masculine and feminine): a lexicon of spanish nouns (@xmath0. the same lexicon translated into german, a trigendered language, would come from a distribution of @xmath1. indeed, languages needing different numbers of gender clusters makes this problem non-trivial. from this, we can compare the similarity to what we would expect for the same lexica if nouns were randomly supplied with gender specifications. that way, we can distinguish meaningful relationships from noise. armed with the first way to quantify communitywise similarity of gender systems, we ask: do gender system similarities reflect linguistic phylogeny, or something else, like areal effects? across 20 languages, we find that our pairwise overlap results measurably align with standard pairwise phylogenetic relationships. zooming in on indo-european, we find that we can recast pairwise similarities into an accurate phylogenetic tree, simply by measuring distance between gender systems and performing hierarchical agglomerative clustering (see §6.2). the primary contribution of this work is a novel metric for lexical typology that measures the pairwise similarity of gender systems. we operationalize gender systems as partitions over a shared set of nouns ( §3). we design and evaluate our measurements of gender system similarity under this formulation ( §4), drawing on insights from community detection. then we recover robust phylogenetic relationships between pairs of gender systems by applying these to 20 gendered languages ( §6) and find that similarity between slavic and romance gender systems does not exceed chance levels. finally, we show that our quantification of gender system similarity allows us to construct phylogenetic trees that closely resemble those posited for indo-european in historical linguistics (e.g., @xcite @xcite . ## background: grammatical gender grammatical gender is a highly fixed classification system for nouns. native speakers rarely make errors in gender recall, which might tentatively argue against tremendous arbitrary variation @xcite . some regularity can surely be found in the associations between gender and various features of the noun, such as orthographic or phonological form, or semantics. with respect to form-based regularities, @xcite devise a system for inferring noun gender (masculine or feminine) from contextual clues and character representations, even in inflected forms of the noun. @xcite also find that phonological form can lead to predictability of gender in two three-gender systems. with respect to word semantics, @xcite quantify the relationship between the gender on inanimate nouns and their distributional word vectors. we can't rely on form. using phonological or orthographic form to derive gender is fraught with complications: particular to our study, epicene nouns (i.e., words that can appear in multiple genders) can pose issues. in german, only gender concord on the definite article and adjectives can disambiguate the gender of some nouns; the same wordform band means "volume" when masculine, but "ribbon" when neuter and "band, musical group" in feminine. another complication with determining gender from the phonological or orthographic form of the noun is that correspondences between are rarely absolute. for example, even though nouns ending in -e are usually 'feminine' in german, this is not universally the case; for example affe, and löwe etc. are masculine. to sidestep these complications, we abstract away from particular word forms and observe the objective consequences of gender over sets of cross-lingual concepts, i.e., indices not word forms, and instead compare those across gender systems (see figure 1 ). which gender systems are likely to be similar? several accounts highlight similarities between the gender systems of phylogenetically-related languages @xcite and argue that they are likely to be at least partially due to historical relations between communities and socio-political factors governing language use. given this, can we recover phylogenetic similarities across gender systems using our methods? if so, this should provide validation that we are indeed measuring at least some of the genuine similarity that exists between gender systems. ## gender systems as partitions any concept can be related to its referents either intensionally or extensionally. while linguistic research has historically sought to uncover the rules for associating a noun with gender in terms of surface features or semantics (see corbett 1991 for an overview), we take an extensional approach. that is, we treat a gender category in a language solely as the set of words it covers. this maps directly to the notion of a community in the network science task of community detection: a community is defined by membership, not by other arbitrary properties, just as a gender here is defined by the union of all nouns it subsumes, not by its phonological realization or contributions to semantics. the disjoint set of communities forms a partition of the set of nouns: each noun is a member of one and only one cluster. although some epicene nouns are present in our investigated languages (see §2), these are very rare. we thus make the simplifying modeling assumption of identifying each word with only a single gender (in our case, the most frequent). this assumption is necessary for our reduction of gender system comparison to clustering evaluation. without it, we would be forced for words like german der/die/das band to consider overlapping or "fuzzy" partitions, which although an intriguing option, will be left for future work. notation. a language's gender system is a partition, named in sans serif (e.g., a). a gender system a has k components called gender classes (i.e., communities, e.g., {a msc , a fem , . . .}); these are in turn sets whose members are items drawn from a finite base set a ⊆ l, where a is a sublexicon selected from the full lexicon l. in our case, a holds all inanimate concepts in our data (see §5). we use ω to name the set of all partitions of @xmath2, inanimate nouns) into k communities. when comparing two languages' respective gender systems, we will use the letters a and b. ## comparing partitions a partition groups items into a set of disjoint categories. we could compare any two gender systems (i.e., partitions) which organize the same nouns by determining how similar their gender labelings are. a first pass at quantifying the similarity of two gender partitions would be to measure simple overlap. we could ask: what fraction of a agrees in gender across languages? that is, for each noun in our multilingual vocabulary, do both languages lexicalize it with the same gender? this is an easily interpretable, accuracy-like measure, bounded by 0 and 1. still, it has no capacity for comparing systems with different numbers of categories; the measure would be handicapped when comparing two-gender systems to three-gender ones. comparing systems with different numbers of categories, though, is a well known problem in the field of community detection. while this looks insurmountable from the gender perspective, where gender categories refer to something we recognize, in community detection, the labels themselves are meaningless-there's no notion of a so-called "cluster 2". the field has circumvented issues arising from comparing systems differing in number of categories by introducing information-theoretic measures to compare partitions. cluster evaluation functions in community detection are, by and large, based on information-theoretic concepts. we define a gender system a's entropy as: how is this notion of entropy for partitions related to the entropy of a probability distribution? these are connected through maximumlikelihood estimation (mle). in our case, the maximum-likelihood estimate that an inanimate noun a is located in a given partition turns out to be the size of that partition divided by n , e.g. we have p mle (msc) = |amsc| /n. recall that the shannon entropy of a distribution p is defined as we have equality between eq. 1 and eq. 2 when we plug the definition of p mle into eq. 2, which is why eq. 1 is considered the entropy of a partition. ## experiments we apply each measure to the gender systems from our swadesh lists, then validate our results on northeuralex. we apply validation to ensure that they are picking up robust similarities as opposed to just reflecting properties of particular word lists. (see github.com/aryamccarthy/ gender-partitions.) we then reconstruct phylogenetic trees of the languages involved. the trees show high agreement with ground truth, compared to random baselines. ## related work there is a baffling dearth of work on quantifying similarity of gender systems. there is, however, ample work on characterizing intensional gender systems, i.e., sets of grammatical rules, that can be divided @xcite into sets of rules based on morphology @xcite @xcite and on phonology @xcite @xcite @xcite . intensional approaches, particularly those with typological leanings, contribute very fine grained research on particular pairwise similarities for particular languages and dialects. although we cannot survey these in detail here, we would love for our measures to contribute findings that can complement these approaches. relatedly, other recent works have investigated grammatical gender and other types of noun classification systems with information theoretic tools. for example, @xcite uses mutual information to quantify the strength of the relationships between declension class, grammatical gender, distributional semantics, and orthographic form respectively in several languages. @xcite , which is arguably closest to this work, measures the strength of semantic relationships between inanimate nouns and verbs or adjectives that takes those nouns as arguments, and that work can be seen as comparing the similarity of nouns clustered by their gender, with the same nouns clustered by the adjectives that modify them or the verbs that take them as arguments. although we adopt information theoretic measures, here there are two other major classes of cluster evaluation measures: set-matching measures, and pair-counting measures, which tally which pairs of items are in the same or different communities. one popular set-matching measure in information retrieval, purity @xcite , is asymmetric and biased by the size and number of communities @xcite . its symmetric form, the f-measure @xcite , has clear bounds but gives no indication of average-case performance. the adjusted rand index (ari; @xcite is the preeminent pair-counting measure. it is related to ami, adjusting the rand index in the same way that ami adjusts mi. ari also computes an expectation, which can be computed over the proper distribution @xcite is empirically better suited to large, balanced clusters. in our case of small and uneven clusters, ami should be preferred @xcite . we can only survey a representative handful of the numerous cluster evaluation measures in the limited space we have here. see @xcite for an outline of desiderata for comparing partitions, as well as a general class of appropriate measures, and for further motivation for ami using a different null model-languages have a fixed number of gender classes, so we select one over n items with k communities, rather than an arbitrary number of communities. ## conclusion we have presented a clean method for comparing grammatical gender systems across languages: by defining gender classes extensionally, we reduced the problem to cluster evaluation from community detection. we validate three metrics by recovering known phylogenic relationships in our languages, with measurable success. separate indo-european branches are no more similar than chance. we emphasize that our methods are not specifically tailored to gender systems. one could apply them more broadly other aspects of the lexicon, e.g. to indo-european verb classes, bantu noun classes, or diachronic time slices of a single language's gender system, data permitting. a related challenge is east and southeast asian numeral classifier systems, which associate nouns with classifiers based largely on the semantic properties of the nouns @xcite @xcite . they display more idiolectal variation, and often more than one classifier can accompany a given noun @xcite , unlike for gender (where this is rare). we note that we could further extend our measures to fuzzy partitions, which remain less explored in community detection, but are a promising avenue for future work.