On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs (2021)

Abstract: We use large-scale corpora in six different gendered languages, along with tools from NLP and information theory, to test whether there is a relationship between the grammatical genders of inanimate nouns and the adjectives used to describe those nouns. For all six languages, we find that there is a statistically significant relationship. We also find that there are statistically significant relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects. We defer deeper investigation of these relationships for future work.

## introduction in many languages, nouns possess grammatical genders. when a noun refers to an animate object, its grammatical gender typically reflects the biological sex or gender identity of that object @xcite @xcite . for example, in german, the word for a boss is grammatically feminine when it refers to a woman, but grammatically masculine when it refers to a man-chefin and chef, respectively. but inanimate nouns (i.e., nouns that refer to inanimate objects) also possess grammatical genders. any german speaker will tell you that the word for a bridge, brücke, is grammatically feminine, even though bridges have neither biological sexes nor gender identities. historically, the grammatical genders of inanimate nouns have been considered more idiosyncratic and * equal contribution in this scientific whirlwind. although laboratory experiments are highly informative, they typically involve small sample sizes. in this paper, we therefore use large-scale corpora and tools from nlp and information theory to test whether there is a relationship between the grammatical genders of inanimate nouns and the adjectives used to describe those nouns. specifically, we calculate the mutual information (mi)-a measure of the mutual statistical dependence between two random variablesbetween the grammatical genders of inanimate nouns and the adjectives that describe them (i.e., share a dependency arc labeled amod) using largescale corpora in six different gendered languages (specifically, german, italian, polish, portuguese, russian, and spanish). for all six languages, we find that the mi is statistically significant, meaning that there is a relationship. we also test whether there are relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects. for all six languages, we find that there are statistically significant relationships for the verbs that take those nouns as direct objects and as subjects. for five of the six languages, we also find that there is a statistically significant relationship for the verbs that take those nouns as indirect objects, but because of the small number of noun-verb pairs involved, we caution against reading too much into this finding. to contextualize our findings, we test whether there are statistically significant relationships between the grammatical genders of inanimate nouns and the cases and numbers of these nouns. a priori, we do not expect to find statistically significant relationships, so these tests can be viewed as a baseline of sorts. as expected, for each of the six languages, there are no statistically significant relationships. to provide further context, we also repeat all tests for animate nouns-a ''skyline'' of sorts-finding that for all six languages there is a statistically significant relationship between the grammatical genders of animate nouns and the adjectives used to describe those nouns. we also find that there are statistically significant relationships between the grammatical genders of animate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects. all of these relationships have effect sizes (operationalized as normalized mi values) that are larger than the effect sizes for inanimate nouns. we emphasize that the practical significance and implications of our findings require deeper investigation. most importantly, we do not investigate the characteristics of the relationships that we find. this means that we do not know whether these relationships are characterized by gender stereotypes, as argued by some cognitive scientists. we also do not engage with the ways that historical and sociopolitical factors affect the grammatical genders possessed by either animate or inanimate nouns @xcite . ## data preparation we use the may 2018 dump of wikipedia to create a corpus for each of the six different gendered languages (i.e., german, italian, polish, portuguese, russian, and spanish). although wikipedia is not the most representative data source, this choice yields language-specific corpora that are roughly parallel-that is, they refer to the same objects, but are not direct translations of each other (which could lead to artificial word choices). we use udpipe to tokenize each corpus @xcite . we dependency parse the corpus for each language using a language-specific dependency parser @xcite , trained using universal dependencies treebanks @xcite . an example dependency tree is shown in figure 1 . we then extract all noun-adjective pairs (dependency arcs labeled amod) and noun-verb pairs from each of the six corpora; for verbs, we extract three types of pairs, reflecting the fact that nouns can be direct objects (dependency arcs labeled dobj), indirect objects (dependency arcs labeled iobj), or subjects (dependency arcs labeled nsubj) of verbs. we discard all pairs that contain a noun that is not present in wordnet (princeton university, 2010). we label the remaining nouns as ''animate'' or ''inanimate'' according to wordnet. next, we lemmatize all words (i.e., nouns, adjectives, and verbs). each word is factored into a set of lexical features consisting of a lemma, or canonical morphological form, and a bundle of three morphological features corresponding to the grammatical gender, number, and case of that word. for example, the german word for a fork, gabel, is grammatically feminine, singular, and genitive. for nouns, we discard the lemmas themselves and retain only the morphological features; for adjectives and verbs, we retain the lemmas and discard the morphological features. for adjectives and verbs, lemmatizing is especially important because it ensures that our tests do not simply reflect the presence of gender agreement, as we describe in section 2.1. however, this means that if the lemmatizer fails, then our tests may simply reflect gender agreement despite our best efforts. to guard against this, we use a state-of-the-art lemmatizer @xcite , trained for each language using universal dependencies treebanks @xcite . we expect that when the lemmatizer fails, the resulting lemmata will be low frequency. we try to exclude lemmatization failures from our calculations by discarding low-frequency lemmata. for each language, we rank the adjective lemmata by their token counts and retain only the highestranked lemmata (in rank order) that account for 90% of the adjective tokens; we then discard all noun-adjective pairs that do not contain one of these lemmata. we repeat the same process for verbs. finally, to ensure that our tests reflect the most salient relationships, we also discard low-frequency inanimate nouns and, separately, low-frequency animate nouns using the same process. we provide counts of the remaining noun-adjective and noun-verb pairs in ## methodology for each language ℓ ∈ {de, it, pl, pt, ru, es}, we define v ℓ adj to be the set of adjective lemmata represented in the noun-adjective pairs retained for that language as defined above. we similarly define v ℓ verb to be the set of verb lemmata represented in the noun-verb pairs retained for that language, as described above. we then define verb to be the sets of verbs that take the nouns as direct objects, as indirect objects, and as subjects, respectively. we also define g ℓ to be the set of grammatical genders for that language (e.g., g @xmath0, fem}), c ℓ to be the set of cases (e.g., c @xmath1, acc, gen, dat}), and n ℓ to be the set of numbers (e.g., n @xmath2, sg}). finally, we define fourteen random variables: to test whether there is a relationship between the grammatical genders of inanimate nouns and the adjectives used to describe those nouns for language ℓ, we calculate the mi (mutual information)-a measure of the mutual statistical dependence between two random variablesbetween g ℓ i and to test for statistical significance, we perform a permutation test. specifically, we permute the grammatical genders of the inanimate nouns 10,000 times and, for each permutation, recalculate the mi between g ℓ i and a ℓ i using the permuted genders. we obtain a p-value by calculating the percentage of permutations that have a higher mi than the mi obtained using the non-permuted genders; if the p-value is less than 0.05, then we treat the relationship between to test whether there are relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects, we calculate mi(g ℓ i , d ℓ i ), mi(g ℓ i , i ℓ i ), and mi(g ℓ , s ℓ i ). again, all probabilities are calculated with respect to inanimate nouns only, and we perform permutation tests to test for statistical significance. we also calculate six nmi variants for each of the three pairs of random variables, using normalizers that are analogous to those in eq. ( 2 ) through eq. ( 7 ). as a baseline, we test whether there are relationships between the grammatical genders of inanimate nouns and the cases and numbers of those nouns-that is, we calculate mi(g ℓ i , c ℓ i ) and mi (g ℓ i , n ℓ i ) using probabilities that are calculated with respect to inanimate nouns only. again, we perform permutation tests (but we do not expect that there will be statistically significant relationships), and we calculate six nmi variants for each pair of random variables using normalizers that are analogous to those in eq. ( 2 ) through eq. ( 7 ). ## results in the first row of figure 3 is analogous to figure 2 , in that each subplot contains six variants of nmi(g ℓ a , a ℓ a ), nmi(g ℓ a , d ℓ a ), and nmi(g ℓ a , s ℓ a ), calculated using normalizers that are analogous to those in eq. ( 2 ) through eq. ( 7 ), for a single language ℓ ∈ {de, it, pl, pt, ru, es}. (as with inanimate nouns, we omit nmi(g ℓ a , i ℓ a ) from each plot because of the small number of noun-verb pairs involved.) for ℓ ∈ {de, it, pl, pt, es}, nmi( and nmi(g es i , d es i ) and nmi(g es i , s es i ) are roughly comparable-again, all regardless of the normalizer. meanwhile, nmi(g ru i , a ru i ) is larger than nmi(g ru i , d ru i ) which is larger than nmi(g ru i , s ru i ) for the normalizers in eq. (2) and eq. (3), while nmi(g ru i , a ru i ) and nmi(g ru i , d ru i ) are roughly comparable and larger than nmi(g ru i , s ru i ) for the other five normalizers. finally, each subplot in figure 4 contains nmi(g ℓ i , a ℓ i ) and nmi(g ℓ a , a ℓ a ), calculated using a single normalizer, for each for each language ℓ ∈ {de, it, pl, pt, ru, es}. each subplot in figure 5 than the nmi values for inanimate nouns. the only exception is polish, where nmi(g pl i , a pl i ) is larger than nmi(g pl a , a pl a ), regardless of the normalizer. ## discussion we find evidence for the presence of a statistically significant relationship between the grammatical genders of inanimate nouns and the adjectives used to describe those nouns for six different gendered languages (specifically, german, italian, polish, portuguese, russian, and spanish). we also find evidence for the presence of statistically significant relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects. however, we caution against reading too much into the relationship for the verbs that take those nouns as indirect objects because of the small number of noun-verb pairs involved. the effect sizes (operationalized as nmi values) for all of these relationships are smaller than the effect sizes for animate nouns. as expected, we do not find any statistically significant relationships for either case or number. we emphasize that our findings complement, rather than supersede, laboratory experiments, such as that of @xcite . we use large-scale corpora and tools from nlp and information theory to test for the presence of even relatively weak relationships across multiple different gendered languages-and, indeed, the relationships that we find have effect sizes (operationalized as nmi values) that are small. in contrast, laboratory experiments typically focus on much stronger relationships by tightly controlling experimental conditions and measuring speakers' immediate, real-time speech production. moreover, although we find statistically significant relationships, we do not investigate the characteristics of these relationships. this means that we do not know whether they are characterized by gender stereotypes, as argued by some cognitive scientists, including @xcite . we also do not know whether the relationships that we find are causal in nature. because mi is symmetric, our findings say nothing about whether the grammatical genders of inanimate nouns cause writers to choose particular adjectives or verbs. we defer deeper investigation of this for future work. we note that each of our tests can be viewed as a comparison of the similarity of two clusterings of a set of items-specifically, a ''clustering'' of nouns into grammatical genders and a ''clustering'' of the same nouns into, for example, adjective lemmata. although (normalized) mi is a standard measure for comparing clusterings, it is not without limitations (see, e.g., newman et al.