The Causal Influence of Grammatical Gender on Distributional Semantics (2024)

Abstract: How much meaning influences gender assignment across languages is an active area of research in linguistics and cognitive science. We can view current approaches as aiming to determine where gender assignment falls on a spectrum, from being fully arbitrarily determined to being largely semantically determined. For the latter case, there is a formulation of the neo-Whorfian hypothesis, which claims that even inanimate noun gender influences how people conceive of and talk about objects (using the choice of adjective used to modify inanimate nouns as a proxy for meaning). We offer a novel, causal graphical model that jointly represents the interactions between a noun’s grammatical gender, its meaning, and adjective choice. In accordance with past results, we find a significant relationship between the gender of nouns and the adjectives that modify them. However, when we control for the meaning of the noun, the relationship between grammatical gender and adjective choice is near zero and insignificant.

## introduction approximately half of the world's languages have grammatical gender @xcite , a grammatical phenomenon that groups nouns together into classes that share morphosyntactic properties @xcite @xcite . among languages that have gender, there is variation in the number of gender classes; for example, some languages have only two classes, e.g., all danish nouns are classed as either common or neuter, whereas others have significantly more, e.g., nigerian fula has around 20, depending on the variety @xcite @xcite . languages also vary with respect to how much gender assignment, i.e., how nouns are sorted into particular genders, is related to the form and the meaning of the noun @xcite @xcite @xcite . some languages group nouns into gender classes that are highly predictable from phonological @xcite @xcite or morphological @xcite @xcite information, while others, such as the dagestanian languages godoberi and bagwalal, seem to be predictable from meaning @xcite @xcite -although, even for most of the strictly semantic systems, there are exceptions. despite this variation, gender assignment is rarely, if ever, wholly predictable from meaning alone. in many languages, there is a semantic core of nouns that are conceptually coherent @xcite @xcite and a surround that is somewhat less semantically coherent. axes along which genders are conceptually coherent often include semantic properties of animate nouns, with inanimate nouns appearing in the surround. for example, in spanish, despite the fact that the nouns table (mesa in spanish) and woman (mujer in spanish) appear in the same gender (i.e., feminine), it is hard to imagine what meaning they share. indeed, some linguists posit that gender assignment for inanimate nouns is effectively arbitrary @xcite @xcite . and, to the extent that gender assignment is not fully arbitrary for inanimate nouns @xcite , many researchers argue there is no compelling evidence showing grammatical gender affects how we conceptualize objects @xcite or the distributional properties of language @xcite . however, not all researchers agree that nonarbitrariness in gender assignment, to the extent it exists, should be assumed to have no bearing on language production. @xcite famously argued for a causal relationship between the gender assigned to inanimate nouns and their usage, in a view colloquially known as the neo-whorfian hypothesis after benjamin whorf @xcite . proponents of this view have studied human associations, under the assumption that people's perceptions of the genders of objects are strongly influenced by the grammatical genders these objects are assigned in their native language @xcite . one manifestation of this perception is the choice of adjectives used to describe nouns @xcite . while this is an intriguing possibility, there are additional lexical properties of nouns that may act as confounders and, thus, finding statistical evidence for the causal effect of grammatical gender on adjective choice requires great care. to facilitate a cleaner way to reason about the causal influence grammatical gender may have on adjective usage, we introduce a causal graphical model to represent the interactions between an inanimate noun's grammatical gender, its meaning, and the choice of its descriptors. this causal framework enables intervening on the values of specific factors to isolate the effects between various properties of languages. our model explains the distribution of adjectives that modify a noun, conditioned on both a representation of the noun's meaning and the gender of the noun itself. upon estimation of the parameters of the causal graphical model, we test the neo-whorfian hypothesis beyond the anecdotal level. first, we validate our model by comparing it to the method presented in prior work without any causal intervention. second, we employ our model with a causal intervention on the noun meaning to test the neo-whorfian hypothesis. that is, we ask a counterfactual question: had nouns been lexicalized with different grammatical genders but retained their same meanings, would the distribution of adjectives that speakers use to modify them have been different? we employ our model on five languages that exhibit grammatical gender: four indo-european languages (german, polish, portuguese, and spanish) and one language from the afro-asiatic language family (hebrew). we find that, at least in wikipedia data, a noun's grammatical gender is indeed correlated with the choice of its descriptors. however, when controlling for a confounder, noun meaning, we present empirical evidence that noun gender has no significant effect on adjective usage. ## a primer on grammatical gender in many languages with grammatical gender, adjectives, demonstratives, determiners, and other categories agree with the noun in gender, i.e., they will systematically change in form to indicate the grammatical gender of the noun they modify. observe the following sentence, a small dog sleeps under the tree., translated into two languages that exhibit grammatical gender (german and polish): a. ein kleiner hund schläft unter dem ## a causal graphical model the technical contribution of this work is a novel causal graphical model that jointly represents the relationship between the grammatical gender of a noun, its meaning, and descriptors. this model is depicted in figure 1 . if properly estimated, the model should enable us to measure the causal effect of grammatical gender on adjective choice in language. we first develop the necessary notation. notation we follow several font and coloring conventions to make our notation easier to digest. all base sets will be uppercase and in calligraphic font, e.g., x . elements of x will be lowercase and italicized, e.g., @xmath0. subsets (including submultisets) will be uppercase and unitalicized, e.g., x ⊂ x . random variables that draw their values from x will be uppercase and italicized, e.g., p(@xmath1. we will use three colors. those objects that relate to nouns will be in blue, those objects that relate to adjectives will be in purple, and those objects that relate to gender will be in green. ## methodology we now outline the methodology of our study, starting with parameter estimation of the model in §5.1 and plug-in estimation of mi(a; g) in §5.2. we conduct two experiments: first, for a point of comparison, we replicate @xcite study to estimate mi(a; g) for each of the five languages ( §5.3); and second, we produce a causal analog of @xcite ( §5.4). using the notation of §3, in the second experiment, we estimate mi do (a; g) for each of the five languages. finally, in §5.5, we discuss our permutation testing methodology. ## results first, we validate our model by comparing the model-based estimation of mi(a; g) to the method presented in @xcite , the plug-in estimation of mi(a; g). then, we employ our causal graphical model to investigate whether there is evidence for the neo-whorfian claim that the grammatical gender of a noun influences the adjective chosen to modify this noun, even when we control for the meaning of the noun. we first validate our model by comparing its results to @xcite plug-in estimate of mi(a; g). if the results of both of these estimates are comparable, we have evidence that our model indeed captures the relation between grammatical gender and adjective choice. we present the results in figure 2 . we observe a substantial relationship between grammatical gender and adjective usage based on the plug-in and model-based mi(a; g) estimates replicating the results of @xcite . the estimates of the model-based mi(a; g) computed using both word2vec and wordnet representations, and the plug-in mi(a; g) lie between 0.2 and 0.5, with the estimates of the model-based approach being consistently higher (with the ex-ception of german) than the estimates of the plug-in mi(a; g). thus, the non-zero estimates of the model-based mi(a; g) indicate that some relationship exists between a noun's grammatical gender and adjective usage. given the above result, we are interested in whether the strength of this relationship is mitigated when controlling for the meaning of a noun. we present the estimates of the model-based mi do (a; g) in for completeness, we test for the presence of a difference between the size of the mi do (a; g) of our model and a model trained on randomly perturbed gender labels based on a subset of adjectives. we reject the null hypothesis that the distributions are exactly the same for all languages and representations' settings. ## discussion evidence against the neo-whorfian hypothesis. we find that the interaction between the grammatical gender of inanimate nouns and the adjectives used to describe those nouns all but disappears when controlling for the meaning of those nouns, for all five analyzed gendered languages. while the order of magnitude of mi do (a; g) measured with our model is significantly different from that of a model trained on random gender labels, it remains minuscule in absolute terms. this minor difference points towards the absence of a meaningful causal relationship between a noun's gender and its modifiers in the languages studied. thus, we provide an additional piece of evidence against the neo-whorfian hypothesis in noun-adjective patterns. a possible weakening of the neo-whorfian hypothesis. although the size of the overall effect is small, it is possible that the effect of gender on adjective choice is stronger for some words than others. future work could explore whether there is evidence of a noticeable effect of gender on adjective choice for a more restricted set of inanimate nouns, e.g., referring to artifacts or body parts. such evidence could perhaps support a weakened version of the neo-whorfian hypothesis. comparing results between word2vec and wordnet. our results hold for both of the word representation conditions, word2vec, and word-net. notably, in comparing the two, we find that using wordnet representations consistently results in a lower mi do (a; g) than word2vec for all languages analyzed in this study. one possible explanation for this difference is that, despite our efforts to make non-contextual word2vec representations, these word2vec representations may still encode some signal regarding gender from the remaining context (such as verb choice or adjacent gendered pronouns in the corpora). if these word2vec representations contain unwanted context-based gender information in addition to the noun meaning, it could result in overestimating mi do (a; g). furthermore, since wordnet representations are created independently from any context within a corpus, they should not contain any signal related to grammatical gender, which may therefore be reflected in the consistently lower mi do (a; g). design choices and limitations. we note several choices in the experimental setup which may influence this analysis. first, while we experiment with northeuralex, which furnishes us with a list of inanimate nouns, the dataset excludes rarer nouns for which an effect might be observed. 9 second, while non-contextual word representations are the current de facto proxy for lexical semantics, they remain a proxy and are fundamentally limited. furthermore, in our effort to estimate word2vec representations for noun meaning without encoding gender-based context, we chose to remove some words in the context but not others. specifically, while we remove adjectives which may carry signals of gender from the training corpora, we do not remove other parts of speech (e.g., verbs) under the reasoning that removing them may damage the training corpora too much for word2vec to effectively learn noun meanings. 10 future work can also explore improved representation methods for noun meaning. for example, @xcite find that creating non-contextual word representations using a combination of word2vec, wordnet, and concept dictionaries can yield a better representation of meaning (i.e., achieving state-of-the-art correlation with the human-annotated similarity scores). third, the corpus choice (and subsequently the noun-adjective pairs on which we conduct our analysis) may factor into the results. it is possible that when applied to other corpora (e.g., more colloquial ones like reddit), this method may yield different results. fourth, the choice of languages analyzed further limits this study to languages with up to three gender classes. future work can investigate languages with more complex gender systems. finally, our modeling approach assumes that the gender of a noun is influenced solely by its meaning. however, prior work has indicated that there are other factors that influence the grammatical gender of nouns such as their phonology and morphology @xcite . therefore, future work should investigate more complex graphical models in order to account for other confounding factors. ## conclusion in this paper, we introduce a causal graphical model that jointly models the interactions between a noun's grammatical gender, its meaning, and adjective choice. we employ our model on five languages that exhibit grammatical gender to investigate the influence of nouns' gender on the adjectives chosen to describe those nouns. replicating the findings of @xcite , we find a substantial correlation between grammatical gender and adjective choice. however, taking advantage of our causal perspective, we show that when controlling for a noun's meaning, the effect of gender on adjective choice is marginal. thus, we provide further evidence against the neo-whorfian hypothesis.