Characterizing non-binary F rench: A first step towards debiasing gender inference (2025)

Abstract: This paper addresses a bias of gender inference systems: their binary nature. Based on the observation that, for French, systems based on pattern-matching of grammatical gender markers in “I am” expressions perform better than machine-learning approaches (Ciot et al. 2013), we examine the use of grammatical gender by non-binary individuals. We describe the construction of a corpus of texts produced by non-binary authors on Reddit, (formely) Twitter and three forums. Our linguistic analysis shows three main patterns of use: authors who use non-binary markers, authors who consistently use one grammatical gender, and authors who use both feminine and masculine markers. Using this knowledge, we make proposals for the improvements of existing gender inference systems based on grammatical gender.

## introduction gender inference constitutes an important domain of nlp research and applications. being able to identify a user's social gender can have many benefits, ranging from commercial (eg. capitalizing on gender-based consumption patterns, @xcite , to civic (e.g. ensuring that people of all genders have equal access to public services and platforms @xcite , and scientific (e.g. properly understanding how gender influences computer-mediated communication, @xcite @xcite . despite researchers and public/private actors becoming more conscious of the complexities of social gender, and a stated emerging desire to include users whose gender does not align with the male/female binary @xcite , in nlp, gender inference is almost always based on a binary conception of social gender. this situation is the result of many factors, among them the fact that many gender inference papers in nlp still adopt an essentialist view of gender (i.e. one in which linguistic patterns are directly attributed to biological aspects of sex/gender), and, as observed by @xcite , training and testing datasets with non-binary (or other) users are lacking. in this way, individuals whose genders do not correspond to male or female are made invisible by current nlp gender inference systems. we consider that this invisibilization constitutes bias against non-binary users (bias statement). building on one of the principles of data feminism ("what gets counted, counts" (d'ignazio and @xcite , we argue that this bias creates both representational and allocational harms (harm statement). the fact that most (if not all) systems fail to recognize the existence of individuals whose gender exists outside the male/female binary is, by definition, a representational harm, (see @xcite , and this misrepresentation of the gender distribution of online spaces hinders research in social science devoted to studying them @xcite . the extent to which binary gender inference systems create allocational harms will depend on their applications: systems that use gender inference to propose beneficial services or products will exclude non-binary users, which could adversely impact their material (social, political and economic) conditions. as discussed above, if we want to debinarize, and therefore debias, gender inference and other nlp systems, a crucial first step is to create datasets composed of contributions from people with a varied set of gender identities. however, creating these datasets is challenging for several reasons: because "non-binary" is a label that encompasses diverse gender identities, because current labeling practices in gender inference datasets are not adequate, and because, in the age of llms, large datasets are favored -and non-binary individuals make up a small (although possibly growing) portion of the population @xcite . another possible reason for the lack of nonbinary representation in nlp datasets may be the focus on the english language (o' @xcite . deep learning techniques, which have been increasingly used in the field of gender inference since 2017, need large amount of data to be accurate. however, in french, and for other languages that have grammatical gender, small datasets and simple methods may be enough to create reliable gender inference systems (i.e., more reliable than most gender inference machine learning/deep learning techniques). this is because in french, speakers have to gender themselves when talking/writing about themselves, providing information about their gender identity. using a deterministic pattern-matching technique based on grammatical gender in "i am" type statements, @xcite reaches a higher accuracy in gender inference than "traditional" machine learning techniques. this type of system may be also used to infer gender beyond the binary, as non-binary speakers may use gender-neutral grammatical markers, which can combine the masculine and the feminine (français.e) or use neomorphemes (françaiz). it implies a shift from machine learning gender inference systems based on "sociolinguistic features" or on less interpretable features (character ngrams, bleached features), to systems based on grammatical gender. these systems rely on linguistic knowledge of grammatical gender. however, to this date, we do not know much about the way non-binary individuals use grammatical gender, because most linguistic studies rely on survey and questionnaire data, which may not reflect actual use of language (for example @xcite . thus, we present the first corpus study of the ways non-binary individuals use of grammatical gender when writing about themselves, in a corpus of computer-mediated communication. we automatically extract grammatical gender in je+être ("i+to be") expressions expressions and classify gender markers into four categories: feminine, masculine, non-binary (use of neomorphemes or combined use of a masculine and a feminine marker), and neutralization (adjectives or nouns that bear no grammatical gender information). we then examine inter-platform and inter-author variation. results show that there is considerable variation between individuals, some sticking to one grammatical gender, and others switching between grammatical genders. we also consider the question of whether there are non-binary markers; one of the candidates is the period (français.e). however, we find that the use of the period is largely limited to a single platform; we conclude that constructing a gendered inference system based on the period would lead to misgendering. our contributions are: ## dataset to create the corpus, we used five platforms: twitter (scraped in 2022 and 2023, before it became x), reddit, and three online forums: betolerant.fr (a forum dedicated to queer identities), forum.assocontact.org, and forums.madmoizelle.com 1 . the data collection approach was different for each platform, depending on their structure and affordances. for example, for reddit, we used a large (preexisting) corpus containing 16,480,376 comments from 21 subreddits; for forums.madmoizelle.com, we extracted data from a single discussion thread dedicated to non-binary identities. ## conclusions we believe that gender inference can serve as a valuable methodological tool in scientific research, particularly when used to shed light on structural inequalities. however, this task should be conducted ethically, with a clear understanding that gender is not binary, fixed, or always externally discernible from texts. we emphasized the need for transparency in the labeling process and for inclusion of gender identities outside the binary. in this paper, we tried to determine whether there is a linguistic signature to being non-binary in written french, in order to assess the possibility of creating a pattern-matching nlp system. our corpus study of grammatical gender in je+être expressions shows that there is not a single distinctive signature that would allow us to infer non-binary gender, but, instead, multiple patterns. this diversity of patterns could be due to (among other things) the instability of non-binary forms, which have emerged in the 21 st century; the fact that the non-binary label encompasses diverse and fluid gender identities; the contexts in which internet users interact; but also, their attitude towards gender and language. one finding was that some non-binary individuals use one grammatical gender fairly or very consistently; thus, there is no way to differentiate them from women and men, based on grammatical gender. other authors use grammatical gender in ways that seem distinct to what women and men do. the first possible linguistic signature we have uncovered is a high amount of variation in the use of feminine and masculine markers; such variation seems much more frequent among non-binary individuals than it is for women and men. adding a measure of entropy to a pattern-matching system would be a way to identify some non-binary authors, but not all. the most distinctive (i.e., distinct from what women and men usually do with grammatical gender in french) is the use of nonbinary gender markers: it seems safe to say that the vast majority of people who use them in "i am" statements are non-binary. there could be some exceptions, however, such as people who wish to conceal their gender identity online, or people using non-binary markers in reported speech or ironically/mockingly. our study provides valuable insight into what these markers look like; creating a gender inference system that extracts these markers in je+être statements using regular expressions would be fairly simple, and it could help debinarize gender inference in french corpora. however, it would be far from an ideal solution, as non-binary gender markers were used by a minority of authors in our corpus. this type of system would thus misgender most non-binary individuals, by classifying them as women or men. furthermore, the productivity of this method would depend greatly on the type of corpus used; texts produced in queer/feminist spaces (such as the madmoizelle forum) seem the most likely to feature this type of grammatical markers. we contend that, using this knowledge, it may be possible to create a system that infers gender beyond the binary in french datasets: it would extract grammatical gender in je+être expressions with a lexicon, using regular expressions to extract nonbinary markers, and integrate measures of diversity. to limit misgendering, we propose the following steps: combining extraction of grammatical gender with extraction of gender-identity statements; manual inspection of samples; and creation of an "unknown gender" category in cases of ambiguity. in our view, the most ethical way to produce a non-binary gender inference system (or any gender inference system) is to emphasize robustness rather than maximizing recall.