Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021)

Abstract: Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.

## introduction as language models are more prolifically used in language processing applications, ensuring a higher degree of fairness in associations made by their learned representations and intervening in any biased decisions they make has become increasingly important. recent work analyzes, quantifies, and mitigates language model biases such as gender, race or religion-related stereotypes in static word embeddings (glove @xcite ) and contextual (e.g., bert @xcite ) representations @xcite @xcite . a bulk of social bias studies on language models have focused on binary gender and the stereotypes associated with masculine and feminine attributes @xcite @xcite . additionally, models often rely on gendered information for decision making, such as in named entity recognition, coreference resolution, and machine translation @xcite @xcite , but the purview of gender in these tasks and associated measures of performance focus on binary gender. while discussing binary gender bias and improving model performance are important, it is important to reshape our understanding of gender in language technologies in a more accurate, inclusive, non-binary manner. current language models can perpetrate harms such as the cyclical erasure of non-binary gender identities @xcite @xcite @xcite . these harms are driven by model and dataset biases due to tainted examples, limited features, and sample size disparities @xcite @xcite , which are consequences of the non-recognition and a lack of understanding of non-binary genders in society @xcite . some recent works attempt to mitigate these harms by building task-specific datasets that are not restricted to binary gender and building metrics that on extension, could potentially measure biases against all genders (cao and @xcite . while such works that intentionally inject real-world or artificially-created data of non-binary people into binary-gendered datasets are well-intentioned, they could benefit from a broader perspective of harms as perceived by non-binary persons to avoid mischaracterizing non-binary genders as a single gender @xcite or perpetuating biases through nonintersectional training examples, i.e. examples that do not capture the interconnected nature of social identities @xcite . in this paper, we conduct a detailed investigation into the representational and allocational harms @xcite related to the treatment of gender with binary predilections in english language technologies. we do so by explaining the complexity of gender and language around it, and surveying non-binary persons with some familiarity with ai on potential harms in common nlp tasks. while the challenges associated with limited or tainted data are loosely hypothesized, they are not well understood. we study the extent of these data challenges and detail how they manifest in the resultant language representations and downstream tasks. we examine both static embeddings (glove) and contextual representations (bert) with respect to the quality of representations (section 4.2) of non-binaryassociated words and pronouns.we highlight how the disparity in representations cyclically propagates the biases of underrepresentation and misrepresentation and can lead to the active misgendering and erasure of non-binary persons in language technologies. we first discuss the complex concepts of gender and bias and their expression in english language. ## harms utilizing and perpetuating the binary construction of gender in english in language technologies can have adverse impacts. we focus on specific tasks within language processing and associated applications in human-centered domains where harms can be perpetrated, motivated by their frequent mention in a survey we conduct (section 3.1). the primary harms we discuss are misgendering and erasure. misgendering: misgendering is the act of accidentally or intentionally addressing someone (oneself or others) using a gendered term that does not match their gender identity. misgendering persons and the associated harms have been studied in contexts of computer vision @xcite and humancomputer interaction @xcite , which highlight its adverse impact on the mental health of non-binary individuals. language applications and their creators can also perpetrate misgendering. for instance, language applications that operationally ask non-binary users to choose between male and f emale as input force non-binary users to misgender themselves @xcite . furthermore, language models which do not explicitly collect gender information are capable of both accidental and intentional misgendering. specifically, language models accidentally misgen-der non-binary persons when there is insufficient information to disambiguate the gender of an individual, and so they default to binary pronouns and binary-gendered terms, potentially based on stereotypes. however, as shown in section 4.2, language models can also misgender non-binary individuals even when their pronouns are provided. erasure: in one sense, erasure is the accidental or intentional invalidation or obscuring of nonbinary gender identities. for example, the language technology genderify, which purportedly "identif[ied] someone's [binary] gender based on their name, email address or username" erased nonbinary people by reductively distributing individuals into binary "gender bins" by their name, based on the assumption that they were cisgender @xcite @xcite . another sense of erasure is in how stereotypes about nonbinary communities are portrayed and propagated (see appendix language applications are capable of automating erasure, in a cyclical fashion @xcite . we posit the cycle of non-binary erasure in text, in which: (i) language applications, trained on large, binary-gendered corpora, reflect the misgendering and erasure of non-binary communities in real life @xcite (ii) this reflection is viewed as a "source of truth and scientific knowledge" @xcite (iii) consequently, authors buy into these harmful ideas and other language models encode them, leading them to stereotypically portray non-binary characters in their works or not include them at all, and @xcite this further amplifies non-binary erasure, and the cycle continues. ## data and technical challenges as a consequence of historical discrimination and erasure in society, narratives of non-binary persons are either largely missing from recorded text or have negative connotations. language technologies also reflect and exacerbate these biases and harms, as discussed in section 3.1, due to tainted examples, limited features, and sample size disparities. these challenges are not well understood. we discuss the different fundamental problems that need to be acknowledged and addressed to strategize and mitigate the cyclical erasure and misgendering of persons as a first step towards building language models that are more inclusive. ## discussion and conclusion this work documents and demonstrates specific challenges towards making current language modeling techniques inclusive of all genders and reducing the extent of discrimination, misgendering, and cyclical erasure it can perpetrate. in particular, our survey identifies numerous representational and allocational harms, as voiced from individuals affected, and we demonstrate and quantify several cases where the roots of these concerns arise in popular language models. some efforts in the nlp community have worked towards countering problems in task-specific data sets with skewed gender tags, due to underrepresentation of non-binary genders. notably, cao and daumé @xcite and cao and daumé iii (2021) introduce a gender-inclusive dataset gicoref for coreference resolution and @xcite propose rewriting text containing gendered pronouns with they as the substituted pronoun to obtain more gender-neutral text. the challenges still remain that (i) not all neopronouns will have sufficient data in real-world text, and (ii) considering non-binary genders as a monolithic third category (i.e. male, female, and gender-neutral) is counter-productive and perceived as harmful (section 3.1). while these efforts are a start in moving away from binary gender, it is questionable if gender should be defined as discrete quantities in language modeling, when in reality, it is of a fluid nature. furthermore, models currently do not account for the mutability of gender and the language around it, and even if they did, they would likely assume there exist well-defined points at which individuals and words transition, which too is detrimental (as documented in our survey, see section 3.1). representing gender is as complex as the concept of gender itself. bucketing gender in immutable, discrete units and trying to represent each, would inevitably result in marginalization of sections of the population to varied extents. as our survey catalogs how pronounced the harms of being consistently misgendered and diminished are, we encourage future work to carefully examine how (and if ) to define and model gender in language representations and tasks. this work sets the interdisciplinary stage for rethinking and addressing challenges with inclusively modeling gender in language representations and tasks. any viable solution cannot simply be a quick fix or patch, but must rely on a bottom-up approach involving affected persons system-wide, such as in annotation and human-in-the-loop mechanisms. simultaneously, research into monitoring language technologies over time to detect harms against nonbinary individuals is critical. it is further paramount to transparently communicate the performance of language technologies for non-binary persons and possible harms. in the case of harm, non-binary individuals must be able to obtain valid recourse to receive a more favorable outcome, as well as have the opportunity to provide feedback on the model's output and have a human intervene.