Measuring Gender Bias in Natural Language Processing: Incorporating Gender-Neutral Linguistic Forms for Non-Binary Gender Identities in Abusive Speech Detection (2023)

Abstract: Predictions from machine learning models can reflect bias in the data on which they are trained. Gender bias has been shown to be prevalent in natural language processing models. The research into identifying and mitigating gender bias in these models predominantly considers gender as binary, male and female, neglecting the fluidity and continuity of gender as a variable. In this paper, we present an approach to evaluate gender bias in a prediction task, which recognises the non-binary nature of gender. We gender-neutralise a random subset of existing real-world hate speech data. We extend the existing template approach for measuring gender bias to include test examples that are gender-neutral. Measuring the bias across a selection of hate speech datasets we show that the bias for the gender-neutral data is closer to that seen for test instances that identify as male than those that identify as female.

## introduction natural language processing (nlp) models and systems are developed by using text content created by humans and they may incorporate biases that exist in the data. these biases can then be reflected in the results produced by these models and systems when they are used in downstream applications @xcite . additionally, word embeddings, which are representations of words and sentences generated from large amounts of natural language text, may also exhibit and even magnify certain features of the data, such as gender stereotypes @xcite @xcite . an issue with existing research is that it considers gender as binary neglecting the fluidity and continuity of gender as a variable @xcite . many of the data resources in nlp currently are inadequate for identifying gender bias as they often have a significant underrepresentation of female or non-binary instances. @xcite . there is a need to incorporate gender-neutral linguistic forms in datasets and algorithms to recognise the non-binary nature of gender. this impacts on algorithms too, such as language models which learn meaningless unstable representations for non-binary associated pronouns and terms @xcite . in this paper, we present an approach to measure gender bias in a downstream task to identify abusive or hate speech that considers male, female, and gender-neutral gender identities. due to the lack of datasets that include gender-neutral linguistic forms, we adjust existing real-world datasets by gender-neutralising a random subset of instances. a challenge with measuring gender bias in natural language training data is the lack of gender identification in the data. one solution to this is to generate synthetic test data with a known gender identity using a template approach known as gbets @xcite . our approach has extended existing binary template definitions to include identity terms that reflect gender neutrality. we use a suite of measures presented by @xcite which are threshold agnostic to measure gender bias. the downstream task we address is abusive and hate speech which involves language that is intended to be harmful and specifically targets individuals based on their affiliation with a particular group, such as their race, gender, sexuality, religion, or other protected characteristics @xcite . hate speech detection models exhibit gender biases towards certain identity terms due to factors such as an uneven distribution of identity terms in hate speech datasets and the excessive use of certain identity terms in hate speech sentences. for instance, some terms, like "women" and "feminism," can be frequently associated with sexist comments in benchmark datasets. these factors can lead to overfitting of the original hate speech detection model, which in turn may result in incorrect generalisations, such as linking the word "women" with a "hateful" label @xcite . we evaluate gender bias on three real-world hate speech datasets that have been adjusted to include data instances with a gender-neutral identity. the findings show that the bias for gender-neutral data is closer to that seen for data that is identified as male than data that is identified as female. the contribution of this work lies in its recognition and exploration of the non-binary nature of gender in the context of measuring and addressing bias in nlp models and systems. while previous research has primarily focused on gender as a binary variable, this study goes beyond the traditional binary categorization and acknowledges the fluidity and continuity of gender identities. by incorporating gender-neutral linguistic forms in datasets we aim to promote gender inclusion and recognise the non-binary spectrum of gender. this approach allows for a more comprehensive understanding of gender bias in nlp and provides insights into the biases present in hate speech detection models. ## related work in supervised learning contexts, there is significant research that identifies and measures bias in downstream nlp tasks. gender and racial biases @xcite , as well as biases against queer individuals @xcite and people with disabilities @xcite have been identified in sentiment analysis tasks. gendered occupational stereotypes are reflected in errors made by co-reference resolution systems @xcite and occupational classification models @xcite . a wide range of research into gender bias studies predominantly focuses on two genders, male and female, not recognising the experiences of individuals who identify as non-binary or gender non-conforming. this is a significant limitation of much of the existing research, as it fails to fully capture the diverse experiences and perspectives of individuals across the gender spectrum. recent research has highlighted the importance of including non-binary identities in nlp studies. studies focusing on neopronouns have shown that language models have difficulties processing them in various languages, including swedish, danish, and english @xcite . also, work by cao and daumé @xcite proposes methods for improving gender inclusivity throughout the machine learning lifecycle, including data collection, model training, and evaluation. a road map toward the integration of inclusive language in translation, with a focus on machine translation tasks, has been discussed in work by @xcite . this work focuses on gender-neutralisation strategies in the context of english-italian translation. moreover, in order to improve support for individuals who identify as non-binary or gender non-conforming, enabling them to self-identify their preferred pronouns and interact with technology in a manner that aligns with their social identity, gender-neutral rewriting models have emerged @xcite in the text generation task. the purpose of a genderneutral rewriter is to automatically identify the gendered language in a text and replace it with genderneutral alternatives. in order to produce genderneutral language, research by @xcite ; @xcite in a relatively similar approach proposed a rule-based and neural approach to automatically rewrite text to be more gender-neutral. the system is designed to identify gender identity words such as "he/she" and replace them with "they". the goal is to promote inclusivity and reduce bias in language use by avoiding gender-specific language that may reinforce gender stereotypes or exclude individuals who do not identify with traditional gender roles. ## approach and evaluation this research aims to explore gender bias in hate speech and offensive language classification, with a specific focus on gender-neutral language. we will accomplish this by analysing commonly used user-generated content datasets, particularly three twitter datasets for abusive content and offensive language identification. these datasets have been used in prior bias detection studies @xcite . abusive language includes various types including stereotypes, offense, abuse, hate speech, threats, etc @xcite . the connections among these phenomena based on previous research, have been visually represented in a work by @xcite and it is shown in figure 1 . current approaches for detecting and mitigating harmful language mainly focus on offensive language, abusive language, and hate speech however with varied and inconsistent definitions @xcite . the difference between offensive language, abusive language, and hate speech lies in their specificity. offensive language is more general, hate speech is more specific, and abusive language falls in between. the hate speech dataset @xcite is a collection of almost 17k tweets consisting of 3,383 samples of sexist content, 1,972 samples of racist content, and 11,559 neutral samples. the dataset is transformed into a binary classification problem by labeling the sexist and racist samples as the "abusive" class and neutral samples as the "non-abusive" class. the abusive tweets dataset is a large-scale crowd-sourced dataset, collected by founta et al. @xcite . the size of the dataset is just under 100k tweets and it is annotated with four labels: hateful, abusive, spam, and none. by combining the none and spam instances into a "non-abusive" class, and the hateful and abusive instances into an "abusive" class, we transform the dataset to a binary classification task, similar to the hate speech dataset. the hatespeech and offensive dataset @xcite is a collection of almost 25k tweets. the majority of tweets are considered to be offensive language (77%), almost 17% are labeled as non-offensive and only almost 6% of the tweets are flagged as hate speech samples. by assigning the "abusive" class label to samples exhibiting hate speech and offensive, and the "non-abusive" label to non-offensive samples, we convert the dataset into a binary classification problem. the hatespeech & offensive dataset contains a significant class imbalance, 83% of the dataset is assigned as abusive while only 17% is assigned as a non-abusive class. in order to create a more balanced dataset for experimental purposes, undersampling was performed on the abusive class during the evaluation by randomly selecting a 17% sample of the abusive data leaving a balanced dataset for this work of 8305 instances. ## results and discussion the high scores on bnsp and bpsn auc metrics results for the abusive tweets dataset show that the model exhibits relatively low bias across all the female and male and neutral subgroups, with high bnsp and subgroup auc scores indicating similar performance to the background group. however, the two hate speech datasets show some level of bias across these figures and it differs between the different subgroups. interestingly the figures for the male and neutral subgroups on the hate speech datasets are much closer to each other and higher than the female subgroup. low values in the bpsn and bnsp auc metrics indicate more bias. so this suggests that the bias for the female subgroup is higher than the male and neutral. looking at what these auc metrics tell us, the bpsn score for females on the hate speech datasets is relatively low with a score of 0.58 in the hatespeech and 0.78 in the hatespeech & offensive dataset. a low bpsn score suggests that the model is more likely to incorrectly classify negative or non-abusive examples from female subgroups as abusive compared to the background groups, which in this case are male and neutral, indicating the model is more likely to predict abuse for the female instances than the male and neutral instances. on the other hand, the bnsp score for the hate speech datasets is lower for male and neutral subgroups than the female subgroup. since the bnsp score measures the difference in false negative rates between the subgroup and the background group the low score in the male and neutral subgroups indicates that the model tends to incorrectly classify abusive examples from both the male and neutral subgroups as non-abusive compared to their respective background group. this suggests that for these hate speech datasets, male and neutral abusive instances are more likely to be missed than female instances. the bpsn and bnsp for genderneutral suggest that the model may be more biased against the gender-neutral subgroup compared to the male subgroup, but less biased compared to the female subgroup. furthermore, the negative values of both the abusive and non-abusive aeg and the hatespeech and hatespeech & offensive datasets suggest that the model is biased towards the female subgroup, as there is a downward shift in scores for this subgroup. this bias is further supported by the low bpsn auc score, which indicates that the model is more likely to make false positive predictions for the female subgroup compared to the background groups. specifically, the negative aeg scores indicate that the model is performing worse for the female subgroup than the reference group, which can contribute to the lower auc score. moreover, positive scores for both the abusive and non-abusive aeg for neutral and male suggest that the model might give more weight or importance to certain features in the neutral and male subgroups when classifying positive and negative examples. this means that the model may be more accurate in classifying positive and negative examples from these two subgroups compared to the background group, with the degree of attribute amplification being relatively small. also, a positive aeg value for the non-abusive class along with a low bnsp indicates that the model is performing better for the male and neutral subgroup for the non-abusive class. overall, these results suggest that the model may exhibit some bias against the neutral and male subgroups, particularly in terms of false negative rates, but the degree of bias is less severe compared to that shown for the female subgroup. looking at the results for hatespeech and hate-speech & offensive datasets we can see that including gender-neutral data in the datasets shows gender bias in the female subgroup, but surprisingly gender-neutral and male results have similar behavior on the bias metrics. there could be several reasons that cause this behavior. given the novelty and limited usage of gender-neutral terms in many societies, they might appear infrequently in training data. consequently, machine learning models could encounter difficulties in comprehending and generating gender-neutral language. for instance, terms like "nibling/pibling" or "sibling" are uncommon in daily speech, and may limit the model's exposure to gender-neutral language. second, gender-neutral forms of specific words, such as "actress" or "waitress," is often associated with the male form, reflecting a common representation found in many datasets. another possible reason might be that the gender-neutral term "they" is the same as the plural "they" which might confuse the model in distinguishing singular and plural they. results show male and gender-neutral subgroups have similar biased behavior according to ## conclusion in this paper, we presented an approach for measuring gender bias in a downstream task of identifying abusive or hate speech that considers male, female, and gender-neutral identities. we adjusted existing real-world datasets by gender-neutralising a random subset of instances and extended existing binary template definitions to include identity terms that reflect gender neutrality. our approach helps address the lack of training data that includes gender-neutral linguistic forms, which is essential for creating more inclusive nlp models and systems by incorporating gender-neutral words through the use of a gender-neutral rewriter. this can lead to more inclusive nlp models and systems. we have evaluated bias towards male, female, and gender-neutral groups and our findings showed that male and gender-neutral groups have similar bias behavior according to the auc bias metrics, while the female group shows a higher bias compared to the others. this approach can help promote more fair and equitable nlp systems by identifying gender bias in the data. while our approach aims to address gender bias in abusive and hate speech detection, there are certain limitations to consider. firstly, the modification of existing datasets by incorporating genderneutral instances relies on the availability of such data. the scarcity of gender-neutral linguistic forms in real-world datasets can pose a challenge in achieving adequate representation. secondly, the template-based approach used to generate synthetic test data may not fully capture the nuances and diversity of gender identities, potentially impacting the generalisability of the results. it is important to acknowledge that the concept of non-binary equivalents for binary gender terms is a subject of ongoing debate and individual preference. while a list of suggested non-binary equivalents has been provided in this paper, it is important to recognise that these terms may not be universally agreed upon or applicable to all non-binary individuals. in future work, we will explore the impact of adjusting datasets to include more gender-neutral identity terms and examine the influence of the dataset size on the results. in addition, a future focus will be on exploring label preservation after gender neutralisation. we will examine the impact of gender-neutralising instances that may be gender stereotypes due to gender roles and consider cases where the resulting text can lose its perceived hatefulness, especially if the assumption is made that the target is a woman/women.