An Explainable Approach to Understanding Gender Stereotype Text (2024)

Abstract: Gender Stereotypes refer to the widely held beliefs and assumptions about the typical traits, behaviours, and roles associated with a collective group of individuals of a particular gender in society. These typical beliefs about how people of a particular gender are described in text can cause harmful effects to individuals leading to unfair treatment. In this research, the aim is to identify the words and language constructs that can influence a text to be considered a gender stereotype. To do so, a transformer model with attention is fine-tuned for gender stereotype detection. Thereafter, words/language constructs used for the model’s decision are identified using a combined use of attention- and SHAP (SHapley Additive exPlanations)-based explainable approaches. Results show that adjectives and verbs were highly influential in predicting gender stereotypes. Furthermore, applying sentiment analysis showed that words describing male gender stereotypes were more positive than those used for female gender stereotypes.

## introduction gender stereotypes (gs) are the perceptions about the typical physical, emotional, and social characteristics displayed by men and women @xcite @xcite . thus, gender stereotypes function as text that can be used to directly or indirectly infer that individual's gender. these perceptions/beliefs assumed by society about an individual based on their gender can lead to gender bias negatively impacting that individual's life. for example, andrich and domahidi (2022) studied descriptions about u.s. political candidates. their study showed that facebook comments posted by users were gender stereotypical in the way that the male candidates were described with stronger masculine traits associated to a political career than the female candidates. this discrepancy and power inequality in traditionally assumed feminine/masculine gender stereotypes has the potential to negatively influence the voters' decisions thus penalizing the candidates based on their gender @xcite . another similar instance occurred during the 2017 labor leadership election in britain. an analysis of the language used in news articles about the candidates showed discrepancies in how they were described that were related to their gender foot_0 . these examples illustrate how language used to describe the subject based on their gender may perpetuate gender stereotypes and lead to gender bias and/or unfair treatment of individuals based on their gender. hence, it is important to understand gender stereotypes that could potentially lead to gender bias and discrimination against individuals based on their gender. the aim of this paper is to use explainable ai (xai) approaches when predicting gender stereotypes to understand the words or language that suggest a gender stereotype. a challenge with using ai prediction models is that they are blackboxes. it makes it hard for humans to understand why models arrived at the particular decisions that they predicted @xcite . therefore, xai approaches aim to improve the transparency and interpretability of ai models by offering explanations as to how or why the predicted result was inferred. xai approaches are generally categorized as transparency design explanations and post-hoc explanations @xcite . transparency design approaches explain how the model functions in the view of the developer such as the model's structure, understanding the individual components of the model, its underlying training algorithm, etc. post-hoc explanations provide an understanding of why a prediction was inferred; the components of the input that influenced the output @xcite . in this work, we use post-hoc explanation approaches such as attention and shap to identify words that influenced the model's prediction of a gender stereotype and anti-gender stereotype text. since the idea of attention was introduced in @xcite , it has been used in understanding text for various nlp tasks as the attention mechanism helps a model to capture the context of words and to focus on the relevant parts of a text when making decisions about the prediction @xcite @xcite . attention captures the importance of the word to the model's prediction corresponding to that particular input text. therefore, it has been considered to be a local-level of explanation surrounding that particular input instance @xcite . on the other hand, xai explanations like shap enable a more sophisticated understanding of how the words are important on a global-level to the whole model. therefore, shap is said to generate global explanations of a model's prediction offering a global understanding of which words are important. our approach is to fine-tune a transformer model with attention to classify textual input as a gender stereotype or anti-gender stereotype. thereafter, using the attention and shap-based explanations, we identify the words that influence the model's decision to categorise the input text as a gender stereotype. in addition, we perform a sentiment analysis on the identified top-influential words to study the emotion associated with the choice of words used for gender stereotypes about men and women. our analysis of top-influential words and language constructs show that adjectives and verbs highly impact gender stereotype predictions. in addition, sentiment analysis shows that gender stereotypes associated with the male gender are more positive than those associated with the female gender. the rest of this paper is structured as follows. section 2 presents the related works on gender stereotypes and gender stereotype detection. section 3 outlines the datasets and model architecture implemented, the explainable approaches used and how we obtain the top-influential words that suggest gender stereotypes. we present the re-sults of our evaluation in section 4 and discuss the observations. we conclude by presenting our key findings and some limitations in our current work. ## related work often gender stereotype and gender bias are considered synonymous though their focus and scope differ @xcite . gender bias is a more specific and technical term that refers to the intentional or unintentional discrimination against individuals based on their gender (costa-jussà, 2019). more generally, gender stereotypes refer to the widely held beliefs and assumptions about the typical traits, behaviors, and roles that are associated with men and women in society @xcite @xcite . although the definition of gender stereotypes roots from the attribution of characteristics or traits to the group, the bias itself rises from the discrimination an individual faces by being assumed and assigned the same characteristics or traits of the group. hence, this paper discusses stereotyping from the perspective of an individual as driven by the motivating examples in the introduction. most of the work in existing literature focuses on identifying and understanding gender bias using ml rather than on gender stereotypes @xcite . for example, researchers investigated the existence and/or the mitigation of gender bias in word embeddings @xcite @xcite , language models @xcite @xcite , coreference resolution @xcite @xcite , machine translation @xcite @xcite , parts-of-speech (pos) tagging @xcite , natural language generation @xcite , etc. existing work on analysing gender stereotypes is mainly focused on the use of pre-defined lexicons of gender-specific words and actions curated through manual and psychological studies @xcite @xcite . herdagdelen and baroni studied the association between gender and actions related to gender stereotypes. they extracted verb-noun pairs from the omcs common sense database and analyzed the occurrence of the verb-noun pairs in the tweets. their results showed that there are clear gender associations with certain actions, such as women being more associated with cooking and cleaning, while men were more associated with driving and building. rubegni et al. explored how children perceive gender stereotypes by analyzing the characters in text written by children in the form of storytelling. they found that male antagonists were described using a limited set of negative adjectives which are demeaning descriptors, while female antagonists were defined using a richer and more varied set of negative qualities. a more recent study by cryan et al. used a selfcompiled dataset of web-posts and news articles which were annotated through crowd-sourcing to identify instances of gender stereotypes. this supervised learning-based method involved training a machine learning model on a set of annotated data to classify texts as to whether the description of an individual in text conformed or contradicted to the intended gender of the subject. the most frequently used words which were used for the gender-conforming and gender-nonconforming predictions were presented in their work. in the past, while machine learning models remained black boxes, using the attention mechanism was a popular approach to understand the predictions of models by looking at the parts of text that were highly attended to as the model was making its decision @xcite . when a transformer model processes each word in the input text, it calculates attention scores for each word. this attention score indicates how much weight or attention the model should give to that word when it decides the predicted class. various studies have found attention to be unreliable explanations @xcite . although the attention score captures the absolute importance of the token, researchers have contradicted the idea of how this instance-level understanding can be approximated to get a global understanding of the feature's importance to the whole model's prediction understanding @xcite . and, the scaling factor used to calculate the attention score can affect the interpretability of the feature's importance in terms of the attention. according to @xcite , attention is not a robust indicator. attention was found to loudly predict the overall relevance of the input components (the words) to a model @xcite . moreover, @xcite question the extent to which attention can provide explainability of feature importance. attention weight measures the relative importance of the token within a specific input sequence. so though the attention score captures the absolute importance of the token, researchers have contradicted the idea of how this instance-level understanding can be approximated to get a global understanding of the feature's importance to the whole model's prediction understanding @xcite . nevertheless, there are works that strongly challenge this claim of the attention not being an explanation of feature importance @xcite . and, researchers have been using the attention score to understand and interpret top words influencing the predictions of machine learning models @xcite . recently, the concept of xai has paved way for these black-box ml model predictions to be interpreted as glass-box explanations @xcite . there are a wide variety of approaches through which these explanations can be derived @xcite @xcite . but most of these are based on post-hoc explanations of a surrogate model that render model-agnostic explanations. some such xai approaches are shap and lime (local interpretable model-agnostic explanations). shap provides a global explanation of the output of any ml model by assigning each feature an importance value (shap value) in the prediction process @xcite . shap values take into account the token interactions based on whether a word is present or absent across the predicted instances and builds a model based on these changes to explain the predictions in the context of other words. work done by @xcite used shap values to study explanation of racial stereotypes. this study identified the words that were most influential in categorizing text into different categories of hate speech based on their shap values. male/female as a biological sex assigned at birth, we consider male/female as a gender. as defined in @xcite , "gender refers to a person's gender identity (how they see themselves or experience their own gender) but also involves other factors such as how a person is perceived by others or experiences differential treatment related to their perceived gender". the stereoset (sset) dataset @xcite ) contains 4 stereotypical categories (gender, race, religion, occupation) of which we use the gender category instances for our research. to create this dataset the authors compiled target terms that represented the different target categories (e.g., for gender "woman", for race "asian", etc.) based on wikidata associations found in triples related to the above categories. then, crowd-workers were asked to write two sentences describing people using these target terms where one sentence suggests a gender stereotype while the other does not. we require the gender of the subject discussed in the text but gender is not explicitly identified in this dataset. we manually labelled the gender identity of the subject as describing a male or a female person. there were 55 instances where the gender of the subject described in the text was not identifiable, these instances were excluded from our analysis. cryan's content (cc) dataset was specifically compiled to study gender stereotyping @xcite . using crowd-sourcing crowd workers were asked to find articles that describe a person (male/female) and label them as whether the description is consistent or contradictory to common gender stereotypes as perceived by that crowd-worker. this dataset has 4 labels, consistent with or contradictory to male/female. translating these labels to a binary classification for our experiments, the male/female consistent labels become gender stereotypes (gs) and the contradictory ones, anti-gender stereotypes (anti-gs). the crowd-workers who were compiling and labelling articles for cryan et al.'s research were also requested to provide their reason for labelling an article as consistent with a gender stereotype or contradictory to a gender stereotype which was not used in their study. reviewing these texts provided as reasons by the annotators, we found them to be valid and direct perceptions of why a person (crowd-worker) would consider a certain text as a gs or an anti-gs. we used these reason texts to generate a dataset which we called cryan's reasons (cr) and labelled it manually as a gs or anti-gs text. to label the data, it was divided into 4 subsets of approximately 1000 text samples each, and 3 annotators labelled each subset of text samples. annotators were asked to label if they considered the text was a gender stereotype or not. they were also asked to select if they thought the text described a "male", "female", "non-binary" gendered person or was "not related to a person". the inter-annotator agreement (iaa) for the gs/anti-gs label for each subset was calculated using the fleiss kappa @xcite . one subset of labelled text samples with an iaa less than 0.8 was dropped and the other 3 subsets with iaas of 0.89, 0.89 and 0.9 were retained giving an average iaa across all retained labelled samples of 0.89. to arrive at a consensus label for the gender and gender stereotype/anti-gs labels, the label assigned to each instance was based on a majority vote, i.e. the value chosen by 2 out of 3 annotators. instances where the 3 raters' gender labels were all different were dropped. then, we removed the instances where the consensus gender label was "not related to a person". only 37 samples were about non-binary people (11 gs and 26 anti-gs). this was not sufficient to train and test a classifier model for our study. therefore, we retained the male and female samples, a total of 3221 samples: 1081 male gs, 958 female gs, 528 male anti-gs and 654 female anti-gs samples. following a similar approach to cryan et al. (2020), we use a transformer model based on the bert architecture, which is a pre-trained deep neural network architecture used to process sequential input data, such as text. we chose bert due to its bidirectional nature. in addition, its context aware embeddings capture relationships between words. and researchers have been successfully fine-tuning bert for downstream tasks in the past within the domain @xcite @xcite . we fine-tuned bert for the gender stereotype detection task on each dataset and added a classification head to predict if a new unseen text was a gs or anti-gs. the pre-trained bert model is fine-tuned on the labeled training datasets and optimized for the best hyper-parameters using optuna @xcite which is an opensource hyper-parameter optimization framework based on bayesian optimization. performance is measured as the average class recall (due to imbalance in the class distribution of the data) over three iterations of 5-fold cross validation on each dataset. the sole use of one xai approach is not a reliable measure of the influential words contributing to the prediction @xcite . attention scores can sometimes be sensitive to noise or outliers in the data leading to misleading interpretations @xcite . and although the fundamental workings of ml models remain unclear, xai methods approximate the model's behaviour based on the predictions. therefore, the post-hoc explanations produced by xai methods like shap alone may not be as fully accurate at capturing how the ml model arrived at a decision (zhong and negre, 2022) either. hence, we looked into capturing the words' importance in making a prediction using more than one approach. abnar and zuidema (2020) state that though shap values are not attention scores, the attention flows which are an extension of attention weights obtained after post-processing align with shap values. so, we use the attention score along with the shap value to identify the words that influence the model's prediction. we combine the attention score and shap values to get an influence score iscore(w i ) for the word w i as shown in equation 1 . we ranked the words in each instances by their influence scores. we selected the top three words with the highest word influence score for analysis. the words with word influence scores lower than these top three were typically article words (a, an, the), prepositions (in, under), conjunctions (and, but) and determiners (some, many). ## results and discussion figure 1 reports the mean and std.deviation of the average class recall on the three datasets across three iterations of 5-fold cross validation for the gender stereotype detection task. ss, cc and cr datasets obtained average class recalls of 0.7, 0.75 and 0.78, respectively with the cr dataset achieving the best performance. further analysis was carried out on the words and type of language constructs that influenced the predictions. ## conclusion gender stereotypes manifest in the way people express themselves through gender expression/characteristic traits described using adjectives or their gender roles/actions described using verbs. these gender stereotypes can prompt harmful effects leading to gender bias if not captured. in this research, we fine-tune a transformer model with attention to classify gender stereotypes. a proposed combination of attention and shap explainable approach is used to identify the words/language constructs that influence a text to be considered as a gender stereotype or not. our findings showed that adjectives (gender expression descriptors) and verbs (gender role descriptors) highly impact a text to suggest a gender stereotype. furthermore, a sentiment analysis of identified top-influential words also revealed that top-influential words used to describe males were more positive than those chosen to describe females. this partiality towards the way in which genders are described represents gender bias where humans evaluate expressions related to men more positively than those related to women.