Do PLM s and Annotators Share the Same Gender Bias? Definition, Dataset, and Framework of Contextualized Gender Bias (2024)

Abstract: Pre-trained language models (PLMs) have achieved success in various of natural language processing (NLP) tasks. However, PLMs also introduce some disquieting safety problems, such as gender bias. Gender bias is an extremely complex issue, because different individuals may hold disparate opinions on whether the same sentence expresses harmful bias, especially those seemingly neutral or positive. This paper first defines the concept of contextualized gender bias (CGB), which makes it easy to measure implicit gender bias in both PLMs and annotators. We then construct CGBDataset, which contains 20k natural sentences with gendered words, from Chinese news. Similar to the task of masked language models, gendered words are masked for PLMs and annotators to judge whether a male word or a female word is more suitable. Then, we introduce CGBFrame to measure the gender bias of annotators. By comparing the results measured by PLMs and annotators, we find that though there are differences on the choices made by PLMs and annotators, they show significant consistency in general.

## introduction plms have achieved success in varieties of nlp tasks @xcite @xcite . however, there is ample evidence showing that these plms trained on real-world text may cause safety problems, such as offensive language, social biases, and toxic behaviors @xcite @xcite . among those unsafe issues, social bias, especially gender bias, is one of the most difficult † equal contribution. ‡ corresponding authors. 1 our dataset is available at https://github.com/ zhushucheng/cgbdataset/ . plms have been shown to learn gender biases from the texts they trained on @xcite @xcite . the subtleness of gender bias makes it more complicated to analyze what reasons may cause plms to express gender bias. algorithms of plms may amplify the bias in the texts @xcite @xcite @xcite . annotators might also bring their biases into plms when they are in nlp annotation tasks @xcite . the former reason may cause plms and annotators share different gender biases as plms only learn gender bias from the texts they trained on. plms and annotators may share the same gender bias according to the latter reason. therefore, our core question is: do plms and annotators share the same gender bias? what might be the reasons why plms and annotators share the same gender bias or not? the answers to these questions may help us better understand the mechanism of bias in nlp and find the correct methods to debias models. figure 1 : the task of measuring cgb involves plms and annotators filling in sentences where gender words have been masked and replaced with male or female words. therefore, we first give our definition of contextualized gender bias (cgb) to expediently measure implicit gender bias in both plms and annotators. the idea of cgb is from the concept indexicality @xcite in sociolinguistics. linguistic features index particular stances and activities that ideologically linked to salient social categories, such as gender @xcite . some contexts always index a particular gender, indicating what behaviors men should perform and what behaviors women should perform. it is the process of social construction of gender through language. inspired by the task of masked language models (mlms), we define the task to measure cgb is to have plms and annotators fill in the sentences that masked gender words with male words or female words, shown as figure 1 . if plms or annotators show tendency to fill in the masked word with a specific gender word in theoretically unbiased context, we think this context indexes a particular gender, demonstrating that plms or annotators over-associate a specific behavior to a specific gender, which is a kind of implicit gender bias, called cgb. rather than directly judging whether a sentence expresses harmful bias towards a specific gender group or not, this definition uses an indirect way to catch the intuition on the sentences index-ing gender, which is easily understandable for all people who even may not be exposed to nlp annotation tasks. in other words, cgb is created to measure the implicit gender bias in both plms and annotators. then, we build a 20k-sentence chinese dataset cgbdataset based on the concept cgb to measure the implicit gender bias in plms and annotators. notice that here our task is to use the dataset to measure the gender bias of annotators instead of inviting annotators to annotate gender bias in the dataset. though many researchers devote to construct reliable datasets and benchmarks on bias @xcite @xcite and offensive language @xcite @xcite , most have been focused on english and only a few works built chinese dataset on this topic @xcite @xcite . besides, some of the datasets are template-based @xcite , which may lead to overestimate the gender bias measured by plms @xcite . our cgbdataset is extracted from natural texts in chinese news, which have diverse sentences and can be used in different nlp tasks. we also introduce a detailed and novel framework cgbframe to measure annotators' gender bias. then, we can compare the results measured by plms and annotators. it is found that though there exists differences on the choices made by plms and annotators, they show significant consistency in general. we demonstrate that the novel consideration of cgb, cgbdataset, and cgbframe are essential for implicit gender bias measurements in both plms and annotators. the contributions of our work can be summarized as follows: in the study of language and gender, the theory of gender performativity is quite important. gender is not a pre-existing fact, but rather something that must be continuously brought into being through the enactment of social practices. performativity refers to the embodied performances of gender that through repetition begin to look as if they are natural and self-evident @xcite @xcite . a main method is through language. from the very beginning of our life, we learn to perform correct gender behaviors through the language around us. that indicates our language usually indexes particular stances and activities that ideologically linked to salient social categories, such as gender. it is the concept indexicality @xcite ) in sociolinguistics. for example, male is always related to work while female is always related to family in our language @xcite . as a result, gender gradually solidifies the differences that should not be caused by gender and may cause unexpected biases and harms @xcite . different from spanish and some of the fusional languages, chinese lacks grammatical gender. in chinese, referential gender ('她' means 'she') and lexical gender ('爸爸' means 'father') are two common ways to express gender (cao and daumé iii, 2020), and we define these two linguistic genders as gender words. regardless of the social regulations of gender, the gender information in context or sentence is only reflected by gender words. that is, when the gender words are masked like the task in mlm, the probabilities of filling in with male or female gender words are theoretically the same. this can be illustrated by the example in figure 1 . according to the given sentence, the probabilities to fill in mask with 'fathers' and 'mothers' should be the same. so, we define this kind of sentence or context as theoretically unbiased context. however, based on our social regulations or experiences, we usually think childcare is the business of mothers. then, annotators and plms all choose 'mothers' to fill in mask. we define that the tendency where plms or annotators choose a particular gender word to fill in the mask in a theoretically unbiased context is contextualized gender bias (cgb). though most of the theoretically unbiased context do not show negative or offensive attitude towards the subject in the context, we articulate that this over-association of plms or annotators may still do harms to specific gender. cgb is subtle and always implicit. sometimes the expression even shows a positive attitude towards women. nonetheless, cgb constrains the specific gender with specific fields, behaviors, and activities, leading to not only do harms to those who are not consistent with the mainstream social norms and regulations, but also erase the uniqueness between person and person. ## cgbdataset we introduce cgbdataset, which contains 20k sentences, extracted from real-world chinese news text. we divide cgbdataset into two parts. one is measuring sentences, which is the main part of cgbdataset to measure cgb in plms and annotators. the other one is objective sentences, which is to measure the accuracy of plms and annotators when the gender word can be definitively inferred. ## cgbframe to measure cgb of annotators, we devise a novel and indirect framework cgbframe for both coarsegrained and fine-grained measurements. due to the complexity and subjectivity of the annotation tasks in some social concepts, such as bias @xcite and intimacy @xcite , the agreement is inevitably lower. though our goal is to use the cgbdataset to measure annotators' cgb, rather than inviting annotators to annotate the dataset, the subjectivity of this task reminds us of putting forward methods to control the quality when measuring annotators' cgb. therefore, we design controllable questions to control the quality of measurement, besides measuring questions, which are to measure cgb of annotators. ## measurement of plms we measured cgb of three widely used plms based on cgbdataset. we used the default parameters and hyperparameters for each model to set the experiment with a rtx2080ti gpu. the ideal plm is that performs high accuracy on objective sentences with low cgb scores on measuring sentences. ## related works gender bias has been found in all fields and tasks of nlp, such as word embeddings @xcite @xcite , coreference resolution (cao and @xcite @xcite , machine translation @xcite , sentiment analysis @xcite , abusive language detection @xcite , and so on. surveys on gender bias in nlp concentrate on how to detect, measure, analyze, and mitigate gender bias in dataset and system @xcite @xcite . according to the causes, manifestations and forms of bias, several studies have classified bias @xcite @xcite . the detection of gender bias in nlp and the construction of dataset to measure and analyze gender bias always depend on the classification and defining of gender bias @xcite . there have been several datasets to detect and measure gender bias @xcite @xcite , or to mitigate gender bias @xcite , by trained annotators or by crowdsourcing @xcite @xcite . in chinese, datasets were built to detect offensive languages @xcite , and social bias @xcite . ## conclusion we define cgb to measure implicit gender bias in plms and annotators. based on the task of mlm, cgbdataset is constructed to measure cgb of both annotators and plms. cgbframe is introduced to better measure cgb of annotators. metrics show high-quality of our dataset and framework. we also measure cgb in popular chinese plms and show that they express cgb. different reasons can be found when plms and annotators share the same or opposite cgb. in the future, different groups of annotators should be included to measure cgb.