Do PLMs and Annotators Share the Same Gender Bias? Definition, Dataset, and Framework of Contextualized Gender Bias (2024)

Abstract: Warning: This paper contains statements of biases and may be upsetting. Pre-trained language models (PLMs) have achieved success in various of natural language processing (NLP) tasks. However, PLMs also introduce some disquieting safety problems, such as gender bias. Gender bias is an extremely complex issue, because different individuals may hold disparate opinions on whether the same sentence expresses harmful bias, especially those seemingly neutral or positive. This paper first defines the concept of contextualized gender bias (CGB), which makes it easy to measure implicit gender bias in both PLMs and annotators. We then construct CGBDataset, which contains 20k natural sentences with gendered words, from Chinese news. Similar to the task of masked language models, gendered words are masked for PLMs and annotators to judge whether a male word or a female word is more suitable. Then, we introduce CGBFrame to measure the gender bias of annotators. By comparing the results measured by PLMs and annotators, we find that though there are differences on the choices made by PLMs and annotators, they show significant consistency in general.1

text: Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 20–32 August 16, 2024 ©2024 Association for Computational Linguistics
Pre-trained language models (PLMs) have achieved success in various of natural language processing (NLP) tasks. However, PLMs also introduce some disquieting safety problems, such as gender bias. Gender bias is an extremely complex issue, because different individuals may hold disparate opinions on whether the same sentence expresses harmful bias, especially those seemingly neutral or positive. This paper first defines the concept of contextualized gender bias (CGB), which makes it easy to measure implicit gender bias in both PLMs and annotators. We then construct CGBDataset, which contains 20k natural sentences with gendered words, from Chinese news. Similar to the task of masked language models, gendered words are masked for PLMs and annotators to judge whether a male word or a female word is more suitable. Then, we introduce CGBFrame to measure the gender bias of annotators. By comparing the results measured by PLMs and annotators, we find that though there are differences on the choices made by PLMs and annotators, they show significant consistency in general.1

heading: 1 Introduction
text: PLMs have achieved success in varieties of NLP tasks (Devlin et al., 2019; Liu et al., 2019; Clark et al., 2020). However, there is ample evidence showing that these PLMs trained on real-world text may cause safety problems, such as offensive language, social biases, and toxic behaviors (Sun et al., 2022; Blodgett et al., 2020; Sheng et al., 2021). Among those unsafe issues, social bias, especially gender bias, is one of the most difficult
†Equal contribution. ‡Corresponding authors. 1Our dataset is available at https://github.com/
zhushucheng/CGBDataset/.
problems to define and detect for the following two reasons. One is that gender bias is sometimes implicit and subtle. Some neutral or even positive attitudes towards women may also hurt them, which is called benevolent sexism (Glick and Fiske, 1996). For example, No man succeeds without a good woman besides him. Wife, mother. This expression shows a positive stereotypical picture with women, but constrains the role of women to the field of family (Zeinert et al., 2021). The second reason is that different groups of people may have varying perspectives on bias. Specifically, men may not recognize bias against women, and vice versa. This group difference can be used to find microaggressions (Breitfeller et al., 2019). Even in the same group, different individuals may hold disparate opinions on whether the same sentence expresses harmful bias based on different perceptions and experiences. This individual difference inevitably causes the low agreement rate when annotators judge whether a sentence demonstrates gender bias or not (Zhou et al., 2022).
PLMs have been shown to learn gender biases from the texts they trained on (Caliskan et al., 2017; Zhao et al., 2018; Rudinger et al., 2018). The subtleness of gender bias makes it more complicated to analyze what reasons may cause PLMs to express gender bias. Algorithms of PLMs may amplify the bias in the texts (Zhao et al., 2017; Bordia and Bowman, 2019; Qian et al., 2019; Webster et al., 2018, 2020). Annotators might also bring their biases into PLMs when they are in NLP annotation tasks (Geva et al., 2019). The former reason may cause PLMs and annotators share different gender biases as PLMs only learn gender bias from the texts they trained on. PLMs and annotators may share the same gender bias according to the latter reason. Therefore, our core question is: do PLMs and annotators share the same gender bias? What might be the reasons why PLMs and annotators share the same gender bias or not? The
20
answers to these questions may help us better understand the mechanism of bias in NLP and find the correct methods to debias models.
Therefore, we first give our definition of contextualized gender bias (CGB) to expediently measure implicit gender bias in both PLMs and annotators. The idea of CGB is from the concept indexicality (Ochs, 1992) in sociolinguistics. Linguistic features index particular stances and activities that ideologically linked to salient social categories, such as gender (Angouri and Baxter, 2021). Some contexts always index a particular gender, indicating what behaviors men should perform and what behaviors women should perform. It is the process of social construction of gender through language. Inspired by the task of masked language models (MLMs), we define the task to measure CGB is to have PLMs and annotators fill in the sentences that masked gender words with male words or female words, shown as Figure 1. If PLMs or annotators show tendency to fill in the masked word with a specific gender word in theoretically unbiased context, we think this context indexes a particular gender, demonstrating that PLMs or annotators over-associate a specific behavior to a specific gender, which is a kind of implicit gender bias, called CGB. Rather than directly judging whether a sentence expresses harmful bias towards a specific gender group or not, this definition uses an indirect way to catch the intuition on the sentences index-
ing gender, which is easily understandable for all people who even may not be exposed to NLP annotation tasks. In other words, CGB is created to measure the implicit gender bias in both PLMs and annotators.
Then, we build a 20k-sentence Chinese dataset CGBDataset based on the concept CGB to measure the implicit gender bias in PLMs and annotators. Notice that here our task is to use the dataset to measure the gender bias of annotators instead of inviting annotators to annotate gender bias in the dataset. Though many researchers devote to construct reliable datasets and benchmarks on bias (Caliskan et al., 2017; May et al., 2019; Nadeem et al., 2021; Nangia et al., 2020) and offensive language (Gehman et al., 2020; Zampieri et al., 2019; Xu et al., 2020), most have been focused on English and only a few works built Chinese dataset on this topic (Tang et al., 2020; Deng et al., 2022; Zhou et al., 2022). Besides, some of the datasets are template-based (Zhao et al., 2018), which may lead to overestimate the gender bias measured by PLMs (Nangia et al., 2020). Our CGBDataset is extracted from natural texts in Chinese news, which have diverse sentences and can be used in different NLP tasks. We also introduce a detailed and novel framework CGBFrame to measure annotators’ gender bias. Then, we can compare the results measured by PLMs and annotators. It is found that though there exists differences on the choices made by PLMs and annotators, they show significant consistency in general. We demonstrate that the novel consideration of CGB, CGBDataset, and CGBFrame are essential for implicit gender bias measurements in both PLMs and annotators.
The contributions of our work can be summarized as follows:
• We propose a concept: contextualized gender bias (CGB). It adapts to the tasks of MLMs and is easy to measure implicit gender bias of both PLMs and annotators.
• We present a new Chinese dataset to measure contextualized gender bias in PLMs and annotators: CGBDataset. It contains 20k sentences, extracted from real-world Chinese news texts.
• We provide a novel framework CGBFrame to measure annotators’ CGB, using an indirect way to catch the intuition of annotators on the sentences indexing gender.
• We compare the results measured by PLMs and annotators, and show that though there exists differences on the choices made by PLMs and annotators, they show significant consistency in general.

heading: 2 Contextualized Gender Bias
text: In the study of language and gender, the theory of gender performativity is quite important. Gender is not a pre-existing fact, but rather something that must be continuously brought into being through the enactment of social practices. Performativity refers to the embodied performances of gender that through repetition begin to look as if they are natural and self-evident (Butler, 1990, 1993, 2004; Angouri and Baxter, 2021). A main method is through language. From the very beginning of our life, we learn to perform correct gender behaviors through the language around us. That indicates our language usually indexes particular stances and activities that ideologically linked to salient social categories, such as gender. It is the concept indexicality (Ochs, 1992) in sociolinguistics. For example, male is always related to work while female is always related to family in our language (Eagly et al., 2000; Wood and Eagly, 2002). As a result, gender gradually solidifies the differences that should not be caused by gender and may cause unexpected biases and harms (Li et al., 2022).
Different from Spanish and some of the fusional languages, Chinese lacks grammatical gender. In Chinese, referential gender (‘她’ means ‘she’) and lexical gender (‘爸爸’ means ‘father’) are two common ways to express gender (Cao and Daumé III, 2020), and we define these two linguistic genders as gender words. Regardless of the social regulations of gender, the gender information in context or sentence is only reflected by gender words. That is, when the gender words are masked like the task in MLM, the probabilities of filling in with male or female gender words are theoretically the same. This can be illustrated by the example in Figure 1. According to the given sentence, the probabilities to fill in MASK with ‘Fathers’ and ‘Mothers’ should be the same. So, we define this kind of sentence or context as theoretically unbiased context. However, based on our social regulations or experiences, we usually think childcare is the business of mothers. Then, annotators and PLMs all choose ‘Mothers’ to fill in MASK. We define that the tendency where PLMs or annotators choose a particular gender word to fill in the
MASK in a theoretically unbiased context is contextualized gender bias (CGB). Though most of the theoretically unbiased context do not show negative or offensive attitude towards the subject in the context, we articulate that this over-association of PLMs or annotators may still do harms to specific gender. CGB is subtle and always implicit. Sometimes the expression even shows a positive attitude towards women. Nonetheless, CGB constrains the specific gender with specific fields, behaviors, and activities, leading to not only do harms to those who are not consistent with the mainstream social norms and regulations, but also erase the uniqueness between person and person.

heading: 3 CGBDataset
text: We introduce CGBDataset, which contains 20k sentences, extracted from real-world Chinese news text. We divide CGBDataset into two parts. One is Measuring Sentences, which is the main part of CGBDataset to measure CGB in PLMs and annotators. The other one is Objective Sentences, which is to measure the accuracy of PLMs and annotators when the gender word can be definitively inferred.

heading: 3.1 Data Source
text: News articles are always regarded as texts with less bias (Lim et al., 2020). According to our definition of CGB, the bias we want to study is implicit and subtle. Hence, news articles are the perfect data source to our task. We selected China’s mainstream official newspapers (e.g. People’s Daily) from 2018 to 2019 (can be publicly accessed) as our corpus. Meanwhile, we chose 16 pairs of common Chinese gender words from a Chinese gender word list (Li et al., 2022). Next, we extracted complete sentences containing gender words from the newspaper corpus, based on punctuation marks at the end of each sentence. We manually filtered out some sentences which cannot be used to measure CGB (Appendix A). We also tried to balance the sentences containing male gender words and female gender words. Finally, there are 20k sentences in the dataset CGBDataset. The length of sentences in the dataset ranges from 9 to 119 characters, with the majority falling between 20 and 36 characters. One gender word in each sentence is masked with placeholder [MASK] or [MASK][MASK] and the female gender word and male gender word can be filled in the placeholder are recorded as well. Unlike the data source of StereoSet (Nadeem et al.,
2021) from templates or CrowS-Pairs (Nangia et al., 2020) from crowdsourcing, our data is from realworld news texts. It provides more diverse sentences, avoiding deliberately generating texts to suit the task which maybe lead to overestimate the gender bias measured by PLMs and annotators. Our dataset also expands the concept of bias, comparing to some of the Chinese datasets (Zhou et al., 2022; Deng et al., 2022), which has already explained in Section 2.

heading: 3.2 Measuring Sentences
text: 19,785 sentences are annotated as Measuring Sentences, which is to measure CGB in PLMs and annotators, just like the example shown in Figure 1. There should be no suggested gender clues for PLMs and annotators to infer the gender word to fill in the sentence. So these sentences are all theoretically unbiased. However, PLMs and annotators sometimes may show tendency towards specific gender according to other irrelevant information, like the over-association with females and childcare. If PLMs and annotators choose a specific gender word to fill in Measuring Sentences, their CGB can be caught.

heading: 3.3 Objective Sentences
text: 215 sentences are annotated as Objective Sentences, which is to measure the accuracy of PLMs and annotators when the gender word can be inferred based on some clues in the sentence. Table 2 in
Appendix C shows that there are 4 types in Objective Sentences: biological sex 1, fixed collocations, semantic relevance, and prior knowledge.

heading: 4 CGBFrame
text: To measure CGB of annotators, we devise a novel and indirect framework CGBFrame for both coarsegrained and fine-grained measurements. Due to the complexity and subjectivity of the annotation tasks in some social concepts, such as bias (Zhou et al., 2022) and intimacy (Pei and Jurgens, 2020), the agreement is inevitably lower. Though our goal is to use the CGBDataset to measure annotators’ CGB, rather than inviting annotators to annotate the dataset, the subjectivity of this task reminds us of putting forward methods to control the quality when measuring annotators’ CGB. Therefore, we design Controllable Questions to control the quality of measurement, besides Measuring Questions, which are to measure CGB of annotators.

heading: 4.1 Target Annotators
text: Before measurement, we need to select target annotators. The idealized results should be that both PLMs and annotators show no bias in Measuring Sentences, but can make the right choices in Ob-
1We acknowledge that while biological sex and gender are often correlated, they are not definitively linked. However, in the CGBDataset, only binary gender is discussed. We strongly recommend expanding the dataset and the discussion to include non-binary identities in the future.
jective Sentences. Hence, our target annotators are those with lower gender bias. Here, we used two psychological inventories to test the gender bias of annotators: MSS (Modern Sexism Scale) (Swim et al., 1995) and ASI (Ambivalent Sexism Inventory) (Glick and Fiske, 1996). These two inventories have already been translated into Chinese, with verification of reliability and validity among Chinese college students (Jia, 2013). We did not use implicit association test (IAT) (Greenwald et al., 1998) because inventories are more convenient as most of the annotators prefer to work online and they cannot take IAT offline. Finally, we selected 3 annotators with low gender bias, who are all college students and in their twenties. Two annotators are female and one is male. They also perform high accuracy in Controllable Questions, indicating that they are in good-quality and representative 2.

heading: 4.2 Measurement Design
text: The basic idea of measurement is matching appropriate gender words through context information, in order to measure CGB indirectly. Because this measurement is subjective and does not have correct answers, we design Type 1 and Type 2 questions to ensure the authenticity and effectiveness of measurement, without telling the annotators the definition of CGB. Both types are indirect measurement methods to catch this subtle CGB. Each type then has Measuring Questions and Controllable Questions. Examples are shown in Figure 2.

heading: 4.2.1 Measuring Questions
text: We designed Type 1 and Type 2 of Measuring Questions to measure annotators’ CGB. All the Measuring Questions are from Measuring Sentences (Section 3.2). Type 1 is a multiple-choice question where there are three words to replace [MASK]. The candidate options include a male word, a female word and a neutral word. The annotators must choose 2 out of the 3 words according to the contexts. They also need to rank the appropriate degree at the same time. If both options are correct without rank, they need to choose ‘q’. When the annotators do not choose both male words and female words, it shows that they think this context may index a specific gender, indicating that they have CGB. For example, No.2 of Type 1 in Figure 2
2We selected these three annotators from a pool of 150 candidates. The final three annotators demonstrated high accuracy in Controllable Questions and showed strong consistency with each other.
shows that the annotator did not choose male word ‘man’ to fill in the sentence. The reason might be that the annotator thought the context, especially the word ‘sewing machine’, indexes female, meaning that the annotator associate female with sewing activity. Type 2 is a probability judgment question, which reverses the opposition of gender words in the original sentence to get a new sentence. The annotators need to judge the occurrence probability of the characters in the brackets based on the current context. When there is a text that does not conform to the impression in the annotator’s experience, the score will be correspondingly lower. No.2 of Type 2 in Figure 2 demonstrates that the annotator considers men seldom doing sewing work. There is no gendered connotation with the term ‘tailor’ in the Chinese language.

heading: 4.2.2 Controllable Questions
text: The subjectivity of our measurement task makes it impossible to quantify the correctness of results. Therefore, we set up two types of Controllable Questions to measure the reliability and quality of annotation results by accuracy and selfconsistency. The first one is Accuracy Controllable Questions, which are all from Objective Sentences (Section 3.3) 3. They have correct answers according to the clues in the sentence, like No.3 of Type 1 and No.1 of Type 2 in Figure 2. Self-consistency Controllable Questions measure whether an annotator can keep himself or herself consistency in the same context between Type 1 and Type 2, like No.2 of Type 1 and No.2 of Type 2 in Figure 2.

heading: 4.3 Measurement Process
text: We first conducted a trial measurement with a scale of 200 questions to each annotator. The objective is to make the annotators familiar with our measurement task. The 200 questions include both Measuring Questions and Controllable Questions. Then, we checked the Controllable Questions. The results would be acceptable when overall accuracy of the Controllable Questions reached 80%. We divided our final measurement task into 10 batches. Each batch would include more than 2,000 questions for each annotator. In this measurement process, we explained and discussed the controversial results they got with the annotators. We collected
3We acknowledge that we overlooked transgender considerations in the Accuracy Controllable Questions, which might lead to transgender bias. For example, in the No.1 of Type 2, a trans man could indeed have eggs.
the reasons why the annotators chose one answer over another and redesigned the Controllable Questions and Measuring Questions accordingly. For every batch, the accuracy of Accuracy Controllable Questions each annotator should be more than 80% and the consistency of Self-consistency Controllable Questions should reach 60%. Otherwise, the annotator needs to redo this batch. If the annotator meets this standard, they will get 100 RMB each batch as a pay. Appendix B shows our measurement metrics of both fine-grained and coarsegrained methods. The whole measurement process was approved by the university ethics review board 2023-09.

heading: 4.4 Measurement Results
text: In the end, each annotator’s accuracy was over 91.97% and consistency was over 83.33%, surpassing the threshold we set, which indicates the measurement’s quality is acceptable.
We compared the correlations of fine-grained scores of Measuring Questions and Accuracy Controllable Questions among three annotators by Pearson’s r, shown in Figure 3. Measuring Questions come from Measuring Sentences, which are theoretically unbiased. The results of each sentence are not completely correlative among the three annotators, which means that the 3 annotators with low gender bias have no absolutely fixed gender tendency in cognition. That is in line with our expectation of the theoretical unbiased contexts. Meanwhile, in the Accuracy Controllable Questions, the strong correlation of the results among the three annotators indicates that they have an obvious gender tendency to each sentence, which is also in line with our expectation of Objective Sentences. The results show that our frame and metric conform to the measurement purpose, and the quality of the measurement results is also reliable.
Additionally, annotators attained Krippendorff’s α = 0.045 on Measuring Questions and α = 0.888 on Accuracy Controllable Questions for coarsegrained result. While α of Measuring Questions is quite low as inter-annotator agreement (IAA) is normally measured, we need to argue that: our task is not to annotate the dataset, but to measure CGB of annotators. So, the low IAA of Measuring Questions does not prove that our measurement cannot obtain a high-quality measurement result. Moreover, the high IAA of Accuracy Controllable Questions demonstrates annotators do show agree-
ment on these Objective Sentences. As a result, our design of Accuracy Controllable Questions is a better estimate of measurement quality and reliability.
At last, we calculated the average fine-grained matrix of each annotator as the final score of annotators for each sentence. Then we gave each sentence a coarse-grained label based on the final fine-grained score. For CGB of all Measuring Sentences measured by the annotators, the average fine-grained score is 0.030, and 9,362 sentences (47.32%) labelled ‘Male’, 8,428 sentences (42.60%) labelled ‘Female’, 1,995 sentences (10.08%) labelled ‘Neutral’ for coarse-grained label. It demonstrates that annotators show a slight male tendency in those theoretical unbiased contexts, indicating that manifold behaviors and activities are defaulted by men in our daily life, and our society accepts that masculine hegemony.

heading: 5 Measurement of PLMs
text: We measured CGB of three widely used PLMs based on CGBDataset. We used the default parameters and hyperparameters for each model to set the experiment with a rtx2080ti GPU. The ideal PLM is that performs high accuracy on Objective Sentences with low CGB scores on Measuring Sentences.

heading: 5.1 Measured Models
text: BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), and ELECTRA (Clark et al., 2020) are three widely used PLMs, which have shown good performance on a range of Chinese NLP tasks.
• BERT (Devlin et al., 2019) is pre-trained on Chinese Wikipedia. We chose three models of BERT which can be applied to our Chinese task. BERT-base4 is pre-trained with character masking. BERT-wwm5 (Cui et al., 2020) is pre-trained with whole word masking. BERTwwm-ext6 extends the pre-trained dataset with other news and question-answer data.
• RoBERTa7 (Liu et al., 2019) outperforms other language models by extending the pretrained data and time.
4https://huggingface.co/ bert-base-chinese
5https://huggingface.co/hfl/ chinese-bert-wwm
6https://huggingface.co/hfl/ chinese-bert-wwm-ext
7https://huggingface.co/hfl/ chinese-roberta-wwm-ext
Table 1: Results of CGB measured by different PLMs. We show the accuracy of Objective Sentences (OS) and bias score of Measuring Sentences (MS) measured by PLMs. We also show the standard deviation (SD) of PB(MS).
BERT-base BERT-wwm BERT-wwm-ext RoBERTa ELECTRA Accuracy of OS 0.819 0.809 0.823 0.842 0.502 Bias score of MS 0.540 0.589 0.627 0.570 0.779 SD of PB(MS) 0.697 0.750 0.800 0.750 0.940
• ELECTRA8 (Clark et al., 2020) has the best performance in many Chinese NLP tasks by a new pre-trained method, which is replaced token detection.

heading: 5.2 Measurement Metrics
text: For each sentence S in CGBDataset, each PLM will give a female word probability pf (S) and a male word probability pm(S). Then, CGB score of sentence PB(S) measured by a PLM can be calculated as
PB(S) = log pm(S)
pf (S) (1)
PB(S) represents the CGB degree measured by PLMs for sentence S. Positive value indicates the PLM indexes the sentence towards male, while negative value indicates the PLM indexes the sentence towards female. The large the absolute value of PB(S) is, the CGB degree measured by the PLM is high. When PB(S) is close to 0, the PLM shows neutral in this sentence.
For Measuring Sentences, we calculate the mean of absolute value of PB(S) as the final bias score
8https://huggingface.co/hfl/ chinese-electra-180g-base-discriminator
of each PLM. For Objective Sentences, we label each sentence ‘Male’ or ‘Female’ by PB(S) and calculate the accuracy of each PLM as PLM should obtain the correct gender word inferred from the clues in Objective Sentences. Our assumption is that a good model should get correct answers in Objective Sentences while remain low CGB in Measuring Sentences.

heading: 5.3 Measurement Results
text: Table 1 shows the results of CGB measured by different PLMs. All PLMs express different CGB. RoBERTa shows the best performance on the accuracy of Objective Sentences and BERT-base and RoBERTa outperform other PLMs with the lowest bias in Measuring Sentences. However, ELECTRA shows the lowest accuracy in Objective Sentences while the highest bias score in Measuring Sentences. It indicates that the most efficient PLM ELECTRA shows higher bias. Here, we need to articulate that bias is a kind of heuristics, which is a simple but efficient mind strategy to allow us to make the least effort when we make daily decisions (Myers et al., 2002). Similarly, PLMs take full advantage of human bias to perform very well in many NLP tasks. What we need to be careful about is the harmful consequence PLMs may bring.
However, those harmful biases, especially those over-associations, are very subtle and difficult for both PLMs and annotators to perceive.

heading: 6 Comparing CGB between Annotators and PLMs

heading: 6.1 Quantitative Analysis
text: We compare the average fine-grained CGB score measured by annotators with PB(S) measured by PLMs, shown in Figure 4. CGB score measured by annotators and all PLMs show significant positive correlations, except ELECTRA, which shows an insignificant negative correlation. It indicates that most of PLMs share the same gender bias as annotators in general. Furthermore, RoBERTa, which performs better on accuracy and bias score, also shows more correlation with annotators. ELECTRA, which performs the worst, shows negative correlation with annotators. Notice that the annotators we chose are with low gender bias. It is supposed that the more similar PLMs share with annotators, the less gender bias PLMs will express.

heading: 6.2 Qualitative Analysis
text: Example 1. 商场里的卫生间要人性化很多，更 适合[MASK][MASK]和宝宝。The toilets in the malls are much more humanized, suiting [MASK]
and babies better.
Example 1 shows PLMs and annotators share the same gender bias. They both correlate females with taking care of babies. Here, PLMs have already associated some activities and behaviors with a specific gender, which is consistent with annotators. This gender bias in PLMs might be from annotators when they annotate data and texts containing those representative gender behaviors according to social and culture norms. It can be called representational bias, which arises when language models capture the correlations between a specific gender and a specific concept (Sun et al., 2019).
Example 2. [MASK]司机醉驾超标近三 倍。The drunk driving of [MASK] drivers exceeded the standard by nearly three times.
Example 2 shows PLMs and annotators share opposite gender bias. PLMs learn gender bias from texts they trained on rather than the annotation process by annotators as they show opposite CGB. Society has historically considered male drivers to be the default, so people seldomly mention ‘male drivers’ and always say ‘female drivers’ to emphasize this phenomenon is rare. As a result, the frequency of ‘female drivers’ is much higher than that of ‘male drivers’. PLMs give the opposite answers with annotators by learning this opposite
association. However, in annotators’ cognition, drunk drivers are usually male. People tend not to provide obvious or external information in the process of speech (Grice, 1975). The frequency of describing a situation in the text does not always correspond to the real world, even different from human subjective cognition. This potential difference between reality and text description is defined as reporting bias (Gordon and Van Durme, 2013).

heading: 7 Related Works
text: Gender bias has been found in all fields and tasks of NLP, such as word embeddings (Bolukbasi et al., 2016; Caliskan et al., 2017; Tan and Celis, 2019; Zhao et al., 2019), coreference resolution (Cao and Daumé III, 2020; Rudinger et al., 2018; Zhao et al., 2018), machine translation (Prates et al., 2020; Cho et al., 2019), sentiment analysis (Kiritchenko and Mohammad, 2018), abusive language detection (Park et al., 2018), and so on. Surveys on gender bias in NLP concentrate on how to detect, measure, analyze, and mitigate gender bias in dataset and system (Blodgett et al., 2020; Sun et al., 2019; Garrido-Muñoz et al., 2021). According to the causes, manifestations and forms of bias, several studies have classified bias (Blodgett et al., 2020; Sun et al., 2019; Friedman and Nissenbaum, 1996; Hitti et al., 2019). The detection of gender bias in NLP and the construction of dataset to measure and analyze gender bias always depend on the classification and defining of gender bias (Breitfeller et al., 2019; Zeinert et al., 2021). There have been several datasets to detect and measure gender bias (Kiritchenko and Mohammad, 2018; Rudinger et al., 2018; Zhao et al., 2018; Dhamala et al., 2021), or to mitigate gender bias (Webster et al., 2018), by trained annotators or by crowdsourcing (Nadeem et al., 2021; Nangia et al., 2020; Breitfeller et al., 2019). In Chinese, datasets were built to detect offensive languages (Deng et al., 2022), and social bias (Zhou et al., 2022; Su et al., 2021).

heading: 8 Conclusion
text: We define CGB to measure implicit gender bias in PLMs and annotators. Based on the task of MLM, CGBDataset is constructed to measure CGB of both annotators and PLMs. CGBFrame is introduced to better measure CGB of annotators. Metrics show high-quality of our dataset and framework. We also measure CGB in popular Chinese PLMs and show that they express CGB. Different
reasons can be found when PLMs and annotators share the same or opposite CGB. In the future, different groups of annotators should be included to measure CGB.
Limitations
The current method and dataset exclude non-binary individuals and gender expressions. However, we believe that the dataset can be expanded to include non-binary identities in the future. Due to budget and time constraints, the types and scale of annotators considered in this study are insufficient. In future research, it is hoped that a more diverse group of annotators can be considered. Additionally, this study did not investigate the most popular large language models (LLMs) currently available. It is hoped that in future research, a comparison can be made between PLMs and LLMs in terms of CGB differences.

heading: Acknowledgments
text: This work is sponsored by CCF-Baidu Open Fund 202323 and 2018 National Major Program of Philosophy and Social Science Fund “Analyses and Researches of Classic Texts of Classical Literature Based on Big Data Technology” (18ZDA238). We thank the reviewers for their useful feedback from GeBNLP2024.

heading: A Sentences Filtered Out
text: We manually filtered out those sentences in which gender words do not refer to gender (e.g. ‘他(he)’ in ‘吉他(guitar) ’ does not have the meaning of ‘he’), or those sentences that are inconsistent with their original meanings when the gender word in those sentences are changed into its opposite (e.g. the opposite gender word of ‘女(female)’ is ‘男(male)’, but ‘男(male)’ cannot replace ‘女(female)’ in the expression ‘生儿育 女(bear and raise children)’ ).

heading: B Measurement Metrics
text: For Type 1, we stipulate that the annotators would get 1 to choose a male word, -1 to choose a female word, and 0 to choose a neutral word for
each question. We designed 2 calculation methods, which are fine-grained method and coarse-grained method. Fine-grained method can show the degree of CGB. If an annotator chooses “q” in annotation, which means the two words are the same in the appropriate rank, the calculation is to add the two scores of the annotations. If an annotator does not choose “q”, the Rank 1 option will get 1.5 weights and the Rank 2 option will get 0.5 weight, and then add the two scores. In the end, there are 7 possible scores for fine-grained method, which are -1.5, -1, -0.5, 0, 0.5, 1, and 1.5. Coarse-grained method can only show the bias direction, towards male, neutral or female. It only has three scores, where 0 for neutral, 1.5 for male, and -1.5 for female.
For Type 2, there are still fine-grained method and coarse-grained method. For fine-grained method, there are 5 scores according to the possibilities chosen by the annotators, which are -1.5 and -0.75 for female, 0 for neutral, and 0.75 and 1.5 for male. For coarse-grained method, there are 3 scores, -1 for female, 0 for neutral and 1 for male.
Finally, we calculate the mean score of the finegrained and coarse-grained methods of each annotator as the metric of each sentence. We both keep the fine-grained and coarse-grained metric of each annotator in each sentence. We regard the fine-grained metric as a continuous value, from -1.5 to 1.5, where the negative value means this sentence indexes female, while positive value means this sentence indexes male, and 0 means this sentence indexes neutral. The absolute value of finegrained metric can represent the degree of annotator’s CGB in this sentence. We give each sentence a label as the coarse-grained metric, which includes ‘Male’, ‘Female’, and ‘Neutral’ according to the fine-grained metric.

heading: C Objective Sentences
text: Table 2 shows the 4 types of Objective Sentences and their examples.

heading: Future Work
text: In the future, different groups of annotators should be included to measure CGB. Limitations
The current method and dataset exclude non-binary individuals and gender expressions. However, we believe that the dataset can be expanded to include non-binary identities in the future. Due to budget and time constraints, the types and scale of annotators considered in this study are insufficient. In future research, it is hoped that a more diverse group of annotators can be considered. Additionally, this study did not investigate the most popular large language models (LLMs) currently available. It is hoped that in future research, a comparison can be made between PLMs and LLMs in terms of CGB differences