Pro-Woman, Anti-Man? Identifying Gender Bias in Stance Detection (2024)

Abstract: Gender bias has been widely observed in NLP models, which has the potential to perpetuate harmful stereotypes and discrimination. In this paper, we construct a dataset GenderStance of 36k samples to measure gender bias in stance detection, determining whether models consistently predict the same stance for a particular gender group. We find that all models are gender-biased and prone to classify sentences that contain male nouns as Against and those with female nouns as Favor. Moreover, extensive experiments indicate that sources of gender bias stem from the fine-tuning data and the foundation model itself.

heading: 1 Introduction
text: The prevalence of unintended social biases in NLP models has been recently identified as a major concern for the field (Caliskan et al., 2017; Chang et al., 2019; Sun et al., 2019; Blodgett et al., 2020; Stańczak and Augenstein, 2021; Thakur et al., 2023). These biases have been found in many subtasks of NLP, ranging from learned word embeddings (Brunet et al., 2019; Dev et al., 2020; Valentini et al., 2023), coreference resolution (Rudinger et al., 2018; Zhao et al., 2018; Cao and Daumé III, 2020), natural language inference (He et al., 2019; Sharma et al., 2021; Anantaprayoon et al., 2023), dialog (Dinan et al., 2020; Zhou et al., 2022; Sicilia and Alikhani, 2023) and machine translation (Stanovsky et al., 2019; Savoldi et al., 2021; Attanasio et al., 2023).
Stance detection aims to identify the attitude (e.g., Favor, Against or None, etc.) of a given text with respect to a specific target of interest (Li and Caragea, 2019; Küçük and Can, 2020; AlDayel and Magdy, 2020; Li et al., 2023b; Zhao et al., 2023; Liu et al., 2023), which can provide valuable insights into decision-making (Li et al., 2021) (e.g., presidential elections, marketing strategies and social media monitoring, etc.). Despite a plethora of
studies showing presence of systematic gender bias in prolifically applied NLP methods, little attention has been paid to the role of gender in stance detection. Schiller et al. (2021) demonstrate that most datasets inevitably inherit the biases of their annotators and overfitting on these dataset biases can result in low robustness in stance detection models. Kaushal et al. (2021) identify the dataset biases as potential spurious correlations of sentiment-stance relations and target-independent lexical choices associated with stance. However, these studies have not taken gender bias into consideration. As compared to previous work, we explore whether stance detection systems tend to associate the stance label with a certain gender, predominantly supporting or opposing the opinions of a certain group of people, and thereby negatively impacting the decision-making.
To identify gender bias as well as understand how it arises in stance detection, we construct a challenging dataset, GenderStance, to explore the predictive differences of models on samples that differ only by gender. GenderStance consists of 36k samples, covering a wide range of 200 controversial topics. Experimental results indicate that state-of-the-art models (Allaway and McKeown, 2020; He et al., 2022; Li et al., 2023a,c; Zhang et al., 2023) are all gender-biased in stance detection, inclined to label sentences containing male nouns as Against and label those with female nouns as Favor. Moreover, we explore how bias can enter into stance detection systems. Results suggest that sources of bias stem from the fine-tuning data and the foundation model itself. To our knowledge, we are the first to evaluate gender bias in stance detection. We argue that current stance detection systems run the risk of making unlicensed inferences, with inherent gender bias possibly resulting in the underrepresentation of different gender groups (Stańczak and Augenstein, 2021; Mehrabi et al., 2021). Our findings highlight the importance
3229
of incorporating gender fairness into the design and evaluation of stance detection systems.

heading: 2 Measuring Gender Bias

heading: 2.1 Problem Formulation
text: Can gender prejudice be observed in current stance detection systems? To evaluate this, we construct a challenging dataset GenderStance that includes two evaluation sets differing only in the gender nouns they contain. Formally, suppose a given training set Ds= {(xsi , tsi , ysi )}Nsi=1 and two gender evaluation sets Dm= {(xmi , tmi )} Ng i=1 and D f= {(xfi , tfi )} Ng i=1, where m and f represent the male and female genders, respectively, xi is a sequence of words, ti is the target and yi ∈ {Against, Favor, None} is the stance label. For a model trained on Ds, we define gender bias as the difference in its stance predictions on Dm and Df .

heading: 2.2 The GenderStance Corpus
text: Here, we introduce how to construct a dataset of 36k sentences to determine whether stance detection models consistently make the same stance prediction to sentences involving a particular gender.
First, we create a domain list by extracting 20 pre-defined categories from Kialo1, which is a structured online debate platform where users provide supporting and opposing claims for each controversial topic. Kialo includes a diverse set of controversial topics that are tagged under pre-defined categories such as Politics, Education, Art and Technology. The complete domain list is shown in Table 1. Subsequently, we create an initial set from Kialo by selecting 10 controversial topics from each domain, along with one supporting and one opposing claims corresponding to each controversial topic (Durmus et al., 2019).
Second, in terms of labels Favor and Against, we create a subset of 24k samples, each with the following structure: “Text: [GEN] believe(s) that [CLAIM]; Target: [TOPIC]”, where [GEN] corresponds to a male or female noun phrase, examples of which are shown in Table 2, [TOPIC] represents a controversial topic2, and [CLAIM] is one of supporting or opposing claims obtained from the previous step. Given that sentences only differ in the gendered noun phrases they contain, the model should make identical predictions towards the tar-
1https://www.kialo.com/tags 2More details of topics are discussed in Appendix B.
get for both males and females. Some examples of GenderStance are shown in Table 3.
Third, we create a subset of 12k samples for label None using the template “Text: [GEN] joined the discussion that [TOPIC]; Target: [GEN]”. Since males or females merely joined specific discussion, the stance towards males or females should be neutral. Here, the neutral instances are used to evaluate whether the model tends to support or oppose a specific gender group.
Our dataset is balanced across genders and has 30 noun phrases for each gender, leading to a total of 36k samples (20 categories × 10 topics × 3 stance labels × 30 noun phrases × 2 genders). The rationale behind our selection of gendered noun phrases is to include a variety of gender distribution characteristics, covering 10 common usages (Caliskan et al., 2017; Kiritchenko and Mohammad, 2018), 10 gender-dominated occupations (Haines et al., 2016; Bhaskaran and Bhallamudi, 2019) and 10 gender-dominated majors (Robnett, 2016; Tellhed et al., 2017). The complete pairs of noun
phrases are shown in Appendix A. We open-source the GenderStance dataset3.

heading: 3 Experimental Settings

heading: 3.1 Datasets
text: The gender bias evaluation was carried out on the models that are trained on two benchmark datasets in stance detection, including Varied Stance Topics (VAST) (Allaway and McKeown, 2020) and SemEval-2016 (Mohammad et al., 2016). VAST includes news comments from the The New York Times that contains a large number of targets from multiple domains. SemEval-2016 is composed of tweet-target pairs centered around six targets, namely Atheism, Climate Change is a Real Concern, Feminist Movement, Hillary Clinton, Legalization of Abortion and Donald Trump. Training, validation and test sets of zero-shot setting are used as provided for VAST dataset. For SemEval-2016, we split the training set of first five targets into training and validation sets using an 85/15 split and test on the last target Donald Trump. The statistics of original VAST and SemEval-2016 datasets are shown in Table 4.

heading: 3.2 Evaluation Metrics
text: We calculate the F1 for each class and adopt the macro-average F1 of all classes as the evaluation metric for zero-shot evaluation on VAST and SemEval-2016, which is consistent with previous work (Allaway and McKeown, 2020; Li et al., 2023c).
We define two additional metrics to measure the degree of gender bias within stance detection models:
• ∆F1: This represents the difference in macroaverage F1 between genders, defined as (F1male-F1female). A higher value serves as an indicator of high bias. We compute ∆F1a and ∆F1f for the Against and Favor labels, respectively.
• ∆P : This represents the difference in the proportion (%) of model predictions on the specific label, defined as (Pmale - Pfemale). A higher value is the indicator of high bias. We compute ∆Pa and ∆Pf for the Against and Favor labels, respectively.
∆P reflects the tendency of model predictions on stance labels, while ∆F1 indicates the impact of
3https://github.com/chuchun8/GenderStance
this tendency on F1 measure. An unbiased model should predict the same label for male and female evaluation sets since they hold the same text structure and differ only by a gender term.

heading: 3.3 Baselines
text: BERT (Allaway and McKeown, 2020) encodes the text-target pair with the BERT model (Devlin et al., 2019), and then perform classification with two fully-connected layers. RoBERTa represents the vanilla RoBERTa-base model (Liu et al., 2019) for stance classification. WS-BERT (He et al., 2022) utilizes the BERT as the base model and encodes Wikipedia knowledge in addition to the text-target pair for classification. KASD (Li et al., 2023a) employs the RoBERTa as the encoding module and proposes a knowledge-augmented framework that infuses both episodic knowledge and discourse knowledge for stance detection. TTS (Li et al., 2023c) employs a teacher-student learning framework that improves target diversity by assigning pseudo stance labels to the augmented targets. We evaluate gender bias with the above baselines that are trained on VAST and SemEval-2016 datasets. In addition, GPT-3.5 (Zhang et al., 2023) and GPT4 are strong zero-shot baselines that directly predict the stance label based on a task description, which are directly applied for measuring the gender bias.

heading: 3.4 Training Settings
text: In our work, we performed all experiments on a single NVIDIA RTX A6000 GPU. The learning rate of baselines is set to 1e-5. AdamW (Loshchilov and Hutter, 2019) is utilized as the optimizer. The model is trained for 4 epochs with early stopping and the patience is 5. We utilized the gpt-3.5-turbo1106 version of GPT-3.5 and gpt-4-1106-preview of GPT-4 for zero-shot evaluations.

heading: 4 Experimental Results
text: The main results of gender bias evaluations are shown in Table 5. Each result is the average of three runs with different initializations. First, we observe that all the stance models tested by us are indeed gender biased. Notably, all models predominantly classify samples containing male nouns as Against and those with female nouns as Favor, as indicated by the positive ∆Pa and negative ∆Pf values. In addition, the non-zero values of ∆F1 for each gender indicate that this tendency has contributed to a large performance gap in stance detection. Positive ∆F1a values and negative ∆F1f values demonstrate that models generally make more accurate predictions on Against with male nouns and Favor with female nouns, which poses representational harm to both gender groups.
Second, Table 5 shows that gender bias varies greatly across models trained on the same dataset, which underscores the substantial impact of the model architectures on bias manifestation. Specifically, WS-BERT and KASD outperform BERT and RoBERTa in the vast majority of the cases, respectively, highlighting the benefits of incorporating external knowledge. Moreover, GPT-3.5 exhibits a high gender bias in the zero-shot setting, confirming the prevalence of gender bias in stance detec-
tion models. Impressively, GPT-4 demonstrates the lowest bias on GenderStance, suggesting GPT-4’s advanced capability to overcome inherent biases.
Third, in terms of training data, although both VAST and SemEval-2016 show similar trends with respect to most metrics, results from Table 5 show that models fine-tuned on SemEval-2016 demonstrate higher bias than those trained on VAST, as evidenced by the higher absolute average score for each metric. This indicates the issue of selection bias (Hovy and Søgaard, 2015; Hovy and Prabhumoye, 2021), a source of bias that is rooted in the data chosen for training models.
To gain a deeper insight into the bias introduced by the training set, we propose a simple rule-based approach to balance noun phrases for each gender within the training data of SemEval-2016. We first identify the gendered terms in each sample with a list of gender pairs and then insert their opposites (e.g., “he” ⇔ “she”) at a random position within the sample. From results in Table 6, we can see that maintaining gender balance in the training set effectively reduces gender bias, while simultaneously achieving comparable or superior macro-average F1 scores on the original dataset (SemEval-2016). The attenuation of gender bias indicates that the gender-balanced training set is important to mitigate gender bias in stance detection.

heading: 5 Implications of Gender Bias in Stance Detection
text: We have so far evaluated gender bias of different models on our GenderStance dataset. In this section, we briefly outline potential implications of our findings in the area of stance detection. First, the model’s behavior misrepresents gender groups by associating male or female nouns with specific stances. This misrepresentation can distort the por-
trayal of genders, suggesting that men are more likely to oppose controversial topics while women are more likely to support them. Such a skewed portrayal contributes to reinforcing gender stereotypes.
Second, since most models run the risk of making biased inferences, individuals might potentially influence the decision-making process, such as altering the support or opposition rates of a particular policy, by purposefully spreading posts with male or female nouns online.

heading: 6 Conclusion
text: In this paper, we construct a dataset GenderStance to test the presence of gender bias in state-of-theart models of stance detection. We consider three groups of models and evaluate them using our dataset. Results show that all models tend to associate gender terms with the stance label, leading to biased predictions. The data used for fine-tuning can be seen as the potential source of gender bias. In addition, stance detection models also contribute to the propagation of gender bias, thereby resulting in unfair treatment of male and female groups. Our dataset can add to the spectrum of NLP benchmarks for evaluating gender bias on relevant application tasks.
Limitations
One limitation of our dataset is that it focuses only on the English language. However, we are keen to expand our dataset to a multilingual setting, including languages such as Chinese and German in future work. Second, we only consider the gender bias in this paper. However, various biases such as race and age may also have a negative impact on the stance detection systems. Third, our dataset only covers binary gender in this paper. We fully acknowledge the importance of including non-binary gender groups in future work. Fourth, the primary goal of this paper is to identify gender bias in stance detection. Consequently, the exploration of debiasing techniques is beyond the scope of this paper and is designated as a promising area for future research.
Ethical Statement
We gather targets and texts solely based on category names and topics from a public debate website. We ensure ethical integrity by not including any useridentifiable information in our constructed dataset.
Besides, it is very important to consider the ethical implications of stance detection systems. As we can see that these systems are gender-biased, and thus a potential harm is that these systems may make incorrect predictions and further mislead the decision-making. Researchers should be aware of potential harms from the misuse of stance detection systems.

heading: Acknowledgements
text: We would like to thank anonymous reviewers for their insightful comments to help improve the paper. Also, we would like to thank Chenye Zhao from University of Illinois at Chicago for helping us run experiments with close-sourced LLMs. This work is supported by the National Natural Science Foundation of China (NSFC) Key Project under Grant Number 62336006, the Pioneer and “Leading Goose” R&D Program of Zhejiang under Grant Number 2022SDXHDX0003 and the Ministry of Science and Technology of China Key Project under Grant Number 2022YFE0204900. Yue Zhang is the corresponding author.

heading: A Pairs of Noun Phrases
text: The complete pairs of noun phrases used in our GenderStance are shown in Table 7.

heading: B Controversial Topics
text: As discussed in Section 2.2, we selected 200 controversial topics from Kialo to construct our dataset. Originally, these topics were presented as claims rather than noun phrases. However, the targets in
most prior work of stance detection are typically in the form of noun phrases (Mohammad et al., 2016; Allaway and McKeown, 2020; Glandt et al., 2021; Li and Caragea, 2023). Therefore, to be consistent with previous work, we manually transformed these claims into noun phrases to serve as targets for the labels Favor and Against. For example, the original claim “executions should be painful” is reformulated as “painful executions”, as shown in Table 3. For the label None, we still use claims as [TOPIC]. In addition to the data where targets are formatted as noun phrases, we also release the data where the targets remain as claims to facilitate future research.

heading: Future Work
text: In addition to the data where targets are formatted as noun phrases, we also release the data where the targets remain as claims to facilitate future research