Stanford MLab at SemEval-2023 Task 10: Exploring GloVe- and Transformer-Based Methods for the Explainable Detection of Online Sexism
Year: 2023

Abstract:
In this paper, we discuss the methods we applied at SemEval-2023 Task 10: Towards the Explainable Detection of Online Sexism. Given an input text, we perform three classification tasks to predict whether the text is sexist and classify the sexist text into subcategories in order to provide an additional explanation as to why the text is sexist. We explored many different types of models, including GloVe embeddings as the baseline approach, transformer-based deep learning models like BERT, RoBERTa, and DeBERTa, ensemble models, and model blending. We explored various data cleaning and augmentation methods to improve model performance. Pretraining transformer models yielded significant improvements in performance, and ensembles and blending slightly improved robustness in the F1 score.

heading: 1 Introduction
text: Online sexism has the potential to inflict significant harm on women (Ortiz, 2023), and it is a serious issue that must be addressed. With the increasing prevalence of social media, it has become easy for groups of people to spread sexist ideas and threaten the safety of others, with online social networks becoming increasingly inundated by sexist comments (Founta et al., 2018).
There have been numerous previous works on the detection of online sexism as a whole (Schütz et al., 2021; Aldana-Bobadilla et al., 2021; Rodríguez-Sánchez et al., 2020), including even applying these models to non-English datasets (Kumar et al., 2021; de Paula et al., 2021; Jiang et al., 2021). However, almost all of these models do not focus on precisely classifying why a certain text conveys sexist sentiments, and instead provide a binary classification for whether the text is sexist.
*Authors are listed alphabetically. All authors contributed equally to this work.
However, this is the task which is most likely to make these models useful for content moderation, since they provide both the moderator and the platform’s users with a consistent explanation for why something was classified as sexist or not. In many cases, the difficulty of using machine classification when doing content moderation is the perception of it being decided by an arbitrary black box.
While the models may empirically have a high accuracy, due to the high social sensitivity of this task, human intervention is almost always required. With laws such as the General Data Protection Regulation (GDPR) established in Europe, which establishes a "right to explanation"(Hoofnagle et al., 2019), there is thus a huge need for model explainability on top of existing performance optimization (Mathew et al., 2020). With more detailed feedback about the categories of sexism, moderators can efficiently mitigate sexist sentiment online in a robust and rules-based manner, and therefore reduce gender-based violence.
As such, given the increasing importance of explainable detection in machine learning models, we propose and compare several natural language processing methods for doing so. We used GloVeand transformer-based models, as well as various data cleaning and augmentation techniques, applying them on Reddit and Gab textual data to detect sexist messages and classify them into various categories of sexism.

heading: 2 Background and Task Setup
text: The data for this task was provided by SemEval Task 10 (Kirk et al., 2022). This labeled data set consisted of 10,000 entries extracted from Gab along with 10,000 entries from Reddit. The dataset is labeled according to the specifications of the required classifier for subtask A, subtask B, and subtask C. In addition to this labeled dataset, there were two unlabeled data sets which each contain 1 million entries from Gab and Reddit that were
1791
provided, which we used to improve our system’s performance.
Subtask A requires a binary classifier to categorize posts into being sexist or non-sexist. Subtask B requires a four-class classification system which categorizes a sexist post according to one of the following categories: (1) threats, (2) derogation, (3) animosity, and (4) prejudiced discussions. Finally, for subtask C, of the posts which are sexist, an 11-class classification system categorizes the posts according to a more specific label of sexism. Subtasks B and C ensure that text which is labeled as sexist is given a specific label for why it is sexist, providing a degree of explainability for those using the model.

heading: 3 System Overview

heading: 3.1 Data Cleaning
text: The training data was taken from Reddit and Gab, and as such, it was essential to clean the data to get consistent formatting. Furthermore, all URL references were removed, hyphens and hashtags were replaced with spaces, all punctuation except apostrophes was removed, and all text was changed to lowercase. Finally, many slang abbreviations were replaced by their expanded forms using the mapping provided by the sms_slang_translator github repository1.

heading: 3.2 Data Augmentation

heading: 3.2.1 Back Translation
text: For Subtask A, since the provided dataset contained far more "Not Sexist" samples than "Sexist" samples, we attempted to use back translation to generate augmented samples of the minority class. Specifically, we translated the minority samples in our training split to Dutch and then back to English, ensuring an even class distribution in our training split. We chose Dutch because it worked empirically (Beddiar et al., 2021) and since Dutch is one of the most similar languages to English, owing to their joint lineage in the West Germanic family. Results with back translation are specifically labeled in the results section.

heading: 3.2.2 Easy Data Augmentation
text: Since back translation did not improve our results in Subtask A, we attempted a different data augmentation approach for Subtask B: Easy Data Augmentation (EDA). We specifically followed a proce-
1https://github.com/rishabhverma17/sms_slang_translator
dure similar to (Kalra and Zubiaga, 2021). We used three operations–– synonym replacement, random insertion, and random swap with a rate of 0.05––to generate augmented samples of the three minority classes in Subtask B (namely "threats, plans to harm and incitement", "animosity", and "prejudiced discussions") in our training split. Much as with back translation, we generated enough samples for each minority class until the number of samples for each class was equal. Results with EDA are specifically labeled in the results section.

heading: 3.3 GloVe-Based Model
text: For our baseline model for Subtask A, we developed a GloVe-based logistic regression model. Although this is not a state-of-the-art model, it provided a useful benchmark for the performance of non-deep learning approaches. We used 50- dimensional GloVe vectors pre-trained on 2 billion tweets from (Pennington et al., 2014) to transform each word in the input text into its vector representation. For each sample’s input text, we averaged the word vectors across the text to create a 50-dimensional input that we fit with a logistic regression model.

heading: 3.4 Transformer-based Models

heading: 3.4.1 BERT
text: BERT (Bidirectional Encoder Representations for Transformers) is a large-language model that has achieved impressive results in NLP experiments (Devlin et al., 2018). It uses a multi-layer, transformer-based encoder architecture and bidirectional self-attention to learn context from both preceding and following sentences. BERT was trained on a language modeling task as well as a next sentence prediction task.
We fine-tuned the BERT model to apply it to our specific Subtasks. We used the existing pre-trained bert-base-uncased tokenizer to preprocess the text for input the BERT model. We added one linear layer with ReLU activation as well as a second linear layer with the Sigmoid (Subtask A) or Softmax (Subtask B and C) activation function to generate the model output. The ultimate prediction was determined either by threshold (for Subtask A) or the argmax of the output vector (for Subtask B and C). For Subtask A, we found that the optimal threshold for a positive prediction was 0.35, as seen in Figure 1. If the model’s output was 0.35 or greater, the text would be classified as sexist. We added dropout
with a rate of 0.5 to help reduce overfitting.
Figure 1: Positive Prediction Thresholds for BERT Classifier

heading: 3.4.2 RoBERTa
text: RoBERTa (Robustly optimized BERT approach), improves on BERT by using a larger-scale model trained on an even larger and cleaner corpus of text using a longer training schedule, larger batch sizes, and a more advanced masking strategy, resulting in improved performance on a wide range of natural language processing tasks (Liu et al., 2019).
We fine-tuned two separate RoBERTa models using the same architecture as our BERT model. We chose to use RoBERTa because it is trained more robustly using dynamic masking as compared to static masking in BERT, and was proven empirically to yield better results. This is because the generation of masks during training means that for each input, the number of different possible masks is much larger than with BERT.
The two separate RoBERTa models included: one existing RoBERTa model that was fine-tuned to classify sexist tweets (ft-RoBERTa)2, and one RoBERTa model that was pre-trained on the provided unlabeled data (pt-RoBERTa)3.
For Subtask A, we fine-tuned the ft-RoBERTa model. For Subtask B, we fine-tuned both the ftRoBERTa and the pt-RoBERTa models. For Subtask C, we fine-tuned the pt-RoBERTa model. For all three Subtasks, we used the same architecture as the BERT model.
2https://huggingface.co/annahaz/xlm-roberta-basemisogyny-sexism-tweets
3https://huggingface.co/HPL/roberta-large-unlabeledlabeled-gab-reddit-task-semeval2023-t10-270000sample

heading: 3.4.3 HateBERT
text: HateBERT is a model that improves upon BERT for the task of abusive language detection (Caselli et al., 2021). More specifically, HateBERT uses training data with abusive language to better focus BERT’s attention on relevant linguistic cues for this task. It employs a refined pre-training procedure using a large dataset of comments from banned subreddits containing offensive, abusive and hateful language to capture the nuances of this type of speech. Overall, these refinements on domain-specific data have allowed HateBERT to empirically achieve superior performance on various English hate speech detection datasets. We used this only for Subtask B.

heading: 3.4.4 DeBERTa
text: DeBERTa (Decoding-enhanced BERT with disentangled attention) is a model that improves upon both BERT and RoBERTa (He et al., 2020). More specifically, DeBERTa uses disentangled attention to better focus on relevant linguistic information, employs a more advanced decoding strategy that captures long-range dependencies, shares parameters between layers to improve efficiency and performance, and uses improved pre-training strategies to capture complex linguistic relationships. Overall, these improvements have led the DeBERTa model to empirically achieve better performance on various downstream NLP tasks.
For Subtasks B and C, we fine-tuned a DeBERTa model for the respective downstream tasks, using the same architecture as the BERT model.

heading: 3.4.5 Sentence-BERT
text: Sentence-BERT (SBERT) is a modification of BERT, with the key innovation being that SBERT is fine-tuned to encode sentences into fixed-length vectors that capture the semantic meaning of the sentence (Reimers and Gurevych, 2019). Since SBERT focuses on generating high-quality sentence embeddings while BERT generates embeddings on the word level, we experimented with the SBERT-based model in hopes of addressing the overfitting we saw when applying other BERTbased models. For Subtask B, we fine-tuned the SBERT model using the same architecture we used for the other BERT-based models.

heading: 3.4.6 Ensembles
text: In hopes of improving model performance, we experimented with ensemble models that involved
concatenating embeddings from two different transformers into a single feature vector and then applying the linear and dropout layers on top of that feature vector to generate the ultimate prediction. We specifically experimented with two ensemble models: concatenating pt-RoBERTa embeddings with DeBERTa embeddings (referred to as Ensemble 1) and concatenating pt-RoBERTa embeddings with SBERT embeddings (referred to as Ensemble 2). Through concatenating the embeddings, we hoped to capture different aspects of the input text that may be better represented by one model over the other and develop a more comprehensive representation of the input text.
For Subtask B, we applied both Ensemble 1 and Ensemble 2, and for Subtask C, we applied Ensemble 1. For each ensemble model, both transformer embeddings were simultaneously fine-tuned on the corresponding downstream task during model training.

heading: 3.4.7 Model Blending
text: To further improve model performance, we experimented with a blending strategy that involved taking the weighted average between the predictions of two different models, giving slightly greater weight to the better-performing model. For Subtask A, we took the weighted average of the ft-RoBERTa and BERT models, multiplying ftRoBERTa’s prediction by 0.6 and BERT’s prediction by 0.4 and summing the results for the ultimate prediction. For Subtasks B and C, we took the weighted average of Ensemble 1 and the ptRoBERTa model. Similar to Subtask A’s blended model, we multiplied Ensemble 1’s predictions by 0.6 and pt-RoBERTa’s predictions by 0.4 and summed them to generate the ultimate prediction.

heading: 4 Experimental Setup
text: To train and evaluate our models, we used an 80-20 split on the provided training dataset to create training and validation datasets for our models. We used a fixed train-val split so we could directly compare the performance of our models. During training, we monitored the Macro F1 score on the validation set, and our program saved the model with the best score. We trained our models with a maximum of 200 epochs, and we used early-stopping to stop training if the training loss did not decrease over 30 epochs. We trained all of our transformerbased models using the Adam optimizer and crossentropy loss.
For our final submission, we trained our model on all the training data provided.

heading: 5 Results

heading: 5.1 Summary
text: As seen in Table 1, the best-performing model for Subtask A was the weighted-average blending strategy, though the difference in F1 scores between the three best-performing models (BERT, ft-RoBERTa, and Weighted Average) was small. ft-RoBERTa had a slightly better score than the plain BERT model. GloVe Vector-based model yielded the worse performance. The data augmentation we performed, such as back-translation, did not lead to any improvements.
For Subtask B, we see in Table 2 that ptRoBERTa clearly sets itself from the pack out of all transformer-based models, achieving an F1 of 0.62. This is very close to the best results achieved by the models involving multiple transformers, as Ensemble 1 and the Weighted Average model achieved F1s of 0.622 and 0.624, respectively. Like with back translation in Subtask A, the EDA strategy we used for Subtask B failed to lead to any improvements. HateBERT, which is pre-trained on abusive text, beat out most of the benchmark BERT-based models except pre-trained RoBERTa, suggesting that the features it learns from abusive text do not perfectly translate across to sexism.
To prove the importance of data cleaning, we ran an experiment with our pt-RoBERTa model on the uncleaned input text, as seen in Table 2. The resulting F1 score of 0.571 was a significant decrease from the score on the cleaned input text.
Our results in Table 3 show that Ensemble 1 outperforms pt-RoBERTa and the Weighted Average model in Subtask C, which was a surprising difference given the results from Subtask B.
The performance of our final models used for submission on the Dev and Test sets can be seen in Table 4 and Table 5, respectively. Our models’ performances on the Dev set were relatively consistent with our Val set results, but the performance on the Test set represented a noticeable decline, especially for Subtasks B and C.

heading: 5.2 Discussion
text: We see from Subtask A that ft-RoBERTa yielded slightly better results than the plain BERT model. However, since we did not test a plain RoBERTa model and because the difference in performance
is very small, it is difficult to tell if this improvement was primarily due to the improvements of RoBERTa over BERT or the transfer-learning from fine-tuning on the Twitter task.
However, when it comes to pre-training on domain-specific data, we can clearly see that this is vital to improving results. In Subtask B, we see that pt-RoBERTa outperformed all single-transformer models, including DeBERTa. This shows that the success of pt-RoBERTa model can be primarily attributed to the more robust embeddings created after pre-training the RoBERTa embeddings on the unlabeled dataset.
For Subtasks B and C, creating ensemble-type models by concatenating embeddings from different transformer models produced slight improvements. Ensemble 2 was middle-of-the-pack in Subtask B since it involved concatenating embeddings from a poorer-performing transformers mode (SBERT). For Ensemble 1, while the improvement was only slight for Subtask B, there is a clear difference in performance between the pt-RoBERTa model’s F1 score and Ensemble 1’s F1 score for Subtask C. This demonstrates how concatenating embeddings from different transformer models can be an effective strategy for creating more robust representations of the input text.
For Subtasks A and B, blending the predictions of the best-performing models led to slight improvements in performance. However, the improvements were small, and for Subtask C, blending the models did not improve results. This indicates that model blending may not be the most optimal approach to improving model performance.
Weighted Average: BERT & ft-RoBERTa (A)
0.802
Weighted Average: Ensemble 1 & pt-RoBERTa (B)
0.628
Ensemble 1 (C) 0.382
Weighted Average: BERT & ft-RoBERTa (A)
0.798
Weighted Average: Ensemble 1 & pt-RoBERTa (B)
0.573
Ensemble 1 (C) 0.354

heading: 6 Conclusion
text: The three subtasks and classification models for online sexism were able to provide explanations for the model predictions by showing intermediate results of the classification. Although this does not
provide a look inside a black box, it is nonetheless a useful explanation for the end user of the model, explaining the specific reason why a text might be sexist.
From our experiments, we saw that transformerbased models like BERT and RoBERTa worked the best to classify sexism in texts, as seen in Subtask A. Data cleaning was essential in improving our results. Furthermore, pre-training transformers models like RoBERTa on domain-specific text substantially improved the performance, rivaling the multi-transformer models in Subtask B.
The models involving concatenating transformer embeddings produced slightly (Subtask B) to significantly better (Subtask C) results, illustrating how combining information from different transformer models produces better representations. Blending model outputs led to slight performance improvements, but these improvements were small in comparison to the improvements seen from pretraining and concatenation.

heading: 7 Limitations and Future Work
text: In this paper, we chose the pre-trained RoBERTa model as a consistent benchmark across all three tasks. Beyond RoBERTa, we used differing models for different tasks by iterating on their performances in previous stages, introducing new models (e.g. SBERT, DeBERTa, HateBERT etc.) for Task B due to the poor performance of some models in Task A and picking the better-performing models in Task B for Task C.
Given the significant improvements shown in the pre-trained RoBERTa model on the unlabeled data, our system could be further improved if we pretrained more robust models, such as DeBERTa, on the unlabeled data. We would like to explore this direction in the future if we had more computational resources.
Ensembles and model blending led to slight performance improvements. However, there are still many combinations and methods of transformer ensembles and model blending we were unable to experiment with due to time constraints. To build a more robust model, we would systematically experiment with other techniques and combinations of ensembles and model blending.
The data augmentation approaches we attempted failed to lead to better results. This could be due to the fact that the augmented data is not in the distribution of the dataset, or it could show that
our model is slightly overfitted. We should like to explore other augmentation strategies since dealing with minority classes would be key in further improving the macro F1 score of our system. To navigate class imbalance, we would like to experiment with more generative models in addition to data augmentation and weighting methods.
Nonetheless, we believe these results show the potential of using pre-trained transformer models coupled with concatenating embeddings in explainable textual detection.

heading: Acknowledgements
text: This research effort would not have been possible without the support of Stanford ACMLab. We would also like to thank Hannah Rose Kirk, Wenjie Yin, Paul Röttger, and Dr. Bertie Vidgen for organizing SemEval 2023 Task 10: Towards the Explainable Detection of Online Sexism. Furthermore, we would like to thank the reviewers for their insightful comments which strengthened the quality of our paper. Finally, we would like to acknowledge Google Colaboratory for its free computing services.

heading: Future Work
text: We would like to explore this direction in the future if we had more computational resources. Ensembles and model blending led to slight performance improvements. However, there are still many combinations and methods of transformer ensembles and model blending we were unable to experiment with due to time constraints. To build a more robust model, we would systematically experiment with other techniques and combinations of ensembles and model blending. The data augmentation approaches we attempted failed to lead to better results. This could be due to the fact that the augmented data is not in the distribution of the dataset, or it could show that
our model is slightly overfitted. We should like to explore other augmentation strategies since dealing with minority classes would be key in further improving the macro F1 score of our system. To navigate class imbalance, we would like to experiment with more generative models in addition to data augmentation and weighting methods. Nonetheless, we believe these results show the potential of using pre-trained transformer models coupled with concatenating embeddings in explainable textual detection