MilaNLP at SemEval-2023 Task 10: Ensembling Domain-Adapted and Regularized Pretrained Language Models for Robust Sexism Detection
Year: 2023

Abstract:
We present the system proposed by the MilaNLP team for the Explainable Detection of Online Sexism (EDOS) shared task. We propose an ensemble modeling approach to combine different classifiers trained with domain adaptation objectives and standard fine-tuning. Our results show that the ensemble is more robust than individual models and that regularized models generate more “conservative” predictions, mitigating the effects of lexical overfitting.1 However, our error analysis also finds that many of the misclassified instances are debatable, raising questions about the objective annotatability of hate speech data. Warning: This paper contains examples of language that some people may find offensive.

heading: 1 Introduction
text: Sexism and misogyny are ever-present in online spaces, creating a hostile environment which can prevent women from enjoying the benefits brought about by the internet (Jane, 2017). Because of its ubiquity, tackling sexism manually has proven ineffective, redirecting efforts towards automatic detection of this phenomenon using natural language processing.
Kirk et al. (2023) presented a novel corpus for the classification of sexist content from Gab and Reddit in English using fine-grained labels. The hierarchical classification of sexism permits the development of classifiers that are both more accurate and explicable. In the Explainable Detection of Online Sexism (EDOS) shared task, systems are required to detect sexist content. The task is divided into three sub-tasks that refer to different levels of granularity (see §2).
In this paper, we propose an ensemble modeling approach to combine different fine-tuned language models. Although ensemble modeling has been
1Code available at https://github.com/MilaNLProc/ milanlp-at-edos.
shown to be beneficial for various tasks in NLP (Garmash and Monz, 2016; Nozza et al., 2016; Fadel et al., 2019; Bashmal and AlZeer, 2021), its potential for hate speech detection has only been explored in a small number of papers (Zimmerman et al., 2018; Plaza-del Arco et al., 2019; Ramakrishnan et al., 2019; Nozza, 2022). Our results confirm that ensemble learning improves robustness and classification performance and we find that regularized fine-tuning leads to higher uncertainty in sexism prediction. The resulting “conservative” models reduce both false positives and false negatives when included in the ensemble. Our system ranked 9th for the binary sexism detection task, with a macro F1-score of 0.8616 (-0.013 compared to the winning team).
Moreover, through a deeper analysis on test instances, we find that many examples in the aggregated dataset are likely mislabelled – showing how to best annotate sexism (and other subjective language phenomena) remains an open question.

heading: 2 Task Overview
text: The EDOS shared task (Kirk et al., 2023) requires participants to build and test automatic models for sexism detection for social media posts. The challenge is framed as a classification task and divided into three hierarchical sub-tasks:
A Binary Sexism detection: The goal of the task is to identify whether the post is sexist or not;
B Multiclass Sexism Categorization: Systems are required to assign every sexist post from phase one of four categories: threats, derogation, animosity, prejudiced discussions;
C Fine-grained Sexism Categorization: Similarly to B, systems are required to identify categories, but now among 11 distinct vectors of sexism, e.g., threat of harms, immutable
2067
gender stereotypes, or supporting system discrimination.
This paper presents a system trained to solve task A. Although we did not address tasks B and C specifically, we provide insights into how errors are distributed across their fine-grained categories (§4.1).

heading: 3 Methodology
text: Our proposed method is an ensemble of four finetuned classifiers. We built each classifier under different training regimes, e.g., using a simple finetuned model or running domain adaptation, or enabling regularisation. See §4 for further discussion on the performance of individual models and the benefits of ensembling them.
To produce a single classification label, we average every model’s prediction and use a .5 threshold to assign the SEXIST label.

heading: 3.1 Models
text: We fine-tuned two classes of models: RoBERTa (Liu et al., 2019) and DeBERTa (He et al., 2021). We chose them based on their generalization capabilities (Choshen et al., 2022) and low hurtful sentence completion score (Nozza et al., 2021, 2022). We used HuggingFace model weights, tokenizers, and code implementation (Wolf et al., 2020).
We did not use any additional data and used the same training and hyperparameter search budget for all the models.

heading: 3.2 Fine-Tuning vs. Domain Adaptation
text: We fine-tuned a pre-trained RoBERTa (ROB) as our initial baseline.2
Recent evidence has shown that further pretraining on in-domain data leads to higher transfer learning capabilities (Gururangan et al., 2020). This approach, commonly referred to as Domain Adaptation, uses data from the same downstream task domain.
We ran domain adaptation using the unlabeled Reddit corpus (1M posts) provided by the task organizers (Kirk et al., 2023) and the Gab Hate Corpus (87K posts) (Kennedy et al., 2022). After concatenating and shuffling the two datasets, we held out 5% as validation data, stratifying on the data source. Our final training dataset counted around 20M words. We used an equivalent batch size of
2https://huggingface.co/roberta-large
1,024, weight decay of 10−2, and a learning rate of 10−5 with a linear warmup across the initial 10% steps for a total of six epochs. We monitored the validation loss every 200 steps and used the best checkpoint at the end.
We adapted only DeBERTa (DEBMLM) as it outperformed RoBERTa in our first tests, using the largest checkpoint available.3 We release PyTorch weights to encourage future research.4

heading: 3.3 Regularization
text: Motivated by recent insights on regularization as a way to improve generalization, we tested two recent techniques that add a regularization term to the classification loss.
Entropy-based Attention Regularization (EAR) (Attanasio et al., 2022b) introduces a penalization term function of how attention weights are distributed. Roughly, the term penalizes the network whenever a token’s self-attention weights have a low-entropy distribution. Intuitively, by forcing a higher entropy, EAR forces a stronger contextualization of token representations.
Robust Representations through Regularized Finetuning (R3F) (Aghajanyan et al., 2020) leverages the intuition that fine-tuning procedures should “move” pre-trained representations much to preserve representational and transferability capabilities. In practice, R3F adds a symmetrical Kullback-Leibler penalty term to the classification loss.
We used EAR and R3F to fine-tune RoBERTa (RoBEAR and RoBR3F, respectively).

heading: 3.4 Hyperparameter Setup
text: We used Optuna (Akiba et al., 2019) to optimize learning rate, weight decay, and EAR and R3F regularization strengths (α and λ, respectively) for a maximum of 20 trials using the Tree-structured Parzen Estimator (TPE) algorithm. Table 1 reports both fixed and variable hyperparameters. We held out 5% of the training set as validation, stratifying over class labels, and early stop training with patience equal to 3.
We repeated each fine-tuning setup (including hyperparameter tuning) with five different initialization seeds.
3https://huggingface.co/microsoft/ deberta-v3-large
4https://huggingface.co/MilaNLProc/ deberta-v3-large-mlm-reddit-gab/

heading: 4 Results
text: We compare the results of each of the models on the provided test set. In addition, we experiment with a series of ensemble models: ROB, RoBR3F, and DEBMLM in combination with RoBEAR as well as an ensemble of all the above models. Overall, our ensemble achieves an F1 score of .862. Full results per model are shown in Table 2.
Although RoBEAR was the worst-performing model, it offers modest gains in performance in combination with other models. The best performing model was the ensemble of DEBMLM + RoBEAR.

heading: 4.1 Error Analysis
text: We manually inspected the false positives (FP) and false negatives (FN) of our ensemble model. Considering the false positives and false negatives only, we can see that the majority are centered around dating advice and sexual relationships. A lot of the examples use incel-speech like “alpha/beta male”, “femoid”, “roasties”, “cucks”, etc. This overlap may have introduced noise in the data as several utterances annotated non-sexist were in fact sexist and vice-versa. In particular, many examples where other forms of hate speech were present were labelled not-sexist.
Confidence Score Analysis Given the similarity between the false positives and negatives, we intuit that they may be closer to the 0.5 confidence threshold. We note that the false negatives and false positives generally have a lower confidence score
closer to the 0.5 threshold than the true positives and negatives (see Figure 1). We analyse the confidence scores for each category and model and find that the models produce lower confidence scores for false positives than true positives, and higher for false negatives than true ones.
In particular, RoBEAR produces more conservative scores (closer to 0.5) across the board. This makes intuitive sense: RoBEAR avoids overfitting on particular words. Although on its own it was the worst-performing model, we leverage its conservative nature by testing ensembles with RoBEAR and the other models (shown in Table 2). Although we find modest improvements in F1 score, the twomodel ensemble with DEBMLM +RoBEAR outperforms our all-model ensemble. Table 4 shows examples that were misclassified by a single model but correctly classified when RoBEAR is taken into account. We note that many of these include incel speech and other keywords.
Category We next consider whether some types of sexism (task B) are more difficult to model than others. We expect that, if all categories are equally difficult, we will find a similar distribution of labels in the misclassified examples as we do overall. Table 3 shows our models accuracy in predicting sexist/not sexist for each category. We use accuracy instead of F1 as a metric because the truth labels here are always positive. Prejudiced discussion (e.g. “I’d be willing to end the 19th Amendment to stop that Emily in her tracks.”) seems to be the most difficult category for the models, possibly because it is more scarce than the other categories (94 examples) and it requires more understanding of the world. Similarly, in terms of Task C, backhanded gendered compliments (e.g., “Bet she’d be hot if
she lost all the fat.”) and supporting mistreatment of individual women (e.g., “If she ever reports a man for sexual harrassment, it should be immediately disregarded.”) are rarer in the dataset despite their diversity, with only 18 and 21 examples respectively.

heading: 4.2 A Note on Data Labelling
text: While examining errors we noticed that there were many examples that were (in the authors’ opinions) likely mislabelled. For example, “Greek authorities move 400 ’asylum seekers’ out of crowded Lesbos
camp: [URL]” was labelled sexist likely because of the reference to the Greek island of Lesbos, also commonly used as a slur for lesbians. In contrast, “Jane will marry some sucker, cheat on him, get divorced, ride the CC, and end up fat, miserable and probably a single mother. You are going to be free, happy and successful!" was labelled ‘not sexist’ despite clearly leaning into gendered tropes and showing prejudice and animosity towards women.
This highlights the oft-discussed issue of the difficulty in obtaining reliable annotations for certain phenomena, particularly phenomena that are as pervasive as sexism. Noisy labels can be attributed to annotator disagreement – particularly for subjective phenomena – and sexism has been shown to have the lowest agreement among annotators (Waseem and Hovy, 2016; Cercas Curry et al., 2021). Although the full annotated dataset (without aggregation) has been published, only aggregated labels are considered during evaluation. Without knowing the exact annotation guidelines and process and who the annotators are, we cannot comment further on this but we note that noisy annotations only make modelling a phenomenon harder.

heading: 5 Related Work
text: Hate speech detection has received a lot of interest in Natural Language Processing in recent years but despite the large number of works in the field, only a small subset focuses on the problem of sexism and misogyny detection. Nozza (2021) demonstrates that hate speech detection techniques do not transfer across diverse hate speech targets, highlighting the necessity for ad-hoc sexism and misogyny detection methodologies and datasets. Indeed, the corpora that were made accessible as a part of shared tasks (Fersini et al., 2018, 2020b, 2022; Basile et al., 2019; Mulki and Ghanem, 2021) made it possible for a range of NLP techniques to be taken towards the challenge of automatically detecting instances of sexism and misogyny on Twitter messages (Indurthi et al., 2019; Fersini et al., 2020a; Attanasio and Pastor, 2020; Lees et al., 2020; Attanasio et al., 2022a, among others).

heading: 6 Conclusion
text: We addressed Task A of the EDOS shared task (Kirk et al., 2023) on sexism detection. We compare domain adaptation to a fine-tuned baseline and tested two regularization techniques. We leveraged the capabilities of each model in an ensemble
that outperformed each of the individual models. We show that EAR regularization improves performance on edge cases when used in an ensemble with other models. Our system ranked 9th for the
binary sexism detection task. Based on our findings, future work should experiment with how best to combine adaptation and regularization for robustness, and continue to investi-
gate how best to collect, annotate and model sexism by e.g., modelling annotator disagreement based on unaggregated labels, following the guidelines set out in the Perspectivist Data Manifesto.5 Additionally, interpretability tools must be developed and integrated into models to improve transparency, accountability, and fairness (Attanasio et al., 2022c, 2023).

heading: Acknowledgements
text: This project has partially received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 949944, INTEGRATOR), and by Fondazione Cariplo (grant No. 2020-4288, MONICA). The authors are members of the MilaNLP group and the Data and Marketing Insights Unit of the Bocconi Institute for Data Science and Analysis.

heading: Future Work
text: Based on our findings, future work should experiment with how best to combine adaptation and regularization for robustness, and continue to investi-
gate how best to collect, annotate and model sexism by e. g. , modelling annotator disagreement based on unaggregated labels, following the guidelines set out in the Perspectivist Data Manifesto. 5 Additionally, interpretability tools must be developed and integrated into models to improve transparency, accountability, and fairness (Attanasio et al. , 2022c, 2023)