UMUT eam at S em E val-2023 Task 10: Fine-grained detection of sexism in E nglish (2023)

Abstract: In this manuscript, we describe the participation of UMUTeam in the Explainable Detection of Online Sexism shared task proposed at SemEval 2023. This task concerns the precise and explainable detection of sexist content on Gab and Reddit, i.e., developing detailed classifiers that not only identify what is sexist, but also explain why it is sexism. Our participation in the three EDOS subtasks is based on extending new unlabeled sexism data in the Masked Language Model task of a pre-trained model, such as RoBERTa-large to improve its generalization capacity and its performance on classification tasks. Once the model has been pre-trained with the new data, fine-tuning of this model is performed for different specific sexism classification tasks. Our system has achieved excellent results in this competitive task, reaching top 24 (84) in Task A, top 23 (69) in Task B, and top 13 (63) in Task C.

## introduction the term sexism refers to any abuse or negative sentiment directed at women on the basis of their sex, or on the basis of their sex combined with one or more identity attributes. it should be noted that sexism is a growing problem on the internet and can have harmful effects on women and other marginalized groups, making online spaces less accessible and unwelcoming, which can perpetuate asymmetries and social injustices. automated tools are already widely used today to identify and evaluate sexist content online. it is important that these tools provide specific ratings and explain why content is considered sexist to improve the interpretability, reliability, and comprehensibility of decisions made by automated tools. however, most only provide generic high-level rating without further explanation. thus, the explainable detection of online sexism (edos) shared task @xcite , proposed at semeval 2023, supports the development of english-language models for sexism detection that are more accurate and explainable, with precise classifications of sexist content on gab and reddit. to this end, the task is divided into three hierarchical subtasks: advances in deep learning have had a major impact on natural language processing (nlp), significantly improving the ability of computation to understand and process human language. transformers is a neural network architecture that has had a major impact on nlp and has outperformed many tasks, such as automatic classification, sentiment analysis, and automatic translations (bozinovski, 2020). fine-tuning of pre-trained transformer models has proven to be a highly effective technique for improving performance on a wide range of nlp tasks. by taking the advantage of the model's pre-trained knowledge and adapting it to the specific task, the model can significantly improve its ability to perform the specific task, such as classification tasks. however, most masked language models (mlm) are pre-trained on massive datasets from different domains, so according to @xcite , additional training with the objective mlm on domain or task specific data prior to fine-tuning for the final task is known to improve the final performance. we participate in all proposed edos subtasks using an approach that is based on extending the mlm task of the roberta-large using new unlabeled sexism-related data and then fine-tuning this model for different specific sexism classification tasks. our system has achieved excellent results in this competitive task, reaching top 24 (84) in task a, top 23 (69) in task b, and top 13 (63) in task c. besides, it is worth noting that our research group has previously experience dealing with hatespeech and sexism identification in works focused on spanish @xcite . the remainder of this paper is organized as follows. section 2 provides a summary of important details about the task setup. section 3 offers an overview of our system for the subtasks. section 4 presents the specific details of our systems. section 5 discusses the results of the experiments, and finally, the conclusions are shown in section 6. ## background we only used the dataset given by organizers. the dataset provided consists of 20,000 entries. of these, 10,000 are sampled from gab and 10,000 from reddit. all entries are first labeled by three trained annotators and disagreements are adjudicated by one of two experts. our first experiments consisted of examining the number of examples of each label to see if the dataset is balanced or not. for task b, we only used the sexist posts from task a. sexist post are classified into 4 general categories: (1) threats, plans to harm and incitement, (2) derogation, (3) animosity, and (4) prejudiced discussions. in ## system overview for solving edos shared task, we built a system, which architecture is depicted in figure 2 . in a nutshell, our system works as follows. first, both labeled and unlabeled datasets were pre-processed. second, we trained the roberta-large mlm with the unlabeled datasets provided by the organizers. finally, once we pre-trained the roberta-large with the unlabeled data, we fine-tuned this model with labeled data for different specific sexism classification task. ## experimental setup for the three subtasks of edos shared task, the corpus we used to train the model is from the organizer, without other external data (see section 2). system performance on all tasks (a, b and c) is evaluated using macro f1 score. macro f1 is calculated as an equal-weighted average of f1 scores for each class in a classification task. it is widely used for imbalanced classification tasks because it assigns equal importance to minority and majority classes. to obtain the classification model for each task, we conducted an hyperparameter optimization stage using raytune @xcite with a tree of parzen estimators (tpe) to select the best combination of the hyperparameters over 10 trials. the hyperparameters evaluated, and their interval range are: (1) weight decay (between 0 and 0.3), (2) training batch size @xcite ), (3) number of training epoch ([1-20]), and (4) learning rate (between 1e-5 and 5e-5). in this case, the best configuration for extended roberta-large with new unlabeled data is 15-20 epoch, a training batch size of 16, a weight decay of 0.183556, and a learning rate of 1.186663e-5 for all tasks (a, b and c). ## results each system was evaluated using the validation split and tested using the test split (see section 2). the official results for the task a, b and c are depicted in ## conclusion in this working notes we have described the participation of umuteam in the edos shared task of semeval 2023. in this shared task, the participants were required to perform precise and explainable detection of sexist content on gab and reddit, i.e., developing detailed classifiers that not only identify