UL & UM 6 P at S em E val-2023 Task 10: Semi-Supervised Multi-task Learning for Explainable Detection of Online Sexism (2023)

Abstract: This paper introduces our participating system to the Explainable Detection of Online Sexism (EDOS) SemEval-2023 - Task 10: Explainable Detection of Online Sexism. The EDOS shared task covers three hierarchical sub-tasks for sexism detection, coarse-grained and fine-grained categorization. We have investigated both single-task and multi-task learning based on RoBERTa transformer-based language models. For improving the results, we have performed further pre-training of RoBERTa on the provided unlabeled data. Besides, we have employed a small sample of the unlabeled data for semi-supervised learning using the minimum class-confusion loss. Our system has achieved macro F1 scores of 82.25\%, 67.35\%, and 49.8\% on Tasks A, B, and C, respectively.

## introduction the advent of diverse social media platforms has enabled both identified and anonymous users to express their views and beliefs freely and openly. however, a number of online users leverage this freedom of expression to abuse other people or a group of people based on their race, ethnicity, culture, religion, gender, and political orientation, to name but a few @xcite . to tackle this issue, social media platforms rely on content moderation techniques to review and filter their users' posts @xcite @xcite . during the past decade, researchers have shown an increased interest in building artificial intelligence-based tools and applications to address the challenges of content moderation automation in social media platforms. for instance, various natural language processing (nlp) research works have been introduced for offensive language @xcite , hate speech @xcite , cyberbullying @xcite @xcite , toxicity @xcite , misogyny @xcite and sexism @xcite detection. the explainable detection of online sexism (edos) shared task introduces three sub-tasks for sexism detection and categorization in the english language @xcite . the aim is to develop nlp models and systems for sexism detection and explainability. it emphasizes improving predictions' interpretability and explainability for fair content moderation decision-making. the edos shared task provides training data for developing an accurate and explainable model for sexism detection using a set of hierarchical labels and tasks. in this paper, we present our participating system to the edos shared task. we have explored both single-task and multi-task learning models. all our models rely on roberta pre-trained transformerbased language model @xcite . to improve the performance of our models, we have conducted domain-adaptive pre-training using the shared task unlabeled data. to do so, on the one hand, we have employed the whole word masking pre-training objective @xcite . on the other hand, we have performed semi-supervised training using the provided labeled data and a sample of 28k entries from the unlabeled data, predicted as sexist texts by our st_3 model (section 3.3), on task b and task c. the minimum class confusion (mcc) loss @xcite is then used to train our model on the sampled unlabeled data. for the official evaluation, we have submitted the results of our multi-task learning model which is trained in a semi-supervised manner and utilizes our adapted roberta-large encoder. our system has achieved macro f1 scores of 82.25%, 67.35%, and 49.8% on tasks a, b, and c, respectively. the explainable detection of online sexism (edos) shared task covers three sub-tasks for sexism detection and categorization in the english language @xcite . the aim is to develop models that detect sexist content and explain why it is sexist. it addresses the challenges of developing an accurate and explainable model for sexism detection using a set of hierarchical labels and tasks. the shared task data is collected from gab and reddit. the task-labeled dataset consists of 20k data instances (entries), where half of the entries are sampled from gab and the other half from reddit. the original dataset is split following the 70/10/20 proportions for training/validation/test sets. in addition to the labeled dataset that is provided for models training on the downstream tasks, the organizers have supplied two unlabelled datasets containing 2m text entries (1m entries from gab and 1m from reddit). ## system overview in this section, we present the employed text encoders, the pre-training procedure, and our model architectures. ## experimental setup all our models are implemented using pytorch foot_0 deep learning framework, pytorch lightning foot_1 , and hugging face transformers foot_2 library. we have conducted our experiments using dell poweredge c4140 server, having 4 nvidia v100 sxm2 32gb. we have performed domain adaptive pre-training using a learning rate of 5 × 10 -5 and a batch size of 8 per gpu device. the number of epochs is fixed to 3. the other hyper-parameters are fixed to their defaults values of the employed pre-training script 4 . for model fine-tuning on the edos sub-tasks, we have fixed the learning to 1 × 10 -5 , the dropout to 0.2, the maximum sequence length to 128, and the number of epochs to 10. the batch size is fixed to 16 for all sub-tasks. all models have been trained on the training set and validated on the provided development set. we have performed model evaluation using the macro averaged recall, precision, and f1-score. ## results in this section, we describe the obtained results of our models as well as our official submissions. task a on the one hand, for single-task learning models, the obtained results on task a show that robertalarge model (st_2) outperforms its base variant (st_1). besides, domain-adaptive pre-training (st_3) improves the performance of our model. on the other hand, the obtained results using multitask learning models demonstrate that using taskspecific attention layers (mt_2) enhances the performance of our system. although the mcc training objective is minimized on tasks b and c, the best f1-score on task a is obtained using the mt_3 model. the best f1-score performance is obtained using mt_3 model that minimizes the mcc loss on task b, and task c. ## conclusion in this paper, we have introduced our participating system to the explainable detection of online sexism (edos) shared task. in order to improve the performance of our model, we have explored domain-adaptive pre-training and semi-supervised learning leveraging the provided unlabeled data. overall, we have assessed the performance of three single-task learning models and three multi-task learning models. the overall evaluation results show that our system has achieved promising results on task b and task c. it is ranked 11th, and 9th on task b, and task c, respectively.