H ausa NLP at S em E val-2023 Task 10: Transfer Learning, Synthetic Data and Side-information for Multi-level Sexism Classification (2023)

Abstract: We present the findings of our participation in the SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS) task, a shared task on offensive language (sexism) detection on English Gab and Reddit dataset. We investigated the effects of transferring two language models: XLM-T (sentiment classification) and HateBERT (same domain - Reddit) for multilevel classification into Sexist or not Sexist, and other subsequent sub-classifications of the sexist data. We also use synthetic classification of unlabelled dataset and intermediary class information to maximize the performance of our models. We submitted a system in Task A, and it ranked 49th with F1-score of 0.82. This result showed to be competitive as it only under-performed the best system by 0.052%F1-score.

## introduction sexism is a form of written or verbal attack on women based on their gender and other identity @xcite . in general, there is a global concern about the prevalence of hate on social media. consequently, many studies have attempted to ensure that the social media remain safe for everyone @xcite . in this paper, we described our experiments for the semeval-2023 task 10: explainable detection of online sexism (edos). the shared task was divided into 3 sub-tasks that are all aimed at predicting fine-grained information about the type of sexism that exists on social media sites of gab and reddit. more information is provided in section 3 and the task description paper @xcite . our models were fine-tuned on two pre-trained language models, namely xlm-t @xcite and hatebert @xcite . while xlm-t is a pretrained language model trained on 198 million tweets, the hatebert on the other hand is a pretrained language model based on bert @xcite that was pretrained on a large scale english reddit dataset for the detection of abusive language. we used only the dataset used by the task organizers in our experiment. see @xcite for a description of the dataset. we made submission only to the task a during the competition phase, and our system achieved a competitive performance of 0.82 f1-score. however, we also described our attempts at creating competitive models for tasks b and c. the main contributions of this paper are as follows: ## related works there is an abundance of literature on the detection of hate speech online. however, only a fraction of such studies focused on sexism detection @xcite . @xcite proposed a chinese dataset and lexicon for the detection of sexism on social media. the proposed dataset and lexicon were created from comments and posts collected on the sina weibo chinese microbloging site and annotated into sexism or non-sexism. those labelled as sexism were further annotated into four categories. finally, each of these categories was labelled according to the target. preliminary results show that context-based models outperform linguistics-based models. waseem and hovy (2016) created a dataset of 16,000 tweets, of which 3,383 tweets were sexist. character n-gram feature was investigated alongside others such as gender and location. the best result was obtained with the combination of char n-gram and gender. at the ibereval 2018 task for the identification of misogyny from spanish and english corpora, canós (2018) used tf-idf features vectors and svm with a linear kernel to develop a system that classifies tweets as misogynistic or not misogynistic for subtask a, and the type and target of the misogynistic tweet for subtask b. the best result was obtained from the spanish dataset for both tasks. another study used tf-idf, user-based, network-based, and text-based features together with classical and deep learning algorithms to automatically detect sexism in tweets. logistic regression (lr), support vector machine (svm) and random forest (rf) were used for the classical machine learning. bidirectional long-short-term memory (bi-lstm) and multilingual bidirectional encoder representation from transformers (mbert) from the deep learning algorithms. the mbert with text features gave the best performance with an accuracy of 0.74. ## task overview the semeval-2023 subtasks aim to create models for the detection of sexist post from gab and reddit. there are three subtask which include: task aa binary classification of statements as either sexist or not; task b -classifies sexist statements into four groups namely, threats, derogation, animosity and prejudiced discussions; task c -involves classification of the sexist statements into 11 fine-grained vectors. this is illustrated in figure 1 . ## system overview in this section, we will elaborate on the main methods for the binary sexism task. ## conclusion in this system description paper, we describe our submission for the subtask a binary classification of comments into sexist and not sexist submitted to the semeval-2023 task 10 -explainable detection of online sexism (edos). we implemented pretrained models using xlm-t and hatebert on the english language twitter. our model achieved competitive result of 82% using f1 score, slightly below the leader with 87%. however, this performance is not exhaustive as we observe great imbalance in the distribution of the dataset and this could have influenced the results. furthermore, we described our attempts at building competitive models for tasks b and c (which we did not submit to the competition). we utilised synthetic data and parent class information to improve the performances of the two models in the respective tasks, and some improvements were observed. finally, we intend to improve the performances of these models by targeting the data imbalance constraint using data augmentation strategies.