SSS at S em E val-2023 Task 10: Explainable Detection of Online Sexism using Majority Voted Fine-Tuned Transformers (2023)

Abstract: This paper describes our submission to Task 10 at SemEval 2023-Explainable Detection of Online Sexism (EDOS), divided into three subtasks. The recent rise in social media platforms has seen an increase in disproportionate levels of sexism experienced by women on social media platforms. This has made detecting and explaining online sexist content more important than ever to make social media safer and more accessible for women. Our approach consists of experimenting and finetuning BERT-based models and using a Majority Voting ensemble model that outperforms individual baseline model scores. Our system achieves a macro F1 score of 0.8392 for Task A, 0.6092 for Task B, and 0.4319 for Task C.

## introduction social media has become a free and powerful tool for communication, allowing people to share their ideas and connect with others worldwide. for women, social media has provided a platform for bringing visibility, empowering, and educating people on women's issues. however, with an increase in the number of users, there is a significant increase in sexism on these platforms has led to online harassment and disinformation, making social media less accessible and unwelcoming, continuing to uphold unequal power dynamics and unfair treatment within society. social media's anonymity and ease of access have made it easier for individuals to spread sexist content online. this has led to a growing interest and needs to detect and identify online content as sexist and explain why that content is sexist. natural language processing (nlp) techniques have shown promise in enabling the automated detection of sexist language and providing insights into its nature of it. this paper describes our contributions to semeval 2023 task 10:explainable detection of online sexism @xcite , which proposed a new dataset of sexist content from gab and reddit to detect and explain online sexist content through three hierarchical classification subtasks. task 10 of semeval 2023 includes three hierarchical subtasks: task a, which is a binary classification task to classify whether a post is sexist or not sexist, task b, a multi-class classification task for sexist posts which further classifies them into four categories and task c, another multi-class classification task to classify the sexist posts into one of eleven fine grained vectors. we conducted detailed experiments on bert-based models, such as roberta @xcite and debertav3 @xcite , using different training methods and loss functions to reduce class imbalance along with hyperparameter tuning. we then ensembled these individual models using a majority voting ensemble and show how these methods affect the model's performance and how an ensemble of these models performs better than the individual baselines. ## experimental setup the training data consists of 14,000 entries out of 20,000 total entries, and these are used for task a. 3,398 of these 14,000 are labelled as sexist, which are further used in tasks b and c. we strictly used the data provided by the organisers to train and test our models in the development and testing phases. we used 80% of the training dataset to train the models, while 20% remaining was used for crossvalidation. upon using both methods, we found that the majority voting ensemble method gives better results than the weighted average ensemble method. using the majority voting ensemble reduced individual model errors, and the impact of outliers was also minimized, leading to a more accurate and robust system. for task a, we used our best three performing models for the ensemble, including the baseline de-bertav3 model, with focal loss and different hyperparameter values. the exact hypermeters used for these three models are mentioned in ## results results for task a, b and c are mentioned in tables 6 and 5 as f1 scores. we have presented our results from the development phase in task f1 score task a 0.8392 task b 0.6092 task c 0.4319 for task b and c, focal loss models performed poorly compared to the baselines despite having some imbalanced classes. we also noticed that using a different training method, like 5-fold cross validation improved the performance over the baseline models for multi-class classification and ensembling them further improved the performance. ## conclusion and future works in this work, we benchmarked various pretrained transformer based models such as bert, roberta and debertav3 and their majority voting ensemble models. this detection and classification of online sexism were done under task 10 at semeval 2023-explainable detection of online sexism (edos). we also used focal loss to deal with class imbalance.debertav3 gave the best f1 score of 0.8348 on task a, 0.6381 on task b, and 0.6381 on task c. roberta gave results close to debertav3 on both task b and c. in future works, we would like to (1) compare the results of the models on different datasets and languages; (2) use unsupervised learning to improve the results of our models;(3) propose a novel custom architecture which provides more robust predictions.