HHS at S em E val-2023 Task 10: A Comparative Analysis of Sexism Detection Based on the R o BERT a Model (2023)

Abstract: This paper describes the methods and models applied by our team HHS in SubTask-A of SemEval-2023 Task 10 about sexism detection. In this task, we trained with the officially released data and analyzed the performance of five models, TextCNN, BERT, RoBERTa, XLNet, and Sup-SimCSE-RoBERTa. The experiments show that most of the models can achieve good results. Then, we tried data augmentation, model ensemble, dropout, and other operations on several of these models, and compared the results for analysis. In the end, the most effective approach that yielded the best results on the test set involved the following steps: enhancing the sexist data using dropout, feeding it as input to the Sup-SimCSE-RoBERTa model, and providing the raw data as input to the XLNet model. Then, combining the outputs of the two methods led to even better results. This method yielded a Macro-F1 score of 0.823 in the final evaluation phase of the SubTask-A of the competition.

## introduction sexism has long existed in society, causing various asymmetries and inequities, and harming some people. given the enormity of data and the fast-paced flow of information on the internet, addressing sexism still poses a significant challenge @xcite . web technologies are rapidly evolving and automated detection is widely used for sexism, except that most tools only give generic categories without further explanation of what sexism is and do not improve the interpretability, trust and understanding of the tools. in the past, computers were frequently used to conduct application tests for sexism by many people. multiple social media platforms have been used to analyze cyberbullying detection using various topics based on deep learning models and transfer learning @xcite . a method has been proposed that can detect and an-alyze three characteristics of twitter tweets: patriarchal behavior, misogyny and sexism @xcite . @xcite demonstrated an increasing pattern of misogynistic content and users as well as violent attitudes. fine-grained, multi-label sexism account classification has been explored, and the first semi-supervised multi-label sexism account classification has been proposed @xcite . @xcite proposed a new hierarchical taxonomy for online misogyny and produced a related labeled dataset. sexism takes many forms on the web, some are fixed words of sexism, some are new words, and many forms of sexism require contextual, temporal and geographical understanding. thus much of this sexism detection is subjective and relative. this also means that sexism data is hard to find, requiring a lot of manual annotation effort. therefore, we participated in the semeval-2023 shared task 10 (kirk et al., 2023), which aims to identify whether utterances contain sexism against women. in this task, we carried out several works as follows. a) firstly, we tried 5 different models on the dataset to compare the experimental results of several models b) secondly, we also tried to enhance the data using methods such as dropout and pseudo-labeling to improve the macro-f1 scores c) finally, we introduced the focal loss function to analyze its effect in subtask-a, we got the best macro-f1 score of 0.84 for the development phase and a macro-f1 score of 0.823 for the final testing phase, obtaining a final rank of 47. the paper is organized as follows: the model, experimental methods and data are presented in section 2; the experimental details are presented in section 3; the analysis of the results is performed in section 4; and the summary is presented in section 5. ## system overview we use sup-simcse-roberta-base @xcite as the first model and use dropout augmented sexist data as this model input to get the first classification output through the fully connected layer. meanwhile, the classification output of the second model is obtained by using the data without enhancement as the xlnet model input. the final result is obtained by calculating the average of the two outputs. the models used here are all sourced from huggingface foot_0 . the composition is shown in figure 1 . ## results and analysis in this section, we report the results of the models and methods presented in sections 2 and 3 and make a basic analysis. from we finally settled on using the sup-simcse-roberta model and used dropout to input the sexist data twice into the model and finally ensemble the output of the xlnet model(srx). we used this model in the final test phase of subtask-a to get a macro-f1 score of 0.823. its performance was also the best in our experiments. ## conclusion we discuss and compare several sexism detection models with different approaches to try to improve model capabilities and report relevant experimental results. the models still have considerable potential for improvement. therefore, in the future plan, we will continue to try to improve the effectiveness of methods such as pseudo labels. and try some large models such as large roberta to achieve better results.