Brainstormers_msec at S em E val-2023 Task 10: Detection of sexism related comments in social media using deep learning (2023)

Abstract: Social media is the media through which people share their thoughts and opinions. This has both its pros and cons which depends on the type of information being conveyed. If any information conveyed over social media hurts or affects a person, such information can be removed as it may disturb their mental health and may decrease their self confidence. During the last decade, hateful and sexist content towards women in being increasingly spread on social networks. The exposure to sexist speech has serious consequences to women’s life and limits their freedom of speech. Sexism is expressed in very different forms: it includes subtle stereotypes and attitudes that, although frequently unnoticed, are extremely harmful for both women and society. Sexist comments have a major impact on women being subjected to it. We as a team participated in the shared task Explainable Detection of Online Sexism (EDOS) at SemEval 2023 and have proposed a model which identifies the sexist comments and its type from English social media posts using the data set shared for the task. Different transformer model like BERT , DistilBERT and RoBERT are used by the proposed model for implementing all the three tasks shared by EDOS. On using the BERT model, macro F1 score of 0.8073, 0.5876 and 0.3729 are achieved for Task A, Task B and Task C respectively.

## introduction there is a lot of information being spread in social media. these can be categorised as necessary and unnecessary information. any information that helps in the growth of a persona and society is considered as necessary information. any information that is in any way not useful to the society or harms the livelihood and mental health of an individual is considered as unnecessary information which can be removed for the benefit of the society. hateful and sexist content towards women that are increasingly spread in the social media net-works has a serious impact on their life and limits their freedom of speech. an exhaustive analysis is required to understand how sexist behaviour and attitudes prevail in social media platforms @xcite . online violence against women can have a serious impact on them as it may affect their mental health or bring hesitation in voicing out their points in social media. hence it is important to find whether the content that is present in social media is sexist and such content should be removed @xcite . it is necessary to explain the reason behind the classification of text as sexist.thus an explainable classification of sexist comments become necessary. by silencing or pushing women out of online spaces, online violence can affect the economic outcomes of those who depend on these platforms for their livelihoods foot_0 . it can also lead to loss of employment and societal status, in cases where online violence impacts their reputation. at this time, although research is beginning to delve into the incidence of sexist content in social media, experimental research examining the effects of these messages is scarce. hashtags on twitter are used to establish topical links. by using hashtags before words or phrases in their posts, users categorise tweets. by searching hashtags, users can find tweets relevant to a specific topic posted by other users on twitter. thus, twitter enables users to disseminate their tweets publicly, and other users can easily track and view those tweets. on twitter, several sexist hashtags have gone viral. some sexist hashtags, such as #getbackinthekitchen, #liestoldbyfemales and #ihatefemaleswho, promote stereotypes and hostility toward women. others, such as #rulesfor-girls and #mygirlfriendnotallowedto, suggest that men are responsible for regulating women's behaviour and that women should submit to male authority. some of the most appalling, such as #thatswhatslutsdo and #itsnotrapeif, promote rape myths and dehumanize women as objects whose only function is sex. not just twitter, this type of behaviour can be seen in many other social media platforms as well. in order to avoid the negative impacts such kind of behaviour has, it is necessary to find such comments and remove them @xcite . it is equally necessary to explain why this action must be taken. this task aims in identifying the content that are sexist and explains why it is marked as sexist. the shared task explainable detection of online sexism (edos) @xcite was a part of semeval 2023 which is based on english comments. the task contains three hierarchical subtasks namely task a, task b and task c. ## data set the data set that are used to implement sexism detection was the training, evaluation and the test dataset that were provided by the organisers of the shared task. each instance of the training dataset had the following informations attached to it: the test data had 4060 instances for which the predictions had to be done using the proposed model. task b was supported with training dataset which had 310, 1590, 1165 and 333 instances under the categories (i) threats, plans to harm and incitement, (ii) derogation, (iii) animosity and (iv) prejudiced discussions respectively. the training dataset of task c had classifed the categories identified in task b into 11 different subcategories. ## system description initially the three dataset provided by the task organizers, namely training dataset, development dataset and testing dataset were collected. training dataset is preprocessed where unnecessary digits, characters and white spaces are removed using tokenization and it is followed by an encoding process. then the model is created. in this system, three models were used namely bert, distilbert and roberta. the preprocessed dataset along with the model created is used for the training phase. each model is then evaluated using a development dataset. the bert model that provided the highest accuracy is taken as final run for submission and was used to find the predictions for the testing dataset. the proposed architecture is represented in figure 1 . the removal of unnecessary information is taken care by the preprocessing phase. all the three sets of data namely training, evaluation and test dataset are preprocessed. this is followed by the process of model building, where pretarined models namely bert, distilbert and roberta were used. in the training phase the pretained models are trained using the preprocessed training dataset. the evaluation of the trained model is carried out in the evaluation phase using the evaluation dataset which makes use of the accuracy as the parameter of evaluation. fine tuning of hyper parameters are performed to improve the accuracy of the proposed system. the labels for the text in the dataset are predicted during the testing phase. contextual embeddings are generated and are used during the training of the model. ## results the metrics that was considered for the evaluation of all the three tasks was macro-f1 score. f1 score the values of the performance metrics namely f1 score and accuracy obtained for different models are shown in all the three tasks were evaluated based on the macro f1 score obtained by the proposed model. the proposed model resulted in an macro f1 score of 0.81 based on which task a was evaluated and we were ranked 61 on the leader board. the accuracy obtained for task a was 0.87. task b had resulted in a macro-f1 score of 0.59 and accuracy obtained was 0.61. we ranked 49 on the leader board under this category. macro-f1 score and accuracy achieved by the proposed model for task c were 0.37 and 0.53 respectively and we obtained 39th rank on the leader board. ## error analysis the f1 score obtained for task a, task b and task c using the proposed bert model shows that more false positive and false negative classification has occurred. one reason for this could be considered as the data imbalance nature of the dataset. considering task a, the number of instances for the class labelled non-sexiest is higher, and the f1 score, precision, and recall associated with this class are high when compared to the class sexiest. the same could also be found with tasks b and c. this represents that the number of misclassifications increases when the number of instances for training is lower, which is associated with data imbalance. data augmentation could be considered sno. text ## conclusions sexism detection has become an important area of research as it is interlinked with different areas of application that includes sentiment analysis, opinion mining, offensive and hate speech detection. having this in mind semeval 2023 had come up with the task of sexism detection which was represented by three tasks namely sexism detection, identifying the categories of sexism related text and identifying whether the text or its rephrase is sexism. we have applied binary classification for task a, b and c. and we have used bert transformer model for training and testing the dataset. all the three tasks were implemented considering the language english. dataset for sexism detection could be created with contextual information which can help in effectively detecting sexism. usage of hybrid approaches where different deep learning models are combined can also facilitate efficient detection of sexism from text. often it could be observed that sexism is not in the text, but could be detected from the intonation or facial expression, which has made mulitmodal sexism detection also as a promising research area.