CNLP-NITS at SemEval-2023 Task 10: Online sexism prediction, PREDHATE!
Year: 2023

Abstract:
Online sexism is a rising issue that threatens women’s safety, fosters hostile situations, and upholds social inequities. We describe a task SemEval-2023 Task 10 for creating Englishlanguage models that can precisely identify and categorize sexist content on internet forums and social platforms like Gab and Reddit as well to provide an explainability in order to address this problem. The problem is divided into three hierarchically organized subtasks: binary sexism detection, sexism by category, and sexism by fine-grained vector. The dataset consists of 20,000 labelled entries. For Task A, pertained models like Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM), which is called CNN-BiLSTM and Generative Pretrained Transformer 2 (GPT-2) models were used, as well as the GPT-2 model for Task B and C, and have provided experimental configurations. According to our findings, the GPT-2 model performs better than the CNN-BiLSTM model for Task A, while GPT-2 is highly accurate for Tasks B and C on the training, validation and testing splits of the training data provided in the task. Our proposed models allow researchers to create more precise and understandable models for identifying and categorizing sexist content in online forums, thereby empowering users and moderators.

heading: 1 Introduction
text: Sexism is a pervasive issue in society that shows up in a variety of linguistic contexts, including online discourse (Paciello et al., 2021). The topic of automatically recognising sexist language in text has several real-world applications, including bias detection in computer-generated material and social media platform moderation (Verge, 2022). Sexism is a major issue in society today, and it has been shown to have negative effects on both men and women. The prevalence of sexism in online communication makes it difficult for people to identify
and address it (Leaper and Brown, 2008). Automatically detecting sexist language in text is an important problem that has many practical applications. For example, social media platforms can use automated systems to detect sexist language and take appropriate actions against users who post offensive content (Rodríguez-Sánchez et al., 2020), this could help reduce the amount of sexism present on these platforms and create a more inclusive environment for all users.
The shared task SemEval-2023 Task 10 (Kirk et al., 2023) was about identifying sexism and proving it by identifing sexist content and explaining why it is sexist enhances the generalization ability, trust, and comprehension of the choices made by automated systems, giving users and moderators more control. This will bring more confidence on the automated system which detects sexist content by providing an explainability for that content. The shared task incorporated with three hierarchical subtasks that each builds on the previous one makes up our main task. The first Subtask, Task A, requires systems to anticipate whether or not a post is sexist and it is a binary classification task. In the second subtask, Task B, systems must predict the category of sexism for posts that are labeled as sexist in Task A. Task B is a four-class classification task. ‘Threats’, ‘Derogation’, ‘Animosity’, and
‘Prejudiced Discussions’ based on prejudice are the categories. The third and final subtask, Task C, requires systems to predict one of 11 fine-grained vectors for postings that are rated as sexist in Task A and categorized in Task B. Task C is an 11-class classification task.
In this shared task SemEval-2023 Task 10, the problem of detecting sexist language and explainability was addressed using two models, one is Convolutional Neural Network (CNN) (Kim, 2014) and Bidirectional Long Short-Term Memory (BiLSTM) (Liu and Guo, 2019) which is interchangeably called throughout the paper CNN-
815
BiLSTM (Ma et al., 2020) and the other is Generative Pretrained Transformer 2 (GPT-2) (Radford et al., 2019). A common type of neural network used in image analysis is the CNN. But, by treating each word as a "channel" in the input, they can also be employed in tasks involving natural language processing. In order to capture particular features, such as n-grams and word embeddings, that are important for the classification task, the CNN layer can then learn various sorts of filters. On the other hand, BiLSTM models are a kind of Recurrent Neural Network (Sherstinsky, 2020) created specifically to handle sequential data, like text. They can recognise long-range relationships in the text because they can keep track of the words that came before them in a phrase. GPT-2 is a Transformer-Based Language Model developed by OpenAI (Radford et al., 2019) that has shown remarkable performance in various natural language processing tasks, such as language generation (Ko and Li, 2020), machine translation (He et al., 2020), and question answering (Puri et al., 2020). To deploy our models a web application called ’PREDHATE!’ has been created that enables users to input any text they desire. The application employs our model’s forecast to indicate whether the text contains sexist content or not.
The paper has been arranged as follows, Section 2 will be the background work and relative work done in the field, Section 3 will be about the data sampling and training, development and test data splits. Section 4 the system overview of the models and were demonstrated. Section 5 gives the experimental setup of our models for the Tasks A,B & C. Section 6 will provide the results how our models performed on these tasks.

heading: 2 Background
text: In recent years, there has been growing interest in the task of detecting sexism and misogyny (Parikh et al., 2021) in natural language text. This is exemplified by research efforts such as SemEval Task 10, which provides a dataset (Kirk et al., 2023) for training and evaluating machine learning models for this task (Zampieri et al., 2019).
One approach to tackling this challenge is to use classic machine learning techniques, such as ngrams, as demonstrated inn (Anzovino et al., 2018). This study presents a dataset for identifying and classifying misogynistic language on Twitter in both Spanish and English. Meanwhile, (Frenda
et al., 2019) applies a similar approach to detect online hate speech against women by combining two datasets. Likewise, we are combining the SemEval Task 10 dataset with other similar ones, which we will be explaining in detail in the section 3.
However, more recent studies have explored the use of advanced deep learning techniques to achieve state-of-the-art results in sexism detection. For instance, (Grosz and Céspedes, 2020) uses GloVe embeddings (Pennington et al., 2014) and modified LSTMs with attention mechanisms (Bahdanau et al., 2014) to detect sexist statements commonly used in the workplace automatically. These studies demonstrate the potential of using deep learning techniques to address the challenge of detecting sexism and misogyny in natural language text.
GPT-2 (Radford et al., 2019) has been used in several related natural language processing (NLP) tasks, including text classification (Anaby-Tavor et al., 2020), sentiment analysis (Alexandridis et al., 2021), and language modeling (Budzianowski and Vulić, 2019).
For example, in a recent study by (Zhang et al., 2022), the authors used a pre-trained GPT-2 model to perform binary classification on a dataset of news articles to predict whether the article is sexist or not sexist. The results showed that the GPT-2 model outperformed several other machine learning models in terms of accuracy.
Another study (Mathew and Bindu, 2020) used a pre-trained GPT-2 model to perform sentiment analysis on a dataset of customer reviews. The authors found that the GPT-2 model was able to achieve state-of-the-art performance in sentiment analysis tasks.
While these studies did not focus specifically on sexism classification, they demonstrate the potential of pre-trained language models like GPT-2 for natural language processing tasks. Similar approaches using GPT-2 could be effective for the sexism classification task (Vaca-Serrano, 2022) as they have done a decent job, for classification of sexism in English and Spanish languages, they have mostly used Encoder-Decoder models for their approach.

heading: 3 Data
text: The organisers have provided the data in the task SemEval-2023 Task 10, which we will be giving a brief overview of the data, how the data has been
split into train, development and test splits as well as the data sampling and annotation in this section.

heading: 3.1 Data Sampling and Annotation
text: (Kirk et al., 2023) have 20,000 entries in their labeled data-set, with 10,000 coming from Gab1 and 10,000 from Reddit2. Three professional annotators first label the entries, then one of two experts decides on disagreements after that. The gold label for Task A is decided by the annotators in unanimity or, in the event of a tie, by an expert review. The gold label for Tasks B and C is chosen based on agreement by at least two annotators or, in the event of a 3-way tie, an expert review. They annotate their data using the "prescriptive paradigm," which includes precise annotation rules, frequent feedback, and training. All annotators and experts are self-identifying women.

heading: 3.2 Training, Development, and Test Data
text: 3,398 of the 14,000 entries in their training set, which has a 70% split, are identified as sexist. For Tasks A, B, and C, there is a single CSV file with the following columns: text (the input text), label sexist (the label for Task A), label category (the label for Task B), and label vector (the label for Task C). Label category and Label vector are set to the string "none" for entries that are not sexist.
To promote cutting-edge training methods, they also made available two unlabeled datasets from
1https://gab.com/ 2https://www.reddit.com/
Gab and Reddit, each with one million entries. Development data is used to grade submissions during the development phase and consists of 2,000 entries with a 10% split. Test data is used to grade submissions during the test phase and consists of 4,000 entries with a 20% split. The development and test datasets contain distinct CSV files for Tasks A, B, and C, with Tasks B and C’s release being spaced out to prevent task-gaming. Table 2 shows the data-set statistics which has been used to train our models.

heading: 4 System Overview
text: For Task A, we use a binary classification approach where we classify whether a post is sexist or not sexist. For this, we use the CNN-BiLSTM model and the GPT-2 model.
The CNN-BiLSTM model uses a combination of convolutional and recurrent layers for feature extraction and sequence modeling, respectively. The input to the model is a sequence of words, which are first embedded into a dense vector representation. The embedded words are then passed through multiple convolutional layers to extract local features, which are then fed to a Bidirectional LSTM layer for sequence modeling. The output of the LSTM layer is then passed through a dense layer for binary classification.
On the other hand, the GPT-2 model is a transformer-based language model that uses many layers to extract contextual information from a given sequence. The input to the GPT-2 model is also a sequence of words, which are embedded into dense vector representations. The embedded words are then passed through multiple transformer layers to extract contextual information. The output of the last transformer layer is then passed through a dense layer for binary classification.
For Task B, we classify the posts that are labeled as sexist by Task A into one of four categories: (1) Threats, (2) Derogation, (3) Animosity, (4) Prejudiced Discussions. For Task C, we classify the posts labeled as sexist by Task A into one of eleven fine-grained vectors. We use the GPT-2 models as in Task A for this classification task for both Task B & C. The only difference is that the output layer is modified to predict one of the four categories instead of binary classification for Task B and again modified to predict one of the eleven fine-grained vectors for Task C. Table 1 shows all the dataset’s classes.
Towards GUI: We have designed a web app, where users can type any text they want. Based on our model’s prediction it will show whether the text is sexist or not. Figure 1 shows an overview of our website ‘PREDHATE!’ which also provided a few features like checking the history and can classify multiple texts at a time. Code is openly available at: https://github.com/human71/predhate.

heading: 5 Experimental Setup
text: The SemEval Task 10 has been divided into 3 subtasks, for each subtask we have created individual models with respect to their respective classifications. Now the experimental setup has been divided into 2 parts as follows.

heading: 5.1 Task A
text: Again for Task A the experimental setup has been divided into 2 parts. one is using CNN-BiLSTM and the other is GPT-2. CNN-BiLSTM: Convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM) models are used in this paper to demonstrate an experimental setting for Task A of the
Sexism Classification Challenge. Task A’s objective is to determine if a particular post is sexist or not sexist.
We feed pre-trained word embedding into our CNN-BiLSTM architecture to build our model. Using the labeled training data supplied by the challenge organizers, we train our model. We divide the data into a 70/30 train-test split and use the training set for model development and verification.
Two primary parts make up the architecture of our CNN-BiLSTM system. The first is a CNN layer, which is intended to identify specific correlations and patterns in the input text. A BiLSTM layer, the second, is created to identify long-term dependencies in the input text. To create a binary classification output, the outputs of these two layers are merged and fed through a fully linked layer with a sigmoid activation function.
The Adam optimiser is used to train our models, while binary cross-entropy serves as our loss function. To avoid overfitting, we additionally apply dropout regularization to our fully connected layer. We use a grid search to fine-tune our hyperparameters over a range of values, and we choose the top-performing model based on validation accuracy. GPT-2: We explore the use of GPT-2, a pre-trained language model, for the task of binary sexism detection (Task A). The use of pre-trained models has been shown to be highly effective in various natural language processing tasks, including text classification.
For this task, we use the pre-trained GPT-2 model with 768 parameters. We fine-tune the model on a given labeled dataset of 14,000 entries, which consists of 10,602 non-sexist posts and 3,398 sexist posts. We split the dataset into 70%
training data, 15% validation data and 15% testing data. We use the Adam optimizer with a learning rate of 1e-5 and a batch size of 8. We train the model for 5 epochs and save the model weights that perform the best on the validation set.
To use the fine-tuned model for predicting sexism in new text, we first tokenize the input text and pass it through the model. We evaluate the performance of the model on our heldout dev dataset of 2,000 entries, which were not labeled either as sexist or not sexist and produced F1-score. In addition, we compare the performance of our GPT-2 model with the previously mentioned CNNBiLSTM model.

heading: 5.2 Task B and C
text: For Task B and C, we use GPT-2 with fine-tuning for multi-class classification. The pre-trained GPT2 model will be fine-tuned on our training data to predict the category of sexism and fine-grained vector of sexism.
We will follow a similar preprocessing pipeline as we did for Task A, with the addition of onehot encoding for the output labels. For Task B, we will use a four-class classification with the following categories: threats, derogation, animosity, and prejudiced discussions. For Task C, we will use an 11-class classification with the following vectors: objectification, sexual violence, physical violence, exploitation, gender discrimination, occupation discrimination, racist discrimination, stereotyping, body shaming, gendered insult, and identity-based insult. We will use the same training, development, and test sets as in Task A. We will fine-tune the pre-trained GPT-2 model on the training set, validate on the development set, and test on the test set. We will use the Adam optimizer with a learning rate of 1e-5, a batch size of 8, and a maximum sequence length of 128 tokens.
We will evaluate the performance of the GPT-2 model for Task B and C using micro-averaged F1 scores, as in Task A. We will compare the performance of the GPT-2 model to the baseline CNNBiLSTM model, as well as to other state-of-the-art models for multi-class text classification.
We will also perform ablation studies to investigate the impact of different model architectures and hyperparameters on the performance of the GPT-2 model. Specifically, have experimented with different learning rates, batch sizes, sequence lengths, and pre-processing techniques to determine the op-
timal configuration for each task.

heading: 6 Experimental Results
text: In this part, we present GPT2 model’s performance throughout training and testing compared to our other approach. We also compare the result with the other existing system submitted on SemEval2023 Task 10.

heading: 6.1 Evaluation Method
text: The Macro F1 score is a single score that strikes a balance between Precision and Recall in a single number. Confusion matrices are often used metrics in classification tasks, and F1 offers a single score that strikes a balance between Precision and Recall in a single number. Values for precision and recall are computed, and there are respectively TP, FN scenarios that are True Positive and FN circumstances that are False Negative. Precision, recall, and F1 score are all split by class and each are given a macro average. The Macro F1 we typically observe is called the macro average. To calculate, we have to find out (1) Macro-average precision by calculating (P1+P2)/2 and (2) Macro-average recall by calculating (R1+R2)/2. Finally, just the harmonic mean of these two will constitute the Macro-average F1.

heading: 6.2 Evaluation of Test Data
text: For Task A we have tested on 1400 lines of data, as the data-set split was 80-10-10%, while, for task B and C the number was 340. We have fine-tuned the classification models for all three using GPT-2 as our base model. Additionally, we tested these classification models with three epoch values and three learning rate values. For the classification models with the best detection performance, the confusion matrix in Figure [3, 4, 5] shows the number of correctly and wrongly classified samples that were acquired. The number of true negatives (TN) in the confusion matrix refers to correctly identified instructions that are benign, whereas the number of true positives (TP) refers to correctly identified instructions that are harmful. The percentage of false positives shows innocent instructions mistaken for malevolent ones. The number of false negatives, on the other hand, shows dangerous instructions mistaken for benign ones. Table [4, 5, 6] shows the precision, recall and the Macro-F1 of the models. From that, we can see the macro average decreased along with the decrease of data-set from A to C.
The most important finding from Table [3, 4] is that the GPT-2 model specifically outperforms our CNN-BiLSTM model for Task A, hence we decided not using this approach for tasks B and C. For Task A, GPT-2 scored 75%, while for B and C, it went down to 40% and 20%.
If there are no predicted samples for a label, precision and F-score are considered ineffective and are set to 0. Because some of our labels appear in our y_pred but not all, our classifier is unable to predict some labels in our y_test. Here, y_test is
the actual values that the dataset has, y_pred is the values our model predicted.

heading: 6.3 Decisions and Problems
text: For evaluation, we use precision, recall, F1-score as metrics. The section 6.2 shows performance of our models on the development and test sets for all three tasks. We also conduct ablation tests to assess how different model elements affect performance as a whole. In the end, we compared our models with the ones that performed the best. The classification model built on GPT-2 that was adjusted
functioned as expected, as indicated in Table 7. Yet, the results show that the model is biased and does not produce the right answers for a certain group, which may be why they perform worse than other groups.

heading: 7 Conclusion
text: As a conclusion, the identification and categorization of sexism in online content is a significant issue in modern culture, where social interactions take place more and more in online settings. Making online places more welcoming and secure for everyone can be achieved with the use of automated techniques for sexism detection and classification especially for women.
In this study, we provided an overview of the 20,000 sexism-labeled entries from the Gab and Reddit dataset as well as a hierarchy of three sexism detection and classification tasks, namely Task A (Binary Sexism Detection), Task B (Category of Sexism), and Task C (Fine-grained Vector of Sexism). The CNN-BiLSTM and GPT-2 models for Task A as well as the GPT-2 model for Tasks B and C were also given, along with their respective experimental settings.
Our findings demonstrated that the GPT-2 model performed better on Task A than the CNN-BiLSTM model, achieving greater accuracy and F1 scores. Also, the use of GPT-2 for Tasks B and C enabled the creation of more thorough and educational classifications, offering greater insights on the kind and degree of the sexism found in online content.
In the future, we hope that our work will stimulate additional investigation into the automated identification and categorization of sexism and that researchers in this field will find our dataset and experimental setups to be a helpful resource. We also hope that the knowledge we learn from this effort will contribute to the development of more welcoming and secure online environments for everyone.

heading: 8 Acknowledgments
text: We would like to thank National Institute of Technology Silchar for providing us the opportunity to participate in the SemEval-2023 Task 10, financial assistance, CNLP & AI laboratorys for the resources and research environment for smooth conduction of research and experiments.

heading: Future Work
text: In the future, we hope that our work will stimulate additional investigation into the automated identification and categorization of sexism and that researchers in this field will find our dataset and experimental setups to be a helpful resource. We also hope that the knowledge we learn from this effort will contribute to the development of more welcoming and secure online environments for everyone