M ar S an at S em E val-2023 Task 10: Can Adversarial Training with help of a Graph Convolutional Network Detect Explainable Sexism? (2023)

Abstract: This paper describes SemEval-2022’s shared task “Explainable Detection of Online Sexism”. The fine-grained classification of sexist content plays a major role in building explainable frameworks for online sexism detection. We hypothesize that by encoding dependency information using Graph Convolutional Networks (GCNs) we may capture more stylistic information about sexist contents. Online sexism has the potential to cause significant harm to women who are the targets of such behavior. It not only creates unwelcoming and inaccessible spaces for women online but also perpetuates social asymmetries and injustices. We believed improving the robustness and generalization ability of neural networks during training will allow models to capture different belief distributions for sexism categories. So we proposed adversarial training with GCNs for explainable detection of online sexism. In the end, our proposed method achieved very competitive results in all subtasks and shows that adversarial training of GCNs is a promising method for the explainable detection of online sexism.

## introduction online platforms have become a major source of communication, where people express their opinions and views. the online world, however, is not immune to sexism, and increasing social media usage has led to the proliferation of hate speech and offensive language @xcite . it is critical to automatically identify such content in order to prevent its spread. this study aims to identify and classify sexism in social media texts. the detection of sexism in online conversations requires identifying offensive language and gender-biased text. sexism involves the use of abusive or negative language directed towards women based on their * equal contribution. listing order is random. gender, or in conjunction with other identity attributes such as race, religion, or gender identity. this form of online hate speech is a significant obstacle to gender equality. women are disproportionately impacted by hateful and offensive content on social media platforms, including misogynistic and sexist posts, which can lead to real-world violence against them @xcite . the automatic detection and categorization of these types of tweets and posts can be a valuable tool for social scientists and policymakers in their efforts to combat sexism through research. detecting and categorizing instances of sexism in social media posts can be a complicated and arduous process, particularly when the content involves sarcasm and various types of sexist language. recent research has identified multiple factors that make identifying sexist content on social media particularly challenging @xcite . these reasons include the use of subtle forms of expression with microaggressions woven into the language, the presence of diverse contexts such as ongoing popular events and conversations, and the potential impact of the platform or language used on determining whether a statement is considered sexist or not. in recent years, graph convolutional networks (gcn) @xcite and language models have shown significant promise in the field of natural language processing. it is possible to develop robust frameworks for detecting and categorizing online sexism posts using gcn's ability to model complex relationships in graph data. this is coupled with the language model's capability to understand the context and semantics of text. adversarial training has been shown to be effective in improving the robustness of neural models by incorporating a small perturbation into the input data during training @xcite . this makes it more difficult for the model to make incorrect predictions. therefore, we propose a novel framework for detecting online sexism posts that combines graph-based models, language models, and adversarial training. our proposed framework leverages the strengths of each component to detect and categorize subtle forms of sexism in social media content. additionally, it can enhance the robustness of the model against adversarial attacks. our code is available on github for researchers. foot_0 this paper is organized as follows: a description of the task and its subtasks can be found in section 2. section 3 reviews related research. the theoretical background of the neural model is presented in section 4. section 5 presents implementation details, while 6 describes experiments and results. the conclusions of the paper can be found in section 7. ## task description explainable detection of online sexism (edos) @xcite is a semeval 2023 task that aims to facilitate the development and assessment of models capable of detecting online sexism in social media posts. human experts in english have manually annotated social media posts for this task. the following three subtasks are involved in edos: binary sexism detection (task a): the first subtask of the edos task is the detection of binary sexism. in this task, systems need to predict whether a given social media post is sexist. category of sexism (task b): for posts that are classified as sexist in task a, the systems have to predict one of four categories: (1) threats, (2) derogation, (3) animosity, or (4) prejudiced discussions. fine-grained vector of sexism (task c): systems have to predict one of 11 fine-grained vectors if posts are classified as sexist in task a. ## related works for the iberlef 2022 dataset, de paula and da silva (2022) implemented multilingual versions of bert, roberta, and the monolingual versions of electra, and gpt2 and achieved the highest results using back-translation to english with bert and roberta language models. with byt5 as a token-free language model, @xcite proposes to combine byte-level modeling with attention-based neural networks through tabular modeling via tabnet, which can take into account platform and language aspects to detect sexism effectively. by incorporating recurrent components, @xcite proposes a neural model for the detection and classification of sexism, combining representations obtained using roberta and linguistic features such as empath and hurtlex. in addition to that, @xcite incorporates external knowledge-specific features into the learning process using emoticons and hashtags. @xcite proposes a large chinese lexicon of abusive and gender-related terms, called sexhatelex, and the first chinese sexism dataset called sina weibo sexism review (swsr). these authors analyse the characteristics of the dataset and how chinese sexism manifests. swsr provides labels at different levels of granularity, including sexism or non-sexism, sexism category, and target type. additionally, the paper uses stateof-the-art machine learning models for the three sexism classification tasks. the research aims to broaden the scope of research in sexism detection and contribute to combating online sexism. a combination of graph convolutional neural networks (gcn) with different edge creation strategies is applied in wilkens and ognibene (2021) as well as an ensemble approach for combining graph embeddings from different gcn models. to capture the linguistic and structural characteristics of sexist posts, the model uses graph-based representations of social media content. a study from @xcite explores the use of deep neural networks, including long-short-term memory (lstms) and convolutional neural networks (cnns), to identify sexism in a social media text. using transfer learning through bidirectional encoder representations from transformers (bert) and distilbert models, as well as data augmentation, the models classify sexism in a dataset of sexism identification in social networks (exist) tasks. with data augmentation, bert and a multi-filter cnn model produce comparable results. the authors in @xcite use @xcite to develop classifiers to detect and annotate different types of sexism. using cnn models, the study detects semantic categories of n-grams and clusters them to improve the classification task by improving the understanding of sexism. as part of the study, the authors examine more precise categories of sexism in social media and examine cnn filters, which can help identify the most important n-grams. to perform document classification, @xcite employ text graph convolutional networks (text gcn). a tensor graph convolutional network based on semantic, syntactic, and sequential graphs is introduced in @xcite for document classification. furthermore, @xcite combine bert and gcn to improve the performance of text classification. gender differences can further compound the challenges of detecting sarcasm and irony. studies have shown that men and women use sarcasm differently, with men often using it as a form of aggression or dominance, while women may use it to express their frustration or as a coping mechanism in situations where they lack power. moreover, societal norms and gender stereotypes can influence the way that sarcasm is perceived and interpreted, with women often being judged more harshly for their use of sarcasm than men. these factors must be considered when attempting to detect sarcasm and irony in written or spoken language (van de @xcite @xcite @xcite . ## system overview this section presents our proposed model which consists of three modules: the encoder module, the gcn module, and the classifier module. the input text is encoded into a feature representation by the encoder module, and relevant features are extracted by the gcn module based on the graph structure. as a final step, the classifier module uses these extracted features to classify the input text. after each epoch, the encoder module updates all node embeddings. figure 1 shows the overview of our proposed model. ## experimental setup this section is structured into two distinct parts. firstly, we describe our data pre-processing approach in detail. secondly, we discuss the implementation details. pre-processing: the dataset contained urls and user mentions, which could confuse the model in labeling the data. therefore, these items were removed during preprocessing. additionally, the text was converted to lowercase and all numbers were removed. these steps help to reduce complexity, improve uniformity, and increase accuracy, especially in tasks like sexism detection. furthermore, the graph constructor utilizes a number of preprocessing techniques, which are only used in the graph constructor, not in the language model. the first step is to remove stop words that are commonly used but do not convey much meaning. subsequently, all words within the text are lemmatized. the last step involves removing words that appear in the text less than three times. implementation details: pytorch is employed to implement the model, which was trained on highperformance nvidia v100 gpus. to fine-tune each subtask, hyper-parameters were adjusted using the development set. the model is trained using the back-propagation algorithm with the adam optimizer, which had a learning rate of 1e-5. an early stopping method is employed by monitoring the validation loss in min mode with patience of five. a batch size of 32 is used for each subtask and cross-entropy loss with class-weights (sklearn is used to calculate class-weights foot_1 ) is used in cases of data imbalance. additionally, a maximum length of 70 tokens is used in all experiments. (only 41 samples have more than 70 tokens.) the transformer consists of eight attention heads, while the position-wise feed-forward layer has a hidden size of 2048. additionally, one layer of bi-lstm with 512 units is used. the gcn layers have output channels set to 1024, followed by a relu function, and a dropout rate of 0.2 is used. the window size in sequential-base graphs is set to ten. ## experiments and analysis this section reviews various baselines and compares them with the proposed model. the first challenge is to identify the most appropriate token contexts based on language models. consequently, we first analyze the performance of different language models for each task, including bertweet, bert-large @xcite , xlm-roberta @xcite , and t5-large @xcite . according to our analysis, bertweet outperforms all other models with a higher macro f1-score. the results of the different language models are presented in ## conclusion our study presents a novel approach for identifying various types of sexist posts by combining graph and language models. to capture contextual information from the input text, we utilize the encoder module, which is composed of a bertweet model, a transformer, and a bidirectional lstm layer. additionally, we utilize syntactic, semantic, and sequence-based graphs to extract information from the graph structure. by integrating context and graph-based features, our proposed model outperforms existing models in identifying sexist posts and their various types. in order to evaluate the performance of the proposed model, several experiments are conducted. in the experiments, it is demonstrated that the com-bination of contextual features extracted from the encoder module and graph-based features provides a reasonable range of results for the detection of online sexism that is explainable due to achieving better performance on fine-grained detection.