IIC _ T eam@Multimodal Hate Speech Event Detection 2023: Detection of Hate Speech and Targets using Xlm-Roberta-base (2023)

Abstract: Hate speech has emerged as a pressing issue on social media platforms, fueled by the increasing availability of multimodal data and easy internet access. Addressing this problem requires collaborative efforts from researchers, policymakers, and online platforms. In this study, we investigate the detection of hate speech in multimodal data, comprising text-embedded images, by employing advanced deep learning models. The main objective is to identify effective strategies for hate speech detection and content moderation. We conducted experiments using four state-of-the-art classifiers: XLM-Roberta-base, BiLSTM, XLNet base cased, and ALBERT, on the CrisisHateMM[4] dataset, consisting of over 4700 text-embedded images related to the Russia-Ukraine conflict. The best findings reveal that XLM-Roberta-base exhibits superior performance, outperforming other classifiers across all evaluation metrics, including an impressive F1 score of 84.62 for sub-task 1 and 69.73 for sub-task 2. The future scope of this study lies in exploring multimodal approaches to enhance hate speech detection accuracy, integrating ethical considerations to address potential biases, promoting fairness, and safeguarding user rights. Additionally, leveraging larger and more diverse datasets will contribute to developing more robust and generalised hate speech detection solutions.

## introduction hate speech on social media has become a major issue, with online platforms being used to denigrate and degrade people or entire groups based on their colour, religion, ethnicity, or handicap @xcite . in the virtual world, the concept of hate speech can be complex and nuanced, making it difficult to address effectively @xcite @xcite . to address the issue, international conventions, and multilateral initiatives have been developed, however, implementing laws in the virtual sphere remains a difficult undertaking. despite social media firms' efforts, suppressing hate speech is an ongoing process. the increasing amount of multimedia content, powered by quicker and more accessible mobile internet, has altered the social media environment. instagram, snapchat, vine, and tiktok have championed multimedia, prompting established behemoths like facebook and twitter to follow suit. this transition has shifted social media from a predominantly textbased environment to one in which video, audio, and photographs take center stage, allowing users to express themselves in more interesting and diverse ways @xcite . the detection of hate speech during political events is especially important for preserving democracy, reducing violence, protecting vulnerable communities, and fostering civil dialogue. effective detection ensures fair elections, platform integrity, national cohesion, and informed decisionmaking while balancing free speech protection with actions to eliminate harmful content. traditional moderation procedures, such as manual text and multimedia inspection, confront substantial restrictions as a result of the massive volume of data created on social media platforms. human moderators are unable to keep up with the exponential increase in content, resulting in delays in recognizing and correcting hate speech, allowing harmful content to propagate unchallenged. furthermore, human moderators' biases and subjective interpretations can lead to inconsistent results. to handle the size and pace of data growth, automated hate speech identification systems are crucial. automated hate speech detection systems utilise artificial intelligence (ai) techniques to analyse large volumes of data and identify content that con-tains hate speech or offensive language these systems rely on natural language processing (nlp) algorithms to preprocess and transform the text data into numerical representations, such as word embeddings. ai models, like recurrent neural networks (rnns), long short-term memory (lstm) networks, or transformers, are then employed to extract contextual and semantic features from the text. the model is trained on labeled datasets, learning to recognize patterns and characteristics indicative of hate speech @xcite @xcite . to address the need for large datasets to train ai models, the crisishatemm dataset @xcite is introduced, aiding the shared task on multimodal hate detection at case 2023 @xcite . this dataset includes two primary tasks, with sub-task 1 focusing on classifying textembedded images into two categories: hate speech and non-hate speech. sub-task 2 involves the classification of targets in the text-embedded images into three categories: individual, organisation, and community. within the datasets, predominantly comprised of images linked to the russia-ukraine conflict, this widespread political event has been the subject of significant hateful language and has been thoroughly examined by researchers @xcite . by employing a subtask-based methodology, this research approach allows for detailed analysis and interpretability, providing valuable insights into hate speech characteristics and target identification in social media content. the dataset's multimodal and contextually relevant annotations facilitate benchmarking and advancements in combating hate speech on social media platforms. the paper proposes a novel approach for hate speech identification utilising the textual model xlm-roberta-base, achieving impressive results. in sub-task 1, the approach achieves an accuracy of 84.65% and an f1 score of 84.63%. in sub-task 2, it demonstrates solid performance with an accuracy of 72.31% and an f1 score of 69.73%. the method effectively detects hate speech in text-embedded images, showcasing its strong performance in this aspect. additionally, the study shows significant improvement in target recognition in images with objectionable text, showcasing the efficacy of the proposed approach in enhancing hate speech detection and target identification. notably, the paper secured the 3rd position in both tasks, signifying its competitive standing within the competition. the accomplishments of this study make a substantial contribution to the field of hate speech identification and further highlight its rank and achievements in the competition. the paper begins with a concise introduction to the problem of hate speech on social media. it then provides a comprehensive review of previous research on the topic and the technological advancements in recent years, including various approaches, methodologies, datasets, and experimental findings. the dataset is described in detail, statistical analysis is used to gain insights, and preprocessing procedures are covered. the article introduces the approach using nlp models (xlm-roberta-base, bilstm, xlnet base cased, and albert) and reports on how well they perform in identifying hate speech. results demonstrate the models' efficacy and the discussion analyses the results and discusses constraints. the conclusion highlights the importance of the research in preventing hate speech while summarising the major contributions. references list the sources used to conduct the study. ## literature survey hate speech on social media is a worrying issue when people utilise online venues to disseminate harmful or discriminatory content, fostering animosity and division. the pervasive effects it has on social cohesiveness, mental health, and actual violence highlight the urgent need for effective content moderation measures to address and reduce this problem. a study @xcite explored the development of a classifier to detect hate speech in web discourses, specifically focusing on race, nationality, and religion themes. they employed sentiment analysis techniques, including subjectivity detection, to identify and rate the polarity of sentiment expressions. by creating a hate speech lexicon based on subjectivity and semantic features, the model effectively classified hate speech. experimental results with a hate corpus demonstrated the practical applicability of the approach in real-world web discourse scenarios. researchers @xcite also tackled the challenge of hate speech detection in online user comments. hate speech, defined as abusive speech targeting specific group characteristics like ethnicity, religion, or gender, poses a significant problem for websites that allow user feedback, leading to negative consequences for their online business and user experience. to address this issue, the paper proposed a novel approach using neural language models to learn distributed low-dimensional representations of comments. these representations are then utilised as inputs to a classification algorithm, effectively addressing the issues of high dimensionality and sparsity that had previously hindered the state-of-the-art hate speech detection methods. as a result, their approach demonstrated high efficiency and effectiveness in detecting hate speech in online comments. the study @xcite addressed the escalating problem of hate speech dissemination in online content and the difficulties confronted by automatic approaches in detecting such content in text. these challenges encompassed the intricacies of language, divergent definitions of hate speech, and limited availability of data for training and testing these systems. additionally, the lack of interpretability in many recent approaches posed a significant hurdle, making it arduous to comprehend the rationale behind the system's decisions. to overcome these obstacles, the paper proposed a multi-view support vector machine (svm) approach, which achieved nearly state-ofthe-art performance in hate speech detection while maintaining simplicity and providing more easily interpretable decisions compared to neural methods. the paper concluded by discussing both technical and practical challenges that still persist in this area, emphasising the need for further research to enhance hate speech detection systems for online content. in 2022, alkomah et al conducted a comprehensive study on hate speech detection systems, reviewing textual features, machine learning models, and datasets @xcite . the analysis of 138 relevant papers revealed that many approaches lack consistency in detecting various hate speech categories. the dominant methods often involve combining multiple deep learning models, while several hate speech datasets were found to be small and unreliable for detection tasks. the study provides valuable insights into the complexities of hate speech and highlights the need for improved approaches and larger, more reliable datasets to effectively combat hate speech and foster healthier online communities. another research in the same year @xcite addressed the pressing need to monitor hate speech on social media platforms, particularly in multimedia content. while text-based filtering has been extensively studied, detecting hate speech in multimedia presents unique challenges. a preliminary study revealed that the speaker's emotional state significantly influences hateful content, prompting the paper to focus on auditory and semantic features. introducing the first multimodal deep learning framework, the study combines emotional auditory features with semantics to detect hate speech effectively. results demonstrate improved detection compared to text-based models. additionally, a new hate speech detection video dataset (hs-dvd) is introduced, filling the gap in available datasets for this purpose. this research contributes to advancing hate speech detection in multimedia, providing a valuable resource to combat hateful content on social media platforms. @xcite conducted a study dedicated to multiaspect hate speech detection on social media. the overwhelming amount of unfiltered toxic content, including cyberbullying, cyberstalking, and hate speech, has become a significant challenge and a focus of active research. the proposed approach utilises a pre-trained bidirectional encoder representations from transformers (bert) model combined with deep learning (dl) models to create ensemble learning architectures. the dl models incorporate bidirectional long-short term memory (bi-lstm) and/or bidirectional gated recurrent unit (bi-gru) on fasttext and glove word embeddings. individual training of these models on multi-label hateful datasets and their combination with bert results in highly effective hate speech detection on social media. by leveraging recent word embedding techniques and dl architectures in conjunction with bert, the study achieves an impressive roc-auc score of 98.63%, significantly enhancing hate speech detection capabilities in multi-aspect scenarios. recently, one more research by @xcite introduced the multi-modal discussion transformer (mdt), a groundbreaking multi-modal graph-based transformer model designed for hate speech detection in online social networks. unlike traditional textonly methods, mdt takes a holistic approach by considering both text and images when labelling a comment as hate speech. the model leverages graph transformers to capture contextual relationships within the entire discussion surrounding a comment and utilises interwoven fusion layers to combine text and image embeddings, rather than processing different modalities separately. comparative evaluations against text-only baselines and extensive ablation studies showcase the superior performance of mdt. the paper concludes by emphasising the significance of multimodal solutions in delivering social value in online contexts and highlights that capturing a holistic view of conversations significantly advances the detection of antisocial behaviour like hate speech. this research presents a promising step towards more effective hate speech detection methods by considering both textual and visual cues in social media discussions @xcite . ## methodology the study utilised figure 1 to split the training data into 80% training sets and 20% validation sets. before model input, a preprocessing step was performed to prepare the textual data. the xlm-roberta-base, bilstm, xlnet base cased, and albert models were fine-tuned on the training sets to enhance hate speech identification. testing predictions were then generated using the finetuned models, demonstrating the effectiveness of the proposed approach in hate speech detection and target identification. ## results and discussion classification results of sub-task 1 are reported in the classification results for sub-task 2 are summarised in table-6, utilising the same four classifiers as in sub-task 1. once again, xlm-robertabase stands out as the best performer, surpassing the other classifiers in all four evaluation metrics. this exceptional performance can be credited to its extensive training on text data, which enables it to comprehend the context of textual content more effectively than traditional nlp models. the precision values further validate that the models have successfully achieved accurate target classification for individual, community, and organisational categories. these results reaffirm the significance of xlm-roberta-base in hate speech detection and target classification within text-embedded images, underlining its potential for advancing research in this field. xlm-roberta-base's superior perfor- ## conclusion this research focuses on text-embedded images used in social media to express opinions and emotions, which unfortunately also serve as platforms for spreading hate speech, propaganda, and extremist ideologies. notably, during the russia-ukraine war, both sides extensively utilised textembedded images for propaganda and hate speech dissemination. the growing abundance of offensive content on social media poses challenges in effectively detecting and moderating such material. to address this issue, we utilise the crisishatemm dataset, an innovative multimodal dataset containing over 4,700 text-embedded images from the russia-ukraine conflict. the dataset is meticulously annotated for hate and non-hate speech, further categorising hate speech into directed and undirected forms and providing annotations for individual, community, and organisational targets. our research involves two subtasks: sub-task 1 focuses on hate speech detection in text-embedded images, while sub-task 2 aims to identify the targets of hate speech. to achieve accurate results, we employ advanced feature extraction techniques and utilise deep learning models for both subtasks, yielding promising outcomes. in sub-task 1, our textual model, xlm-roberta-base, demonstrated superior performance, achieving the highest accuracy on test(unseen) data with a recall of 85.08%, precision of 84.76%, f1 score of 84.63%, and accuracy of 84.65%. additionally, in sub-task 2, the xlm-roberta-base model outperformed other approaches, achieving a recall of 68.94%, precision of 71.05%, f1 score of 69.73%, and accuracy of 72.31% these results highlight the effectiveness of our approach in hate speech detection and target identification in text-embedded images during the russia-ukraine conflict. exploring multimodal approaches, accessing larger and more diverse datasets, fine-tuning strategies, addressing biases, integrating automated systems with social media platforms, extending detection to multiple languages, and improving interpretability and contextual understanding are all part of the future of hate speech detection and content moderation. exploring zero-shot and few-shot learning methodologies, as well as addressing ethical concerns, are also essential. in summary, future research promises effective and responsible ways for combating hate speech on social media through the use of ai developments.