Transformers at HSD -2 L ang 2024: Hate Speech Detection in A rabic and T urkish Tweets Using BERT Based Architectures (2024)

Abstract: Over the past years, researchers across the globe have made significant efforts to develop systems capable of identifying the presence of hate speech in different languages. This paper describes the team Transformers’ submission to the subtasks: Hate Speech Detection in Turkish across Various Contexts and Hate Speech Detection with Limited Data in Arabic, organized by HSD-2Lang in conjunction with CASE at EACL 2024. A BERT based architecture was employed in both the subtasks. We achieved an F1 score of 0.63258 using XLM RoBERTa and 0.48101 using mBERT, hence securing the 6th rank and the 5th rank in the first and the second subtask, respectively.

## introduction hate speech is defined as the usage of expressions or phrases which are hostile, offensive, or threatening in nature. hate speech is usually targeted against an individual or a group of individuals, highlighting those unique characteristics that distinguish those individuals. some people use online platforms, such as twitter, to spread hateful content at the click of a button. access to the internet, along with social media platforms such as twitter, instagram, and facebook, can enable anyone, anywhere in the world, to share their ideas with millions of people across the globe within a few milliseconds @xcite . with the advancing technological age, it is getting easier to spread hateful content thousands of miles across the globe, even without revealing their identity, due to the increased anonymity offered by online platforms. automated detection of hateful content has become crucial for enhancing content moderation to mitigate societal harm. social media platforms encourage the users to report any hate speech content that violates the hateful conduct policy so that appropriate action can be taken. however, it is still visible to many users, which necessitates the use of an automated system to detect and curb such content @xcite . the task organized by hsd-2lang foot_0 at case 2024 aimed at identifying the presence of hate speech in turkish and arabic languages (gökçe uludogan, 2024). the task was divided into two subtasks, as listed below: in the past few years, natural language processing (nlp) has experienced major breakthroughs, especially in the hate speech identification domain. some of which are long short term memory @xcite and the gated recurrent units @xcite . but, there has been a paradigm shift with the introduction of transformers @xcite . arabic language is one of the six official languages of the united nations. arabic is also a critical and strategically useful language @xcite . with 18.55 million users, turkey had the 7 th highest number of twitter users in 2023 @xcite . turkish is also one of the most widely spoken languages of the turkic language family. both arabic and turkish are very different from the english language. the orthography of both languages significantly differs from english due to the right-to-left text orientation and the utilization of connecting letters. the presence of word elongation, common ligatures, zero-width diacritics, and allographic variants leads to further complications. the morphology is extremely intricate, showcasing a wealth of morphemes that are used as prefixes, suffixes, or even circumfixes. these elements can denote various grammatical features such as case, number, gender, and definiteness, among others, resulting in a sophisticated morphotactic system @xcite . ## related work researchers have made multiple efforts in the past to automatically detect the presence of hate speech in arabic and turkish, in the past. a variety of different approaches have been used in the past to address this problem. abuzayed and elsayed (2020) compared 15 classical and neural network models to classify arabic tweets based on the presence of hate speech. to solve the problem efficiently, a "quick and simple" approach was used. the experiments were conducted on a collection of 8,000 tweets, and it was found that neural learning models outperformed the classical ones. the best classifier was a joint architecture of a convolution and recurrent neural network. the classifier used data after pre-processing, in which the punctuation, foreign characters, numbers, repeated characters, and diacritics were removed from the text. the remaining arabic text was then normalized. in the approach adopted by husain (2020), extensive pre-processing was performed. the preprocessing step involved seven different steps. the work showed the improvement that pre-processing the data makes by retaining only the important content and performing dimensionality reduction. the first step in pre-processing was the conversion of emojis and emoticons to a textual label to ensure that the meaning conveyed by them did not suffer due to their removal. next, since the arabic dialect exists in various different forms, the variations in the different forms were normalized. the words were then categorized and then letter normalization was performed. this was followed by hashtag segmentation, where the '#' symbol was removed and the text following it was left untouched. after this, the numbers, more than two consecutive spaces, and the occurrence of more than three repetitive characters was removed along with arabic stop words. lastly, to address the data imbalance, upsampling was performed. neural networks and discourse analysis techniques were used by @xcite to identify the presence of hate speech in turkish text. a hierarchical attention network and bert 3 dataset description ## methodology in nlp, text classification in languages with limited resources and code-mixed nature, has been a prominent problem. it can be defined as assigning text labels depending upon the content, context and intention of it. researchers have devised multiple models to tackle this problem. many of these models followed a transformer based approach and have been pre-trained on large corpora of text and figure 2: label generation for unseen data ## results and discussion multilingual transformer models were used to detect the presence of hate speech in arabic and turkish tweets. the bert based architectures were finetuned to improve their performance further. the data imbalance present in the data of both the subtasks was addressed by performing random undersampling on the data to ensure that the number of tweets for both the classes are equal. it was found that the model performed better when the unprocessed data was used to train the model. hence, the text data was used without any pre-processing to train the model. the methodology followed to finetune the transformers for both the subtasks has been summarized in figure 1 . the label generation for the testing data has been summarised in figure 2 for both the subtasks. the models achieved an f1 score of 0.63258 and 0.48101 in subtask a and subtask b, respectively. overall, the highest f1 score achieved was 0.69644 and 0.68354 in subtask a and subtask b respectively by the teams ranked 1 st in the shared task. ## conclusion and future work hate speech detection is the process of classifying text based on the presence or absence of hateful content. the aim of the shared task organized by hsd-2lang in conjunction with case at eacl 2024 was to automatically detect whether the tweet was hateful or not in nature. in this paper, we discussed our use of two multilingual bert based transformers in the hate speech detection in turkish and arabic tweets shared task. we achieved an f1 score of 0.63258 in subtask a with xlmr and an f1 score of 0.48101 in subtask b with mbert with the discussed approaches. transformers have shown great potential in the field of nlp and have consistently outperformed the classical models. hence, combining different transformer models using ensembling techniques can help improve performance. also, since limited resources are available for both arabic and turkish, the performance may be further enhanced by combining multiple datasets.