B ham NLP at S em E val-2020 Task 12: An Ensemble of Different Word Embeddings and Emotion Transfer Learning for A rabic Offensive Language Identification in Social Media (2020)

Abstract: Social media platforms such as Twitter offer people an opportunity to publish short posts in which they can share their opinions and perspectives. While these applications can be valuable, they can also be exploited to promote negative opinions, insults, and hatred against a person, race, or group. These opinions can be spread to millions of people at the click of a mouse. As such, there is a need to develop mechanisms by which offensive language can be automatically detected in social media channels and managed in a timely manner. To help achieve this goal, SemEval 2020 offered a shared task (OffensEval 2020) that involved the detection of offensive text in Arabic. We propose an ensemble approach that combines different levels of word embedding models and transfers learning from other sources of emotion-related tasks. The proposed system ranked 9th out of the 52 entries within the Arabic Offensive language identification subtask.

## introduction while microblogging platforms can be used positively and productively, they are also frequently used for destructive purposes such as spreading offensive messages. people who wish to spread hate can use these channels to quickly and easily reach millions of people at the click of a button. to prevent this spread, there is a need for a system that can automatically identify messages that contain offensive language. a significant number of studies have focused on the detection of offensive text in english. however, there is a lack of research that focuses on detecting offensive text in arabic @xcite . to contribute to the emerging body of knowledge in this domain, a shared task 'offenseval 2020 -task 12' was conducted at the semeval 2020 @xcite . one of the subtasks of this shared task is 'offensive language identification'. the organisers released multilingual datasets for five languages, including arabic, which is the focus of this paper. the shared task involved some significant challenges. the distribution of the targeted classes was unbalanced: 19% of the tweets were labelled as offensive, while the remaining were labelled as inoffensive. also, the tweets employed a variety of dialects and sub-dialects. unlike modern standard arabic (msa), dialectical arabic can take a variety of forms, and there is a general lack of rules and standards. to address these challenges, we proposed an ensemble approach that combines three different models (detailed in section 3). we believe that integrating multiple kinds of classifiers can help overcome the weaknesses and realise the advantages of each. our proposed system ranked 9th out of 52 participants for the arabic offensive language identification subtask. the remainder of this paper is organized as follows. section 2 provides a brief overview of the current literature on the detection of offensive arabic text. section 3 includes an overview of the methodology, including the experimental setup by which the proposed system was tested. section 4 provides an evaluation, analysis and discussion of the outcomes. finally, section 5 concludes the paper with suggestions for future research. ## related work the identification of offensive text in the english language has been studied in depth, and multiple offensive categories have been identified, including racist speech, religious hate speech, and sexism @xcite @xcite @xcite . in contrast, only a few studies have focused on identifying offensive language in arabic (al-hassan and al-dossari, 2019). one of these few studies was performed by @xcite who produced an extensive corpus of arabic tweets that were manually classified according to three categories: clean, obscene and offensive. a further contribution to the body of knowledge was offered by @xcite , who developed a corpus that consisted of offensive comments that had been posted on youtube in arabic. the dataset consisted of 16,000 egyptian, libyan and iraqi comments that were subsequently categorized into one of three classes: offensive, inoffensive and neutral. the researchers trained a support vector machine (svm) classifier to detect the offensive comments. the results indicated that the use of an n-gram feature could enhance the accuracy of the classifier, while using a combination of an n-gram with stemming diminished system performance. research that was performed by @xcite was limited to the use of arabic to share religious hate. they developed and ranked a lexicon of the religious hate terms that were in most common use and tested various classifiers. more recently, @xcite extended their existing corpus of offensive arabic words and used it to construct a large-scale training model that could be used to automatically detect offensive tweets. the researchers developed a character-level deep learning algorithm that was subsequently employed to classify each tweet as either offensive or inoffensive. @xcite created the largest arabic-language dataset to date and employed specific tags for hate speech and vulgarity. they subsequently analyzed the dataset to identify the topics that were most commonly associated with offensive tweets and to obtain meaningful insights into the use of hate speech among members of the arab community. finally, they proved the effectiveness of the system by performing a range of large-scale experiments on the dataset using svm techniques and generated promising results (@xmath0.7). ## methodology the proposed ensemble system consists of three different classifier models. we pre-processed the raw tweets as inputs (see section 3.1) to the models. sections 3.2, 3.3 and 3.4 describe the three different models which are then combined using an ensemble technique as described in section 3.5. ## experiment results our models were trained and tested on the training and dev sets released by the organisers. details to the data can be found in @xcite . the official evaluation metrics for this shared task came from macro-f1. the results of our models are presented in in addition, we assessed three pre-trained word embeddings: two word-level models (mazajak and ara2vec) and a char-level model. we engaged in a variety of evaluation exercises during which we compared the performance of each model individually and in combination. the char-level model and mazajak obtained an f1-score of 0.87, outperforming ara2vec (0.86) on the dev dataset. although the char-level model was trained on a dataset that consisted of just 10 million tweets, it achieved a result comparable to mazajak, which was trained on 250 million tweets. however, by combining the three models, we enhanced the accuracy of the results by around 1%. we noted that a char model can adequately address the oov problem. for example, in arabic, ( -alklbwbh), meaning female puppy, represents an offensive word. however, the mazajak and ara2vec pre-trained embeddings did not detect this word as potentially offensive. the char-level model adequately captured its meaning by encoding this word close to other related words that either have the same semantic meaning or a predominantly different form of this word. ## conclusion in this work, we proposed an ensemble approach that combines different levels of word embedding models and transfer learning from other sources of emotion-related tasks. the proposed system ranked 9th out of the 52 entries within the arabic offensive language identification subtask. in future studies, we hope to enhance the performance of our model by applying additional pre-processing techniques and more effectively exploiting a list of offensive words. additionally, we will investigate various methods by which data can be augmented into our training datasets to make them more robust.