CHILLAX - at A rabic Hate Speech 2022: A Hybrid Machine Learning and Transformers based Model to Detect A rabic Offensive and Hate Speech (2022)

Abstract: Hate speech and offensive language have become a crucial problem nowadays due to the extensive usage of social media by people of different gender, nationality, religion and other types of characteristics allowing anyone to share their thoughts and opinions. In this research paper, We proposed a hybrid model for the first and second tasks of OSACT2022. This model used the Arabic pre-trained Bert language model MARBERT for feature extraction of the Arabic tweets in the dataset provided by the OSACT2022 shared task, then fed the features to two classic machine learning classifiers (Logistic Regression, Random Forest). The best results achieved for the offensive tweet detection task were by the Logistic Regression model with accuracy, precision, recall, and f1-score of 80%, 78%, 78%, and 78%, respectively. The results for the hate speech tweet detection task were 89%, 72%, 80%, and 76%.

## introduction twitter and similar social media platforms users are from every race, religion, nationality, and background communicate and freely share their opinions and beliefs. the down side to this is that it is easy to exploit these social media platforms by sharing offensive and hate speech content that targets and threatens individuals or groups based on common characteristics, or identities. despite the considerable efforts that social media platforms are making to prevent such content from spreading, it is still threatening the online communities and users are still seeing it on many platforms. it is imperative to detect and prevent such content from appearing on social media platforms, thus motivating our research on arabic offensive and hate speech detection. pre-trained language models based on transformer @xcite such as gpt @xcite , bert @xcite , xlnet @xcite , and roberta @xcite have been shown to be effective for learning contextualized language representation achieving state-of-the-art results on a wide variety of natural language processing tasks. recent research has adopted the methodology of fine-tuning a pre-trained language model by simply adding a fully connected neural layer specific to the downstream task the model is being fine-tuned for, such as sarcasm detection @xcite and hate speech detection @xcite . research has shown that due to the numerous layers present in transformer models, simply feeding the output of the transformer's encoder final layer to the fully connected neural layer would restrict the power of the pretrained representations of the language @xcite . @xcite shows that different combinations of different output layers of the transformers encoder layers result in distinct performance on various tasks like named entity recognition task. it is found that the most contextualized representations of input text tend to occur in the middle layers, while the top layers are for language modeling @xcite . this research used different transformers' encoder layers as feature extractors. then we used the extracted features to feed into two classical machine learning classifiers; logistic regression and random forest. we used the marbert pre-trained transformer model as it was trained on a large arabic tweets corpora and has proved to be efficient in similar tasks such as sentiment analysis, where it scored 0.93 f1-score on the arsas dataset @xcite . also, we used logistic regression as a classifier as it proved superior on binary classification problems as the current subtasks. we conducted our experiments on the osact2022 dataset for subtask 1: arabic offensive, and subtask2: hate speech tweets detection. one of the challenges of the osact2022 dataset are that the dataset is imbalanced; the number of tweets on both subtasks aren't equal across the two classes, offensive and hate speech detection. we tackled this problem by using data augmentation techniques to achieve a balanced class distribution in the dataset to prevent the classifiers from biasing towards the majority class. the research paper is organized as follows. section 2 gives a brief overview of related work. section 3 explain in details our methodology and proposed model. section 4 presents the experiment results and evaluation metrics. section 5 concludes our research and our potential future work. ## related work recently, the interest in detecting hate speech has increased rapidly, attracting the attention of many researchers trying to develop various models and methods to extract hate features and hateful content. several research studies were conducted to study hate speech and offensive language in online communities and social media over arabic content. @xcite ) investigate 15 classical and neural learning models with tf-idf and word embedding as feature representations of the osact-2020 dataset; their best classifier (a joint architecture of cnn and rnn) achieved 73% macro f1-score on the development dataset and 69% on the test dataset with word embedding as feature representations. @xcite ) investigate several neural network models that are based on cnn and rnn to detect hate speech in arabic tweets and also evaluates recent bert model on the task of arabic hate speech detection. they built a hate speech dataset containing 9,316 annotated arabic tweets. they conducted experiments on that dataset and an out-domain dataset showing that the cnn model achieves an f1-score of 79% and au-roc of 89%. @xcite proposed an innovative deep learning approach for the detection of cyber hate speech. the detection of hate speech on twitter in the arabic region, in particular, using a word embedding mechanism for feature representation and fed to a hybrid cnn and lstm neural network that achieved a 71% f1-score on a dataset that is collected from the twitter api. @xcite collected a 3,000 tweet dataset from twitter where they experimented bow and tf-idf methods for feature representation and classical machine learning models (svm, nb, rf) and concluded that tf-idf with svm achieved the best results of 82% f1-score. so far, related work shows that word embeddings, tf-idf, and bow have been used as the feature representation for text experimented with traditional machine learning models as well as deep learning neural networks. in most cases, deep learning neural networks show superior results to classical machine learning models. this further motivates our approach to experiment with the feature representation of the different combinations of hidden layers in pretrained transformer models on classical machine learning models. ## results and discussion before training the model, the training dataset was split into 70% for training and 30% for testing to evaluate the model, along with the development and test datasets provided by the osact2022 shared task.