Is it Offensive or Abusive? An Empirical Study of Hateful Language Detection of A rabic Social Media Texts (2024)

Abstract: Among many potential subjects studied in Sentiment Analysis, widespread offensive and abusive language on social media has triggered interest in reducing its risks on users; children in particular. This paper centres on distinguishing between offensive and abusive language detec- tion within Arabic social media texts through the employment of various machine and deep learning techniques. The techniques include Naïve Bayes (NB), Support Vector Machine (SVM), fastText, keras, and RoBERTa XML multilingual embeddings, which have demon- strated superior performance compared to other statistical machine learning methods and dif- ferent kinds of embeddings like fastText. The methods were implemented on two separate corpora from YouTube comments totalling 47K comments. The results demonstrated that all models, except NB, reached an accuracy of 82%. It was also shown that word tri-grams en- hance classification performance, though other tuning techniques were applied such as TF-IDF and grid-search. The linguistic findings, aimed at distinguishing between offensive and abu- sive language, were consistent with machine learning (ML) performance, which effectively classified the two distinct classes of sentiment: offensive and abusive.

## introduction social media streams such as x (previously known as twitter) and youtube apply individual policies to control the content posted by internet users @xcite . despite having in place automatic methods, there is no guarantee that all the unsuitable content is detected. that is because it is challenging to completely filter out all slang, misspelling, and dialectal terms. the filters, based on sentiment analysis techniques, detect the targeted content typically depend on keyword lists, rule-based approaches, and machine learning algorithms to classify sentiments. the significance of sentiment analysis escalates with the growth of unsuitable content disseminated across social media platforms on a daily basis. it is a necessary pre-requisite for categorising personal opinions into positive, negative, or neutral. accordingly, this paper concentrates on establishing consistent definitions of unsuitable language prevalent on social media discourse and developing a robust sentiment analysis classifier to detect offensive and abusive language in arabic. classifying positive and negative text poses plenty of challenges. however, one of the most difficult challenges is dealing with dialectal terms in arabic. millions of arab users on social media use a combination of modern standard arabic (msa) and dialects to post their opinions. the complexity of arabic dialects is underscored by their common linguistic characteristics intertwined with polysemous words that share identical structures but harbour multiple meanings, which can influence the classification process in machine learning. for instance, the verb ‫ي‬ ُ ‫ب‬ ‫قَ‬ ّ ِ ‫ي‬ (yubaqi), typically meaning "to remain" or "to keep in one's possession", acquires a distinct interpretation in omani dialects, particularly in buhla and al-hamra foot_0 . the influence of polysemous words among dialects is not a major problem in a non-offensive context. however, it is a significant issue when it occurs in sensitive contexts such as gender, religion and race @xcite . the following sentence: ‫و‬ ‫ت‬ ‫يج‬ ‫ك‬ ‫ﺍ‬ ‫ل‬ ‫م‬ ‫م‬ ‫ح‬ ‫و‬ ‫ن‬ ‫ه‬ ‫ت‬ ‫ق‬ ‫و‬ ‫ل‬ ‫ه‬ ‫ذ‬ ‫ﺍ‬ ‫ف‬ ‫ن‬ accordingly, disseminating offensive, obscenities, profanities and insulting content on social media by using informal or dialectal language may possibly constitute a challenge to classifiers trained on sentiment analysis to detect harmful content. inexperienced social media users such as children and teenagers could be affected by viewing undetected abusive and offensive sentiments. making consistent definitions for offensive, abusive and clean content by specifying their exact linguistic and cultural features is a significant technical challenge in terms of classifying the three classes by means of the existing tools for arabic text. as @xcite commented, there is no clear explanation regarding why offense is an inappropriate way to behave. it is enough to say that it is not part of the culture and moral rules. coughlan (2016), gathered interesting results in a case study showing that three-quarters of social media users aged between 11 to 12 years old had faked their ages to browse adult content, while two-thirds of children did not report offensive content on social media. the results of this behaviour go in par with the findings of @xcite , who states that the use of strong language by children may possibly reflect on their ability to be ethical and responsible parents in the future. ## data by reviewing the literature about offensive language, two separate datasets from previous studies form the corpus of this paper; @xcite and @xcite . it was collected from various youtube channels in an effective way, where the videos uploaded on those channels display celebrities in controversial footage with the aim of provoking viewers to use strong language in response. this led to a rich corpus of offensive words being collected. the annotation which is binary has only two classes: offensive and non-offensive. the inter-annotation agreement is reasonably good (71%). a strong point in this study is that it did not collect the data based on predefined profane words as the previous studies have done, for the reason that it lessens the ability of the predictivity of the tools proposed. despite the fact that it being highlighted as the largest dataset in tackling arabic offensive language, it appears that the dataset utilised by @xcite , is larger than alakrots. mubaraks includes 1100 tweets and 32k comments collected from the aljazeera channel on youtube covering various topics, such as politics, society, the economy and science. the annotation classes are obscene, offensive or clean. the inter-annotation agreement is relatively high, 87%. 2 the datasets are publicly available at: https://github. com/etconlp/offensive-detection.git moving the argument along, remarkably, there are certain offensive ideas that are found in many discussions on arabic social media regardless of what the users are commenting about. for example, it is common to notice offensive remarks on ideas relating to the sunni-shii conflict 3 , complaints about terrorism and comparing people to jews when their behaviour is very poor. ## results to examine how pre-processing and stemming affect the classification performance positively or negatively, four versions of each dataset were examined in the implementation of nb and svm. the versions included the following: pure dataset (pipeline), stemmed dataset, pre-processed dataset, in addition to the stemmed and pre-processed dataset. the best results were obtained by linear kernel and rbf kernel (b) by assigning various features for instance cache size (200), gamma (scale), and max iterations (-1). their f-score is 79%. the results show that the improvement in performance of fasttext is slightly higher than nb and svm. despite the fact those results are the best in each feature, the best f-score was obtained by using the word tri-gram feature. ## conclusion this paper concentrated on classifying and distinguishing offensive and abusive language on social media, youtube in particular. nb, svm and fasttext, keras, and roberta xml multilingual embeddings algorithms were implemented on two separated datasets, binary and multi-class, comprising 47k comments in total, and demonstrated high performance in relation to classification. the fasttext algorithm surpassed the others by achieving 82% accuracy. the tests on fasttext confirmed that using the word tri-gram feature improves the accuracy of this classification topic. it is also important to note that ability of classifying offensive and abusive languages shown in the results tried to prove the definitions of offensive and abusive language agreed in the literature review, that language which contains hateful and aggressive remarks is offensive, whereas language that includes vulgar, pornographic and sexual remarks is abusive. however, lack of balance in the amount of offensive and abusive comments led to lower accuracy. in the future, we will work on collecting a multiclass dataset that is large enough and balanced to run more deep learning models to enhance classification.