A rabic Offensive Language on T witter: Analysis and Experiments (2021)

Abstract: Detecting offensive language on Twitter has many applications ranging from detecting/predicting bullying to measuring polarization. In this paper, we focus on building a large Arabic offensive tweet dataset. We introduce a method for building a dataset that is not biased by topic, dialect, or target. We produce the largest Arabic dataset to date with special tags for vulgarity and hate speech. We thoroughly analyze the dataset to determine which topics, dialects, and gender are most associated with offensive tweets and how Arabic speakers useoffensive language. Lastly, we conduct many experiments to produce strong results (F1 =83.2) on the dataset using SOTA techniques.

## introduction disclaimer: due to the nature of the paper, some examples herein contain highly offensive language and hate speech. they don't reflect the views of the authors in any way. this work is an attempt to help fight such speech. much recent interest has focused on the detection of offensive language and hate speech in online social media. offensiveness is often associated with undesirable behaviors such as trolling, cyberbullying, online extremism, political polarization, and propaganda. thus, offensive language detection is instrumental for a variety of application such as: quantifying polarization @xcite , trolls and propaganda account detection @xcite , hate crimes likelihood estimation @xcite ; and predicting conflicts @xcite . in this paper, we describe our methodology for building a large dataset of arabic offensive tweets. given that roughly 1-2% of all arabic tweets are offensive @xcite , targeted annotation is essential to efficiently build a large dataset. since our methodology does not use a seed list of offensive words, it is not biased by topic, target, or dialect. using our methodology, we tagged a 10,000 arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. further, we labeled tweets as vulgar or hate speech. to date, this is the largest available dataset, which we plan to make publicly available along with annotation guidelines. we use this dataset to characterize arabic offensive language to ascertain the topics, dialects, and users' gender that are most associated with the use of offensive language. though we suspect that there are common features that span different languages and cultures, some characteristics of arabic offensive language are language and culture specific. thus, we conduct a thorough analysis of how arab users use offensive language. next, we use the dataset to train strong arabic offensive language classifiers using state-of-the-art representations and classification techniques. specifically, we experiment with static and contextualized embeddings for representation along with a variety of classifiers such as transformer-based and support vector machine (svm) classifiers. the contributions of this paper are as follows: many recent papers have focused on the detection of offensive language, including hate speech @xcite @xcite @xcite @xcite . offensive language can be categorized as: vulgar, which include explicit and rude sexual references, pornographic, and hateful, which includes offensive remarks concerning people's race, religion, country, etc. @xcite . prior works have concentrated on building annotated corpora and training classification models. concerning corpora, hatespeechdata.com attempts to maintain an updated list of hate speech corpora for multiple languages including arabic and english. further, semeval 2019 ran an evaluation task targeted at detecting offensive language, which focused exclusively on english @xcite . for semeval 2020, they extended the task to include other languages including arabic @xcite . as for classification models, most studies used supervised classification at either word level @xcite , character sequence level @xcite , and word embeddings @xcite . the studies used different classification techniques including naïve bayes @xcite , support vector machines (svm) @xcite , and deep learning @xcite @xcite classification. the accuracy of the aforementioned system ranged between 76% and 90%. earlier work looked at the use of sentiment words as features as well as contextual features @xcite . the work on arabic offensive language detection is relatively nascent @xcite @xcite @xcite . @xcite suggested that certain users are more likely to use offensive languages than others, and they used this insight to build a list of offensive arabic words and to construct a labeled set of 1,100 tweets. @xcite used supervised classification based on a variety of features including user profile features, textual features, and network features. they reported an accuracy of nearly 90%. @xcite used supervised classification based on word n-grams to detect offensive language in youtube comments. they im-proved classification with stemming and achieved a precision of 88%. @xcite focused on detecting religious hate speech using a recurrent neural network. arabic is a morphologically rich language with a standard variety called modern standard arabic (msa), which is typically used in formal communication, and many dialectal varieties that differ from msa in lexical selection, morphology, phonology, and syntactic structures. in msa, words are typically derived from a set of thousands of roots by fitting a root into a stem template and the resulting stem may accept a variety of prefixes and suffixes. though word segmentation, which greatly improves word matching, is quite accurate for msa @xcite , with accuracy approaching 99%, dialectal segmentation is not sufficiently reliable, with accuracy ranging between 91-95% for different dialects @xcite . since dialectal arabic is ubiquitous in arabic tweets and many tweets have creative spellings of words, recent work on arabic offensive language detection used character-level models @xcite . ## experiments we conducted an extensive battery of experiments on the dataset to establish strong arabic offensive language classification results. though offensive tweets have finer-grained labels where offensive tweet could also be vulgar and/or hate speech, we conducted coarser-grained classification to determine if a tweet was offensive or not. for classification, we experimented with several tweet representation and classification models. for tweet representations, we used: the count of positive and negative terms, based on a polarity lexicon; static embeddings, namely fasttext and skip-gram; and deep contextual embeddings, namely bert base-multilingual and arabert (antoun et al., 2020). ## conclusion and future work in this paper we presented a systematic method for building an arabic offensive language tweet dataset that does not favor specific dialects, topics, or genres. we developed detailed guidelines for tagging the tweets as clean or offensive, including special tags for vulgar tweets and hate speech. we tagged 10,000 tweets, which we plan to release publicly and would constitute the largest available arabic offensive language dataset. we characterized the offensive tweets in the dataset to determine the topics that illicit such language, the dialects that are most often used, the common modes of offensiveness, and the gender distribution of their authors. we performed this breakdown for offensive tweets in general and for vulgar and hate speech tweets separately. we believe that this is the first detailed analysis of its kind. lastly, we conducted a large battery of experiments on the dataset, using crossvalidation, to establish a strong system for arabic offensive language detection. we showed that using an arabic specific bert model (arabert) and static embeddings trained on tweets produced competitive results on the dataset. for future work, we plan to pursue several directions. first, we want explore target specific offensive language, where attacks against an entity or a group may employ certain expressions that are only offensive within the context of that target and completely innocuous otherwise. second, we plan to examine the effectiveness of cross dialectal and cross lingual learning of offensive language.