WOLI at S em E val-2020 Task 12: A rabic Offensive Language Identification on Different T witter Datasets (2020)

Abstract: Communicating through social platforms has become one of the principal means of personal communications and interactions. Unfortunately, healthy communication is often interfered by offensive language that can have damaging effects on the users. A key to fight offensive language on social media is the existence of an automatic offensive language detection system. This paper presents the results and the main findings of SemEval-2020, Task 12 OffensEval Sub-task A Zampieri et al. (2020), on Identifying and categorising Offensive Language in Social Media. The task was based on the Arabic OffensEval dataset Mubarak et al. (2020). In this paper, we describe the system submitted by WideBot AI Lab for the shared task which ranked 10th out of 52 participants with Macro-F1 86.9% on the golden dataset under CodaLab username “yasserotiefy”. We experimented with various models and the best model is a linear SVM in which we use a combination of both character and word n-grams. We also introduced a neural network approach that enhanced the predictive ability of our system that includes CNN, highway network, Bi-LSTM, and attention layers.

## introduction social media platforms such as facebook and twitter provide an online space for individuals or groups to express their opinions often with limited restrictions. hence, social media users may use filthy, sexual, or offensive language @xcite . offensive phrases could ridicule or insult an individual or a group @xcite . many applications involve offensive language detection, for instance, parental control or adult content filters; identifying hate speech that incites or presage violence @xcite ; identifying troll accounts @xcite ; conflict detection, which is often lead up to verbal hostility @xcite ; quantifying the intensity of polarisation @xcite . in addition to the huge effort required by human annotators, the risk of applying discrimination or favouritism should be placed under consideration. moreover, a manual annotation task by human annotators would increase the system response times drastically, since a computer-based solution can perform this task much faster than humans. with the rapid growth in the amount of user-generated content in the social media services, the scalability of manual filtering highlights the need for automating the process of on-line hate-speech detection @xcite . offenseval, semeval-2020 task 12 sub-task a @xcite aims to train a model on arabic tweets collected from twitter, annotated by human annotators to identify the offensive language within the messages. in this paper, we describe the system submitted by widebot ai lab for the shared task of offensive language identification in arabic and discuss the results of our experiments and it is organised as follows: section 2 discusses important details about the task and data used. it additionally demonstrates the previous work done on this task. section 3 discusses all the datasets used in training. section 4 presents an overview of the key algorithms. section 5 explains the procedures of the experiments and evaluation metrics. section 6 compares the results. section 7 shows the conclusions and future works. ## background the goal of this task is to distinguish between offensive and non-offensive messages. offensive messages often contain insults, threats, and any form of targeted profanity. the dataset provided for the task is about 10,000 annotated messages. each message is assigned one of the following two labels: ## dataset this subsection shows the used dataset beyond the provided training data by arabic offenseval shared task @xcite which are twitter examples with labels not for non-offensive examples and off for offensive examples. we faced a dataset imbalance problem since the provided dataset has 1,915 examples for off and 8,085 examples for not, so we decided to use the following additional datasets in our experiment: ## proposed models this subsection shows the key algorithms and modeling decisions we used in our experiments. we used a logistic regression model on a bag of words of the training dataset as our baseline and other two main systems which are linear svm from classical models and the other one is based on recently developed neural network architectures. ## experimental setup we train different models for the classification task. in this section, we explain the preprocessing and training phases. the classifiers we experimented with include svm with features of a word and character n-grams and the chba network. we experimented using both the original dataset only and the external datasets mentioned before. we have chosen the svm model with the offenseval dataset and other external datasets since it performs the best score in our experiments until the time of submission, but we did further experiments using the chba network on the offenseval dataset only with hyperparameter optimisation which did better than the submitted one. the evaluation metric of this task is macro-f1 which calculates the unweighted-mean f1 of the two classes since the imbalance of the data classes affects the macro-f1, and usually, the score is penalised by the minority classes. ## conclusion in this paper, we presented the results and the main findings of semeval-2020 task 12 offenseval -arabic task on identifying and categorising offensive language in social media in which we carried out different experiments with linear svm using different features and chba network. we also used a feature selection approach to deal with the high sparsity of the data, also we have shown how stacking the tf-idf on word n-grams and character n-grams of the task dataset with external datasets could improve the performance of the model. however, hyperparameter optimisation and the usage of pre-trained word embeddings for the chba network showed its effectiveness in this task as it outperformed the linear svm model knowing that we just used the dataset provided by offenseval only. future work includes investigating text representations, improving the deep learning model, and releasing widebot conversational dataset from different dialects for offensive language identification tasks.