C yber T ronics at S em E val-2020 Task 12: Multilingual Offensive Language Identification over Social Media (2020)

Abstract: The SemEval-2020 Task 12 (OffensEval) challenge focuses on detection of signs of offensiveness using posts or comments over social media. This task has been organized for several languages, e.g., Arabic, Danish, English, Greek and Turkish. It has featured three related sub-tasks for English language: sub-task A was to discriminate between offensive and non-offensive posts, the focus of sub-task B was on the type of offensive content in the post and finally, in sub-task C, proposed systems had to identify the target of the offensive posts. The corpus for each of the languages is developed using the posts and comments over Twitter, a popular social media platform. We have participated in this challenge and submitted results for different languages. The current work presents different machine learning and deep learning techniques and analyzes their performance for offensiveness prediction which involves various classifiers and feature engineering schemes. The experimental analysis on the training set shows that SVM using language specific pre-trained word embedding (Fasttext) outperforms the other methods. Our system achieves a macro-averaged F1 score of 0.45 for Arabic language, 0.43 for Greek language and 0.54 for Turkish language.

## introduction offensive language is very common in social media, now-a-days. individual users frequently take advantage of the perceived anonymity of computer mediated communication, using this to engage in behavior that many of them would not consider in real life. the semeval-2020 task 12 (offenseval) challenge focuses on prediction of presence of offensive language using the social media. the main goal of this task is to instigate discussion on the creation of reusable benchmarks for evaluating proposed algorithms by exploring issues of evaluation methodology and other processes related to the creation of test collections. the given corpora are developed using the posts and comments over twitter, a popular social media. @xcite organized a multilingual offensive language classification task with a particular focus on twitter posts. it has released different corpora for the individual languages, e.g., arabic @xcite ), danish ( sigurbergsson et al. (2020) ), english @xcite ), greek @xcite ) and turkish (coltekikin (2020)), and intended to identify and capture the offensive language. all the languages except english have only one task to be performed, e.g., sub-task a which is to identify offensive language. for english, the task has been divided in three different sub-tasks, namely, sub-task b for offense type categorization in which the offense type is categorized into either targeted or untargeted, and sub-task c focuses on identification of target offense. in this paper, different machine learning and deep learning frameworks have been proposed to accomplish the given task. each of the proposed systems has been implemented using language specific fasttext, pre-trained word vectors developed over crawling the web. support vector machine (svm) is widely used for text categorization as introduced by @xcite . @xcite recommended the linear kernel for text categorization as it performs well when there exists a lot of features. hence linear svm has been used in our experiments. convolutional neural network (cnn) had been invented by @xcite for extraction of local features, which later had been proven to be the standard choice for image processing tasks. also, @xcite introduced long short term memory (lstm) to capture implicit ordering of the sequence data in terms of words and sentences, which is widely used in various natural language processing tasks. subsequently, svm as machine learning classifier and hybrid network of cnn with lstm as deep learning classification system are proposed and those have been implemented for efficiently identifying the presence of offensive languages in text. the results on the test set submitted to the challenge suggest that these frameworks achieve reasonably good performance. however, there are some submissions for this task, whose performances are better than our proposed framework. the paper is organised as follows. related literature reviews have been provided in section 2. the corpora used in these experiments is described in section 3. the proposed machine learning and deep learning frameworks are explained in section 4. section 5 describes the experimental evaluation. the conclusion is presented in section 6. ## related work identifying offensive language over social media has been an increasingly trending issue over the past few years. @xcite showed a survey of different text mining approaches for effectively identifying these issues. each existing language contains it's own language rule which comprises of different syntactic and semantic guidelines. for this, to accomplish the same goal over different languages, it requires different methodologies and approaches. ## data description the corpora released as part of the semeval 2020 @xcite ) are the collection of posts or comments from a set of users over twitter. each of the corpora is divided into two categories -offensive(off) and not-offensive(not). figures 1 shows the distribution of the classes in the data provided for arabic, greek and turkish languages, respectively. the distributions clearly show the imbalance in class labels. the overview of each of the corpora is presented in ## proposed methodologies the given corpus for each of the languages of semeval-2020 task 12 @xcite ) is further divided into two sets, namely, training set and validation set. the new training set is developed by randomly choosing 80% tweet samples from not and off categories; the rest 20% of these categories form the validation set. to train our models, support vector machine -a classical machine learning text classification algorithm is utilized. as a deep learning model, the hybrid model integrating networks of cnn with lstm has been trained for accomplishing the task. our proposed framework consists of three crucial layers: data preprocessing, feature extraction and text categorization. a high-level overview of our proposed framework and steps of producing clean text from raw text is shown in figure 2 . we have kept this architecture uniform for offensive language detection for all the three languages. ## experimental results we have reported the performance of both the machine learning and deep learning frameworks on our validation set for each of the corpora in ## conclusion in this paper, we have reported our proposed frameworks and their corresponding performances in sub-task a of semeval-2020 task 12 offenseval, for the languages of arabic, greek and turkish: multilingual offensive language identification in social media (offenseval 2020). we have learned that our proposed frameworks did not perform well because of two possible issues which are as follows: firstly, all the language corpora are highly imbalanced, i.e., insufficient quantity of tweet samples in either of the classes and secondly, for the low resource language, e.g., greek, the qualities of produced word vectors are much lower than those of other resourceful languages. we understand that an increase in the volume of data could add more potential to our framework, and use of external resources of data could be advantageous.