TAC at S em E val-2020 Task 12: Ensembling Approach for Multilingual Offensive Language Identification in Social Media (2020)

Abstract: Usage of offensive language on social media is getting more common these days, and there is a need of a mechanism to detect it and control it. This paper deals with offensive language detection in five different languages; English, Arabic, Danish, Greek and Turkish. We presented an almost similar ensemble pipeline comprised of machine learning and deep learning models for all five languages. Three machine learning and four deep learning models were used in the ensemble. In the OffensEval-2020 competition our model achieved F1-score of 0.85, 0.74, 0.68, 0.81, and 0.9 for Arabic, Turkish, Danish, Greek and English language tasks respectively.

## introduction with the growing and easy access to social media without check and balance, ethical and moral values are somehow put aside. one of the main issues is abusive behaviour, cyberbullying and offensive language. recently a lot of work @xcite ,(i orts, 2019) was performed to tackle this problem. most of the research work was done for languages like english and arabic @xcite , and less work for languages like danish, turkish and greek. the semeval 2020 task 12 @xcite consisted of identification of offensive words in different languages such as arabic, turkish, english, greek and danish. this paper proposed an ensemble approach for offensive language identification in social media text. we have participated in all languages tasks. for arabic, turkish, danish and greek language, we used fasttext embeddings. for english subtask a, we used fasttext embeddings combined with glove embeddings. the datasets were skewed, for which we used the focal loss as it gives more weightage to less occurred class in the dataset. this paper is organized as follows. section 2 presents related work in the area of offensive language detection. section 3 deals with more detail of provided dataset and the methodology used for the tasks. results are discussed in section 4. section 5 concludes the paper and presents some ideas for future work. using artificial neural network @xcite on a dataset comprised of 3000 turkish tweets, f1 score of 91% was obtained. a study @xcite comprised of twitter tweets and instagram post showed that using multinomial naive bayes cyberbullying can be classified with an accuracy of 84 % using tf-idf as features. turkish language sub-task organizer (c ¸öltekin, 2020) collected the tweets and classified them as offensive vs non offensive tweets with an f1-score of 77.3%. they categorized the offensive tweets into further categories as targeted or non targeted. if the category is targeted, it is further categorized as group targeted, individual targeted or other. they used both character grams and words grams as features and linear svm as classifier. sigurbergsson and derczynski (2020), the task organizers for danish language, got an f1 score of 70% while classifying offensive vs non offensive text using logistic regression. authors have categorized their data in a similar hierarchical way as proposed by @xcite . they collected the data from twitter, facebook and reddit. their data is skewed as 88% of the posts were labelled as not offensive and rest as offensive. in low resource language like danish, because of over fitting in term of word distribution on training data, accuracy drop when moving from one domain such as film/movie reviews to another domain such as company reviews. so an offline approach to solve this problem was proposed @xcite . the authors trained a base model, created k copies of training data, then corrupted these copies based on weights of base model. finally they train on corrupted training data. this tended to increase performance on unseen data and help in inter domain classification. using combination of lexicon and word2vec trained on greek wikipedia @xcite , support vector machine classifier tended to achieve an f1 score of 77.85%. the dataset collected from greek e-shopping website consisted of product reviews. using tf-idf uni-gram features and linear support vector machine @xcite ) macro f1 score of 80% was achieved. they tried different deep learning models and lstm and gru with attention mechanism resulted in f1 score of 89% . english offensive language identification task is similar with the task held last year @xcite , where more than 100 team submitted their papers. team nuli @xcite achieved highest f1 score using bert uncased model. hassan and al-dossari (2019) did a literature review on detection of hate speech on social media. they covered multilingual corpus in their survey. ## conclusion and feature work in this paper, we classified offensive language in multiple languages text such as english, arabic, danish, greek and turkish. we proposed a similar ensemble pipe line for all language tasks. we use count vectorizer, word level tf-idf, character level tf-idf as feature for logistic regression. two variants of recurrent neural network and convolution neural network each are applied to classify offensive language. we found that character level tf-idf work better in almost all cased, except english, where training was performed on olid dataset. in deep learning methodology, instead of using traditional binary cross entropy as loss functions, we tried focal loss. instead of fixed learning rate, cyclic learning rate was used. as future work, we will try multilingual bert and language specific bert model such as arabert @xcite . for english language task, we will try to overcome hardware limitations and train our ensemble pipeline on larger dataset.