A delaide C y C at S em E val-2020 Task 12: Ensemble of Classifiers for Offensive Language Detection in Social Media (2020)

Abstract: This paper describes the systems our team (AdelaideCyC) has developed for SemEval Task 12 (OffensEval 2020) to detect offensive language in social media. The challenge focuses on three subtasks – offensive language identification (subtask A), offense type identification (subtask B), and offense target identification (subtask C). Our team has participated in all the three subtasks. We have developed machine learning and deep learning-based ensembles of models. We have achieved F1-scores of 0.906, 0.552, and 0.623 in subtask A, B, and C respectively. While our performance scores are promising for subtask A, the results demonstrate that subtask B and C still remain challenging to classify.

## introduction the surge of internet and social media technologies provides a wealth of opportunities for cybercrime, and has led to the unprecedented social crisis of online abuse. despite the illegality of such behaviour, most social media platforms such as facebook, twitter, instagram are susceptible to online bullying due to their openness and anonymisation. the sheer amount of offensive language generation vastly exceeds the capacity of manual detection. therefore, there is a crucial need for urgent development of technological solutions. the automated identification of offensive language has been recognised as a subtask of nlp only recently and most of the advances have occurred in the last few years. the offensive language identification as an nlp problem is inherently complex and challengingeven for humans (aside from the offensive language's victims) due to many variants of language used by harassers such as coarse language, sarcasm, intimidation, and colloquialisms. people also tend to use coarse language in a friendly manner, without an intention to harm anyone. therefore, it is important to identify whether a post or a tweet is offensive and whether it is targeted at an individual or a group. in this paper, we focus on identifying offensive posts extracted from the twitter platform. the training dataset contained more than 9 million tweets and they were annotated using a semisupervised approach. our team has participated in english versions of all three subtasks organised by @xcite , i.e., subtask a: offensive language identification (offensive or not), subtask b: offense type identification (targeted insults and threats or untargeted), and subtask c: offense target identification @xcite . in this paper, we discuss the models developed for each subtask along with the performance. the advancement of offensive language identification has many benefits for social media and online communities to protect their users. ## related work the identification of unacceptable language in social media and online communities has attracted attention from researchers in related fields such as cyberbullying @xcite , aggression @xcite , hate speech @xcite , abusive language @xcite , and offensive language @xcite . @xcite is one of the first studies to create a dataset for offensive language detection by categorising tweets into hate speech, offensive but not hate speech, and neither. their work utilised various features such as n-grams, tf-idf, readability scores, and sentiments to build machine learning models like logistic regression and svm. recently, multiple classification tasks like offenseval @xcite and hateval @xcite contributed to the advancement of the research field by creating datasets and tasks to identify offensive language, type, and target @xcite and hate speech, target, and aggressiveness @xcite . the systems developed for these tasks used cutting-edge nlp, machine learning and deep learning techniques. some key systems for offenseval and hateval such as fermi @xcite used universal sentence encoder to build a svm model, nlpr@sapol @xcite used an ensemble of deep learning models like openai finetune and transformer, while nuli @xcite developed a bert-based model. although some systems have achieved reasonable performance (e.g., 0.82 f1-score for subtask a of offenseval by nuli), most other systems still lack 'good' performance for other subtasks such as identifying target and type of offenses. some of these challenges focus on specific problems like hate speech against minorities (women, migrants) @xcite while offenseval classification tasks focus on 'general' offensive language available on twitter. ## conclusion this paper presents the description of the systems we developed subtask a and c, we have developed ensembles of models using distilbert. in subtask b, our best performing model was an ensemble developed using logistic regression, linearsvc, robert, xlnet and bert. we have achieved promisi competition. despite the good results we have obtained for subtask b and c using the held systems could be further improved by this newer ensemble model was created after the official deadline for subtask c, and hence we could not submit it for the challenge. yet, after the official, labelled test dataset of subtask c was made available at the end of the challenge, we evaluated our system and observed a macro averaged f1 ranked 2nd amongst all submissions for subtask c.