LIIR at S em E val-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification (2020)

Abstract: This paper presents our system entitled ‘LIIR’ for SemEval-2020 Task 12 on Multilingual Offensive Language Identification in Social Media (OffensEval 2). We have participated in sub-task A for English, Danish, Greek, Arabic, and Turkish languages. We adapt and fine-tune the BERT and Multilingual Bert models made available by Google AI for English and non-English languages respectively. For the English language, we use a combination of two fine-tuned BERT models. For other languages we propose a cross-lingual augmentation approach in order to enrich training data and we use Multilingual BERT to obtain sentence representations.

## introduction nowadays, with an exponential increase in the use of social media platforms such as facebook and twitter by people from different educational and cultural backgrounds, the need for automatic methods for recognizing and filtering offensive languages is necessary @xcite . different types of offensive content like hate speech @xcite , aggression @xcite and cyberbullying @xcite can be very harmful to the user's mental health, especially to children and youth @xcite . the offenseval 2019 competition @xcite was an attempt to build systems capable of recognizing offensive content in social networks for the english language. the offenseval 2019 organizers defined three subtasks: whether a message is offensive or not @xcite , what is the type of the offensive message (subtask b), and who is the target of the offensive message (subtask c). this year, they have extended the competition to several languages while the subtasks remain the same as in offenseval 2019. offenseval 2020 @xcite features a multilingual dataset with five languages including english, danish, turkish, greek, and arabic. this article presents our approaches to semeval-2020 task 12: offenseval 2 -multilingual offensive language identification in social media. we have participated in subtask a for all languages. the goal of subtask a is recognizing if a sentence is offensive or not. for the english language, we separately finetune two bidirectional encoder representations of the bert transformer architecture @xcite on two different datasets, and we use the combination of these two models for training our classifier. also, we perform an extensive preprocessing for the english language. for other languages, we enhance the provided training dataset for each language using a cross-lingual augmentation approach, then, we train a classifier by fine-tuning a multilingual bert (mbert) @xcite using the augmented dataset with a linear classification layer on top. our proposed augmentation approach, inspired by the works of @xcite , translates each training sample into three other languages, then, adds the original training sample concatenated with every translation to the training set. liir has achieved a competitive performance with the best methods proposed by participants in the competition for the english track. also, empirical results show that our cross-lingual augmentation approach is effective in improving results. the rest of the article is organized as follows. the next section reviews related works. section 3 describes the methodology of our proposed models. we will discuss experiments in section 4 and the results are presented in section 5. finally, the last section contains the conclusion of our work. ## methodology in this section, we present the proposed methods in more detail. we have participated in subtask a -categorizing a given sentence as 'offensive' or 'not-offensive' -for the english, turkish, arabic, danish, and greek languages. this year, offenseval organizers have provided labeled training data for all the languages except for english where they just have provided unlabeled training data. therefore, we propose two different approaches in this paper, one for the english language, and one for the other languages. for the english language, we fine-tune two bert models separately on two different datasets, and we use the combination of these two models in training our classifier. for the other languages, a cross-lingual augmentation approach is used for enriching each language's training set, and we fine-tune an mbert model to obtain sentence representations. in the following subsections, we describe our cross-lingual augmentation technique, and detail the proposed models. ## conclusion in this paper, we have presented our models for recognizing offensive content in the semeval-2020 task 12 subtask a for all languages. we have fine-tuned two bert models on two different datasets for the english language. moreover, we have fine-tuned an mbert model on the augmented training sets for the other languages by implementing a cross-lingual augmentation approach. the evaluation results show that the proposed systems are capable of effectively recognizing offensive content in language. as future work, we intend to investigate other augmentation techniques and utilize language-specific resources to improve the performance of our method in languages other than the english language. furthermore, we plan to address the problem of imbalance in the training set.