cs@ D ravidian L ang T ech- EACL 2021: Offensive Language Identification Based On Multilingual BERT Model (2021)

Abstract: This paper introduces the related content of the task “Offensive Language Identification in Dravidian LANGUAGES-EACL 2021”. The task requires us to classify Dravidian languages collected from social media into Not-Offensive, Off-Untargeted, Off-Target-Individual, etc. This data set contains actual annotations in code-mixed text posted by users on Youtube, not from the monolingual text in textbooks. Based on the features of the data set code mixture, we use multilingual BERT and TextCNN for semantic extraction and text classification. In this article, we will show the experiment and result analysis of this task.

## introduction one of the manifestations of the rapid development of the internet is the increasing number of users, which allows more and more people from different languages, different regions, and different cultures to communicate, but the frequent appearance of offensive speech will affect this harmonious atmosphere @xcite . this has become a serious problem for users of online communities and social media platforms. in such a multilingual social environment, it has become a serious problem for users of online communities and social media platforms @xcite @xcite . this task is to identify offensive language content from dravidian languages (tamil-english, malayalam-english, and kannada-english) collected from social media. the dravidian civilization of the indus valley civilisation (3,300-1,900 bce) is believed to have flourished in the northwestern indian subcontinent (tamil). the dravidian languages were first documented in tamil-brahmi script engraved on cave walls in tamil nadu's madurai and tirunelveli districts in the 6th century bce. tamil is india's oldest language. agglutinative languages are dravidian languages. subject-object-verb is the word order (sov). there is a clusivity distinction in most dravidian languages. all we need to do is to classify it into not-offensive, offensive-untargeted, offensive-targeted-individual, offensive-targetedgroup, offensive-targeted-other, or not-in-indentedlanguage. to further extract semantic information, we adopt a text classification method based on the hierarchical connection between bert and textcnn, which is a combined model of bert and textcnn. use multilingual bert to vectorize each word, obtain the semantic features of the text, and construct the text mapping matrix. use textcnn convolutional neural network to perform convolution operation on text mapping matrix, get the output of all or part of the hidden layer, get the semantic feature matrix of the text, then use the pooling algorithm to reduce the dimension of the semantic feature matrix of the text to obtain the semantic feature vector of the text. ## related work nowadays, offensive, false, and other remarks on social media have become issues that we have to pay attention to. to solve this problem, many scholars have done a lot of research activities. it proposed a cnn-gram deep learning architecture @xcite for hate and offensive language detection in social media and compared its performance with the current baseline method, the model shows higher robustness. @xcite used the functional template described by yarowsky to model hate speech as a classification problem to detect hate speech on the internet. it trained the classifier through semi-supervised machine learning technology (epstein and mengibar, 2015), training the classifier to determine if there are potentially offensive terms in the text. @xcite proposed a detection scheme when distinguishing hateful content on social media. it is a combination of recurrent neural network (rnn) classifiers. it contains various functions related to user-related information and achieves relatively high classification quality. @xcite used random forest methods to identify twitter hate speech datasets and compared them with the accuracy results of neural networks and adaboost. @xcite found that in sentiment classification, logistic regression, random forest classifier, and decision tree performed relatively well, svm performed poorly, and its heterogeneity was also poor. it used a hate-bert model retrained @xcite ) on ral-e (offensive and hateful reddit english data set) for the detection of abusive language and found that hatebert is r tosuperio the corresponding conventional bert model. @xcite fine-tuned the bert and modeled the bert as a basic neural network. this model can enhance its detection performance and achieve good accuracy at a low computational cost. @xcite proposed a deep convolution model that uses unsupervised pre-trained word embedding to classify objectionable text. it used the pre-trained arabic language model arabert in the task of offensive language detection, which @xcite showed good performance in classification tasks. @xcite proposed a stack weighted ensemble (swe) model with five independent classifiers to detect hate speech. @xcite tried to use deep learning convolutional neural network models and creating cnn models to classify twitter's hate speech text. @xcite proposed a systematic method to construct a data set of tweets that do not support specific dialects and topics and use crossvalidation to establish offensive language system detection. ( chakravarthi et al., 2020a, 2021 ## data the common feature of the data sets in three different languages is code-mixed. code-mixed refers to words in multiple different languages that may appear in the same sentence. the data we used in this task is mainly text mixed with english ((tamil-english, malayalam-english, and kannada-english)). labels distribution of malayalam training set and validation set. in the training set, not offensive: 88.4%, not malayalam: 8.04%, offensive targeted insult individual: 1.49%, offensive untargetede: 1.19%, offensive targeted insult group: 0.88%. in the validation set, not offensive: 89%, not malayalam: 8.15%, offensive targeted insult individual: 1.2%, offensive untargetede: 1%, of-fensive targeted insult group: 0.65%. labels distribution of kannada training set and validation set. in the training set, not offensive: 57.01%, not kannada: 24.48%, offensive targeted insult individual: 7.83%, offensive targeted insult group: 5.29%, offensive untargetede: 3.41%, offensive targeted insult other: 1.98%. in the validation set, not offensive: 54.83%, not kannada: 24.58%, offensive targeted insult individual: 8.49%, offensive targeted insult group: 5.79%, offensive untargetede: 4.25%, offensive targeted insult other: 2.06%. labels distribution of tamil training set and validation set. in the training set, not offensive: 72.23%, offensive untargetede: 8.27%, offensive targeted insult group: 7.28%, offensive targeted insult individual: 6.67%, not-tamil: 4.14%, offensive targeted insult other: 1.29%. in the validation set, not offensive: 72.77%, offensive untargetede: 8.11%, offensive targeted insult group: 6.72%, offensive targeted insult individual: 7%, not tamil: 3.92%, offensive targeted insult other: 1.48%. the data sets we can use are the training set and validation set in three languages(tamil, malayalam, and kannada) provided by the task organizer team. code mixing is the main feature of the data set provided by the task organizer. there are five different categories in the malayalam dataset. each dataset of kannada and tamil has six different categories. there is an imbalance of category labels in the data sets of the three languages. ## methods because the deep learning model can learn the complex distribution characteristics of data through deep artificial neural networks and nonlinearity. especially the use of deep learning in tasks related to text data has attracted more and more attention @xcite . ## conclusion for this task, we use multilingual bert and textcnn to complete the detection of offensive speech. the above is our description of this task. the result of this task may not be ideal. therefore, in future work, we try to use new models and improve methods to obtain better results. this task has given us a better understanding of the detection of speech on social media. it not only has an understanding of the importance of the detection of offensive speech but also improved our ability to solve such problems. in the future, we will continue to learn about social media speech detection.