Categorizing Offensive Language in Social Networks: A C hinese Corpus, Systems and an Explainable Tool (2020)

Abstract: Recently, more and more data have been generated in the online world, filled with offensive language such as threats, swear words or straightforward insults. It is disgraceful for a progressive society, and then the question arises on how language resources and technologies can cope with this challenge. However, previous work only analyzes the problem as a whole but fails to detect particular types of offensive content in a more fine-grained way, mainly because of the lack of annotated data. In this work, we present a densely annotated data-set COLA

## introduction in modern society, the occupation of offensive language on the online world, such as social media, is becoming a paramount concern. offensive language differs considerably, ranging from pure abuse to more rigorous types of writing. thus, offensive language is hard to be automatically identified. however, it's essential to track this; for example, the appearance of offensive language on social media is related to hate crimes in a real social situation. @xcite . moreover, it can be pretty troublesome to distinguish fine-grained offensive language because few general definitions exist @xcite . recently, researchers have proposed some guidelines to identify the type and the attributes of offensive language @xcite ]. however, the online world's offensive language is a general category containing specific examples of profanity or insult. in our work, "offensive language" in the online world is defined in more detail and fine-grained. and to the best of our knowledge, though offensive language identification being a burgeoning field, there is no data-set yet for chinese. in this context, we present a sizeable chinese classification corpus of offensive language called cola. then, we employ a deep dilated capsule network to extract hierarchical structure. we further design hierarchical attention to aggregate and fully utilize information within a hierarchical representation. correctly, each sentence is embedded into capsules and incrementally distilled into task-relevant categories during the hierarchical attention process. what is more, we present an explanation tool, which proves that our work for the chinese language seizes the pattern of offensive language in some points, and almost correctly identifies different varieties of offensive language, like hate speech and cyberbullying. in summary, our work aims at answering the two questions a) how lrs can cope with a huge amount of offensive language in the online world, and b) can lt provide means to process such language data at high speed? the major contributions are highlighted as follows: ## data collection in this section, we describe the data set and how we annotate data set. we create a large-scale data-set that annotates offensive texts in chinese. the texts are crawled from youtube and weibo: 18.7k comments in total. three annotators categorised these texts in four classes: neutral, insulting, antisocial, and illegal. we build a chinese dataset from social media that people can communicate on internet, such as sina weibo 0 and youtube comments. our released cola contains user-generated comments from different social media platforms, and as we know, it is the first of its kind. and, the dataset is marked as capture different types of offensive language. we propose four automatic classification systems, each designed to work for the chinese language. ## proposed methods in this part, we describe the categorizing task to be performed, how we perform the task and the excellent methods are proposed for especially this task. ## experiments and results for the training details, the method of cross-entropy loss was used in cnn, rnn, and bert. and we used adam as the optimizer, and the learning rate is set as 0.001. and also, we added dropout and early stop trick. the dropout trick randomly abandons a certain proportion of nodes in the training process to prevent the occurrence of over-fitting. finally, dropout is adopted as 0.5. the effects of the model are evaluated by early stop technology on the validation set after each iteration. when the validation set's evaluation result no longer improves in n consecutive rounds, the iterative process is truncated, and the process of training is suspended. the number of n has impacts on the time consumed and whether the model is converge or not. we implemented our models with pytorch, moreover, we finetuned the pre-trained based model, bert, on two nvidia rtx 2080ti gpus. we also performed an ablation analysis of our hacn model. the weighted f1 drops 1.1% when we remove the hierarchical attention module. we carry out several experiments during the evaluation phase, and the best experiment is taken into account for the evaluation phase. the systems are evaluated with the official competition metric, precision, recall, and macro averaged f1 score. what is more, this task is multiple classifications, so macro average, weighted average, micro average are also employed. in the filtering of insulting language in chinese, the best performing model achieves a macro averaged f1-score of 94.05%. in the situation of antisocial language, the best performing system for the chinese achieves a macro averaged f1-score of 97.18%. finally, in the detection of illegal language, the state of the art nlp model for the chinese is up to a macro averaged f1-score of 67.30%. however, we adopted a weighted average as a uniform evaluating method due to the unbalanced classes. our hierarchical attention capsule network (hacn) is the best-performed model with an excellent result of 94.86%. ## analysis in our experiment, we can find that rnn has an acceptable performance, however, there are some obvious shortcomings. when handling unbalanced data, such as a high recall rate, fail to classify certain classes because the class lacks visible character. it is also necessary to remark that the experiments are performed with the default parameters. thus there is an additional field for improvement with some finetuning, which we plan to consider for future research. moreover, we note that the label distribution is extremely imbalanced because there might be a bias introduced by the algorithms. extending from the experiment above, we present a comparison experiment in figure , where we record the valid accuracy over time and spot trends with different systems. figure 3 illustrates that our proposed hacn model can quickly converge to its stable equilibrium values. in the meantime, the starting point shows that hacn can get a promising result in a short time (after the first epoch). figure 4 shows times spent in the testing step for 3,759 test sentences when using different systems. the curves generated with these results suggested that our proposed hacn model can achieve the classification at the quickest speed. considering the deployment in a sound system, we only compared the testing step. nevertheless, we believe we still have substantial advantages in training because there is no need to pretrain the model on a large number of data compared with bert. furthermore, we show a deep analysis of the mis-classified cases in the evaluation process of our experiments. thus, we do a manual analysis for those mis-classified samples. this analysis aims at getting a deep comprehension of the areas our classifiers are lacking in. our model fails to classify some metaphorical offensive words. it is usual for human to use euphemisms to tone down swear words in some situations. for some other cases, our model classifies some profanity text as offensive, which is actually not offensive. the classifier could miss these word variants, especially when the word variant is the only offensive word in the given sentence. in another situation, the word "sucks" is the only word that is often used offensively. however, the given tweet is not offensive because the author only describes their mood instead of insulting someone else. these misclassifications seem to indicate that proceedings of the 19th china national conference on computational linguistics, pages 1045-1056, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 the classifier reacts to trigger words with negative connotations but may not be capable of interpreting the words concerning the broader context. 8 conclusion and future work this work presents a chinese corpus of offensive language crawled from microblog entries and video comments and manually categorized into 4 categories, and several models, including an allegedly novel architecture: hierarchical attention capsule network, for classification tested on the corpus. we describe the data-set (with simple baselines) and then talk about the modeling with both standard and non-standard methods and tools for explanations. for the dataset, we present an annotated corpus of offensive language in the online world, consisting of sentences and the corresponding annotations. the corpus consists of 18707 sentences annotated with four classes, including neutral language, insulting language, antisocial language, and illegal language. we also present several systems used for the classification of offensive language. the baselines are svm, rnn, cnn, and bert. what's more, we present a novel capsule network (hacn) with hierarchical attention to model the semantic structure. the best f1 score of 94.86% is achieved when using hacn. finally, we propose an explanation tool to illustrate what our systems have learned. identifying offensive language in the online world is also interdisciplinary, as it overlaps with psychology, sociology, and economics, while also raising legal and ethical questions, so we expect it to attract a broader audience. thus, in the future, we would like to bring the ideas and research achievements of other related fields to deliver and share technology and solutions for offensive language from online user-generated content.