Toxic Language Detection in Social Media for B razilian P ortuguese: New Dataset and Multilingual Analysis (2020)

Abstract: Hate speech and toxic comments are a common concern of social media platform users. Although these comments are, fortunately, the minority in these platforms, they are still capable of causing harm. Therefore, identifying these comments is an important task for studying and preventing the proliferation of toxicity in social media. Previous work in automatically detecting toxic comments focus mainly in English, with very few work in languages like Brazilian Portuguese. In this paper, we propose a new large-scale dataset for Brazilian Portuguese with tweets annotated as either toxic or non-toxic or in different types of toxicity. We present our dataset collection and annotation process, where we aimed to select candidates covering multiple demographic groups. State-of-the-art BERT models were able to achieve 76% macro-F1 score using monolingual data in the binary case. We also show that large-scale monolingual data is still needed to create more accurate models, despite recent advances in multilingual approaches. An error analysis and experiments with multi-label classification show the difficulty of classifying certain types of toxic comments that appear less frequently in our data and highlights the need to develop models that are aware of different categories of toxicity.

## introduction social media can be a powerful tool that enables virtual human interactions, connecting people and enhancing businesses' presence. on the other hand, since users feel somehow protected under their virtual identities, social media has also become a platform for hate speech and use of toxic language. although hate speech is a crime in most countries, identifying cases in social media is not an easy task, given the massive amounts of data posted every day. therefore, automatic approaches for detecting online hate speech have received significant attention in recent years @xcite @xcite . in this paper, we focus on the analysis and automatic detection of toxic comments. our definition of toxic is similar to the one used by the jigsaw competition, 1 where comments containing insults and obscene language are also considered, besides hate speech. 2 systems capable of automatically identifying toxic comments are useful for platform's moderators and to select content for specific users (e.g. children). nevertheless, there are multiple challenges specific to process toxic comments automatically, e.g. (i) toxic language may not be explicit, i.e. may not contain explicit toxic terms; (ii) there is a large spectrum of types of toxicity (e.g. sexism, racism, insult); (iii) toxic comments correspond to a minority of comments, which is fortunate, but means that automatic data-driven approaches need to deal with highly unbalanced data. although there is some work on this topic for other languages -e.g. arabic @xcite and german @xcite -, most of the resources and studies available are for english @xcite @xcite @xcite ). 3 for portuguese, only two previous works are available @xcite and their datasets are considerably small, mainly when compared to resources available for english. we present told-br (toxic language dataset for brazilian portuguese), a new dataset with twitter posts in the brazilian portuguese language. 4 915 a total of 21k tweets were manually annotated into seven categories: non-toxic, lgbtq+phobia, obscene, insult, racism, misogyny and xenophobia. each tweet has three annotations that were made by volunteers from a university in brazil. volunteers were selected taking into account demographic information, aiming to create a dataset as balanced as possible in regarding to demographic group biases. this is then the largest dataset available for toxic data analysis in social media for the portuguese language and the first dataset with demographic information about annotators. 5 we experiment with brazilian portuguese @xcite and multilingual @xcite bert models @xcite for the binary task of automatically classifying toxic comments, since similar models achieve state-of-the-art results for the same task in other languages @xcite . models fine-tuned on monolingual data achieve up to 76% of macro-f 1, improving 3 points over a baseline. besides, bert-based approaches with multilingual pre-trained models enable transfer learning and zero-shot learning. the offenseval 2019 olid dataset @xcite is then used to experiment with (i) transfer-learning: where both olid and told-br are used to fine-tune bert; and, (ii) zero-shot learning: where bert is fine-tuned using only olid. results highlight the importance of language-specific datasets, since transfer learning does not improve over monolingual models and zero-shot learning achieves only a macro-f 1 of 56%. an error analysis is performed using our best model, where the worst-case scenario, i.e., classifying toxic comments as non-toxic, is further investigated, taking into account the fine-grained categories. results show that categories with fewer examples in the dataset (racism and xenophobia) are more likely to be mislabelled than other classes, with the best performance being achieved by majority classes (insult and obscene). we also analyse the amount of data needed in order to achieve the best performance in binary classification. models trained with few examples are only accurate in predicting the majority class (non-toxic). as the number of instances grow, the performance on the minority class (toxic) improves significantly. section 2 presents an overview of relevant previous work. section 3 shows details about the told-br dataset. material and methods are presented in section 4, whilst results are discussed in section 5. finally, section 6 shows a final discussion and future work. ## related work although multiple researchers have addressed the topic of hate speech (e.g. @xcite hovy (2016), chung et al. (2019), basile et al. (2019) ), we focus the literature review on previous work related to toxic comments detection, the topic of our paper. due to space constraints, we only describe papers that create and use twitter-based datasets and/or focus on the brazilian portuguese language. english @xcite present a dataset with around 25k tweets annotated by crowdworkers as containing hate, offensive language, or neither. they build a feature-based classifier with tf-idf transformation over n-grams, part-ofspeech information, sentiment analysis, network information (e.g., number of replies), among other features. their best model, trained using logistic regression, achieves a macro-f 1 of 90. founta et al. (2018) also rely on crowd-workers to annotate 80k tweets into eight categories: offensive, abusive, hateful speech, aggressive, cyberbullying, spam, and normal. they perform an exploratory approach to identify the categories that cause most confusion to crowd-workers. their final, large-scale annotation is done using four categories: abusive, hateful, normal, or spam. offenseval is a series of shared tasks focusing on offensive comments detection @xcite . the olid dataset (used in the 2019 edition) has around 14k tweets in english manually annotated as offensive or nonoffensive. the best model for the relevant task a (offensive versus non-offensive) uses a bert-based classifier and achieves 82.9 of macro-f 1. german a shared task (organized as part of ger-meval 2018) aimed to classify tweets in german categorized into offensive or non-offensive @xcite . they make available a manually annotated dataset with approximately 8.5k tweets. the best system achieved 76.77 of f 1-score and was a feature-based ensemble approach. arabic @xcite present a dataset with 1.1k manually annotated tweets into obscene, offensive, or clean. they experiment with lexicalbased approaches that achieve a maximum of 60 f 1-score. @xcite create a dataset with tweets in the levantine dialect of arabic manually annotated into normal, abusive, or hate (with approximately 5k tweets). the authors use featurebased approaches to induce models for ternary and binary scenarios, with best systems achieving 74.4 and 89.6 of f 1-score, respectively. spanish carmona et al. (2018) present a shared task aiming to detect aggressive tweets in mexican spanish. they manually annotate 11k tweets into aggressive or non-aggressive. the best system is a feature-based approach with macro-f 1 of 62. hindi @xcite present a dataset of around 3.6k tweets in hinglish (spoken hindi written using the roman script). the dataset was annotated into three classes not offensive, abusive and hate-inducing by ten nlp researchers. a convolutional neural network (cnn) architecture with transfer learning is used, where the model is trained with both hinglish and english data (from @xcite ), achieving 71.4% of f 1-score. portuguese de pelle and moreira (2017) make available a dataset with 1, 250 comments, extracted from comment sessions of g1.globo.com website, and annotated them into categories of offensive or non-offensive. the offensive class was also subdivided into racism, sexism, lgbtq+phobia, xenophobia, religious in-tolerance, or cursing. they experiment with binary classification, using n-grams as features to svm and naivebayes models. best results are achieved with svm reaching a weighted f1 score between 77 and 82, depending on different label interpretations. @xcite describe a dataset with 5, 668 tweets classified as hate vs. non-hate, with the hate class further classified following a fine-grained hierarchy. experiments with binary classification show a f 1 score of 78 using an lstm-based architecture. multilingual hasoc was a shared task aiming to classify hate speech and offensive comments in english, german, and hindi @xcite . their dataset contains around 7k tweets and facebook posts manually annotated. sub-task a sep-arates posts into hate speech or offensive versus neither; and, sub-task b separates posts containing hate speech or offence into three categories: hate speech, offensive or profane. best performing systems in all languages used deep learning approaches. for offenseval 2020 @xcite , a more extensive training data is available for english (over 9m tweets), although the annotation was made semi-automatically. arabic, danish, greek, and turkish datasets are also available with manually annotated labels. for all languages, best models are achieved using some variation of bert. our work is different from previous approaches because we (i) release a large-scale dataset for a language other than english, that was created with the aim to reduce demographic biases; (ii) experiment with multilingual approaches, including transfer learning and zero-shot-learning; (iii) perform an analysis of the amount of data needed to train reliable models; and, (iv) experiment with multilabel classification, providing first insights into this challenge task. ## dataset in this section, we describe the procedure adopted to create told-br and present its main features. ## materials and methods in this section, we describe the techniques, tools, and other materials used in our experimental evaluation. as mentioned before, we restrict our experiments on the dataset labelled as positive when at least one annotator considers the example as toxic. we then investigate the effects of the number of instances in the training data, different algorithms to train a classification model, various scenarios considering single-and multilingual models, and perform an initial experiment with multi-label classification. we use bag-of-words (bow) to represent the examples and an automl model to build the baseline model (bow+automl). for this, we use the auto-sklearn foot_1 library @xcite . for our bert-based models, we use the simpletransformers foot_2 library, that allows easy training and evaluation. we use default arguments for parameter tuning and define a seed to allow for reproducibility. two versions of pretrained bert language models are applied: brazilian portuguese bert 10 (souza et al., 2019), and multilingual bert foot_4 @xcite . told-br is used to fine-tune bert-based models for our monolingual experiments, with monolingual bert (br-bert) and multilingual bert (m-bert-br). although m-bert-br refers to the multilingual version of bert, we refer to these two models as "monolingual models," as we trained using the dataset with brazilian portuguese sentences alone. using the multilingual model, we also carry out experiments in which we add data in english to train the models either through transfer learning or zero-shot learning. for these experiments we use the olid data, concatenating the training and test splits into a single dataset. for transfer learning, we merged olid and told-br to obtain a model with both languages as input, aiming to assess whether extra data in english helps in building better models (m-bert(transfer)). for zero-shot learning, olid is used alone at training time, building a model that did not have access to any data in brazilian portuguese (m-bert(zero-shot)). through these experiments, we can assess the advantages of monolingual models, whether data from another language can directly benefit the classification, and whether a specific monolingual dataset is necessary or not. we experiment with different sizes of the training set to assess the influence of the volume of data on the classification. for that, we evaluate the results on random subsets of the data. the size of each partition varies in a range between 10% and 100% adding 10% of the data at each iteration. for each step, we repeat the classification three times to minimise the probability of reporting results obtained by chance. our best model (m-bert-br) is used for this experiment (c.f. section 5). evaluation for binary classification is done in terms of precision, recall and, f 1-score per class and macro-f 1. we also analyse the confusion matrices of our systems in order to better visualise the performance of our models in each class, mainly focusing on an analysis of false negatives. although we mainly focus on binary classification, an initial approach for multi-label classification is also presented. we use the adaptation for the multi-label classification scenario available in simpletransformers. in this case, the transformer's output consists of six neurons, each representing one of the labels. these neurons are considered independent in the training and prediction process. thus, when an output neuron is activated, we set the label represented by this neuron to positive. besides, we evaluate the performance of a baseline based on bow+automl, where we train an automl model for multilabel classification. evaluation is done in terms of hamming loss and average precision @xcite . ## results and discussion this section shows the results of our experiments in classifying toxic comments using told-br. ## concluding remarks in this paper, we present told-br: a dataset for the classification of toxic comments on twitter in brazilian portuguese. through a wide and comprehensive analysis, we demonstrated the need for this dataset for studies on automatic classification of toxic comments. we highlight that monolingual approaches for this task still outperform multilingual experiments and that large-scale datasets are needed for building reliable models. also, we show that there are still challenges to be overcome, such as the naturally significant class imbalance when dealing with multi-label classification. as future work, in addition to deal with class imbalance, we intend to evaluate if aggregating classes with high divergences between annotators can build more reliable models. besides, we intend to assess the benefits of adding unlabelled data to told-br to use semi-supervised techniques.