H ate BERT : Retraining BERT for Abusive Language Detection in E nglish (2021)

Abstract: We introduce HateBERT, a re-trained BERT model for abusive language detection in English. The model was trained on RAL-E, a large-scale dataset of Reddit comments in English from communities banned for being offensive, abusive, or hateful that we have curated and made available to the public. We present the results of a detailed comparison between a general pre-trained language model and the retrained version on three English datasets for offensive, abusive language and hate speech detection tasks. In all datasets, HateBERT outperforms the corresponding general BERT model. We also discuss a battery of experiments comparing the portability of the fine-tuned models across the datasets, suggesting that portability is affected by compatibility of the annotated phenomena.

## introduction the development of systems for the automatic identification of abusive language phenomena has followed a common trend in nlp: feature-based linear classifiers @xcite @xcite , neural network architectures (e.g., cnn or bi-lstm) @xcite @xcite , and fine-tuning pre-trained language models, e.g., bert, roberta, a.o., @xcite . results vary both across datasets and architectures, with linear classifiers qualifying as very competitive, if not better, when compared to neural networks. on the other hand, systems based on pre-trained language models have reached new state-of-the-art results. one issue with these pretrained models is that the training language variety makes them well suited for general-purpose language understanding tasks, and it highlights their limits with more domain-specific language varieties. to address this, there is a growing interest in generating domain-specific bert-like pretrained language models, such as alberto @xcite or tweeteval @xcite for twitter, biobert for the biomedical domain in english @xcite , finbert for the financial domain in english @xcite , and legal-bert for the legal domain in english @xcite . we introduce hatebert, a pre-trained bert model for abusive language phenomena in social media in english. abusive language phenomena fall along a wide spectrum including, a.o., microaggression, stereotyping, offense, abuse, hate speech, threats, and doxxing @xcite . current approaches have focus on a limited range, namely offensive language, abusive language, and hate speech. the connections among these phenomena have only superficially been accounted for, resulting in a fragmented picture, with a variety of definitions, and (in)compatible annotations @xcite . @xcite introduce a graphical visualisation (figure 1 ) of the connections among abusive language phenomena according to the definitions in previous work @xcite @xcite @xcite . when it comes to offensive language, abusive language, and hate speech, the distinguishing factor is their level of specificity. this makes offensive language the most generic form of abusive language phenomena and hate speech the most specific, with abusive language being somewhere in the middle. such differences are a major issue for the study of portability of models. previous work @xcite @xcite has addressed this task by conflating portability with generalizability, forcing datasets with different phenomena into homogenous annotations by collapsing labels into (binary) macro-categories. in our portability experiments, we show that the behavior of hatebert can be explained by accounting for these difference in specificity across the abusive language phenomena. our key contributions are: (i.) additional evidence that further pre-training is a viable strategy to obtain domain-specific or language varietyoriented models in a fast and cheap way; (ii.) the release of hatebert, a pre-trained bert for abusive language phenomena, intended to boost research in this area; (iii.) the release of a large-scale dataset of social media posts in english from communities banned for being offensive, abusive, or hateful. ## hatebert: re-training bert with abusive online communities further pre-training of transformer based pretrained language models is becoming more and more popular as a competitive, effective, and fast solution to adapt pre-trained language models to new language varieties or domains @xcite @xcite , especially in cases where raw data are scarce to generate a bert-like model from scratch @xcite . this is the case of abusive language phenomena. however, for these phenomena an additional predicament with respect to previous work is that the options for suitable and representative collections of data are very limited. directly scraping messages containing profanities would not be the best option as lots of potentially useful data may be missed. @xcite have used tweets about controversial topics to generate offensive-loaded embeddings, but their approach presents some limits. on the other hand, @xcite have shown the effectiveness of using messages from potentially abusive-oriented on-line communities to generate so-called hate embeddings. more recently, @xcite have shown that biased word embeddings can be beneficial. we follow the idea of exploiting biased embeddings by creating them using messages from banned communities in reddit. ## experiments and results to verify the usefulness of hatebert for detecting abusive language phenomena, we run a set of experiments on three english datasets. offenseval 2019 @xcite the dataset contains 14,100 tweets annotated for offensive language. according to the task definition, a message is labelled as offensive if "it contains any form of non-acceptable language (profanity) or a targeted offense, which can be veiled or direct." @xcite ). the dataset is split into training and test, with 13,240 messages in training and 860 in test. the positive class (i.e. messages labeled as offensive) are 4,400 in training and 240 in test. no development data is provided. abuseval @xcite this dataset has been obtained by adding a layer of abusive lan-guage annotation to offenseval 2019. abusive language is defined as a specific case of offensive language, namely "hurtful language that a speaker uses to insult or offend another individual or a group of individuals based on their personal qualities, appearance, social status, opinions, statements, or actions." @xcite . the main difference with respect to offensive language is the exclusion of isolated profanities or untargeted messages from the positive class. the size of the dataset is the same as offenseval 2019.the differences concern the distribution of the positive class which results in 2,749 in training and 178 in test. hateval @xcite the english portion of the dataset contains 13,000 tweets annotated for hate speech against migrants and women. the authors define hate speech as "any communication that disparages a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nationality, religion, or other characteristics." @xcite . only hateful messages targeting migrants and women belong to the positive class, leaving any other message (including offensive or abusive against other targets) to the negative class. the training set is composed of 10,000 messages and the test contains 3,000. both training and test contain an equal amount of messages with respect to the targets, i.e., 5,000 each in training and 1,500 each in test. this does not hold for the distribution of the positive class, where 4,165 messages are present in the training and 1,252 in the test set. all datasets are imbalanced between positive and negative classes and they target phenomena that vary along the specificity dimension. this allows us to evaluate both the robusteness and the portability of hatebert. we applied the same pre-processing steps and hyperparameters when fine-tuning both the generic bert and hatebert. pre-processing steps and hyperparameters (table a. 3) are more closely detailed in the appendix b. the in-domain results confirm the validity of the re-training approach to generate better models for detection of abusive language phenomena, with hatebert largely outperforming the correthe portability experiments were run using the best model for each of the in-dataset experiments. our results show that hatebert ensures better portability than a generic bert model, especially when going from generic abusive language phenomena (i.e., offensive language) towards more specific ones (i.e., abusive language or hate speech). this behaviour is expected and provides empirical evidence to the differences across the annotated phenomena. we also claim that hatebert consistently obtains better representations of the targeted phenomena. this is evident when looking at the dif- train model offenseval 2019 abuseval hateval p r p r p r offenseval 2019 bert --.510 .685 .479 .771 hatebert --.553 .696 .480 .767 abuseval bert .776 .420 --.545 .571 hatebert .836 .404 --.565 .567 hateval bert .540 .220 .438 .241 --hatebert .473 .183 .365 .191 -- ## conclusion and future directions this contribution introduces hatebert base uncased, foot_4 a pre-trained language model for abusive language phenomena in english. we confirm that further pre-training is an effective and cheap strategy to port pre-trained language models to other language varieties. the in-dataset evaluation shows that hatebert consistently outperforms a generic bert across different abusive language phenomena, such as offensive language (offen-seval 2019), abusive language (abuseval), and hate speech (hateval). the cross-dataset experiments show that hatebert obtains robust representations of each abusive language phenomenon against which it has been fine-tuned. in particular, the cross-dataset experiments have provided (i.) further empirical evidence on the relationship among three abusive language phenomena along the dimension of specificity; (ii.) empirical support to the validity of the annotated data; (iii.) a principled explanation for the different performances of hatebert and bert. a known issue concerning hatebert is its bias toward the subreddit r/fatpeoplehate. to address this and other balancing issues, we retrieved an additional 1.3m messages. this has allowed us to add 712,583 new messages to 12 subreddits listed in table a .1, and identify three additional ones (r/uncensorednews, r/europeannationalism, and r/farright), for a total of 597,609 messages. this new data is currently used to extend hatebert. future work will focus on two directions: (i.) investigating to what extent the embedding representations of hatebert are actually different from a general bert pre-trained model, and (ii.) investigating the connections across the various abusive langauge phenomena. the collection of banned subreddits has been retrieved from a publicly available collection of reddit, obtained through the reddit api and in compliance with reddit's terms of use. from this collection, we generated the ral-e dataset. ral-e will be publicly released (it is accessible also at review phase in the supplementary materials). while its availability may have an important impact in boosting research on abusive language phenomena, especially by making natural interactions in online communities available, we are also aware of the risks of privacy violations for owners of the messages. this is one of the reasons why at this stage, we only make available in ral-e the content of the message without metadata such as the screen name of the author and the community where the message was posted. usernames and subreddit names have not been used to retrain the models. this reduces the risks of privacy leakage from the retrained models. since the training material comes from banned community it is impossible and impracticable to obtain meaningful consent from the users (or redditers). in compliance with the association for internet researchers ethical guidelines foot_7 , we consider that: not making available the username and the specific community are the only reliable ways to protect users' privacy. we have also manually checked (for a small portion of the messages) whether it is possible to retrieve these messages by actively searching copy-paste the text of the message in reddit. in none of the cases were we able to obtain a positive result. there are numerous benefits from using such models to monitor the spread of abusive language phenomena in social media. among them, we mention the following: (i.) reducing exposure to harmful content in social media; (ii.) contributing to the creation of healthier online interactions; and (iii.) promoting positive contagious behaviors and interactions @xcite . unfortunately, work in this area is not free from potentially negative impacts. the most direct is a risk of promoting misrepresentation. hatebert is an intrinsically biased pre-trained language model. the fine-tuned models that can be obtained are not overgenerating the positive classes, but they suffer from the biases in the manually annotated data, especially for the offensive language detection task @xcite . furthermore, we think that such tools must always be used under the supervision of humans. current datasets are completely lacking the actual context of occurrence of a messsage and the associated meaning nuances that may accompany it, labelling the positive classes only on the basis of superficial linguistic cues. the deployment of models based on hatebert "in the wild" without human supervision requires additional research and suitable datasets for training. we see benefits in the use of hatebert in research on abusive language phenomena as well as in the availability of ral-e. researchers are encouraged to be aware of the intrinsic biased nature of hatebert and of its impacts in real-world scenarios.