Improving Cross-Domain Hate Speech Detection by Reducing the False Positive Rate (2021)

Abstract: Hate speech detection is an actively growing field of research with a variety of recently proposed approaches that allowed to push the state-of-the-art results. One of the challenges of such automated approaches – namely recent deep learning models – is a risk of false positives (i.e., false accusations), which may lead to over-blocking or removal of harmless social media content in applications with little moderator intervention. We evaluate deep learning models both under in-domain and cross-domain hate speech detection conditions, and introduce an SVM approach that allows to significantly improve the state-of-the-art results when combined with the deep learning models through a simple majority-voting ensemble. The improvement is mainly due to a reduction of the false positive rate.

## introduction a commonly used definition of hate speech is a communication that disparages a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nationality, religion, or other characteristics @xcite . the automated detection of hate speech online and related concepts, such as toxicity, cyberbullying, abusive and offensive language, has recently gained popularity within the natural language processing (nlp) community. robust hate speech detection systems may provide valuable information for police, security agencies, and social media platforms to effectively counter such effects in online discussions @xcite . despite the recent advances in the field, mainly due to a large amount of available social media data and recent deep learning techniques, the task remains challenging from an nlp perspective, since on the one hand, hate speech, toxicity, or offensive language are often not explicitly expressed through the use of offensive words, while on the other hand, non-hateful content may contain such terms and the classifier may consider signals for an offensive word stronger than other signals from the context, leading to false positive predictions, and further removal of harmless content online @xcite . labelling non-hateful utterances as hate speech (false positives or type ii errors) is a common error even for human annotators due to personal bias. several studies showed that providing context, detailed annotation guidelines, or the background of the author of a message improves annotation quality by reducing the number of utterances erroneously annotated as hateful @xcite @xcite . we assess the performance of deep learning models that currently provide state-of-the-art results for the hate speech detection task @xcite both under in-domain and crossdomain hate speech detection conditions, and introduce an svm approach with a variety of engineered features (e.g., stylometric, emotion, hate speech lexicon features, described further in the paper) that significantly improves the results when combined with the deep learning models in an ensemble, mainly by reducing the false positive rate. we target the use cases where messages are flagged automatically and can be mistakenly removed, without or with little moderator intervention. while existing optimization strategies (e.g., threshold variation) allow to minimize false positives with a negative effect on overall accuracy, our method reduces the false positive rate without decreasing overall performance. ## methodology hate speech detection is commonly framed as a binary supervised classification task (hate speech vs. non-hate speech) and has been addressed using both deep neural networks and methods based on manual feature engineering @xcite . our work evaluates and exploits the advantages of deep neural networks as means for extracting discriminative features directly from text and of a conventional svm approach taking the advantage of explicit feature engineering based on task and domain knowledge. in more detail, we focus on the approaches described below. ## error analysis we performed a quantitative analysis of the obtained results focusing on the false positive rate: f p @xmath0, the probability that a positive label is assigned to a negative instance; we additionally report positive predictive value: p p @xmath1, the probability a predicted positive is a true positive, for the examined models in the in-domain and cross-domain settings ( we note that the svm approach shows the lowest fpr and the highest ppv in all the considered settings, except when training on the olid dataset and testing on the frenk dataset. combining bert and roberta with svm through the ensemble approach reduces the false positive rate in three out of four settings, when compared to bert and roberta in isolation, and contributes to the overall improvement of the results in all the considered settings. the improvement brought by combining bert and roberta with svm is higher in the majority of cases than combining bert and roberta with either cnn or lstm. measuring the correlation of the predictions of different models using the pearson correlation coefficient revealed that svm produces highly uncorrelated predictions when compared to bert and roberta. an analogous effect for deep learning and shallow approaches was observed in @xcite . the majority of the erroneous false positive predictions produced by the svm approach contain offensive words used in a non-hateful context (avg. 78.8% messages over the four settings), while for bert and roberta this percentage is lower in all the settings (avg. 68.7% and 69.7%, respectively), indicating that bert and roberta tend to classify an instance as belonging to the hate speech class even if it is not explicitly contains offensive terms. our findings suggest that the svm approach improves the results mainly by reducing the false positive rate when combined with bert and roberta. this strategy can be used to address one of the challenges that social media platforms are facing: removal of content that does not violate community guidelines. ## conclusions we showed that one of the challenges in hate speech detection: erroneous false positive decisions, can be addressed by combining deep learning models with a robust feature-engineered svm approach. the results are consistent within the indomain and cross-domain settings. this simple strategy provides a significant boost to the state-ofthe-art hate speech detection results.