Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing (2022)

Abstract: Research to tackle hate speech plaguing online media has made strides in providing solutions, analyzing bias and curating data. A challenging problem is ambiguity between hate speech and offensive language, causing low performance both overall and specifically for the hate speech class. It can be argued that misclassifying actual hate speech content as merely offensive can lead to further harm against targeted groups. In our work, we mitigate this potentially harmful phenomenon by proposing an adversarial debiasing method to separate the two classes. We show that our method works for English, Arabic German and Hindi, plus in a multilingual setting, improving performance over baselines.

## introduction online hate speech has become a pernicious phenomenon of modern society and a lot of effort is being expended in tackling this challenge. while there has been plenty of work to develop automatic methods for hate speech detection @xcite , this has proven to be a difficult challenge to tackle with impractically poor performance. in the nlp community, a prevailing convention is to frame this problem as a three-way classification: between hate speech, offensive language and neither @xcite @xcite @xcite . while this convention allows for the application of more traditional nlp pipelines, performance has been low @xcite especially when it comes to generalization to unseen data @xcite , with even humans struggling to distinguish hate speech @xcite . in our work we also adopt the wide-spread 3class definition of hate speech, where hate speech @xcite . we see the confusion between hate speech and offensive language, with numerous false negatives. we argue that these are very insidious mistakes that could lead to further harm against target groups. with our adversarial debiasing method, we can separate these two classes further and thus minimize this type of error as well as increase overall performance. an observation that can be made from the way classifiers operate is that oftentimes hate speech is misclassified as offensive language and viceversa @xcite . we showcase this in figure 1 . we argue that for hate speech detection models to be trustworthy, we need to work along two axes: increasing overall efficiency while keeping false negatives (i.e., hate speech marked as offensive language) to a mini- with that goal in mind, we propose a novel method to separate the two classes (hate speech and offensive language) via adapting adversarial debiasing methods to the problem at hand. this model stabilizes and improves classifier behavior and performance, increasing metrics across all classes, while at the same time keeping performance for hate speech content stable (or improving upon it). we thus strike a balance between performance overall and specifically for hate speech. we experiment with different architectures for the classifier (bert @xcite , distilbert @xcite ) and the adversary (bert, distilbert and lstms @xcite ). we perform hyperparameter tuning on english data before applying our findings on several languages (german, arabic and hindi) diverse in script, typography and grammar, as well as on a multilingual task setting using mbert. to more objectively frame the benefit of our method, we compare against a battery of baselines, while we also perform error analysis to identify patterns where our method helps. in summary, our contributions foot_0 are: i) employing adversarial debiasing to separate hate speech and offensive language ii) showing that our method works in keeping false negatives to a minimum and increasing f1-scores on multiple english datasets iii) generalizing our findings to other languages, including a multilingual setting. ## related work for hate speech detection, supervised learning approaches are often used. @xcite provide a comprehensive survey on the earlier research of hate speech detection. in more recent work, focus has been placed on various classification methods and curation of datasets @xcite @xcite @xcite . in @xcite , the prevailing definition of the task as a three-way classification was formulated concretely. in their work, despite the high overall accuracy, over 30% of hate speech was misclassified as offensive language, which saliently sheds light on this pervasive challenge in hate speech detection. this finding was corroborated more recently in @xcite , where the state-of-the-art bert model @xcite was applied on a hate speech detection task, with over 60% of hate speech misclassified as offensive language. in the other direction, efforts have also been made to tackle false positives @xcite . further, recent efforts in hate speech detection have increased language coverage from english to multiple languages around the globe, including hindi @xcite , arabic @xcite , levantine @xcite , indonesian (ibrohim and budi, 2019), danish (sigurbergsson and derczynski, 2020) as well as more general multilingual data @xcite @xcite and code-mixing @xcite . a similar methodology to adversarial debiasing was applied to recidivism prediction @xcite . there, racial biases existing in criminal history datasets were mitigated through adversarial training. this method was also applied in hate speech research to minimize bias against aae text @xcite . in this case, adversarial debiasing was employed to counteract the disproportionate labeling of aae text as offensive or hate speech. these works have shown the potential of adversarial debiasing methods in training fairer models. ## data since we wanted to evaluate the 3-class setting (hate speech, offensive language and neither), we either used datasets that already utilized these classes or equivalent ones (for example, in @xcite offensive language is called abusive language). overall, we made use of seven datasets. a summary of each dataset is presented in founta18. founta et al. (2018 ) contains 100k english samples collected from twitter. the definition of hate speech is the usual definition (as described in §1), while the abusive language class is defined as any impolite content using profanity, which is equivalent to the definition of offensive language. thus, we regard it as offensive language for our experiments. hasocen19. @xcite is an english hate speech dataset of 6k samples from twitter and facebook. the samples were labeled into four categories: hate speech, offensive language, profanity, and normal. offensive language is defined as unacceptable language in the absence of insults and abuse. the profanity class expands on this definition to include swear words. we merged the two classes, because both classes meet our definition. ## adversarial debiasing in this section, we detail our adversarial debiasing scheme. in this setup two models are trained in conjunction: the classifier (predictor) and the adversary. the classifier is predicting the actual class of an example, while the adversary learns to predict a protected variable. for the classifier, we compare the performance of three different models. and for the adversary, we investigate three different architectures, loss functions and protected variables foot_1 . in a first step, the models are trained and evaluated with davidson17, founta18 and hasocen19 (the english datasets in our experiments). ## experimental setup for each dataset, we either use the provided training, development and testing set splits, or we sample them at 80:10:10 rates randomly. then, we further downsample (to the number of examples in the smallest class according to each dataset) the training set classes to generate a uniform training set. for hasocen19, hasocde19, hasochin19 and germeval18, the dataset was already split in training and test sets. the original rates are presented in for each experiment, we train for five epochs and keep the best-performing model across the epochs as evaluated on the development set. then, we compute this model's performance on the heldout test set. we repeat this process three times and average the results. ## results results are summarized in for davidson17 and founta18, our method does not provide positive findings. in david-son17 both tph and overall performance are lower, while in founta18, adversarial debiasing does provide a performance boost, but it comes at the expense of tph. for these two datasets, adversary swear was applied as well, improving the tph for davidson17 but not for founta18. results are better for the final english dataset, hasocen19. there, even though tph drops substantially (10%), overall performance increases by more than 0.2 f1 points. without an adversary, accuracy and f1-scores suffer, making for sub-par classifiers biased heavily towards hate speech. instead, with our method, more separation is achieved and the model manages to separate the two contentious classes (hate speech and offensive language) with greater efficiency. whereas before the classifier would not be practical due to low accuracy, with our method f1-scores and accuracy increase to acceptable levels. hasocde19 follows the same pattern, with the vanilla model being unable to provide strong overall results, instead becoming biased towards hate speech and dropping the rest of the classes. with our method, better balance is struck and we see an improvement of 0.15 f1-score over the vanilla model. in germeval18, we see stronger performance gains both for the tph (+5.7% over the nonadversary model) and overall (+0.02 in f1-score). even though the class-weighting baseline does for hasochin19, we observe the same pattern as the other hasoc family of datasets, although to a lesser extent. in l-hsab19, we show that the simple class-weighting baselines is better than both the vanilla and adversary models. finally, in the multilingual setting, we get mixed results. compared to the vanilla model, adversarial debiasing offers a better tph score with minimal drop in performance (while macro f1 increases by 0.02 too). against the baseline model, while the baseline has a higher tph, in all the other metrics performance is worse. in synopsis, apart from l-hsab19, performance is better when using the adversarial debiasing method, either for the tph or the overall f1score metrics. for the multilingual dataset, performance of our method is mixed, improving upon the vanilla model on the tph and upon the baseline model on the other metrics. all in all, our method manages to strike a better balance between tph and overall performance and we thus believe these models are more applicable to a real-world scenario where both axes need to be taken into consideration. ## error analysis we observe that a few samples of hate speech are misclassified as offensive language by the vanilla model without the adversary, but correctly predicted by the adversarial model. in in english, we see that hateful speech was marked as merely offensive by the vanilla model, potentially because no slur was used, but only some offensive language ('c*nt', 'ass' and 'bad ass'). the model failed to take into account the context in which these words were used, or failed to pick up innocuous words used here as slurs (for example, 'orangutan'). the adversarial model was able to make correct predictions, potentially because it is not putting as much weight on individual words, but the combinations between them. in german, the vanilla model's shortcomings are again centered around a lack of slurs. in both examples, there are no direct slurs so the model interprets it as offensive because of the overall negative sentiment (created through phrases such as 'böse männer', meaning 'evil men' and 'sexuelle gewalt', meaning 'sexual violence'). in one of the examples, the model misses that 'froschfresser' (meaning 'frog eaters') is used as a slur. the adversarial model again shows an ability to expand from keyword-based predictions to a better understanding of context. in hindi, while the example is merely offensive, the vanilla model has marked it as hateful, potentially because of the politically heavy 'terrorist' term. the adversarial model has not put as much weight on the word and thus made a correct prediction. in arabic, the vanilla model again misses that innocuous words are used as slurs (eg., 'dogs'), marking the text as offensive instead of hateful. ## conclusion in hate speech detection efforts, it can be observed that a lot of classifiers struggle with the hate speech and offensive language classes. a lot of models trained on current datasets misclassify hate speech as offensive language. we argue that this type of error is particularly insidious, since it can lead to targeted groups getting exposed to harmful content more often. further, a lot of hate speech classifiers are impractical, either having a low true positive rate for hate speech or low performance overall. we propose a method to both increase the true positive rate for hate speech and to stabilize the classifiers in general. we base our method on the adversarial debiasing setup, where in our instance we are trying to support the classifier in separating the hate speech and offensive language classes. we evaluate on seven hate speech datasets spanning four languages, plus a multilingual set we create by combining all data. our method is at best performing just as well for all datasets ex