Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate Speech (2021)

Abstract: Automatic hate speech detection is hampered by the scarcity of labeled datasetd, leading to poor generalization. We employ pretrained language models (LMs) to alleviate this data bottleneck. We utilize the GPT LM for generating large amounts of synthetic hate speech sequences from available labeled examples, and leverage the generated data in fine-tuning large pretrained LMs on hate detection. An empirical study using the models of BERT, RoBERTa and ALBERT, shows that this approach improves generalization significantly and consistently within and across data distributions. In fact, we find that generating relevant labeled hate speech sequences is preferable to using out-of-domain, and sometimes also within-domain, human-labeled examples.

## introduction hate speech refers to the expression of hateful or violent attitudes based on group affiliation such as race, nationality, religion, or sexual orientation. in light of the increasing prevalence of hate speech on social media, there is a pressing need to develop automatic methods that detect hate speech manifestation at scale @xcite . automatic methods of hate speech detection typically take a supervised approach that heavily depends on labeled datasets. however, the difficulty of collecting hate speech samples often leads to biased data sampling techniques, focusing on a specific subset of hateful terms or accounts. consequently, relevant available datasets are limited in size, highly imbalanced, and exhibit topical and lexical biases. several recent works have indicated these shortcomings, and shown that classification models trained on those datasets merely memorize keywords, where this results in poor generalization @xcite . in this work, we seek to improve hate speech generalization using large pretrained language models (lms). we focus our attention on the transformerbased language encoder of bert @xcite and its variants, all of which have been pretrained on massive heterogeneous corpora. in classification, the network parameters of the pretrained models are adapted to a target task using supervised training via a model finetuning procedure @xcite . due to the deep language representations encoded in these large lms, they typically achieve improved performance in low-resource classification settings @xcite . yet, large volumes of high-quality labeled examples must be provided to achieve high model generalization on the target task. in order to improve the performance of pretrained lm classifiers when labeled data is limited, it has been suggested to continue pretraining the models using unlabeled in-domain text, or expose the models to unlabeled task-related data @xcite . as hate speech is scarce and diverse, constructing a large and representative corpus of relevant texts is nontrivial, and attempts to continue pretraining bert using some of the existing datasets have not yielded improvements so far @xcite . in this work, we rather extend the available manually-curated hate speech datasets with large amounts of generated labeled examples. we employ synthetic text sequences generated using the lm of gpt2 @xcite , having it been biased to generate hate (and non-hate) speech using the human-labeled examples @xcite . we then augment the existing gold-labeled datasets with large amounts of synthetic examples, increasing their size from tens to hundreds of thousands of labeled examples. in experiments using the lms of bert, roberta @xcite and albert @xcite , we show substantial and consistent improvements using the synthetic data. remarkably, we observe improved generalization in cross-dataset evaluation, sometimes even surpassing the respective within-dataset results, and show gains in comparison to out-of-domain authentic labeled examples. as of today, it is not common practice to incorporate mass amounts of synthetic data for finetuning lm classification models. our findings therefore have implications for text classification in general, and hate detection in particular. ## related work a recent related work @xcite synthesized new examples from existing training data with the objective of improving multi-class classification. they finetuned gpt2 by prepending the class label to text samples, and used the finetuned model to generate new labeled sentences conditioned on the class label. a bert classifier was then trained on both the existing and the synthesized data. while similar to our approach, they focused on balancing topical multi-class datasets, generating a small number of examples per class from a handful samples. another work generated up to several thousands of examples per class with the goal of dataset balancing @xcite . previous attempts to augment hate speech datasets using synthetic examples similarly focused on remedying the class imbalance within those datasets as means for improving generalization. @xcite proposed several data augmentation techniques, including word swapping and replacement, and class-conditional recurrent neural language generation. they achieved limited performance gains. cao and lee (2020) proposed a gan architecture to guide the generation of hateful texts, and showed average 5% improvement in terms of hate detection f1 using lstm and cnn classifiers. they too focused on dataset balancing, using limited amounts of synthetic data. in this work, we apply sequence generation at large scale, increasing the original dataset size by magnitudes of order. we previously observed that this data augmentation approach improves the performance of a cnn-based hate speech classifier @xcite . here, we apply pretrained lms for extensive data synthesis, and then leverage this data in finetuning pretrained lm text classifiers. performance-wise, classifiers based on pretrained lms achieve favorable results in resource limited settings, and we show that largescale data generation and augmentation further boosts performance, significantly improving the generalization of hate speech detection. ## methods we follow the approach by @xcite , comprised of the following steps. (i) given a dataset d i that consists of hate and non-hate labeled examples {d i h , d i nh }, we generate additional classconditioned synthetic text sequences. we utilize gpt2, a lm that had been pretrained using mass amounts of web text for this purpose. 1 in order to bias the model towards the genre of micro-posts, hate speech, and the topics and terms that characterise each dataset, we continue training gpt2 from its distribution checkpoint, serving it with the labeled text sequences. concretely, we adapt distinct gpt2 models per dataset and class, i.e., for each dataset d i , we obtain two models, g i h and g i nh . (ii) in text synthesis, we provide no prompt to the respective gpt2 model, that is, the token sequences are generated unconditionally, starting from the empty string. similar to the labeled datasets, we generate sequences that are relatively short, up to 30 tokens. (iii) presumably, not all of the text sequences generated by g i h are hateful. we utilize the labeled examples d i for finetuning a bert classifier on hate detection, and apply the resulting classifier to the sequences generated by g i h . we then only maintain those sequences that are perceived as hateful by the model, setting a threshold over the classifier confidence scores. in our experiments, following manual tuning, we set the threshold to 0.7, discarding about two thirds of the generated hate speech sequences. finally, we augment the labeled examples d i with an equal number of hate and non-hate synthetic examples. additional technical details are given in the appendix. in all cases, we follow the standard practice of passing the final layer [cls] embedding to a taskspecific feedforward layer, while finetuning the pretrained models using labeled examples @xcite . in finetuning, we extend d i with varying amounts of generated sequences. ## experiments we wish to assess whether and to what extent the generated synthetic data is sufficiently relevant and diverse for improving the generalization of pretrained lms on the hate detection task. we therefore consider both within-and cross-dataset setups. datasets ## conclusion we evaluated several large transformer-based language models, which yield state-of-the-art hate detection results when finetuned using existing labeled datasets, and boosted their performance by augmenting those datasets with large amounts of generated data. we demonstrated strong positive impact of data augmentation across models and datasets, improving hate detection generalization on unseen examples. while large amounts of authentic task-related data may be available for finetuning in some domains or tasks, this is not the case for hate speech. our main finding is that large lms can be used for synthetic data enrichment, and yield even better results than related humanlabeled datasets. these results hold promise for overcoming sparsity and biases of labeled data. ethical statement hate speech generation is sensitive and must not be maliciously misused.