Generation-Based Data Augmentation for Offensive Language Detection: Is It Worth It? (2023)

Abstract: Generation-based data augmentation (DA) has been presented in several works as a way to improve offensive language detection. However, the effectiveness of generative DA has been shown only in limited scenarios, and the potential injection of biases when using generated data to classify offensive language has not been investigated. Our aim is that of analyzing the feasibility of generative data augmentation more in-depth with two main focuses. First, we investigate the robustness of models trained on generated data in a variety of data augmentation setups, both novel and already presented in previous work, and compare their performance on four widely-used English offensive language datasets that present inherent differences in terms of content and complexity. In addition to this, we analyze models using the HateCheck suite, a series of functional tests created to challenge hate speech detection systems. Second, we investigate potential lexical bias issues through a qualitative analysis on the generated data. We find that the potential positive impact of generative data augmentation on model performance is unreliable, and generative DA can also have unpredictable effects on lexical bias.

## introduction even though large language models have been found to have a tendency to encode and propagate undesirable social bias @xcite , going as far as generating toxic sequences starting from non-toxic prompts @xcite , the use of synthetic data for offensive language detection has been found to be potentially helpful in improving models (e.g. @xcite ; @xcite ; d' @xcite ). indeed, data augmentation (da) through generation has the potential to mitigate some of the known issues of smaller datasets, which are common in offensive language detection, such as lack of linguistic variation and risk of overfitting @xcite . furthermore, synthetic data can overcome privacy issues related to the use of social media data obtained without user consent in research experiments. it can also mitigate dataset decay, an issue affecting reproducibility, since online messages, especially abusive ones, tend to be deleted over time, while synthetic examples do not present this issue @xcite . while generative da has been shown to be potentially useful for the task of detecting offensive and abusive language online in multiple works, several aspects and implications of it remain unexplored. first of all, generative da has mostly been shown to work for offensive language detection when starting with a single specific dataset and using a specific generation setup, with no investigation of the impact of different generation setups on the quality of the augmented data, as well as little exploration of cross-dataset or cross-domain performance. the first aim of our work is therefore that of assessing the robustness of models across different sources of variation as follows: i) we train and test our models using four english offensive language datasets, testing both within dataset and cross-dataset performance; ii) we simulate two lowresource scenarios, in which we start with different quantities of gold examples; iii) we compare four different generation setups, of which two were used in previous work and two are novel; iv) we experiment with different thresholds for filtering the generated data prior to using it for training. our second aim for this work is that of conducting a qualitative analysis on the generated data, with a focus on lexical bias. in order to do this we compute the correlation between tokens in offensive texts using pmi 1 , and we test the models trained on augmented data on the hatecheck suite @xcite , which includes a series of functional tests aimed at finding model weaknesses. ## related work model-based data augmentation exploiting large language models (llms) such as gpt-2 @xcite has been found effective for various nlp tasks. one method that has been shown to be promising is fine-tuning gpt-2 on annotated data, and then using it to generate additional similar data. the most common approach is prepending labels to sequences during fine-tuning, and then using labels as prompts for the model to generate sequences belonging to specific classes @xcite @xcite . similar methods have also been successfully applied to abusive language classification. for instance, @xcite find that da using a fine-tuned gpt-2 model leads to performance improvements in very low-resource scenarios. @xcite use a conditional variant of gpt-2 based on reinforcement learning, where lexical features for each class are extracted from the entire dataset and then used for generation. @xcite use gpt-2 to generate synthetic hate speech data. they find that the addition of large amounts of synthetic data helps classification when starting from datasets containing thousands of labeled instances. while d' @xcite follow the label-prepending approach of @xcite , @xcite train a separate generative model on data belonging to each class. both approaches are found effective, but they have never been comparatively evaluated. to our knowledge, the robustness of models trained using generation-based da has not been analyzed in depth. while @xcite test their models cross-dataset, results are presented in a setup in which models are trained on 4 datasets together and tested on a fifth one. this setup presupposes that multiple datasets can be used at once for training models. however, this might not always be the case when da is needed, so we evaluate crossdataset setups in which only one dataset is available for training. in addition, to our knowledge ours is the first work to pair a robustness analysis with a qualitative analysis of lexical bias in the context of generative da for this task. ## methods we aim at comparing the performance of different data augmentation setups, both novel and already employed in previous work. we test them in withindataset and cross-dataset scenarios, to assess the impact of synthetic data on model robustness across setups. below is an overview of the process we follow, whose specifics are detailed in section 5. 1. we randomly undersample the training data, obtaining the data subset x consisting of n examples (sec. 5.1). ## experimental setting we structure our experiments along three axes of variation, with the aim of assessing their impact on model performance. the explored dimensions are further detailed in the following subsections. we experiment with two probability thresholds, in order to assess whether the confidence of the classifier is associated with generated data quality. each model is tested on its own test data (withindataset) and on the test data for the other datasets (cross-dataset). ## results in this section we report the results of our experiments. each experiment is run 10 times, with different random seeds. the metric we use to evaluate models is macro-f1 score. in order to reliably compare the distributions of results across runs, we use almost stochastic order (aso) @xcite in its implementation by @xcite . following their findings, we use τ = 0.2 as a threshold for statistical significance. 8 ## qualitative analysis in order to estimate the quality of the generated examples and the impact of the prompting method, we randomly select a subset of 10 generated examples for every dataset / data size combination for manual analysis. we find that there are some clear differences between the prompting setups, and that the methods that exploit prompting in natural language (nl-prompt and cloze-prompt) tend to generate the most realistic examples. tag-prompt tends to often generate strings of random special characters, resulting in very low quality data, while the 1/label setup often results in sequences that appear out of domain. some examples of the generated texts can be found in appendix b. ## conclusions in this work, we presented an evaluation of both existing and novel data augmentation setups based on generative large language models for offensive language detection. we investigated the robustness of such models, testing them in within-dataset and cross-dataset scenarios, and performed a qualitative analysis on the augmented data. we found that while generative da can positively impact model performance in some cases, especially when low amounts of gold data are available, this positive effect is not consistent across setups, making generative da unreliable in the setups we tested. in addition to this, we found that generative da can potentially introduce lexical bias from the pre-trained generative model into the augmented data, as well as increase the reliance of models on identity terms and slurs, which could have unintended effects on classification. overall, although it might improve classification performance in some cases, we advise against using generative da for this task, since it is computationally intensive and it does not appear to consistently make models perform better or be more robust.