Don’t Augment, Rewrite? Assessing Abusive Language Detection with Synthetic Data (2024)

Abstract: Research on abusive language detection and content moderation is crucial to combat online harm. However, current limitations set by regulatory bodies and social media platforms can make it difficult to share collected data. We address this challenge by exploring the possibility to replace existing datasets in English for abusive language detection with synthetic data obtained by rewriting original texts with an instruction-based generative model.We show that such data can be effectively used to train a classifier whose performance is in line, and sometimes better, than a classifier trained on original data. Training with synthetic data also seems to improve robustness in a cross-dataset setting. A manual inspection of the generated data confirms that rewriting makes it impossible to retrieve the original texts online.

## introduction abusive language detection 1 online has been a topic of interest to the nlp research community for the last ten years, with a number of datasets retrieved from social media platforms and annotated to be used for training and evaluation @xcite . however, more and more restrictions are now being set that limit resharing of this kind of data even for research purposes (e.g. recent changes in x/twitter terms of use). if data is used for commercial purposes, even more restrictions apply, while national and international organizations are enforcing regulations to protect users' privacy and limit data transfer, like the european general data protection regulation. 2 datasets created to train abusive language detection 1 in this work, we use abusive as an umbrella term to encompass both implicit and explicit attacks, including hate speech and offensive language @xcite ). in the light of these restrictions, it is likely that the availability of data to pursue research on abusive language detection and content moderation will represent an issue in the near future. we therefore address the following question in this work: would it be possible to replace existing datasets for abusive language detection with synthetic ones by maintaining the same classification performance? performing this task with a good accuracy would present several advantages. for example, it would be possible to freely share datasets without the risk of disclosing user information or infringing terms of use and regulations. furthermore, it would mitigate the problem of data degradation, which makes social media datasets unusable after few years from release, since hateful content is frequently deleted @xcite . beside the benefits in terms of privacy, high-quality synthetic data could also reduce the need for human effort in dataset annotation, providing an effective way to automatically create large amounts of training data. we present a first set of experiments in this direction by using a generative language model to rewrite two different abusive language datasets, comparing two prompt types. we then evaluate the quality of the generated data by using it for training abusive language detection classifiers. ## background the use of synthetic data has been proposed as a way to mitigate some of the known issues with hate speech datasets @xcite , such as data scarcity @xcite and negative psychological impact on annotators @xcite . one of the applications of synthetic data is data augmentation (da), in which new data is generated starting from a small set of gold instances, and which is mainly meant to address lowresource settings. although da based on large language models (llms) has been found to potentially lead to performance improvements for many nlp tasks @xcite , it has shown mixed results with regards to abusive language identification @xcite . other work has focused on using fully synthetic data for training models in a few-shot and zeroshot setting @xcite , but models trained on synthetic data were found to achieve worse performance than models trained on gold data, especially for subjective tasks. differently from da and the above work, we propose a rewriting approach, meant to address a scenario where enough gold data has been collected and annotated in the past, but cannot be reshared. ## data in this work, we experiment with the creation of synthetic data starting from two different english datasets revolving around abusive language. first, the multi-domain agreement dataset (mda) by @xcite , which contains tweets annotated for offensive language with different inter-annotator agreement levels, spanning across three topics: the black lives matter movement, covid-19 and the 2020 us elections. we use the set of this dataset containing 2,700 tweets, balanced across agreement levels, labels, and topics. second, we use the measuring hate speech (mhs) corpus @xcite , which contains social media posts from multiple platforms annotated for hate speech. we use ∼10% of this corpus for training (3,013 examples) given its large size, to reflect the size of the mda dataset. we then test each model on both datasets to evaluate cross-dataset generalization. furthermore, we also test our models on the hatecheck test suite @xcite , a set of functional tests for finding specific weaknesses of hate speech detection models. for all datasets, we first carry out a pre-processing step in which we remove all user mentions and urls, replacing them with '@user' and 'url' respectively. further details about the splits we use are reported in appendix a. ## methods previous works experimenting with generative llms to augment abusive language datasets mostly exploit fine-tuning of generative models on existing gold data @xcite , trainable components for task-specific decoding @xcite , or humans in the loop to evaluate generated sequences @xcite . however, with the growing size of generative llms, making them more expensive to fine-tune, and their zero-shot capabilities thanks to instruction tuning, these models can often carry out numerous tasks without requiring any further fine-tuning @xcite . because of this, we use a freely available instruction-tuned model in our experiments, llama-2 chat 7b @xcite , through the huggingface library @xcite . 3 the now widespread use of instruction-tuned generative large language models has also led to numerous efforts towards alignment, ideally in order to minimize inappropriate, offensive or unethical uses @xcite . while this is often preferable for many applications, it can make the creation of synthetic abusive language detection datasets complex, as these models are tuned to avoid producing abusive content due to its potentially harmful uses. because of this, we frame the task not as the generation of new, unseen data, but rather as the rewriting of existing gold sequences, so that i) the synthetic sequences are semantically close to existing data, inspired by the simple changes applied by easy data augmentation @xcite , and ii) the synthetic sequences cannot in principle be traced back to any existing social media posts or their original posters. ## evaluation since our main research question focuses on the possibility to replace abusive language datasets with synthetic data by maintaining the same performance level, we train classifiers on synthetic data derived from mda and mhs and evaluate them on the gold test sets of both, including cross-dataset testing to assess model robustness. additionally, we test our models with the hatecheck test suite. for our classification experiments, we fine-tune a roberta large classifier (355m parameters) @xcite on the original and synthetic data for both gold datasets and for both prompting types. we select this model because it was the best performing one on the mhs dataset, as reported in @xcite , and it outperformed bert on mda @xcite . furthermore, previous work showed that no relevant differences among bert-like models could be observed with respect to the impact of synthetic data on model performance for hate speech detection @xcite . further details are reported in appendix b. ## conclusion in this work, we perform a first exploration of abusive language detection using synthetic data generated through rewriting. we show that this is a promising research direction, since models trained on synthetic data can achieve a classification performance on par with models based on gold data, and even show better robustness in some cross-dataset settings. furthermore, it was not possible to trace back the original data starting from the synthetic ones, even through a manual search online. rewriting original texts seems to be an effective strategy both through paraphrasing and formality style transfer. we believe this approach to be a step forward for the development of datasets and systems for subjective tasks that are more privacy-aware and comply with existing regulations on personal data sharing, anonymization, and right to be forgotten.