Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD , a G erman Hate Speech Dataset (2024)

Abstract: Hate speech detection models are only as good as the data they are trained on. Datasets sourced from social media suffer from systematic gaps and biases, leading to unreliable models with simplistic decision boundaries. Adversarial datasets, collected by exploiting model weaknesses, promise to fix this problem. However, adversarial data collection can be slow and costly, and individual annotators have limited creativity. In this paper, we introduce GAHD, a new German Adversarial Hate speech Dataset comprising ca. 11k examples. During data collection, we explore new strategies for supporting annotators, to create more diverse adversarial examples more efficiently and provide a manual analysis of annotator disagreements for each strategy. Our experiments show that the resulting dataset is challenging even for state-of-the-art hate speech detection models, and that training on GAHD clearly improves model robustness. Further, we find that mixing multiple support strategies is most advantageous. We make GAHD publicly available at https://github.com/jagol/gahd.

## introduction robust hate speech detection is essential for addressing and analyzing online hate on a large scale. hate speech detection models are typically trained on datasets sourced from social media or newspaper comment sections @xcite . however, such datasets are known to have systematic gaps and biases, which leads to models that suffer from lexical overfitting and poor generalisability @xcite @xcite . dynamic adversarial data collection (dadc), seeks to address this issue, by tasking annotators to create texts that trick a model, the target model, into incorrect classifications @xcite . @xcite to improve a german hate speech classifier. we start with a target model trained on existing datasets. then, in each round (r1-r4), annotators try to trick the target model using a different method. after each round, we train a new target model including the new adversarial examples. the newly-created data is added to the training data, and the target model is then retrained on all data, making it more robust. this process is repeated across multiple rounds. @xcite use dadc to create an english hate speech dataset, and show that training on their data substantially improves model robustness. however, dadc is time-consuming, expensive, and can result in a homogenous dataset, unless annotators explore diverse strategies for tricking the target model. in this paper, we introduce gahd, a new german adversarial hate speech dataset, collected with four rounds of dadc. to address the limitations of prior dadc work, we use a new strategy in each round to support annotators in finding diverse adversarial examples, in a time-efficient manner. figure 1 shows our improved dadc process: in r1, the first round, we let annotators come up freely with their own adversarial examples. for r2, we provide the annotators with english-to-german translated adversarial examples as candidates to validate or reject, and as a way to inspire new, derived examples. in r3, annotators validate sentences from german newspapers that the target model labeled as hate speech. due to their origin, it is unlikely that these sentences are hate speech, which makes them likely adversarial examples. for r4, we task annotators with creating contrastive examples by modifying previously collected examples in a way that flips their label. gahd contains 10,996 adversarial examples, with 42.4% labeled as hate speech. 1,300 entries are paired with a contrastive example. evaluating the target model after each round demonstrates large improvements in model robustness, with almost 20 percentage point increases in macro f 1 on the gahd test split (in-domain), and german hate-check test suite (out-of-domain) @xcite . we further evaluate the contribution of individual rounds, while controlling for data size, observing that rounds with manually-crafted examples are more effective, but that mixing multiple rounds with different data collection strategies leads to more consistent improvements. finally, we benchmark a range of commercial apis and large language models (llms) on gahd, finding that the apis generally struggle, with only gpt-4 achieving over 80% macro f 1 . in summary, our contributions are: ## related work dynamic adversarial data collection there is a growing body of work demonstrating that dadc improves the robustness and generalisability of nlp models on a wide range of tasks @xcite @xcite @xcite @xcite . dadc further leads to datasets that are more syntactically and lexically diverse than non-adversarial data @xcite . a branch of research building on this paradigm, exploring how dadc can be made more efficient, has shown that data augmentation for adversarial data improves model generalisation @xcite and that supporting annotators by generating suggestions can improve the annotator efficiency and model tricking rate @xcite . two previous papers applied dadc to hate speech. the first created an english hate speech dataset over four rounds of dadc @xcite . in contrast to our work, the authors relied on manually crafted examples and rule-based perturbations. the second paper uses dadc to create an english test suite for emoji-based hate speech @xcite . hate speech datasets hate speech detection datasets are typically sourced from social media, and are annotated on a post-level for binary or ternary classification @xcite @xcite . sometimes more fine-grained annotations schemes are employed @xcite @xcite . adversarial datasets for hate speech can be categorized into collected web-sourced datasets @xcite , manually created datasets @xcite , and generated datasets @xcite @xcite . a range of adversarial attacks and perturbations on hate speech detection models have been proposed and analyzed @xcite @xcite @xcite , leading to research on how to defend against such attacks @xcite . finally, the goal of preventing models from relying on spurious correlations has motivated contrastive data augmentation @xcite and automatic counterfactual data augmentation for sexism and hate speech detection @xcite . ## conclusion in this paper, we presented gahd, a german adversarial hate speech dataset produced via dynamic adversarial data collection (dadc). across four rounds of data collection, we explored new strategies for supporting the annotators in efficiently creating diverse examples by suggesting candidates for validation or inspiration. in total, gahd comprises 10,996 examples (42.4% hate speech), including 1,300 contrastive examples. our experiments showed that (1) training on gahd clearly improves the robustness of hate speech detection models, demonstrated by increases of 18-20 percentage points on gahd and hatecheck, (2) supporting annotators with a variety of methods not only increases their efficiency but also leads to more effective examples, and (3) gahd is challenging, even for state-of-the-art llms and content moderation apis. our results highlight the benefits of supporting annotators in finding adversarial examples. future work could explore more annotator support strategies for dadc. specifically, llm-based augmentations @xcite , such as perturbations and counterfactuals @xcite @xcite present a promising avenue.