On the Effectiveness of Adversarial Robustness for Abuse Mitigation with Counterspeech (2024)

Abstract: Recent work on automated approaches to counterspeech have mostly focused on synthetic data but seldom look into how the public deals with abuse. While these systems identifying and generating counterspeech have the potential for abuse mitigation, it remains unclear how robust a model is against adversarial attacks across multiple domains and how models trained on synthetic data can handle unseen user-generated abusive content in the real world. To tackle these issues, this paper first explores the dynamics of abuse and replies using our novel dataset of 6,955 labelled tweets targeted at footballers for studying public figure abuse. We then curate DynaCounter, a new English dataset of 1,911 pairs of abuse and replies addressing nine minority identity groups, collected in an adversarial human-in-the-loop process over four rounds. Our analysis shows that adversarial attacks do not necessarily result in better generalisation. We further present a study of multi-domain counterspeech generation, comparing Flan-T5 and T5 models. We observe that handling certain abuse targets is particularly challenging.

## introduction online abuse is a significant societal challenge, with public figures often bearing the brunt of its toxic impact. being exposed to such abusive behaviour can have detrimental effects on the mental well-being of victims and even on bystanders who witness it @xcite @xcite . the approach of using counterspeech (or counter-abuse) to directly resist abusive or harmful content has received considerable interest for understanding its effective usage in real-life scenarios (e.g. saltman @xcite @xcite and automating techniques (e.g. @xcite . while counterspeech is promising, the research field faces many challenges. from a theoretical abusive content: @xcite you are useless mate please leave our club, possibly the worst defence i have ever seen at my club. authentic counterspeech 1: hold dat. lol you proved that you don't know anything about football. people defending him as well mate. #bbcfootball authentic counterspeech 2: you obviously see what you want to see. for christs sake he was sensational for us 90% of his time here. . . let him be happy. ## plf dataset: authentic counterspeech collection and annotation to our knowledge, there is no abusive language data that targets public figures and is paired with authentic counterspeech. we start our study by collecting such data, focusing on abuse against premium league footballers (plf) in the uk on twitter/x. 2 the data collection is conducted based on three steps, following the work done by @xcite . firstly, we collect 3,127,640 tweets directed at plf in the period of 08/08/2021 to 01/04/2022, using twitter api v1.1. the tweets are retrieved based on a list of player accounts from england's top football divisions (808 from the men's premier league). secondly, we apply an abuse classifier to automatically identify abusive tweets using footballers personal attacks classifier @xcite , which is suitable because it also targets abuse against footballers. the abusive tweets returned are reviewed by the authors for quality control, resulting in 4,556 abusive tweets in total. the last step is to collect responses to the abusive tweets, i.e. a response as a direct reply to abuse. we collect the first twenty replies to each abusive tweet and discard the replies that are less than 10 words. retweets and replies to replies are not included. this data allows us to characterise how abuse is responded to online. 3 2 for simplicity, we refer to the dataset as plf dataset throughout the paper. ## counterspeech generation the last set of experiments is to assess generation performance across targets and datasets. we adopt the same data used in the adversarial attack on counterspeech detection ( §4.1) for the task of counterspeech generation. we conducted the experiments with t5 @xcite and flan-t5 @xcite . the training inputs are structured as [abuse] response : [counterspeech]. ## discussion based on the results presented in the previous sections, we discuss four high-level key observations. authentic counterspeech differs from synthetic ones in terms of style, language and nature. online users often adopt informal language and conversational styles, reflecting the diverse linguistic expressions prevalent on the internet. their counterspeech tends to be short, emotive, and direct and may incorporate internet slang to engage with a focused target audience. on the contrary, with the aim of catering to a wide audience, experts tend to employ a formal and educational tone, using evidence-based arguments to substantiate their claims. the language in expert counterspeech is usually structured and precise (see appendix a.3). despite these differences, it is unclear what responses the two types of counterspeech elicit and how they compare in fostering constructive dialogue and having a positive impact on the target audience. in future work, it will be imperative to consider the nuances and the dynamics of abuse and counterspeech while developing abuse intervention tools (e.g. user studies). ## conclusion we study the adversarial robustness of counterspeech classifiers by collecting four iterations of datasets, dynacounter, covering multiple domains. our analysis indicates that the characteristics of authentic counterspeech are distinct from synthetic/expert alternatives, which points towards future opportunities for investigating the role of different types of counterspeech for abuse mitigation. our extensive evaluation of counterspeech generation shows that while flan-t5 and t5 can produce decent responses to abuse against various targets, certain pitfalls should be addressed.