Understanding Counterspeech for Online Harm Mitigation (2024)

Abstract: Counterspeech offers direct rebuttals to hateful speech by challenging perpetrators of hate and showing support to targets of abuse. It provides a promising alternative to more contentious measures, such as content moderation and deplatforming, by contributing a greater amount of positive online speech rather than attempting to mitigate harmful content through removal. Advances in the development of large language models mean that the process of producing counterspeech could be made more efficient by automating its generation, which would enable large-scale online campaigns. However, we currently lack a systematic understanding of several important factors relating to the efficacy of counterspeech for hate mitigation, such as which types of counterspeech are most effective, what are the optimal conditions for implementation, and which specific effects of hate it can best ameliorate. This paper aims to fill this gap by systematically reviewing counterspeech research in the social sciences and comparing methodologies and findings with natural language processing (NLP) and computer science efforts in automatic counterspeech generation. By taking this multi-disciplinary view, we identify promising future directions in both fields.

## introduction the exposure of social media users to online hate and abuse continues to be a cause for public concern. volumes of abuse on social media continue to be significant in absolute terms @xcite , and some claim they are rising on platforms such as twitter where, at the same time, content moderation appears to be becoming less of a priority @xcite . receiving abuse can have negative effects on the mental health of targets, and also on others witnessing it @xcite . in the context of public figures, the impact on the witnesses (bystanders) is arguably even more important, as the abuse is potentially witnessed by a large volume of people. in addition, politicians and other prominent actors are driven out of the public sphere precisely because of the vitriol they receive on a daily basis @xcite , raising concerns for the overall health of democracy. within this context, research on mechanisms for combating online abuse is becoming ever more important. one such research angle is the area of "counterspeech" (or counter-narratives): content that is designed to resist or contradict abusive or hateful content figure 1 : counterspeech dynamics. (1) perpetrator(s) generate hate speech. this may be witnessed by either targets and/or bystanders. (2) counterspeaker(s) respond with counterspeech, which may be directed at the perpetrator(s), bystanders (e.g. to provide alternative perspectives), or other targets (e.g. in support). counterspeakers may themselves be targets or bystanders, or could be members of organised counterspeech groups. they can have inor out-group identities with respect to either the perpetrator(s) or the target(s). counterspeech is directed at recipients, who can be one or more of (a) the perpetrator(s), (b) the target(s), or (c) other bystanders. both counterspeakers and targets can be individual or multiple (one-to-one, one-tomany and so on). @xcite @xcite , also see figure 1 . such coun-understanding counterspeech for online harm mitigation terspeech (as we will elaborate more fully below) is an important potential tool in the fight against online hate and abuse as it does not require any interventions from the platform or from law enforcement, and may contribute to mitigating the effects of abuse @xcite @xcite without impinging on free speech. several civil organisations have used counterspeech to directly challenge hate, and facebook has launched campaigns with local communities and policymakers to promote accessibility to counterspeech tools. 1 similarly, moonshot and jigsaw implemented the redirect method, presenting alternative counterspeech or counter videos when users search queries that may suggest an inclination towards extremist content or groups. 2 the detection and generation of counterspeech is important because it underpins the promise of aipowered assistive tools for hate mitigation. identifying counterspeech is vital also for analytical research in the area: for instance, to disentangle the dynamics of perpetrators, victims and bystanders @xcite @xcite , as well as determining which responses are most effective in combating hate speech @xcite @xcite . automatically producing counterspeech is a timely and important task for two reasons. first, composing counterspeech is time-consuming and requires considerable expertise to be effective @xcite . recently, large language models have been able to produce fluent and personalised arguments tailored to user expectations addressing various topics and tasks. thus, developing counterspeech tools is feasible and can provide support to civil organisations, practitioners and stakeholders in hate intervention at scale. second, by partially automating counterspeech writing, such assistive tools can lessen practitioners' psychological strain resulting from prolonged exposure to harmful content @xcite . however, despite the potential for counterspeech, and the growing body of work in this area, the research agenda remains a relatively new one, which also suffers from the fact that it is divided into a number of disciplinary silos. in methodological terms, meanwhile, social scientists studying the dynamics and impacts of counterspeech (e.g. @xcite @xcite often do not engage with computer scientists developing models to detect and generate such speech (e.g. @xcite . this disconnection may increase the time and effort for tackling online harms. the aim of this review article is to fill this gap, by providing a comprehensive, multi-disciplinary overview of the field of counterspeech covering computer science 3 and the social sciences over the last ten years. we make a number of contributions in particular. firstly, we outline a definition of counterspeech and a framework for understanding its use and impact, as well as a detailed taxonomy. visualised in figure 1 , such a framework helps delineate the interaction of hate speech and responses within people involved in the conversations (i.e. perpetrators, targets and bystanders). we review research on the effectiveness of counterspeech, bringing together perspectives on the impact it makes when it is experienced. thus, computer scientists can adeptly approach counterspeech studies and develop effective tools based on our analysis. we also analyse technical work on counterspeech, looking specifically at the task of counterspeech generation, scalability, and the availability and methodology behind different datasets. importantly, across all studies, we focus on commonalities and differences between computer science and the social sciences, including how the impact of counterspeech is evaluated and which specific effect of hate speech it best ameliorates. we draw on our findings to discuss the challenges and directions of open science (and safe ai) for online hate mitigation. for computer scientists, we provide evidence-based recommendations for automatic approaches to counterspeech tools using natural language processing (nlp). similarly, for social scientists, we set out future perspectives on interdisciplinary collaborations with ai researchers on mitigating online harms, including conducting large-scale analyses and evaluating the impact of automated interventions. taken together, our work offers researchers, policymakers and practitioners the tools to further understand the potentials of automated counterspeech for online hate mitigation. ## background interest in investigating the social and computational aspects of counterspeech has grown considerably in the past five years. however, while extant work reviews the impact of counterspeech on hate mitigation @xcite @xcite , none have systematically addressed this issue in combination with computational studies in order to synthesise social scientific insights and discuss the potential role of automated methods in reducing harms. @xcite present a focused (2016-2018) systematic review of research into the impact of counter-narratives on prevention of violent radicalisation. they cate3 while most studies on computational approaches to counterspeech included in this review adopt natural language processing techniques, we use 'computer science' to broadly cover the research field in which the studies are done. on the computational side, some work reviews the use of counterspeech in social media using natural language processing, including work outlining counterspeech datasets @xcite , discussing automated approaches to counterspeech classification @xcite and generation @xcite , and work focusing on system evaluation @xcite . however, nlp work from computer sciences is not typically informed by important insights from the social sciences, including the key roles of intergroup dynamics, the social context in which counterspeech is employed, and the mode of persuasion by which counterspeech operates. taking an interdisciplinary approach, we join work from the computer and social sciences. ## review methodology taking a multi-disciplinary perspective, we systematically review work on counterspeech from computer science and the social sciences published in the past ten years. to ensure broad coverage and to conduct a reproducible review, we follow the systematic methodology of @xcite . the search and inclusion process is shown in figure 2 . we used keyword terms related to counterspeech to search three key databases (acl anthology, arxiv, and scopus) that together offer a broad coverage of our target literature. we included the search terms 'counter-speech', 'counternarratives', 'counter-terrorism', 'counter-aggression', 'counter-hate', 'counter speech', 'counter narrative', 'countering online hate speech', 'counter hate speech', and 'counter-hate speech'. we also included 34 publications that we had identified previously from other sources, but that were not returned by keyword search due to not including relevant keywords or not being indexed in the target search repositories. the search covers the data within the period between 2005 and 2023. of the returned results, we include all publications that concern (1) analysis of the use and effectiveness of interventions against hateful or abusive language online, (2) characteristics of counterspeech users or recipients, or (3) data and/or implementation designed for counterspeech (e.g., counterspeech classification or generation). these inclusion criteria were applied by two of the authors. following this process, we include 100 papers for analysis in this review. each of the papers was read by at least one of the co-authors of the article. our review is divided into several sections (the results of which are presented sequentially below). first, we examine definitional characteristics of counterspeech, looking at how the term itself is defined, how different taxonomies have been created to classify different types of counterspeech, and the different potential purposes attributed to it. based on the definitional characteristics, we examine studies that have looked at the impact of counterspeech, discussing the different analytical designs employed and analysing evidence of the results. following this, we discuss computational approaches to counterspeech, focusing in particular on both detection and generation. finally, we examine ethical issues in the domain of counterspeech, and also speculate about future perspectives and directions in the field. ## defining counterspeech counterspeech is multifaceted and can be characterised in several different ways. in most studies in the field use one of three main terms: counterspeech, counter-narratives @xcite @xcite and hope speech @xcite . these three terms broadly refer to a similar concept: content that challenges and rebuts hateful discourse and propaganda @xcite @xcite @xcite using non-aggressive speech @xcite @xcite . there are some differences between the terms. @xcite considers counter-narratives as intentional strategic communication within a political, policy, or military context. additionally, the term counter-narrative also refers to narratives that challenge a much broader view or category such as forms of education, propaganda, and public information @xcite . such counter-narratives are often discussed in the context of the prevention of violent extremism. hope speech, meanwhile, could be seen as a particular type of counterspeech: it promotes positive engagement in online discourse to lessen the consequences of abuse, and places a particular emphasis on delivering optimism, resilience, and the values of equality, diversity and inclusion @xcite . in this paper, we review work that relates to all of these three concepts, and largely make use of the catch-all term counterspeech, while acknowledging the slight differences between the concepts. ## the impact of counterspeech while we have delineated the characteristics of counterspeech, its concrete effects on harm mitigation remain debated. the methods applied for evaluating the effectiveness of counterspeech vary considerably across studies in the field. in this section we provide an evidence-based analysis of counterspeech's efficacy, examining how it is used in real-life scenarios and its influence based on eight aspects. research design a wide range of methodologies have been adopted to assess the impact of counterspeech on hate mitigation, including observational studies @xcite @xcite , experimental @xcite @xcite and quasiexperimental designs @xcite . in observational studies, investigators typically assess the rela-tionship between exposure to counterspeech and outcome variables of interest without any experimental manipulation. for instance, a longitudinal study of german political conversations on twitter examined the interplay between organized hate and counterspeech groups @xcite . there is also an ethnographic study interviewing counterspeakers on facebook to understand external and internal practices for collectively intervening in hateful comments, such as how to build effective counterspeech action and keep counterspeakers engaged @xcite . for experimental and quasi-experimental designs, both aim at estimating the causal effects of exposure to different kinds of counterspeech on outcome variables in comparison with controls (no exposure to counterspeech). ## computational approaches to counterspeech in this section, we switch the focus to look at nlp literature on counterspeech emerging from the field of computer science. we tackle three subjects in particular: the datasets being used in these studies, approaches to counterspeech detection, and approaches to counterspeech generation. ## future perspectives of the many promising abuse intervention experiments that we review, results are not always consistent, demonstrating weak claims or limited success (applicable only to certain settings). possible reasons include short-term experiments, small sample sizes and nonstandardised experimental designs. to improve this, effective interventions should come with the characteristics of scalability, durability, reliability, and specificity. in this section, we highlight key distinctions and over-laps across areas that have and have not been explored in social sciences and computer science, discuss ethical issues related to evaluating counterspeech in real-life settings and automating the task of counterspeech generation, and identify best practices for future research. distinctions and overlaps across areas by recognizing the commonalities and differences between social sciences and computer science, we pinpoint the unique contributions of each discipline and encourage interdisciplinary collaborations to address complex societal challenges and better understand human behaviour with the help of computational systems. ethical issues, risks and challenges of conducting counterspeech studies effective evaluation of counterspeech not only identifies users who may need help, but also safeguards human rights and reinforces a stronger sense of responsibility in the community. this discussion is based on the authors' opinion and not stemming from the review. responsible open-source research can facilitate reproducibility and transparency of science. recently, reproducible research has been deemed critical in both social sciences @xcite and computer science, and low replication success is found despite using materials provided in the original papers @xcite . to tackle this issue, a few initiatives for transparent research have been proposed, advocat-ing researchers to state succinctly in papers how experiments are conducted (e.g., stimuli, mechanisms for data selection) and evaluated, including a 21 word solution @xcite and open science framework. 5 furthermore, practising data sharing encourages researchers to be responsible for fair and transparent experimental designs, and to avoid subtle selection biases that might affect substantive research questions under investigation @xcite . at the same time, when handling sensitive or personal information, data sharing should adhere to research ethics and privacy standards @xcite . for instance, in the case of hate speech, using synthetic examples or de-identification techniques is considered a good general practice for ensuring the safety of individuals @xcite . furthermore, effective and ethical counterspeech relies on the accuracy and robustness of detecting online hate speech: an innocent speaker may be publicly targeted and shamed if an utterance is falsely classified as hate speech -either directly or indirectly as in endto-end response generation. for example, google's jigsaw api (google jigsaw, 2022), a widely used tool for detecting toxic language, makes predictions that are aligned with racist beliefs and biases-for example it is less likely to rate anti-black language as toxic, but more likely to mark african american english as toxic @xcite . it is thus important to make sure that the underlying tool is not biased and well-calibrated to the likelihood that an utterance was indeed intended as hate speech. for example, the 'tone' of counterspeech could be used to reflect the model's confidence. a related question is free speech: what counts as acceptable online behaviour, what sort of speech is deemed inappropriate, in which contexts, and should be targeted by counterspeech? a promising direction for answering this complex question is participatory design to empower the voices of those who are targeted @xcite . in sum, there is a trade-off between risks and benefits of counterspeech generation. following the 'good samaritan' law: automating counterspeech provides timely help to victims in an emergency which is protected against prosecution (even if it goes wrong). similar legislation is adopted by other countries, including the european union, australia and the uk. under this interpretation, well-intentional counterspeech (by humans and machines) is better than doing nothing at all. 5 https://osf.io/ best practices we provide best practices for developing successful intervention tools. 1. bear in mind practical use cases and scenarios of hate-countering tools. a single intervention strategy is unlikely to diminish online harm and successful counterspeech interventions would benefit from personalisation. to design successful counterspeech tools, it is important to consider the purposes of counter messages (e.g., support victims and debunk stereotypes), the speakers (e.g., practitioners, authorities and high-profile people), recipients (e.g., ingroup/outgroup, political background and education level), the content (e.g., strategy, style, and tones), intensity (e.g., one message per week/month), and the communication medium (e.g., videos, text, and platforms). ## conclusion online hate speech is a pressing global issue, prompting scientists and practitioners to examine potential solutions. counterspeech, content that directly rebuts hateful content, is one promising avenue. while nlp researchers are already beginning to explore opportunities to automate the generation of counterspeech for the mitigation of hate at scale, research from the social sciences points to many nuances that need to be considered regarding the impact of counterspeech before this intervention is deployed. taking an interdisciplinary approach, we have attempted to synthesize the growing body of work in the field. through our analysis of extant work, we suggest that findings regarding the efficacy of counterspeech are highly dependent on several factors, including methodological ones such as study design and outcome measures, and features of counterspeech such as the speaker, target of hate, and strategy employed. while some work finds counterspeech to be effective in lowering further hate generation from the perpetrator and raising feelings of empowerment in bystanders and targets, others find that counterspeech can backfire and encourage more hate. to understand the advantages and disadvantages of counterspeech more deeply, we suggest that empirical research should focus on testing counterspeech interventions in real-world settings which are scalable, durable, reliable, and specific. researchers should agree on key outcome variables of interest in order to understand the optimal social conditions for producing counterspeech at scale by automating its generation. we hope that this review helps make sense of the variety of types of counterspeech that have been studied to date and prompts future collaborations between social and computer scientists working to ameliorate the negative effects of online hate.