Contextualized Graph Representations for Generating Counter-Narratives against Hate Speech (2024)

Abstract: Hate speech (HS) is a widely acknowledged societal problem with potentially grave effects on vulnerable individuals and minority groups. Developing counter-narratives (CNs) that confront biases and stereotypes driving hateful narratives is considered an impactful strategy. Current automatic methods focus on isolated utterances to detect and react to hateful content online, often omitting the conversational context where HS naturally occurs. In this work, we explore strategies for the incorporation of conversational history for CN generation, comparing text and graphical representations with varying degrees of context. Overall, automatic and human evaluations show that 1) contextualized representations are comparable to those of isolated utterances, and 2) models based on graph representations outperform text representations, thus opening new research directions for future work.

## introduction hate speech (hs) is a widespread problem in society with severe repercussions at both personal and societal levels. at the individual level, it can lead to severe psychological and emotional impacts on individuals who are targeted, e.g., fear of becoming the target of physical violence @xcite or increased suicide rates @xcite . at a systemic level, it can create a feedback loop between offline violence and online hs @xcite , e.g., encouraging school violence @xcite . the pervasiveness of hs in the digital age makes its countering a pressing issue @xcite . several approaches exist to counter online hs, including legal sanctions, content regulation, and counter-speech @xcite . the latter consists in responding to hs through counternarratives (cns), which are non-negative responses to hs, targeting extreme statements through factbound arguments or alternative perspectives @xcite . cns aim to promote understanding between individuals and are regarded as an approach that does not pose normative or censoring issues @xcite . furthermore, cns can play a social role in educating those exposed to biased narratives, providing evidence-based responses and exposing misinformation @xcite . cns naturally appear online and are typically authored by community members addressing the phenomenon of hate speech or carefully curated by a mediating party like ngo operators. significant cognitive effort and time investment are needed to create cns manually, which is not feasible at large scale @xcite . this became the primary motivation for exploring automatic methods for cn generation, a task initially proposed by @xcite . as some of these automatic methods rely on data-hungry approaches, research efforts have been directed at providing high-quality datasets containing manually created cns. their provenance ranges from mined instances on social media platforms like twitter @xcite , gab or reddit @xcite , or instances manually created by trained ngo operators @xcite . currently, automatically generated counternarratives tend to be generic and often fail to engage users effectively @xcite . as @xcite observe, current automatic cn generation methods are ineffective in persuading authors against their expressed biases due to a lack of personalization and contextualization. this is partially because authority is insufficient to change the course of the conversation; instead, establishing psychological and epistemic common ground is required @xcite . ## related work in this section, we begin by describing the research on counter-narrative generation, with a particular focus on the role of dialogue context. we then shift our attention to the use of graph representations for textual data. ## research framework the research questions guiding this work are: to answer the stated research questions, we generate cns under several conditions. we first compare the generated cns utilizing only the targeted hs utterance against the ones utilizing the previous dialogue history. then, we compare the generated cns using only text against the ones relying on graph representations. ## experimental setup data we work with the dialoconan dataset @xcite , consisting of 3,059 dialogues between a hs author and a cn author. the average length of these dialogues is 5.43 turns. we divide the data into 80/10/10 for train/dev/test, resulting in 2,447/306/306 instances, respectively. we consider the last cn as the ground truth (avg. @xmath0.8), the last hs as the one to be addressed (avg. @xmath1.1), and the rest as discourse context (avg. @xmath2.4, 3.4 turns). ## results we present automatic evaluation in firstly, regarding graph representations, we observe that models based on graphs outperform those using only text representations, as evidenced by both the automatic metrics and human evaluation. the benefits of encoding contextual information are less pronounced. in the text modality, automatic metrics hint at better performance for the contextualized models (9 out of 13 metrics); yet this is not reflected in the human evaluation. in the graph modalities, the contextualized representations outperform the non-contextualized ones in the majority of automatic (7 out of 13) and human metrics (4 out of 6). furthermore, we observe a slight preference for got over ekg, which is not a graph model specifically tailored to represent dialogue discourse. finally, we note that the non-contextualized got model increases the counterspeech strength and reduces the average toxicity of the generated cns. this improvement is not reflected in the effectiveness score in human annotation. the discrepancy may indicate a mismatch between automatic and human evaluation in contextualized settings. annotators were explicitly asked to consider the previous dialogue history in their judgments, however the automatic metrics only evaluate a given cn independently of its context (see limitations). ## conclusion in this work, we explored the usage of graphs for encoding discourse contextual information for cn generation, providing initial evidence of the benefits of contextualized graph representations for this task. further research could investigate the impact of specific graph representation models (i.e., other syntactic or semantic types of graphs) for capturing context in the task of cn generation. furthermore, future work could involve the intersection with knowledge-grounded cn generation, using knowledge graph repositories like wikidata for supporting factually-enhanced cns, potentially further improving their effectiveness.