Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study (2022)

Abstract: In this work, we present an extensive study on the use of pre-trained language models for the task of automatic Counter Narrative (CN) generation to fight online hate speech in English. We first present a comparative study to determine whether there is a particular Language Model (or class of LMs) and a particular decoding mechanism that are the most appropriate to generate CNs. Findings show that autoregressive models combined with stochastic decodings are the most promising. We then investigate how an LM performs in generating a CN with regard to an unseen target of hate. We find out that a key element for successful ‘out of target’ experiments is not an overall similarity with the training data but the presence of a specific subset of training data, i. e. a target that shares some commonalities with the test target that can be defined a-priori. We finally introduce the idea of a pipeline based on the addition of an automatic post-editing step to refine generated CNs.

## introduction hate speech (hs) has found fertile ground in social media platforms. actions undertaken by such platforms to tackle online hatred consist in identifying possible sources of hate and removing them by means of content deletion, account suspension or shadow-banning. however, these actions are often interpreted and denounced as censorship by the affected users and political groups @xcite . for this reason, such restrictions can have the opposite effect of exacerbating the hostility of the haters @xcite . an alternative strategy, that is looming on the horizon, is based on the use of counter narratives. cns are "all communicative actions aimed at refuting hate speech through thoughtful and cogent reasons, and true and fact-bound arguments" @xcite . as a de-escalating * now at the university of stuttgart, germany. measure, cns have been proven to be successful in diminishing hate, while preserving the freedom of speech @xcite . an example @xmath0, @xmath1: hs: women are basically childlike, they remain this way most of their lives. soft and emotional. it has devastated our once great patriarchal civilizations. cn: without softness and emotions there would be just brutality and cruelty. not all women are soft and emotional and many men have these characteristics. to perpetuate these socially constructed gender profiles maintains norms which oppress anybody. based on their effectiveness, cns have started being employed by ngos to counter online hate. since for ngo operators it is impossible to manually write responses to all instances of hate, a line of nlp research has recently emerged, focusing on designing systems to automatically generate cn suggestions @xcite @xcite @xcite . in this study, our main goal is to compare pre-trained language models (lm) and decoding mechanisms in order to understand their pros and cons in generating cns. thus, we use various automatic metrics and manual evaluations with expert judgments to assess several lms, representing the main categories of the model architectures, and decoding methods. we further test the robustness of the fine-tuned lms in generating cns for an unseen target. results show that autoregressive models are in general more suited for the task, and while stochastic decoding mechanisms can generate more novel, diverse, and informative outputs, the deterministic decoding is useful in scenarios where more generic and less novel (yet 'safer') cns are needed. furthermore, in out-of-target experiments we find that the similarity of targets (e.g. jews and muslims as religious groups) plays a crucial role for the effectiveness of portability to new targets. we finally show a promising research direction of leveraging gold human edits for building an additional automatic post-editing step to correct errors made by lms during generation. to the best of our knowledge, this is the first study systematically analysing state of the art pre-trained lms in cn generation. ## related work in this section we first discuss standard approaches to hate countering and studies on cn effectiveness on social media platforms, then the existing cn data collection and generation strategies. hate countering. nlp has started addressing the phenomenon of the proliferation of hs by creating datasets for automatic detection @xcite @xcite @xcite . several surveys provide a review on the existing approaches on the topic @xcite @xcite , also addressing the ethical challenges of the task @xcite . still, automatic detection of hs presents some drawbacks @xcite . first of all, the datasets might include biases, and the models tend to replicate such biases @xcite @xcite . moreover, the end goals for which hs detection is employed are often charged with censorship of the freedom of speech by concerned users @xcite . in this scenario, ngos have started employing cns to counter online hate. cns have been shown to be effective in reducing linguistic violence @xcite @xcite @xcite ; moreover, even if they might not influence the view of extremists, they are still effective in presenting alternative and non-hateful viewpoints to bystanders @xcite . ## methodology in this section, we present the cn dataset, the language models, and the decoding mechanisms employed for our experiments. ## evaluation metrics we use several metrics to evaluate various aspects of the cn generation. overlap metrics. these metrics depend on the n-gram similarity of the generated outputs to a set of reference texts in order to assess the quality. we used our gold cns as references and the cns generated by the different models, as candidates. in particular, we employed three bleu variants: bleu-1 (b-1), bleu-3 (b-3) and bleu-4 (b-4) @xcite , and rouge-l (rou) @xcite . diversity metrics. they are used to measure how diverse and novel the produced cns are. in particular, we utilized repetition rate (rr) to measure the repetitiveness across generated cns, in terms of the average ratios of non-singleton n-grams present in the corpus @xcite . it should be noted that rr is calculated as a corpus-based repetition score , i.e. inter-cn, instead of calculating intra-cn repetition of n-grams only. we also used novelty (nov) @xcite , based on jaccard similarity, to compute the amount of novel content that is present in the generated cns as compared to the training data. human evaluation metrics. albeit more difficult to attain, human judgments provide a more reliable evaluation and a deeper understanding than automatic metrics @xcite . to this end, we specified the following dimensions for the evaluation of cns. suitableness (sui): measures how suitable a cn is to the hs in terms of semantic relatedness and in terms of adherence to cn guidelines foot_0 ; grammaticality (grm): how grammatically correct a generated cn is; specificity (spe): how specific are the arguments brought by the cn in response to the hs; choose-or-not (cho): determines whether the annotators would select that cn to post-edit and use it in a real case scenario as in the set up presented by @xcite ; is-best (best): whether the cn is the absolute best among the ones generated for an hs (i. e. whether the annotators would pick up exactly that cn if they had to use it in a real case scenario). the first three dimensions are rated with a 5points likert scale and follow the evaluation procedure described by @xcite , whereas both choose-or-not and is-best are binary ratings (0, 1). choose-or-not allows for multiple cns to be selected for the same hs, while only one cn can be selected for is-best for each hs. toxicity. foot_1 it determines how "rude, disrespectful, or unreasonable" a text is. toxicity has been employed both to detect the bias present in lms @xcite and as a solution to mitigate such bias @xcite . syntactic metrics. a high syntactic complexity can be used as a proxy for an lm's ability of generating complex arguments. we used the syntactic dependency parser of spacy foot_2 for the task, focusing on the following measures: maximum syntactic depth (msd): the maximum depth among the dependency trees calculated over each sentence composing a cn. average syntactic depth (asd): the average depth of the sentences in each cn. number of sentences (nst): the number of sentences composing a cn. ## experiments we performed two sets of experiments: first, we assessed how lms perform in the task of generating cns with different decoding mechanisms. then, we selected the best model from the first round of experiments and tested its generalization capabilities when confronted with an unseen target of hate. ## conclusion in this work, we focus on automatic cn generation as a downstream task. first, we present a comparative study to determine the performances and peculiarities of several pre-trained lms and decoding mechanisms. we observe that the best results (in term of novelty and specificity) overall are achieved by the autoregressive models with stochastic decoding: gpt-2 with the top k decoding mechanism, and dialogpt with the combination top pk . at the same time deterministic decoding can be used when more generic yet 'safer' cns are preferred. then, we investigate the performances of lms in zero-shot generation for unseen targets of hate. hence, we fine-tuned 5 different versions of gpt-2, leaving out the examples pertaining to one target at each turn. we find out that for each configuration/version, there is a subset of the training data which is more influential with respect to the generated data (i. e. a target that shares some commonalities with the test target that can be defined a-priori). finally, we introduce an experiment by training an automatic post-editing module to further improve the cn generation quality. the notable human evaluation results paves the way for a promising future direction that decouples cn generation from its 'final refinement'.