Generating Counter Narratives against Online Hate Speech: Data and Strategies (2020)

Abstract: Recently research has started focusing on avoiding undesired effects that come with content moderation, such as censorship and overblocking, when dealing with hatred online. The core idea is to directly intervene in the discussion with textual responses that are meant to counter the hate content and prevent it from further spreading. Accordingly, automation strategies, such as natural language generation, are beginning to be investigated. Still, they suffer from the lack of sufficient amount of quality data and tend to produce generic/repetitive responses. Being aware of the aforementioned limitations, we present a study on how to collect responses to hate effectively, employing large scale unsupervised language models such as GPT-2 for the generation of silver data, and the best annotation strategies/neural architectures that can be used for data filtering before expert validation/post-editing.

## introduction owing to the upsurge in the use of social media platforms over the past decade, hate speech (hs) has become a pervasive issue by spreading quickly and widely. meanwhile, it is difficult to track and control its diffusion, since nuances in cultures and languages make it difficult to provide a clear-cut distinction between hate and dangerous speeches @xcite . the standard approaches to prevent online hate spreading include the suspension of user accounts or deletion of hate comments from the social media platforms (smps), paving the way for the accusation of censorship and overblocking. alternatively, to weigh the right to freedom of speech, shadow-banning has been put into use where the content/account is not deleted but hidden from smp search results. still, we believe that we must overstep reactive identifyand-delete strategies to responsively intervene in the conversations @xcite . in this line of action, some non-govermental organizations (ngos) train operators to intervene in online hateful conversations by writing counter-narratives. a counter-narrative (cn) is a non-aggressive response that offers feedback through fact-bound arguments and is considered as the most effective approach to withstand hate messages @xcite . to be effective, a cn should follow guidelines similar to those provided in the 'get the trolls out' project foot_0 , in order to avoid escalating the hatred in the discussion. still, manual intervention against hate speech is not scalable. therefore, data-driven nlg approaches are beginning to be investigated to assist ngo operators in writing cns. as a necessary first step, diverse cn collection strategies have been proposed, each of which has its advantages and shortcomings @xcite @xcite . in this study, we aim to investigate methods to obtain high quality cns while reducing efforts from experts. we first compare data collection strategies depending on the two main requirements that cn datasets must meet: (i) data quantity and (ii) data quality. finding the right trade-off between the two is in fact a key element for an effective automatic cn generation. to our understanding none of the collection strategies presented so far is able to fulfill this requirement. thus, we test several hybrid strategies to collect data, by mixing niche-sourcing, crowd-sourcing, and synthetic data generation obtained by fine-tuning deep neural architectures specifically developed for nlg tasks, such as gpt-2 @xcite . we propose using an author-reviewer framework in which an author is tasked with text generation and a reviewer can be a human or a classifier model that filters the produced output. finally, a validation/post-editing phase is conducted with ngo operators over the filtered data. our findings show that this framework is scalable allowing to obtain datasets that are suitable in terms of diversity, novelty, and quantity. ## related work we briefly focus on three research aspects related to hate online, i.e. available datasets, methodologies for detection, and studies on the effectiveness of the textual intervention. in the following section instead, we will focus on a few methodologies specifically devoted to hs-cn pairs collection. hate datasets. several datasets have been collected from smps including twitter @xcite @xcite , facebook @xcite , whatsapp @xcite , and forums @xcite , in order to perform hate speech classification @xcite @xcite . hate detection. most of the research on hatred online focuses on hate speech detection @xcite @xcite employing features such as lexical resources @xcite , sentiment polarity @xcite and multimodal information @xcite to build a classifier. hate countering. recent work has proved that counter-narratives are effective in hate countering @xcite @xcite @xcite . several cn methods to counter hatred are outlined and tested by @xcite , @xcite , and @xcite . ## cn collection approaches three prototypical strategies to collect hs-cn pairs have been presented recently. crawling (crawl). @xcite focus on the intuition that cns can be found on smps as responses to hateful expressions. the proposed approach is a mix of automatic hs collection via linguistic patterns, and a manual annotation of replies to check if they are responses that counter the original hate content. thus, all the material collected is made of natural/real occurrences of hs-cn pairs. crowdsourcing (crowd). @xcite propose that once a list of hs is collected from smps and manually annotated, we can briefly instruct crowd-workers (non-expert) to write possible responses to such hate content. in this case, the content is obtained in controlled settings as opposed to crawling approaches. ## characteristics of the datasets regardless of the hs-cn collection strategy, datasets must meet two criteria: quality and quantity. while quantity has a straightforward interpretation, we propose that data quality should be decomposed into conformity (to ngos guidelines) and diversity (lexical & semantic). additionally, hs-cn datasets should not be ephemeral, which is a structural problem with crawled data since, due to copyright limitations, datasets are usually distributed as a list of tweet ids @xcite . by generating the data through crowdsourcing or nichesourcing, the problem is avoided. quantity. while the crawl dataset is very small and ephemeral, representing more a proof of concept than an actual dataset, the crowd dataset involved more than 900 workers to produce ≈ 41k cns. on the other hand, the niche dataset is constructed by the participation of ≈ 100 expertoperators to obtain ≈ 4k pairs (in three languages) and resorted to hs paraphrasing and pair translation to obtain the final ≈ 14k hs-cn pairs. evidently, employing non-experts, e.g, crowdworkers or annotators, is preferable in terms of data quantity. quality. in terms of quality, we consider that diversity is of paramount importance, since verbatim repetition of arguments can become detrimental for operator credibility and for the cn intervention itself. following @xcite , we distinguish between (i) lexical diversity and (ii) semantic diversity. while lexical diversity focuses on the diversity in surface realization of cns and can be captured by word overlapping metrics, semantic diversity focuses on the meaning and is harder to be cap-tured, as in the case of cns with similar meaning but different wordings (e.g.,"any source?" vs. "do you have a link?"). (i) semantic diversity & conformity. to model semantic diversity and conformity, we focus on the cn 'argument' types that are present in various datasets. argument types are useful in assessing content richness @xcite . in a preliminary analysis, crowd cns are observed to be simpler and mainly focus on 'denouncing' the use of profanity while niche cns are found richer with a higher variety of arguments. on the other hand, crawl cns can cover diverse arguments to a certain extent while being highly prone to contain profanities. to perform a quantitative comparison, we randomly sampled 100 pairs from each dataset and annotated them according to the cn types presented by @xcite the results are reported in ## cn generation through author-reviewer architecture since none of the aforementioned approaches alone can be decisive for creating proper cn datasets, we propose a novel framework that combines crowdsourcing and nichesourcing to obtain new quality data while reducing collection cost/effort. the key elements of this combination are: (i) there must be an external element in the framework that produces hs-cn candidates, (ii) non-experts should prefilter the material to be presented/validated by experts. thus, we settle on the author-reviewer modular architecture @xcite . in this architecture the author has the task of generating a text that conveys the correct propositional content (a cn), whereas the reviewer must ensure that the author's output satisfies certain properties. the reviewer finally evaluates the text hostile "hell is where u belong! stupid f***t... go hang yourself!!" denouncing "the n word is unacceptable. please refrain from future use." fact "the majority of sexual assaults are committed by a family member, friend, or partner of the victim, and only 12% of convicted rapists are muslim. it is not the religion, its the individuals, whether they're muslim or not." -niche. the author-reviewer architecture that we propose differs from the previous studies in two respects: (i) it is used for data collection rather than for nlg, (ii) we modified the original configuration by adding a human reviewer and a final postediting step. we first tested four different author configurations, then three reviewer configurations keeping the best author configuration constant. a representation of the architecture is shown in figure 1 . gpt-2 is an unsupervised transformer-based @xcite lm trained on a dataset of 8 million web pages, capable of generating coherent text and can be fine-tuned and/or conditioned on various nlg tasks. we used the medium model, which was the largest available during our experimentation and contains 345 million parameters, with 24 layers, 16 attention heads, and hidden state size of 1024. we fine-tuned two models with gpt-2, one on niche and one on crowd datasets for counter-narrative generation. niche -training and test data. we have split 5366 pairs of hs-cn for training and the rest (1288 pairs) for testing. in particular, the original hs-cn pairs, one hs paraphrase, and the pairs translated from fr and it were kept for training while the other hs paraphrases were used for testing. see @xcite for further details. crowd -training and test data. although the crowd dataset was created for dialogue level hs-cn, we could extract hs-cn pairs by selecting the dialogues in which only 1 utterance was labeled as hs. therefore, we could guarantee that the crowd-produced cns are exactly for the labeled utterance. we then applied a 80/20 training and test split, obtaining 26320 and 6337 pairs. baselines. in addition to the fine-tuned gpt-2 models, we also evaluate two baseline models. considering the benefits of the transformer architectures on parallelization and learning long-term dependencies over recurrent models @xcite , we have implemented the baseline models using transformer architecture. the models have been trained similar to the base model described by @xcite with 6 transformer layers, batch size of 64, 100 epochs, 4000 warmup steps, input/output dimension of 512, 8 attention heads, inner-layer dimension of 2048, and drop-out rate of 0.1. we used nucleus sampling also for the baselines with a p value of 0.9 during decoding. in brief, we have trained four different configu-rations/models as authors: metrics. we report both standard metrics (bleu @xcite , bertscore @xcite ) concerning the lexical and semantic generation performances and a specific diversity metric (rr) regarding the generation quality. as a second quality metric, we report novelty (wang and wan, 2018) based on jaccard similarity function (a variant of the same metric is used also by @xcite ). while diversity is used to measure the ability of the model to produce diverse/varied responses with respect to the given input hs, novelty is used to measure how different the generated sequences are with regard to the training corpus @xcite . ## the reviewer the task of the reviewer is a sentence-level confidence estimation (ce) similar to the one of machine translation @xcite . in this task, the reviewer must decide whether the author output is correct/suitable for a given source text, i.e. a hate speech. consistently with the mt scenario, one application of ce is filtering candidates for possible human post-editing, which is conducted by the ngo operator by validating the cn. we tested three reviewer configurations: ## ngo operators experiments to verify that the author-reviewer approach can boost hs-cn data collection, we run an experiment with 5 expert operators from an ngo. we compared the filtering strategies to reveal the best depending on several metrics. ## conclusions to counter hatred online and avoid the undesired effects that come with content moderation, intervening in the discussion directly with textual responses is considered as a viable solution. in this scenario, automation strategies, such as natural language generation, are necessary to help ngo operators in their countering effort. however, these automation approaches are not mature yet, since they suffer from the lack of sufficient amount of quality data and tend to produce generic/repetitive responses. considering the aforementioned limitations, we presented a study on how to reduce data collection effort, using a mix of several strategies. to effectively and efficiently obtain varied and novel data, we first propose the generation of silver counter-narratives -using large scale unsupervised language models -then a filtering stage by crowd-workers and finally an expert validation/post-editing. we also show promising results obtained by replacing crowd-filtering with an automatic classifier. as a final remark, we believe that the proposed framework can be useful for other nlg tasks such as paraphrase generation or text simplification.