Counterspeeches up my sleeve! Intent Distribution Learning and Persistent Fusion for Intent-Conditioned Counterspeech Generation
Year: 2023

Abstract:
Counterspeech has been demonstrated to be an efficacious approach for combating hate speech. While various conventional and controlled approaches have been studied in recent years to generate counterspeech, a counterspeech with a certain intent may not be sufficient in every scenario. Due to the complex and multifaceted nature of hate speech, utilizing multiple forms of counter-narratives with varying intents may be advantageous in different circumstances. In this paper, we explore intent-conditioned counterspeech generation. At first, we develop IntentCONAN, a diversified intent-specific counterspeech dataset with 6831 counterspeeches conditioned on five intents, i.e., informative, denouncing, question, positive, and humour. Subsequently, we propose QUARC, a two-stage framework for intent-conditioned counterspeech generation. QUARC leverages vector-quantized representations learned for each intent category along with PerFuMe, a novel fusion module to incorporate intent-specific information into the model. Our evaluation demonstrates that QUARC outperforms several baselines by an average of ~10% across evaluation metrics. An extensive human evaluation supplements our hypothesis of better and more appropriate responses than comparative systems. Warning: This work contains offensive and hateful text that some might find upsetting. It does not represent the views of the authors.

heading: 1 Introduction
text: The quantity and accessibility of information on the Internet are constantly growing in the 21st century. This has made it increasingly simpler for users on social media to post hateful or attacking speech, all while hiding behind the veil of anonymity (Mondal et al., 2017). Hate speech (Awal et al., 2021; Chakraborty and Masud, 2022) is an offensive dialogue that uses stereotypes to communicate a hateful ideology, and it can target several protected
qualities such as gender, religion, colour, and disability (Chetty and Alathur, 2018). This type of cyberhate could have long-term implications for both individuals and communities (Masud et al., 2022). Outlawing or regulating hate speech does not appear to be beneficial because it rarely improves the situation and may be interpreted as interfering with free speech (Chandrasekharan et al., 2017). Prohibiting hateful speech has also been demonstrated to have unexpected consequences, but more importantly, it introduces a curb to the opportunity to defend against potential harm with positive, unbiased, and informed statements that could incite change. The best strategy for fending off offensive online remarks is counterspeech (Wright et al., 2017; Schieb and Preuss, 2016). Past initiatives such as WeCounterHate1 and GetTheTrollsOut2 have proven to make a difference; however, the sheer volume of online hate speech (Cao et al., 2021) necessitates the development of a trustworthy and effective counterargument system.
Motivation: Every circumstance that necessitates counterspeech is distinct. Prior work (Zhu
1http://www.wecounterhate.com/. 2https://getthetrollsout.org/.
5792
and Bhat, 2021) in this domain is limited to generating one counterspeech instance for every hate speech. However, while appropriate, a single counterspeech style could fail to produce the desired effect on the attacker and bystanders alike. Mathew et al. (2019) showed that different victimized communities could be perceptible to different types of counterspeeches. The authors analyzed comments from YouTube and compared the popularity of various intents of counterspeeches for different affected communities like POC, LGBT+, and Jews. They concluded that most likes and replies were received by different kinds of counterspeech instances for different communities – e.g., facts and humor in the case of LGBT+. These observations indicate that a counterspeech generation model would benefit from a diverse output pool, and generating appropriate counterspeeches for different scenarios would provide a better opportunity to educate the attacker and the general public. We support our argument with an example in Figure 1. For a given hate speech, we generate counterspeeches from Generate-Prune-Select (GPS) (Zhu and Bhat, 2021) – a popular counterspeech generation model, and fine-tuned DialoGPT (Zhang et al., 2020b). Though the counterspeeches with intents question and denouncing, respectively, are semantically appropriate and can be used as valid responses, we argue that the legitimacy of the evidence supporting the Holocaust would be best addressed by a factual/informative counterspeech. To the best of our knowledge, this paper presents the first successful pipeline for intent-controlled counterspeech generation.
Our Contribution: We propose a novel task of intent-specific counterspeech generation that aims to generate a counterspeech for a given hate speech and a desired counterspeech intent. In total, we consider five counterspeech intents, namely – informative, question, denouncing, humor, and positive. We curate IntentCONAN, an intent-specific counterspeech generation dataset consisting of 6, 831 counterspeeches for 3, 583 hate speech instances. Further, we propose QUARC, a novel two-phased counterspeech generation framework. In the first stage, QUARC learns vector-quantized representations for every intent and leverages the learned representations to generate desired intent-specific counterspeech in the second stage. Our comparative analysis and human evaluation demonstrate
QUARC’s superior performance over several baselines both empirically and qualitatively.
In brief, we make the following contributions: • Novel task – Intent-specific counterspeech gen-
eration, which results in a diverse pool of counterarguments for a given hate speech. • Novel dataset – IntentCONAN with 6831 counterarguments for 3583 hate speeches spanning across five counterspeech intents. • Novel model – QUARC, a two-phased intentspecific counterspeech generation framework. • Evaluation – An extensive comparison and human evaluation to quantify the efficacy of our approach w.r.t state-of-the-art baselines.
Reproducibility: We open-source the code and dataset at: https://github.com/ LCS2-IIITD/quarc-counterspeech.

heading: 2 Related Works
text: Counterspeech Datasets: An effective counterspeech can de-escalate the conversation and positively affect the audience of the counterspeech (Benesch et al., 2016). However, the scale limitations in manual counterspeech generation have prompted the automatic generation of counterspeech. The first bottleneck is the availability of hate speechcounterspeech (HS-CS) datasets of high quality. Several strategies have been employed for dataset curation. Qian et al. (2019) focused on a crowdsourcing approach in which non-expert crowdworkers were instructed to write responses to hate speeches from Reddit and Gab. The first large-scale HS-CS dataset, CONAN (Chung et al., 2019), ensured quality by relying on niche-sourcing NGO experts to generate counterspeech. Further, to address the shortcomings of manual curation of datasets, Tekiroğlu et al. (2020) presented a hybrid approach of dataset curation in which language models are trained on seed datasets of HS-CS pairs to generate new pairs validated and edited by annotators. Recently, Fanton et al. (2021) created Multi-Target CONAN, which contains labels for different target communities, and the counterspeeches are generated through a semi-automatic mechanism.
Automatic Counterspeech Generation: Qian et al. (2019) made an initial attempt to automatically generate counterspeeches using a Seq2Seq model. Zhu and Bhat (2021) employed a three-task pipeline consisting of an encoder, grammar check, and counterspeech retrieval based on hate speech
for generating diverse counterspeeches. While research has shown the potency of using conditioned counterspeech depending on the context (Mathew et al., 2019; Hangartner et al., 2021), the generation task is still in its infancy. Recently, Saha et al. (2022) proposed CounterGEDI, a model to control attributes like politeness, detoxification, and emotions of the generated counterspeeches using class-conditioned language models. However, the model does not include specific intents described in Benesch et al. (2016).
Controlling Methods for Generation: Prior studies on controlled language generation aimed to enforce user-specified constraints while generating texts. These approaches can exploit constraints at inference time (Dathathri et al., 2020) or be applied during the training of the model (Wu et al., 2021). For controlled dialogue generation, Lin et al. (2021) used a series of lightweight adapters on top of a language model for high-level control of dialogues generated. In other work, Keskar et al. (2019) finetuned separate models for each attribute. While the above models show promising results for the task of controlled generation, we find that these models cannot be used directly for generating controlled counterspeeches with hate speech and intent as the input. This is due to the scarcity of counterspeeches for each intent and the overlap between the intents that make it harder for the model to learn the differences.

heading: 3 Dataset
text: We begin by analyzing existing works to determine the intent categories for IntentCONAN. CONAN (Chung et al., 2019) derives nine intent categories from Benesch et al. (2016), whereas Mathew et al. (2019) defined seven intent categories with minor variation. In contrast, due to the scarcity of data points, we club a few semantically-similar intents together in IntentCONAN, e.g., we combine the positive and affiliation intents as positive. In total, we consider five intent categories, i.e., informative, question, denouncing, humor, and positive in this work. Table 7 in Appendix A highlights the relationship among the three sets of intent categories.
The publicly-available Multi-Target CONAN dataset (Fanton et al., 2021) consists of ~5, 000 HS-CS pairs. However, it does not comprise any intent label for the counterspeeches. First, we extract the HS-CS pairs and clean them to remove redundancy. Subsequently, we employ three do-
main experts3 to annotate the existing CS with an intent and write new CS for the remaining intent categories. Although, we capped the annotations at the values mentioned in Table 1 so as to not induce repetitiveness; i.e. not every hate speech in IntentCONAN has five counterspeeches. The count-wise statistics are: 5 CS-per-HS: 10%, 4 CS-per-HS: 5%, 3 CS-per-HS: 20%, 2 CS-per-HS: 10%, and 1 CS-per-HS: 55%. An example of annotated counterspeeches for various intents is shown in Table 2.
Annotation Guidelines: Prior to the annotation, we make sure that the annotators have a comprehensive understanding of the field-manual4 for “responding to online abuse”. In our pilot study, we conduct several rounds of deliberation with all annotators over the understanding of the counterspeech. In particular, annotators consider the following objectives for every intent of speech: Estab-
3The annotators are experts in NLP and social media. 4https://onlineharassmentfieldmanual.
pen.org/.
lish the Goal: Each type of counterspeech necessitates a distinct fundamental idea, speech style, and goal. De-escalate: Each counterspeech instance should be written in a manner that would neutralize the situation and, ideally, not provoke retaliation or further hate speech. Avoid Hostile Language: Under no circumstance was threatening speech, name-calling, profanity, or hostility to be displayed while annotating counterspeech instances. Subsequently, annotators label and write the intentspecific counterspeeches for 3, 583 distinct hate speech instances. Table 1 shows IntentCONAN’s detailed statistics. Appendix A contains more information about the dataset.

heading: 4 Proposed Methodology
text: In this section, we define the architecture and the structural details of our proposed framework, QUARC. Our key insight is that a counterspeech instance can be decomposed into two distinct components – its semantics and intent. In particular, we can convey the semantics of the same counterspeech (which can be regarded as the compositional message) in multiple manners, such as through humor, as a question, in an informative manner, etc., depending upon the desired intent. More formally, given the counterspeech yi, the semantics si and the intent ci, we posit that there exists a function ζ such that yi admits a factorization yi ∼ ζ(yi|si, ci). The primary goal of our method is to learn contextuallyrich representations to seamlessly integrate the desired intent information with the semantics of the counterspeech to yield effective intent-conditioned counterspeeches. To this end, we design a novel two-phase training pipeline in which we attempt to learn the vector-quantized representations of each intent and propose a fusion mechanism, PerFuMe, to integrate this information into the model.
Let us denote the dataset D = {(x1, t1, c1, y1), · · · , (xn, tn, cn, yn)}, where xi denotes the ith hate-speech instance, ti denotes the target of xi, yi denotes the counterspeech corresponding to xi, and ci denotes the category/intent of yi. Our end goal is to learn a stochastic counterspeech generation function χ, such that yi ∼ χ(·|xi, ci). We decompose this task into two phases, where we design two models: CLIME and COGENT. CLIME is designed to learn the quantized codebook vectors corresponding to each intent. This is done by learning a functional mapping ζ, which aims to reconstruct the
counterspeech yi from its semantic encoding zsi and the intent encoding efi corresponding to ci as ŷi ∼ ζ(·|zsi , efi ). For COGENT, we utilize the Intent Codebook C, assimilated through CLIME to learn χ, which takes as input the semantic encoding of the hate speech xsi , as well as the encoding of desired intent efi , to yield ỹi ∼ χ(·|xsi , e f i ). The overall architecture is depicted in Figure 2.
4.1 Codebook Learning Model (CLIME)
The overall purpose of CLIME is to learn the codebook representations for each intent category. It comprises two modules: ITEM and QUINCE; ITEM is utilized to generate the semantic encoding, while QUINCE is utilized to procure the representation of the desired intent. The representations obtained from these modules are passed through our novel fusion mechanism, PerFuMe, and the emitted output is passed onto the decoder for the reconstruction of the original counterspeech. Note that CLIME does not utilize the hate speech instance xi, and solely works on the counterspeech yi and its intent ci in a reconstructive fashion.
Intent-Unaware Semantic Encoding Module (ITEM): The counterspeech yi is first tokenized into its sub-word embeddings yti ∈ Rn×D, where n is the maximum input length and D is the latent dimension of the model. These embeddings are then passed through the semantic encoder, ϕs, which is parameterized by a BART encoder, to yield the semantic representation zis ∼ ϕs(zsi |yti) ∈ Rn×D. It is crucial that the information contained in zsi reflects only the semantics of the counterspeech, and not the intent, in order to enable effective learning of intent representations separately. If the intent information were distilled within zsi , the model would not need to rely on the codebook vector efi to reconstruct the sample, rendering the learned intent distribution trivial. To combat this, we train an intent classification module on top of zsi , and use a gradient-reversal layer to expunge intent-specific information from within zsi . The intent classifier is trained jointly with the reconstruction module.
Quantized Intent Encoding Module (QUINCE): The tokenized embedding yti is passed to the intent encoder, ϕi (parameterized by a BART encoder), to obtain the form encoding, zfi ∼ ϕi(z f i |yti). To learn a globally applicable quantized distribution for all intents, we employ a codebook similar to a VQ-VAE (van den Oord et al., 2017). The intent-
codebook, C ∈ R|C|×D, is a matrix where each row corresponds to the embedding of one intent. Our aim is to jointly learn the codebook for further utilization in generating intent-conditioned counterspeeches. We accomplish this by using the reconstruction objective as well as using a loss function similar to van den Oord et al. (2017), which moves the pooled version of zfi closer to the codebook vector efi corresponding to ci (e f i = C(ci)), and vice versa, using a stop-gradient operator, sg(.). sg(.) is defined as identity and zero during forward and backward propagation, respectively. Since the semantic encoding zsi has had its intent-specific information stripped through the gradient reversal layer, this information must be distilled in the quantized efi in order to facilitate effective reconstruction. Reconstruction: The generated embeddings zsi and efi (from ITEM and QUINCE, respectively) are then passed into our adaptive-gated fusion mechanism, PerFuMe to yield zi ∈ Rn×D. zi is then given to the decoder as input to generate ŷi ∼ ζ(·|zsi , efi ), the reconstructed output. We train the model by minimizing the negative loglikelihood of ŷi with respect to the reference yi as well as incorporating auxiliary losses from ITEM and QUINCE, as follows:
L = ED[−logpζ(yi|zsi , efi )+||z fp i −sg(e f i )||2+
||sg(zfpi )− e f i ||2 + logp(ci|zsi )] (1)
where zsi ∼ ϕs(·|yi), efi = C(ci), z f i ∼ ϕi(·|yi) and zfpi = Pool(z f i ).

heading: 4.2 Conditioned Counterspeech Generation
text: Model (COGENT)
The objective of the second phase is to generate counterspeeches that are conditioned on the desired intent, given an input hate speech. This is achieved through the utilization of COGENT, which comprises TREAD, a module designed to map the input hate speech xi to a semantic encoding of the counterspeech, which can then be fused with the codebook vector efi corresponding to the specified intent as learned through CLIME. The following sections provide a more in-depth description of the functions of these modules.
Target-Aware Semantic Mapping Module (TREAD): The hate speech xi is passed through the semantic encoder ϕs to obtain its semantic representation x̂si ∼ ϕs(·|xi) ∈ Rn×D. – Target Information Incorporation: Since the semantics of the hate speech should inherently possess discriminative characteristics to determine the intended target of hate speech, we explicitly strengthen x̂si by incorporating target category ti through a joint classification loss. x̂si is passed through a target classification module to yield t̂i ∈ R|T |, where |T | denotes the total number of target categories in the dataset. t̂i denotes the probability distribution over all targets for xi and is optimized via the negative log-likelihood loss with the actual target ti. – Semantic Mapping: The semantic representation x̂si encompasses information about the semantics of hate speech; however, we require the semantics
of the corresponding counterspeech to coalesce with the desired intent. To facilitate this, we define a mapping function ξ, which maps the semantics of hate speech to the desired counterspeech as ẑsi ∼ ξ(·|x̂si ). In practice, ξ is parameterized by a multi-layered Transformer Encoder (Vaswani et al., 2017), which is learned jointly. We term the parameterized version of ξ as the contextual mapper.
Counterspeech Generation: The semantic mapping of counterspeech, ẑsi ∈ Rn×D is then fused with the codebook vector efi through PerFuMe and passed to the decoder to yield the generated counterspeech ỹi ∼ χ(·|ẑsi , efi ) ∈ Rn×D. COGENT is trained by minimizing the negative loglikelihood loss of generating yi, as well as the auxiliary target loss as follows:
L = ED[−logp(yi|ẑsi , efi )− logp(ti|x̂si )] (2)
with x̂si ∼ ϕs(·|xi), ẑsi ∼ ξ(·|x̂si ) and efi = C(ci).

heading: 4.3 Persistent Fusion Mechanism with
text: Adaptive Gating (PerFuMe)
Coalescing intent-specific information with the semantics of a counterspeech can prove to be a challenging task as the model may not pay heed to the desired intent and generate a counterspeech that respects the desired semantics but has a different form than required. To address this problem, we propose PerFuMe, a persistent fusion module where we repeatedly synchronize the intentencoded information with the semantic information to ensure that the desired form is not overlooked. We also enhance this fusion procedure with adaptive gating, where we design two distinct gates to control the degree of semantic and intent-specific information leveraged during integration.
More formally, let the semantic and intentspecific information be denoted by zsi ∈ RN×D and efi ∈ R1×D, respectively. e f i is stacked on top of itself N times to obtain ẽfi ∈ RN×D. We obtain ẑi ∈ RN×D as:
ẑi = a(. . . a((a((z s i⊕ẽfi )W1+b1)⊕ẽ f i )W2+b2)
· · · ⊕ ẽfi )Wk + bk) (3)
where a denotes a non-linear activation function, ⊕ represents concatenation, W1,W2 . . .Wk ∈ R2D×D, and b1, b2 . . . bk ∈ RN×D are trainable
matrices. We also introduce two gates, s-gate and igate, which control the flow of semantic and intentspecific information, respectively.
µs = σ(z s iWs1 + ẽ f i Wi2 + bs) µi = σ(z s iWs2 + ẽ f i Wi1 + bi) (4)
Ws1,Ws2,Wi1,Wi2 ∈ RD×D, and bs, bi ∈ RN×D are trainable parameters. µs and µi are designed to filter the information emitted from the semantic and intent-specific encodings, respectively.
zsemi = µs ⊙ zsi + (1− µs)⊙ ẽfi zinti = (1− µi)⊙ zsi + µi ⊙ ẽfi (5)
where ⊙ denotes the Hadamard product. Finally, we resolve the information obtained from s-gate (zsemi ), i-gate (z int i ) and the persistent fusion mechanism (ẑi) to produce the fused matrix zi = ẑi + z sem i ⊙ zinti , where zi ∈ RN×D.

heading: 5 Experimental Setup and Results
text: In this section, we delineate an exhaustive analysis of our model’s performance and also carry out a predictive comparison against text generation models using both human and automatic evaluation.
Comparative Systems: • Generate Prune Select (GPS) (Zhu and Bhat, 2021) uses a three-stage pipeline for generating counterspeeches. The first stage generates a large number of counterspeeches using an autoencoder architecture which is further pruned using a grammatical model. Finally, the most suitable counterspeeches are chosen for hate speech using a vector-based response selection model. • Plug And Play Language Model (PPLM) (Dathathri et al., 2020) We utilize finetuned GPT-2 as the base language model for PPLM. • In addition, we fine-tune DialoGPT (Zhang et al., 2020b) and BART (Lewis et al., 2020) on IntentCONAN as well. For all four comparative models, we provide the desired intent as prompt.
Evaluation Metrics: We employ Rouge (Lin and Hovy, 2003) and Meteor (Banerjee and Lavie, 2005) scores to evaluate the syntactic correctness of the generated counterspeech. Given that Rouge and Meteor metrics primarily assess surface-level overlap, their standalone usage may not provide a comprehensive evaluation of the effectiveness of the generated counterspeech instances, considering the possibility of multiple correct outputs. To address this limitation, we augment these metrics by
incorporating measures of semantic richness and conducting thorough human evaluations to ensure a more comprehensive assessment. For semantic richness, we report BERTScore (BS) (Zhang et al., 2020a) along with cosine similarity (SS) obtained from a sentence-transformers model (all-miniLMv2) (Reimers and Gurevych, 2019). Moreover, to check the efficacy of the models in incorporating the desired intent in the generated counterspeeches, we compute category accuracy (CA) through an intent classification (IC) model.
Result Analysis: The results are reported in Table 3. We observe that QUARC beats the baselines across all metrics. In terms of lexical similarity, GPS is the best-performing baseline as it demonstrates high scores on R1, R2, RL, and Meteor. However, QUARC reports higher scores by a margin of ~10% on the syntactic similarity measures except for R2. On the semantic similarity measure, QUARC outperforms the best baseline (GPS) by ~2% and ~5% on BS and SS scores, respectively. This demonstrates the ability of our framework to generate semantically coherent counterspeeches to a given hate speech. In the context of generating intent-conditioned counterspeeches, CA evaluates the appropriateness of the generated counterspeeches. We observe that the majority of the baselines are notably inferior in producing outputs corresponding to the desired intent. For instance, while GPS is able to produce syntactically and semantically coherent outputs, it falls short in terms of accurately preserving the intended intent and is outperformed by our framework by 79%. Due to the explicit design of our pipeline, QUARC is able to efficaciously generate counterspeeches that preserve the desired intent (c.f. Appendix C).

heading: Method Div Nov
text: To obtain a deeper insight into the performance of QUARC and the best baselines (GPS and BART), we compute novelty and diversity in line with Wang and Wan (2018) (c.f. Table 4). These metrics measure the lexical dissimilarity between the generated instances and the training corpus, as well as the dissimilarity between the generated instances themselves. They convey the degree of originality and variety in the generated text and can serve as an indicator of over-reliance on memorization of the training corpus or repetition of outputs. The scores achieved by QUARC (0.68 and 0.67) indicate a relatively higher degree of originality and variety in the generated text, while GPS exhibits the lowest scores across these metrics, indicating a tendency to memorize data from the training set and generate similar outputs more frequently. This might also be one of the reasons for its relatively high performance on similarity metrics. We must observe, however, that a high score on these metrics, when accompanied by a low semantic and syntactic similarity, marks that the generated outputs may not be entirely aligned with the intended ones.
Ablation Study: To dissect and identify the individual components that drive our framework, we perform multiple ablations on its architecture. In the standard version of QUARC, we initialize the codebook vectors by using the mean-pooled version of the representations of each intent obtained from an intent classifier. When we remove this initialization strategy and initialize the codebook vectors randomly for CLIME, we observe a slight drop-off in all metrics. However, when we retain this initialization strategy and directly use these vectors in COGENT without undergoing the first phase, a higher drop in almost all scores (except CA) is observed. The drop is especially high in diversity, which demonstrates that the generated texts have more repeated tokens across test samples, explaining the higher CA score as compared to the first ablation.
We performed another ablation in which we added a memory bank component to COGENT. We
stored the semantic representations zsi of each counterspeech instance in training set in a memory bank in the first phase while utilizing CLIME. When we perform contextual mapping in the TREAD module inside COGENT to map the semantics of the hate speech x̂si to that of the corresponding counterspeech ẑsi , we used the representations stored in the memory bank to align ẑsi and z s i closer to each other via an auxiliary loss given by ||zsi − ẑsi ||2. However, this ended up degrading the performance, perhaps due to the overfitting and lack of generalization owing to the relatively smaller training set size. We performed another ablation in which we removed all residual connections from both CLIME and COGENT to see its effect, and we noted a similar drop in performance, In the last two ablations, we again noted a large drop in diversity, which demonstrates that both CLIME and residual connections are critical in generating non-repetitive distinct counterspeeches.
Qualitative Analysis: For qualitative evaluation, we report the outputs of QUARC and the best baseline (GPS) for two instances in Table 5. In each case, we show the outputs for two desired CS intents. We observe that QUARC does a fair job in generating CS with the desired intents in three out of four cases, whereas the intents of generated CS in GPS align with the desired intent in only one out of four cases – even for the correct case, GPS produces an incoherent statement. For H2 with the desired humor intent, both QUARC and GPS commit mistakes for the intent (i.e., question for QUARC and denouncing for GPS); however, the output is a valid CS, ignoring the desired intent. Our analysis suggests that GPS and other baselines perform poorly in generating the desired intent-conditioned
CS as compared to QUARC.
Human Evaluation: Given the limitations of empirical evaluation in holistically assessing the efficacy of generation models, we conduct a comprehensive human evaluation on a random subset of the generated counterspeeches from QUARC and GPS (detailed instructions in Appendix E). The subset was uniformly distributed across intents. We ask our evaluators5 to rate the outputs on the following metrics: Independent CS (IC) denotes whether the generated instance can be considered as CS without any context; Conditioned CS (CC) shows whether the generated output is an appropriate response to the given hate speech; Adequacy (A) depicts whether the generated CS is grammatically sound, coherent and fluent; Toxicity (T) indicates whether the output can be considered toxic. For each of the above metrics, the evaluators are instructed to rate every counterspeech on a 5-point Likert scale. For example, considering the Toxicity metric (T), a score of 1 denotes that the counterspeech can be considered completely non-toxic, 3 denotes neutral and 5 denotes highly toxic. Category Accuracy (CA) determines if the counterspeech adheres to the desired intent; here the evaluators are told to assign the counterspeech to one of the five intents to the best of their ability.
The results of the human evaluation (c.f. Table 6) indicate that QUARC outperforms the best baseline by a significant margin in all metrics except toxicity. These results demonstrate that the outputs generated by our model are not only more effectively recognized as counterspeeches but are also
5A total of 60 evaluators in the field of NLP and social science participated, having ages between 20-30 years with 60:40 male to female ratio.

heading: Model Human Evaluation MetricIC ↑ CC ↑ A ↑ T ↓ CA ↑
text: more closely aligned with the intended response to the consumed hate speech. Moreover, the results attest to the efficacy of our intent-specific representation and fusion-based approach through the CA metric. We observe fair agreement (κ = 0.32) on Fleiss’ Kappa scale amongst the evaluators (Fleiss and Cohen, 1973).
Congruence: We introduce Implicit Similarity (IS), a metric that utilizes implicit feedback from human evaluation to reflect the similarity between intent pairs. Intuitively, the core idea behind IS is that when different evaluators assign a different intent category to the same counterspeech, there exists a certain affinity between those categories. As an example, if evaluator A assigns the intent Informative to a counterspeech, and evaluator B assigns the intent Positive to the same counterspeech, then there exists a certain similarity between the intents Informative and Positive. The strength of this affinity can be approximated via its relative frequency of occurrence, and the method for its computation is described below.
We calculate IS for every possible intent pair; since there are 5 intents, there are a total of 5C2 = 10 distinct pairs. Let the counterspeech yi be generated in response to the hate speech xi with the desired intent ci. The human evaluators are asked to classify the intent of yi from the defined set of 5 intents – {I1, I2, I3, I4, I5} without knowledge of the actual intent ci. Each evaluator from the group of N evaluators assigns the intent for yi and we obtain the relative frequency of the classified intents as
Vi = {I1 : vi1, I2 : vi2, I3 : vi3, I4 : vi4, I5 : vi5}, where ∑5 j=1 vij = 1, and vij denotes the fraction of evaluators that assigned yi to the intent class Ij . The implicit similarity for a pair of intents (Ia, Ib) for the ith counterspeech is computed as ISa,bi = via × vib × NS, where NS = 4 is the normalizing factor applied to standardize the range of ISa,bi to [0, 1] (since the maximum value of via × vib is 0.25). ISa,bi is indicative of the similarity between a pair of intents, as a higher value of ISa,bi deems that the same sample was assigned to both Ia and Ib consistently by evaluators (without knowledge of the desired ci), and thus, there exists a certain affinity between these intent classes. Hence, we compute the overall implicit similarity between (Ia, Ib) for the set of K counterspeeches given to the human evaluators as ISa,b = ∑K k=1 IS a,b k /K. Note that IS is calculated without the knowledge of the desired intent ci to provide a more faithful picture.
We plot the learnt representation of each intent category (after dimensionality reduction through PCA) along with the computed IS scores (Figure 3). We note that the IS scores closely align with the distances between the learnt representations. This congruence not only demonstrates the robustness of the learnt representations, but also provides a key insight into a critical factor behind the superior performance of QUARC (more details in Appendix D).

heading: 6 Conclusion
text: In an effort to address the pervasive issue of hateful speech on the internet, we proposed the novel task of intent-conditioned counterspeech generation. We developed IntentCONAN, the first intent-specific dataset for diverse counterspeech generation. Further, to benchmark the dataset, we proposed a novel framework (QUARC) that decomposes the task into two phases – CLIME learns the intent distribution which is subsequently leveraged by COGENT to generate the intent-conditioned counterspeeches. We conducted an extensive evaluation (i.e., empirical, qualitative, and human) to establish the effectiveness of QUARC.

heading: Acknowledgement
text: Authors acknowledge the partial support of Logically and Infosys Center of AI (CAI), IIIT Delhi.

heading: Limitations
text: The current work marks the first step towards intentconditioned counterspeech generation, and as we noted, even though our model excels in fluency, a larger and more diverse dataset paired with knowledge grounding is necessary to improve and ensure factual correctness. Although the annotators kept the quality of counterspeech as high as possible, it is possible that this data is not at par with other datasets that are annotated by more skilled NGO operators, as is the case with the Multi-Target CONAN dataset (Fanton et al., 2021). A more large-scale annotation of our dataset with higher instances for under-represented target communities would hence be beneficial to learn more accurate distributions of every counterspeech class. Another limitation of the current work is that it exhibits a slightly higher-degree of toxicity compared to the baseline. It, therefore, pertains to accounting for lowering the amount of toxicity present in the generated counterspeeches as future research. Lastly, humor in counterspeech is a very subjective topic, and inspite of including only a few datapoints from that class as compared to the others in our dataset, it is likely that QUARC could generate vague and/or offensive text under the pretext of humor. We intend on keeping the dataset private and only provide access for research and educational purposes.

heading: Ethics Statement
text: We recognize that combating online hate speech can be a delicate matter, and we fully acknowledge that research in this domain might raise ethical and moral concerns. This work is simply the beginning of efforts to create a consistent and diversified compendium of counterspeeches for every hateful instance. We also agree that models used to automate counterspeech could end up producing factually erroneous statements, and a more efficient method of incorporating real-world knowledge into these models is required. On the other hand, even if generative models could perform well, there is still a pressing need for a large-scale counterspeech dataset with a more diversified response pool to ensure a net positive outcome. Furthermore, while a deployable model for counterspeech is not completely feasible as of now, there are organizations like United Against Hate6 who are making considerable contributions to mitigate hate online.
6https://www.united-against-hate.org/.

heading: A Dataset

heading: A.1 Objectives for the Annotators
text: Following are the guidelines strictly adhered to by the annotators while constructing counterspeeches for the five intents:
Informative Counterspeech: Informative or Factual statements are the most popular form of counterspeech in which a counterspeech seeks to debunk or fact-check the claims made by the attacker. We found some instances of counterspeech in the CONAN dataset which were inconsistent with the definition of facts, a few being:
(HS) “Don’t the Moslem’s Holy books say ’kill infidels wherever they may be hiding’”
(CS) No, it doesn’t.
(HS) “Muslims are the ones who invented the slave trade.”
(CS) Slavery has been rampant in early societies of all kinds. I am sure the Romans did not take inspiration for their slave trade directly from Muslims.
Therefore, to ensure the validity of the counterspeeches without having to fact-check every statement from the Multi-Target CONAN dataset, we decide to rename the class to "Informative Counterspeech"; this seems more apt, and makes writing counterspeeches for our annotators easier. Furthermore, because our annotators were unfamiliar with facts from each target community, we relied on official sources like Red Cross, The Holocaust Encyclopedia, RAINN, The Anti-Defamation League, Brookings, and credible news sources like CNN, HuffingtonPost (among others) to verify that the annotations were factually correct as far as possible for this class.
Questioning Counterspeech: For this class, the annotators were instructed to frame countermeasures in the form of questions that would challenge the speaker’s chain of reasoning and compel them to either answer convincingly or recant their original remark. If necessary, factual information was to be obtained from a pre-determined pool of data sources, as indicated in the preceding section.
Denouncing Counterspeech: This category of counterspeech needed to be handled with caution, as denouncing can sometimes be used to propagate obscene language. Our annotators were directed to convey the impression that the opinions put forth by the hate speaker are not acceptable without using name-calling or profanity.
Humorous Counterspeech: A heated dispute or discussion can be effectively defused by humor and sarcasm (Mathew et al., 2019). By highlighting how absurd it is, humor undercuts the hate speech and aids in diverting the attention of those following the dialogue online. Annotators were asked to construct a sentence that would not incite resentment from other users while also making sure that it would not contain any controversial ideas or terms. It should be mentioned that the annotators had prior knowledge of the sarcasm and humour that are well-received on social media.
Positive Counterspeech: The use of empathy and positive reinforcement in hate speech can lead to a decline in online animosity (Hangartner et al., 2021). Regardless of the severity of the hate speech, the annotators make an effort to compose a courteous, polite, and civil statement. Furthermore, we argue that if bystanders who are following the discourse online are a member of the group impacted by the comment, they would be instilled with a sense of support and humanness.

heading: A.2 Dataset Statistics
text: Figure 4 gives an overview of our dataset: IntentCONAN. Figures 4a and 4b show the distributions of the target communities in the hate speech and intents across the counterspeeches, respectively.
For a more fine-grained perspective, Figure 4c and 4d show the uniform distributions of intents in the data splits and the intents across target communities. Figures 4e and 4f depicts the average token lengths for the five intent classes and eight target communities.

heading: B Additional Details on Experiments
text: Experimental Setup: All the experiments were performed using a Tesla V100 and an RTX A6000
GPU. Our model (and the BART baseline) was trained for 20 epochs with the initial learning rate of 8e-5 using AdamW as the optimizer and a linear scheduler, with 10% of the total steps as warmup having a weight decay of 0.03. Training the model took an average time of 3 hours with a batch size of 32, and the model with the best validation loss was employed for testing. We used the base version of BART (140M parameters) from the transformers library (Wolf et al., 2020) for parameterizing both ϕs and ϕi. The baselines were trained using the recommended hyperparameter settings. To compute the ROUGE score, we use the rouge library in python with the default arguments, we compute METEOR through nltk (bir), semantic similarity by using the all-miniLM-v2 model from the sentence-transformers library (Reimers and Gurevych, 2019) and BERTScore using the original bert-score library. To check the efficacy of the models in incorporating the desired intent in the generated counterspeeches, we train an Intent Classification (IC) model on IntentCONAN for intent classification of each counterspeech instance, which achieves 75% accuracy on the test set for classification (we utilize the base version of RoBERTa). The IC model is used to classify whether the generated counterspeeches are compatible with the desired intent, and the accuracy
obtained across the generated samples is reported as the category accuracy.

heading: C Analysis of Intent-Conditioning
text: In order to systematically evaluate the effects of intent conditioning, we begin by analyzing the accuracy of the IC model for each intent separately. The results are depicted in Figure 5. From the bar chart, we observe that the accuracy of the intents – informative and question, is higher than the other intents, while humor displays the lowest accuracy. To obtain a more comprehensive understanding, the confusion matrix illustrates that the intents denouncing and positive tend to be recognized as informative by the IC model in some cases, while humor can also be recognized as informative and denouncing. Since the IC model is susceptible to errors, it is hard to say with certainty whether the generated counterspeech belongs to the desired intent, or whether the model has misclassified it. Hence, we utilize the confusion matrices from human evaluation and design a new metric in the next section for analyzing the intent conditioning due to the inherent reliability of human evaluators.
D Interpretability and Robustness of Intent Representations
A key advantage afforded by our approach is the exploration of interpretability, which is enabled by our paradigm of learning the intent representations separately. The intent representations illustrated in Figure 3 (left) depict that the intents positive and denouncing are both mapped closely to informative, and are slightly farther away from each other, while question and humor are considerably distant to all other intents. This observation is further supported by computing the cosine similarity in the original dimension of the representations (Fig. 7c). To assess the robustness of the obtained representations, we use implicit feedback from human evaluations to gauge the similarity between intents. We employ two strategies: (i) we design a new metric, Implicit Similarity (IS) to compute the similarity between pairs of intents implicitly through human evaluation responses without the knowledge of the actual intent; (ii) we utilize the intent information and use the confusion matrices obtained from human evaluation (Fig 6a) for this purpose.
We plot the IS values for each intent pair in Figure 7. The IS scores for the pairs (I,D) and (I, P ) are the highest, followed by the pair (P,D),

heading: Informative

heading: Denouncing
text: while the lowest scores are achieved by the pairs (P,H), (Q,P ), (I,H) and (Q,H). Interestingly, the IS scores closely align with the distances between the intent representations in the scatter-plot in Figure 3. This demonstrates the robustness of the intent representations learned by QUARC and highlights a critical factor responsible for its performance, as the representations align with the proximity that is inherently captured by evaluators.

heading: Explicit Similarity through Human Evaluation:
text: To further analyze the intent representations, we also utilize the desired intent ci to generate the confusion matrices for human evaluation in Figure 6. We observe a similar pattern to that observed through IS, as we can see that the bottom-right 3× 3 square has a darker shade as compared to the rest of the matrix, denoting that the Informative, Positive and denouncing intents are closer together when compared to other pairings.

heading: E Human Evaluation
text: The evaluators recruited were well-versed in the field of NLP and social media. The form provided to them contained the descriptions of terminology
such as Hate Speech and Counterspeech, and Intents. For further clarity, a few lines of description for each intent along with an example were also shown. The form also included information on the format of the questionnaire; the evaluators were made aware of how the evaluation data would be used in the study and were warned against the possibility of encountering foul or offensive language that could be upsetting.
Analysis: As shown in Figure 6, our model generates intent-identifiable outputs across all intents, with the exception of the humor, where the outputs were often assigned to denouncing. Conversely, GPS fails to effectively condition on intent, as evidenced by the mismatch between desired and obtained intents, with decent performance only on informative, perhaps due to its prevalence in the training set.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Section 7. Limitations
3 A2. Did you discuss any potential risks of your work? Section 8. Ethical Considerations
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract and Section 1. Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3. Dataset and Section 4. Methodology
3 B1. Did you cite the creators of artifacts you used? Section 3. Dataset. We extend the original dataset and cite the dataset.
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Section 1: Introduction, Section 2: Related Works
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 1: Introduction
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Section 3. Dataset
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Table 3
C 3 Did you run computational experiments? Section 5.Experimental Setup and Results
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix B. Additional details on Experiments. Experimental setup
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix B. Additional details on Experiments. Experimental setup
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix C: Further Analysis
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix B. Additional details on Experiments. Experimental setup
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 5 : Experimental Setup and Results
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Appendix D: Human Evaluation Details
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 5 : Experimental Setup and Results
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Section 2: Related Works
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Section 5 : Experimental Setup and Results

heading: Future Work
text: It, therefore, pertains to accounting for lowering the amount of toxicity present in the generated counterspeeches as future research. Lastly, humor in counterspeech is a very subjective topic, and inspite of including only a few datapoints from that class as compared to the others in our dataset, it is likely that QUARC could generate vague and/or offensive text under the pretext of humor. We intend on keeping the dataset private and only provide access for research and educational purposes