A Fine-Grained Taxonomy of Replies to Hate Speech
Authors: Xinchen Yu
Ashley Zhao
Eduardo Blanco
Lingzi Hong
Venue: EMNLP 2023 Main (2023)
Keywords: counterspeech
hate speech
taxonomy

Abstract: Countering rather than censoring hate speech has emerged as a promising strategy to address hatred. There are many types of counterspeech in user-generated content: addressing the hateful content or its author, generic requests, well-reasoned counter arguments, insults, etc. The effectiveness of counterspeech, which we define as subsequent incivility, depends on these types. In this paper, we present a theoretically grounded taxonomy of replies to hate speech and a new corpus. We work with real, user-generated hate speech and all the replies it elicits rather than replies generated by a third party. Our analyses provide insights into the content real users reply with as well as which replies are empirically most effective. We also experiment with models to characterize the replies to hate speech, thereby opening the door to estimating whether a reply to hate speech will result in further incivility.

A Fine-Grained Taxonomy of Replies to Hate Speech
Xinchen Yu!
Ashley Zhao²
Eduardo Blanco'
Lingzi Hong'
'University of Arizona
TAMSS University of North Texas
3 University of North Texas
xinchenyu@arizona.co edu
Abstract
Hateful post: You are full of fat feek liberuls who jump to say person died because they didn't wear flimsy piece of cloth not because they made awful life decisions and stuffed their faces.
Countering rather than censoring hate speech has emerged as promising strategy to address hatred. There are many types of counterspeech in user-generated content: addressing the hateful content or its author, generic requests, well-reasoned counter arguments, insults, etc. The effectiveness of counterspeech, which we define as subsequent incivility, depends on these types. In this paper, we present theoretically grounded taxonomy of replies to hate speech and a new corpus. We work with real, user-generated hate speech and all the replies it elicits rather than replies generated by third party. Our analyses provide insights into the content real users reply with as well as which replies are empirically most effective. We also experiment with models to characterize the replies to hate speech, thereby opening the door to estimating whether a reply hate speech will result in further incivility.
Reply addressing the author: You like don't like here. So why are you loser here? [subsequent hate: 2]
Reply addressing the content: The fe*king virus killed them. not the obesity. If they didn't 's get the virus, they would still be alive. [subsequent hate: 0]
Table I: An excerpt from two Reddit conversations. The first reply disagrees with the hateful post by attacking author, and there are two additional hateful com- ments in the subsequent conversation. The second reply disagrees with the hateful post by addressing the con- tent, and there are no subsequent hateful comments.
counterspeech into finer- wner-grained categorics has en- hanced our understanding of interactions between hate speech and counterspeech. Therefore, has the potential to identify effective interventions to deal with hate speech and further promote con- structive discourses. Existing studies mostly focus the identification categorization of counter- speech and ignore replies that do counter but may effective in de-escalating online hatred (i.e.. neutral replies). In addition, few studies working on categorizing counterspeech have (a) differenti- ated the target of counterspeech (i.c., addressing the author or the hateful content), or (b) grounded the categorization argument mining theories.
1 Introduction
Counterspeech refers to "direct response that counters hate speech" (Mathew et al., 2019). is a remedy to address hate speech (Richards and Calvert, 2000) by spelling out the hateful content conveyed in an utterance, or challenging, ques- tioning, rejecting, disputing or confronting the hate (Langton, 2018; Goffredo et al., 2022). Dis- tinct from approaches hate mitigation by content moderation (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018), counterspeech is preferable as it does not interfere with the principle of free and open public spaces for debate (Schieb and Preuss, 2016; Chung et al., 2019).
Counterspeech addressing the author hate speech can lead to completely different conver- sational outcomes than counterspeech addressing hateful content. Table shows two replies counter- ing hateful post. While both replies use hostile language (Mathew et al., 2019), the former dis- agrees with the hateful post by denigrating the au- thor (L... you loser), and results in two additional hateful comments in the subsequent conversation. The latter reply contradicts the content of the hate- ful post and backs up with credible rationale
Recently, the NLP communities have contributed сorpora for the detection (Mathew al., 2019; He al., 2021; Albanyan and Blanco, 2022; Yu et al., 2022) and catcgorization (Mathew et al., 2019; Gof- fredo et al., 2022) of counterspeech. Categorizing

---

Authors
Source (Lang.)
Size
Balance
Hate
Categorization
Taget
He et al. (2021)
Twitter (en)
2,290
22.6%
Vidgen al. (2021)
Reddit (en)
27,494
0.8%
Albanyan and Blanco (2022)
Twitter (en)
11,304
20.0%
4
Yu et al. (2022)
Reddit (en)
6,846
23.7%
4
Mathew al. (2019)
YouTube (en)
11,093
49.5%
Goffredo al. (2022)
Reddit (it)
624
13.0%
Ours
Reddit (en)
3,654
41.2%
Table 2: Summary counterspeech datasets from user-generated web content in terms of Size, counterspeech class Balance ratio, and the inclusion of Hate comment, fine-grained counterspeech Categorization, and Target of counterspeech. We are the first to include a fine-grained categorization of counterspeech that spells out the target.
that does not result in additional hateful comments. Addressing the author rather the content is usually considered weaker form of disagree- ment (Graham, 2008). Previous works in argument mining have looked at counterarguments that attack either the author of an argument (Habernal et al., 2018) or the content of an argument (Wachsmuth et al., 2018 8; Jo et al., 2020; Alshomary et al., 2021). Counterargumennt and counterspeech share sim- ilar purpose: to present an alternative stance statement (Chung. 2022), but counterspeech is not necessarily counterargument (see Section 2).
belongs to. Additionally, present a qualita- tive error analysis.
2 Related Work
Hate Speech There have been several stud- ies on hate speech, including efforts on defin- ing taxonomies for hate speech. Examples in- clude hate speech against both individuals and groups (Zampieri al., 2019), against some spe- cific identities (Waseem et al., 2017; Fortuna and Nunes, 2018), against only a single identity (Guest et al., 2021; Vidgen and Yasseri, 2020), or cate- gories for implicit hate (EISherief et al., 2021).
In this paper, we present a categorization of coun- terspeech that is well-grounded on theories in argu- ment mining and builds a bridge between these two related yet unconnected research areas. Specif- ically, our categorization differentiates between counterspeech that addresses the author of hate speech and the hateful content itself. Therefore, could provide useful insights into counterspeech generation and evaluation strategies. Importantly, our categorization can help uncover how language usage in different types of replies is tied to the future conversation trajectory.
Counterspeech
a strategy to address hate speech that does not require content removal, and it has received increasing attention. Although a few works have built fine-grained counterspeech taxonomies (Mathew al., 2019; Goffredo et al., 2022; Allaway al., 2023), they do not differen- tiate between the target of the counterspeech (i.e., the author of hate or the hate itself). In addition, they fail to convey the intensity of hostile tone counterspeech (Benesch et al., 2016; Mathew et al.. 2019). For example, insulting somebody's 's intelli- gence is more uncivil than asking them to leave the discussion. There have been curated with the help of trained opera- tors (Qian et al., 2019; Chung al., 2019; Fanton et al., 2021) 1) or by generative models (Allaway al., 2023). However, synthetic counterspeech is not as rich as genuine counterspeech and rarely used by real users. For example, "/... are inappropri- ate" and "/... should be avoided" made up 5.6% of replics in the synthetic counterspeech dataset by Qian et al. (2019). On the other hand, found no real users using this kind of generic language counter hate speech in our dataset. In this paper, we work with counterspeech written by regular people out of their own motivations and desires.
The main contributions are follows:!
taxonomy of replies to hate speech with categories indicating (a) whether a reply disagrees with hate speech and (b) whether replics that disagree address the author of hate speech or the content as well as secondary categories;
A theoretically-gronnde
few synthetic datasets that
Using the taxonomy to create annotated dataset of replies to hate speech from Reddit; Comparing different types of replies hate speech with respect (a) language usage and (b) their relation conversational outcomes; Building models to predict the primary and secondary categories a reply to hate speech
'Data available at https: thtp::: https///gthabb.co///nheeee// com/xinchenyu/
counter_taxonomy

---

Table summarizes existing counterspeech datasets from user-generated web content. We are the first to propose fine-grained taxonomy of replies to hate speech that differentiates be- tween (a) replies countering and not countering hate speech and (b) replies that counter hate speech by addressing the author of hate speech and the hateful content. Our work also complements recent efforts to capture and understand counterspeech replies that attack the author of the hate speech (Al- banyan and Blanco, 2022). We note two differ- ences. First, attacking the author only a subset addressing the author; we consider broader set of scenarios, for example. accusing or blaming the author's behavior (namely, Accusation). Second, they consider two types of counterspeech, while we work with fine-grained categorization.
Reply to Hate
Counter Author
Insult
Accusation
Sarcasm and humor
Counter Content
Claim
Justification
Request
Sarcasm and humor
Sympathy
Non-counter Agree
Non-counter Neutral
Figure I: Taxonomy of replies to hate speech.
Counterspeech vs Counterargument A coun- terargument is considered an ad-hominem argu- ment if it argues against the author rather than the content (Habernal et al., 2018). Otherwise, it argues against the content of the argument by denying the premises, conclusion, the reasoning between them (Walton, 2009; Wachsmuth al., 2018). There are two main differences between counterspeech and counterarguments. First, coun- terspeech does not necessarily attack a hateful comment-it may provide suggestions to the author of the hateful comment (e.g.. "Please be nice to oth- ers"). On the other hand, counterargument always attacks either argument its author. Sccond, when addressing the content, counterspeech com- ment does not require supporting evidence, while counterargument usually does (Chung, 2022).
and (b) the analysis of how they relate to ineffec- tive and uncivil communication. Figure shows the taxonomic structure.
3.1 Counter Author
Counter Authors refers to the language that dis- agrees with the hateful comment by addressing the author or some features of the author's char- acter (Tindale, 2007). The secondary categories comprise three sub-types: Insult, Accusation, and Sarcasm and humor.
Insult refers to language that explicitly attacks the author of the hateful comment, including (a) vulgar insults (Holgate et al., 2018: Habernal et al., 2018); "your hole"" (b) intelligence insults (Habernal et al., 2018; Vidgen et al., 2021): "so stapid I can't 't help you"; (c) dehumanization (Leader Maynard and Benesch, 2016; Vidgen et al., 2021): "you are just trash"; or (d) threatening language (Zampieri et al., 2019): "beat me if you dare". Using insults is usually considered the worst way to disagree with hate speech (Graham, 2008).
3 A Taxonomy of Replies to Hate Speech
We present new taxonomy to categorize replies to hate speech that comprises four primary cate- gories and eight secondary categories. bridges the gap between argumentation theories (Haber- nal et al., 2018; Habernal and Gurevych, 2017: Wachsmuth et al., 2018) and counterspeech en- countered in genuine user-generated web data. The taxonomy covers two dimensions of arguments: logos (i.e., logical arguments) and pathos (i.e., ap- pealing to emotion) (Aristotle and Kennedy, 1991). Therefore, our categories are mutually exclu- sive but represent principal types of replies to hate speech. By detailing argument components, this taxonomy facilitates (a) the evaluation of logical fallacies and emotional appeals in counterspeech
Accusation refers language that (a) blames or spells out the behavior or intention of the author of the hateful comment, including but not limited to lying (Holgate et al., 2018), trolling (Mihaylov and Nakov, 2016), ignorance (Jain et al., 2014), name-callin name-calling (Kenski et al., 2020), etc., (b) asks the author to leave the discussion, (c) accuses the author of belonging an affiliation or identity- based group (Vidgen et al., 2021) in negative way (e.g., incels, radical right or left).
Sarcasm and humor refers to language that uses sarcasm (Waseem and Hovy, 2016; Justo et al.,

---

2014) humor (Fortuna and Nunes, 2018; Gof- fredo et al., 2022) address the author of the hate- ful comment. For example, "Keep trying! You will make a good comment eventually" uses sarcasm to mean the opposite of what it (explicitly) states.
"It is so frustrating when people don't listen to him but try to paint a picture of him".
3.3 Non-counter Agree / Non-counter Neutral
Non-counter Agree refers to agreeing with the hate- ful comment, including instances that not only agree but also include additional hate. On the other hand, Non-counter Neutral is language that nei- ther agrees nor disagrees with the hateful comment. Some examples include shifting discussion topics, pointing to external sources, and sharing stories.
3.2 Counter Content
Counter Content includes language that disagrees by addressing the content of the hateful comment. There are five sub-types in the secondary categories: Claim, Justification, Request, Sareasm and humor, and Sympathy. We provide details below for these sub-types except for Sarcasm and humor, as the definition is similar as the definition of the sub-type by the same name under Counter Author. Indeed, the only difference is that the sarcasm or humor is directed towards the hateful comment.
4 Data Collection and Annotation
We choose Reddit as the source of data and use the PushShift API to retrieve whole conversation threads.2 To mitigate topic and author biases that keyword sampling may introduce (Wiegand et al., 2019; Vidgen et al., 2021), we use community- based sampling. We select 35 subreddits that are thought to be hateful (Qian et al., 2019; Guest et al., 2021; Vidgen al., 2021), including sub- reddits such as r/MensRights, r/Seduction, and r/PurpleillDebate. We refer the reader Ap- pendix A for the full list. There are a total of 1,382,596 comments from 5,325 submissions.
Claim is an assertion put forward (Toulmin et al., 1979) that shows disagreement with the hateful comment by rebutting its premise. evidence, or conclusion, or targeting the reasoning between them (Walton, 2009) without providing any jus- tifications reasons for supporting the claim.
Justification refers to language that provides one or more justifications as evidence or reason to oppose the hateful content (Albanyan and Blanco, 2022). For example, the following reply is justified: "This is racist. Non-white men are also represented in
4.1 Identifying Hateful Comments
As the prevalence of online hate in the wild is very low (0.1 (0.1% in English language social media (Vid- gen et al., 2021)), we first build three models identify candidate comments with uncivil content in the 1.382,596 comments using pre-trained mod- els (Liu et al., 2019) with the corpora by Davidson et al. (2017), Qian et al. (2019), and Vidgen et al. (2021). We consider a comment as candidate com- ment if any of the three classifiers predicts it to be uncivil. There comments. Uncivil comments cover broader cases than hateful comments (Davidson et al., 2020). To ensure quality. we validate candidates via crowd- sourcing. We choose Amazon Mechanical Turk (MTurk) as the crowdsourcing platform. Anno- tators are provided with a detailed definition and examples of hate speech (Ward, 1997; Davidson et al., 2017; Vidgen et al., 2021). Only crowdwork- that pass short 10-question qualification test checking for understanding can keep working on our task (minimum correct answers: 90%). Each candidate comment is annotated by three annota- tors. The Krippendorff's s a coefficient (Krippen-
this infographic, not only the white men".
Request is the language that (a) questions the valid- ity of the content in the hateful comment and asks for more evidence or a justification (Walton, 2005; Chung al., 2021), or (b) makes suggestions the author of the hateful comment in a non-negative way (e.g. "Hey there are definitely better ways to say it"). We include Request in Counter Content based on our observations that making suggestions is usually closely related to the content of the hate- ful comment. Prior work includes speech that uses sympathy or kindly makes suggestions in the same category (namely, positive tone) (Mathew et al., 2019). Instead, we include the latter in Request and the former in a new sub-type, Sympathy.
total of 5,469 candidate uncivil
Sympathy refers to language that expresses the feeling of sorrow or puts in good word for some- one being attacked in the hateful comment (Sosca and Caragea, 2022). The target being attacked by hate speech is sometimes the group the au- thor of counterspeech affiliates with. For example, they may share identities or belong to the same affiliation based group (Mathew et al., 2019), as in
2nttps://pushhhift.ie io/api -ataappppppammmerss..

---

Primary
Secondary
p-value
Bonf.
Insult
252
50
Textual factors
Accusation
168
33
Total tokens
444
Counter Author
Sarcasm & humor
83
17
Second pronoun
Total
503
100
Question mark
Negation
Claim
361
36
Name entity (norp)
Justification
269
26
Request
227
23
Sentiment and social role factors
Counter Content
Sarcasm & humor
79
8
Disgust words
Sympathy
66
7
Negative words
Total
1,002
100
Positive words
Polite words
Non-counter Agree
Total
791
100
Male words
Non-counter Neutral Total
1,358
Female words
Table 3: Percentage of replics each category. Non- counter Agree and Non-counter Neutral do not have secondary categories, so only the total is shown.
Table Linguistic analysis comparing the replies in Counter Author and Counter Content comments. Num- ber arrows indicates the p-value Hoam-Whitney U test; one: p<0.05, two: p<0.01, and three: p<0.001). Ar- row direction indicates whether higher values correlate with Counter Author (up) or Counter Content (down). A check mark (v) indicates that the statistical test passes the Bonferroni correction.
dorff, 2011) among a total of 26 annotators is 0.65, which indicates substantial agreement (Artstein and Poesio, 2008). The labels are obtained using the majority vote. 1,065 out of the total 5,469 candi- date comments were labeled by the crowdworkers hateful comments.
substantial agreement. The final corpus consists 3,654 replies to hateful comments, each of which is assigned a category in our taxonomy. Only 3% of the replies are labeled with multiple categories and excluded in the following analyses.
4.2 Annotating Replies to Hateful Comments Next, we collect all the direct replies to each hateful comment that are 50 tokens or fewer. We reserve longer replies, which are more likely to fall into multiple categories, for future work. We label the replies using our fine-gres fine-grained taxonomy (Section 3). A reply that falls into more than one subcategory will be labeled with multiple categories.
We refer the reader to Appendix B for data statement providing more details about our corpus.
Corpus Analysis
Table presents the percentage of labels that be- long to primary and secondary categories. For the primary categories, the most frequent label is Non-counter Neutral, accounting for 37.2% of the replics followed by Counter Content (27.3%), Non-counter Agree (21.6%) and Counter Author (13.8 (3...).. To surprise, when disagreeing with the hateful comment by addressing the author, most replies use Insult (50.1%). When disagrecing by addressing the content, Claim is the most frequent secondary category (36.0%).
Since fine-grained categories are too subtle for MTurk workers, we hire three rescarch assistants to be our expert annotators. The annotators under- went 4 weeks of training and are either native or fluent English speakers. There are total of four annotation phases. In each phase, we collect anno- tations for 1,000 replies (except the last phase). In the first phase, we walk them through 100 replics, resolve disugreements, and refine confusing label definitions. We then let them independently work on the next 150 replies. Pairwise Cohen's be- tween annotators is above 0.70, which considered substantial (Artstein and Poesio, 2008), so each an- notator then labels an independent partition of the data with 250 replics. We repeat the above steps for the three phases to ensure high annota- tion quality and keep the time spent on annotations reasonable. The Krippendorff's coefficients for the 150 double-annotated replies in each phase are 0.78, 0.73, 0.68, and 0.71, which are considered
5.1 Linguistic Cues
Past work has found that lexicon-based features can differentiate between Counter replies from Non-counted replies to hateful comments (Mathew et al., 2019; Albanyan and Blanco, 2022). In study, explore whether linguistic and lexicon- based features can distinguish Counter Author from Counter Content. We analyze the linguistic char- acteristics of replies to hateful comments when

---

Counter vs Non-counter
Author
One sub-type vs other terminal nodes
Content
Insult
Sarcasm
Justification
Request
Sympathy
Number of uncivil comments Number civil comments Number of total comments
111
t1 f 11
444
11
M1
Table 5: Analysis of conversational outcomes by comparing the replies that are (a) Counter vs Non-counter, (b) when Counter, Counter Author vs Counter Content, and (c) selected sub-type and all other terminal nodes in the taxonomy (i.e., the eight subtypes, Non-counter Agree, and Non-counter Neutral). We refer the two subtypes named Sarcasm and humor as Surcasm. Arrow direction indicates whether higher values correlate with the former in each comparisons (up). Tests having at least one arrow have passed the Bonferroni correction.
replies are Counter Author or Counter Content to shed light on the differences between the lan- guage people
ments, and (c) the number of total comments. Un- civil comments have been identified in Section 4.1; all the other comments are considered civil. We verified the reliability of this method by manually annotating a sample of 150 comments and obtained Cohen's s = 0.67 between the manual and auto- mated labels. This is considered substantial agree- ment and suggests that the automated labeling is sound measure of whether comment is civil or uncivil. Results are shown in Table
when disagreeing with hateful comments. All factors we consider are based on counts of (a) textual features or (b) words indi- cating sentiment and social roles. We check for negation cues (Fancellu et al., 2016), mentions of all named entities, 3 and sentiment and social role factors using the Sentiment Analysis and Cognition Engine (SEANCE) lexicon (Crossley et al., 201 7). We run Mann- Whitney U test (Mann and Whitney, 1947) and report the results in Table 4.
Counter vs Non-counter Replies that dis- agree with the hateful comments (Counter) elicit (a) fewer civil comments and (b) more uncivil com- ments in the subsequent conversations. Thus com- bating hatred may elicit wider discussions but at the same time introduce additional uncivil behaviors.
We observe several interesting findings:
Regarding textual features, disagreeing by ad- dressing the author uses more second pro- nouns, while addressing the content uses more tokens, question marks, and negation cues. Regarding sentiment and social role features, there significantly more negative and dis- gusting words in replies when addressing the author, as well as less positiveness, politeness, and words related to the social roles of men (i.e., buddy, boy, actor) and women (i.e., aunt, bride, daughter).
Counter Author vs Counter Content When disagreeing with hateful comments, addressing the author correlates with fewer civil comments. This indicates that arguing against hatred in public on- line conversations may escalate hatred, but address- ing the hate speakers may be even worse.
5.2 Conversational Outcomes
A selected sub-type all other terminal nodes Regarding the number of uncivil comments, Insult replies correlate with more subsequent uncivil com- ments. Surprisingly, we found that some Counter Content sub-types may attract more follow-up dis- cussions and incubate additional uncivil behaviors (Justification, Request, and Sympathy). Our find- ings are consistent with previous work showing that when correcting misstatements, language toxicity increases (Mosleh al., 2021).
Do replies hateful comments result in more or less civility follow-up conversations according to our fine-grained taxonomic categorization? Sim- ilarly to the previous analysis, we look for sta- tistically significant differences in conversational outcomes between (a) Counter and Non-counter groups. (b) Counter Author and Counter Content groups, and (c) selected sub-types replics and all other terminal nodes in the taxonomy (i.e., the eight subtypes, Non-counter Agree, and Non-counter Neutral). We consider several properties of the sub- sequent conversation after reply: (a) the number uncivil comments, (b) the number of civil com-
Experiments
We experiment with models to solve two tasks:
Primary category classification. Given a reply to hateful comment, these models determine
Phttps://spacy.ion htw.://sagggggggiiiiiffeeeeeutte Liguisticcffettures

---

Non-counter
Counter Author
Counter Content
W. Average
Agree
Neutral
R
F1
R
F1
R
R
Fl
P
F1
Majority
0.00
0.00
0.00
0.36
1.00
0.53
0.00
0.00
0.00
0.00
0.00
0.00
0.13
0.36
0.19
Random
0.24
0.24
0.24
0.36
0.26
0.30
0.14
0.23
0.18
0.20
0.21
0.21
0.26
0.23
0.24
Hate
0.40
0.23
0.29
0.68
0.55
0.50
0.09
0.15
0.38
0.47
0.42
0,43
0.43
0.39
Replyf
0.53
0.34
0.42
0.55
0.59
0.57
0.70
0.52
0.59
0.43
0.59
0.49
0.54
0.52
0.52
-pretrainf
0.46
0.55
0,50
0.54
0.58
0.56
0.71
0.55
0.62
0.54
0.48
0.51
0.55
0.54
0.54
+blend
0.46
0.48
0.47
0.55
0.67
0.60
0.78
0.56
0.65
0.51
0.43
0.47
0.56
0.55
0.55
Hatc+Replyt
0.56
0.45
0.50
0.58
0.70
0.63
0.74
0.59
0.66
0.56
0.56
0.56
0.60
0.59
0.59
preetrint*
0.52
0.48
0.50
0.61
0,56
0.58
0,68
0.59
0.63
0.51
0.65
0.57
0.57
0.57
0,57
+blend#$
0.53
0.45
0.49
0.61
0.71
0.66
0.74
0.59
0.66
0.56
0.59
0.58
0.61
0.60
0.60
Table 6: Results obtained with several models. We indicate statistical significance (McNemar's test (McNemar, 1947) over the weighted average) as follows: & indicates statistically significant (p < 0.05) results with respect to the Hate model, and & with respect the Reply model. We only show results pretraining and blending with the best instances (stance and EDA using only the Reply and pretraining and blending respectively, and counterspeech and EDA using Hale+Reply and pretraining and blending respectively). Training with the hate comment + reply coupled with blending and data augmentation yields the best results (FI: 0.60).
if it is Counter Author, Counter Content, Non- counter Agree, or Non-counter Neutral; 2. Counterspeech sub-type classification. Given reply that disagrees with a hateful com- ment (i.e., Counter Author Counter Con- tent), these models determine the sub-type of counterspeech from our ontology.
ing with related tasks is beneficial. Pretraining takes place prior to training with our task. The re- lated corpora that we use are hate speech (Davidson et al., 2017), sentiment (Rosenthal et al., 2017), sar- casm (Ghosh al., 2020), counterspeech (Yu et al., 2022), and stance (Pougué-Biyong al., 2021).
Blending Additional Data We also experiment with a complementary approach: blending addi- tional corpora during the training process (Shnarch et al., 2018). We use both our corpus and additional corpora to train for m blending epochs, and then use only our corpus train for another epochs. Besides re-using the corpora used for pretraining (see above), we also create additional instances by adapting EDA (Easy Data Augmentation) to aug- ment our own corpus (Wei and Zou, 2019). Specif- ically, we use Synonym Replacement, Random Insertion, Random Swap, and Random Deletion.
We use a 70-15-15 split for each task. All of our models are neural classifiers with the RoBERTa transformer (Liu et al., 2019) as the main compo- nent. We use the pretrained models by Hugging- Face (Wolf et al., 2020) and Pytorch (Paszke et al., 2019) to implement our models.
6.1 Primary Category Classification
The neural classifiers consist of the RoBERTa trans- former and another two fully connected layers to make predictions. To find out whether adding the hate comment would be beneficial, we consider three textual inputs: (a) the hate comment; (b) the reply to the hate comment; and (c) the hate com- ment and the reply. When considering both the hate comment and the reply. we concatenate them using [SEP] special token. The baseline models we use are the majority and random baselines. For the for- mer, the majority label is predicted WWnn-countrr Neutral, Table 5). For the latter, a random label out of the four primary categories predicted.
6.1.1 Quantitative Results
Table 6 shows the results per label and weighted averages. We provide here results pretraining and blending with the most beneficial tasks: stance for pretraining and EDA for blending. Using only the reply input offers competitive performance with F1 scores up to 0.52 compared with the random bascline (F1: 0.52 vs. 0.24). Using both the hate comment and the reply as input yields much better results (F1: 0.59 vs 0.52). Consistent with previous work (Yu et al., 2022), this indicates that including the conversational context does help. Finally, the network that blends the corpus augmented by EDA
In addition to standard supervised learning, we explore two strategies to improve performance:
Pretraining with Related Tasks We experiment with several corpora to investigate whether pretrain-

---

Error Type
Example
Ground Truth
Predicted
Rhetorical
26 Hate: p*ek worthless inbreds who've ve contributed nothing to society,
question
Reply: Where your contributions? doubt there's 's any.
Author
Content
Irony
21 Hate: Retarded republicans fear everything
Reply: It's amazing how broken you have to be to believe in their positions as whole.
Author
Agrec
Discourse
Hate: Nothing. These hoes are just glorified bums. Treat em like they
information
act.. low-grade prostitutes.
Reply: At least prostitutes are honest about it.
Agree
Neutral
General knowledge
8
Hate: This comment section is smelling of finknng Incels. But this girl is loser. Put ker back in jail.
Reply: Last I checked, 'incels' were people who think they" unlovable, not guys who hale attempted murder.
Content
Neutral
Metaphorical language
8 Hate: This c built like toad. Reply: She reminds me Porsche 911.
Agree
Neutral
Table Most common error types made by the best model (predictions by hate comment + reply blending).
P
R
Fl
quired to make correct predictions. In other words, the reply itself looks neutral and does not contain hate speech, but the combination of both the hateful comment and the reply does convey hatred. Errors may also occur when general knowledge is required to understand hateful content (8%) or when the re- ply uses Metaphorical language (EISherief et al., 2021) to implicitly express hatred (5%). In the bot- tom example from Table 7, a woman is referred to as a "toad" and "Porsche 9/1".
Insult
0.62
0.66
Accusation
0.50
0.28
0.36
Sarcasm and humor
0.30
0.33
0.32
Claim
0.61
0,55
0.58
Justification
0.64
0.65
0.65
Request
0.69
0.73
0.71
Sympathy
0.21
0.60
0.32
Weighted Average
0.58
0.57
Table 8: Results of counterspeech type classification obtained with the best model.
6.2 Counterspeech Sub-type Classification
and takes both the hate comment and the reply as input yields the best results (F1: 0.60).
We further build models for a 7-way classification task in which the input is a Counter Author Counter Content reply along with the hate com- ment and the output is the sub-type of counter- speech presents in the reply. Table 8 shows the results. We retrain the best-performing system from Table 6: (Hate + Reply blending). The Sarcasm and humor class in Counter Author and Counter Content are combined together consid- ering that the prevalence of both classes in our corpus is low (Counter Author: 2.3%; Counter Content: 2.2%). Results show that predicting the fine-grained counter sub-type for Counter reply challenging task (F1: 0.57). Regarding recall, the model performs better when predicting Request. Justification, or Insult, compared with Accusation and Sarcasm and humor.
6.1.2 Qualitative Analysis
To further understand the challenges in pri- mary category classification,
manually ana- lyze 100 random errors made by our best model (Hate++eply+blend), We discovered set of error types grounded on the language used in the replies and hateful comment. Table 7 lists and exemplifies the most common error types.
Rhetorical questions (Schmidt and Wiegand, 2017) are the most common (26%) type of errors. the example, the model fails realize that the reply does not ask for further validation but instead accuses the author the hateful comment making no contributions. Irony (Nobata et al., 2016; Qian et al., 2019) is also common error type (21 %) and requires reasoning and understanding. The ex- ample uses irony to attack the author instead of
Conclusion
In this paper, we introduce a diteoreticallyggoondde taxonomy of fine-grained categories of replies hate speech. The taxonomy allows us to differen- tiate replies that disagree by addressing the author
showing agreement ("amazing how broken").
We also found that 18% of errors occur when discourse information (de Gibert et al., 2018) is re-

---

of hate speech from addressing the content. We annotate noo-Large-bu--suustatiie amount of Red- dit comments based on this taxonomy, thereby en- abling the research community to better understand and model replies to hateful comment.
resolve disagreements, which requires expertise knowledge and quite time-consuming. We fo- cus on replies that fall into only one category, as most comments on social media are short. How- ever, there are also replies that adopt a combination of strategies to respond to hatred, for example, pro- viding justification and showing sympathy at the same time. We reserve such complicated cases for future studies. Finally, while we have set qual- ification test and provide detailed instructions crowdworkers and in-house annotators, people's perceptions hate speech may still vary slightly. We obtain the ground truth of hateful comments using majority vote.
We further analyze whether different types of replies hate speech differ in linguistic features and conversational outcomes in the subsequent con- versations. We find that disagreeing with the hatred by addressing the author uses lcss positive and po- lite words, which indeed relates to less civil conver- sational outcomes compared with disagreeing by addressing the content. Experimental results of a 4- way classification not perfect but encouraging: taking the hateful comment into account coupled with blending and data augmentation yields signifi- cant improvements. Our qualitative analysis of the most common errors show that a lot of errors are made when replies contain rhetorical questions or use irony to show disagreements.
Ethical Considerations
The study has been through careful consideration of the risks and benefits to ensure that the research is conducted in an ethical manner. First, there are no identifiable participants in the process. The data we collected from Reddit is public available and is further anonymized by removing user names and other personally identifiable information for stor- age, annotation, and analysis. Second, we have in- formed crowdworkers and research assistants about the annotation tasks and obtained their content to participate. They were warned in the instructions that the content might be offensive or upsetting. They were also encouraged to stop the annotation process whenever the felt upset or overwhelming. We compensated them with $8.5 per hour. Third, the examples in this work included showcase the severity of the problem with hate speech. They are taken from actual web data and no way re- flect the opinion of the authors. The data will be shared based on methodological, legal, and ethi- cal considerations (Weller and Kinder- Kurlanda. 2016). Finally, we also acknowledge the risk as- sociated with releasing the dataset. However, we believe the benefit of shedding light how people react to hatred outweighs any risks associated with the dataset release.
Our findings point towards several opportuni- ties for promoting healthier interactions in online platforms. For example, while our results show that disagreeing with hate speech may clicit addi- tional incivility, we never claim that counterspeech is counterrroductive. Counterspeech effectiveness can be further assessed based on changes in behav- ior or beliefs undergone by authors of hate speech. needed to guide such research line. Our results have further revealed that replies that kindly make suggestions or show empathy to- wards the target of hate speech may attract more follow-up discussions and more uncivil comments. Additional work is needed to better understand this phenomenon and whether it is present across dif- ferent types of hate speech, for example, misogyny, homophobia, or Islamophobia.
User studies
Limitations
Our work has several limitations. One is that we identify uncivil comments automatically with clas- sifiers. Although we validate the reliability of la- bels obtained by classifiers with a small sample, some uncivil comments that we work with are actu- ally not uncivil. Likewise, we are unable to capture some uncivil comments, for example, implicit hate speech (EISherief et al., 2021). Additionally, our corpus consists of 3,654 replies to hate speech, which is not large. The samples are limited as our manual annotation effort needs annotators to un- derstand and label the graiined categories and
References
Abdullah Albanyan and Eduardo Blanco. 2022. Pin- pointing fne-grained relationships between hateful tweets and replics. Proceedings of the AAA/ Confer- ence on Artificial Intelligenne, 36(10))00418-1046
Emily Allaway, Nina Taneja, Sarah-Jane Leslie, and Maarten Sap. 2023. Towards countering essentialism through social bias reasoning.

---

Milad Alshomary. Shahbaz Syed, Arkajit Dhar, Martin Potthast, and Henning Wachsmuth. 2021. Counter- argument generation by attacking weak premises. In Findings of the Association for Computational Lin- guistics: ACL-UJCLLP 2021, pages 1816-1827, On- line. Association for Computational Linguisties.
Ona de Gibert, Naiara Perez, Aitor Garcia-Pablos, and Montse Cuadros. 2018. Hate speech dataset from white supremacy forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 1 11-20, Brussels, Belgium. Association for Computational Linguistics.
Aristotle and G.A. Kennedy, 1991. On Rhetoric: A Theory of Civic Discourse. Oxford University Press.
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaish- navi Anupindi, Jordyn Seybolt, Mummun De Choud- hury, and Diyi Yang. 2021. Latent hatred: A bench- mark for understanding implicit hate speech. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 345-363, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Comput. Linguist., 34(4):555-55.
Emily M. Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computaaionnll Linguistics, 6:587-604 604.
Federico Fancellu, Adam Lopez, and Bonnie Webber. 2016. Neural networks for negation scope detection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics Voolume
Susan Benesch, Derek Ruths, P Kelly Dillon, Mohammad Haji Saleem, and Lucas Wright. field
I: Long Papers), pages 495-504, Berlin, Germany. Association for Computational Linguistics.
2016. Counterspeech on twitter: study. wwwpeeeeehhotttttttree-eee-----e twwtter----taee-ee fee---fielddsuud counterspeech-onn
https: https://daaggroussseeedd /
Margherita Fanton, Helena Bonaldi, Serra Sinem Tekiroglu. and Marco Guerini. 2021. Human-in- the- loop for data collection: multi- target counter narra- tive dataset to fight online hate speech. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 2226-3240, Online. Association for Computational Linguistics.
Accessed: 2023-08-28.
Yi-Ling Chung. 2022. Counter Narrative Generation for Fighting Online Hate Speech. Ph.D. thesis, Uni- versity of Trento, Via Calepina, Trento, Italy.
Yi-Ling Chung, Marco Guerini, and Rodrigo Agerii 2021. Multilingual counter narrative type classifi- cation. In Proceedings of the 8th Workshop Ar- gument Mining, pages 125-132, Punta Cana. Do- minican Republic. Association for Computational Linguistics.
Paula Fortuna and Sérgio Nunes. 2018. A survey automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 5 1-330
Yi-Ling Chung. Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN COunter NArratives through nichesoureing: mul- tilingual dataset of responses to fight online hate speech. In Proceedings of the 57th Annual Meet- ing of the Association for Compulational Linguistics, pages 2819-2822 Florence, Italy, Association for Computational Linguistics.
Debanjan Ghosh, Avijit Vajpayee, and Smaranda Mure- san. 2020. A report on the 2020 sarcasm detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, pages 1-11, On- line. Association for Computational Linguistics.
Pierpaolo Goffredo, Valerio Basile, Bianca Cepollaro, and Viviana Patti. 2022. Counter- An Italian corpus for online counterspeech in ecological con- texts. In Proceedings of the Sixth Workshop on On- line Abuse and Harms (WOAH), pages 57-66, Seattle, Washington (Hybrid). Association for Computational Linguistics.
Scott A Crossley, Kristopher Kyle, and Danielle S Mc- Namara. 2017. Sentiment analysis and social cogni- tion engine (seance): An automatic tool for sentiment, social cognition, and social-order analysis. Behavior research methods, 49(3):803-82) 821.
Paul Graham. 2008 How to disagree. http:// www. paulgraham. com/disagress sagree. html. Accessed: 2023-04-13.
Sam Davidson, Qiusi Sun, and Magaalna Wojeieszak. 2020. Developing a new classifier for automated identification of incivility in social media. In Pro- ceedings of the Fourth Workshop on Online Abuse and Harms, pages 95-101, Online. Association for Computational Linguistics.
Ella Guest, Bertie Vidgen, Alexandros Mittos, Nishanth Sastry, Gareth Tyson, and Helen Margetts. 2021. An expert annotated dataset for the detection of online misogyny. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, pages 1336-1350, Online. Association for Computational Linguistics.
Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15-18, 2017, pages 512-515. AAAI Press.
Ivan Habernal and Iryna Gurevych. 2017, Argumenta- tion mining in nser-generated web discourse, Com- putational Linguistics, 43( (): 125-

---

Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name- calling: Dy- namics and triggers of ad hominem fallacies in web argumentation. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguisties: Human Lan- guage Technologies, Volume (Long Papers), pages 386-396, New Orleans, Louisiana. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, 0bss/197.11
Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochasti- cally larger than the other. The annals of mathemati- cal statistics, pages 50-60.
Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakr- ishnan, Diyi Yang, and Srijan Kumar. 2021. Racism is a virus: Anti-asian hate and counterspeech in so- cial media during the covid-19 crisis. In Proceedings of the 2021 IEEF/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 22,, page 90-94, New York, NY, USA. Association for Computing Machinery.
Binny Mathew, Punyajoy Saha, Hardik Tharad, Subham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, and Animesh Mukherjee. 2019. Thou shalt not hate: Countering online hate speech. In Pro- ceedings of the Thirteenth International Conference on Web and Social Media, ICWSM 2019, Munich, Germany, June 11-14, 2019, pages -69-380. AAAI Press.
Eric Holgate, Isabel Cachola, Daniel Preofiuc-Pietme and Junyi Jessy Li. 2018. Why swear? analyzing vulgar expressions.
Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions pereentages, Psychometrika, 12(2):153-157.
and inferring the intentions In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 4414, Brussels, Belgium. Association for Com- putational Linguistics.
Todor Mihaylov and Preslav Nakov. 2016. Hunting for troll comments in news community forums. In Proceedings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pages 399-405, Berlin, Germany. As- sociation for Computational Linguistics.
Siddharth Jain, Archna Bhatia, Angelique Rein, and Eduard Hovy. 2014. A corpus of participant roles in contentious discussions. In Proceedings of the Ninth International Conference and Evaluation (LREC'14), pages 1751-1756, Reyk- javik, Iceland. European Language Resources Asso- ciation (ELRA).
Language Resources
Mohsen Moslch, Cameron Martel, Dean Eekles, and David Rand. 2021. Perverse downstream conse- quences of debunking: Being corrected by another user for posting false political news increases sub- sequent sharing of low quality, partisan, and toxic content in twitter field experiment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. CHI '21, New York, NY, USA. Association for Computing Machinery.
Yohan Jo, Seojin Bang. Emaad Manzoor, Eduard Hovy, and Chris Reed. 2020, Detecting attackable sen- tences in arguments. In Proceedings of the 2020 Conference on Empirical Methods Natural Lan- guage Processing (EMNLP), pages 1-23, Online. As- sociation for Computational Linguistics.
Chikashi Nobata, Joel R. Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive lan- guage detection in online user content. In Proceed- ings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 15, 2016, pages 145-153. ACM.
Raquel Justo, Thomas Corcoran, Stephanie M Lukin, Marilyn Walker, and M Inés Torrss. 2014. Extract- ing relevant knowledge for the detection of sarcasm and nastiness in the social web. Knowledge-Bond Based Systems, 1221124333
Kate Kenski, Kevin Coe, and Stephen A Rains. 2020. Perceptions of uncivil discourse online: An exami- nation of types and predictors. Communication Re- search, 47(6):795-8144
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
Klaus Krippendorff. 2011.
Computing krippen- dorff's alpha-reliability. https://reposi tory. upenn. lerinn.edu//ssee 08.
Accessed: 2023-02-
John Pougué-Biyong Valentina Semenova. Alexandre Matton, Rachel Han, Aerin Kim, Renaud Lambiotte, and Doyne Farmer. 2021. DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates. In Thirty-fifth Conference on Neu- ral Information Processing Systems Datasets and Benchmarks Truck (Round 2).
Rae Langton. 2018. Blocking counter-speech. New work on speech acts, 144:156.
Jonathan Leader Maynard and Susan Benesch. 2016. Dangerous speech and dangerous ideology: An inte- grated model for monitoring and prevention. Geno- cide Studies and Prevention, 9(3).
Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and

---

Samuel R. Bowman. 2020. jiant: software toolkit for research on general-purpose purpose text understanding models. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics: System Demonstrations, pages 109-117, Online. As- sociation for Computational Linguisties.
Human Language Technologies, pages 2289-2303, Online. Association for Computational Linguistics.
Bertie Vidgen and Taha Yasseri. 2020. Detecting weak and strong islamophobic hate speech on social me- dia. Journal of Information Technology & Politics, 17(1):66-78.
Jing Qian, Anna Bethke, Yinyin Liu,, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Empirical Proceedings of the 2019 Conference
Henning Wachsmuth, Shahbaz Syed, and Benno Stein, 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 241-251, Melbourne, Australia. Association for Computational Linguistics.
Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 4755- 4764, Hong Kong, China. Association for Computa- tional Linguistics.
Douglas Walton, 2005. Fundamentals of critical argu- mentation. Cambridge University Press.
Robert Richards and Clay Calvert, 2000. Counter- speech 2000: A new look at the old remedy for bad speech. BYU L. Rev., page 553.
Douglas Walton. 2009. Ohjections, rebuttals and refuta- tions. In &th international conference of the Ontario Society for the Study of Argumentation (OSSA), 3-6 June 2009 Windsor, ON, Canada, pages 1-10. On- tario Society for the study of Argumentation (OSSA).
Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. SemEval-20177 task 4: Sentiment analysis Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemžEval-2) 2017), pages 502- 518, Vancouver, Canada. Association for Computa- tional Linguistics.
Kenneth D Ward. 1997. Free speech and the devel- opment of liberal virtues: An examination of the controversies involving flag-burning and hate speech. U. Miami L. Rev, 52:733.
Carla Schieb and Mike Preuss. 2016. Governing hate speech by means of counterspeech facebook. In 66th ica annual conference. pages 1-23, Fukuoka, Japan.
Zeerak Waseem, Thomas Davidson, Dana Warmsley. and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Lan- guage Online, pages 78-84, Vancouver, BC, Canada. Association for Computational Linguistics.
Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language pro- cessing. Proceedings of the Fifth International Workshop on Natural Language Processing for So- cial Media, pages 1-10, Valencia, Spain. Association for Computational Linguistics.
Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the Student Research Workshop, SRW@ HLT-NAACL 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, San Diego California, USA, June 12-17, 2016, pages 88 88-93. The Associa- tion for Computational Linguistics.
Eyal Shnarch, Carlos Alzate, Lena Dankin, Mar- tin Gleize, Yufang Hou, Leshem Choshen, Ranit Aharonov, and Noam Slonim. 2018 Will it blend? blending weak and strong labeled data in a neural network for argumentation mining. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volme 2: Short Papers). pages 399-605. Melbourne, Australia. Association for Computational Linguistics.
Jason Wei and Kai Zou. 2019. EDA: Easy data augmen- tation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNLP),, pages 6382-6388, Hong Kong, China. As- sociation for Computational Linguistics.
Tiberiu Sosea and Comnlla Caragea. 2022. Ensynet: A dataset for encouragement and sympathy detection. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5444-5449.
Christopher W Tindale. 2007. Fallacies and argument appraisal. Cambridge University Press.
Katrin Weller and Katharina E Kinder-Kurlanda. 2016. A manifesto for data sharing in social media research. In Proceedings of the 8th ACM Conference on Web Science, pages 172-
S. Toulmin, R.D. Rieke, and A. Janik. 1979. An Intro- duction to Reasoning. Macmillan.
Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, and Rebekah Tromble. 2021. Introducing CAD: the contextual abuse dataset. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics:
Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics:

---

Human Language Technologies, Volume (Long and Short Papers), pages 602-608, Minneapolis, Min- nesota. Association for Computational Linguistics.
includes hateful comments and their direct replies with annotations indicating the fine-grained cate- gory of the replies.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- rie Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz. Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao. Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
To identify hateful comments, we first select 35 subreddits considered to be hateful and scrape whole conversation threads from these subreddits, As the prevalence of online hate in the wild is very low, we further build pretrained language models to automatically identify candidate uncivil comments. A set of 5,469 candidate uncivil comments is then collected based on our previous efforts. Finally we assign all the candidate uncivil comments to an- notators for validation. Inter-annotator agreement (Krippendorff's s a)) is 0.65, indicating substantial agreement; over 0.8 would be nearly perfect. We collect 1,065 candidates comments that were la- beled as hateful.
Xinchen Yu, Eduardo Blanco, and Lingzi Hong. 2022. Hate speech and counter speech detection: Conver- sational context does matter. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies. pages 5918-5930, Seat- tle. United States. Association for Computational Linguistics.
After identifying hateful comments, we collect direct replies to them and manually annotate their types in the taxonomy we work with. Three anno- tators participate in the annotations. All the 3,654 replies to hateful comments were annotated by at least one annotator. We set a total of four phases. In each phase, we randomly select 150 replies to be annotated by three annotators. If inter-annotator agreement is above 0.60, which is substantial agree- ment, then each annotator works on an indepen- dent partition of the replies. The final version of the rulcs used to scrape comments, identify hate- ful comments and annotate replies to hateful com- ments are detailed in Section 4.
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies. Volume I (Long and Short Papers), pages 1415-148 1420, Minneapolis, Minnesota. Association for Computational Linguisties.
A Subreddit List
We provide here the list of subreddits we have selected: r/antiwork, r/antheism, r/bakchodi, rbbindingffisaac, r/changemyview, r/conspiracy, /DankMemes, r/Drama, r /feaar/eeeeeeeeeeannggttttteee //Fminismm r/GenZedongg h/t.aaaaaaaaaaaaaaaaaaCaaaaaaaa rjjusmecckbeardthinggs r/No r/KotakulnAction, Vlmnoong Tohes, r/MensRights, r/Sino, //etaCanada, r/modernwarfare, r/Seduction, r/playrust, r/PurplePillDeebate, r/PussyPass, r/ShitPolitiesSays. r/4Chan, rShindedditSSays. r/worldnews, r/SubredditDram TTrucReddit, r/DotA2, r/TumbirnAAction r/ //ToooChromssomsss //BlackPeopleTTiine
B.2 Language Variety
The data collection process was carried out from April to July 2022. In Reddit, more than 95% of comments are written in English. We also use SpaCy to make sure that each reply is in English. Information on the specific type of English not available.
B.3 Speaker Demographic
I/
The 3,654 replies along with the hateful comments are posted by Reddit users. We do not require comments to come from verified accounts. As per Reddit age restrictions, the minimum user age is 13 years for both authors of hateful comments and replies. Speakers are not reachable and thus demo- graphic information about the speakers is limited.
B Data Statement
As per the recommendations by Bender and Fried- man (2018), we provide a data statement to better understand the new data presented in this paper.
B.1 Curation Rationale
B.4 Annotator Demographic
We create a new dataset to interpret replics to hate speech in user- ever-generated web content. The dataset
Three annotators are part of the annotation process and development of annotation guidelines. All of

---

them are women and their ages range from 18 to 35 years old. Ethnic backgrounds are as follows: are from Asian and from North America. All the annotators are highly proficient English. Socioeconomic backgrounds are as follows: all an- notators reported that they are in the middle class. Educational background are as follows: under- graduate, 1 graduate student, and I with a doctoral degree. 2 of them work in NLP-related research areas, and 1 majors in computer science.
All crowd workers are self-reported to be over 18 years old. They need to pass Adult Content Qualification in order to work on our tasks. The other demographic information is limited.
B.5 Speech Situation
Text in our corpus is retrieved from Reddit between April to July of 2022. Modality of text is written by Reddit. Reddit allows users to edit their comments, and
the version of the comments available of July 2022. The interactions are asyn- chronous and replies and hateful comments cannot appear in Reddit simultaneously. The intended au- dience could be any user on the internet.
C Training Details
Our dataset pre-processed by removing URLs. removing symbols, removing any additional spaces, and at the end, converting all words to lower-case. The neural model takes about half an hour on aver- age to train on single NVIDIA TITAN Xp.
We use the implementation by Pruksachatkun et al. (2020) and fine-tune the RoBERTa (base ar- chitecture; 12 layers) (Liu et al., 2019) model for each of the three training settings. For each setting, we set the hyperparameters be the same when the input the hateful comment, the reply, or both (Table 9).

---

Epochs
Batch size
Learning rate
Dropout
Patience
reply
le-5
0.5
pretrain
0.5
blend
0.5
Table 9: Hyperparameters used to fine-tune RoBERTa individually for each training setting. We accept default settings for the other hyperparameters as defined in the implementation Pruksachatkun al. (2020).


Reviews:
[{"review_id": "s2zmY8XkW0", "reviewer": "Reviewer_v5qd", "summary": "", "questions": "1. With the Section 6 Experiments, it is clearly hard to model or classify these categories. How do you propose to improve on these models besides naive finetuning or augmentation with EDA? Do you see a way for better sampling techniques and how it would perform cross domains?  ", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reasons to Reject**\n\nRE: They do not compare their formalism with existing proposed formalisms like ([1]. [2]) and datasets like CONAN.?\n* Thank you for the references. We found them relevant and will add them to the related work. \n* In the domain of counterspeech, CONAN and the work by Mathew et al., (2019) mainly built on or simply followed the taxonomies by Benesch et al., (2016) with slight modifications. We conducted detailed analyses of these taxonomies to ensure the new taxonomy covers all different types of counterspeech. There are overlaps in some categories (i.e., line 193, 208, 248, 258), however, the previous taxonomies fail to convey some important information. Most notably:\n    * the intensity of hostile tones does vary: intelligence insults [Insult] are more uncivil than asking someone to leave the discussion [Accusation]. \n    * the strategies of positive tones can be different: kindly provide suggestions [Request] and show sympathy [Sympathy].\n\n* Our work has two salient distinctions: (1) we indicated whether the counterspeech addresses the author of hate speech or the content (i.e., in Table 2) and (2) we formalized the taxonomy based on argumentation theories (i.e., line 165-172). These yield several advantages: \n    * In counterspeech generation, Counter-content is generated by focusing on the content of hate speech, while Counter-author may have greater flexibility.\n    * In the evaluation, the relevance of the content in counterspeech to hate speech is a crucial factor for quality evaluation. But this factor may not be necessarily required for counterspeech.\n    * For implications, Counter-content might provide guidance on what and why the content is inappropriate. While Counter-author is usually not that informative.\n\n* Thank you for the suggestion, we will expand Table 2 and the discussion in Section 2 in the extra page if the paper is accepted.\n\n**Questions for Authors**\n\nRE: How do you propose to improve on these models besides naive finetuning or augmentation with EDA? Do you see a way for better sampling techniques and how it would perform cross domains?\n* These are insightful suggestions. We will explore different methods in our future work. Specifically, we are very interested in experimenting with different Parameter-Efficient Fine-Tuning methods, for example,  prompt tuning and prefix tuning. We are also very curious about the performance of the latest LLMs (Llama 2, T5, etc.) on our dataset. \n* For cross-domain performance, we will acknowledge in our limitations that replies on Reddit may not be representative of all platforms. Nevertheless, the methodology we employed in establishing the taxonomy can be extrapolated to other datasets.\n\n=====\nWe refer the reader to Appendix C for the information necessary to replicate the results. We will also make our dataset and code publicly available on GitHub.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The annotated data collected based on this taxonomy can help with designing better counterargument frameworks to help promote safe conversations and content moderation operators. Further, the authors provided some analysis of the conversational outcomes based on the countering strategy (author, content, or non-counter). They provide a preliminary attempt at automatic category classification and counterspeech sub-type classification, with scope for potential improvements in future work. \n", "reasons_to_reject": "The authors present an annotated dataset based on a proposed framework and present an analysis of this dataset. However, they do not compare their formalism with existing proposed formalisms like ([1]. [2]) and datasets like CONAN. Further, any human study of how the formalism presents itself in a controlled study could be interesting vs in the wild like Reddit. \n\n[1] Allaway, E., Taneja, N., Leslie, S.J. and Sap, M., 2023. Towards countering essentialism through social bias reasoning. arXiv preprint arXiv:2303.16173.\n[2] Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language.\n", "questions_to_authors": "1. With the Section 6 Experiments, it is clearly hard to model or classify these categories. How do you propose to improve on these models besides naive finetuning or augmentation with EDA? Do you see a way for better sampling techniques and how it would perform cross domains?  ", "paper_topic_and_main_contributions": "The authors introduce a taxonomy of counterspeech replies to hate speech with a classification on countering personal attacks from the author to countering the content of the hate speech reply or refraining from countering, grounded in social psychology. They annotate a decent sample of Reddit comments based on this taxonomy and analyze the types of replies used by humans in the wild. \n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Allaway, E., Taneja, N., Leslie, S.J. and Sap, M., 2023. Towards countering essentialism through social bias reasoning. arXiv preprint arXiv:2303.16173.\n[2] Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language.\n"}}, {"review_id": "HbQJklS7LQ", "reviewer": "Reviewer_LEi4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reasons to Reject**\n\nRE: It would give the study depth if a larger (noisy) dataset were also created for experiments of weak supervision, to further investigate the value of the manually annotated data.\n* This is an excellent suggestion that we have thought of earlier. We would be interested to explore this method in our future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tWell studied theoretical work to propose taxonomy for Counterspeech, being the first dataset to propose a fine-grained categorization of counterspeech including the target of counterspeech.\n2.\tManual annotation efforts to create dataset of 3654 samples.\n3.\tThis work studies overlooked conversational patterns in the form of counterspeech.\n", "reasons_to_reject": "1. It would give the study depth if a larger (noisy) dataset were also created for experiments of weak supervision, to further investigate the value of the manually annotated data.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper studies “countering” as a form of replies to initial hatespeech content. “Counters” can be of various forms – counter arguments, insults, etc,. \nThis work categorizes the types of “counters” into a taxonomy, and further investigates the language usage, and if the final outcome of the conversation remains hateful.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "x8dKcjNlQD", "reviewer": "Reviewer_hSjz", "summary": "", "questions": "1. Does Non-counter Agree / Non-counter Neutral include supporting victims?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reasons to Reject**\n\nRE: The paper lacks discussion around existing taxonomies for categorising counterspeech: how existing taxonomy compares with the one proposed, and why a new taxonomy is needed (is it really needed?).\n* Regarding counterspeech, CONAN and the work by Mathew et al., (2019) mainly built on or simply followed the taxonomies by Benesch et al., (2016) with slight modifications. We conducted detailed analyses of previous taxonomies on counterspeech to ensure the new taxonomy covers all different types of counterspeech. There are overlaps in some categories (i.e., line 193, 208, 248, 258), however, the previous taxonomies fail to convey some important information. Most notably:\n    * the intensity of hostile tones does vary: intelligence insults [Insult] are more uncivil than asking someone to leave the discussion [Accusation]. \n    * the strategies of positive tones can be different: kindly provide suggestions [Request] and show sympathy [Sympathy].\n* Our work has two salient distinctions: (1) we indicated whether the counterspeech addresses the author of hate speech or the content (i.e., in Table 2) and (2) we formalized the taxonomy based on argumentation theories (i.e., line 165-172). These yield several advantages: \n    * In counterspeech generation, Counter-content is generated by focusing on the content of hate speech, while Counter-author may have greater flexibility.\n    * In the evaluation, the relevance of the content in counterspeech to hate speech is a crucial factor for quality evaluation. But this factor may not be necessarily required for counterspeech.\n    * For implications, Counter-content might provide guidance on what and why the content is inappropriate. While Counter-author is usually not that informative.\n* Thank you for the excellent suggestion, we will revise accordingly to make the distinctions more straightforward and clearer (Table 2 and Section 2).\n\n\n**Questions for the Authors**\n\nRE: Does Non-counter Agree / Non-counter Neutral include supporting victims? \n* Supporting or saying good words for victims would belong to Sympathy (line 251-260).\n\n\n**Missing References**\n\nRE: In line 125 ( \"...\" is generic and rarely used by real users, but common in synthetic counterspeech), references or statistics are needed.\n* Thank you! We will add the following in Section 2: We observe “[...] are inappropriate” and “should be avoided” made up of 5.6% of the replies in the synthetic counterspeech dataset (Qian et al., 2019). While in the comments by real users, we found no one actually uses these kinds of language.\n\n**Typos Grammar Style And Presentation Improvements**\n\nRE: For the statement in the third paragraph of the introduction, I reckon this connection/assumption is a bit too strong and would require large-scale analysis or relevant evidence to back it up. [...] It may be better to include relevant references or soften the statement.\n* Thank you for the suggestion. We will tone down the statement.\n\nRE: The sentence in lines 355-357 can be deleted as it is repeated in lines 341-343.\n* Thank you for the suggestion. We will delete the content accordingly.\n\nRE: Line 542: in our corpus is low\n* Thank you. We will fix this typo and proofread our paper carefully.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. A taxonomy of replies to hate speech based on argumentation theories which can encourage studies on projecting persuasion to the design of counterspeech that can be potentially effective in hate mitigation. However, I think this taxonomy may be more applicable to specific platforms like Reddit or forums compared to Twitter in which not much argumentative discussion is present in hate speech domain.\n2. An analysis that looks into the dynamics of hate speech and replies", "reasons_to_reject": "The paper lacks discussion around existing taxonomies for categorising counterspeech: how existing taxonomy compares with the one proposed, and why a new taxonomy is needed (is it really needed?). What types of benefits does this new taxonomy bring and what limitations exist in the existing taxonomies? For instance, several categories in the proposed taxonomy overlap with existing taxonomy like Benesh's [1], which are generally expressed in different/argumentative terms (e.g., claims vs facts/hypocrisy; accusation vs denouncing). For this reason, it is not clear why and how this new taxonomy can add knowledge to what we already knew about counterspeech.\n\n[1] Benesch, S., Ruths, D., Dillon, K. P., Saleem, H. M., and Wright, L. (2016). Counterspeech on Twitter: A field study. Dangerous Speech Project.\n", "questions_to_authors": "1. Does Non-counter Agree / Non-counter Neutral include supporting victims?", "paper_topic_and_main_contributions": "This paper proposes a fine-grained taxonomy of replies to hate speech that addresses hateful content or its author. This taxonomy is applied to a Reddit dataset the authors collected. It also provides corpora and linguistic analyses of the interaction between hateful content and its replies, giving insights into what replies can elicit civil/uncivil outcomes.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "1. In line 125 ( \"...\" is generic and rarely used by real users, but common in synthetic counterspeech), references or statistics are needed."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. For the statement in the third paragraph of the introduction, I reckon this connection/assumption is a bit too strong and would require large-scale analysis or relevant evidence to back it up. There can be other reasons why addressing hate speakers received further hateful comments and why addressing hateful content did not, e.g., users did not see it, replies have been removed etc. It may be better to include relevant references or soften the statement.\n2. The sentence in lines 355-357 can be deleted as it is repeated in lines 341-343.\n3. Line 542: in our corpus *is* low \n"}}]

Author rebuttal:

Meta-review: Reviewers generally found that this was an interesting and fairly sound paper: The topic of focus is important, and the proposed counterspeech taxonomy is a useful addition that other researchers in this area could adopt/build on. 

The main concern the reviewers had was with the lack of comparison to other frameworks for counterspeech. I think that addressing this concern, as the authors have started to do in their responses, is important, because it would better situate their work in this area. The meta-reviewer’s opinion is that the choice to work from an argument-mining perspective is super interesting and compelling; comparisons to other taxonomies would have the additional benefit of clarify why that’s the case. (Per one of the reviewers: how does the new taxonomy “add knowledge to what we already knew?”) I suggest that the authors heed our suggestions to expand their discussion on this point (alongside other reviewers’ comments).
Decision: Accept-Main