CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network
Authors: Sreyan Ghosh
Manan Suri
Purva Chiniya
Utkarsh Tyagi
Sonal Kumar
Dinesh Manocha
Venue: EMNLP 2023 Main (2023)
Keywords: hate-speech
hyperbolic
social-good
implicit
TL;DR: The paper proposes a novel architecture to detect implicit hate speech in online conversations

Abstract: The tremendous growth of social media users interacting in online conversations has led to significant growth in hate speech affecting people from various demographics. Most of the prior works focus on detecting explicit hate speech, which is overt and leverages hateful phrases, with very little work focusing on detecting hate speech that is implicit or denotes hatred through indirect or coded language. In this paper, we present CoSyn, a context synergized neural network that explicitly incorporates user- and conversational-context for detecting implicit hate speech in online conversations. CoSyn introduces novel ways to encode these external contexts and employs a novel context interaction mechanism that clearly captures the interplay between them, making independent assessments of the amounts of information to be retrieved from these noisy contexts. Additionally, it carries out all these operations in the hyperbolic space to account for the scale-free dynamics of social media. We demonstrate the effectiveness of CoSyn on 6 hate speech datasets and show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%. We make our code available.

CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network
Sreyan Ghosh I*
Manan Sur Purva Chiniya Utkarsh Tyagi'*
Sonal Kumar Dinesh Manocha'
University of Maryland College Park, USA, MIDAS Labs, IITT-Delhi, India (sreyang, pchiniya, utkarsht, sonalkum, JmanochaJ@und edu, manansuri270ggaii. com
Abstract
User 1
PLOC, some bellevers have been going to cow shelters once week cover in cow cling uniee it Immunity against, help recover from, COVID-19, PNMME 2e0 mi 23/ae, 2013
The tremendous growth of social media users interacting in online conversations has led to sigmificant growth in hate speech affecting peo- ple from various demographics. Most prior works focus detecting explicit hate speech, which is overt and leverages hateful phrases, with very little work focusing on detecting hate speech that is implicit or de- notes hatred through indirect or coded language. In this paper, present CoSyn, context synergized neural network that explicitly incor- porates user- and conversational -oontext de- tecting implicit hate speech in online conversa- tions. CoSyn introduces novel ways to encode these external contexts and employs novel context interaction mechanism that clearly cap- tures the interplay between them, making inde- pendent assessments of the amounts of informa- tion to be retrieved from these noisy contexts. Additionally, carries out all these operations in the hyperbolic space account for the scale- free dynamics of social media. We demonstrate the effectiveness of CoSyn on 6 hate speech datasets and show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% 57.8%. We make our code available
@ueer] #1OC working hand produce more standupp comeedinss
zone en--25
Hatefu
Pluser2 They already produce comedians...
@ @ @
Non-hatelul
Figure I: Illustration of social media conversation tree with implicit hate speech. User posts a factual statement about practices people follow location. In response, User 2 implies hate through sarcastic statement, to which User 3 elaborates with positive stance. Clearly, these utterances even difficult for humans to classify as hate or not without proper conversational context.
certain
implicit hate speech (Jurgens al., 2019). Fig. il- lustrates Twitter conversation where users convey hate implicitly through sarcasm and conversational context.
1 Introduction
Societal Problem and Impact. Though a consid- erable amount of research has been done de- tecting explicit hate speech (Schmidt and Wiegand, 2019), detecting implicit hate speech is a greatly understudied problem in the literature, despite the social importance and relevance of the task. In the past, extremist groups have used coded language to assemble groups for acts of aggression (Gubler and Kalmoe, 2015) and domestic terrorism (Piazza, 2020) while maintaining deniability for their tions (Dénigot and Burnett, 2020). Because lacks clear lexical signals, implicit hate utterances evade keyword- based detection systems (Wiegand et al., 2019). 1), and even the most advanced neural architec- tures may not effective in detecting such utter-
Hate speech defined as the act of making utter- ances that can potentially offend, insult, or threaten a person or a community based on their caste, re- ligion, sexual orientation, or gender (Schmidt and Wiegand, 2019). For social media companies like Twitter, Facebook, and Reddit, appropriately deal- ing with hate speech has been a crucial challenge (Tiku and Newton, 2015). Hate speech can take the form of overt abuse, also known as explicit hate speech (Schmidt and Wiegand, 2017), or can be ut- tered in coded or indirect language, also known as
'ntts://gittubbcom//eee//////// Sreyan8SSS//onnn
"These authors contributed equally to this work.

---

ances (Caselli al., 2020).
Main Contributions. In this paper, we propose CoSyn, a novel neural network architecture for de- tecting implicit hate speech that effectively incorpo- rates external contexts like conversational and user. Our primary aim is to classify whether a target ut- terance (text only) implies hate or not, including the ones that signal hate implicitly, CoSyn jointly models the user's personal context (historical and social) and the conversational dialogue context in conversation trees. CoSyn has four main compo- nents: (1) To encode text utterances, we train a transformer sentence encoder and promote it to learn iavs-invrriant representations. This helps us to handle keyword bias, long-standing problem in hate speech classification (Garg et al., 2022). (2) We start by modeling the user's personal historical context using a novel Hyperbolic Fourier Atten- tion Network (HFANN. HIAN models diverse and scale-free user engagement of a user on social me- dia by leveraging Discrete Fourier Transform (Coo- ley and Tukey, 1965) and hyperbolic attention on past user utterances. (3) We next model the user's personal social context using a Hyperbolic Graph Convolutional Network (HGCN) (Chami et al., 2019). HGCN models the scale-free dynamics of social networks using hyperbolic learning, leverag- ing the social connections between users, which act as edges in the graph. (4) Finally, to jointly model a user's 's personal context and the conversational dialogue context, we propose a novel Context Syn- ergized Hyperbolic Tree-LSTM (CSHT), CSHT effectively models the scale-free nature of conver- sation trees and clearly captures the interaction between these context representations in the hy- perbolic learning framework. We describe all our components in detail in Section 2.7. To summarize, our main contributions are as follows:
Current Challenges. Current state-of-the-art hate speech detection systems fail to effectively detect implicit and subtle hate (Ocampo et al., 2023). De- tecting implicit hate speech is difficult for multi- ple reasons: (1) Linguistic nuance and diversity: Implicit hate can be conveyed through sarcasm, humor (Waseem and Hovy, 2016), euphemisms (Magu and Luo, 2018), circumlocution (Gao and Huang, 2017), and other symbolic or metaphorical languages (Qian et al., 2018). (2) Varying context: Implicit hate can be conveyed through everything from dehumanizing comparisons (Leader Maynard and Benesch, 2016), and stereotypes (Warner and Hirschberg. 2012) to threats, intimidation, and in- citement to violence (Sanguinetti et al., 2018; For- tuna and Nunes, 2018). (3) Lack of sufficient lin- guistic signals: Unlike parent posts, which con- tain sufficient linguistic cues through background knowledge provided by the user, replies or com- ments to the parent post are mostly short and context-less reactions to the parent post. These factors make implicit hate speech difficult to detect and emphasize the need for better learning systems.
Why prior work is insufficient. (EISherief et al., 2021) define implicit hate speech as "coded or in- direct language that disparages person or group." They also propose the first dataset, Latent Hatred, benchmark model performance on implicit hate speech classification and show that existing state- of-the-art classifiers fail to perform well on the benchmark. Though Latent Hatred builds on an ex- haustive 6-class taxonomy, it ignores implicit hate speech that is though it accounts for majority of implicit hate speech online (Modha et al., 2022; Hebert et al.. 2022). Lin (2022) builds on Latent Hatred and pro- pose one of the first systems to classify implicit hate speech leveraging world knowledge through knowledge graphs (KGs). However, beyond the fact that their system is restricted to only English due to the unavailability of such KGs in other lan- guages, their system also fails to capture any kind of external context, which is vital for effective hate speech detection (Sheth et al., 2022). Thus, first extend the definition of implicit hate speech include utterances that convey hate only in the context of the conversational dialogue (example in 1).. Next, we propose a novel neural learning system to solve this problem.
even
We introduce CoSyn, the first neural network architecture specifically built to detect implicit hate speech in online conversations. CoSyn leverages the strengths of existing research and introduces novel modules to explicitly take into account user and conversational con- text integral to detecting implicit hate speech.
Through extensive experimentation, we show that CoSyn outperforms all our baselines quantitatively on hate speech datasets with absolute improvements of 1.24% 57.8%.
We also perform extensive ablative experi- ments and qualitative comparisons to prove

---

the efficacy of CoSyn.
expx TT HD and the logarithmic map log, 1H + TH are used to project Euclidean vectors to the hyperbolic space and vice versa re- spectively.
2 Methodology
Fig. provides clear pictorial representation of our proposed model architecture (algorithm shown in Algorithm 1). We provide an overview describ- ing various operations in Hyperbolic Geometry in Appendix 2.2; and we request our readers to re- fer to that for more details. In this section, we describe the three constituent components of our proposed CoSyn, which model different aspects of context, namely, the author's personal histori- cal context, the author's personal social context, and both the personal historical and social contexts with the conversational context.
expx(1) = x +
(tanh
Ax
|||||
e |1l|
(1)
2
x # y | ① vl
log, (0) = A
tanh-1(1l-an rByll)
(2)
where 2, y E HD, Ta. Further, perform operations in the Hyperbolic space, the
following basic hyperbolic operations are described below:
2.1 Background, Notations, and Problem Formulation.
Mobius Addition adds pair of points T, y HP as,
Let's suppose we have N distinct conversation trees, where each tree is denoted by Tn E T = (To...., .n..... .... TT]]. Each tree Tn has JTn individual utterances denoted by Tn "t""....., 1T is authored by one of the L users in the dataset denoted by mary aim of CoSyn is to classify each utterance t into its label yoh E {0,1} where and I indicate whether the utterance is hateful or not. Addition- ally, each hateful utterance (y,² 1) is labeled with d C [0, 1) where 0 and indicate whether the hateful utterance explicitly or implicitly hate- ful. " is used only for analysis. In the following subsections, "user" refers to the author of an utter- ance in a conversation tree to be assessed.
x #Qy y:y: = (1+ 2(x, y) + 13 x x+ (1 |->1))
Th ....,................... )..).. :.))) Each individual utterance
+ (z, v) 10 11
{uo. ,u,, "", ".....,,, u..) The pri-
Möbius Multiplication @ multiplies vectors x E HP and W D'DD given by,
W = exp, W logo
(4)
Möbius Element-wise Multiplication @ performs element-wise multiplication on x, y H.P
y: tanh | tanh -aah--((a ((lvll
|ry|
(5)
|y]
2.2 Hyperbolic Geometry: Background
Hyperbolic Geometry. A Riemannian manifold is D-dimensional real and smooth manifold defined with an inner product on tangent space INr T x T. IR each point æ M, where the tangent space TaM is D-dimensional vector space representing the first-order local ap- proximation of the manifold around a given point A Poincaré ball manifold is hyperbolic space defined constant negative curvature Rieman- nian manifold, denoted by (HP.0 .g) where manifold HD_ {x E RD_ | < 1} and the Riemannian metric is given by 9x >2.g where gF I de- notes the identity matrix, i.e., the Euclidean metric tensor and Az = I- -|| To perform opera- tions the hyperbolic space, the exponential map
Hyperbolic Distance between points æ. y C D given by:
ds = 2 tanh -taann-((1 -1 $ yll)
(6)
Fourier Transform. The Fourier transform breaks down a signal into its individual frequency com- ponents. When applied to a sequence with E [O, - 1], the Discrete Fourier Transform (DFT) generates new representation X₁ for each value of where complishes this by computing a sum of the origi- nal input tokens nn, multiplied by twiddle factors [e 2mrkn/N JN,, where n is the index of each token in the sequence. Thus, DFT is expressed as:
< N 1. DFT

---

N-1 X₁
Σ Tne 2rikn/NN
0 OSksN-1 (7)
n=0
(a) Frequency distri- bution for the social graph.
(b) Frequency distri- bution for conversa- tion trees.
2.3 Bias-invariant Encoder Training
CoSyn, like other works in literature, builds on the assumption that linguistic signals from text utter- ances can effectively enable the detection of hate speech in online conversations. Thus, our primary aim is to learn encoder e(-) that can effectively generate vector representations R. for text utter- ance where is the dimension of the vector. Specif- ically, we fine-tune a SentenceBERT (Reimers and Gurevych, 2019) and solve an additional loss pro- posed in (Mathew et al., 2021) using self-attention maps and ground- -tuth hate spans. Keyword bias is a cong-standing problem in hate speech classi- fication, and solving the extra objective promotes learning bias-invariant sentence representations.
Graph
Convertation
Nean Degres
1822
Node Dagroo
2066 00
2,80 20
P(x)
(c) Properties of the social graph and conversation trees.
Figure 2: Various properties of the social graph and conversation trees averaged across datasets. The fit- ting of the node distribution in the power law ( e [2, 3)) (Choromański al., 2013) and low hyperbolicity (Barabási and Bonabeau, 2003) 5 indicates the scale- free nature of conversations and social graphs.
Fourier Attention Network (HFAN) followed by modeling the user's social context by passing Whist through Hyperbolic Graph Convolution Network (HGCN). To be precise, learning personal user con- text takes the form of el (H sarrricallttterrnaat + HFAN - Uhist - HGCN + Uu. We next de- scribe, in detail, HFAN and HGCN.
2.4 Modeling Personal User Context
Modeling personal user context, in the form of au- thor profiling, for hate speech classification has shown great success in the past because hateful users (users prone to making hate utterances) share common stereotypes and form communities around them. They exhibit strong degrees of homophily and have high reciprocity values (Mathew et al., 2019). We hypothesize that this will prove to be especially useful in classifying conversational im- plicit hate speech, which on its own lacks clear lexical signals or any form of background knowl- edge. Thus, our primary aim is to learn a vector representation U4, E Rd for the user u who has authored the utterance i to be assessed, where d is the dimension of the vector. For our work, we also explore the importance of the personal historical context of user enable better author profiling. Studies show that past social engagement and lin- guistic styles of their utterances on social media platforms play an important role assessing the user's ideology and emotions (Xiao et al., 2020). Thus we propose novel methodology for author profiling that is more intuitive, explainable, and effective for our task. Our primary aim is to model the user's (author of the utterance to be assessed) personal context and generate user representations Uy which can then be used for hate speech classi- fication. To achieve this, we first encode a user's historical utterances hist using our Hyperbolic
2.5 Hyperbolic Fourier Attention Network
User engagement on social media often diverse and possesses scale-free properties (Sigerson and Cheng, 2018). Thus to account for the natural irregularities and effectively model a user's per- sonal historical context, we propose a novel Hyper- bolic Fourier Attention Network (HFAN). The first step is to encode historical utterances, H" using our encoder c(-), made by the user u, denoted by Hu {hi h2g h Next, apply 2D DFT to the embeddings (ID DFT along the tempo- ral dimension and ID DFT along the embedding dimension) to capture the most commonly occur- ring frequencies (ideologies, opinions, emotions, etc.) in the historical utterances made by the user using:
Uf 4fforrier = =xp (Fa (logāi (e(HH)))))
(8)
where F is the DFT operation. The 2D DFT operation helps highlight the most prominent fre- quencies, which signifies holistic understanding of the user's sociological behavior. Next, we hy- pothesize that the frequencies most relevant to a conversation can act as the most important clues to

---

HEAN
HGCK
CSET
Pomncae ball tranaa Comnsaormatiin
Parent
2D Founier Transform
Flyeetoolic Attention
Hyper Activation
o
Dis 2001 Losicr
Hypeebolis GCN
Tux howran Paoddr
O
Hiserrcal
Reply
Panne
Ealoppech Abour
Ferura
Social Gogg
Rinary callagy
Ponncae Ball Transformation
Hyparbulic Activation
Context Seneext-yyeegieeed Hypurbolic Tree LSTM cell
Figure 3: Illustration of CoSyn. CoSyn detects if target utterance, authored by a social media user, implies hate or not leveraging three main components: (1) HFAN: This models the user's personal historical context by employing Fourier Transform and Hyperholic Attention on the user's historical tweets. (2) HGCN: This models the user's social context through hyperbolic graph learning on its relations with other users. (3) CSHT: This jointly models the user's personal context and the conversational context to finally classify if the utterance is hateful.
assessing how user would react to other utterances in the conversation tree (for c.g., a user's stance on the contribution of particular political party to recent riot in discussion). Additionally, these fre- quencies may change over time. Thus, to account for the latter factor first, pass the embeddings through Hyperbolic GRU (Zhang and Gao, 202 1), which first projects these embeddings from the Eu- clidean the hyperbolic space using a Poincaré ball manifold and then effectively captures the se- quential temporal context on time-aligned historic user utterances. The Hyperbolic GRU is a modi- fied version of the conventional GRU, which per- forms Mobius operations on the Poincaré model (addition, multiplication, and bias). We request readers refer to (Zhang and Gao, 2021) for more details. Next, to account for the former factor, we perform Hyperbolic Attention (Zhang and Gao, 2021), which first linearly projects the utterance embeddings within Poincaré space and then con- structs the final user embedding Whiss via Einstein midpoint:
and Gao, 2021). C.c the sentence centroid, which is randomly initialized and learnable.
2.6 Hyperbolic Graph Convolutional Network
After obtaining the user representations 2hhist for every user in the dataset, we model social rela- tionships between the users Each edge {u,, Uz) E & represents one of the four types of relations: 1) User u, retweets tweet posted by uj. 2) User u, mentions user tj in tweet 4. 3) User u, replies post by user uj or 4) User n, follows user uj. Each vertex e E V rep- resents the user representations Uy HGCN (Chami et al., 2019) modifies the conventional GCN and performs neighbor aggregation using graph convo- lutions in the hyperbolic space to enrich user's historical context representations learned through HFAN using social context. Social network nections between users on a platform often possess hierarchical and scale-free structural properties (de- gree distribution of nodes follows the power law as seen in Fig and decreases exponentially with a few nodes having a large number of connections (Barabási and Bonabeau, 2003)). To model such complex hierarchical representations HGCN per- forms all operations in the Poincaré space. We first project our representations li the hyperbolic space using a Poincaré ball manifold (exp* (.)) with a sectional curvature 1/K. Formally, the fea- ture aggregation based at the 7th HGCN layer is denoted by:
graph G (V, E).
a; = exp Bpddc (chc, hgc)) cn)
ai7 (hi:)) )
Uhist Σ
h
(10)
Ei ary (hz))
where h&c and h are the utterance encoding of utterance h2 obtained from fourier projected from the Poincaré space into the Lorentz space and Klein space for computational convenience (Zhang

---

as the user embeddings of the author of the post, denoted as uu to define computational gates oper- ating the hyperbolic space for CSHT cells. As defined in Subsection2.2 O O & represent Möbius Dot Product, Möhius Addition and Möbius matrix multiplication, respectively.
0 = 00 (FM (Aol-I)w(i)) (11)
where - and - 1/K, the hyperbolic cur- and i, respectively. À D-1/2AD-1/2 is the degree normalized adjacency matrix, W is trainable network parameter, O the output of the 1th layer and FM is the Frechet Mean operation. σ. - is the hyperbolic non- linear activation allowing varying curvature each layer.
vatures at layers
Considering multiple input nodes tk, (fx, it) E E, we implement multiple hyperbolic forget gates incorporating the outputs h2 of the input nodes with the current node's combined user and post representation rj.
w((x) X2 C W Mu:
rj
2.7 Context-Synergized Hyperbolic Tree-LSTM
fyk: expo (a (log,, (r, ① U & he 1 Db(r)
(12)
To model the conversational context in con- versation trees effectively, we propose Context- Synergized Hyperbolic Tree-LSTM (CSHT). CSHT presents several modifications and improve- ments over Tree-LSTM (Tai al., 2015), includ- ing (1) incorporating both the user's personal con- text and the conversation context while clearly cap- turing the interactions between them, and (2) op- erating in the hyperbolic space, unlike the origi- nal TSe,LSTM, which operates in the Euclidean space. Conversation trees social media pos- sess hierarchical structure of message propaga- tion, where certain nodes may have many replies; e.g., nodes that include utterances from popular users. Such phenomena lead to the formation of hubs within the conversation tree, which in- dicates scale-free and asymmetrical properties of the conversation tree (Avin et al., 2018). The conversation tree is T represented using T (V, E). where vertex utterance X3 c(ty) is part of the conversa- tion tree T) and edge E E represents one of the three relations between the utterances: (1) parent-comment, (2) comment-reply or (3) reply- reply. CSHT is modeled as a Bi-Directional Tree- LSTM [SSH (X,;4,)) ČSHT (K) where Un is the user representation of the user who au- thored the utterance to be assessed obtained from the HGCN.
The input gate i, and the intermediate memory gate uj corresponding to the representation for the current utterance given by:
ij = exp, a (log, (w(l)) & X, + U
(13)
hy + b()) " = expo (tanh (log, (W(")
2 0 U
hy 1
(14)
The input gate my and the intermediate memory gate $j corresponding to the user representation from the social graph are given by:
m, = exp. (o (log. (W(m)
@ U
(15)
U(m) & hy + b (m)
8; = expo (tanh llog. (W(s) U
(16)
U($) hy + bbb
V represents the encoded
The output gate for the tree cell is then calculated as follows:
0, =xxx (o (w(o) Uu U((o hi (
The parameters (()), U(w), (w) are learnable parameters in the respective gate w. We obtain the cell state for the current cell ey by combining the representations from the multiple forget gates fik= kvi,, s.t. (t&, (j) E E, well gates correspond- ing to the tweet and user representations as follows:
In order to understand the signals in CHST, we focus specific node ty. We gather informa- tion from all input nodes 4a where (4,,4) E, and combine their hidden states hy using Einstein's midpoint to produce an aggregated hidden state hy for node ty. We use the hyperbolic represen- tation of the current post, denoted as X,. as well
Cj = uy my C3j 3 Σ fyx O (18)
These equations are applied recursively by blend- ing in tree representations at every level. The out- put of the current cell, h2 is given by:

---

())) -hi- oj expo (tanh (cz))
to our task, and we describe about the working of each baseline in detail in Appendix B. Specifically, we compare CoSyn with Sentence-BERT (Reimers and Gurevych, 2019), ConceptNet, HASOC (Fa- rooqi et al., 2021), 1), Conc-Perspective (Pavlopoulos et al., 2020), CSI (Ruchansky et al., 2017). GCAN (Lu and Li, 2020), HYPHEN (Grover et al., 2022), FinerFact (Jin et al., 2022), GraphNLI (Agarwal et al., 2022), DUCK (Tian al., 2022), MRIL (Sawhney et al., 2022a) and Madhu (Madhu et al., 2023).
(19)
Final Prediction Layer. We concatenate the node output from CSHT hy. projected to the Eu- clidean space, with the Euclidean utterance fea- tures for the given tweet, X, to form a robust representation incorporating features of the tweet along with social and conversational context de- rived from the CSHT network of CoSyn. This concatenated representation passed through a final classification layer to obtain the final predic- tion ff Softmax (MLP (log. (hz) ;1))). For training, we optimize a binary cross-entropy loss.
3.3 Hyperparameters
We decide on the best based on the best validation set performance us- ing grid search. The optimal hyperparameters are found to be, batch-size b = 32, HGCN output em- bedding dimension 9 512, hidden dimension of CHST h 768, latent dimension of HFAN I 1= 100, learning rate Ir 1.3e weight decay 3. and dropout rate ā 0.41.
for CoSyn
3 Experiments and Results
3.1 Dataset
We evaluate CoSyn on conversational hate speech datasets; namely. Reddit (Qian et al., 2019), GAB (Qian et al., 2019), DIALOCONAN (Bonaldi et al., 2022), CAD (Vidgen al., 2021), ICHCL (Modha al., 2021) and Latent Hatred (EISherief al.. 2021). Appendix A provides dataset statistics. Ta- ble reports the micro-Fi scores averaged across runs with 3 different random seeds.
3.4 Results and Ablations
Table compares the performance of CoSyn with all our baselines on 6 hate speech datasets. As we see, Cosyn outperforms all our baselines both on the entire dataset and on the implicit subset, thus showing its effectiveness for implicit hate speech detection. Despite the ambiguous nature of implicit hate speech, CoSyn has high precision scores for implicit hate speech, thus implying that it mitigates the problem of false positives better than other base- lines. One common observation that our bias invariant SentenceBERT approach emerges as the most competitive baseline to CoSyn, thereby re- inforcing that most prior work do not effectively leverage external context.
Implicit Hate Annotations. The original datasets do not have annotations to indicate if an utterance denotes implicit or explicit hate speech. Thus, for our work, we add extra annotations for evaluat- ing how CoSyn fares against baselines in un- derstanding the context for detecting implicit hate speech. Specifically, we use Amazon Mechani- cal Turk (MTurk) and ask three individual MTTurk workers to annotate if a given utterance conveys hate implicitly or explicitly. The primary factor was the requirement of conversational context to understand the conveyed hate. At stage (1), we provide them with the definition of hate speech and examples of explicit and implicit hate speech. At stage (2), we provide complete conversations and ask them to annotate implicit or explicit in a binary fashion. Cohen's Kappa Scores for inter-annotator agreement are provided with the dataset details in Appendix A.
When evaluated on the entire dataset, CoSyn achieves absolute improvements in the range of 5.1 % 35.2% on Reddit, 4.0% 45.0% on CAD, 4.8% 22.4% on DIALOCONAN, 3.9% 42.7% on GAB, 9.7% 38.4% on ICHCL and 5.8% 31.6% on Latent Hatred ober our baselines. When eval- uated on the implicit subsets, CoSyn achieves ab- solute improvements in the range of 5.1% 57.9%
3.2 Baselines
on Reddit, 8.6% - 31.3% on CAD, 18.2% 40.7% on DIALOCONAN, 2.9% 28.5% on GAB, 8.2%
Due to the acute lack of prior work in this space, beyond just hate speech classification, we also com- pare our method with systems proposed in the thriv- ing fake news detection literature, which consid- ers conversational and user contexts to detect fake news. Some of these systems had to be adapted
19.5% on ICHCL and 1.2% 18.1 1 % Latent Hatred.
Table ablates the performance of CoSyn, re- moving one component at a time to show the sig- nificance of each. All results have been averaged

---

Uneral
Impliihi
Ovenal
Impleli
Oreaal
C.P.
R.F.
C.
RR
P.
Beacker
Redii
CAP
BEALOCONAN
11.5
81Ak
7605
71.01
18.06
66:89
61 1.30
41.18
19:1
08:01
53.00
10:00
66.55
17.50
60.33 10
76-65
50.43
48:08
78:05
Oecephhir
Cree- Proppcttoo
56.21
51.89
01.38
20.25
39
32.35
27.35
33.90
10.11
12.50
37.47
13.34
15.00
11:34
28.00
15.35
13.16
1/1
20.55
31.30
11.30
18.95
OSI
53.77
36.93
54.90
33.43
10.25
34.00
20.59
5530
63.3 11
18.03
43.14
44.25
42.00
30.35
43.25
53.01
39.3
26:25
28.90
2465
306
11.55
OCAN
54.75
07:34
51.99
23.25
30300
18:58
13.35
13.50
59.31
77.08
07.39
44.34
45:0
43:15
33.75
31.58
11.10
32.67
91.33
52.90
29086
14.20
18.00
HYTIEEN
5264
55.12
56.28
22:01
21.10
23.30
31.00
62:0
46.93
12.91
70.22
13:36
23.06
3487
38.00
21.34
21/33
13:39
15:19
11:30
10.12
15.00
13:00
31.5
18.11
00.10
09:15
30.6
5.50
50.00
18.1
14.45
13.05
13.00
55.00
38.50
54.01
31.11
18.31
13.58
16.09
11.0
RCN
16:01
00.61
0.00
11:15
18.56
16.00
11.57
1160
37 18
13.41
33.30
35.17
13.7
1960
31.7
06 40:10
5531
53.89
76.91
53.19
15.00
MAL
28201
55.33
0661
31.00
10.12
13:38
36.01
32.09
14.41
38.21
15.39
40.0
42:0
15.02
21.25
40082
13.63
42.23
23.00
22.31
23.00
20:77
15:31
Marh
10.08
65.38
76.83
76.79
13.79
71.31
77.11
71.14
90:17
05.31
66.13
41.27
10.67
31.63
16.05
13.00
55.35
6.55
79.41
1966
01.02
11:53
13.36
23.03
16.75
57.39
59.27
56.01
38.19
53/0
53.90
43.25
52.95
54.9
51.39
45.02
54.00
Cocopphii
19:79
10:77
68:31
70.31
56.24
16.27
18.11
78.91
17.00
15.13
15.57
17.33
54.15
10:09
53.46
HASSXC
1946
43.44
36.13
16.3
22:11
15.39
12:1
12.33
72.09
12.31
36.31
13.39
11.36
15.23
12.11
3067
52.22
48.33
2132
57.22
42.51
15.21
Coes Peopoctr
53,47
40:23
36.34
18-25
20.54
0.35
26.29
15.45
71.18
20.14
12.25
31.18
25.39
32.14
33.46
2382
51.22
53.09
48.35
-1.11
20.12
42:31
14:37
33.24
23.15
18:34
54.17
75.30
66.35
76.90
31.11
33:15
23.64
16.35
13:11
70
12.06
13.90
18.99
21:25
17:39
14222
72:11
35.22
15.30
14.30
55.30
54.55
59.34
-1.34
31,55
36/71
52.55
3600
15:04
59.11
67.43
71.97
32.27
11.02
33.35
34.00
18:00
12.11
15.22
56.54
$2.00
42.47
16.08
12.00
Gupl NLI
50.00
53.23
47.15
12.13
15.00
39.58
47.00
32:30
5:11
42.58
63.33
36.33
31.29
35.10
67.65
35:15
33.10
1534
48.07
44.55
36.34
12:43
13.00
DUCK
82:76
06.33
66.31
35.00
3530
34.2
36.23
24.00
79.66
75.42
78.30
57.55
31.64
18.12
33.48
33.20
2530
35.00
53.15
28.15
3003
$2.46
26.16
BCN
13:41
41.1
11:
15.11
56.55
16
01
7118
77.6
51.05
55.00
10.77
08.68
51-46
37.11
51:30
06.71
17.0
WL.55
46.36
15.03
41.00
5126
47.65
$5.09
66.12
Table 1: Result comparison of CoSyn with our baselines on 6 hate speech datasets. We compare performance on both the overall dataset and the implicit subset. C.F: and R.F, indicate F; scores measured on only comments and replies, respectively. CoSyn outperforms all our baselines with absolute improvements in the range of 3.4% - 45.00 when evaluated on the entire dataset and 1.2% 57.9% when evaluated on only the implicit subset. - -inicaates conversation trees in the datasct did not have replics.
across all 6 datasets. Some major observations in- clude (1) CoSyn sees a steep drop in performance when used without user context (we model only the conversation context with a vanilla Tree-LSTM in the hyperbolic space). This proves the effec- tiveness of the user's personal context. Addition- ally, modeling user context with HFAN and HGCN proves to be more effective than just feeding mean- pooled historical utterance embeddings into CSHT. (2) Modeling in the Euclidean space results in an overall F1 drop of 3.8%. We reinforce the effective- ness of modeling in the hyperbolic space through our findings in Fig. 2.
Abtatiena
F;
Dharddl
P
R
F;
P
Implict
Commectt F;
Roply F.
CaSyu (Murs)
70.33
09:33
76:82
$6.09
54.17
58.64
46:33
49:32
DET
51.52
57.34
45:23
46:92
45.04
65.92
53.28
$2.00
45:12
46:32
BIANN HEDON
66.39
66.71
08:5
51.14
96.78
43:91
0.33
Lar Choacel
62.11
63.94
64.21
48.72
08.64
49.55
59.5
41.19
BICEST -a OaCCHST
68.62
58.23
65.36
55.20
52.78
57.59
46.22
43.3
Hyporhote- Baclskon
56-18
67.21
5433
51.1
45.44
47.41
Table 2: Ablation study Cosyn. Results are averaged
across all 6 datasets.
earlier, prior efforts have focused primarily on classifying overt abuse or explicit hate speech (Schmidt and Wiegand, 2019) with little or no work on classifying implicit hate speech or that conveyed in coded language. The lack of research on this topic can also be attributed to existing datasets being skewed towards explicitly abusive text (Waseem and Hovy, 2016). Recently, this area of research has seen growing interest, with several datasets and benchmarks released for evaluating the performance of existing hate speech classifiers in identifying implicit hate speech (Caselli et al., 2020; EISherief et al., 2021; Sap et al., 2019). One common observation is that most prior systems from literature fail to identify implicit hate utterances (EISherief et al., 2021). Lin el al. (Lin, 2022) proposes one of the first systems to classify implicit hate speech leveraging world knowledge. However, they evaluate their performance on only Latent Hatred, which lacks conversational context. Additionally, acquiring
3.5 Results Analysis
In Fig. 4, we show test set predictions from 4 differ- ent conversation trees from the ICHCL dataset with an attempt to study the effect of conversational and author context for target utterance classification. We notice that hateful users possess high degrees of homophily and are predominantly hateful over time. It also reveals limitation of CoSyn where insufficient context from the parent leads to a false positive (comment in the 4th example).
4 Related Work
Prior research on identifying hate speech on social media has proposed systems that can flag online hate speech with remarkable accuracy (Schmidt and Wiegand, 2019). However, as mentioned

---

User
Today reppeted apnpoON colled. the apake would Ietter If they to bad about and irsead to work
310C, some belloeno have boon going to shatten once
VVV
cang Inow: then inmuuniyy againd, them CDV33-19
#TAPE
V
Cenrve
Even in such sentire tragedy, the comclence of #POIDON not medkanigg Dribersoiit..
X
x
tlatoo highty educated ruling pertcil
x
F
User
R
breetiing,
He sel olscssed and dt drandsands
Unee
neygen and onygen rélated peont, COMID-19 19 veddines taken.
high-
VVV
User 10
reoovering
VV8
User 11
Gae
High neeting and eocuse collage cartee ann orginhinng
X X
triwiic JAMS cid -NAME> go?
*
x
User
H
User
E
Thank yay #tOC prees weas About time, #LDC neads events.
propa tiss
bet then starcing with Eirorisss, AWANE
Hotbhil Engagement Scale
Pred ons mode by: 1. GemencaBert Cosatiiation Head Coßyt. wrong
Non-utefull
Howwts
Hartorzzd Hatetsl Engagementt
X
impanterss
Coßyn
Figure 4: We study 4 conversation trees in the ICHCL dataset, including the prediction of different classifiers on the utterance to assessed, the historical engagement of the author of the utterance, and the social relations between the different authors.
world knowledge through knowledge graphs (KGs) requires language-specific KGs, and short utterances in conversation trees make the retrieval weak. In the past, to classify context-sensitive hate speech in existing open-source datasets, researchers incorporated conversational context for hate speech classification (Gao and Huang, 2017: Pérez et al., 2022). However, these sys- tems employ naive fusion and fail to leverage structural dependencies between utterances in the conversation tree. Another line of work explores author profiling using community-based social context (connections a user has with other users) (Pavlopoulos al., 2017; Mishra et al., 2018). However, the representations are not learned end-to-end and employ naive fusion.
Conclusion
In this paper, we present CoSyn, a novel learning framework to detect implicit hate speech in online conversations. CoSyn jointly models the conversa- tional context and the author's historical and social context in the hyperbolic space to classify whether target utterance is hateful. Leveraging these texts allows CoSyn effectively detect implicit hate speech ubiquitous in online social media.
Acknowledgement
This work was supported by ARO grants W91 and 1911NF2110066.
Limitations
Hyperbolic networks have been explored earlier for tasks that include modeling user interactions in social media, like suicide ideation detection (Sawhney et al., 2021b, 2022b), fake news detection (Grover et al., 2022), online time stream modeling (Sawhney al., 2021a), etc. All these works show that modeling the hierarchical and scale-free nature of social networks and data generated online benefits from modeling in the hyperbolic space over Euclidean space.
In this section, we list down some potential limita- tions of CoSyn:
Lacking world knowledge is one of CoSyn's potential limitations. The inclusion of world knowledge could serve as crucial context, enhancing CoSyn's performance this task (Sheth et al., 2022). We would like to explore this as part of future work.
Table 2 clearly demonstrates that CoSyn's 's ef- fectiveness relies on the cohesive integration

---

of all its components. Therefore, as direc- tion for future rescarch, our focus will be on enhancing the performance of individual com- ponents.
Ines Chami, Zhitao Ying. Christopher Ré, and Jure Leskovec. 2019. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32.
Krzysztof Choromański, Michal Matuszak, and Jacek Mickisz. 2013. Scale-free graph with preferential tachment and evolving internal vertex structure. Jour- nal of Statistical Physics, (116))1115-1181
Ethics Statement
Our institution's ' Institutional Review Board (IRB) has granted approval for this study. In the annota- tion process, we took precautions by including a cautionary note the instructions, alerting annota- tors to potentially offensive or distressing content. Annotators were also allowed to discontinue the labeling process if they felt overwhelmed.
James Cooley and John W Tukey, 1965. An algo- rithm for the machine calculation of complex fourier series. Mathematics computation, 19(90):297- 301.
Quentin Dénigot and Heather Burnett. 2020. Dogwhis- tles as identity-hased interpretative variation. In Pro- ceedings of the Probability and Meaning Conference (PaM 2020), pages 17-25. ACL.
Additionally, in light of the rapid proliferation of offensive language on the internet, numerous A0- based frameworks have emerged for hate speech detection. Nevertheless, a significant drawback of many current hate speech detection models is their narrow focus explicit or overt hate speech, thereby overlooking the identification of implicit expressions of hate that hold equal potential for harm. CoSyn could ideally identifying implicit hate speech with remarkable accuracy, preventing targeted communities from experiencing increased harm online.
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaish- navi Anupindi, Jordyn Seybolt, Mummun De Choud- hury, and Diyi Yang. 2021. Latent hatred: A bench- mark for understanding implicit hate speech. In Pro- ceedings of the 2021 Conference Empirical Meth- ods in Natural Language Procexsing, pages 345-363, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics.
Zaki Mustafa Farooqi, Sreyan Ghosh, and Rajiv Ratn Shah. 2021. Leveraging transformers for hate speech detection conversational code-mixed tweets. arXiv preprint urXiv:2112.09986
References
Paula Fortuna and Sérgio Nunes 2018. survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 51(4):1-30.
Vibhor Agarwal, Sagar Joglekar, Anthony Young, and Nishanth Sastry. 2022. Graphnli: A graph-based nat- ural language inference model for polarity prediction online debates. In Proceedings the WWW 2022, pages 2729-2737.
Lei Gao and Ruihong Huang. 2017. Detecting on-
line hate speech using context aware models. arXiv preprint arXiv: 1710.07395
Chen Avin, Zvi Lotker, David Peleg, Yvonne-Ane Pignolet, and Itzik Turkel. 2018. Elites social net- works: An axiomatic approach to power balance and price's square root law. PloS one, 13(10)xe005820.
Tanmay Garg. Sarah Masud, Tharun Suresh, and Tanmoy Chakraborty. 2022. Handling bias in toxic speech detection: A survey. arXiv preprint arXiv:2202.00126.
Albert-Lászie Barabási and Eric Bonabeau. 2003. Scale- free networks. Scientific american, 288(5)660-69.
Karish Grover, SM Angara. Md Akhtar, Tanmoy Chakraborty, et al. 2022. Public wisdom mat- ters! discourse-aware hyperbolic fourier co-attention for social-text classification. arXiv preprint arXiv:2209.33017
Helena Bonaldi, Sara Dellantonio, Serra Sinem Tekiroglu, and Marco Guerini. 2022. Human- machine collaboration approaches to build a dialogue dataset for hate speech countering. In Proceedings of the 2022 Conference on Empirical Methods Nat- ural Language Processing, pages 8031 1-11--88 1-8049, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Joshua Gubler and Nathan Kalmoe. 2015. Violent rhetoric in protracted group conflicts: Experimental evidence from israel and india. Political Research Quarterly, 68(4):651-664
Liam Hebert, Lukasz Golab, and Robin Cohen. 2022. Predicting hateful discussions on reddit using graph transformer networks and communal context. In 2022 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), pages 9-17.
Tommaso Caselli, Valerio Basile, Jelena Mitrović, Inga Kartoziya, and Michael Granitzer. 2020. feel of- fended, don't it be abusive! implicit/explicit messages offensive and abusive language. Proceedings of the 12th LREC.

---

Yiqiao Jin, Xiting Wang, Ruichao Yang. Yizhou Sun,
content identification in english and indo-aryan lan- guages and conversational hate speech. In Forumfor Information Retrieval Evaluation, pages 1-3.
Wei Wang, Hao Liao, and Xing Xie. 2022. Towards fine-grained reasoning for fake news detection. In Proceedings of the AAAI, volume 36, pages 5746- 5754.
Nicolas Ocampo, Ekaterina Sviridova, Elena Cabrio, and Serena Villata. 2023. An in-depth analysis of implicit and subtle hate speech messages. In Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 1989-2005.
David Jurgens, Libby Hemphill, and Eshwar Chan- drasekharan. 2019. A just and comprehensive strat- egy for using NLP address online abuse. In Pro- ceedings of the 57th Annual Meeting of the ACL, pages (658-6666. ACL.
John Pavlopoulos, Prodromos Malakasiotis, Juli Baka- gianni, and Ion Androutsopoulos. 2017. Improved abusive comment moderation with user embeddings. In Proceedings of the 2017 EMNLP Workshop: Nat- ural Language Processing meets Journalism, pages 51-55. ACI..
Jonathan Leader Maynard and Susan Benesch. 2016. Dangerous speech and dangerous ideology: An inte- grated model for monitoring and prevention. Geno- cide Studies and Prevention, 9(3).
Jessica Lin. 2022. Leveraging world knowledge in implicit hate speech detection. arXiv preprint arXiv:2212. 14100.
John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. Tox- icity detection: Does context really matter? arXiv preprint 2xxv:2006.00998
Yi-Ju Lu and Ch Li. 2020. Gcan: Graph- aware co-attention networks for explainable fake news detection on social media. arXiv preprint 2xXi:2004.11644:
Juan Manuel Pérez, Franco Luque, Demian Zayat, Martín Kondratzky, Agustín Moro. Pablo Serrati, Joaquín Zajac, Paula Miguel, Natalia Debandi, Agustín Gravano, et al. 2022. Assessing the impact of contextual information in hate speech detection. arXiv preprial 1rX0v:2210.00665.
Hiren Madhu, Shrey Satapara, Sandip Modha, Thomas Mandl, and Prasenjit Majumder. 2023. Detecting offensive speech conversational code-mixed di- alogue on social media: A contextual dataset and benchmark experiments. Expert Systems with Appli- cations, 113:1.
James A Piazza. 2020.
Politician hate speech and International Interactions.
domestic terrorism. 46(3):431-453.
Rijul Magu and Jiebo Luo. 2018. Determining code words in euphemistic hate speech using word embed- ding networks. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 93-100. ACL.
Jing Qian. Anna Bethke, Yinyin Liu, Elizabeth M. Belding Royer, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Conference on Empirical Methods in Natural Language Processing.
Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019. Spread of hate speech in online social media. In Proceedings of the 10th ACM www. pages 173-182.
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018. Leveraging intra-user and inter-user representation learning for automated hate speech detection. In NAACL-HLT 2018, pages 118-123. ACL.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukher- jec. 2021. Hatexplain: A benchmark dataset for ex- plainable hate speech detection. In Proceedings of the AAAI, volume 35, pages 14867-14875.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese nett-ne..orks. arXiv preprint arXiv:1908. 10084.
Pushkar Mishra, Marco Del Tredici, Helen Yan- nakoudakis, and Ekaterina Shutova. 2018 Author profiling for abuse detection. In Proceedings of the 27th ICCL, pages 1088-1098 ACL
Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on CIKM, pages 797-806.
Sandip Modha, Thomas Mandl, Prasenjit Majumder, Shrey Satapara, Tithi Patel, and Hiren Madhu. 2022. Overview of the hasoc subtrack at fire 2022: Identifi- cation of conversational hate-speech in hindi-english code-mixed and german language. Working Notes of FIRE.
Manuela Sanguinetti. Fabio Poletto, Cristina Bosco. Viviana Patti, and Marco Stranisci. 2018. An italian twitter corpus of hate speech against immigrants. In Proceedings of the eleventh LREC.
Sandip Modha, Thomas Mandl, Gautam Kishore Shahi, Hiren Madhu, Shrey Satapara, Tharindu Ranasinghe, and Marcos Zampieri. 2021. Overview of the hasoc subtrack at fire 2021: Hate speech and offensive
Maarten Sap. Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A Smith, and Yejin Choi. 2019. Social bias frames: Reasoning about social and power implica- tions of language. arXiv preprint arXiv: 19] 1911199999999 1.03891.

---

Ramit Sawhney, Shivam Agarwal, Atula T. Neerkaje, and Kapil Jayesh Pathak. 2022a. Orthogonal multi- manifold enriching of directed networks. In Pro- ceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 6074-6086. PMLR.
Nitasha Tiku and Casey Newton, 2015. Twitter weo: suck at dealing with abuse.". The Verge, 4(2):2015.
Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, and Rebekah Tromble. 2021. Introducing cad: the contextual abuse dataset. In Proceedings of the 2021 Conference of the North American Chap- ler of the Association for Computational Linguistics: Human Language Technologies, pages 2289-2303,
Ramit Sawhney, Shivam Agarwal, Atula Tejaswi Neerkaje, Nikolaos Aletras, Preslav Nakov, and Lu- cie Flek. 2022b. Towards suicide ideation detection through online conversational context. In Proceed- ings of the 45th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, SIGIR '22, page 17 1727, New York, NY, USA. Association for Computing Machinery.
William Warner and Julia Hirschberg, 2012. Detecting hate speech on the world wide web. In Proceed- ings of the Second Workshop on Language in Social Media, pages 19 26. ACL.
Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88 93, ACL.
Ramit Sawhney, Shivam Agarwal, Megh Thakkar, Ar- nav Wadhwa, and Rajiv Ratn Shah. 2021a. Hyper- bolic online time stream modeling. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, STGIR 221, page 1682-1686, New York, NY, USA. Association for Computing Machinery.
Michael Wiegand. Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In NAACL-HLI 2019, pages 602- 608. ACL.
Ramit Sawhney, Harshit Joshi, Rajiv Ratn Shah, and Lu- cie Flek. 2021b. Suicide ideation detection via social and temporal user representations using hyperbolic learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 2176-2190, Online. Association for Computational Linguistics.
Zhiping Xiao, Weiping Song, Haoyan Xu, Zhicheng Ren, and Yizhou Sun. 2020. Timme: Twitter adeology-detection via multi-task multi-relational em- bedding. In Proceedings of the 26th ACM SIGKDD, pages 2258-228880 -268.
Chengkun Zhang and Junbin Gao. 2021. Hype-ham: Hyperbolic hicrarchical attention network for seman- tic embedding. In Proceedings of the twenty-Ninth International Conference on International Joint Con- ferences on Artificial Intelligence, pages 3990-3996.
Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language pro- cessing. In Fifth International Workshop on Natural Language Processing for Social Media, pages 1-10. ACI.
A Dataset Details
Reddit. The Reddit hate speech intervention dataset (Qian et al., 2019) has 5,020 conversations, including 22,324 comments. On average, each con- versation consists of 4.45 comments, and the length of each comment is 58.0 tokens. 5,257 comments labeled hate speech, and 17,067 are labeled non- hate speech. Most conversations, 3,847 (76.6%). contain hate speech. Each conversation with hate speech has 2.66 responses on average, for a to- tal of 10,243 intervention responses. The average length of the intervention responses is 17.96 tokens. User history, the timestamp for each post, and the username for each post were fetched using the Red- dit API². Dataset statistics can be found in Table 3. The Cohen's 's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.78, 0.74, and 0.71.
Anna Schmidt and Michael Wiegand. 2019. survey on hate speech detection using natural language pro- cessing. In Fifth International Workshop on Natural Language Processing for Social Media, April 3, 2017, pages 1-10. ACI..
Amit Sheth. Valerie L Shalin, and Ugur Kursuncu. 2022. Defining and detecting toxicity on social me- dia: context and knowledge are key. Neurocomput- ing, 12-318.
Leif Sigerson and Cecilia Cheng. 2018 Scales for mea- suring user engagement with social network sites: A systematic review of psychometric properties. Com- puters in Human Behavior, 83:87-105.
Kai Sheng Tai, Richard Socher, and Christopher D Man- ning. 2015. Improved semantic representations from tree-structured long short-tery term memory networks. arXiv preprint arXii:/503.000
Lin Tian, Xiuzhen Jenny Zhang, and Jey Han Lau. 2022. Duck: Rumour detection on social media by mod- elling user and comment propagation networks. In NAACL-IILT 2022, pages 4939-4949.
GAB. The GAB (Qian et al., 2019) 11 11,825 con- versations, consisting of 33,776 posts. On average,
hhtps//wwww.redddt.cmm/////

---

each conversation consists of 2.86 posts, and the average length of each post is 35.6 tokens. 14,614
Algorithm I CoSyn: Implicit Hatespeech Detection
Given:
N distinct conversation T indesed T.
posts are labeled hate speech, and 19,162 are la-
L distinct users where cach user is indexed by Mi historical utterances of user H
beled non-hate speech. Nearly all the conversa- tions, 11, (94.5%), contain hate speech. 31,487 intervention responses were originally collected for conversations with hate speech, or 2.82 responses per conversation on average. The average length of the intervention responses is 17.27 tokens. User history was fetched using the GAB API². The meta- data for each fetched post provides information like the author and timestamp for a post. Dataset statis- ties can be found in Table 6. The Cohen's s Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.85, 0.82, and 0.91.
each tree Tn = 110
>J" utterances
Initialize:
e(. +- c(JTn) Un + c(H
b Bias-invarriant Encoder Training
D'ncode historical utterances using HPAN
Map -HGGN(M
Moddllng Personal Context
Process:
Considkring 17th belonging to user M2
23 =(t!")
CSHT(X,
DUSHI D-Final Prediction
Softmax (MLP ([lgg (by) ;2G]))
return 25"
0.65, 0.73, and 0.69.
ICHCL. The ICHCL dataset (Modha et al., 2021) (Identification of Conversational HateSpeech in Code-Mixed Languages) consists of hind-english code-switched Twitter conversations. The primary task is identify comments and replies that can be considered acceptable when considered alone but may appear hateful, profane, or offensive when the context of a parent tweet is considered. Binary classification of such contextual posts was consid- ered in this subtask. Around 7,000 code-mixed postings in English and Hindi were downloaded from Twitter and annotated in-house by the authors. User history was fetched using the Twitter API3, Dataset statistics can be found in Table 8. The Co- hen's 's Kappa Scores for inter-annotator agreement for pairs of annotators annotating for implicit hate speech were 0.72, 0.81, and 0.74.
DIALOCONAN. The DIALOCONAN dataset (Bonaldi et al., 2022) has more than 3K dialog- ical interactions between two interlocutors, one acting as the hater and the other as the NGO op- erator, for a total of more than 16K turns. The previous tweets of a particular user were consid- ered to be the history for that. Dataset statistics can be found in Table 7. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.79, 0.78, and 0.79.
CAD. The CAD dataset (Vidgen et al., 2021) is an annotated dataset of ~ 25,000 Reddit entries. The dataset is labeled with 6 conceptually different cate- gories, including Identity- -iiritt--, Afflliationddireteed Counter Speech, Non-hateful Slurs, and Neutral. The dataset is also annotated with salient subcategories, such as whether per- sonal abuse is directed at a person in the conversa- tion thread or someone outside it. This taxonomy offers greater coverage and granularity of abuse than previous work. Each entry can be assigned to multiple primary and/or secondary categories. The dataset is also annotated in context, where each en- try is annotated in the context of the conversational thread it is part of. Every annotation also has la- bel for whether contextual information was needed make the annotation. User history, the times- tamp for each post, and the username for each post were fetched using the Reddit API Dataset statis- tics can be found in Table 5. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were
Latent Hatred. The Latent Hatred dataset (EISh- erief et al., 2021) is hate speech dataset of 27K Gab messages annotated according to a 6-class tax- onomy that includes White Grievance, Incitement to Violence, Inferiority Language, Irony, Stereo- types and Misinformation, and Threatening and Intimidation. Comments for every post, the user's ss user history, and the timestamp for each post were fetched using the Twitter API Dataset statistics can be found in Table 4. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annota- tors annotating for implicit hate speech were 0.91, 0.96, and 0.89.
B Baseline Descriptions
Sentence-BERR w/ Classification Head We use our bias-invariant Utterance Encoder trained and

---

Split
Convs.
Comments
Replies
Hate
Non-Hate
Implicit
Explict
Split
Coms.
Comments
Replics
Implaeit
Explicit
6755
2065
11317
Train
4613
3094
Val
3608
1348
2280
706
3755
1097
Table 3: Dataset statistics for the Reddit dataset.
Table Dataset statistics for the ICHCL dataset.
Split
Convs.
Comments
Replics
Hate
Non-Hate
Implicit
Explictt
Train
12485
418
3836
8649
3246
590
propagation-bast based user's features.
Val
4162
83
1309
2853
1096
213
HYPHEN (Grover et al., 2022) HYPHEN or
Text
4162
76
2373
1789
2138
235
Discourse-Aware Hyperbolic Fourier Co-Attention, proposed for social-text classification and incor- porates conversational discourse knowledge with
Table 4: Dataset statisties for the Latent Hatred Dataset.
Split
Coms.
Comments
Replics
Hate
Non-Fate
Implici
Explicit
Abstract Meaning Representation graphs and em-
12704
2511
11073
1307
Val
4526
1169
834
3692
375
459
ploys co-attention in the hyperbolic space.
Test
5307
4884
963
4342
405
560
FinerFact (Jin et al., 2022) FinerFact, originally proposed for fake news detection, incorporates a fine- fine-grained reasoning framework by introducing mutual reinforcement-basee method for incorpo- rating human knowledge and designs prior-aware aware bi-channel kernel graph network to model subtle differences between picces evidence.
Table 5: Dataset statistics for the CAD dataset
Split
Comss
Comments
Replics
Hate
Non-Hate
Implicit
Explictt
Train
18300
3264
6948
6083
12212
2438
3663
Val
6100
1789
2336
1975
4125
737
1238
Test
6100
1809
2315
1976
4124
241
1235
Table 6: Dataset statistics for the Gab dataset
GraphNLI (Agarwal al., 2022) Graph-based Natural Language Inference Model (GraphNLI) proposes a graph-based deep learning architecture that effectively captures both the local and the global context in online conversation trees through graph-based walks.
Comments
Replics
Hale
Non-Hate
Implicit
Explict
Train
13675
7393
5837
5838
4262
Val
2436
1520
1218
1218
365
Test
2514
160
1591
1257
1257
336
Table Dataset statistics for the DIALOCONAN dataset.
DUCK (Tian al., 2022) Rumour detection with user and comment networks (DUCK), similar CoSyn, employs graph attention networks to jointly model the contents and the structure of social media conversation trees, well as the network of users who engage in these conversations.
inferred on utterances in isolation for this baseline.
ConceptNet Similar to Sentence-BEiR but fused with world knowledge from ConceptNet inspired from (EISherief et al., 2021; Lin, 2022).
Additional Details
HASOC (Farooqi et al., 2021) We use the system proposed by the winning solution in the HASOC 2021 challenge. The authors propose to train a transformer encoder by concatenating parent- comment-reply chains separated by the SEP token. Conc-Persper SenePPerspective (Pavlopoulos et al., 2020) Simi- lar to HASOC, but context concatenation is done only during inference and not during training.
Model Parameters:
Sentence-BERT has R 110M parameters with 12-layers of encoder, 768-hidden- state, 2048 feed-forward hidden-state and 8-heads. CoSyn has 18M parameters.
Compute Infrastructure: All our experiments are conducted on a single NVIDIA A100 GPU, An entire CoSyn training pipeline takes x 40 minutes.
CSI (Ruchansky al., 2017) Capture, Score, and Integrate (CST), originally proposed for fake news classification, implements neural network to jointly learn the temporal pattern of user activity on a given utterance, and the user characteristic based on the behavior of users.
Implementation Software and Packages: We im- plement all our models in PyTorch and use the HuggingFace implementations of SeRTenceBERT.
GCAN (Lu and Li. 2020) Graph-aware Co- Attention Network (GCAN), originally proposed for fake news classification, implements a neu- ral network that uses the target utterance and its
Potential Risks: CoSyn relies training data that may contain biases inherent in the data sources.
'htpss//ppytorch.org/
hhtps://hugginggfcc..o/

---

The detection system may inadvertently amplify reinforce existing socictal biases if these biases are not adequately addressed. For example, the training data is biased towards specific demograph- ics or ideologies, the model might exhibit unfair treatment or misclassification of certain groups, leading to potential discrimination or harm. Un- derstanding the contextual nuances of language is complex task. CoSyn might also be prone to weergeeneralization, potentially resulting in false positives misclassification of non-hateful speech. The risk is that legitimate expressions of opinion or controversial yet non-hateful statements may be mistakenly flagged as hate speech. This could in- advertently lead to censorship or suppression of freedom of speech, limiting open dialogue and crit- ical discussions important social issues.


Reviews:
[{"review_id": "hHKmpBVGo0", "reviewer": "Reviewer_ArYr", "summary": "", "questions": "Will the additional implicit hate speech annotation be publicly available?\n\nAs the authors also claim, implicit hate speech is more challenging than explicit hate speech. On the other hand, several models, including the proposed one in Table-1, show better performance for implicit hate speech than the overall hate speech, which is counter-intuitive.  This could be originated from the annotation of the implicit hate speech conducted via crowdsourcing. I would like to hear from the authors what their opinion is.\n\nMoreover, the details of the annotation study are missing. Is there any quality check for the annotators (e.g., via control questions)? \n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer ArYr,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Reasons To Reject\n> 1. Despite its strength in the methodological part, this paper is missing an error analysis section, which I found very critical for NLP papers. Providing several examples where the model systematically fails (for both explicit and implicit cases), and a more profound outlook on the results in terms of the characteristics of the datasets would improve the impact of the paper.\n\n**Ans:** We thank you for the feedback. Due to space constraints, we only provide one example with error analysis in Figure 4 on Page 8. Since the rebuttal does not allow additional materials, this is what we promise to do with the final version of the paper (with the extra page allotted to us):\n- Add one more example in the main paper similar to Figure 4.\n- Add at least 3 more examples in the Appendix similar to Figure 4.\n\n### Questions For The Authors\n> 1. Will the additional implicit hate speech annotation be publicly available?\n\n**Ans:** Thank You for the question. Yes of course we will make them publicly available!\n\n> 2.  As the authors also claim, implicit hate speech is more challenging than explicit hate speech. On the other hand, several models, including the proposed one in Table-1, show better performance for implicit hate speech than the overall hate speech, which is counter-intuitive. This could be originated from the annotation of the implicit hate speech conducted via crowdsourcing. I would like to hear from the authors what their opinion is.\n\n**Ans:** We appreciate the reviewer's feedback and analysis. We notice that this phenomenon occurs only in approximately 7 instances across 6 datasets and 12 models (10% of the total experiments). The Implicit F1 score is a composite metric derived from three individual scores: Parent, Comment, and Reply F1 scores. Firstly, we notice certain cases where a data imbalance exists among these categories, which could affect the overall performance. Given that Parent tweets typically offer richer contextual cues, the model's superior performance in this category could have affected the final Implicit F1 score. Nevertheless, it's noteworthy that a significant performance gap persists between the overall and implicit Comment and Reply F1 scores, suggesting that detecting implicit hate speech remains a challenge across these tweet types.\nWe concur with the reviewer's observation and attribute this phenomenon to other factors like data imbalance, model bias, or potential overfitting to specific characteristics.\n\n>  3. Moreover, the details of the annotation study are missing. Is there any quality check for the annotators (e.g., via control questions)?\n\n**Ans:** We sincerely apologize for missing these details. As mentioned in the paper in Section 3.1, we follow a two-stage annotation process where: During stage (1), we provide the annotators with the definition of hate speech and examples of explicit and implicit hate speech. During stage (2), we provide complete conversations and ask them to annotate implicit or explicit in a binary fashion. We will now elaborate on Step 1 (with the control questions) and include these details in the final version of our paper (with the extra page allotted to us).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "It is a very well-written and well-structured paper. The problem definition is formulated clearly. The methods and details of the architecture were explained transparently, contributing to its reproducibility.  Their evaluation seems sound and comprehensive: They evaluate their method on several datasets and compare their results with the performance of the existing methods on implicit hate speech.", "reasons_to_reject": "I have stated my concerns in the “Questions for the Author section.” Despite its strength in the methodological part, this paper is missing an error analysis section, which I found very critical for NLP papers. Providing several examples where the model systematically fails (for both explicit and implicit cases), and a more profound outlook on the results in terms of the characteristics of the datasets would improve the impact of the paper. ", "questions_to_authors": "Will the additional implicit hate speech annotation be publicly available?\n\nAs the authors also claim, implicit hate speech is more challenging than explicit hate speech. On the other hand, several models, including the proposed one in Table-1, show better performance for implicit hate speech than the overall hate speech, which is counter-intuitive.  This could be originated from the annotation of the implicit hate speech conducted via crowdsourcing. I would like to hear from the authors what their opinion is.\n\nMoreover, the details of the annotation study are missing. Is there any quality check for the annotators (e.g., via control questions)? \n", "paper_topic_and_main_contributions": "This paper provides a new method of implicit hate speech detection for conversational context by incorporating three aspects: (i) the author’s historical context, (ii) the author’s social context, and (iii) conversational context. Their method of inclusion of this information has been compared to numerous baselines and SOTA methods. \nThey extend the annotation scheme for six commonly used hate speech datasets by employing crowdsourcing annotation (with three raters per utterance). \nSuppose this additional annotation for implicitness is made publicly available. In that case, it can be a valuable source for the researcher who tackles implicit hate speech detection. However, I have some concerns about the quality of the crowdsourcing annotations that I mentioned later on.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8bII3exLrB", "reviewer": "Reviewer_4YGb", "summary": "", "questions": "- Social media posts in Figure 1 might not be hateful conversations as per the definition by Schmidt and Wiegand (2019)). The utterance in the parent post might be cryptic. \n\n- Authors choose the Hyperbolic Fourier Attention Network (HFAN) for author profiling or to learn historical context; for social context, they use the Hyperbolic Graph Convolution Network (HGCN). And Context Synergized Hyperbolic Tree-LSTM (CSHT) to learn the conversational context. The micro F1 score of the ICHCL 2021 dataset is 0.8953. I would like to advise the author to look at the paper by  (Madhu et al., 2023) where the reported macro F1 score is 0.892 using simple architecture (SentenceBERT and LSTM). I would like to ask the authors why they chose such a complex model for the experiment.\n\n-Are the results reported by the authors statistically significant?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 4YGb,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Reasons To Reject\n> 1. The authors did not mention the compelling reasons for designing the proposed model.\n\n**Ans:** Figure 2 and Section 3 describe in detail each component in CoSyn. We list the motivation behind each component below. We would also like to know if any section in the paper is not well-motivated; we will try our best to address and rewrite them in the final version of the paper.\n\n- User engagement on social media is often diverse and possesses scale-free properties. Thus, to account for the natural irregularities and effectively model a user’s personal historical context, we propose a novel Hyperbolic Fourier Attention Network (HFAN) that 1. Use Fourier Transform to model abstract frequencies or patterns appearing in the historical utterances by a user, and 2. Using hyperbolic representation to model the scale-free nature of historical discourse on social media.\n- HGCN has been used to account for the social context of a user defined by their social graph and the historical activity of the users. HGCN modifies the conventional GCN and performs neighbor aggregation using graph convolutions in the hyperbolic space to enrich a user's historical context representations learned through HFAN using social context.\n- To model the conversational context in conversation trees effectively, we propose Context-Synergized Hyperbolic Tree-LSTM (CHST). CSHT presents several modifications and improvements over Tree-LSTM, including (1) incorporating both the user's personal context and the conversation context while clearly capturing the interactions between them and (2) operating in the hyperbolic space, unlike the original TreeLSTM to account for the asymmetric properties of conversation trees on social media.\n\nCoSyn utilizes the advantages of established research while introducing innovative components that specifically consider user and conversation context. This is crucial for effectively identifying implicit hate speech.\n\n### Questions For The Authors\n> 1. Social media posts in Figure 1 might not be hateful conversations as per the definition by Schmidt and Wiegand (2019)). The utterance in the parent post might be cryptic. \n\n**Ans:** We thank you for the insightful comment. We agree with your comments. Moreover, we don't think the parent post is hateful. Therefore, we mark it with a green border, with the key “Non Hateful.” Only the comment and the reply to the comment are hateful, which has a red border. We will make this more clear in the final version of our paper.\n\n> 2. Authors choose the Hyperbolic Fourier Attention Network (HFAN) for author profiling or to learn historical context; for social context, they use the Hyperbolic Graph Convolution Network (HGCN). And Context Synergized Hyperbolic Tree-LSTM (CSHT) to learn the conversational context. The micro F1 score of the ICHCL 2021 dataset is 0.8953. I would like to advise the author to look at the paper by  (Madhu et al., 2023) where the reported macro F1 score is 0.892 using simple architecture (SentenceBERT and LSTM). I would like to ask the authors why they chose such a complex model for the experiment.\n\n**Ans:** We thank you for the important question. We sincerely apologize for missing this baseline. Upon request, in Table 1, we present results on all 6 datasets for the system proposed by  (Modha et al., 2023). \n\n|      |          Overall     |       | |             |   Implicit    |       |        |\n|--------------|---------------|-------|----------|-------------|-------|-------|--------|\n|              | F_1           | P     | R        | F_1         | P     | R     | C. F_1 | R. F_1 |\n|  Reddit   |        |       |          |             |       |       |        |        |\n| Modha et Al  | 70.58    | 65.28 | 76.82    | 76.79 | 77.79 | 75.81 | 27.39  | 23.34  |\n| **CoSyn (ours)** | 76.23        | 79.07| 73.58    | 81.12       | 80.15 | 82.12 | 60.23  | 59.12 |\n|       GAB       |           |       |          |             |       |       |        |        |\n| Modha et Al  | 52.94   | 45.19 | 63.91    | 41.15 | 48.72 | 35.61 | 26.84  | 17.77  |\n| **CoSyn (ours)** | 66.71        | 64.43 | 69.15    | 45.00          | 37.01 | 57.38 | 46.22 | 38.29  |\n|      CAD        |           |       |          |             |       |       |        |        |\n| Modha et Al  | 50.47   | 55.72 | 46.13    | 51.77 | 50.67 | 52.91 | 36.63  | -      |\n| **CoSyn (ours)** | 73.26       | 70.07 | 76.75    | 57.59      | 59.27 | 56.01| 38.19 | –      |\n|     ICHCL         |         |       |          |             |       |       |        |        |\n| Modha et Al  | 82.01   | 84.63 | 79.55    | 39.56 | 39.00    | 40.14 | 39.81  | 39.72  |\n| **CoSyn (ours)** | 89.53         | 90.55 | 88.53   | 46.03      | 46.26 | 45.82 | 46.85  | 45.89 |\n|    DIALOCONAN          |    |       |          |             |       |       |        |        |\n| Modha et Al  | 46.45   | 40.06 | 55.26   | 36.96 | 29.44 | 49.64 | 47.62  | 33.82  |\n| **CoSyn (ours)** | 51.02  | 52.92 | 49.25   | 52.98       | 54.67 | 51.39 | 48.77  | 54.00     |\n|   Latent Hatred           |  |       |          |             |       |       |        |        |\n| Modha et Al  | 59.52   | 47.14 | 80.72    | 40.72 | 33.56 | 51.75 | 28.19  | -      |\n| **CoSyn (ours)** | 64.65         | 61.92 | 67.63    | 53.28       | 47.66 | 55.49 | 40.12  | –      |\n\n\n\nTable 1: Result comparison for  (Modha et al., 2023) and CoSyn on all 6 datasets.\n\n\nAs we clearly see, (Modha et al., 2023) perform close to our SentenceBert baseline. Still, CoSyn outperforms it by an average of 9.9% on overall F1 performance and by an average of 8.18% on implicit hate speech F1. We hypothesize that this is because the architecture proposed by Modha et al. is similar to the SentenceBert baseline but with an extra LSTM head. \nThese results also help prove that CoSyn, though more complex than most other architectures in terms of architecture design, effectively captures external context and helps improve implicit hate speech detection. The CoSyn architecture captures different aspects of the context by coordinating between the multiple individual components.\n\n> 3. Are the results reported by the authors statistically significant?\n\n**Ans:** We thank you for the question. As mentioned in the last line of Section 3.1, all scores in Table 1 are micro-F1 scores averaged across 3 runs with 3 different random seeds.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed model, CoSyn, beat the baseline of the six conversation hate speech datasets.\n\n- Codes are available to reproduce the results\n", "reasons_to_reject": "The authors did not mention the compelling reasons for designing the proposed model. ", "questions_to_authors": "- Social media posts in Figure 1 might not be hateful conversations as per the definition by Schmidt and Wiegand (2019)). The utterance in the parent post might be cryptic. \n\n- Authors choose the Hyperbolic Fourier Attention Network (HFAN) for author profiling or to learn historical context; for social context, they use the Hyperbolic Graph Convolution Network (HGCN). And Context Synergized Hyperbolic Tree-LSTM (CSHT) to learn the conversational context. The micro F1 score of the ICHCL 2021 dataset is 0.8953. I would like to advise the author to look at the paper by  (Madhu et al., 2023) where the reported macro F1 score is 0.892 using simple architecture (SentenceBERT and LSTM). I would like to ask the authors why they chose such a complex model for the experiment.\n\n-Are the results reported by the authors statistically significant?", "paper_topic_and_main_contributions": "This paper focuses on detecting implicit hate in social media conversational dialogue. The authors propose a model that exploits the user’s personal context, social context, and conversation context to predict hateful posts. The authors tested the proposed model Cosyn in six conversational hate speech datasets and claimed that their model outperformed the baseline of each dataset.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "ICHCL 2021 dataset full paper:\nS. Satapara, S. Modha, T. Mandl, H. Madhu, P. Majumder, Overview of the HASOC subtrack at FIRE 2021: Conversational hate speech detection in code-mixed language, in: P. Mehta, T. Mandl, P. Majumder, M. Mitra (Eds.), Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, volume 3159 of CEUR Workshop Proceedings, CEUR-WS.org, 2021, pp. 20–31. URL: http: //ceur-ws.org/Vol-3159/T1-2.pdf\n\nBenchmark ICHCL 2021 dataset\n\nH. Madhu, S. Satapara, S. Modha, T. Mandl, P. Majumder, Detecting offensive speech in conversational code-mixed dialogue on social media: A contextual dataset and benchmark experiments, Expert Systems with Applications (2022) 119342. URL: https://doi.org/10. 1016/j.eswa.2022.119342\n"}}, {"review_id": "Y2pFKTGQdR", "reviewer": "Reviewer_cZN1", "summary": "", "questions": "How long are the conversations on average? And what is the average number of tokens per comment? \nIs it possible to feed these to LLMs if the context lengths fit?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer cZN1,\n\nWe thank you for the detailed analysis of our paper and the insightful comments. In what follows, we try to address all our comments point-by-point.\n\n### Reasons To Reject\n> 1. The paper is difficult to follow. Some of the motivations are not clearly explained. For example, it is unclear as to why the Fourier transform is required.\n\n**Ans:** We apologize if you found it difficult to follow. While other reviewers explicitly appreciated the paper's clarity, we would like to know more about which Sections are difficult to follow so we can improve on them.\n\nAs mentioned in Section 2.4, ​​we apply 2D DFT to the embeddings (1D DFT along the temporal dimension and 1D DFT along the embedding dimension) to capture the most commonly occurring frequencies (ideologies, opinions, emotions, etc.) in the historical utterances made by the user. Appendix A describes the DFT formulation, which fundamentally helps analyze frequency patterns and capture periodicities in embedded textual data. More precisely, applying the 2D DFT on embeddings, the model can efficiently identify the most recurring frequency components, and in our model, these repetitive patterns could correspond to consistently used themes, emotions, or ideologies in a user's historical utterances.\n\n> 2. The idea of using conversation trees and hyperbolic learning is not novel. For example, [1] studies similar problems for different tasks. It is unclear as to what the technical novelties are compared to these recent works apart from changing the task.\n\n**Ans:** We thank you for the question. We have highlighted novel contributions in our work compared to prior art, including [1]. We list them down as follows:\n1. We do not use the original TreeLSTM used in [1]. Instead, we propose CSHT, which introduces a novel context interaction mechanism that clearly captures the interplay between user and conversational contexts. Unlike TreeLSTM cells, which take a single embedding as input, CSHT cells take in 2 inputs (both user and conversational contexts) and independently assess the amounts of information to be retrieved from both.\n2. We are the first to introduce the idea of leveraging multiple diverse contexts for implicit hate speech detection and discuss why such contexts are important for the task (last paragraph of Section 1 and Section 2). Subsequently, we propose the CoSyn architecture, which introduces a novel learning methodology leveraging all these contexts intuitively and clearly. CoSyn, as we see in Table 1, improves performance over most baselines.\n3. None of the individual components in CoSyn have been directly proposed in prior art. We are inspired by prior art for some of our operations in the components, but the formulation of each individual component is novel.\n\nWe do not propose hyperbolic space learning as our major novelty. Instead, we show how different modules in CoSyn come together to solve the implicit hate speech detection problem. Modeling in the hyperbolic space just helps us consider the scale-free properties of social media posts and gives CoSyn an extra boost.\n\n> 3. Furthermore, these recent baselines should also be adopted in the performance comparison.\n\n**Ans:** We thank you for the suggestion. As requested, we compare the performance of CoSyn with the two suggested methodologies from the literature [1,2] and show the results in the following Table 1. We will also include these in the final version of our paper.\n\n|      |              | Overall      | |        |    Implicit   |      |        |\n|--------------|---------------|-------|----------|--------|--------|-------|--------|\n|              | F_1           | P     | R        | F_1    | P      | R     | C. F_1 | R. F_1 |\n| Reddit    |        |       |          |        |        |       |        |        |\n| HCN          | 56.01         | 55.64 | 56.38   | 30.03  | 31.18  | 28.96 | 26.90   | 21.67  |\n| MRIL         | 58.91         | 57.30  | 60.61    | 31.76  | 30.12  | 33.58 | 36.67  | 32.05  |\n| **CoSyn (ours)** | 76.23        | 79.07 | 73.58   | 81.12 | 80.15  | 82.12 | 60.23 | 59.12  |\n|              |               |       |          |        |        |       |        |        |\n|  GAB            |           |       |          |        |        |       |        |        |\n| HCN          | 52.51         | 53.88 | 51.21    | 41.60   | 46.11  | 37.89 | 24.25  | 28.97  |\n| MRIL         | 61.21         | 60.75 | 61.67    | 40.92  | 33.65  | 52.19 | 42.01  | 33.89  |\n| **CoSyn (ours)** | 66.71        | 64.43 | 69.15    | 45.00     | 37.01  | 57.38 | 46.22  | 38.29 |\n|  CAD            |           |       |          |        |        |       |        |        |\n| HCN          | 34.63         | 37.18 | 32.41   | 27.65  | 22.30   | 36.37 | 23.87  | -      |\n| MRIL         | 34.41         | 35.27 | 33.59    | 40. 65 | 42.90   | 38.62 | 21.35  | -      |\n| **CoSyn (ours)** | 73.26         | 70.07 | 76.75    | 57.59  | 59.27  | 56.01 | 38.19  | –      |\n|              |               |       |          |        |        |       |        |        |\n|   ICHCL           |         |       |          |        |        |       |        |        |\n| HCN          | 65.36         | 60.45 | 71.14    | 27.66  | 27.03 | 28.32 | 25.03  | 27.99  |\n| MRIL         | 66.71         | 67.87 | 65.59    | 30.22  | 31.65  | 28.91 | 29.01  | 29.89  |\n| **CoSyn (ours)** | 89.53         | 90.55 | 88.53    | 46.03  | 46.26  | 45.82 | 46.85  | 45.89  |\n|    DIALOCONAN          |    |       |          |        |        |       |        |        |\n| HCN          | 39.65         | 34.67 | 46.30     | 25.31  | 23.89  | 26.91 | 22.19  | 15.90   |\n| MRIL         | 40.82         | 39.45 | 42.28    | 25.09  | 22.31  | 28.66 | 20.77  | 16.31  |\n| **CoSyn (ours)** | 51.02         | 52.92 | 49.25   | 52.98  | 54.67  | 51.39 | 48.77  | 54.00     |\n|              |               |       |          |        |        |       |        |        |\n|   Latent Hatred           |  |       |          |        |        |       |        |        |\n| HCN          | 50.47         | 52.39 | 48.68    | 30.11  | 32.96  | 27.71 | 21.32  | -      |\n| MRIL         | 57.32         | 54.88 | 59.98    | 40.16  | 37.80   | 42.83 | 37.61  | -      |\n| **CoSyn (ours)** | 64.65         | 61.92 | 67.63   | 53.28 | 47.66  | 55.49 | 40.12 | –      |\n|              |               |       |          |        |        |       |        |        |\n\n\nTable 2: Result comparison of HCN  and MRIL with CoSyn on 6 hate speech datasets.\n\n\nAs we clearly see, CoSyn improves over HCN [1] by an average of 20.46% on overall performance and an average of 25.61% on implicit hate speech detection performance. On the other hand, CoSyn improves over MRIL [2] on overall performance by an average of 17% and an average of 21.2% on implicit hate speech detection.\n\n> 4. The architecture involves quite a few complex components, which may add additional computational overload. An analysis of the complexity will be interesting to see. The hyperbolic operations are often time-consuming and have an unstable training process\n\n**Ans:** We thank you for the insightful comments. We now analyze the complexity of CoSyn. We will add these details to the final version of the paper.\n\n**Complexity**:\n\nBias Invariant Encoder: For a sequence of length t tokens, the complexity of encoding it is O(t^2), given that a transformer-based model is used.\n\nHFAN: For a given user, d past utterances with t tokens each are considered user history. The bias invariant encoder encodes these utterances, which follows with a complexity of O(d*t^2). A 2D Fourier transform is applied to the bias invariant representations, which costs O(d*t(log(t)+log(d)). The computational complexity of hyperbolic operations is asymptotically bounded by the complexity of calculating the vector norm, which is used to compute various projections and perform Mobius operations O(d*t). The complexity for the Mobius GRU, therefore, is then bound by the sequence length, which is O(t) in this case, once the hyperbolic projection has been accounted for. This is followed by Hyperbolic projection and computation of the Einstein mid-point, which is proportional to O(d^2), as followed by this discussion. O(d^2+t+d*t+d*t(log d + log t) + d*t^2).\n\nHGCN: The key computation in each layer of a GCN is the multiplication of the adjacency matrix and the node feature matrix. Consider a user graph with n nodes, and m dimensional features; the hyperbolic activations are of the order of O(n*m^2). The key computation in each layer of a HGCN is the multiplication of the adjacency matrix and the node feature matrix, which is O(n^2*m), but for sparse graph representations used by use, this can further be approximated to O(e*m) where e represents the number of edges in the graph. For L layers, therefore, the complexity of the HGCN is O(L*e*m + m*n^2).\n\nCHST: CHST is modeled with a tree-LSTM base. The complexity, therefore, depends on the number of nodes in the tree: O(p).\n\nThe overall complexity combining the three components is thus: O(p) + O(e*m+n*m^2) + O(d^2+t+d*t+d*t(log d + log t) + d*t^2). \n\nConsidering asymptotic complexity with respect to constant dimensions, the complexity can be expressed as a function of input size as O(p) + O(n). The stated complexities should be seen in light of the intricate mechanisms each architecture employs to fulfill its role effectively. We do not notice any major instability in CoSyn training. Due to the platform's limitations, we cannot provide an image that plots the loss against the training steps. However, we will add this to the final version of the paper.\n\n### Questions For The Authors\n> 1. How long are the conversations on average? And what is the average number of tokens per comment?\n\n**Ans:** Thank You for the question. The conversation chains, on average (averaged across 6 datasets), are 17 utterances long with an average of 42 tokens per conversation. More details can be found in Tables 3,4,5,6,7 and 8 in the Appendix.\n\n> 2. Is it possible to feed these to LLMs if the context lengths fit?\n\n**Ans:** Feeding conversation chains to instruction-tuned LLMs might be an interesting direction of research we would like to explore as future work. However, no prior-work explores the prompts to use or the learning mechanism to be employed. Also, the maximum number of tokens per conversation across 6 datasets for our experiments is 13728, which might cross the context length of modern LLMs.\nAdditionally, it should be noted that open-source LLMs might be unable to answer with their safety features. Additionally, there would be no clear way to feed user context (historical and social) within the conversational context.\n\n### References\n\n[1] Sawhney, Ramit, et al. \"Towards suicide ideation detection through online conversational context.\" Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 2022.\n\n[2] Sawhney, Ramit, et al. \"Orthogonal Multi-Manifold Enriching of Directed Networks.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\t¬The proposed architecture, CoSyn is technically sound. The authors use hyperbolic learning to model the scale-free properties of the network. \n2.\tThe experiments on 6 different datasets are good and the authors show performance improvements compared to baseline methods. \n3.\tThe qualitative analysis is interesting and gives nice interpretations", "reasons_to_reject": "1.\tThe paper is difficult to follow. Some of the motivations are not clearly explained. For example, it is unclear as to why the Fourier trasnsform is required. \n2.\tThe idea of using conversation trees and hyperbolic learning is not novel. For example [1], study similar problems for different tasks. It is unclear as to what the technical novelties are compared to these recent works apart from changing the task. \n3.\tFurthermore, these recent baselines should also be adopted in the performance comparison.\n4.\tThe architecture involves quite a few complex components which may add additional computational overload. An analysis on the complexity will be interesting to see. The hyperbolic operations are often time consuming and have unstable training process", "questions_to_authors": "How long are the conversations on average? And what is the average number of tokens per comment? \nIs it possible to feed these to LLMs if the context lengths fit?", "paper_topic_and_main_contributions": "The authors study the problem of hatespeech detection on socal media. They build a neural network to capture implicit hate speech that uses user and conversational contexts. The authors experiment on 6 different hate-speech datasets and show improvements.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Towards Suicide Ideation Detection Through Online Conversational Context\n https://dl.acm.org/doi/abs/10.1145/3477495.3532068 also see references by https://proceedings.mlr.press/v151/sawhney22a.html"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper under review presents a novel approach to implicit hate speech detection in conversational contexts. It incorporates the author's historical context, social context, and conversational context, comparing its methodology with various baselines and state-of-the-art (SOTA) methods. Additionally, the paper extends the annotation scheme for six widely used hate speech datasets through crowdsourced annotations. While there are some concerns regarding the quality of these annotations, the paper's clarity, thorough evaluation, and the superiority of the CoSyn model over baselines make it a valuable addition to the field, with potential significance for future research in implicit hate speech detection.

Three reviewers (ArYr, 4YGb, and cZN1) provided their feedback and insights on the paper. All reviewers acknowledge the paper's contributions to implicit hate speech detection and its comparative analysis against baselines and state-of-the-art methods. ArYr commends the paper's precise problem formulation, transparent explanations of methods, and comprehensive evaluation on multiple datasets. 4YGb highlights the success of the proposed model, CoSyn, in outperforming baselines across six conversational hate speech datasets and the availability of code for result reproducibility. cZN1 praises the paper's technical soundness, particularly using hyperbolic learning to model network properties and the interesting qualitative analysis providing interpretations.
Reviewer Concerns and Suggestions:

Some concerns, although addressable, have been raised by the reviewers as well. ArYr recommends adding an error analysis section and providing examples of where the model systematically fails for both explicit and implicit cases. 4YGb notes that the paper lacks a clear explanation of the compelling reasons for designing the proposed model. cZN1 asked for clarity regarding the use of Fourier transforms and the addition of a complexity analysis or a way to assess the computational load of the proposed model compared with baselines.

In summary, while the paper receives recognition for its contributions and strengths, the authors should address concerns related to error analysis and computational complexity to enhance the paper's impact. Future paper readers will also appreciate some changes in the manuscript to better motivate the inclusion of specific design elements like Fourier transform.
Decision: Accept-Main