Venue: EMNLP 2023

CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

Sreyan Ghosh\({}^{1*}\)   Manan Suri\({}^{2,3*}\) Purva Chiniya\({}^{1*}\) Utkarsh Tyagi\({}^{1*}\)

**Sonal Kumar\({}^{1*}\) Dinesh Manocha\({}^{1}\) \({}^{1}\)University of Maryland College Park, USA, \({}^{2}\)MIDAS Labs, IIIT-Delhi, India {sreyang, pchiniya, utkarsht, sonalkum, dmanocha}@umd.edu, manansuri27@gmail.com**

###### Abstract

The tremendous growth of social media users interacting in online conversations has led to significant growth in hate speech affecting people from various demographics. Most of the prior works focus on detecting _explicit hate speech_, which is overt and leverages hateful phrases, with very little work focusing on detecting hate speech that is _implicit_ or denotes hatred through indirect or coded language. In this paper, we present **CoSyn**, a **c**ontext **sv**nergized neural network that explicitly incorporates user- and conversational-context for detecting _implicit hate speech_ in online conversations. CoSyn introduces novel ways to encode these external contexts and employs a novel context interaction mechanism that clearly captures the interplay between them, making independent assessments of the amounts of information to be retrieved from these noisy contexts. Additionally, it carries out all these operations in the hyperbolic space to account for the scale-free dynamics of social media. We demonstrate the effectiveness of CoSyn on 6 hate speech datasets and show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%. We make our code available1.

Footnote 1: [https://github.com/Yeyan88/CoSyn](https://github.com/Yeyan88/CoSyn)

+
Footnote †: These authors contributed equally to this work.

## 1 Introduction

Hate speech is defined as the act of making utterances that can potentially offend, insult, or threaten a person or a community based on their case, religion, sexual orientation, or gender Schmidt and Wiegand (2019). For social media companies like Twitter, Facebook, and Reddit, appropriately dealing with hate speech has been a crucial challenge Tiku and Newton (2015). Hate speech can take the form of overt abuse, also known as _explicit hate speech_Schmidt and Wiegand (2017), or can be uttered in coded or indirect language, also known as _implicit hate speech_Jurgens et al. (2019). Fig.1 illustrates a Twitter conversation where users convey hate implicitly through sarcasm and conversational context.

**Soicetal Problem and Impact.** Though a considerable amount of research has been done on detecting explicit hate speech Schmidt and Wiegand (2019), detecting implicit hate speech is a greatly understudied problem in the literature, despite the social importance and relevance of the task. In the past, extremist groups have used coded language to assemble groups for acts of aggression Gubler and Kalmoe (2015) and domestic terrorism Piazza (2020) while maintaining deniability for their actions Denigot and Burnett (2020). Because it lacks clear lexical signals, implicit hate utterances evade keyword-based detection systems Wiegand et al. (2019), and even the most advanced neural architectures may not be effective in detecting such utter

Figure 1: Illustration of a social media conversation tree with _implicit hate speech_. User 1 posts a factual statement about practices people follow in a certain location. In response, User 2 implies hate through a sarcastic statement, to which User 3 elaborates with a positive stance. Clearly, these utterances are even difficult for humans to classify as hate or not without the proper conversational context.

 

ances (Caselli et al., 2020).

**Current Challenges.** Current state-of-the-art hate speech detection systems fail to effectively detect implicit and subtle hate OCampo et al. (2023). Detecting implicit hate speech is difficult for multiple reasons: **(1)** Linguistic nuance and diversity: Implicit hate can be conveyed through sarcasm, humor Waseem and Hovy (2016), euphemisms Magu and Luo (2018), circumlocution Gao and Huang (2017), and other symbolic or metaphorical languages Qian et al. (2018). **(2)** Varying context: Implicit hate can be conveyed through everything from dehumanizing comparisons Leader Maynard and Benesch (2016), and stereotypes Warner and Hirschberg (2012) to threats, intimidation, and incitement to violence Sanguinetti et al. (2018); Fortuna and Nunes (2018). **(3)** Lack of sufficient linguistic signals: Unlike parent posts, which contain sufficient linguistic cues through background knowledge provided by the user, replies or comments to the parent post are mostly short and context-less reactions to the parent post. These factors make implicit hate speech difficult to detect and emphasize the need for better learning systems.

**Why prior work is insufficient.**ElSherief et al. (2021) define implicit hate speech as "coded or indirect language that disparages a person or group." They also propose the first dataset, _Latent Hatred_, to benchmark model performance on implicit hate speech classification and show that existing state-of-the-art classifiers fail to perform well on the benchmark. Though _Latent Hatred_ builds on an exhaustive 6-class taxonomy, it ignores implicit hate speech that is conversational-context-sensitive even though it accounts for a majority of implicit hate speech online Modha et al. (2022); Hebert et al. (2022). Lin (2022) builds on Latent Hatred and propose one of the first systems to classify implicit hate speech leveraging world knowledge through knowledge graphs (KGs). However, beyond the fact that their system is restricted to only English due to the unavailability of such KGs in other languages, their system also fails to capture any kind of external context, which is vital for effective hate speech detection Sheth et al. (2022). Thus, we first extend the definition of implicit hate speech to include utterances that convey hate only in the context of the conversational dialogue (example in Fig.1). Next, we propose a novel neural learning system to solve this problem.

**Main Contributions.** In this paper, we propose CoSyn, a novel neural network architecture for detecting implicit hate speech that effectively incorporates external contexts like conversational and user. Our primary aim is to classify whether a target utterance (text only) implies hate or not, including the ones that signal hate implicitly. CoSyn jointly models the user's personal context (historical and social) and the conversational dialogue context in conversation trees. CoSyn has four main components: **(1)** To encode text utterances, we train a transformer sentence encoder and promote it to learn bias-invariant representations. This helps us to handle keyword bias, a long-standing problem in hate speech classification Garg et al. (2022). **(2)** We start by modeling the user's _personal historical context_ using a novel **Hyperbolic Fourier Attention Network (HFAN)**. HFAN models diverse and scale-free user engagement of a user on social media by leveraging Discrete Fourier Transform Cooley and Tukey (1965) and hyperbolic attention on past user utterances. **(3)** We next model the user's _personal social context_ using a **Hyperbolic Graph Convolutional Network (HGCN)**Chami et al. (2019). HGCN models the scale-free dynamics of social networks using hyperbolic learning, leveraging the social connections between users, which act as edges in the graph. **(4)** Finally, to _jointly model a user's personal context and the conversational dialogue context_, we propose a novel **Context Synergized Hyperbolic Tree-LSTM (CSHT)**. CSHT effectively models the scale-free nature of conversation trees and clearly captures the interaction between these context representations in the hyperbolic learning framework. We describe all our components in detail in Section 2.7. To summarize, our main contributions are as follows:

* We introduce CoSyn, the first neural network architecture specifically built to detect implicit hate speech in online conversations. CoSyn leverages the strengths of existing research and introduces novel modules to explicitly take into account user and conversational context integral to detecting implicit hate speech.

* We also perform extensive ablative experiments and qualitative comparisons to prove 

the efficacy of CoSyn.

## 2 Methodology

Fig. 3 provides a clear pictorial representation of our proposed model architecture (algorithm shown in Algorithm 1). We provide an overview describing various operations in Hyperbolic Geometry in Appendix 2.2; and we request our readers to refer to that for more details. In this section, we describe the three constituent components of our proposed **CoSyn**, which model different aspects of context, namely, the author's personal historical context, the author's personal social context, and both the personal historical and social contexts with the conversational context.

### Background, Notations, and Problem Formulation.

Let's suppose we have \(N\) distinct conversation trees, where each tree is denoted by \(T_{n}\in T=\left\{T_{0},\cdots,T_{n},\cdots,T_{N}\right\}\). Each tree \(T_{n}\) has \(J^{T_{n}}\) individual utterances denoted by \(T_{n}=\left\{t_{0}^{T_{n}},\cdots,t_{j}^{T_{n}},\cdots,t_{J}^{T_{j}}\right\}\). Each individual utterance \(t_{j}^{T_{n}}\) is authored by one of the \(L\) users in the dataset denoted by \(u\in\left\{u_{0},\cdots,u_{l},\cdots,u_{L}\right\}\). The primary aim of CoSyn is to classify each utterance \(t\) into its label \(y^{gt}\in\left\{0,1\right\}\) where 0 and 1 indicate whether the utterance is hateful or not. Additionally, each hateful utterance (\(y_{i}^{gt}=1\)) is labeled with \(y^{ip}\in\left\{0,1\right\}\) where 0 and 1 indicate whether the hateful utterance is explicitly or implicitly hateful. \(y^{ip}\) is used only for analysis. In the following subsections, "user" refers to the author of an utterance in a conversation tree to be assessed.

### Hyperbolic Geometry: Background

#### Hyperbolic Geometry.

A Riemannian manifold is a \(D\)-dimensional real and smooth manifold defined with an inner product on tangent space \(g_{\mathbf{x}}:\mathcal{T}_{\mathbf{x}}\mathcal{M}\times\mathcal{T}_{\mathbf{x}}\mathcal{ M}\rightarrow\mathbb{R}\) at each point \(\mathbf{x}\in\mathcal{M}\), where the tangent space \(\mathcal{T}_{x}\mathcal{M}\) is a \(D\)-dimensional vector space representing the first-order local approximation of the manifold around a given point \(\mathbf{x}\). A Poincare ball manifold is a hyperbolic space defined as a constant negative curvature Riemannian manifold, denoted by \(\big{(}\mathbb{H}^{D},g\big{)}\) where manifold \(\mathbb{H}^{D}=\big{\{}\mathbf{x}\in\mathbb{R}^{D}:\|\mathbf{x}\|<1\big{\}}\) and the Riemannian metric is given by \(g_{\mathbf{x}}=\lambda_{\mathbf{x}}^{2}g^{E}\) where \(g^{E}=\mathbf{I}_{D}\) denotes the identity matrix, i.e., the Euclidean metric tensor and \(\lambda_{\mathbf{x}}=\frac{1}{1-\||\mathbf{x}\||\|^{2}}\). To perform operations in the hyperbolic space, the exponential map \(\exp_{\mathbf{x}}:\mathcal{T}_{\mathbf{x}}\mathbb{H}^{D}\rightarrow\mathbb{H}^{D}\) and the logarithmic map \(\log_{\mathbf{x}}:\mathbb{H}^{D}\rightarrow\mathcal{T}_{\mathbf{x}}\mathbb{H}^{D}\) are used to project Euclidean vectors to the hyperbolic space and vice versa respectively.

\[\exp_{x}(v):=x\oplus\bigg{(}\tanh\bigg{(}\frac{\lambda_{x}\|v\|}{2}\bigg{)} \frac{v}{\|v\|}\bigg{)} \tag{1}\]

\[\log_{x}(y):=\frac{2}{\lambda_{x}}\tanh^{-1}(\|-x\oplus y\|)\frac{-x\oplus y }{\|-x\oplus y\|} \tag{2}\]

where \(\mathbf{x}\), \(\mathbf{y}\in\mathbb{H}^{D}\), \(\mathbf{v}\in\mathcal{T}_{\mathbf{x}}\mathbb{H}^{D}\). Further, to perform operations in the Hyperbolic space, the following basic hyperbolic operations are described below:

#### Mobius Addition \(\oplus\)

adds a pair of points \(x,y\in\mathbb{H}^{D}\) as,

\[x\oplus y:=\frac{\left(1+2\langle x,y\rangle+\|y\|^{2}\right)x+\left(1-\|x\|^ {2}\right)y}{1+\langle x,y\rangle+\|x\|^{2}\|y\|^{2}} \tag{3}\]

#### Möbius Multiplication \(\otimes\)

multiplies vectors \(x\in\mathbb{H}^{D}\) and \(W\in\mathbb{R}^{D^{\prime}\times D}\) given by,

\[\mathbf{W}\otimes x:=\exp_{o}\left(\mathbf{W}\log_{0}(x)\right) \tag{4}\]

#### Mobius Element-wise Multiplication \(\odot\)

performs element-wise multiplication on \(x,y\in\mathbb{H}^{D}\),

\[x\odot y:=\tanh\bigg{(}\frac{\|xy\|}{y}\tanh^{-1}(\|y\|)\bigg{)}\frac{\|xy\|}{ \|y\|} \tag{5}\]

#### Hyperbolic Distance

between points \(\mathbf{x},\mathbf{y}\in\mathbb{H}^{D}\) is given by:

\[d_{\text{B}}:=2\tanh^{-1}(\|-x\oplus y\|) \tag{6}\]

#### Fourier Transform.

The Fourier transform breaks down a signal into its individual frequency components. When applied to a sequence \(x_{n}\) with \(n\in[0,N-1]\), the Discrete Fourier Transform (DFT) generates a new representation \(X_{k}\) for each value of k, where \(0\leq k\leq N-1\). DFT accomplishes this by computing a sum of the original input tokens \(x_{n}\), multiplied by twiddle factors \([e^{-2\pi ikln/N}]\), where \(n\) is the index of each token in the sequence. Thus, DFT is expressed as 

\[X_{k}=\sum_{n=0}^{N-1}x_{n}e^{-2\pi ikn/N},\quad 0\leq k\leq N-1 \tag{7}\]

### Bias-invariant Encoder Training

CoSyn, like other works in literature, builds on the assumption that linguistic signals from text utterances can effectively enable the detection of hate speech in online conversations. Thus, our primary aim is to learn an encoder \(\mathrm{e}(.)\) that can effectively generate vector representations \(\mathbb{R}^{d}\) for a text utterance where \(d\) is the dimension of the vector. Specifically, we fine-tune a SentenceBERT Reimers and Gurevych (2019) and solve an additional loss proposed in Mathew et al. (2021) using self-attention maps and ground-truth hate spans. Keyword bias is a long-standing problem in hate speech classification, and solving the extra objective promotes learning bias-invariant sentence representations.

### Modeling Personal User Context

Modeling personal user context, in the form of author profiling, for hate speech classification has shown great success in the past because hateful users (users prone to making hate utterances) share common stereotypes and form communities around them. They exhibit strong degrees of _homophily_ and have high reciprocity values Mathew et al. (2019). We hypothesize that this will prove to be especially useful in classifying _conversational implicit hate speech_, which on its own lacks clear lexical signals or any form of background knowledge. Thus, our primary aim is to learn a vector representation \(\mathcal{U}_{u}\in\mathbb{R}^{d}\) for the user \(u\) who has authored the utterance \(t\) to be assessed, where \(d\) is the dimension of the vector. For our work, we also explore the importance of the personal historical context of a user to enable better author profiling. Studies show that past social engagement and linguistic styles of their utterances on social media platforms play an important role in assessing the user's ideology and emotions Xiao et al. (2020). Thus we propose a novel methodology for author profiling that is more intuitive, explainable, and effective for our task. Our primary aim is to model the user's (author of the utterance to be assessed) personal context and generate user representations \(\mathcal{U}_{u}\), which can then be used for hate speech classification. To achieve this, we first encode a user's historical utterances \(\mathcal{U}_{u}^{hist}\) using our Hyperbolic Fourier Attention Network (HFAN) followed by modeling the user's social context by passing \(\mathcal{U}_{u}^{hist}\) through a Hyperbolic Graph Convolution Network (HGCN). To be precise, learning personal user context takes the form of \(\mathrm{e}(HistoricalUTterances)\)\(\rightarrow\) HFAN \(\rightarrow\)\(\mathcal{U}_{ut}^{hist}\)\(\rightarrow\) HGCN \(\rightarrow\)\(\mathcal{U}_{u}\). We next describe, in detail, HFAN and HGCN.

### Hyperbolic Fourier Attention Network

User engagement on social media is often diverse and possesses scale-free properties Sigerson and Cheng (2018). Thus to account for the natural irregularities and effectively model a user's personal historical context, we propose a novel Hyperbolic Fourier Attention Network (**HFAN**). The first step is to encode historical utterances, \(H^{u}\), using our encoder \(\mathrm{e}(.)\), made by the user \(u\), denoted by \(H^{u}\in\{h_{0}^{u},\cdots,h_{s}^{u},\cdots,h_{S}^{u}\}\). Next, we apply 2D DFT to the embeddings (1D DFT along the temporal dimension and 1D DFT along the embedding dimension) to capture the most commonly occurring frequencies (ideologies, opinions, emotions, etc.) in the historical utterances made by the user using:

\[\mathcal{U}_{u}^{fourier}=\exp_{6}^{c}\left(\mathcal{F}_{\text{s}}\left( \mathcal{F}_{\text{h}}\left(\log_{0}^{c}\left(e(H^{u})\right)\right)\right)\right) \tag{8}\]

where \(\mathcal{F}\) is the DFT operation. The 2D DFT operation helps highlight the most prominent frequencies, which signifies a holistic understanding of the user's sociological behavior. Next, we hypothesize that the frequencies most relevant to a conversation can act as the most important clues to

Figure 2: Various properties of the social graph and conversation trees averaged across datasets. The fitting of the node distribution in the power law \((\gamma\in[2,3])\) Choromanski et al. (2013) and low hyperbolicity Barabási and Bonabeau (2003)\(\delta\) indicates the scale-free nature of conversations and social graphs.

 

assessing how a user would react to other utterances in the conversation tree (for e.g., a user's stance on the contribution of a particular political party to a recent riot in discussion). Additionally, these frequencies may change over time. Thus, to account for the latter factor first, we pass the embeddings through a Hyperbolic GRU Zhang and Gao (2021), which first projects these embeddings from the Euclidean to the hyperbolic space using a Poincare ball manifold and then effectively captures the sequential temporal context on time-aligned historic user utterances. The Hyperbolic GRU is a modified version of the conventional GRU, which performs Mobius operations on the Poincare model (addition, multiplication, and bias). We request our readers to refer to Zhang and Gao (2021) for more details. Next, to account for the former factor, we perform Hyperbolic Attention Zhang and Gao (2021), which first linearly projects the utterance embeddings within Poincare space and then constructs the final user embedding \({\cal U}_{u}^{hist}\) via Einstein midpoint:

\[\alpha_{i}=\exp\left(-\beta_{h}d_{\mathcal{L}}\left({\bf c}_{h\mathcal{L}},{ \bf h}_{s\mathcal{L}}^{u}\right)-c_{h}\right) \tag{9}\]

\[{\cal U}_{u}^{hist}=\sum_{S}\Bigg{[}\frac{\alpha_{i}\gamma\left({\bf h}_{i \mathcal{K}}^{u}\right)}{\sum_{l}\alpha_{l}\gamma\left({\bf h}_{i\mathcal{K}} ^{u}\right)}\Bigg{]}\,{\bf h}_{i\mathcal{K}}^{u}, \tag{10}\]

where \({\bf h}_{s\mathcal{L}}\) and \({\bf h}_{s\mathcal{K}}\) are the utterance encoding of utterance \(h_{s}\) obtained from \({\cal U}_{u}^{fourier}\) projected from the Poincare space into the Lorentz space and Klein space for computational convenience Zhang and Gao (2021). \({\bf c}_{ss\mathcal{L}}\) is the sentence centroid, which is randomly initialized and learnable.

### Hyperbolic Graph Convolutional Network

After obtaining the user representations \({\cal U}_{u}^{hist}\) for every user \(u\) in the dataset, we model social relationships between the users as a graph \(\mathcal{G}\) = (\(\mathcal{V}\), \(\mathcal{E}\)). Each edge \(e=\{u_{i},u_{j}\}\in\mathcal{E}\) represents one of the four types of relations: 1) User \(u_{i}\) retweets a tweet posted by \(u_{j}\), 2) User \(u_{i}\) mentions user \(u_{j}\) in a tweet \(t_{t}\), 3) User \(u_{i}\) replies to a post by user \(u_{j}\) or 4) User \(u_{i}\) follows user \(u_{j}\). Each vertex \(v\in\mathcal{V}\) represents the user representations \({\cal U}_{u}\). **HGCN**Chami et al. (2019) modifies the conventional GCN and performs neighbor aggregation using graph convolutions in the hyperbolic space to enrich a user's historical context representations learned through HFAN using social context. Social network connections between users on a platform often possess hierarchical and scale-free structural properties (degree distribution of nodes follows the power law as seen in Fig 2 and decreases exponentially with a few nodes having a large number of connections Barabasi and Bonabeau (2003)). To model such complex hierarchical representations, HGCN performs all operations in the Poincare space. We first project our representations \(\mathcal{U}\) onto the hyperbolic space using a Poincare ball manifold (\(exp^{K}\)(.)) with a sectional curvature -1/K. Formally, the feature aggregation based at the \(i^{th}\) HGCN layer is denoted by:

Figure 3: Illustration of **CoSyn**. CoSyn detects if a target utterance, authored by a social media user, implies hate or not leveraging three main components: (1) **HFAN**: This models the user’s personal historical context by employing Fourier Transform and Hyperbolic Attention on the user’s historical tweets. (2) **HGCN**: This models the user’s social context through hyperbolic graph learning on its relations with other users. (3) **CSHT**: This jointly models the user’s personal context and the conversational context to finally classify if the utterance is hateful.

 

\[{\bf O}^{(i)}=\sigma^{\otimes^{K_{i-1},K_{i}}}\left({\rm FM} \left(\tilde{\bf A}\hskip 0.5pt{\bf O}^{(i-1)}{\bf W}^{(i)}\right)\right) \tag{11}\]

where -I/\(K_{i-1}\) and -I/\(K_{i}\) are the hyperbolic curvatures at layers \(i\) - 1 and \(i\), respectively. \(\tilde{\bf A}\) = \(D^{-1/2}AD^{-1/2}\) is the degree normalized adjacency matrix, \(W\) is a trainable network parameter , \({\bf O}^{i}\) is the output of the \(i^{th}\) layer and \({\rm FM}\) is the Frechet Mean operation. \(\sigma^{\otimes^{K_{i-1},K_{i}}}\) is the hyperbolic nonlinear activation allowing varying curvature at each layer.

### Context-Synergized Hyperbolic Tree-LSTM

To model the conversational context in conversation trees effectively, we propose Contextual Synergized Hyperbolic Tree-LSTM (**CSHT**). CSHT presents several modifications and improvements over Tree-LSTM Tai et al. (2015), including (1) incorporating both the user's personal context and the conversation context while clearly capturing the interactions between them, and (2) operating in the hyperbolic space, unlike the original TreeLSTM, which operates in the Euclidean space. Conversation trees on social media possess a hierarchical structure of message propagation, where certain nodes may have many replies; e.g., nodes that include utterances from popular users. Such phenomena lead to the formation of hubs within the conversation tree, which indicates scale-free and asymmetrical properties of the conversation tree Avin et al. (2018). The conversation tree is \(T\) represented using \(\mathcal{T}\) = (\(\mathcal{V}\), \(\mathcal{E}\)), where vertex \(v\in\mathcal{V}\) represents the encoded utterance \(\mathcal{X}_{j}\) = \({\rm e}(t_{j})\) (\(t\) is part of the conversation tree \(T\)) and edge \(e\in\mathcal{E}\) represents one of the three relations between the utterances: (1) parent-comment, (2) comment-reply or (3) replyly. CSHT is modeled as a Bi-Directional Tree-LSTM where \(\mathcal{U}_{u}\) is the user representation of the user \(u\) who authored the utterance to be assessed obtained from the HGCN.

In order to understand the signals in CHST, we focus on a specific node \(t_{j}\). We gather information from all input nodes \(t_{k}\) where \(\{t_{k},t_{j}\}\in\mathcal{E}\), and combine their hidden states \(h_{k}\) using Einstein's midpoint to produce an aggregated hidden state \(\widetilde{h_{j}}\) for node \(t_{j}\). We use the hyperbolic representation of the current post, denoted as \(\mathcal{X}_{j}\), as well as the user embeddings of the author of the post, denoted as \(\mathcal{U}_{u}\), to define computational gates operating in the hyperbolic space for CSHT cells. As defined in Subsection2.2\(\odot,\oplus,\otimes\) represent Mobius Dot Product, Mobius Addition and Mobius matrix multiplication, respectively.

Considering multiple input nodes \(t_{k},\{t_{k},t_{j}\}\in\mathcal{E}\), we implement multiple hyperbolic forget gates incorporating the outputs \(h_{k}\) of the input nodes with the current node's combined user and post representation \(r_{j}\).

\[\begin{gathered} r_{j}={\bf W}^{({\tt fx})}\otimes \mathcal{X}_{j}\oplus{\bf W}^{({\rm fg})}\otimes\mathcal{U}_{u},\\ f_{jk}=\exp_{o}\Big{(}\sigma\left(\log_{o}\Big{(}r_{j}\oplus{ \bf U}^{({\tt f})}\otimes h_{k}\oplus{\bf b}^{({\tt f})}\Big{)}\right)\!\Big{)} \end{gathered} \tag{12}\]

The input gate \(i_{j}\) and the intermediate memory gate \(u_{j}\) corresponding to the representation for the current utterance are given by:

\[\begin{gathered} i_{j}=\exp_{o}\sigma\left(\log_{o}\Big{(}{\bf W }^{({\tt i})}\otimes\mathcal{X}_{j}\oplus{\bf U}^{({\tt i})}\otimes\\ \tilde{h_{j}}\oplus{\bf b}^{({\tt i})}\Big{)}\right)\\ u_{j}=\exp_{o}\big{(}\tanh\big{(}\log_{o}\big{(}{\bf W}^{({\tt u })}\otimes\mathcal{X}_{j}\oplus{\bf U}^{({\tt u})}\otimes\\ \widetilde{h_{j}}\oplus{\bf b}^{({\tt u})}\big{)}\big{)}\bigg{)} \end{gathered} \tag{13}\]

The input gate \(m_{j}\) and the intermediate memory gate \(s_{j}\) corresponding to the user representation from the social graph are given by:

\[\begin{gathered} m_{j}=\exp_{o}\Big{(}\sigma\left(\log_{o}\Big{(}{ \bf W}^{({\tt m})}\otimes\mathcal{U}_{u}\oplus\\ {\bf U}^{({\tt m})}\otimes\widetilde{h_{j}}\oplus{\bf b}^{({\tt m })}\Big{)}\Big{)}\right)\\ s_{j}=\exp_{o}\Big{(}\tanh\Big{(}\log_{o}\Big{(}{\bf W}^{({\tt s })}\otimes\mathcal{U}_{u}\oplus\\ {\bf U}^{({\tt s})}\otimes\widetilde{h_{j}}\oplus{\bf b}^{({\tt s })}\Big{)}\Big{)}\Big{)}\end{gathered} \tag{14}\]

The output gate for the tree cell is then calculated as follows:

\[\small o_{j}=\exp_{o}\Big{(}\sigma\left(\log_{o}\Big{(}{\bf W}^{({\tt o})} \otimes\mathcal{U}_{u}\oplus{\bf U}^{({\tt o})}\otimes\widetilde{h_{j}}\oplus {\bf b}^{({\tt o})}\Big{)}\right)\!\Big{)} \tag{16}\]

The parameters \({\bf W}^{({\bf w})},{\bf U}^{({\tt w})},{\bf b}^{({\tt w})}\) are learnable parameters in the respective gate \({\bf w}\). We obtain the cell state for the current cell \(c_{j}\) by combining the representations from the multiple forget gates \(f_{jk}\), \(k\forall t_{k},s.t.\{t_{k},t_{j}\}\in\mathcal{E}\), as well as gates corresponding to the tweet and user representations as follows:

\[c_{j}=i_{j}\odot u_{j}\oplus m_{j}\odot s_{j}\oplus\sum_{k}f_{jk}\odot c_{k} \tag{17}\]

These equations are applied recursively by blending in tree representations at every level. The output of the current cell, \(h_{j}\) is given by 

\[{\rm CHST}(\mathcal{X}_{j},\mathcal{U}_{j})=h_{j}=o_{j}\odot \exp_{o}\left(\tanh\left(c_{j}\right)\right) \tag{19}\]

Final Prediction Layer.We concatenate the node output from CSHT \(h_{j}\), projected to the Euclidean space, with the Euclidean utterance features for the given tweet, \(\mathcal{X}_{j}\) to form a robust representation incorporating features of the tweet along with social and conversational context derived from the CSMT network of CoSyn. This concatenated representation is passed through a final classification layer to obtain the final prediction \(y^{gt}_{j}={\rm Softmax}\left({\rm MLP}\left(\left[\log_{o}\left(h_{j}\right); \mathcal{X}_{j}\right]\right)\right)\). For training, we optimize a binary cross-entropy loss.

## 3 Experiments and Results

### Dataset

We evaluate CoSyn on 6 conversational hate speech datasets; namely, Reddit Qian et al. (2019), GAB Qian et al. (2019), DIALOCONAN Bonaldi et al. (2022), CAD Vidgen et al. (2021), ICHCL Modha et al. (2021) and Latent Hatred ElSherief et al. (2021). Appendix A provides dataset statistics. Table 1 reports the micro-F\({}_{1}\) scores averaged across 3 runs with 3 different random seeds.

**Implicit Hate Annotations.** The original datasets do not have annotations to indicate if an utterance denotes implicit or explicit hate speech. Thus, for our work, we add extra annotations for evaluating how CoSyn fares against our baselines in understanding the context for detecting implicit hate speech. Specifically, we use Amazon Mechanical Turk (MTurk) and ask three individual MTurk workers to annotate if a given utterance conveys hate implicitly or explicitly. The primary factor was the requirement of conversational context to understand the conveyed hate. At stage (1), we provide them with the definition of hate speech and examples of explicit and implicit hate speech. At stage (2), we provide complete conversations and ask them to annotate implicit or explicit in a binary fashion. Cohen's Kappa Scores for inter-annotator agreement are provided with the dataset details in Appendix A.

### Baselines

Due to the acute lack of prior work in this space, beyond just hate speech classification, we also compare our method with systems proposed in the thriving fake news detection literature, which considers conversational and user contexts to detect fake news. Some of these systems had to be adapted to our task, and we describe about the working of each baseline in detail in Appendix B. Specifically, we compare CoSyn with Sentence-BERT Reimers Gurevych (2019), ConceptNet, HASOC Farooqi et al. (2021), Conc-Perspective Pavlopoulos et al. (2020), CSI Ruchansky et al. (2017), GCAN Lu and Li (2020), HYPHEN Grover et al. (2022), FinerFact Jin et al. (2022), GraphNLI Agarwal et al. (2022), DUCK Tian et al. (2022), MRIL Sawhney et al. (2022a) and Madhu Madhu et al. (2023).

### Hyperparameters

We decide on the best hyper-parameters for CoSyn based on the best validation set performance using grid search. The optimal hyperparameters are found to be, batch-size \(b=32\), HGCN output embedding dimension \(g=512\), hidden dimension of CHST \(h=768\), latent dimension of HFAN \(l=100\), learning rate \(lr=1.3e^{-2}\), weight decay \(\beta=3.2e^{-4}\), and dropout rate \(\delta=0.41\).

### Results and Ablations

Table 1 compares the performance of CoSyn with all our baselines on 6 hate speech datasets. As we see, Cosyn outperforms all our baselines both on the entire dataset and on the implicit subset, thus showing its effectiveness for implicit hate speech detection. Despite the ambiguous nature of implicit hate speech, CoSyn has high precision scores for implicit hate speech, thus implying that it mitigates the problem of false positives better than other baselines. One common observation is that our bias invariant SentenceBERT approach emerges as the most competitive baseline to CoSyn, thereby reinforcing that most prior work do not effectively leverage external context.

When evaluated on the entire dataset, CoSyn achieves absolute improvements in the range of 5.1% - 35.2% on Reddit, 4.0% - 45.0% on CAD, 4.8% - 22.4% on DIALOCONAN, 3.9% - 42.7% on GAB, 9.7% - 38.4% on ICHCL and 5.8% - 31.6% on Latent Hatred ober our baselines. When evaluated on the implicit subsets, CoSyn achieves absolute improvements in the range of 5.1% - 57.9% on Reddit, 8.6% - 31.3% on CAD, 18.2% - 40.7% on DIALOCONAN, 2.9% - 28.5% on GAB, 8.2% - 19.5% on ICHCL and 1.2% - 18.1% on Latent Hatred.

Table 2 ablates the performance of CoSyn, removing one component at a time to show the significance of each. All results have been averaged



across all 6 datasets. Some major observations include (1) CoSyn sees a steep drop in performance when used without user context (we model only the conversation context with a vanilla Tree-LSTM in the hyperbolic space). This proves the effectiveness of the user's personal context. Additionally, modeling user context with HFAN and HGCN proves to be more effective than just feeding mean-pooled historical utterance embeddings into CSHT. (2) Modeling in the Euclidean space results in an overall F1 drop of 3.8%. We reinforce the effectiveness of modeling in the hyperbolic space through our findings in Fig. 2.

### Results Analysis

In Fig. 4, we show test set predictions from 4 different conversation trees from the ICHCL dataset with an attempt to study the effect of conversational and author context for target utterance classification. We notice that hateful users possess high degrees of homophily and are predominantly hateful over time. It also reveals a limitation of CoSyn where insufficient context from the parent leads to a false positive (comment in the 4th example).

## 4 Related Work

Prior research on identifying hate speech on social media has proposed systems that can flag online hate speech with remarkable accuracy Schmidt and Wiegand (2019). However, as mentioned earlier, prior efforts have focused primarily on classifying overt abuse or explicit hate speech Schmidt and Wiegand (2019) with little or no work on classifying implicit hate speech or that conveyed in coded language. The lack of research on this topic can also be attributed to existing datasets being skewed towards _explicitly_ abusive text Waseem and Hovy (2016). Recently, this area of research has seen growing interest, with several datasets and benchmarks released for evaluating the performance of existing hate speech classifiers in identifying implicit hate speech Caselli et al. (2020); ElSherief et al. (2021); Sap et al. (2019). One common observation is that most prior systems from literature fail to identify implicit hate utterances ElSherief et al. (2021). Lin _et al._Lin (2022) proposes one of the first systems to classify implicit hate speech leveraging world knowledge. However, they evaluate their performance on only _Latent Hatred_, which lacks conversational context. Additionally, acquiring

\begin{table} width=center 

\end{table}
Table 1: Result comparison of CoSyn with our baselines on 6 hate speech datasets. We compare performance on both the overall dataset and the implicit subset. C.F\({}_{1}\) and R.F\({}_{i}\) indicate F\({}_{1}\) scores measured on only comments and replies, respectively. CoSyn outperforms all our baselines with absolute improvements in the range of 3.4% - 45.0% when evaluated on the entire dataset and 1.2% - 75.9% when evaluated on only the implicit subset. – indicates conversation trees in the dataset did not have triples.

\begin{table} width=center 

\end{table}
Table 2: Ablation study on Cosyn. Results are averaged across all 6 datasets.

 

world knowledge through knowledge graphs (KGs) requires language-specific KGs, and short utterances in conversation trees make the retrieval weak. In the past, to classify context-sensitive hate speech in existing open-source datasets, researchers incorporated conversational context for hate speech classification Gao and Huang (2017); Perez et al. (2022). However, these systems employ naive fusion and fail to leverage structural dependencies between utterances in the conversation tree. Another line of work explores author profiling using community-based social context (connections a user has with other users) Pavlopoulos et al. (2017); Mishra et al. (2018). However, the representations are not learned end-to-end and employ naive fusion.

Hyperbolic networks have been explored earlier for tasks that include modeling user interactions in social media, like suicide ideation detection Sawhney et al. (2021b, 2022b), fake news detection Grover et al. (2022), online time stream modeling Sawhney et al. (2021a), etc. All these works show that modeling the hierarchical and scale-free nature of social networks and data generated online benefits from modeling in the hyperbolic space over Euclidean space.

## 5 Conclusion

In this paper, we present CoSyn, a novel learning framework to detect implicit hate speech in online conversations. CoSyn jointly models the conversational context and the author's historical and social context in the hyperbolic space to classify whether a target utterance is hateful. Leveraging these contexts allows CoSyn to effectively detect implicit hate speech ubiquitous in online social media.

### Acknowledgement

This work was supported by ARO grants W911NF2310352 and W911NF2110026.

### Limitations

In this section, we list down some potential limitations of CoSyn:

1. Lacking world knowledge is one of CoSyn's potential limitations. The inclusion of world knowledge could serve as a crucial context, enhancing CoSyn's performance in this task Sheth et al. (2022). We would like to explore this as part of future work.
2. Table 2 clearly demonstrates that CoSyn's effectiveness relies on the cohesive integration

Figure 4: We study 4 conversation trees in the ICHCL dataset, including the prediction of different classifiers on the utterance to be assessed, the historical engagement of the author of the utterance, and the social relations between the different authors.

 

of all its components. Therefore, as a direction for future research, our focus will be on enhancing the performance of individual components.

## Ethics Statement

Our institution's Institutional Review Board (IRB) has granted approval for this study. In the annotation process, we took precautions by including a cautionary note in the instructions, alerting annotators to potentially offensive or distressing content. Annotators were also allowed to discontinue the labeling process if they felt overwhelmed.

Additionally, in light of the rapid proliferation of offensive language on the internet, numerous A0-based frameworks have emerged for hate speech detection. Nevertheless, a significant drawback of many current hate speech detection models is their narrow focus on explicit or overt hate speech, thereby overlooking the identification of implicit expressions of hate that hold equal potential for harm. CoSyn could ideally identifying implicit hate speech with remarkable accuracy, preventing targeted communities from experiencing increased harm online.

## References

* Agarwal et al. (2022) Vibhor Agarwal, Sagar Joglekar, Anthony P Young, and Nishanth Sastry. 2022. Graphnl: A graph-based natural language inference model for polarity prediction in online debates. In _Proceedings of the WWW 2022_, pages 2729-2737.
* Avin et al. (2018) Chen Avin, Zvi Lotker, David Peleg, Yvonne-Anne Pignolet, and Itzik Turkel. 2018. Elites in social networks: An axiomatic approach to power balance and price's square root law. _PloS one_, 13(10):e0205820.
* Barabasi and Bonabeau (2003) Albert-Laszlo Barabasi and Eric Bonabeau. 2003. Scale-free networks. _Scientific american_, 288(5):60-69.
* Bonaldi et al. (2022) Helena Bonaldi, Sara Dellantonio, Serra Sinem Tekiroglu, and Marco Guerini. 2022. Human-machine collaboration approaches to build a dialogue dataset for hate speech countering. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 8031-8049, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Caselli et al. (2020) Tommaso Caselli, Valerio Basile, Jelena Mitrovic, Inga Kartoziya, and Michael Granitzer. 2020. I feel offended, don't be abusive! implicit/explicit messages in offensive and abusive language. In _Proceedings of the 12th LREC_.
* Chami et al. (2019) Ines Chami, Zhitao Ying, Christopher Re, and Jure Leskovec. 2019. Hyperbolic graph convolutional neural networks. _Advances in neural information processing systems_, 32.
* Choromanski et al. (2013) Krzysztof Choromanski, Michal Matuszak, and Jacek Miekisz. 2013. Scale-free graph with preferential attachment and evolving internal vertex structure. _Journal of Statistical Physics_, 151(6):1175-1183.
* Cooley and Tukey (1965) James W Cooley and John W Tukey. 1965. An algorithm for the machine calculation of complex fourier series. _Mathematics of computation_, 19(90):297-301.
* Denigot and Burnett (2020) Quentin Denigot and Heather Burnett. 2020. Dogwhistles as identity-based interpretative variation. In _Proceedings of the Probability and Meaning Conference (PaM 2020)_, pages 17-25. ACL.
* ElSherief et al. (2021) Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupandi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021. Latent hatred: A benchmark for understanding implicit hate speech. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 345-363, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Farooqi et al. (2021) Zakai Mustafa Farooqi, Sreyan Ghosh, and Rajiv Ratn Shah. 2021. Leveraging transformers for hate speech detection in conversational code-mixed tweets. _arXiv preprint arXiv:2112.09986_.
* Fortuna and Nunes (2018) Paula Fortuna and Sergio Nunes. 2018. A survey on automatic detection of hate speech in text. _ACM Computing Surveys (CSUR)_, 51(4):1-30.
* Gao and Huang (2017) Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context aware models. _arXiv preprint arXiv:1710.07395_.
* Garg et al. (2022) Tanmay Garg, Sarah Masud, Tharum Suresh, and Tammoy Chakraborty. 2022. Handling bias in toxic speech detection: A survey. _arXiv preprint arXiv:2202.00126_.
* Grover et al. (2022) Karish Grover, SM Angara, Md Akhtar, Tanmoy Chakraborty, et al. 2022. Public wisdom matters! discourse-aware hyperbolic fourier co-attention for social-text classification. _arXiv preprint arXiv:2209.13017_.
* Gubler and Kalmoe (2015) Joshua R Gubler and Nathan P Kalmoe. 2015. Violent rhetoric in protracted group conflicts: Experimental evidence from israel and india. _Political Research Quarterly_, 68(4):651-664.
* Hebert et al. (2022) Liam Hebert, Lukasz Golab, and Robin Cohen. 2022. Predicting hateful discussions on reddit using graph transformer networks and communal context. In _2022 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT)_, pages 9-17 

Yiqiao Jin, Xiting Wang, Ruichao Yang, Yizhou Sun, Wei Wang, Hao Liao, and Xing Xie. 2022. Towards fine-grained reasoning for fake news detection. In _Proceedings of the AAAI_, volume 36, pages 5746-5754.
* Jurgens et al. (2019) David Jurgens, Libby Hemphill, and Eshwar Chandrasekharan. 2019. A just and comprehensive strategy for using NLP to address online abuse. In _Proceedings of the 57th Annual Meeting of the ACL_, pages 3658-3666. ACL.
* Maynard and Benesch (2016) Jonathan Leader Maynard and Susan Benesch. 2016. Dangerous speech and dangerous ideology: An integrated model for monitoring and prevention. _Geoncide Studies and Prevention_, 9(3).
* Lin (2022) Jessica Lin. 2022. Leveraging world knowledge in implicit hate speech detection. _arXiv preprint arXiv:2212.14100_.
* Lu and Li (2020) Yi-Ju Lu and Cheng-Te Li. 2020. Gcan: Graph-aware co-attention networks for explainable fake news detection on social media. _arXiv preprint arXiv:2004.11648_.
* Madhu et al. (2023) Hirem Madhu, Shrey Satapara, Sandip Modha, Thomas Mandl, and Prasenjit Majumder. 2023. Detecting offensive speech in conversational code-mixed dialogue on social media: A contextual dataset and benchmark experiments. _Expert Systems with Applications_, 215:119342.
* Magu and Luo (2018) Rijul Magu and Jiebo Luo. 2018. Determining code words in euphemistic hate speech using word embedding networks. In _Proceedings of the 2nd Workshop on Abusive Language Online (ALW2)_, pages 93-100. ACL.
* Mathew et al. (2019) Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019. Spread of hate speech in online social media. In _Proceedings of the 10th ACM WWW_, pages 173-182.
* Mathew et al. (2021) Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In _Proceedings of the AAAI_, volume 35, pages 14867-14875.
* Mishra et al. (2018) Pushkar Mishra, Marco Del Tredici, Helen Yannakoudakis, and Ekaterina Shutova. 2018. Author profiling for abuse detection. In _Proceedings of the 27th ICCL_, pages 1088-1098. ACL.
* Modha et al. (2022) Sandip Modha, Thomas Mandl, Prasenjit Majumder, Shrey Satapara, Tithi Patel, and Hiren Madhu. 2022. Overview of the hasoc subtrack at fire 2022: Identification of conversational hate-speech in hindi-english code-mixed and german language. _Working Notes of FIRE_.
* Modha et al. (2021) Sandip Modha, Thomas Mandl, Gautam Kishore Shahi, Hiren Madhu, Shrey Satapara, Tharindu Ranasinghe, and Marcos Zampieri. 2021. Overview of the hasoc subtrack at fire 2021: Hate speech and offensive content identification in english and indo-aryyan languages and conversational hate speech. In _Forum for Information Retrieval Evaluation_, pages 1-3.
* Ocampo et al. (2023) Nicolas Ocampo, Ekaterina Sviridova, Elena Cabrio, and Serena Villata. 2023. An in-depth analysis of implicit and subtle hate speech messages. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 1989-2005.
* Pavlopoulos et al. (2017) John Pavlopoulos, Prodromos Malakasiotis, Julii Bakagianni, and Ion Androutsopoulos. 2017. Improved abusive comment moderation with user embeddings. In _Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism_, pages 51-55. ACL.
* Pavlopoulos et al. (2020) John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithun Thain, and Ion Androutsopoulos. 2020. Toxicity detection: Does context really matter? _arXiv preprint arXiv:2006.00998_.
* Perez et al. (2022) Juan Manuel Perez, Franco Luque, Demian Zayat, Martin Kondratzky, Agustin Moro, Pablo Serrati, Joaquin Zahjac, Paula Miguel, Natalia Debandi, Agustin Gravano, et al. 2022. Assessing the impact of contextual information in hate speech detection. _arXiv preprint arXiv:2210.00465_.
* Piazza (2020) James A Piazza. 2020. Politician hate speech and domestic terrorism. _International Interactions_, 46(3):431-453.
* Qian et al. (2019) Jing Qian, Anna Bethke, Yinvin Liu, Elizabeth M. Belding-Royer, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In _Conference on Empirical Methods in Natural Language Processing_.
* Qian et al. (2018) Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018. Leveraging intra-user and inter-user representation learning for automated hate speech detection. In _NAACL-HLT 2018_, pages 118-123. ACL.
* Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. _arXiv preprint arXiv:1908.10084_.
* Ruchansky et al. (2017) Natali Ruchansky, Sungyoung Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In _Proceedings of the 2017 ACM on CIKM_, pages 797-806.
* Sanguinetti et al. (2018) Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Marco Stranisci. 2018. An italian twitter corpus of hate speech against immigrants. In _Proceedings of the eleventh LREC_.
* Sap et al. (2019) Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. 2019. Social bias frames: Reasoning about social and power implications of language. _arXiv preprint arXiv:1911.03891_ 

Ramit Sawhney, Shivam Agarwal, Atula T. Neerkaje, and Kapil Jayesh Pathak. 2022a. Orthogonal multi-manifold enriching of directed networks. In _Proceedings of The 25th International Conference on Artificial Intelligence and Statistics_, volume 151 of _Proceedings of Machine Learning Research_, pages 6074-6086. PMLR.
* Sawhney et al. (2022b) Ramit Sawhney, Shivam Agarwal, Atula Tejaswi Neerkaje, Nikolaos Aletras, Preslav Nakov, and Lucie Flek. 2022b. Towards suicide ideation detection through online conversational context. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_, SIGIR '22, page 1716-1727, New York, NY, USA. Association for Computing Machinery.
* Sawhney et al. (2021a) Ramit Sawhney, Shivam Agarwal, Megh Thakkar, Arnav Wadhwa, and Rajiv Ratn Shah. 2021a. Hyperbolic online time stream modeling. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_, SIGIR '21, page 1682-1686, New York, NY, USA. Association for Computing Machinery.
* Sawhney et al. (2021b) Ramit Sawhney, Harshti Joshi, Rajiv Ratn Shah, and Lucie Flek. 2021b. Suicide ideation detection via social and temporal user representations using hyperbolic learning. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2176-2190, Online. Association for Computational Linguistics.
* Schmidt and Wiegand (2017) Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In _Fifth International Workshop on Natural Language Processing for Social Media_, pages 1-10. ACL.
* Schmidt and Wiegand (2019) Anna Schmidt and Michael Wiegand. 2019. A survey on hate speech detection using natural language processing. In _Fifth International Workshop on Natural Language Processing for Social Media, April 3, 2017_, pages 1-10. ACL.
* Sheth et al. (2022)mit Sheth, Valerie L Shalin, and Ugur Kursuncu. 2022. Defining and detecting toxicity on social media: context and knowledge are key. _Neurocomputing_, 490:312-318.
* Sigerson and Cheng (2018) Leif Sigerson and Cecilia Cheng. 2018. Scales for measuring user engagement with social network sites: A systematic review of psychometric properties. _Computers in Human Behavior_, 83:87-105.
* Tai et al. (2015) Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. _arXiv preprint arXiv:1503.00075_.
* Tian et al. (2022) Lin Tian, Xiuzhen Jenny Zhang, and Jey Han Lau. 2022. Duck: Rumour detection on social media by modelling user and comment propagation networks. In _NAACL-HLT 2022_, pages 4939-4949.
* Tibuk and Newton (2015) Natasha Tibuk and Casey Newton. 2015. Twitter ceo:"we suck at dealing with abuse.. _The Verge_, 4(2):2015.
* Vidgen et al. (2021) Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, and Rebekah Tremble. 2021. Introducing cad: the contextual abuse dataset. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2289-2303.
* Warner and Hirschberg (2012) William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In _Proceedings of the Second Workshop on Language in Social Media_, pages 19-26. ACL.
* Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In _Proceedings of the NAACL Student Research Workshop_, pages 88-93. ACL.
* Wiegand et al. (2019) Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In _NAACL-HLT 2019_, pages 602-608. ACL.
* Xiao et al. (2020) Zhiping Xiao, Weiping Song, Haoyan Xu, Zhicheng Ren, and Yizhou Sun. 2020. Timme: Twitter ideology-detection via multi-task multi-relational embedding. In _Proceedings of the 26th ACM SIGKDD_, pages 2258-2268.
* Zhang and Gao (2021) Chengkan Zhang and Junbin Gao. 2021. Hype-han: Hyperbolic hierarchical attention network for semantic embedding. In _Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence_, pages 3990-3996.

## Appendix A Dataset Details

**Reddit.** The Reddit hate speech intervention dataset Qian et al. (2019) has 5,020 conversations, including 22,324 comments. On average, each conversation consists of 4.45 comments, and the length of each comment is 58.0 tokens. 5,257 comments are labeled hate speech, and 17,067 are labeled non-hatee speech. Most conversations, 3,847 (76.6%), contain hate speech. Each conversation with hate speech has 2.66 responses on average, for a total of 10,243 intervention responses. The average length of the intervention responses is 17.96 tokens. User history, the timestamp for each post, and the username for each post were fetched using the Reddit API2. Dataset statistics can be found in Table 3. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.78, 0.74, and 0.71.

Footnote 2: [https://www.reddit.com/dev/api/](https://www.reddit.com/dev/api/)

**GAB.** The GAB Qian et al. (2019) 11,825 conversations, consisting of 33,776 posts. On average, 

each conversation consists of 2.86 posts, and the average length of each post is 35.6 tokens. 14,614 posts are labeled hate speech, and 19,162 are labeled non-hate speech. Nearly all the conversations, 11,169 (94.5%), contain hate speech. 31,487 intervention responses were originally collected for conversations with hate speech, or 2.82 responses per conversation on average. The average length of the intervention responses is 17.27 tokens. User history was fetched using the GAB API3. The metadata for each fetched post provides information like the author and timestamp for a post. Dataset statistics can be found in Table 6. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.85, 0.82, and 0.91.

Footnote 3: [https://www.npmjs.com/package/gab-api](https://www.npmjs.com/package/gab-api)

**DIALOCONAN.** The DIALOCONAN dataset [1] has more than 3K dialogical interactions between two interlocutors, one acting as the hater and the other as the NGO operator, for a total of more than 16K turns. The previous tweets of a particular user were considered to be the history for that. Dataset statistics can be found in Table 7. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.79, 0.78, and 0.79.

**CAD.** The CAD dataset [13] is an annotated dataset of \(\approx\) 25,000 Reddit entries. The dataset is labeled with 6 conceptually different categories, including Identity-directed, Person-directed, Affiliation-directed, Counter Speech, Non-hateful Slurs, and Neutral. The dataset is also annotated with salient subcategories, such as whether personal abuse is directed at a person in the conversation thread or someone outside it. This taxonomy offers greater coverage and granularity of abuse than previous work. Each entry can be assigned to multiple primary and/or secondary categories. The dataset is also annotated in context, where each entry is annotated in the context of the conversational thread it is part of. Every annotation also has a label for whether contextual information was needed to make the annotation. User history, the timestamp for each post, and the username for each post were fetched using the Reddit API4. Dataset statistics can be found in Table 5. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.65, 0.73, and 0.69.

Footnote 4: [https://www.reddit.com/dev/api/](https://www.reddit.com/dev/api/)

**ICHCL.** The ICHCL dataset [14] (Identification of Conversational HateSpeech in Code-Mixed Languages) consists of hind-english code-switched Twitter conversations. The primary task is to identify comments and replies that can be considered acceptable when considered alone but may appear hateful, profane, or offensive when the context of a parent tweet is considered. Binary classification of such contextual posts was considered in this subtask. Around 7,000 code-mixed postings in English and Hindi were downloaded from Twitter and annotated in-house by the authors. User history was fetched using the Twitter API5. Dataset statistics can be found in Table 8. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.72, 0.81, and 0.74.

Footnote 5: [https://developer.twitter.com/en/docs/twitter-api](https://developer.twitter.com/en/docs/twitter-api)

**Latent Hatred.** The Latent Hatred dataset [1] is a hate speech dataset of 27K Gab messages annotated according to a 6-class taxonomy that includes White Grievance, Incitement to Violence, Inferiority Language, Irony, Stereotypes and Misinformation, and Threatening and Intimidation. Comments for every post, the user's user history, and the timestamp for each post were fetched using the Twitter API6. Dataset statistics can be found in Table 4. The Cohen's Kappa Scores for inter-annotator agreement for 3 pairs of annotators annotating for implicit hate speech were 0.91, 0.96, and 0.89.

Footnote 6: [https://developer.twitter.com/en/docs/twitter-api](https://developer.twitter.com/en/docs/twitter-api)

## Appendix B Baseline Descriptions

**Sentence-BERT w/ Classification Head** We use our bias-invariant Utterance Encoder trained and 

inferred on utterances in isolation for this baseline.

**ConceptNet** Similar to Sentence-BERT but fused with world knowledge from ConceptNet inspired from ElSherief et al. (2021); Lin (2022).

**HASOC**Farooqi et al. (2021) We use the system proposed by the winning solution in the HASOC 2021 challenge. The authors propose to train a transformer encoder by concatenating parentment-reply chains separated by the SEP token.

**Conc-Perspective**Pavlopoulos et al. (2020) Similar to HASOC, but context concatenation is done only during inference and not during training.

**CSI**Ruchansky et al. (2017) Capture, Score, and Integrate (CSI), originally proposed for fake news classification, implements a neural network to jointly learn the temporal pattern of user activity on a given utterance, and the user characteristic based on the behavior of users.

**GCAN**Lu and Li (2020) Graph-aware Co-Attention Network (GCAN), originally proposed for fake news classification, implements a neural network that uses the target utterance and its propagation-based user's features.

**HYPhen**Grover et al. (2022) HYPPHEN or Discourse-Aware Hyperbolic Fourier Co-Attention, proposed for social-text classification and incorporates conversational discourse knowledge with Abstract Meaning Representation graphs and employs co-attention in the hyperbolic space.

**FinerFact**Jin et al. (2022) FinerFact, originally proposed for fake news detection, incorporates a fine-grained reasoning framework by introducing a mutual reinforcement-based method for incorporating human knowledge and designs a prior-aware bi-channel kernel graph network to model subtle differences between pieces of evidence.

**GraphNLI**Agarwal et al. (2022) Graph-based Natural Language Inference Model (GraphNLI) proposes a graph-based deep learning architecture that effectively captures both the local and the global context in online conversation trees through graph-based walks.

**DUCK**Tian et al. (2022) Rumour detection with user and comment networks (DUCK), similar to CoSyn, employs graph attention networks to jointly model the contents and the structure of social media conversation trees, as well as the network of users who engage in these conversations.

## Appendix C Additional Details

**Model Parameters:** Sentence-BERT has \(\approx\) 110M parameters with 12-layers of encoder, 768-hidden-state, 2048 feed-forward hidden-state and 8-heads. CoSyn has \(\approx\) 18M parameters.

**Compute Infrastructure:** All our experiments are conducted on a single NVIDIA A100 GPU. An entire CoSyn training pipeline takes \(\approx\) 40 minutes.

**Implementation Software and Packages:** We implement all our models in PyTorch 7 and use the HuggingFace 8 implementations of SentenceBERT.

Footnote 7: [https://pytorch.org/](https://pytorch.org/)

Footnote 8: [https://huggingface.co/](https://huggingface.co/)

**Potential Risks:** CoSyn relies on training data that may contain biases inherent in the data sources.

\begin{table}
\begin{tabular}{p{39.8pt} p{25.6pt} p{25.6pt} p{25.6pt}|p{25.6pt} p{25.6pt} p{25.6pt}} \hline
**Split** & **Cours.** & **Comments** & **Replies** & **Hate** & **Non-Hate** & **Implicit** & **Explicit** \\ \hline Train & 13820 & 5264 & 6948 & 6088 & 12212 & 2435 & 3653 \\ Val & 6100 & 1789 & 2336 & 1975 & 4125 & 737 & 1238 \\ Test & 6100 & 1809 & 2315 & 1976 & 4124 & 741 & 1235 \\ \hline \end{tabular}
\end{table}
Table 6: Dataset statistics for the Gab dataset

\begin{table}
\begin{tabular}{p{39.8pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt}} \hline
**Split** & **Cours.** & **Comments** & **Replies** & **Hate** & **Non-Hate** & **Implicit** & **Explicit** \\ \hline Train & 12485 & 418 & – & 3835 & 8649 & 3246 & 590 \\ Val & 4162 & 83 & – & 1309 & 2853 & 1096 & 213 \\ Test & 4162 & 76 & – & 2273 & 1789 & 2138 & 235 \\ \hline \end{tabular}
\end{table}
Table 4: Dataset statistics for the Latent Hatred Dataset.

\begin{table}
\begin{tabular}{p{39.8pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt}} \hline
**Split** & **Cours.** & **Comments** & **Replies** & **Hate** & **Non-Hate** & **Implicit** & **Explicit** \\ \hline Train & 1382 & 3604 & 6755 & 2065 & 11317 & 1160 & 905 \\ Val & 4461 & 1155 & 2314 & 653 & 3808 & 388 & 265 \\ Test & 4461 & 1181 & 2280 & 706 & 3735 & 400 & 306 \\ \hline  & & & & & & & \\ \end{tabular}
\end{table}
Table 3: Dataset statistics for the Reddit dataset.

\begin{table}
\begin{tabular}{p{39.8pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt}} \hline
**Split** & **Cours.** & **Comments** & **Replies** & **Hate** & **Non-Hate** & **Implicit** & **Explicit** \\ \hline Train & 12485 & 418 & – & 3835 & 8649 & 3246 & 590 \\ Val & 4162 & 83 & – & 1309 & 2853 & 1096 & 213 \\ Test & 4162 & 76 & – & 2273 & 1789 & 2138 & 235 \\ \hline \end{tabular}
\end{table}
Table 3: Dataset statistics for the Reddit dataset.

\begin{table}
\begin{tabular}{p{39.8pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt} p{25.6pt}} \hline
**Split** & **Cours.** & **Comments** & **Replies** & **Hate** & **Non-Hate** & **Implicit** & **Explicit** \\ \hline Train & 13584 & 12704 & – & 2511 & 11073 & 1004 & 1507 \\ Val & 4526 & 4469 & – & 844 & 3692 & 375 & 459 \\ Test & 5307 & 4884 & – & 965 & 432 & 405 & 560 \\ \hline \end{tabular}
\end{table}
Table 5: Dataset statistics for the CAD dataset 

The detection system may inadvertently amplify or reinforce existing societal biases if these biases are not adequately addressed. For example, if the training data is biased towards specific demographics or ideologies, the model might exhibit unfair treatment or misclassification of certain groups, leading to potential discrimination or harm. Understanding the contextual nuances of language is a complex task. CoSyn might also be prone to over-generalization, potentially resulting in false positives or misclassification of non-hateful speech. The risk is that legitimate expressions of opinion or controversial yet non-hateful statements may be mistakenly flagged as hate speech. This could inadvertently lead to censorship or suppression of freedom of speech, limiting open dialogue and critical discussions on important social issues.