Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social Text Classification
Authors: Karish Grover
S M Phaneendra Angara
Md Shad Akhtar
Tanmoy Chakraborty
Venue: NeurIPS 2022 Accept (2022)
Keywords: Social Text Classification
Hyperbolic Geometry
Fourier Transform

Abstract: Social media has become the fulcrum of all forms of communication. Classifying social texts such as fake news, rumour, sarcasm, etc. has gained significant attention. The surface-level signals expressed by a social-text itself may not be adequate for such tasks; therefore, recent methods attempted to incorporate other intrinsic signals such as user behavior and the underlying graph structure. Oftentimes, the public wisdom expressed through the comments/replies to a social-text acts as a surrogate of crowd-sourced view and may provide us with complementary signals. State-of-the-art methods on social-text classification tend to ignore such a rich hierarchical signal. Here, we propose Hyphen, a discourse-aware hyperbolic spectral co-attention network. Hyphen is a fusion of hyperbolic graph representation learning with a novel Fourier co-attention mechanism in an attempt to generalise the social-text classification tasks by incorporating public discourse. We parse public discourse as an Abstract Meaning Representation (AMR) graph and use the powerful hyperbolic geometric representation to model graphs with hierarchical structure. Finally, we equip it with a novel Fourier co-attention mechanism to capture the correlation between the source post and public discourse. Extensive experiments on four different social-text classification tasks, namely detecting fake news, hate speech, rumour, and sarcasm, show that Hyphen generalises well, and achieves state-of-the-art results on ten benchmark datasets. We also employ a sentence-level fact-checked and annotated dataset to evaluate how Hyphen is capable of producing explanations as analogous evidence to the final prediction.

Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classification
Karish Grover
S.M. Phaneendra Angara LinkedIn
Md. Shad Akhtar IIT Delhi India
IIT Delhi India karishh1947@@iiitd ac. in
India Langar@llinkedin com
shad. akhtar@iiitd.ca ac. in
Tanmoy Chakraborty ITT Delhi
India
tanchakčee. iitd. ac.
Abstract
Social media has become the fulcrum of all forms of communication. Classifying social texts such as fake news, rumour, sarcasm, etc. has gained significant atten- tion. The surface-level signals expressed by social-text itself may not be adequate for such tasks; therefore, recent methods attempted to incorporate other intrinsic signals such as user behavior and the underlying graph structure. Oftentimes, the "public wisdom' expressed through the comments/replies to a social-text acts as a surrogate of crowd-sourced view and may provide with complementary signals. sate--of-thh-ar methods on social-text classification tend to ignore such a rich hierarchical signal. Here, we propose Hyphen, a discourse-ane aware hyperbolic spec- tral co-attention network. Hyphen is a fusion of hyperbolic graph representation learning with a novel Fourier co-attention mechanism an attempt to generalise the social-text classification tasks by incorporating public discourse. We parse public discourse as an Abstract Meaning Representation (AMR) graph and use the powerful hyperbolic geometric representation to model graphs with hierarchical structure. Finally, we equip it with a novel Fourier co-attention mechanism capture the correlation between the source post and public discourse. Extensive experiments on four different social-text classification tasks, namely detecting fake news, hate speech, rumour, and sarcasm, show that Hyphen generalises well, and achieves ootte--ffthh-ar results on ten benchmark datasets. We also employ sentence-level fact-checked and annotated dataset to evaluate how Hyphen is capable of producing explanations as analogous evidence to the final prediction. Code is available at: hhttps:/ggttae wwwtthuww........tteeeeeeee IIIDD--IIID/Hppheeee
Introduction
Social media has become a significant source of communication and information sharing. Min- ing texts shared on social media (aka social-texts) are indispensable for multiple tasks online offence detection, sarcasm identification, sentiment analysis, fake news detection, etc. Despite the proliferation of research in social computing, there gap capturing the heterogeneous signals beyond the standalone source text processing. Predictive models incorporating signals such as user profiles
underlying user interaction networks 5 7 and metadata information 12 13 are far and few in between. These heterogeneous signals are challenging to obtain and may always be available on different platforms (e.g.. Reddit does not provide explicit user interaction network: YouTube does not release user activities publicly). On the other hand, comment
[
2 3
[10
[1
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Fake News
User Comments
Says Virginia Del. Mark Cole "wants schools check children's genitals before using bathroom."
Tweet fDr (48044020000411
Anty guesses XXDXY
would deain persor
Tweet CC:
5000003000 1220
Mark Cova filed a ailiaal would foree schools check the gonitais of their studeniss-p in onter ensure that they are using
Wall, sradumyy dows not alwayys coroeiit will sser dieornooone complen and sngmary, so would fave to be staa. inspection
faciiiiea reserved for thair mogiomiccl sex". It's hard to imagine any
that cid not mwo ueoechhlt inspudingg the geritale of childian bakore using the but roats
Any student who willyll and know knowingy vicutes this sact on shall labll
Tweet (D: sESa (ecol//s0000000000 172118
parally
evaeeo $50.
this uneeforoebble
Lusal school boardh all curelop and imp huuel diii ledd require school restrocm, lodker
only be usad
ransgender people.
showam.coom that for usw spacific garder sciely usine by individuaass anatomicc designation. on. Notarlyy iranegendeeee ch lonen solating them single person bottroom, which the Department Justice ruied June lution students
matches such gender provides ellgmatzzing
Tweet (D:
what abo making pollicians brain. baftra they'r allowed to stand?
doconcy
Figure 1: A motivating example (taken from our dataset) showing how user comments act as analogous evidence for fake news article. The third comment hints towards possible sense of harassment being brought out by the highlighted portion of the text (red) and that it is possible fake news.
threads following a source post are an equally rich source of heterogeneous signals, which are casier to obtain and uniformly available across social media platforms and forums. We hypothesise that such public discourse carries complementary and rich latent signals (public wisdom, worldly knowledge, fact busting, opinions, emotions, etc.), which would otherwise be difficult obtain from just standalone source-post analysis. Therefore, public discourse can be used in unison with the source posts to enhance social-text classification tasks. Figure hintts towards the motivation behind using public discourse as an implicit proxy for social-text classification.
[
In this work, we propose Hyphen, a discourse-aware hyperbolic spectral co-attention network that amalgamates the source post and its corresponding public discourse through a novel framework to perform generalised social-text classification. We parse individual comments on a source post as separate Abstract Mcaning Representation (AMR) graphs [1]. and merge them into one macro-AMi AMR, representing mass perception and public wisdom about the source post. The AMR representation inherently abstracts away from syntactic representations, i.e., sentences which are similar in meaning are assigned similar AMRs, even if they are not identically worded [13]. The resultant macro-AMR graph represents the semantic information in rich hierarchical structure [16] [7]. Hyphen aims
to effectively utilise the hierarchical properties of the macro-AMR graph by using the hyperbolic manifold 18 for representation learning.
In order to fuse the source post with the public discourse, we propose a novel Fourier co- co-attention mechanism on the hyperbolic space. It computes pair-wise attention between user comments and the source post, thereby capturing the correlation between them. On a typical social media post, several users express their opinions, and there are several messages being conveyed by the source post itself, some of which are more relevant and/or common than the others. We use a novel discrete Fourier transform based [19] sublayer to filter the most-common user opinions expressed in the macro-AMM and most prominent messages being conveyed by the source post. Fourier transform is essentially measurement of energy (i.e., strength of prevalence) of particular frequency within signal. We can extend this notion to quantify how dominant a particular frequency within a signal. Building on this, we hypothesise that the time-domain signal isomorphically represents various user comments on the source post, and the Fourier transform over the comment representations yields the most-commonly occurring user frequencies (stance, opinions, interpretation, wisdom, etc.). Similarly, the Fourier transform over the sentence-level representations of the source post renders the most intense messages and facts being conveyed by it.
We perform extensive experiments with Hyphen on four social-text classification tasks detecting fake news, hate speech, rumour, and sarcasm, on ten benchmark datasets. Hyphen achieves of the-art results across all datasets when compared with suite of generic and data-specific baselines. Further, to evaluate the efficacy of hyperbolic manifold and Fourier co-attention in Hyphen, we perform extensive ablation studies, which provide empirical justification behind the superiority of Hyphen. Finally, we show how Hyphen excels in producing explainability.
2

---

2 Related Work
Generic social-text classification. There have been some attempts to arrive at a general architecture for social-text classification. Bi-RNODE [20] proposes to use recurrent neural ordinary differential equations by considering the time of posting. CBS-L [21] considers transformation of document representation from the traditional r--gram feature space center-based similarity (CBS) space solve the issue of co-variate shift. Pre-trained Transformer-basee models like RoBERTa-baa base [22],
BERTweet [23], CliRicalioBERTT (24], etc. also deliver benchmark results on generic social-text classification [25). FNet [26] proves to be competent at modeling semantic relationships by replacing the self- layer Transformer encoder with a standard, con-aaraettri Fourier transform.
Use of public discourse in social-text classification. Multiple approaches have been proposed to public discourse
attribute for classifying the social media posts. TCNN-URG [27] utilises a CNN-based network to encode the content, and a variational autoencoder for modeling user comments in fake news detection. CSI 28
is hybrid deep learning model that utilizes subtle clues from text, user responses, and the source post, while modeling the source post representation using an LSTM-based network. Zubiaga al.
29 use public discourse for rumour stance detection using sequential classifiers. Lee et al. [30] propose sentence-level distributed representation for the source post guided by the conversational structure. CASCADE [I] and CUE-CNN [32] use stylometric and personality traits of users in unison with the discussion threads to learn contextual representations for sareasm detection. dEFEND 33 and GCAN p propose to use co-attention over user comments and other social media attributes for detecting fake news and other social texts. The performance of most of these models deteriorate when extended to multiple tasks and fail to filter out the least relevant parts of their respective imput modalities. Moreover, they operate on the Euclidean manifold, and therefore, overlook the representation strength hyperbolic geometry in modeling hierarchical structures. Hyphen overcomes these limitations of the existing methods.
Hyperbolic representation learning. Hyperbolic representation learning has gained significant attention in tasks in which the data inherently exhibits a hierarchical structure. HGCN (34) and HAT 335 achieve state-of-the-art results in graph classification owing to their powerful representation ability to model graphs with hierarchical structure. Unlike these two, H2H-GCN (36 directly works on the hyperbolic manifold to keep global hyperbolic structure, instead of relying on the tangent space, Furthermore, the recent GIL model [37 captures more informative internal structural features with low dimensions while maintaining conformal invariance of both Euclidean and hyperbolic spaces. However, for social-text classification, none of the above approaches simultaneously consider the source- and seccourse-guider representations. We build on this limitation and use public comments in unison with the source post to further contextualise and improve a social-text classifier.
3 Background
Hyperbolic geometry. Riemannian manifold (M, g) of dimension n a real and smooth manifold equipped with an inner product langent space IN T..M x T.M + R at each point E M. where the tangent space TaM is an n-dimensional vector space and can be seen as a first-order local approximation of M around point In particular, hyperbolic space (H ). constant negative curvature Riemannian manifold, is defined by the manifold H = {x E R" c||=||| < 1} equipped with the following Riemannian metrie: Iff >2 where dr T- and gB In is the Euclidean metric tensor. The connections between hyperbolic space and tangent space are established by the exponential map expε Thllll + IH? and the logarithmic map log H + Talll as follows.
expī(v) = x G. tanh (e
(ل 2
Vellvl
log (y)
tanh
(Vell-v G. vI)
(2)
Voxg
| G.
vl
where x, y IHI. TT and @B represents Möbius addition as follows,
a Dc y (1 20(x y)
cllyll
c||æ
10
(3)
+ 20(2, 3)
+
3

---

Further, the generalization for multiplication in hyperbolic space can be defined by the Möbius matrix-vector multiplication between vector x E H (0) and matrix M E RR=x as shown below,
[[ |x||
Mx
M 8. tanh
tanh (aan-"((ellx)))) (Vellx)))
Mx
Hyperbolic space has been studied in differential geometry under five isometric models [18] This work mostly confines to the Poincaré ball model. It is a compact representation of the hyperbolic space and has the principled generalizations of basic operations (c.g., addition, multiplication). We use P, & in the superscript, to denote the Poincaré and Huclidean manifolds, respectively. We provide more insights into these models in Appendix A.T
Discrete Fourier Transform. The Fourier transform decomposes a function into its constituent frequencies. Given sequence } with n E [0, N - 1]. the Discrete Fourier Transform (DFT) is defined as, Xx = En=0 Ine N nk where S k S N 1. For each k the DFT generates a new representation X₃ as a sum of the original input tokens In, with the twiddle factors 38 19 39
4 Architecture of Hyphen
In this section, we lay out the structural details of Hyphen (see Figure 2 for the schematic diagram). We propose individual pipelines for learning representations of the source post and the user comments on the hyperbolic space. We then combine both the representations using novel hyperbolic Fourier co-attention mechanism that helps in simultaneously attending to both the representations. Lastly, we pass it to feed-forward network for the final classification. Without loss of generality, we denote the Poincaré ball model (P) as H (hyperbolic space) throughout the paper.
4.1 Encoding public discourse
In this section, discuss the pipeline for encoding the public discourse. We parse the user comments into an AMR (Abstract Meaning Representation) [14 graph. The individual comment-level AMR graphs are merged to form a macro-A.M (discussed below), representing the global public wisdom and latent frequencies in the discourse. Next, we learn representations of the macro-AMR using
HGCN (Hyperbolic Graph Convolutional Network) 40). This yields a representation for public discourse containing rich latent signals.
Macro-AMR graph creation. Considering social media post containing several user comments C= [C1, C,,.... Cm]]. we obtain an AMR (Abstract Meaning Representation) [14] graph for each user comment. We merge all the comment-level AMR graphs into one macro-AMR (post-level) while preserving the structural context of the subgraphs (comment-level), Figure (a) contains the schematic for an example macro-AMM graph. In particular, we adopt three strategies - (a) Add global dummy-node: We add dummy node and connect it to all the root nodes of the comment-level AMRs, and add a comment tag COMMENT to the edges. The dummy node ensures that all the AMRs are connected, so that information can be exchanged during graph encoding. (b) Concept merging: Since we consider comments made on a particular post, these comments will essentially discuss the same topic. Therefore, multiple user comments can have identical mentions, resulting repeated concept nodes in the comment-level AMRs. We identify such repeated concepts, and add edge with label :SAME starting from earlier nodes to later nodes (here later and earlier refer to the temporal order of the ongoing conversation on social media post). (c) Inter- comment co-reference resolution: A major challenge for conversational understanding is posed by pronouns, which occur quite frequently in such social media comments. We conduct co-reference resolution on the comment-level AMRs to identify co-reference clusters containing concept nodes that refer to the same entity. We add edges labeled with the label COREF between them, starting from earlier nodes to later nodes in co-reference cluster to indicate their relation. Such types of connections can further enhance commens information exchange. This step results in a post-level AMR graph Gamr [g. 92 ... ......".. Ih',, representing relations between various subgraphs {g: = (ui, e < < m (different user comments will correspond to different subgraphs). The merged-AMR presents global view of the public wisdom and interpretations.
Hyperbolic graph encoder. We adopt the Poincaré ball model of HGCN to encode the lest- AMR graph and form user comment representations. Since different comments correspond to different

---

Feod Forward
TEP
Hyperbolic Fourier Co-Amention
cap
Hyper User Interpectation Encoder
bankd-))
coler
-
exps
Hyperbolic Candidate Post Encoder
AMR Pursing Merging
Fouiler Trassfiom
logel
(a) Hyphen model architeciure
Hyperbolic Fourier
Heporbtlc Aedoutiin
To
Euattiion OIB
Xolinaa CRU
Mecros NAR
Regobalit OCN
(c) Hyperbolic candidate post encodor
(d) Hyperbolic user interpretation encoder
Figure 2: Dissecting the primary components of Hyphen. The overall model architecture shown in (a) contains two parallel pipelines to encode the candidate post and user comments; (c) encodes the candidate post's sentences using an attention-enhanced hyperbolic word encoder (Section 4.2). and (d) uses hyperbolic GCN to encode the merged AMR containing latent user interpretations and form subgraph embeddings g. (Section 4.1 The final representations from (c) and (d), i.e., SP and GP, are then passed (b), which first transforms these through Fourier sublayer and then computes the co-attention between user interpretation and the source post sentences in the hyperbolic space (Section 4.3
subgraphs of the post-level AMR, we ultimately aggregate the node representations form subgraph embeddings. Each subgraph embedding represents how individual users interpret the source post (their opinion). this section, we summarize the graph encoder architecture. Given a post-level AMR graph Gamr = (V, E') and the Euclidean input node features, denoted by (x..E) E RRd where dy is the input embedding dimension for entities in the AMR graph, we first map the input from Euclidean to Hyperbolic space. Therefore, we interpret x as a point in the tangent space T. and map it to H with 0.P expe expe(xx.E) (x..E) Our graph encoder then stacks multiple hyperbolic graph convolution layers to perform message passing (see Appendix [A. for the background on HGCN), Finally, aggregate the hyperbolic node embeddings 4.7 'iev at the last layer to form subgraph (comments) embeddings shown in Figure ( d). We take the mean of the node embeddings for nodes present in subgraph yield the aggregated subgraph embedding: g," expo (E logo I.P ) //(eg])
Finally we
Here, g," 222dd the operator represents the number of nodes present in subgraph vi and is the number of layers HGCN. Therefore, the output for this encoder is G [g g2 ..... gm P), where G. H22,, xm the matrix containing the learned representations for user interpretations (comments).
5

---

4.2 Hyperbolic Candidate Post Encoder
Inspired by we propose to learn the source post content representations through hierarchical attention network in the hyperbolic space. We know that not all sentences in a source post might contain relevant information. We thus employ hierarchical attention- network to capture the relative importance of various sentences. Consider the input embedding of the 4th word appearing in the 7th sentence as X;t, in the candidate post. We utilise a hyperbolic word-level encoder (see Appendix A.3 AA.3 for the background of Hyperbolic Hierarchical Attention Network (HHAN)) to learn sKw,, the representation of the hth sentence. Now, similar the word-level encoder, we utilize Möbius- GRU units to encode each sentence in the source post. We capture the sentence-level context to learn the sentence representation sf from the sentence vector P obtained from the word-level encoder. Specifically, we use Poincaré ball model based Möbius-GRU to encode different sentences. We obtain outputs from the Möbius- GRU as s." = [GRU (s?") GRÜ mob((? as shown in Figure e).). Here, sp is the final context-aware representation for the 1th sentence in the souree post in the hyperbolic space (Poincaré ball model), i.e., sP E IH?2,, where d. is the input embedding dimension for the words X in the source post's sentences. This finally gives us S [s1" ", ss2 ..... Sn ", where S E H22dxn is the matrix containing the learned candidate post representations.
4.3 Hyperbolic Fourier Co-Attention
We hypothesise that the evidence for various social-text classification tasks can be unveiled by investigating how different parts of the post are interpreted by different users, and how they correlate to different user opinions. Therefore, we develop hyperbolic Fourier co-attention mechanism to
model the mutual influence between the source social media post (i.e., S [s] P $2 ..... Sn " and user comments (interpretation) embeddings (i.e., G [g p 82 82"", ,.... gm ") where S E HHr,d xn
and G E H.e ("). ). Co-Attention 42 enables the learning of pairwise attentions, i.e., learning to attend based on computing word-level affinity scores between two representations. Once we have the public discourse (Section[4) 4.1) and the social media text (Section 4.2 embeddings in the hyperbolic space, the next step is Fourier sublayer, which applies a 2D DFT to its (sequence length, hidden dimension) embedding input one 1D DFT along the sequence dimension, Fseq and one 1D DFT along the hidden dimension, F
S expo (Freg (Fr
G exp6 (Freg (Fr (logo (GP)))
(5)
The intuition behind taking the Fourier transform the user interpretation embeddings can be thought of as an attempt to capture the most commonly occurring frequuncnfse (public wisdom, worldly knowledge, fact busting, opinions, emotions, etc.) in the public discourse. These frequencies signify how the source post is being received by most of the people. Further, the Fourier transform over the source post embeddings hints towards the most prominent messages conveyed by the source post. This is depicted in Figure b). Next, we compute a proximity matrix F E Rm The affinity (proximity) matrix F can thought to transform the issrrinterpretation attention space to the candidate post attention space, and vice versa for its transpose F
It is computed as:
F = tanh (logō (S/PT &e W We)) 8g logó (G/P))
(6)
where Wag E 2d, x2d, is matrix of learnable parameters. The operator @. is the Möbius Multiplication operator (Equation 4. and @g is the simple euclidean matrix multiplication. By treating the affinity matrix as a feature, we can learn to predict candidate post and user interpretation attention maps HPP H HoP E HH given by
expo (tanh (M:::(WW (W, 8e SPP tee expo (W @. GP)) & F D) H expo (tanh (W, Ee / @ expo W. & StP)) &g F ())
H
(7)
where We E I 2xdd, Wg E 2d are learnable parameters, & is the latent-dimension used in computing co-attention and @. is the Möbius Addition operator (Equation 3) We can then generate
the attention weights of source words and interaction users through the Softmax function:
af softmax
T. &e H)))).
2 softmax(loggs
[w]]
T Kg
& H
(8)
'The relative ordering of Fizy and Fi in Equation S is immaterial because the two ID DFTs commute 266

---

where a E R lm and a E R are the vectors of attention probabilities for each sentence in the source story and each user comment, respectively. Was: Whg E R are learnable weights. Eventually, we can generate the attention vectors of source sentences and user interpretation through weighted sum using the derived attention weights, given by
g
where & E II22d and g E RRIx2,, are the learned co-attention feature vectors that depict how sentences the source post are correlated to the user interpretations. Finally, we have a feed forward network which yields the final classification output as FFN[s gg), .g"], where [-] is the concatenation operator. Equipped with co-attention learning, our model is further capable of generating suitable explanations (Section by looking into the co-attention weights between different frequencies of users and in the source post.
Experiments
Datasets. We evaluate the performance of Hyphen on four different social-text classification tasks
across ten datasets (c.f. Table D - -ii fake news detection (Politifact (43. Gossipcop [3 Anti Vax
44]), (ii) hate speech detection (HASOC 45)), (iii) rumour detection (Pheme [46), Twitter15 [47)
Twitter16 [47]. RumourEval [4 and (iv) sarcasm detection (FigLang- Teinnn-Titter449 FigLang- Reddit
49). We augmented the datasets with public comments/replies to suite our experimental setting (see
the Appendix B for details on dataset preparation).
Experimentation details. For both the hyperbolic encoders in Figures (c)- (d), we adopt the Poincaré model of the respective frameworks. Due to lim-
é
Dataset
SOTA-1
SOTA-2
SOTA-3
poets
[per post)
Policfoct
415
29
+IUNN-URG
28
HPA-BLSTM
50
TCST 28
Gossipacp
Antavi
3797
"INNNUUGG
HPX-BLSTM
TCST
HASOC
712
CRNN
HPA-BI STM
*CS1
ited machine precision, is possible that the (.) and logū() maps might sometimes return points that are not exactly lo- cated on the manifold. To avoid this and to ensure that points remain on the manifold and tangent vec- tors remain on the right tangent space, we clamp the maximum norm to 1 - -14 For optimization on the hyperbolic space, dimension, see Equation 5) for hyperbolic co-attention, we run grid search over = 50, 80, 128, 256, and finally all other hyper-parameters in the Appendix B We run all experiments for 100 epochs with early stopping patience of 10 epochs, NVIDIA RTX A6000 GPU.
Twitter5
GCAN
Twittrll is
362
27
BEGON S1
OCAN
AARD
Pheme
6435
DEGUN
*RumourGAN
Ramourevalt
446
17
CNN
DeCluF 56
1STM
MTL
Figlang Twitler
5000
LST
LSTM, CNN-LSIM.
Ensemble (SVM.
MI.P| 50
Ensemble (SVM.
Figlang Reddit
58
MLP1
Table 1: The statistics of the datasets and the chosen data specific base- lines for four social-text classification tasks. * denotes those baseline models which utilise public discourse. denotes the dataset with three classes, and the remaining datasets have two levels.
Riemannian Adam from Geoopt 61 To find the optimal & (latent
128. For HGCN, we use two layers with curvatures K; K₂ -1. We detail
Curvature for our implementation. Hyphen learns the hyperbolic representations for public discourse and source-post text simulatancously and applies a novel Fourier co-attention mechanism over the obtained embeddings. However, to be able to do so, we need to ensure that the curvatures of the hyperbolic manifolds (in our case Poincaré ball model) are same (or product space of both manifolds). To ensure consistency across both the pipelines (public discourse encoder (Section 4.1 and cource-post encoder (Section 4.2) ). in Hyphen take the constant negative curvature
As addressed in the limitations (See Section 7). another promising approach for Hyphen could be to consider the product space of both the manifolds before applying the co-attention mechanism.
Baseline methods. We compare Hyphen with two sets of baselines (c.f. Table (i) Generic neural baselines: We employ those models that are often used for social-text classification tasks and have
7

---

Task
Dataset
spta-s baseline
Generic neural baseline
Hyphen
SOTA-1
SOTA-2
SOTA-3
HAN
dEFEND
BERT
RoBERIa
Euch.
Hyper
Pre.
0.712
0.894
0.847
0.852
0.902
0.911
0.924
0.951
0.972
Politifact
Rec.
0.785
0.868
0.897
0.958
0,956
0.904
0.903
F1
0.827
0.881
0.871
0.902
0.928
0.905
0.906
0.940
0.968
Fake News
Pre.
0.715
0.684
0.752
0.818
0.729
0.764
0.771
0.786
0.791
Detection
Gossipcop
Rec.
0.521
0.662
0.638
0.742
0.782
0.761
0.775
0.776
0.788
Fl
0.603
0.673
0.682
0.778
0.755
0.762
0.772
0.781
0.816
Pre.
0.829
0.865
0.901
0.806
0.935
0.943
0.948
0.941
0.951
ANTiVax
Rec.
0.825
0.864
0.912
0.862
0.934
0,941
0.961
0.937
0.927
F1
0.872
0.865
0,908
0.833
0.935
0.942
0.939
0.937
0.945
Hate Speech Detection
Pre.
0.531
0.652
0.686
O. 0.658
0.667
0.646
0.647
0.712
0.748
HASOC
Rec.
0.529
0.697
0.699
0.681
0.672
0.651
0.661
0.703
0.718
F1
0.591
0.634
0.698
0.614
0.657
0.641
0.648
0.702
0.713
Pre.
0.785
0.821
0
0.852
Pheme
Roc.
0.783
0.791
0.841
0.779
0.842
0.862
0.851
0.843
0.875
F1
0.782
0.801
0.844
0.799
0.841
0
0.844
Pre.
0,866
0.824
0.928
0.929
0.851
0.899
0.913
0.943
0.961
Twitter15
Roc.
0.794
0.829
0.954
0.839
0.849
0.891
0.909
0.937
0.968
Rumour
F1
0.811
0.825
0.941
0.881
0.848
0.891 1
0. 0.908
0.936
0.957
Detection
Pre.
0,871
0.759
0.901
0.941
0.892
0,921
0.895
0.944
0.946
Twitter16
Rec.
0.751
0,763
0.942
0.842
0.888
0.918
0.891
0.936
0,937
F1
0.778
0.759
0.919
0.889
0.887
0,919
0 0.892
0.937
0.938
Rumour
Pre.
0.545
0,583
0.571
0.653
0.631
U.356
0.602
0.746
0.721
Eval
Rec.
0.676
0.777
0.888
0.444
0.555
0.533
0.602
0.686
0.718
F1
0.598
0.667
0.695
0.518
0.573
0.533
0.595
0.697
0.712
FigLang
Pre.
0.701
0.741
0.751
0.734
0.758
0.797
0.822
0.811
0.823
Twitter
Rec.
0.669
0.746
0.718
0.798
0.796
0.832
Sarcasm
F1
0.681
0,741
0.752
0.721
0.757
0.797
0.801
0,812
0.822
Detection
FigL.agg
0.395
0.671
0.723
0.691
0.712
Reddit
Rec.
0.605
0.677
0.683
0.664
0.634
0.696
0.688
0.697
0.704
F1
0.585
0.667
0.678
0.665
0.631
0.677
0.689
0.698
Table 2: Performance comparisons (Precision (Pre.), Recall (Rec.) and F1 score) of various baselines against Hyphen-hyperboo
(Hyper.) and papphen-euciide
(Eucli.). The best (resp. 2nd
ranked) method is marked in bold (resp. underline). See Table [1 for other abbreviations.
been shown perform comparatively. We consider different variations of the Transformer model and those who use social context as an auxiliary signal for social-text classification (dEFEND (3). (ii) Data-specific baselines: We experimented with many data-specific and task-specific baselines and chose top three for every dataset based on the performance. Since top three models are data- data-specific, we call them with generic names- (a) SO (b) SOTA-2, and (c) SOTA-3, respectively.
Performance comparison. Table 5 shows the performance comparison. The content-based pre- trained models, BERT and RoBERTa la outperform dEFEND which uses both the source content and user comments. We observe that dEFEND performs better than all the data-specific baselines because of the sophisticated use of co-attention. By incorporating public comments along with the social pegrfiicanpeeroommancc post, Hyphen shows significan performance improvement over all the baselines. We observe that
while the performance improvement over baselines is significant (~~ 4%: p 0.005) on datasets like Politifact, Gossipcop, and Twitter15, the performance improvement is not that significant (p < 0.05) on AntiVax and FigLang (Reddit). This is due to the fact that in the latter datasets, there are less number of comments available per the source posts (see Table T.. On Politifact and Gossipcop, hyphen-hyperbol.. has a performance gain of 3.9% and 3.8%, over the best baselines models, RoBERTa and dEFEND, respectively. Note that even when compared to the pre-trained T Transformer architectures, Hpyhen shows decent improvement, while for the non- non-Tranfformer based baselines like HAN, there is a performance gain of 11 11.2% even on the AntiVa dataset. We explain the data-specific baselines, their modalities, and detailed analyses of their performance in the Appendix B
Ablation study. We perform ablations with two variants of our model, namely and Hyphen seseeen-ucldddenen in which Hyperbolic and Euclidean represent the underlying manifold.
Effect of public wisdom. When we remove user comments (Hyphen w/o comments: we consider only source post, get rid of the co-attention block for this analysis as we have just one modality, and keep the Fourier transform layer to capture the latent messages in the candidate post), the performance degrades. Table 3 shows that for Gossipcop and Politifact datasets, performance degradation of 7.23% and 7.4%, respectively. Due to the presence of less number of comments per post in the Anti dataset, the performance degradation is not that significant (i.e.. 1.05%, < 0.1). On some datasets like FigLang (Twitter), Pheme and Twitter15, Hyphen- hyperbolic w/o comments records a significant performance degradation (p- < 0.001) of 8.47%,
has a
We also perform statistical significance t-test comparing Hyphen and the other baselines.
8

---

7.4% and 6.24%, respectively, Even Hyphen-eucl euclidean w/o comments sees a fall in Fl score of 6.38% 6.4% and 5.45% for Twitter15, T and FigLang (Twitter), respectively. Since this is a content-only pipeline, in many cases, the model is outperformed by pre-trained Transformer models. Effect of hyperbolic space. We evaluate Hyphen's 's performance by replacing the hyperbolic manifold with Euclidean. We observe that in support of our initial hypothesis, Styphen-hyperbol1 outperforms Hyphen-euclidenn (see Table 3). The former records a considerable gain of 3.55% and 2.85% Fl score Gossipcop and Politifact datasets, respectively, over the latter. For the Vati dataset, a smaller increment of 0.83% can be attributed to the less number of user comments available in the dataset. Note that for the variant, gypphen-hyperbolie w/o comments, there is performance degradation as compared to Hyphen-euclicn intuitive the sole advantage of hyperbolic space lies in capturing the inherent hierarchy of the macro-AMR graphs. Therefore, in case of a content-only model, Hyphen-eucl idean performs better. On Pheme and 1witter15, the former achieves a significant F1 score gain (p < 0.005) of 3. 12% and 3.18% respectively. Due to less number of user comments in RumourEval, FigL.ng (Twitter) and Figlang (Reddit), the performance gain is less significant (p- 0.05), i.e., 1.44%, 1.01% and 0.47% respectively. It should be noted that this behaviour demonstrates the effectiveness of Hyphen in early detection. Even with less number of user comments available, Hyphen achieves performance boost over the baselines, and thus can be extremely effective in tasks like detecting fake news, where early detection is of great significance.
on Gossipcop and Politifact. This is
Effect of Fourier transform layer. Table 3 shows that including the Fourier transform layer to capture the most prominent user opinions about the source post and the most common (latent) messages conveyed by the source post, boosts the overall performance of Hyphen. There is an improvement of 5.6% F1 score on Gossipcop and 1. 1.74% on Politifact due to the Fourier layer in frrphen-hyperboll. lcc. Because of the less number of comments per post in AntiVax, there is a smaller increment of 0.87% F1 score. Even for on Gossipcop and ~ 1% on Politifact. Hyphen-hyperbolic shows significant improvement (p < 0.001) of 4.49%, 4.34%, and 4.13% Fl score on FigLang (Twitter), Pheme and Twitter15, respectively. For Hyphen-euclidean there is an increase of 5. 10% on FigLang (Twitter) and 3.53% on FigLang (Reddit). Hyphen-eucliden euclidean and of 4.49% and 4.34% in FI score, respectively, over all datasets. On applying co-attention over the outputs of Fourier transform layer, we are able to attend better to both the representations simultaneously, and thus the model's ability to capture the correlation between the two increases.
there is an increase of 2. 2.28%
hypherbolyybbb record an average increase
Explainability
Here, we demonstrate how Hyphen excels at providing explanations for social-text classifica- tion tasks. Using the hyperbolic co-attention weights a (Equation g). we can provide an im- plicit rank list of sentences present in the source post in the order of their relevance to the final prediction. For instance, consider the scenario of fake news detection. A fake news is often created by manipulating selected parts of a true information. The generated rank list of sen- tences in this case would correspond to the sentences in the news article, which are possible misinformation. Furthermore, manual verification of all sentences in a news article is tedious, and therefore, a rank list based on the level of corck-wor of sentences is convenient. To evaluate the performance of Hyphen in gener- ating explanations, we consider Politifact, and for each source post, we manually annotate sentences present in the source post based on their relevance to the final level (fake/real) (see Appendix C for dataset annotation details). The annotators also rank sentences of source post in the order of their check- worthness... We expect our model to produce similar list of sentences for the source post. We use dEFEND [3 as baseline for comparing the rank correlations, and evaluate the rank list produced by Hyphen against this ground-truth annotated rank list using Kendall's 7 and Spearman's 's p rank correlation coefficients. dEFEND is the only model among the chosen baselines, which produces a similar rank link using attention weights in an attempt to provide explanations (See Appendix C for sample rank lists generated by dEFEND and Hyphen). Table 4 shows that the explanations produced by dEFEND have almost no correlation (r 0.0231, p 0.0189) to the annotated rank list. On the contrary, Hyphen- hyperbolic shows high positive correlation
Model
Kendall's
Spearmon''s
LEFEND
0.0231 1 0.053
0.0189 1. 0.012
Hyphen-
0.4013 0.072
0.4236 + 0.072
Bypbee--ypeebolli
0.4983 0.055
0.5532 + 0.045
Table 4: Performance of Hyphen and dEFEND in providing explanations on Politifact. + denotes std. dev. across random runs.

---

Euclidean
Hyperbollc
Dataset
Model
Precision
Recall
Precision
Recall
Ryphen
0,9515
0,9364
0,9401
0,9722
0,9612
0.9686
Politifact
Hyphen w/o comments
0.9166
0.8802
0,8979 (1 4.22%)
0.8461
0.9615
0.8963 () 7.23%)
Hyphen w/o Fourier
0.9091
0.9523
0.9302
0.99%)
0.9341
0,9623
0.9512 1.74%)
Hyphen
0.7862
0,7763
0,7812
0.7913
0,7884
0,8167
Gossipcop
Hyphen x/o commenta
0.7557
0.7578
0.7551 (4 2.61
0.7511
0.7734
0.7407 (4 7.60%)
Ryphen w/o Fourier
0.7751
0.7695
0.7584 () 2.28%)
0.7611
0.7812
0.7607 (, 5.60%)
Ryphen
0,9409
0.9375
0,9373
0.9511
0.9275
0.9456
ANTIVax
Hyphen w/o commenta
0.9202
0.9187
0.9192
181%%)
0.9417
0.9346
0.9351
105%%
Ryphen x/o Fourler
0.9315
0.9281
0.9286 (1 0.87%)
0.9365
0. 0,9281
0.9369 (1 0,87%)
Hyphen
0.7121
0.7031
0.7031
0.7481
0.7187
0.7132
HASOC
Hyphen w/o commenta
0.7122
0.6718
0.6693
3.38%)
0.6747
0.6718
0.6717 4.15%)
Hyphen w/o Fourier
0.6909
0.6718
0.6762 (1 (.69%)
0.7019
0.7031
0.6933 () 1.99%)
Hyphen
0.8445
0.8751
0.8757
Pheme
Ryphen w/o commenta
0,8264
0,8142
0.8161 (1 2.84%
0.8121
0.7968
0.8017 (L 7.40%
Hyphen w/o Fourier
0.8304
0.8203
0.8215 () 2.30%)
0,8411
0.8301
0.8323 (J) 4.34%)
Hyphen
0,9437
0,9375
0.9367
0,9703
0,9687
0.9685
Twitter15
Hyphen w/o comments
0.8782
0.8751
0,8729 (1 6.38%)
0.9078
0.9062
0.9061 (J 6.24%)
Hyphen w/o Fourier
0,9082
0.9062
0.9063
3.04%)
0.9444
0,9375
0.9272 (J 4.1%)
Hyphen
0.9444
0,9363
0.9372
0,9464
0,9375
0.9382
Twitter16
Hyphen w/o commants
0.9021
0.8751
0.8732 (1 6.40%)
0.9196
0.9061
0.902 () 3.40%)
Nyphen w/o Fourier
0.9071
0,9054 3.18%)
0.9067
0,9062
0.9155
2.27%)
Ryphen
0,7465
0,6862
0.6979
0,7219
0,7187
0.7123
RumourEval
Hyphen w/o comments
0.6776
0.6364
0.6611 (4 3.68%)
0.6941
0.6875
O. 0.6898
(L 2.25%
Hyphen v/o Fourier
0.7045
0,6875
0.6762 2.17%)
0.7433
0.6875
0.6743
3.80%)
Hyphen
0.8115
0.8025
0.8121
0.8235
0
0.8222
Figl.nng_ Twitter
Hyphen w/o comments
0.7656
0.7583
0.7576
5.45%)
0.7555
0.7375
0.7375 (L 8.47%)
Hyphen w/o Fourier
0.7624
0,7617
0.7611 (1 5,10%)
0.7779
0.7968
0.7773 ( 4.49%
Ryphen
0,7071
0.6979
0.6971
0,7107
0.7043
0,7018
FigLang Reddit
Hyphen w/o comments
0.6685
0.6511
0.6489
4.82%)
0.6743
0.6514
0.6513 5.05%)
Hyphen w/o Fourier
0.6687
0.6642
0.6618 (1 3.53%)
0.7091
0.6971
0.6961 ( 0.57%
Table 3: Ablation study showing the effect of public discourse, hyperbolic manifold, and Fourier
transform layer on the performance of Hyphen for all four tasks. The decrease in performance the ablation version of Hyphen w.r.t its original one is shown within parenthesis.
(r 0.4983. p = 0.5532). Hyphen- eucl idean also shows comparable performance. The results present the efficacy of Hyphen in providing decent explanations for social-text classification.
Model augmentation for explainability. To provide explanations, we rule out the Fourier sub-layer from Hyphen. This is done because on taking the Fourier transform of source-post and comments' representations (Equation 5), we cannot assert an ordered mapping from the spectral domain to the sentence representations. Such an order necessary for us have a mapping between the co-attention weights and the source-pos sentences they were derived from. Without such mapping, Hyphen would not be able generate a rank-list based on the sentences in the source-post.
7 Conclusion
Public wisdom on social media carries diverse latent signals which can be used in unison with the source post to enhance the social-text classification tasks. Our proposed Hyphen model uses novel hyperbolic Fourier co-attention network to amalgamate both these information. Apart from the state-of-the-ard performance in social-text classification, Hyphen shows the potential of generating suitable explanations to support the final prediction and works well in a generalised discourse-aware setting. In the future, mixed-curvature learning in product spaces (62) and syyperbooictoo-yyperhhihe 36 representations could be employed to boost the learning capabilities of Hyphen.
Limitations. Hyphen resorts to using tangent spaces for computing Fourier co-attention which is inferior because tangent spaces are only a local approximation of the manifold. One may further incorporate other signals such as user interaction network and user credibility into the model.
Acknowledgment
T. Chakraborty would like to acknowledge the support of the Linkedln faculty research grant. We would like to acknowledge the support of the data annotators Arnay Goel, Samridh Girdhar, Abhijay Singh, and Siddharth Rajput, for their help in annotating the Politifact dataset.
10

---

References
[1] Elise Fehn Unsvåg and Bjöm Gambäck. The effects of user features on twitter hate speech detection. In Proceedings of the 2nd workshop abusive language online (ALW2), pages 75-85, 2018.
[2] Kai Shu, Xinyi Zhou, Suhang Wang, Reza Zafarani, and Huan Liu. The role of user profiees for fake news detection. In Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pages 436-439, 2019.
[3] Anshu Malhotra, Luam Totti, Wagner Meira Jr, Ponnurangam Kumaraguru, and Virgilio Almeida. Studying user footprints in different online social networks. In 2012 IEEE/ACM Inter- national Conference on Advances in Social Networks Analysis and Mining, pages 1065-1070. IEEE, 2012.
[4] Hani Nurrahmi and Dade Nurjanah. Indonesian twitter cyberbultying detection using text classification and user credibility. In 2018 International Conference on Information and Communications Technology (ICOIACT), pages 543-548. IEEE, 2018
[5] Xiaoyu Yang, Yuefei Lyu, Tian Tian, Yifei Liu, Yudong Liu, and Xi Zhang. Rumor detection on social media with graph structured adversarial learning. In Proceedings the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 1417-1423, 2021.
[6] Adrien Guille, Hakim Hacid, and Cécile Favre. Predicting the temporal dynamics of information diffusion in social networks. arXiv preprint arXiv: 2013.
[7] Ioannis Pitas. Graph-based social media analysis, volume 39. CRC Press, 2016.
[8] Adrien Guille, Hakim Hacid, Cecile Favre, and Djamel A Zighed. Information diffusion in online social networks: A survey. ACM Sigmod Record, 42(2):17-28, 2013.
[9] Yi-Ju Lu and Cheng-Te Li. Gcan: Graph-aware co-attention networks for explainable fake news detection social media. arXiv preprint arX1v:2004.11645 2020.
[10] Kai Shu, Suhang Wang, and Huan Liu. Beyond news contents: The role of social context for fake news detection. Proceedings of the twelfth ACM international conference on web search and data mining, pages 312-320, 2019.
[1I] Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yer Ven Kan. Fang: Leveraging social context for fake news detection using graph representation. In Proceedings of the 29th ACM international conference information & knowledge management, pages 1165-1174, 2020.
[12] Arkaitz Zubiaga, Maria Liakata, and Rob Procter. Exploiting context for rumour detection in social media. In International conference social informatics, pages 109-123. Springer, 2017.
[13] Lei Gao and Ruihong Huang. Detecting online hate speech using context aware models. arXiv preprint 1XXv::710.00395. 2017.
[14] Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgeseu, Kira Griffitt, UIf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. Abstract meaning representation (amr) 1.0 specification. Parsing on Freebase from Question-Answer Pairs. In Proceedings of the 2013 Conference Empirical Methods in Natural Language Processing. Seattle: ACL, pages 1533-1544, 2012.
[15] Jeffrey Flanigan, Sam Thomson, Jaime G Carbonell, Chris Dyer, and Noah A Smith. A discriminative graph-based parser for the abstract meaning representation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1426-1 4426-1436, 2014.
[16] Peiyi Wang. Liang Chen, Tianyu Liu, Damai Dai, Yunbo Cao, Baobao Chang, and Zhifang Sui. Hierarchical curriculum learning for amr parsing. 2021.
[17] Bang Liu, Ting Zhang, Fred X Han, Di Niu, Kunfeng Lai, and Yu Xu. Matching natural language sentences with hierarchical sentence factorization. In Proceedings of the 2018 World Wide Web Conference, pages 1237-1246, 2018
[18] James W Cannon, William Floyd, Richard Kenyon, Walter R Parry, et al. Hyperbolic geometry. Flavors of geometry, 31(59-115))2 1997.

---

[19] James W Cooley and John W Tukey. An algorithm for the machine calculation of complex fourier series. Mathematics of computation, 19(90):27-3301 1965.
[20] Maunika Tamire, Srinivas Anumasa, and P. K. Srijith. Bi-directional recurrent neural ordinary differential equations for social media text classification, 2021.
[21] Geli Fei and Bing Liu. Social media text classification under negative covariate shift. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2347-2356, 2015.
[22] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1692, 2019.
[23] Dat Quoe Nguyen, Thanh Vu, and Anh Tuan Nguyen. Bertweet: A pre-trained language model for english tweets. arXiv preprint arXiv:2005. 10200, 2020.
[24] Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. Publicly available clinical bert embeddings. arXiv preprini vrXiv:1904033323 2019.
[25] Yuting Guo, Xiangjue Dong, Mohammed Ali Al-Garadi, Abeed Sarker, Cecile Paris, and Diego Mollá Aliod. Benchmarking of transformer-basee pre-trained models on social media text classification datasets. In Proceedings of the The 18th Annual Workshop of the Aus- tralasian Language Technology Association, pages 86-91, Virtual Workshop, December 2020. Australasian Language Technology Association.
[26] James Teorph Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms, 2021.
[27] Feng Qian, Chengyue Gong, Karishma Sharma, and Yan Liu. Neural user response generator: Fake news detection with collective user intelligence. In IJCAI, volume 18, pages 3834-3840, 2018.
[28] Natali Ruchansky, Sungyong Seo, and Yan Liu. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 797-806, 2017.
[29] Arkaitz Zubiaga, Elena Kochkina, Maria Liakata, Rob Procter, Michal Lukasik, Kalina Bontcheva, Trevor Cohn, and Isabelle Augenstein. Discourse-awarr rumour stance classi- fication in social media using sequential classifiers. Information Processing & Management, 54(2):273299) 2018.
[30] Kangwook Lee, Sanggyu Han, and Sung-Hyon Myaeng. A discourse-aware neural network- based text model for decument-level text classification. Journal of Information Science, (4(6):715-7355 2018
[31] Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. Cascade: Contextual sarcasm detection in online discussion forums, 2018.
[32] Silvio Amir, Byron C. Wallace, Hao Lyu, and Paula Carvalho Mário J. Silva. Modelling context with user embeddings for sarcasm detection in social media, 2016.
[33] Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 395-405, 2019.
[34] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32, 2019.
[35] Yiding Zhang, Xiao Wang, Xunqiang Jiang, Chuan Shi, and Yanfang Ye. Hyperbolic graph attention network, 2019.
[36] Jindou Dai, Yuwei Wu, 7hi Gao, and Yunde Jia, A -xperbolie-oo--yyeroooii graph convolutional network. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 154-163, 2021.
[37] Shichao Zhu, Shirui Pan, Chuan Zhou, Jia Wu, Yanan Cao, and Bin Wang. Graph geometry interaction learning, 2020.
[38] Matteo Frigo and Steven G Johnson. The design and implementation of fftw3, Proceedings of the IEEE, 93(2):216 231, 2005.
12

---

[39] John Guibas, Morteza Mardani, Zongyi Li, Andrew Tao, Anima Anandkumar, and Bryan Catanzaro. Adaptive fourier neural operators: Efficient token mixers for transformers, 2021.
[40] Ines Chami, Rex Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks, 2019.
[41] Chengkun Zhang and Junbin Gao. Hype-han: Hyperbolic hierarchical attention network for semantic embedding. In Proceedings of the Twenty-Ninth International Conference International Joint Conferences on Artificial Intelligence, pages 3990-3996, 2021.
[42] Jiasen Lu. Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question image co- attention for visual question answering. Advances neural information processing systems, 29, 2016.
[43] Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news social media. Big data, 873):171-188, 2020.
[44] Kadhim Hayawi, Sakib Shahriar, Mohamed Adel Serhani, Ikbal Taleb, and Sujith Samuel Mathew, Anti-vax: novel twitter dataset for covid-19 vaccine misinformation detection. Public health, 203:23-30, 2022.
[45] Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. Overview of the hasoc track at fire 2019 Hate speech and offensive content identification in indo-curopean languages. In Proceedings of the 11th forum for information retrieval evaluation, pages 14-17, 2019.
[46] Cody Buntain and Jennifer Golbeck. Automatically identifying fake news in popular twitter threads. In 2017 IEEE International Conference on Smart Cloud (SmariCloud), pages 205-215, 2017.
[47] Jing Ma, Wei Gao, and Kam-Fai Wong. Rumor detection on Twitter with tree-structured recursive neural networks. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1980-1989, Melbourne, Australia, July 2018. Association for Computational Linguistics.
[48] Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, and Leon Derczynski. Semeval-2019 task Rumoureval, determining rumour veracity and support for rumours. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 845 854, 2019.
[49] Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. A report on the 2020 sarcasm detection shared task. arXiv preprini arXiv:2005.05814 2020.
[50] Han Guo, Juan Cao, Yazi Zhang, Junbo Guo, and Jintao Li. Rumor detection with hierarchi- cal social attention network. In Proceedings of the 27th ACM international conference information and knowledge management, pages 943-951, 2018.
[51] Zhixian Chen, Tengfei Ma, Zhihua Jin, Yangqiu Song, and Yang Wang. Bigen: A bi-directional low-pass filtering graph neural network. arXiv preprint
Aal::200100511 2021.
[52] Yun-Zhu Song, Yi-Syuan Chen, Yi-Ting Chang, Shao- Yu Weng, and Hong-Han Shuai. Adversary-aware rumor detection. In Findings of the Association for Computational Linguisties: ACL-IJCNLP 2021, pages 1371-1382, 2021.
[53] Matthew Korban and Xin Li. Ddgen: A dynamic directed graph convolutional network for action recognition. In European Conference on Computer Vision, pages 761-776. Springer, 2020.
[54] Jing Ma, Wei Gao, and Kam-Fai Wong. Detect rumors on twitter by promoting information campaigns with generative adversarial learning. In The world wide web conference, pages 2049-3053 2019.
[55] Qi Huang, Chuan Zhou, Jia Wu, Luchen Liu, and Bin Wang. Deep spatial-tempora structure learning for rumor detection on twitter. Neural Computing and Applications, pages 1-11, 2020.
[56] Aoshuang Ye, Lina Wang, Run Wang, Wenqi Wang, Jianpeng Ke, and Danlei Wang. An end-to-end rumor detection model based on feature aggregation. Complexity, 2021, 2021.
13

---

[57] Mostafa Karimzadeh, Samuel Martin Schwegler, Zhongliang Zhao, Torsten Braun, and Susana Sargento. Mtl-Istm: Multi-task learning- lstm for urban traffic flow forecasting. In 2021 International Wireless Communications and Mobile Computing (IWCMC), pages 564-569, 2021.
[58] Deepak Jain, Akshi Kumar, and Geetanjali Garg. Sarcasm detection in mash-up language using soft-attention based bi-directional 1stm and feature-rich enn. Applied Soft Computing. 91:106198, 2020.
[59] Jens Lemmens, Ben Burtenshaw, Ehsan Lotfi, Ilia Markov, and Walter Daclemans. Sarcasm detection using an ensemble approach. In Proceedings of the Second Workshop on Figurative Language Processing, pages 264-269, Online, July 2020. Association for Computational Linguistics.
[60] Amit Kumar Jena, Aman Sinha, and Rohit Agarwal. C-net: Contextual network for sarcasm detection. In Proceedings of the Second Workshop Figurative Language Processing, pages 61-66, Online, July 2020. Association for Computational Linguistics.
[61] Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in pytorch, 2020.
[62] Albert Gu, Frederic Sala, Beliz Gunel, and Christopher Ré. Learning mixed-curvature rep- resentations in product spaces. In International Conference on Learning Representations, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 7
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] We release the Code and Data used as part of the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See experimentation details in Section
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See Section 6
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See experimentation details in
Section 5
4. If you are using existing assets (c.g., code, data, models) or curating/relcasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
14

---

(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "t-MiqB7steh", "reviewer": "Reviewer_jzcw", "summary": "Social-text classification is challenged by the brevity and lack of context in individual posts. Previous work has tried to augment predictive models with external (to the post) sources of information such as signals derived from users or their networks. However, such signals may not always be readily available. In contrast, public discourse is usually available in the form of comments or replies to a given post. This paper, proposes Hyphen, a novel model that exploits public discourse to improve social-text classification. The model consists of three main modules: candidate post encoder, public discourse encoder and co-attention between the representations induced by the encoders. \n\nTo encode the pubic discourse, individual comments are first represented as AMR graphs which are then merged into a single macro-AMR graph. The macro-AMR is then passed into a Hyperbolic Convolutional Neural Network to induce embedding representations.\nThe candidate post is encoded through a Hyperbolic Hierarchical Attention Network — individual word representations are transformed into sentence representations, using bi-directional Mobius-GRU layers with attention. Similarly, the final document representation is obtained  with a higher-level Mobius-GRU layer with attention that processes the sentence representations.\nBoth induced representations are then combined via an Hyperbolic Fourier Co-attention mechanism that produces affinity scores between representations. The representations are first transformed via a 2D Discrete Fourier Transform to extract the most salient (high “frequency”) features. Then an affinity matrix is learned such that it can produce attention weights between source words and user interactions, which are used to obtain the final weight aggregated representations for the source post and the public discourse signals. These attention weights can also be used to interpret the predictions: by ranking candidate sentences wrt to attention scores we can get a sense of which parts of the input most contribute to the predictions.\n\nThe evaluation performed on a variety of datasets and tasks such as detection of fake news, rumors, hate-speech and sarcasm, shows that the proposed model is effective. ", "questions": "N/A", "limitations": "Yes, for the most part. However, given that this work entails collecting and analyzing personal data from social media, a discussion on potential negative societal impact is missing. ", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper addresses an important problem in social-text classification: how to augment predictive models with external information? The proposed approach seems to be effective at leveraging public discourse associated to a target post by exploiting the hierarchical nature of public discourse (in social-texts). Another benefit of this model is the Fourier co-attention module that allows models to focus on the most important parts of the input. Furthermore, this mechanism can afford some degree of interpretability by inspecting the attention weights attributed to each sentence on a given input. However, claims of model interpretability based on attention must be taken with a grain of salt as previous work has shown that “Attention is Not Explanation” (Jain and Wallace, 2019).\n\nOverall the paper is well written and organized, and the main claims of the paper are supported by experimental evidence. However, the notation is a bit convoluted and thus hard to follow with the various superscripts and subscripts. Perhaps the authors can polish and simplify the notation on the final draft. There are also broken references (e.g. line 192 refers to Figure (e) which does not exist; line 272 refers a Table 5 that does not exist; line 261 discusses optimizing k and points to Eq. 5 but this equation has no k). ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "M_YtvqpxF9w", "reviewer": "Reviewer_NXDS", "summary": "This paper proposes to apply hyperbolic fourier co-attention mechanism to aggregate public wisdom in social text classification. In social text classification tasks, e.g. fake news detection and hate speech detection, previous works have proved that the comments or reactions from other users on the internet could contribute a lot in boosting the performance of text classification models. This work proposes to use a co-attention mechanism to aggregate the public wisdom within the Abstract Meaning Representation (AMR) graph constructed from comments and reactions. More specifically, they replace the operations like addition and mutiplication in original attention and Fourier transforms with the corresponding definitions in hyperbolic geometry. The experiment results show that their proposed works outperform both task specific SOTA and generic SOTA nerual models.", "questions": "Based on my understanding, the reason why the Hyperbolic geomerty works is that it can capture hierachy structure more effectively. Would the authors like to do some visualization to the learnt embedding to verify that the model does learn a good embedding?", "limitations": "See the weakness. Also, I think that the idea of aggregating the public wisdom in social text classification is well known. The main contribution of this work is in designing neural architectures to capture such wisdom better.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The performance of the proposed model is impressive.\n2. The idea to capture the hierarchy structure in the AMR graph is convincing.\n3. I appreciate that the authors did so many experiments for the evaluation.\n\nWeakness:\nMy main concern is about the comparison of task specific SOTA. The issue is that the data specific SOTAs are usually pre-BERT models. As a result, their backbones for capturing linguistic cues are generally weak. For example, the CSI model simply computes the mean of all word embeddings of the text for the detection. From the table 2, we can also see that those generic neural models almost defeat all those data specific SOTAs. As a result, it is hard to say that the proposed Fourier Hyperbolic Co-Attention mechanism can bring better boosts compared to the technologies applied in the data specific SOTAs. To address my concern, I expect the authors could either claim that those data specific SOTAs can not use BERT or provide some results of the data specific SOTAs with BERT backbone (not necessary on all SOTA or all datasets, since the time for rebuttal will be limited. Only several common SOTAs, like CSI, on several datasets would be helpful.)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Hz7SiiYutll", "reviewer": "Reviewer_Lrhk", "summary": "This paper addresses the problem of social media classification. In detail, it targets fake news, hate speech, rumour, and sarcasm detection on ten datasets. The approach combines hyperbolic graph representation learning with Fourier co-attention. In detail, the authors encode the candidate with an attention-enhanced hyperbolic word encoder and use abstract meaning representations to learn hyperbolic graph representations of posts aligned with a candidate. During the evaluation, they compare their approach with generic neural baselines (i.e. Bert) and data-specific baselines - like CNN-based approaches. They found statistically significant improvements in the majority of the tasks. The final ablation study examinates the effect of incorporating social media posts and using the Fourier co-attention mechanism, \n", "questions": "- How was a candidate and its post encoded?\n- Does table 2 shows the average results over multiple runs using different seeds?\n", "limitations": "No limitations", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method shows constant improvement over different tasks, especially when the number of posts is large. Moreover, it shows that it can gain even from an extensive collection of posts. \n- Different aspects of this method are examined in detail during the ablation study and show their importance. \n- Apart from the pure performance evaluation, the authors include a concluding evaluation regarding the interpretability of single sentence importance from the candidates based on their approach. They found a clear improvement in the baseline.  \n\nWeaknesses:\n- The authors compared their approach to pre-trained language models - like Bert - and found overall tasks a constant performance improvement. From the paper, it is unclear how social media posts were presented to these models. In the case of concatenating the candidate and its posts, parts of these posts could be ignored by the models. Because they have a fixed token limitation and most datasets have many relevant social media posts.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0LO8ygVIBnv", "reviewer": "Reviewer_niN4", "summary": "This paper proposes to leverage the public wisdom expressed in social media comments and replies to augment social text classification. To this end, this work proposes Hyphen, which consists of hyperbolic graph representation learning and a Fourier co-attention mechanism. Extensive experiments on 10 datasets demonstrate the effectiveness of Hyphen.", "questions": "1. I appreciate that the authors leverage 10 datasets for evaluation. Many of these datasets are well-explored in previous literature, and they often use different dataset splits. Maybe I missed it, but the authors did not seem to provide the train/validation/test split information of these 10 datasets. It would be helpful for readers to know if the authors followed the splits of any previous work or created splits in these experiments.\n\n2. Did the authors implement \"SOTA-1\", \"SOTA-2\", and \"SOTA-3\" in Table 2 or directly quoted the reported results in these papers? It does not seem clear to me what the answer is. If the results are directly quoted, did the authors make sure that the train/validation/test split is identical?\n\n3. The idea of using social discourse such as user comments or interactions on social media to augment social text classification is not new, for example, dEFEND. I hope the authors could better justify how introducing the maths of Hyperbolic Fourier Co-Attention actually help Hyphen achieve this goal.\n\n4. While BERT and RoBERTa are indeed powerful models, the state-of-the-art of NLP has progressed from there and many improved LMs have emerged. In the spirit of [1], I suggest the authors check out newer LMs [2,3] as sota baselines against Hyphen.\n\n5. I appreciate the authors' efforts in annotating the PolitiFact dataset for experiments in Section 6. Did the authors use crowdsourcing for these annotations? If so, please answer the questions in Checklist 5 accordingly.\n\n[1] Bowman, Samuel. \"The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail.\" Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.\n\n[2] Clark, Kevin, et al. \"Electra: Pre-training text encoders as discriminators rather than generators.\" arXiv preprint arXiv:2003.10555 (2020).\n\n[3] He, Pengcheng, et al. \"Deberta: Decoding-enhanced bert with disentangled attention.\" arXiv preprint arXiv:2006.03654 (2020).", "limitations": "The authors discussed the limitations of this work in Section 7. However, the authors did not discuss the potential negative social impacts of this work. I look forward to discussing these issues with the authors and update my score accordingly.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- proposes to leverage social media discourse for the task\n- extensive experiments on 4 tasks, 10 datasets\n\nWeaknesses:\n- certain important experiment details are missing\n- justification of the Hyperbolic Fourier Co-Attention for social text classification\n- no societal impact discussions", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept