Public Wisdom Matters ! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classification
Abstract
Social media has become the fulcrum of all forms of communication.
Classifying social texts such as fake news, rumour, sarcasm, etc. has gained significant attention.
The surface-level signals expressed by a social-text itself may not be adequate for such tasks; therefore, recent methods attempted to incorporate other intrinsic signals such as user behavior and the underlying graph structure.
Oftentimes, the ‘public wisdom’ expressed through the comments/replies to a social-text acts as a surrogate of crowd-sourced view and may provide us with complementary signals.
State-of-the-art methods on social-text classification tend to ignore such a rich hierarchical signal.
Here, we propose Hyphen , a discourse-aware hyperbolic spectral co-attention network.
Hyphen is a fusion of hyperbolic graph representation learning with a novel Fourier co-attention mechanism in an attempt to generalise the social-text classification tasks by incorporating public discourse .
We parse public discourse as an Abstract Meaning Representation (AMR) graph and use the powerful hyperbolic geometric representation to model graphs with hierarchical structure.
Finally, we equip it with a novel Fourier co-attention mechanism to capture the correlation between the source post and public discourse.
Extensive experiments on four different social-text classification tasks, namely detecting fake news, hate speech, rumour, and sarcasm, show that Hyphen generalises well, and achieves state-of-the-art results on ten benchmark datasets.
We also employ a sentence-level fact-checked and annotated dataset to evaluate how Hyphen is capable of producing explanations as analogous evidence to the final prediction. Code is available at: https://github.com/LCS2- IIITD/Hyphen .
Introduction
Social media has become a significant source of communication and information sharing.
Mining texts shared on social media ( aka social-texts) are indispensable for multiple tasks – online offence detection, sarcasm identification, sentiment analysis, fake news detection, etc.
Despite the proliferation of research in social computing, there is a gap in capturing the heterogeneous signals beyond the standalone source text processing.
Predictive models incorporating signals such as user profiles [1, 2, 3, 4], underlying user interaction networks [5, 6, 7, 8, 9] and metadata information [10, 11, 12, 13], are far and few in between.
These heterogeneous signals are challenging to obtain and may not always be available on different platforms (e.g., Reddit does not provide explicit user interaction network; YouTube does not release user activities publicly).
On the other hand, comment threads following a source post are an equally rich source of heterogeneous signals, which are easier to obtain and uniformly available across social media platforms and forums.
We hypothesise that such public discourse carries complementary and rich latent signals (public wisdom, worldly knowledge, fact busting, opinions, emotions, etc.), which would otherwise be difficult to obtain from just standalone source-post analysis.
Therefore, public discourse can be used in unison with the source posts to enhance social-text classification tasks.
Figure 1 hints towards the motivation behind using public discourse as an implicit proxy for social-text classification.
In this work, we propose Hyphen , a discourse-aware hyperbolic spectral co-attention network that amalgamates the source post and its corresponding public discourse through a novel framework to perform generalised social-text classification.
We parse individual comments on a source post as separate Abstract Meaning Representation (AMR) graphs [14], and merge them into one macro-AMR, representing mass perception and public wisdom about the source post.
The AMR representation inherently abstracts away from syntactic representations, i.e., sentences which are similar in meaning are assigned similar AMRs, even if they are not identically worded [15].
The resultant macro-AMR graph represents the semantic information in a rich hierarchical structure [16, 17].
Hyphen aims to effectively utilise the hierarchical properties of the macro-AMR graph by using the hyperbolic manifold [18] for representation learning.
In order to fuse the source post with the public discourse, we propose a novel Fourier co-attention mechanism on the hyperbolic space.
It computes pair-wise attention between user comments and the source post, thereby capturing the correlation between them.
On a typical social media post, several users express their opinions, and there are several messages being conveyed by the source post itself, some of which are more relevant and/or common than the others.
We use a novel discrete
Fourier transform based [19] sublayer to filter the most-common user opinions expressed in the macro-AMR and most prominent messages being conveyed by the source post.
Fourier transform is essentially a measurement of energy (i.e., strength of prevalence) of a particular frequency within a signal.
We can extend this notion to quantify how dominant a particular frequency is within a signal.
Building on this, we hypothesise that the time-domain signal isomorphically represents various user comments on the source post, and the Fourier transform over the comment representations yields the most-commonly occurring user frequencies (stance, opinions, interpretation, wisdom, etc.).
Similarly, the Fourier transform over the sentence-level representations of the source post renders the most intense messages and facts being conveyed by it.
We perform extensive experiments with Hyphen on four social-text classification tasks – detecting fake news, hate speech, rumour, and sarcasm, on ten benchmark datasets.
Hyphen achieves state-ofthe-art results across all datasets when compared with a suite of generic and data-specific baselines.
Further, to evaluate the efficacy of hyperbolic manifold and Fourier co-attention in Hyphen , we perform extensive ablation studies, which provide empirical justification behind the superiority of
Hyphen . Finally, we show how Hyphen excels in producing explainability.
Related Work
Generic social-text classification.
There have been some attempts to arrive at a general architecture for social-text classification.
Bi-RNODE [20] proposes to use recurrent neural ordinary differential equations by considering the time of posting.
CBS-L [21] considers transformation of document representation from the traditional n -gram feature space to a center-based similarity (CBS) space to solve the issue of co-variate shift.
Pre-trained Transformer-based models like RoBERTa-base [22],
BERTweet [23], ClinicalBioBERT [24], etc. also deliver benchmark results on generic social-text classification [25].
FNet [26] proves to be competent at modeling semantic relationships by replacing the self-attention layer in a Transformer encoder with a standard, non-parametric Fourier transform.
Use of public discourse in social-text classification.
Multiple approaches have been proposed to use public discourse as an attribute for classifying the social media posts.
TCNN-URG [27] utilises a CNN-based network to encode the content, and a variational autoencoder for modeling user comments in fake news detection.
CSI [28] is a hybrid deep learning model that utilizes subtle clues from text, user responses, and the source post, while modeling the source post representation using an LSTM-based network.
Zubiaga et al.
[29] use public discourse for rumour stance detection using sequential classifiers.
Lee et al.
[30] propose sentence-level distributed representation for the source post guided by the conversational structure.
CASCADE [31] and CUE-CNN [32] use stylometric and personality traits of users in unison with the discussion threads to learn contextual representations for sarcasm detection.
dEFEND [33] and GCAN [9] propose to use co-attention over user comments and other social media attributes for detecting fake news and other social texts.
The performance of most of these models deteriorate when extended to multiple tasks and fail to filter out the least relevant parts of their respective input modalities.
Moreover, they operate on the Euclidean manifold, and therefore, overlook the representation strength of hyperbolic geometry in modeling hierarchical structures.
Hyphen overcomes these limitations of the existing methods.
Hyperbolic representation learning.
Hyperbolic representation learning has gained significant attention in tasks in which the data inherently exhibits a hierarchical structure.
HGCN [34] and HAT [35] achieve state-of-the-art results in graph classification owing to their powerful representation ability to model graphs with hierarchical structure.
Unlike these two, H2H-GCN [36] directly works on the hyperbolic manifold to keep global hyperbolic structure, instead of relying on the tangent space.
Furthermore, the recent GIL model [37] captures more informative internal structural features with low dimensions while maintaining conformal invariance of both Euclidean and hyperbolic spaces.
However, for social-text classification, none of the above approaches simultaneously consider the source- and discourse-guided representations.
We build on this limitation and use public comments in unison with the source post to further contextualise and improve a social-text classifier.
Background
Hyperbolic geometry.
A Riemannian manifold ( M , g ) of dimension n is a real and smooth manifold equipped with an inner product on tangent space
g x : T x M × T x M
→ R at each point x ∈ M , where the tangent space T
x M is an n -dimensional vector space and can be seen as a first-order local approximation of M around point x .
In particular, hyperbolic space ( H nc , g c ) , a constant negative curvature Riemannian manifold, is defined by the manifold H nc = { x ∈ R n : c ∥ x ∥ < 1 } equipped with the following Riemannian metric: g c x = λ 2 x g E , where λ x = 2 1 − c ∥ x ∥ 2 , and g E =
I n is the
Euclidean metric tensor.
The connections between hyperbolic space and tangent space are established by the exponential map exp c x : T x H nc
→ H nc , and the logarithmic map log c x : H nc → T x H nc , as follows,
[Equation]
where x , y ∈ H nc , v ∈ T x H nc , and ⊕ c represents M ¨ obius addition as follows,
[Equation]
Further, the generalization for multiplication in hyperbolic space can be defined by the M ¨ o bius matrix-vector multiplication between vector x ∈ H nc \ { 0 } and matrix M ∈ R m × n as shown below,
[Equation]
Hyperbolic space has been studied in differential geometry under five isometric models [18].
This work mostly confines to the Poincaré ball model.
It is a compact representation of the hyperbolic space and has the principled generalizations of basic operations (e.g., addition, multiplication).
We use P , E in the superscript, to denote the Poincaré and Euclidean manifolds, respectively.
We provide more insights into these models in Appendix A.1.
Discrete Fourier Transform.
The Fourier transform decomposes a function into its constituent frequencies.
Given a sequence { x n } with n ∈ [0 , N − 1] , the Discrete Fourier Transform (DFT) is defined as, X k = (cid:80) N − 1 n =0 x n e − 2 πiN nk , where 0 ≤ k ≤ N − 1 .
For each k , the DFT generates a new representation X k as a sum of the original input tokens x n , with the twiddle factors [38, 19, 39].
Architecture of Hyphen
In this section, we lay out the structural details of Hyphen (see Figure 2 for the schematic diagram).
We propose individual pipelines for learning representations of the source post and the user comments on the hyperbolic space.
We then combine both the representations using a novel hyperbolic Fourier co-attention mechanism that helps in simultaneously attending to both the representations.
Lastly, we pass it to a feed-forward network for the final classification.
Without loss of generality, we denote the
Poincaré ball model ( P ) as H nc (hyperbolic space) throughout the paper.
4.1 Encoding public discourse
In this section, we discuss the pipeline for encoding the public discourse.
We parse the user comments into an AMR (Abstract Meaning Representation)
[14] graph.
The individual comment-level AMR graphs are merged to form a macro-AMR (discussed below), representing the global public wisdom and latent frequencies in the discourse.
Next, we learn representations of the macro-AMR using a HGCN (Hyperbolic Graph Convolutional Network) [40].
This yields a representation for public discourse containing rich latent signals.
Macro-AMR graph creation .
Considering a social media post containing several user comments
C =
[ c 1 , c 2 , ..., c m ] , we obtain an AMR (Abstract Meaning Representation) [14] graph for each user comment.
We merge all the comment-level AMR graphs into one macro-AMR (post-level) while preserving the structural context of the subgraphs (comment-level).
Figure 2(a) contains the schematic for an example macro-AMR graph.
In particular, we adopt three strategies – (a)
Add a global dummy-node : We add a dummy node and connect it to all the root nodes of the comment-level AMRs, and add a comment tag :COMMENT to the edges.
The dummy node ensures that all the AMRs are connected, so that information can be exchanged during graph encoding.
(b) Concept merging : Since we consider comments made on a particular post, these comments will essentially discuss the same topic.
Therefore, multiple user comments can have identical mentions, resulting in repeated concept nodes in the comment-level AMRs.
We identify such repeated concepts, and add an edge with label :
SAME starting from earlier nodes to later nodes (here later and earlier refer to the temporal order of the ongoing conversation on a social media post).
(c) Intercomment co-reference resolution : A major challenge for conversational understanding is posed by pronouns, which occur quite frequently in such social media comments.
We conduct co-reference resolution on the comment-level AMRs to identify co-reference clusters containing concept nodes that refer to the same entity.
We add edges labeled with the label :COREF between them, starting from earlier nodes to later nodes in a co-reference cluster to indicate their relation.
Such types of connections can further enhance cross-comments information exchange.
This step results in a post-level AMR graph G amr =
[ g 1 s , g 2 s , ..., g ms ] , representing relations between various subgraphs { g is =
( v is , e is ) | 1 ≤ i ≤ m } (different user comments will correspond to different subgraphs).
The merged-AMR presents a global view of the public wisdom and interpretations.
Hyperbolic graph encoder .
We adopt the Poincaré ball model of HGCN [40] to encode the post-level AMR graph and form user comment representations.
Since different comments correspond to different
T r a n s po s e
Fourier Transform
E i n s t e i n
M i dpo i n t
A gg .
M ob i u s G
R U
H yp e r bo
li c
A c ti v a ti on
E u c li d ea n G R U
H yp e r bo li c G C N
M e r g e d
A
M R
Hyperbolic User Interpretation Encoder
Hyperbolic Fourier Co-Attention
Hyperbolic Candidate Post Encoder
AMR Parsing and Merging
Feed Forward :
SAME :SAME :COREF :COREF :COMMENT :COMMENT :COMMENT:COREF
Transpose subgraphs of the post-level AMR, we ultimately aggregate the node representations to form subgraph embeddings.
Each subgraph embedding represents how individual users interpret the source post (their opinion).
In this section, we summarize the graph encoder architecture. Given a post-level AMR graph G amr = ( V , E ) and the Euclidean input node features, denoted by ( x 0 , E ) ∈ R d g , where d g is the input embedding dimension for entities in the AMR graph, we first map the input from Euclidean to Hyperbolic space. Therefore, we interpret x E as a point in the tangent space T
o H d g c and map it to H d g c with x 0 , P = exp c 0 ( x 0 , E ) .
Our graph encoder then stacks multiple hyperbolic graph convolution layers to perform message passing (see Appendix A.2 for the background on HGCN).
Finally, we aggregate the hyperbolic node embeddings ( x L, P i ) i ∈V at the last layer to form subgraph (comments) embeddings as shown in Figure 2(d).
We take the mean of the node embeddings for nodes present in a subgraph to yield the aggregated subgraph embedding: g P j = exp c 0 ( (cid:80) ni ∈ v js log c 0 ( x L, P i ) / | v js | ) .
Here, g P j ∈ H 2 d g c , the operator | .
| represents the number of nodes present in subgraph v js , and L is the number of layers of HGCN.
Therefore, the output for this encoder is G P =
[ g 1 P , g 2 P , ..., g m P ] , where G P ∈ H 2 d g × m is the matrix containing the learned representations for user interpretations (comments).
4.2 Hyperbolic Candidate Post Encoder
Inspired by [41], we propose to learn the source post content representations through a hierarchical attention network in the hyperbolic space.
We know that not all sentences in a source post might contain relevant information.
We thus employ a hierarchical attention-based network to capture the relative importance of various sentences.
Consider the input embedding of the t th word appearing in the i th sentence as x it , in the candidate post.
We utilise a hyperbolic word-level encoder (see
Appendix A.3 for the background of Hyperbolic Hierarchical Attention Network (HHAN)) to learn s K w i , the representation of the i th sentence.
Now, similar to the word-level encoder, we utilize M ¨ o biusGRU units to encode each sentence in the source post.
We capture the sentence-level context to learn the sentence representation s P i from the sentence vector s P w i obtained from the word-level encoder.
Specifically, we use Poincaré ball model based M ¨ o bius -GRU to encode different sentences.
We obtain outputs from the M ¨ o bius -GRU as s P i =
[ −−−→ GRU mob ( s P w i ) , ←−−− GRU mob ( s P w i )] as shown in Figure 2(e).
Here, s P i is the final context-aware representation for the i th sentence in the source post in the hyperbolic space (Poincaré ball model), i.e., s P i ∈
H 2 d s c , where d s is the input embedding dimension for the words x it in the source post’s sentences.
This finally gives us S P =
[ s 1 P , s 2 P , ..., s n P ] , where S P ∈ H
2 d s × n c is the matrix containing the learned candidate post representations.
4.3 Hyperbolic Fourier Co-Attention
We hypothesise that the evidence for various social-text classification tasks can be unveiled by investigating how different parts of the post are interpreted by different users, and how they correlate to different user opinions.
Therefore, we develop a hyperbolic Fourier co-attention mechanism to model the mutual influence between the source social media post (i.e., S P =
[ s 1 P , s 2 P , ..., s n P ] ) and user comments (interpretation) embeddings (i.e., G P =
[ g 1 P , g 2 P , ..., g m P ] , where S P ∈ H 2 d s × n c and G P ∈
H 2 d g × m c ).
Co-Attention [42] enables the learning of pairwise attentions, i.e., learning to attend based on computing word-level affinity scores between two representations.
Once we have the public discourse (Section 4.1) and the social media text (Section 4.2) embeddings in the hyperbolic space, the next step is a Fourier sublayer, which applies a 2D DFT to its (sequence length, hidden dimension) embedding input – one 1D DFT along the sequence dimension, F seq , and one 1D DFT along the hidden dimension, F h :[-
-]
[Equation]
The intuition behind taking the Fourier transform over the user interpretation embeddings can be thought of as an attempt to capture the most commonly occurring frequencies (public wisdom, worldly knowledge, fact busting, opinions, emotions, etc.) in the public discourse.
These frequencies signify how the source post is being received by most of the people.
Further, the Fourier transform over the source post embeddings hints towards the most prominent messages conveyed by the source post.
This is depicted in Figure 2(b).
Next, we compute a proximity matrix F E ∈ R m × n .
The affinity (proximity) matrix F E can be thought to transform the user-interpretation attention space to the candidate post attention space, and vice versa for its transpose F E⊤ .
It is computed as:
[Equation]
where W sg ∈ R 2 d s × 2 d g is a matrix of learnable parameters.
The operator ⊗ c is the M ¨ o bius
Multiplication operator (Equation 4), and ⊗ E is the simple euclidean matrix multiplication.
By treating the affinity matrix as a feature, we can learn to predict candidate post and user interpretation attention maps H s P ∈ H k × n c and H g P ∈
H k × m c , given by
[Equation]
where W s ∈ R k × 2 d s , W g ∈ R k × 2 d g are learnable parameters, k is the latent-dimension used in computing co-attention and ⊕ c is the M ¨ o bius Addition operator (Equation 3).
We can then generate the attention weights of source words and interaction users through the Softmax function:
[Equation]
where a s E ∈ R 1 × m and a g E ∈ R 1 × n are the vectors of attention probabilities for each sentence in the source story and each user comment, respectively.
w hs , w hg ∈ R 1 × k are learnable weights.
Eventually, we can generate the attention vectors of source sentences and user interpretation through weighted sum using the derived attention weights, given by
[Equation]
where ˆ s E ∈ R 1 × 2 d s and ˆ g E ∈ R 1 × 2 d g are the learned co-attention feature vectors that depict how sentences in the source post are correlated to the user interpretations.
Finally, we have a feed forward network which yields the final classification output as ˆ y =
FFN [ˆ s E , ˆ g E ] , where [ . ] is the concatenation operator.
Equipped with co-attention learning, our model is further capable of generating suitable explanations (Section 6) by looking into the co-attention weights between different frequencies of users and in the source post.
Experiments
Datasets.
We evaluate the performance of Hyphen on four different social-text classification tasks across ten datasets (c.f. Table 1) – (i) fake news detection (Politifact [43], Gossipcop [43], AntiVax [44]), (ii) hate speech detection (HASOC [45]), (iii) rumour detection (Pheme [46], Twitter15 [47],
Twitter16 [47], RumourEval [48]), and (iv) sarcasm detection (FigLang-Twitter[49], FigLang-Reddit [49]).
We augmented the datasets with public comments/replies to suite our experimental setting (see the Appendix B for details on dataset preparation). [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
FiglangTwitter
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] FiglangReddit
[Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] Experimentation details
For both the hyperbolic encoders in Figures 2(c)- (d), we adopt the Poincaré model of the respective frameworks.
Due to limited machine precision, it is possible that the exp c 0 ( . ) and log c 0 ( . ) maps might sometimes return points that are not exactly located on the manifold.
To avoid this and to ensure that points remain on the manifold and tangent vectors remain on the right tangent space, we clamp the maximum norm to 1 − e − 14 .
For optimization on the hyperbolic space, we use Riemannian Adam from Geoopt [61].
To find the optimal k (latent dimension, see Equation 5) for hyperbolic co-attention, we run grid search over k = 50 , 80 , 128 , 256 , and finally use k = 128 .
For HGCN, we use two layers with curvatures K 1 = K 2 = -1.
We detail all other hyper-parameters in the Appendix B.
We run all experiments for 100 epochs with early stopping patience of 10 epochs, on a NVIDIA RTX A6000 GPU. Curvature for our implementation .
Hyphen learns the hyperbolic representations for public discourse and source-post text simulataneously and applies a novel Fourier co-attention mechanism over the obtained embeddings.
However, to be able to do so, we need to ensure that the curvatures of the hyperbolic manifolds (in our case Poincaré ball model) are same (or a product space of both manifolds).
To ensure consistency across both the pipelines (public discourse encoder (Section 4.1) and source-post encoder (Section 4.2)), in Hyphen we take the constant negative curvature c = − 1 .
As addressed in the limitations (See Section 7), another promising approach for Hyphen could be to consider the product space of both the manifolds before applying the co-attention mechanism.
Baseline methods.
We compare Hyphen with two sets of baselines (c.f. Table 1) – (i) Generic neural baselines: We employ those models that are often used for social-text classification tasks and have [Table] [Table] Rec.
[Table] [Table] [Table] Hate Speech Detection
HASOC
F
[Table]
[Table]
Pre.
[Table]
Rec.
[Table]
F
[Table]
[Table]
[Table]
Twitter15 F
[Table]
Pre.
[Table]
Rec.
[Table]
Twitter16 F
[Table]
[Table]
[Table]
[Table]
RumourDetection
[Table]
SarcasmDetection
FigLangReddit
[Table]
Table been shown to perform comparatively.
We consider different variations of the Transformer model and those who use social context as an auxiliary signal for social-text classification (dEFEND [33]).
(ii)
Data-specific baselines :
We experimented with many data-specific and task-specific baselines and chose top three for every dataset based on the performance.
Since top three models are data-specific, we call them with generic names – (a) SOTA-1, (b) SOTA-2, and (c) SOTA-3, respectively.
Performance comparison.
Table 5 shows the performance comparison.
The content-based pretrained models, BERT and RoBERTa, outperform dEFEND which uses both the source content and user comments.
We observe that dEFEND performs better than all the data-specific baselines because of the sophisticated use of co-attention.
By incorporating public comments along with the social post, Hyphen shows significant 2 performance improvement over all the baselines.
We observe that while the performance improvement over baselines is significant ( ∼ 4% ; p < 0 .
005 ) on datasets like
Politifact, Gossipcop, and Twitter15, the performance improvement is not that significant ( p < 0 . 05 ) on AntiVax and FigLang (Reddit).
This is due to the fact that in the latter datasets, there are less number of comments available per the source posts (see Table 1).
On Politifact and Gossipcop,
Hyphen - hyperbolic has a performance gain of 3 . 9% and 3 .
8% , over the best baselines models,
RoBERTa and dEFEND, respectively.
Note that even when compared to the pre-trained Transformer architectures, Hpyhen shows decent improvement, while for the non-Transformer based baselines like
HAN , there is a performance gain of 11 .
2% even on the AntiVax dataset.
We explain the data-specific baselines, their modalities, and detailed analyses of their performance in the Appendix B.
Ablation study.
We perform ablations with two variants of our model, namely Hyphen-hyperbolic and Hyphen-euclidean , in which Hyperbolic and Euclidean represent the underlying manifold.
■
Effect of public wisdom.
When we remove user comments ( Hyphen w/o comments : we consider only source post, get rid of the co-attention block for this analysis as we have just one modality, and keep the Fourier transform layer to capture the latent messages in the candidate post), the performance degrades.
Table 3 shows that for Gossipcop and Politifact datasets, Hyphen - hyperbolic has a performance degradation of 7 . 23% and 7 . 4% , respectively.
Due to the presence of less number of comments per post in the AntiVax dataset, the performance degradation is not that significant (i.e., 1 . 05% , p < 0 . 1 ). On some datasets like FigLang (Twitter), Pheme and Twitter15, Hyphenhyperbolic w/o comments records a significant performance degradation ( p < 0 . 001 ) of 8 .
7 . 4% and 6 .
24% , respectively.
Even Hyphen - euclidean w/o comments sees a fall in F1 score of 6 . 38% , 6 . 4% and 5 . 45% for Twitter15, Twitter16 and FigLang (Twitter), respectively. Since this is a content-only pipeline, in many cases, the model is outperformed by pre-trained Transformer models.
■ Effect of hyperbolic space.
We evaluate Hyphen ’s performance by replacing the hyperbolic manifold with Euclidean.
We observe that in support of our initial hypothesis, Hyphen - hyperbolic outperforms Hyphen - euclidean (see Table 3).
The former records a considerable gain of 3 .
55% and 2 .
85% F1 score on Gossipcop and Politifact datasets, respectively, over the latter.
For the AntiVax dataset, a smaller increment of 0 .
83% can be attributed to the less number of user comments available in the dataset.
Note that for the variant, Hyphen - hyperbolic w/o comments , there is a performance degradation as compared to Hyphen - euclidean on Gossipcop and Politifact.
This is intuitive as the sole advantage of hyperbolic space lies in capturing the inherent hierarchy of the macro-AMR graphs.
Therefore, in case of a content-only model, Hyphen - euclidean performs better.
On Pheme and Twitter15, the former achieves a significant F1 score gain ( p < 0 . 005 ) of 3 . 12% and 3 . 18% respectively. Due to less number of user comments in RumourEval, FigLang (Twitter) and
Figlang (Reddit), the performance gain is less significant ( p < 0 . 05 ), i.e., 1 . 44% , 1 . 01% and 0 .
47% respectively.
It should be noted that this behaviour demonstrates the effectiveness of Hyphen in early detection .
Even with less number of user comments available, Hyphen achieves performance boost over the baselines, and thus can be extremely effective in tasks like detecting fake news, where early detection is of great significance.
■ Effect of Fourier transform layer.
Table 3 shows that including the Fourier transform layer to capture the most prominent user opinions about the source post and the most common (latent) messages conveyed by the source post, boosts the overall performance of Hyphen .
There is an improvement of 5 . 6% F1 score on Gossipcop and 1 .
74% on Politifact due to the Fourier layer in
Hyphen - hyperbolic .
Because of the less number of comments per post in AntiVax, there is a smaller increment of 0 .
87% F1 score.
Even for Hyphen - euclidean , there is an increase of 2 .
28% on Gossipcop and ∼ 1 % on Politifact.
Hyphen - hyperbolic shows a significant improvement ( p < 0 . 001 ) of 4 . 34% , and 4 . 13% F1 score on FigLang (Twitter), Pheme and Twitter15, respectively.
For Hyphen - euclidean , there is an increase of 5 .
10% on FigLang (Twitter) and 3 .
53% on FigLang (Reddit).
Hyphen - euclidean and Hyphen - hyperbolic record an average increase of 4 .
49% and 4 .
34% in F1 score, respectively, over all datasets.
On applying co-attention over the outputs of Fourier transform layer, we are able to attend better to both the representations simultaneously, and thus the model’s ability to capture the correlation between the two increases.
Explainability
Here, we demonstrate how Hyphen excels at providing explanations for social-text classification tasks.
Using the hyperbolic co-attention weights a s E (Equation 8), we can provide an implicit rank list of sentences present in the source post in the order of their relevance to the final prediction.
For instance, consider the scenario of fake news detection.
A fake news is often created by manipulating selected parts of a true information.
The generated rank list of sentences in this case would correspond to the sentences in the news article, which are possible misinformation.
Furthermore, manual verification of all sentences in a news article is tedious, and therefore, a rank list based on the level of check-worthiness of sentences is convenient. [Table]
To evaluate the performance of Hyphen in generating explanations, we consider Politifact, and for each source post, we manually annotate sentences present in the source post based on their relevance to the final level (fake/real) (see Appendix C for dataset annotation details).
The annotators also rank sentences of a source post in the order of their check-worthiness.
We expect our model to produce a similar list of sentences for the source post.
We use dEFEND [33] as a baseline for comparing the rank correlations, and evaluate the rank list produced by Hyphen against this ground-truth annotated rank list using Kendall’s τ and Spearman’s ρ rank correlation coefficients.
dEFEND is the only model among the chosen baselines, which produces a similar rank link using attention weights in an attempt to provide explanations (See Appendix C for sample rank lists generated by dEFEND and Hyphen ).
Table 4 shows that the explanations produced by dEFEND have almost no correlation ( τ = 0 .
0231 , ρ = 0 .
to the annotated rank list.
On the contrary, Hyphen - hyperbolic shows a high positive correlation [Table] [Table] [Table]
[Table]
[Table]
( τ = 0 . 4983 , ρ = 0 . 5532) .
Hyphen - euclidean also shows comparable performance.
The present the efficacy of Hyphen in providing decent explanations for social-text classification. Model augmentation for explainability.
To provide explanations, we rule out the Fourier sub-layer from Hyphen . This is done because on taking the Fourier transform of source-post and comments’ representations (Equation 5), we cannot assert an ordered mapping from the spectral domain to the sentence representations. Such an order is necessary for us to have a mapping between the co-attention weights and the source-post sentences they were derived from. Without such a mapping,
Hyphen would not be able to generate a rank-list based on the sentences in the source-post.
Conclusion
Public wisdom on social media carries diverse latent signals which can be used in unison with the source post to enhance the social-text classification tasks.
Our proposed Hyphen model uses a novel hyperbolic Fourier co-attention network to amalgamate both these information.
Apart from the state-of-the-art performance in social-text classification, Hyphen shows the potential of generating suitable explanations to support the final prediction and works well in a generalised discourse-aware setting.
In the future, mixed-curvature learning in product spaces [62] and hyperbolic-to-hyperbolic [36] representations could be employed to boost the learning capabilities of Hyphen .
Limitations .
Hyphen resorts to using tangent spaces for computing Fourier co-attention which is inferior because tangent spaces are only a local approximation of the manifold.
One may further incorporate other signals such as user interaction network and user credibility into the model. Acknowledgment
T. Chakraborty would like to acknowledge the support of the LinkedIn faculty research grant.
We would like to acknowledge the support of the data annotators - Arnav Goel, Samridh Girdhar, Abhijay
Singh, and Siddharth Rajput, for their help in annotating the Politifact dataset.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] We release the Code and Data used as a part of the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See experimentation details in Section(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section(d)
Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See experimentation details in Section4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Hyperbolic space has been studied under five isometric models [18]. In this section, we discuss the Poincaré ball and Lorentz models, which are utilized in the source-post encoder.
Hyphen relies on the Poincaré ball model. Poincaré ball model . The Poincaré ball (cid:0) B dc , g B (cid:1) of radius 1 / (cid:112) | c | , equipped with Riemannian metric g B and constant negative curvature c ( c < 0) , is a d -dimensional manifold
B dc = { x ∈ R d : c ∥ x ∥ 2 < − 1 } , where g B is conformal to the Euclidean metric g E = I d with conformal factor λ c x = 2 / (1 + c ∥ x ∥ 2 ) .
The distance between two points x , y ∈ B dc is measured along a geodesic and is given by d c B ( x , y )
= (2 / (cid:112) | c | ) tanh − 1 ( (cid:112) | c |∥ − x ⊕ c y ∥ ) .
Lorentz model .
With constant negative curvature c ( c < 0)
and equipped with Riemannian metric g L , the Lorentz model (cid:0) L dc , g L (cid:1) is the Riemannian manifold L dc = { x ∈ R d +1 : ⟨ x , x ⟩ L = 1 /c } , where g L = diag
([ − 1 , 1 , ..., 1]) n .The distance between two points x , y ∈ L dc is given by d c L ( x , y ) =
(1 / (cid:112) | c | ) cosh − 1 ( c ⟨ x , y ⟩ L ) , where ⟨ x , y ⟩ L is the Lorentzian inner product.
Klein model .
With constant negative curvature c ( c < 0) , the Klein model is also the Riemannian manifold K dc = { x ∈ R
d : c ∥ x ∥ 2 < − 1 } .
The isomorphism between the Klein model and Poincaré ball can be defined through a projection on or from the hemisphere model.
A.2 Hyperbolic Graph Convolutional Network (HGCN)
Given a graph G =
( V , E ) and Euclidean input features ( x E i )
i ∈V , HGCN [40] can be interpreted as transforming and aggregating neighbours’ embeddings in the tangent space of the center node and projecting the result to a hyperbolic space with a different curvature, at each stacked layer.
Suppose that x i aggregates information from its neighbors ( x j ) j ∈N ( i ) .
Further, we use H in superscript to denote the hyperbolic manifold.
Then, the aggregation operation A GG can be formulated as, A GG K ( x H ) i =
exp K x H i (cid:16)(cid:80)
j ∈N ( i )
[Equation]
ij log K x
[Equation]
i ( x H j ) (cid:17) , where, w ij = S OFTMAX
[Equation]
( i ) ( MLP (log K
o
( x H i )
|| log K
o
( x H j ))) .
More precisely, HGCN applies the Euclidean non-linear activation in T o H d,K ℓ − 1 and then maps back to H d,K ℓ , as σ ⊗
Kℓ − 1 ,Kℓ ( x H ) =
exp K ℓ o ( σ (log K ℓ − 1 o ( x H ))) .
Therefore, the message passing in a HGCN layer can be shown as, x ℓ, H i
[Equation]
, where − 1 /K ℓ − 1 and − 1 /K ℓ are the hyperbolic curvatures at layer ℓ − 1 and ℓ , respectively.
As a final step, we can use the hyperbolic node embeddings at the last layer ( x L, H i ) i ∈V for downstream tasks.
A.3 Hyperbolic Hierarchical Attention Network (HyperHAN)
HyperHAN [41] learns the source document representation through a hierarchical attention network in the hyperbolic space.
Consider the input embedding of the t th word appearing in the i th sentence as x it , in the candidate document.
The Euclidean hidden state of x it within the sentence can be constructed using forward and backward Euclidean-GRU layers as: h E it =
[ −−−→ GRU ( x it ) , ←−−− GRU ( x it )] .
We denote the Klein and Lorentz models using K and L in superscript, respectively.
Zhang and Gao [41] aim to jointly learn a hyperbolic word centroid c L w from all the training documents.
c L w can be considered as a baseline for measuring the importance of hyperbolic words based on their mutual distance.
To learn c L w , they consider another layer upon hidden state h E it as: h E ′ it =
tanh ( W w h E it
+ b w ) .
The next step is activating h E ′ it as h L ′ it .
The word-level attention weights are then computed as α it by: α it = exp( − β w d L ( c L w , h L ′ it ) − c w ) .
After capturing the hyperbolic attention weights, the semantic meaning of words appearing in the same sentences is aggregated via Einstein midpoint: s
[Equation]
i
[Equation]
, where, γ (cid:0) h K it (cid:1) =
[Equation]
is the so-called
Lorentz factor, and s K w i is the learned representation for the i th sentence.
Similar to the word-level encoder, Mobiüs -GRU units are utilized with aggregation using Einstein midpoint to encode each sentence in the source post, yielding the final document level representation.
B Experiments
Dataset preparation.
In this section, we list out the dataset collection and augmentation procedure.
Since we need both source-post text and public discourse information, we augment all the datasets to yield sufficient comments per social media post.
Politifact and Gossipcop [43] were collected from two fact-verification platforms PolitiFact and GossipCop, and contain news content with two labels (fake or real) and social context information.
After scraping the tweets corresponding to the news articles in the datasets, we get 837 and 19266 news articles (Politifact and Gossipcop respectively) which have atleast 1 tweet available.
Finally, we filter the news articles with atleast 3 comments which gives us datasets with 415 and 2813 news articles (source posts) for Politifact and Gossipcop respectively.
AntiVax [44] is a novel Twitter dataset for COVID-19 vaccine misinformation detection, with more than 15,000 tweets annotated as fake or not.
We manually scrape the user comments corresponding to the tweets present in the dataset, which resulted in 3797 tweets (2865 real andfake) with atleast one user comment.
HASOC [45] was taken from the HASOC 2019 sub-task B with over 3000 tweets (comments and replies) from 82 conversation threads labelled as hate speech or not.
Due to the available labels, we consider the top level comments on the 82 conversation threads as separate tweets and the corresponding replies as the public discourse.
This yields a dataset with 712 tweets with public discourse (in contrast to the original 82 conversation threads).
Pheme [46] is a collection of 6425 Twitter rumours and non-rumours conversation threads related to 9 events and each of the samples is annotated as either True, False or Unverified.
RumourEval was introduced in
SemEval-2019 Task 7, and has 446 twitter and reddit posts belonging to three categories: real , fake and unverified rumour.
Twitter15 and Twitter16 [47] consist of source tweets (1490 andresp.)
along with the sequence of re-tweet users.
We choose only true and fake rumour labels as the ground truth.
Since there is no discourse available, we scrape the user comments corresponding to the posts and filter the tweets with atleast one comment giving us 543 and 362 source-tweets respectively.
FigLang (Twitter) and FigLang (Reddit) [49] are FigLang 2020 shared task datasets withsamples each labelled as either sarcasm or not.
Experimentation details.
We adopt a pre-trained AMR parser from the AMRLib 3 library and use the parse_xfm_bart_base model to generate the comment-level AMRs.
We resolve co-references on the comment-level AMRs using an off-the-shelf model AMRCoref 4 , which yields the various co-reference clusters.
Finally, we convert all the merged AMRs to the Deep Graph Library 5 (DGL) format.
All node embeddings for AMR are initialised using 100D Glove embeddings 6 .
Data-specific hyperparameters have been laid out in Table B.
[Figure]
Batchsize
[Figure]
Batchsize
Maxsents
Maxcoms
Politifact
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
1e
[Figure]
[Figure]
[Figure]
1e
[Figure]
1eFigLangTwitter 1e1e-
FigLangReddit 1e1e-
- tences considered in a source post while training, max coms : Max.
comments on post considered while training.
Data-specific baselines.
We experiment with three different baselines per dataset.
We compare the model performance using F1 Score,
Precision and Recall.
These models are known to have reported representative results on the benchmark datasets.
For Fake news detection , (v) TCNN-URG [27] utilises a CNN-based network for encoding news content, and a variation auto-encoder (VAE) for modelling the user comments (vi) CSI [28] is a hybrid deep learning model that utilizes subtle clues from text, responses, and source post, while modelling the news representation using an LSTM-based network, for fake news detection, and lastly (vii)
HPA-BLSTM [50] learns news representations through a word-level, post-level, and event-level user engagements on social media.
These turned out to have representative performance for fake news detection and therefore, we consider them as baselines for Politifact, Gossipcop, and AntiVax datasets (which are related to the task of fake news detection).
In addition to CSI and HPA-BLSTM as baselines for Hate speech detection on the HASOC dataset, we use CRNN as a baseline due to the ability of CNNs to capture the sequential correlation in text.
In rumour detection , for Twitter15 and Twitter16 datasets, (v) AARD [52] uses a weighted-edge transformer-graph network and position-aware adversarial response generator to capture the malicious user attacks while spreading rumours, (vi) GCAN [9] employs a dual co-attention mechanism between source social media post and the underlying propagation patterns, and (vii) BiGCN [51], utilizes the original graph structure information and the latent correlation between features assisted by bidirectional-filtering.
Further, for
Pheme dataset we use (v) RumourGAN [54] which adheres to a GAN-based approach, where the generator is designed to produce uncertain or conflicting opinions (voices), complicating the original conversational threads in order to penalise the discriminator to learn better, (vi) DDGCN [53], which models spatial and temporal features of human actions from their skeletal representations, and (vii)
STS-NN [55].
On RumourEval, (v) DeClarE [56] provides a strong baseline.
Moreover simple yet effective models like (vi) CNN and (vi) MTL-LSTM show comparable performance, and hence are included in our set of baselines.
For the task of (d) Sarcasm detection on Figlang (Twitter) and
Figlang (Reddit) datasets, we use (v) CNN + LSTM [58], (vi) an ensemble of CNN, LSTM, SVM and MLP [59], and lastly (vii) C-Net [60] for efficient sarcasm classification.
C Explainability
Data annotation.
To evaluate the efficacy of Hyphen in producing suitable explanations, we factcheck and annotate the Politifact dataset on a sentence-level.
Each sentence has the following possible labels – true , false , quote , unverified , non_check_worthy or noise .
The annotators were further supposed to arrange the fact-checked sentences in the order of their check-worthiness.
We take the help of four expert annotators in the age-group of 25-30 years.
The final labels for a sentence were decided on the basis of majority voting amongst the four annotators.
To decide the final rank-list (since different annotators might have different opinions about the level of check-worthiness of the sentences), the fourth annotator compiled the final rank-list by referring to the fact-checked ranklists by the first three annotators using Kendall’s τ and Spearman’s ρ rank correlation coefficients, and manually observing the similarities between the three rank-lists.
The compiled list is then cross-checked and re-evaluated by the first three annotators for consistency.
Explainability evaluation.
To evaluate the performance of Hyphen against the annotated rank-list, we measure the rank-correlation between the two.
If Hyphen predicts a news article in Politifact to be fake, we filter the sentences in the ground-truth annotation with the label fake (in the order of their check-worthiness).
We adopt a similar procedure in case Hyphen predicts a news article to be true.
This is done because if a news article is fake, we aim to identify the sentences in the article which are misinformation and thus most relevant to the final prediction.
Finally, we compare the filtered ground-truth rank-list with the rank-list produced by Hyphen using Kendall’s τ and Spearman’s ρ coefficients.
Figure 3 shows sample rank-lists produced by Hyphen - hyperbolic and dEFEND [33].
dEFEND
[Table]
[Table]
[Table]
and
[Table]
“unprecedented
[Table]
The
[Table]
be
[Table]
“This
[Table]
in
[Table]
for
[Table]
receiving
[Table]
[Table]
said in
[Table]
[Table]
State spending that benefits the president may be considered a violation of the domestic emolument clause, which says the president “shall not receive” any emolument, other than fixed compensation, from “the United States, or any of them.
The case advances a very high-profile attempt to see if President Trump is violating the emoluments clause of the U.S. Constitution, which precludes him from receiving gifts from foreign or state governments.
Attorney General Karl A. Racine and Maryland Attorney General Brian E. Frosh can now subpoena the Trump organization, thereby forcing them to preserve documents in relation to President Trump’s alleged indiscretions.
Federal Judge Peter J. Messitte has just ruled in favor of two attorney generals seeking to subpoena the Trump organization relating to President Trump unlawfully receiving emoluments from foreign and domestic governments.
D.C.
President Trump has been accused of profiting from the presidency, and this case will seek to prove that assertion.
Hyphen
Federal Judge Peter J. Messitte has just ruled in favor of two attorney generals seeking to subpoena the Trump organization relating to President Trump unlawfully receiving emoluments from foreign and domestic governments.
D.C.
Attorney General Karl A. Racine and Maryland Attorney General Brian E. Frosh can now subpoena the Trump organization, thereby forcing them to preserve documents in relation to President Trump’s alleged indiscretions.
The case advances a very high-profile attempt to see if President Trump is violating the emoluments clause of the U.S. Constitution, which precludes him from receiving gifts from foreign or state governments.
The Trump Organization will be compelled to comply with the court’s ruling.
“This ruling is an important first step in our litigation
[Table]
[Figure]