DUCK : Rumour Detection on Social Media by Modelling User and Comment Propagation Networks (2022)

Abstract: Social media rumours, a form of misinformation, can mislead the public and cause significant economic and social disruption. Motivated by the observation that the user network — which captures \textit{who} engage with a story — and the comment network — which captures \textit{how} they react to it — provide complementary signals for rumour detection, in this paper, we propose DUCK (rumour \underline{d} etection with \underline{u} ser and \underline{c} omment networ \underline{k} s) for rumour detection on social media. We study how to leverage transformers and graph attention networks to jointly model the contents and structure of social media conversations, as well as the network of users who engaged in these conversations. Over four widely used benchmark rumour datasets in English and Chinese, we show that DUCK produces superior performance for detecting rumours, creating a new state-of-the-art. Source code for DUCK is available at: https://github.com/ltian678/DUCK-code .

## introduction social media platforms bring easy access to a wealth of information. on the flip side, social media has also accelerated the spread of misinformation @xcite . rumours, a form of misinformation typically defined as stories or statements with unverified truth value @xcite , can mislead the public and cause significant economic and social disruption. since the seminal work on prediction of information credibility on social media by @xcite , automatic rumour detection on social media has attracted significant research, which aims to detect rumour stories (in contrast to news articles by credible news sources) or determine their veracity -true, false or unverified. although the task is related to fake news detection, the use of social media for propagation means that various social context features can be leveraged for detection. this is a contrast to fever-style fake news detection @xcite which relies mainly on a source of world knowledge (e.g. wikipedia) to fact-check stories. early studies of rumour detection focus on supervised learning algorithms incorporating features manually engineered from post contents, user profiles as well as information propagation patterns @xcite @xcite @xcite . recent neural approaches typically explore fusing different feature representations for rumour detection. sequence processing models such as bert are used to encode the contents of social media conversations, e.g. source posts and the stream of comments @xcite , while graph models have been experimented to model the structure of social media conversations @xcite @xcite . although prior approaches explored a combination of content and user features for rumour detection, how to leverage pretrained sequence and graph networks to model them effectively remains under-explored. research found that misinformation propagates differently from genuine information on social media @xcite . reply comments of a story contain user opinions and captures how users react to the story, which provides a strong signal for understanding the truthfulness of a story. on the other hand, the network of users who reply to and repost/retweet a story captures who engage with it, which provides a complementary signal. most studies @xcite @xcite ) typically use only one of these signals. in this paper we propose duck (rumour detection with user and comment networks), a framework that jointly models the user and comment propagation networks. our study presents an extensive exploration on how we can best model these networks, and compared to previous studies, there are several key differences: (1) we model comments as both (i) a stream to capture the temporal nature of evolving comments; and (ii) a network by following the conversational structure (see figure 1 for an illustration); (2) our comment network uses a sequence model to encode a pair of comments before feeding them to a graph network, allowing our model to capture the nuanced characteristics (e.g. agreement or rebuttal) exhibited by a reply; and (3) when modelling the users who engage with a story via graph networks, we initialise the user nodes with encodings learned from their profiles and characteristics of their "friends" based on their social networks. we conduct experiments on four widely used benchmark rumour datasets in english and chinese, and show that duck produces superior performance, creating a new state-of-the-art. although both users and comments provide complementary signals for our task, the comments have a stronger impact. also, when modelling the network of users who engage with a story, incorporating the social relations of users proves to be very beneficial. ## related work early studies of rumour detection focus on supervised learning algorithms incorporating engineered features from post contents, user profiles as well as information propagation patterns @xcite @xcite @xcite . @xcite analysed lexical content and information propagation based on allport's theory of transmission @xcite . it identified 53 features within six categories to represent a rumour message, from semantic meaning to information transmission. recent research focuses on neural models to automatically extract features for rumour detection. sequence processing models leverage the textual contents from the source posts and user reply comments for rumour detection. signals such as writing style, stance and opinions as well as emotions are extracted from the text for rumour detection. @xcite introduce linguistic features to represent writing styles and other features based on sensational headlines from twitter to detect misinformation. to detect rumours as early as possible, @xcite incorporate reinforcement learning to dynamically decide how many responses are needed to classify a rumour. @xcite explore the relationship between a source tweet and its comments by transferring stance prediction model to classify rumours. most of these approaches model user comments as a sequence of posts and ignore the conversational structure among the comments. graph neural models leverage information propagation patterns for rumour detection. @xcite experiment with using convolutional and recurrent neural networks to process user features in the retweet propagation path of stories, and @xcite present a tree-structure recursive neural network to model information propagation for rumour detection. @xcite propose a bi-directional graph network to model the upward and downward information propagation structure among user comments to distinguish false from true rumours. there are also studies combining signals from contents, users and propagation networks for rumour and fake news detection. an ensemble deep learning architecture is presented in @xcite , which incorporates source post content and retweet network. @xcite propose to learn representations for misinformation detection based on the heterogeneous graph of news, news sources, users and their stances in comments. leveraging dual attention mechanism on source text and user propagation structure, @xcite leverage dual attention mechanism on source text and user propagation structure via graph attention networks for fake news detection task. all these studies largely model the superficial characteristics of comments and users, e.g. comments are represented using static features such as bag-of-words @xcite and users with simple features extracted from their profiles @xcite @xcite . deeper interactions, such as the relation between a post and its reply and the social relations of users (e.g. followers) , remain under-explored. beyond rumour detection, recent studies explore combining modern pretrained language models and graph models for modelling texts and their interactions. using the fever dataset, @xcite use pretrained models to perform semantic role labelling to understand the relation between clauses in evidence passages and then encode the s c cn uf un rvnn (ma et al., 2018) rnn+cnn (liu and wu, 2018) p rt multitask (li et al., 2019) p stance-bert (tian et al., 2020) bi-gcn (bian et al., 2020) gcan (lu and li, 2020) p rt duck (our work) p+s rt+rp ## problem statement let @xmath0, x 1 , x 2 , ..., x n } be a set of stories, where a story x i consists of a source post and its reply comments, defined as x @xmath1, u 0 , p 0 , t 0 ), ..., (c m , u m , p m , t m )}, where c refers to the textual content of the post (empty string if it is a repost/retweet), u is the user id who submits the post, p is the parent post id that the current post replies to (null if it is a source post, e.g. p 0 = null), and t the timestamp of the post. each story x i is associated with a groundtruth label y @xmath2, where y represents the label set (binary or 4 classes depending on the rumour dataset). our goal is to learn a classifier from the labelled rumour dataset, that is f : @xmath3. ## methodology the overall architecture of our rumour detection approach is presented in figure 1 . it consists of four modules: (1) comment tree: models the comment network by following the reply-to structure using a combination of bert @xcite and graph attentional networks @xcite ; (2) comment chain: models the comments as a stream using transformer-based sequence models; ## experiments and results in this section, we first introduce the datasets for our experiments and then present the performance of duck by comparing against a number state-ofthe-art models. ## conclusion we presented duck, a social media rumour detection approach that models both the network of users who interact with a story as well as their comments/opinions. our approach is unique in how we model the comment as a graph (with bert and gat) and also as a stream (with transformers) and the user networks together with their peer relations (with gat and gae). we conduct extensive experiments over four popular rumour benchmark datasets to evaluate duck. we found that the comment network contains the strongest signal for predicting rumours, and social relations are important for modelling the user network. duck substantially outperforms all benchmark methods consistently, creating a new state-of-the-art.