Semantic Oppositeness Assisted Deep Contextual Modeling for Automatic Rumor Detection in Social Networks (2021)

Abstract: Social networks face a major challenge in the form of rumors and fake news, due to their intrinsic nature of connecting users to millions of others, and of giving any individual the power to post anything. Given the rapid, widespread dissemination of information in social networks, manually detecting suspicious news is sub-optimal. Thus, research on automatic rumor detection has become a necessity. Previous works in the domain have utilized the reply relations between posts, as well as the semantic similarity between the main post and its context, consisting of replies, in order to obtain state-of-the-art performance. In this work, we demonstrate that semantic oppositeness can improve the performance on the task of rumor detection. We show that semantic oppositeness captures elements of discord, which are not properly covered by previous efforts, which only utilize semantic similarity or reply structure. We show, with extensive experiments on recent data sets for this problem, that our proposed model achieves state-of-the-art performance. Further, we show that our model is more resistant to the variances in performance introduced by randomness.

## introduction social media changed the ecosystem of the world wide web by making it possible for any individual, regardless of their level of knowledge of web technologies, to create and maintain profiles online. at the same time, various social media provided these individuals with means to tap into the infor-mation disseminated by others (e.g., facebook by adding friends, twitter by following). by virtue of other mechanisms, such as facebook pages and twitter lists, the reach of each individual was then extended to the range of thousands-to-millions of users. new content, in the form of posts, is created on social media sites each passing second. the rapidity of this post creation is such, that it is possible to claim that social media reflect a near real-time view of the events in the real world @xcite . while it was, indeed, beneficial in terms of volume of data, to have private individuals be content creators and propagators of information, this created significant issues, from the perspective of veracity of the data. this gave rise to a challenge of detecting fake news and rumors (which, in this study, we refer to as the task of rumor detection). the need for rumor detection has come to the forefront, in light of its momentous impacts on political events @xcite and social @xcite or economic @xcite trends. manual intervention on this task would require extensive analysis of and reasoning about various sources of information, resulting in long response times, which are intolerable, given the impact of these rumors, and the rate at which they spread. thus, automatic rumor detection, toward which we contribute in this paper, has become an important area of contemporary research. @xcite define any piece of information, of which the veracity status was questionable at the time of posting, as a rumor. they further claim that a rumor may later be verified to be true or false by other authorized sources. we follow their definition in this work; thus, we define the task of rumor detection as: given a piece of information from a social network, predict whether the piece of information is a rumor or not using the conversations which were induced by the said piece of information. the initial piece of information could be a tweet or a user post, and the induced conversation would be the replies from other users (which we use as contextual information). following the conventions in the literature, in this work, we refer to a main post and its replies as a thread. in this paper, we utilize the semantic oppositeness proposed by @xcite to improve the rumor detection task, which has so far been restricted to only considering semantic similarity. we further prove that semantic oppositeness is well-suited to be applied to this domain, under the observation that rumor threads are more discordant than those of non-rumors. we further observe that, within rumor threads, false rumor threads continue to be clamorous; while true rumor threads settle into inevitable acquiescence. we claim that semantic oppositeness can help in distinguishing this behavior as well. we propose word-level self-attention mechanism for the semantic oppositeness to augment the tweet level self-attention mechanism for the semantic similarity. we model the explicit and implicit connections within a thread, using a relevancy matrix. unlike a regular adjacency matrix, our relevancy matrix recognizes the coherence of each sub-tree of conversation rooted at the main post, while acknowledging that, by definition, for this task, the main tweet must be directly related all the rest of the tweets, regardless of the degrees of separation that may exist between them. we conduct extensive experiments to compare our proposed model with the state-of-the-art studies conducted on the same topic. to the best of our knowledge, this work is the first to utilize semantic oppositeness in rumor detection. in summary, our contributions in this paper include: the remainder of this paper is organized as follows: section 2 presents related work, and then section 3 provides a formal definition of the problem, along with our proposed solution. it is followed by section 4 discussing experiments and results. finally the section 5 concludes the paper. ## related work semantic oppositeness is the mathematical counterpart of semantic similarity (de silva and dou, 2019). while implementations of semantic similarity @xcite are more widely used than those of semantic oppositeness, there are a number of studies which work on deriving or using semantic oppositeness @xcite @xcite @xcite . however, it is noted that almost all of these studies are reducing oppositeness from a scale to either bipolar scales @xcite or simple anonymity @xcite . the study by @xcite proves that this reduction is incorrect and proposes an alternative oppositeness function. their follow-up study, de @xcite creates a word embedding model for this function. in this study, we use the oppositeness embeddings created by them. rumor detection task has been approached on three fronts, according to @xcite : feature engineering, propagation-based, and deep learning. in the feature engineering approach, posts are transformed into feature representations by handdesigned features and sent to a statistical model to be classified. in addition to textual information, structural evidences @xcite and media content @xcite are also utilized. given that this approach depends heavily on the quality of the hand-designed feature sets, it is neither scalable, nor transferable to other domains. the propagation-based approach is built on the assumption that the propagation pattern of a rumor is significantly different to that of a non-rumor. it has been deployed to detect rumors in social networks @xcite . however, this method does not pay any heed to the information in the post content itself. as expected, deep learning approach, automatically learns effective features @xcite @xcite . @xcite claim that these discovered features capture the underlying representations of the posts, and hence, improve the generalization performance, while making it easy to be adapted into a new domain or a social medium for the purpose of rumor detection. @xcite propose a slide window-based system for feature extraction. none of these state-of-the-art work attempt to check rumour veracity akin to attempts by hamidian and diab (2019a) and @xcite . instead, they attempt to do classification on the already established baseline. thus, our work also follow the approach of the former rather than the latter. the work by @xcite does focus on rumor detection and classification. however, they are not using the data sets common to the state-of-the-art work mentioned above to evaluate their approach. our work is most related to the rumor detection model on twitter by means of deep learning to capture contextual information @xcite . however, we also derive inspiration from earlier work on the same topic @xcite , which utilized the tree-like structures of the posts, and the work by de @xcite , which introduced the oppositeness embedding model. the early work by @xcite uses recursive neural networks (rvnn) for the construction of the aforementioned tree-like structures of the posts, based on their tf-idf representations. the following work by @xcite acknowledges the usefulness of considering the innate similarities between replies, but further claims that only considering the replies along the treelike structure only exploits the explicit relations between the main posts and their replies, and thus ignores the implicit relations among the posts from different branches based on their semantics. under this claim, they disregard the tree-like structure entirely. in our work, we preserve the idea of considering semantic similarities to discover the implicit relationships among posts, as proposed by @xcite . however, we augment the model and reintroduce the explicit relationships proposed by @xcite in a balancing of information between implicit and explicit. further, we note that all these prior works have been solely focused on the similarity between the posts and have ignored the oppositeness metric. to the best of our knowledge, we are the first to utilize oppositeness information in the rumor detection task. ## methodology we use a recent work @xcite on rumor detection as our baseline. their work, in turn, was heavily influenced by the earlier work on rumor detection in twitter @xcite . a tweet set i is defined as shown in equation 1, where r 0 is the initial tweet and r 1 , r 2 , . . . , r t are replies, such that t is the count of replies. each tweet r i is a sequence of words w 1 , w 2 , ..., w n , such that n is the count of words. we tokenize the tweets; and in this work, tokens and words are used interchangeably. we also define the relevance matrix m , which carries the information of the tree structure of the tweet tree in equation 2, where a b denotes that a and b belong to the same tree in the forest obtained by eliminating the initial tweet. we show the process in fig 1 as well. our input is the pair @xmath0, m ), which differs from @xcite , where only i was used as the input. the entire data set is represented by d. following the convention of (veyseh et al., 2019) which is our baseline, we classify each pair (i, m ) into four labels: 1) not a rumor (nr); 2) false rumor (fr); 3) true rumor (tr); and 4) unrecognizable (ur), it should be noted that the distinction between "false rumor" and "true rumor" is drawn from the truthfulness of r 0 . in this work what we do with the above described relevance matrix m is to augment the implicit relationship consideration using the high level structure of the explicit relationships, hence bringing in the best-of-both-worlds. in summary, the set of edges in the relevancy matrix is a super-set of the set of edges in the adjacency matrix. in addition to the edges that were in the adjacency matrix, the relevancy matrix also has edges that carry implicit connection information. thus, by definition, the relevancy matrix is more figure 1: relevance matrix building: 1) original tweet reply tree; 2) obtain the forest by temporarily removing the root (main tweet); 3) consider each tree in the forest to be fully connected graphs, and obtain the relevance matrices; 4) obtain the full relevance matrix by putting together the matrices from the previous step and considering the main tweet to be connected to all the other tweets. word embeddings [each of size: 300] classifier l label mlp rumour classifier [with ff of 200 units] classifier l thread mlp latent label classifier [with ff of 100 units] mlp l main tweet-level self-attention ... ... ... ... ... ... main tweet reply 1 reply n word-level self-attention oppositeness embeddings [each of size: 300] descriptive of the thread compared to the adjacency matrix. ## conclusion rumors and fake news are a significant problem in social networks, due to their intrinsic nature of connecting users to millions of others and giving any individual the power to post anything. we introduced a novel method for rumor detection, based on semantic oppositeness, in this paper. we demonstrated the effectiveness of our method using data sets from twitter. compared to previous work, which only used explicit structures in the reply relations or semantic similarity, our model learns both explicit and implicit relations between a main tweet and its replies, by utilizing both semantic similarity and semantic oppositeness. we proved, with extensive experiments, that our proposed model achieves state-of-the-art performance, while being more resistant to the variances in performance introduced by randomness.