Coupled Hierarchical Transformer for Stance-Aware Rumor Verification in Social Media Conversations (2020)

Abstract: The prevalent use of social media enables rapid spread of rumors on a massive scale, which leads to the emerging need of automatic rumor verification (RV). A number of previous studies focus on leveraging stance classification to enhance RV with multi-task learning (MTL) methods. However, most of these methods failed to employ pre-trained contextualized embeddings such as BERT, and did not exploit inter-task dependencies by using predicted stance labels to improve the RV task. Therefore, in this paper, to extend BERT to obtain thread representations, we first propose a Hierarchical Transformer, which divides each long thread into shorter subthreads, and employs BERT to separately represent each subthread, followed by a global Transformer layer to encode all the subthreads. We further propose a Coupled Transformer Module to capture the inter-task interactions and a Post-Level Attention layer to use the predicted stance labels for RV, respectively. Experiments on two benchmark datasets show the superiority of our Coupled Hierarchical Transformer model over existing MTL approaches.

## background recent years have witnessed a profound revolution in social media, as many individuals gradually turn to different social platforms to share the latest news and voice personal opinions. meanwhile, the flourish of social media also enables rapid dissemination of unverified information (i.e., rumors) on a massive scale, which may cause serious harm to our society (e.g., impacting presidential election decisions @xcite ). since manually checking a sheer quantity of rumors on social media is naturally labor-intensive and timeconsuming, it is crucial to develop an automatic rumor verification approach to mitigate their harmful effect. rumor verification is typically defined as a task of determining whether the source claim in a conversation thread is false rumor, true rumor, or unverified rumor @xcite . in the literature, much work has been done for rumor verification @xcite @xcite @xcite . among them, one appealing line of work focuses on exploiting stance signals to enhance rumor verification @xcite , since it is observed that people's stances in reply posts usually provide important clues to rumor verification (e.g., in fig. 1 , if the source claim is denied or queried by most replies, it is highly probable that the source claim contains misinformation and is false rumor). this line of work has attracted increasing attention in recent years. a number of multi-task learning (mtl) methods have been proposed to jointly perform stance classification (sc) and rumor verification (rv) over conversation threads, including sequential lstm-based methods @xcite , tree lstm-based methods @xcite , and graph convolutional network-based methods @xcite . these mtl approaches are mainly constructed upon the mtl2 framework proposed in @xcite , which aims to first learn shared representations with shared layers in the low level, followed by learning task-specific representations with separate stance-specific layers and rumor-specific layers in the high level. although these mtl approaches have shown the usefulness of stance signals to rumor verification, they still suffer from the following shortcomings: to address the above two shortcomings, we explore the potential of bert for stance-aware rumor verification, and propose a new multi-task learning model based on transformer @xcite , named coupled hierarchical transformer. our main contributions can be summarized as follows: evaluations on two benchmark datasets demonstrate the following: first, compared with existing single-task models, our hierarchical transformer brings consistent performance gains on macro-f 1 for both sc and rv tasks. second, our coupled hierarchical transformer outperforms the state-ofthe-art multi-task learning approach by 9.2% and 6.3% on macro-f 1 for the two benchmarks, respectively. the second line of work applies variants of several neural network models to automatically capture important features among all the propagated posts @xcite @xcite . different from these studies, the goal in this paper is to leverage stance classification to improve rumor verification with a multi-task learning architecture. stance-aware rumor verification: the recent advance in rumor verification is to exploit stance information to enhance rumor verification with different multi-task learning approaches. specifically, @xcite and @xcite respectively proposed two multi-task learning architectures to jointly optimize stance classification and rumor verification based on two different variants of rnn, i.e., gru and lstm. more recently, kumar and carley (2019) proposed another multi-task lstm model based on tree structures for stanceaware rumor verification. our work bears the same intuition to these previous studies, and aims to explore the potential of the pre-trained bert to this multi-task learning task. ## methodology in this section, we first formulate the task of stance classification (sc) and rumor verification (rv). we then describe our single-task model for sc and rv, followed by introducing our multi-task learning framework for stance-aware rumor verification. ## experiments in this section, we first evaluate our single-task model on both stance classification (sc) and rumor verification (rv), followed by evaluating our multitask learning model on rv. finally, we perform further analysis to provide deeper insights into our proposed multi-task learning model. ## conclusion in this paper, we first examined the limitations of existing approaches to stance classification (sc) and rumor verification (rv). to tackle these limitations, we first proposed a single-task model (i.e., hierarchical transformer) for sc and rv, followed by designing a multi-task learning framework with a coupled transformer module to capture intertask interactions and a post-level attention layer to use stance distributions for the rv task. experiments on two benchmarks show the effectiveness of our single-task and multi-task learning methods.