Continually Detection, Rapidly React: Unseen Rumors Detection Based on Continual Prompt-Tuning (2022)

Abstract: Since open social platforms allow for a large and continuous flow of unverified information, rumors can emerge unexpectedly and spread quickly. However, existing rumor detection (RD) models often assume the same training and testing distributions and can not cope with the continuously changing social network environment. This paper proposed a Continual Prompt-Tuning RD (CPT-RD) framework, which avoids catastrophic forgetting (CF) of upstream tasks during sequential task learning and enables bidirectional knowledge transfer between domain tasks. Specifically, we propose the following strategies: (a) Our design explicitly decouples shared and domain-specific knowledge, thus reducing the interference among different domains during optimization; (b) Several technologies aim to transfer knowledge of upstream tasks to deal with emergencies; (c) A task-conditioned prompt-wise hypernetwork (TPHNet) is used to consolidate past domains. In addition, CPT-RD avoids CF without the necessity of a rehearsal buffer. Finally, CPT-RD is evaluated on English and Chinese RD datasets and is effective and efficient compared to prior state-of-the-art methods.

## introduction online platforms such as social media are facing new and ever-evolving cyber threats at the information level -rumor. a rumor is an unconfirmed claim related to an object, event, or issue of public concern that is spread when its integrity is unknown @xcite . it is very necessary to study automated rd, because rumors are extremely harmful to society and manual detection is time-consuming and labor-intensive @xcite . however, automated rd has significant challenges and still faces the following difficulties: previous work @xcite @xcite @xcite usually assume the same distribution of training and testing data, and have difficulty coping with changing social network environments. in fig. 1 (a) , there are no updates to the model regardless of how many unseen events will appear in the future, which requires the unreasonable assumption that the model's generalization capability is enough. in addition, as the social network environment changes, the original training set data will become outdated @xcite , so that the model's ability on more recent events will diminish. for example, the covid-19 pandemic caused massive trouble for existing rd models @xcite . in social media, the ideal rd model must continuously detect event stream and react rapidly to emergencies. to study this ability, we propose a continual dynamic event transfer (cdet) setup (illustrated in fig. 1 (b) ), which makes the rd model dynamically updates the parameters θ over a series of sequential events and assume that each event goes through three stages from burst to normalization: zero-shot stage with no samples, few-shot stage with a small number of samples, and fullshot stage with large-scale labeled samples. this is because rumors must be detected early to avoid the social harm caused by their spread, but there are only a few or no labeled samples in the early stages of an event. existing rd models still face many challenges in the cdet setup: (1) catastrophic forgetting: when a neural model is trained in a sequence of tasks, the downstream tasks may catastrophically interfere with the upstream tasks. (2) knowledge transfer and accumulation: transfer the knowledge learned from upstream tasks for rapid generalization, accumulating knowledge from downstream tasks to better cope with upstream tasks; (3) parameter explosion: previous research @xcite often require dynamically expanding neural modules for each task, which is undoubtedly aggravating for pre-trained language models (plm) with billions of parameters, with limited memory; (4) data privacy: after learning a task, training data is usually discarded due to user privacy concerns @xcite . this requires models to share learned parameters, rather than saving data to retrain the model. to address the above challenges, a novel framework called continual prompt-tuning rd (cpt-rd) has been proposed. technically, cpt-rd can be seen as a continuously migrated version of p-tuning v2 @xcite . from fig. 2 , we can clearly decouple domain-specific knowledge (tuning parameter) and shared knowledge (frozen parameter). this provides the basis for achieving memory of task-specific parameters and bidirectional domain knowledge transfer. bidirectional knowledge transfer includes: (1) forward knowledge transfer: cpt-rd has various prompt initialization strategies to adapt fast frozen layer 1 our main contributions are: (1) we propose a cdet setup in rd for evaluating rapid generalization and continual detection problems simultaneously. (2) to completely avoid the cf encountered in continual detection, we optimize and store domain-specific soft-prompt for each event domain and use it selectively. (3) we propose various forward knowledge transfer strategies to deal with early emergency rumors and accumulate detection experience through tphnet for backward knowledge transfer. (4) our experiments on the collected chinese and english social media rd datasets demonstrate the superior performance and efficiency of our proposed method. ## methodology in this section, we first present the task definition and execution process of cpt-rd in cdet, then elaborate prompt encoding method and bidirectional knowledge transfer strategy. ## experiments to evaluate the proposed cpt-rd, we closely follow the settings proposed in prior works @xcite , and conduct comprehensive experiments. in particular, we mainly consider whether cpt-rd effectively addresses the four challenges mentioned in sec.1. we carefully compare cpt-rd with state-of-theart methods of different categories under proper experiment settings. moreover, we conduct extensive ablation studies to provide a deeper understanding of our method. ## conclusion we explore how to continually detect a social network environment with frequent unseen domains in rd. the novel framework continual prompt-tuning rd (cpt-rd) is proposed, which includes various knowledge transfer techniques. in the face of emergency domains, cpt-rd can use soft-prompt initialization strategies to achieve fast generalization. there is also a taskconditioned prompt-wise generator network (tph-net) in terms of continual accumulation of detection knowledge. future works include extending usage of cpt-rd to task agnostic scenarios and designing more diverse knowledge transfer strategies.