M eta A dapt: Domain Adaptive Few-Shot Misinformation Detection via Meta Learning (2023)

Abstract: With emerging topics (e.g., COVID-19) on social media as a source for the spreading misinformation, overcoming the distributional shifts between the original training domain (i.e., source domain) and such target domains remains a non-trivial task for misinformation detection. This presents an elusive challenge for early-stage misinformation detection, where a good amount of data and annotations from the target domain is not available for training. To address the data scarcity issue, we propose MetaAdapt, a meta learning based approach for domain adaptive few-shot misinformation detection. MetaAdapt leverages limited target examples to provide feedback and guide the knowledge transfer from the source to the target domain (i.e., learn to adapt). In particular, we train the initial model with multiple source tasks and compute their similarity scores to the meta task. Based on the similarity scores, we rescale the meta gradients to adaptively learn from the source tasks. As such, MetaAdapt can learn how to adapt the misinformation detection model and exploit the source data for improved performance in the target domain. To demonstrate the efficiency and effectiveness of our method, we perform extensive experiments to compare MetaAdapt with state-of-the-art baselines and large language models (LLMs) such as LLaMA, where MetaAdapt achieves better performance in domain adaptive few-shot misinformation detection with substantially reduced parameters on real-world datasets.

## introduction recently, significant progress has been made in misinformation detection due to the improvements in developing machine learning-based methods @xcite @xcite . such methods include large language models (llms), which can be fine-tuned for detecting and responding to rumors on social media platforms @xcite @xcite weights source text et al., 2023). however, misinformation on emerging topics remains an elusive challenge for existing approaches, as there exists a large domain gap between the training (i.e., source domain) and the target distribution (i.e., target domain) @xcite . for instance, existing models often fail to detect early-stage misinformation due to the lack of domain knowledge (see figure 1 ). with the increase of emerging topics (e.g., covid-19) on social media as a source of misinformation, the failure to distinguish such earlystage misinformation can result in potential threats to public interest @xcite . to tackle the problem of cross-domain early misinformation detection, one possible solution is crowdsourcing, which collects domain knowledge from online resources @xcite @xcite . another alternative approach is to transfer knowledge from labeled source data to unlabeled target data with domain adaptive methods @xcite @xcite @xcite . however, the former methods use large amounts of human annotations while the latter approaches require extensive unlabeled examples. as such, existing methods are less effective for detecting cross-domain early misinformation, where neither large amounts of annotations nor tar-get domain examples can be provided for training. despite the insufficiency of early misinformation data, limited target examples and annotations can often be achieved at minimal costs @xcite @xcite . nevertheless, previous approaches are not optimized to learn from the source data under the guidance of limited target examples @xcite @xcite . such methods are often unaware of the adaptation objective and thus fail to maximize the transfer of source domain knowledge. to fully exploit existing data from different domains, we consider a cross-domain few-shot setting to adapt misinformation detection models to an unseen target domain @xcite @xcite . that is, given a source data distribution and access to limited target examples, our objective is to maximize the model performance in the target domain. an example of such application can be adapting a model from fake news detection to covid early misinformation detection, where abundant fake news from existing datasets can be used for training the model under the guidance of limited covid misinformation examples. in this paper, we design metaadapt, a few-shot domain adaptation approach based on meta learning for early misinformation detection. specifically, we leverage the source domain examples (i.e., source task) and train a model to obtain the task gradients. then, we evaluate the updated model on the few-shot target examples (i.e., meta task) to derive second-order meta gradients w.r.t. the original parameters. we additionally compute the similarity between the task gradients and meta gradients to select more 'informative' source tasks, such that the updated model adaptively learns (from the source data) to generalize even with a small number of labeled examples. in other words, the meta model learns to reweight the source tasks with the objective of optimizing the model performance in the target domain. therefore, the resulting model can optimally adapt to the target distribution with the provided source domain knowledge. to show the efficacy of our meta learning-based adaptation method, we focus on the early-stage misinformation of covid-19 and demonstrate the performance of metaadapt on real-world datasets, where metaadapt can consistently outperform state-ofthe-art methods and large language models by demonstrating significant improvements. we summarize our contributions as follows foot_0 : ## preliminary we consider the following problem setup for domain adaptive few-shot misinformation detection: labeled source data and k-shot target examples (i.e., k examples per class) are available for training. the objective of our framework is to train a misinformation detection model f that is optimized for the target domain performance using both source and few-shot target examples. data: our research is defined within the scope of single-source adaptive misinformation detection (i.e., we study the adaptation problem from a single source domain to the target domain). we denote d s as the source domain and d t as the (differ- weights source task 2 ) source task 1 (e.g. mexico's richest loses billions on trump victory. → fake) data flow task gradients meta gradients initial model θ weights source label target label few-shot examples (e.g. coronavirus can be treated by injecting disinfectant. → fake) meta loss task loss updated model ϕ 1 updated model ϕ 1 sim-based gradient rescaling ent) target domain. in our setting, labeled source data and limited target examples can be used for training. the few-shot adaptation is performed in two-fold: (1) an initial model is updated upon multiple batches of sampled source data examples (i.e., source tasks) respectively; (2) the updated models are evaluated on the few-shot target examples respectively (i.e., meta task) to compute the meta loss, followed by updating the initial parameters using the derived second-order derivatives. the input data is defined as follows: model & objective: the misinformation detection model is represented by a function f parameterized by θ. f takes textual statements as input and predicts the probability of input as true information, i.e., y (i) = arg max(f (θ, x (i) )). for optimization, our objective is to maximize the model performance on target data x t (note x t ̸ = x ′ t ) from the target domain d t . mathematically, this can be formulated as the optimization problem of minimizing the loss l of θ over target data x t : ## methodology provided with labeled source data and k-shot target examples, we first present our meta adaptation framework for domain adaptive few-shot misinformation detection. to improve the adaptation performance, we introduce a second-order meta learning algorithm metaadapt based on learnable learning rate and task similarity. an illustration of metaadapt is provided in figure 2 . upon deployment, the adapted models achieve considerable improvements thanks to the adaptive optimization and similarity-guided meta adaptation. ## conclusion in this paper, we explore meta learning in domain adaptive few-shot misinformation detection. we propose metaadapt, a meta learning-based approach for cross-domain few-shot adaptation. metaadapt is the first to leverage few-shot target examples for exploiting the source domain knowledge under the guidance of limited target examples. experiment results demonstrate the effectiveness of our method by achieving promising results on multiple challenging source-target dataset combinations, where metaadapt can significant outperform