# Cross-Domain Fake News Detection Using a Prompt-Based Approach
## 2 Related Work  
Large language models (LLMs) have been observed to effectively grasp a vast amount of implicit knowledge [9; 10], and prompting these models holds significant promise for tackling a diverse range of NLP tasks [11]. PLMs utilize two primary training methodologies: pre-training, where models are trained from the ground up using extensive datasets, and fine-tuning, where these models are adapted to specific downstream tasks without requiring complete re-training. In supervised learning scenarios, a classification layer is typically added atop these models to facilitate predictions. However, with the advent of models boasting millions or even billions of parameters, task-specific fine-tuning has presented significant challenges. According to [12], the transformative capabilities of LLMs have led to the emergence of a novel paradigm known as \"pre-training then prompting\", which harnesses PLMs for downstream NLP tasks. This approach differs from the traditional \"pre-training then fine-tuning\" paradigm, which requires updating the model parameters using labeled datasets. Unlike the conventional \"pre-training then fine-tuning\" approach, where model parameters are updated using labeled datasets, \"pre-training then prompting\" enables the direct inference of test data labels. This is achieved by providing LLMs with a few input-output examples, followed by test inputs, without requiring parameter updates. This innovative paradigm streamlines the adaptation of LLMs to specific tasks, offering a more efficient and flexible approach to NLP problem-solving. The concept of prompt-based few-shot fine-tuning has garnered considerable attention. This approach aims to enhance the few-shot learning capabilities of PLMs by integrating label-specific verbalizers and prompts tailored to align with the language models' architecture [13]. Prompt-based few-shot fine-tuning, which adapts models using limited examples within a structured prompting format, has been extensively researched for moderately sized PLMs like BERT [14]. For instance, in [7], the authors introduced Pattern-Exploiting Training (PET), a novel semi-supervised training approach designed to enhance the performance of language models in low-resource settings. PET operates by transforming input examples into cloze-style phrases, which serve as prompts to guide the language model's understanding of the task at hand. These reformulated phrases are then leveraged to assign soft labels to a substantial pool of unlabeled data. In the study described in [13], the authors adapted PET for tasks involving the prediction of multiple tokens. They demonstrated that when combined with ALBERT, both PET and its iterative variant (iPET) surpassed the performance of GPT-3 on SuperGLUE, even when trained with only 32 training examples. PET and iPET achieved this superior performance while utilizing only 0.1% of the parameters of GPT-3. Additionally, the training process with PET was completed within several hours on a single GPU without the need for expensive hyperparameter optimization. These authors also highlighted that similar levels of performance could be achieved without unlabeled data. The authors of [15] introduced LM-BFF (\"Better Few-Shot Fine-Tuning of Language Models\"), a collection of straightforward and complementary techniques designed for fine-tuning language models using only a small set of annotated examples. Key components of their approach include (1) prompt-based fine-tuning coupled with an innovative pipeline for prompt generation automation, and (2) a refined method for dynamically and selectively integrating task demonstrations into each context. These authors conducted a systematic evaluation across various NLP tasks, encompassing classification and regression, to assess the few-shot performance of their methods. Their experiments reveal significant improvements over standard fine-tuning approaches in low-resource settings, with absolute performance enhancements of up to 30% and an average improvement of 11% across all tasks.