# Cross-Domain Fake News Detection Using a Prompt-Based Approach
## 3 Methodology
### Task Formulation  
Our work tackles FND as a zero-shot binary text classification problem, leveraging the capabilities of PLMs like GPT2 [19]. We achieve this using prompt-based fine-tuning. This framework is agnostic to the specific PLM chosen, allowing for flexibility in applying it to various pre-trained models. Our method employs a prompt-based approach to adapt the model for the specific task of fake news classification. In text classification, the aim is to categorize an input text \(X=\{x_{0},x_{1},\cdots,x_{i}\}\) into classes \(Y=\{y_{0},y_{1}\}\). The prompt-based fine-tuning approach utilizes a template \(P(h)\) to guide the model toward the desired task. This template acts as a guiding instruction, for example, in classifying news articles. Additionally, a verbalizer is used in our model. The verbalizer consists of relevant words from the vocabulary \(V\) associated with each class. It essentially maps the model's predicted word probabilities to task-specific labels. This mapping function, denoted as \(f:v\in V\to y\in Y\), translates predicted word probabilities into class probabilities, effectively transforming the classification task into a cloze-style language modeling problem. We can illustrate this process with the following example. Given a sequence of tokens \(x_{i}\), the process entails constructing a prompt input by concatenating it with a standardized prompt sentence. Formally, \(x_{i}+\) prompt is defined as \(x_{i}\) appended with the prompt sentence: \"The label of this news is \(\underset{-}{\text{ }}\)\". Within this formulation, a masked token is utilized to signify the position where the model predicts the class label \(y_{i}\), belonging to the set \(Y\), for the input \(x_{i}\). Subsequently, the \(x_{i}+\) prompt is inputted into the language model (LM) to generate a probability distribution for each word \(v\) in the vocabulary \(V\), representing potential candidates to fill the masked position. Prior to it being inputted into the LM, the input text is encoded alongside the prompt, employing necessary truncation and padding  
Figure 1: Figure (**a**) illustrates the prompt-based GPT2 baseline, while (**b**) presents the proposed _context-aware_ prompt-based GPT2-D architecture. First, the input text \(x_{i}\) plus the prompt is fed into a natural language template. The pre-trained language model (PLM) then makes a prediction by filling in the blank and mapping the result to the default class (fake/real) using the verbalizer. The key distinction is that the architecture in (**b**) employs a contextualized prompt by injecting domain-based information ([DOMAIN]) to enhance performance. techniques to ensure that the entirety of the prompt is integrated within the model's input. Then, the probability of label \(y\) for the \(x_{i}\) prompt can be calculated as follows:  
\[p(y|x_{i})=LM(\frac{}{}=v|x_{i})|v\in V_{y} \tag{1}\]  
\[p(fake|x_{i})=\frac{e^{V_{fake}\cdot h}}{e^{V_{fake}\cdot h+e^{V_{real}\cdot h}}} \tag{2}\]  
where \(h\) is the hidden representation. The training process involves sampling a subset of labeled data while optimizing the model's parameters using binary cross-entropy loss (BCEL) between the predicted label (i.e., the label \(j\) with the highest probability) and the true label distribution:  
\[BCEL=-\sum_{i=1}^{|S|}\sum_{j=1}^{|L|}y_{j}^{i}\log P_{j}^{i} \tag{3}\]  
The model is iteratively trained for three epochs. In the second scenario, we chose to utilize the conditional context-aware prompt approach (Figure 1b) to explore the impact of incorporating domain information as contextual input data for the model. Like the baseline approach (Figure 1a), this approach constructs the input text by concatenating the provided news text with a prompt. However, this prompt now includes a placeholder denoted by [DOMAIN]. For example, consider the news text \"Celebrity announces shocking news\" and the prompt \"This news is about [DOMAIN], the label of this news is\".