# Multi-Stage Prompt Tuning for Political Perspective Detection in Low-Resource Settings
## 1 Introduction  
Political perspective detection in news media is more challenging than that in political texts. It is a document-level classification task that seeks to identify author biases (e.g., left-leaning, right-leaning, and no bias) in news articles. Political perspective detection in news media is vital in various applications, such as political polling and news recommendation systems [1, 2]. In addition, identifying perspective differences can help lay the foundation for the automatic detection of false content and rumors [2]. Identifying the author biases in news articles may be more difficult than in political texts. This is primarily attributed to the fact that news articles endeavor to maintain credibility and an appearance of impartiality. This may be a challenging task even for people (if they lack political knowledge). Moreover, with long sequences and a large amount of jargon, labeling the political perspective on documents would be labor-intensive and expensive. Under these circumstances, studying political perspective detection systems in low-resource settings plays a critical role. Prompt-based learning (i.e., discrete prompting and prompt tuning), which freezes all parameters of a pre-trained language model and steers the model by prepending naturallanguage or continuous prompts to the input, has achieved notable results in low-resource settings [3, 4, 5]. In [3], the authors prepend task instructions in a natural language form and a few examples to the input to perform downstream tasks. They achieve promising results in a wide range of natural language processing (NLP) tasks without updating parameters. In [4], they reformulate input examples as cloze-style phrases to help language models understand a given task and achieve significant improvements over fine-tuning (i.e., model tuning), modifying all model parameters, in natural language understanding (NLU) tasks. In another line of work [5], they propose prefix-tuning, which prepends trainable continuous prompts (i.e., prefix) to the input and outperforms fine-tuning in low-data settings for natural language generation (NLG) tasks. Various methods have emerged to learn a new task with very few samples, but prompt-based learning basically benefits from prior knowledge stored in pre-trained language models. Despite promising results, some challenges remain in applying prompts to political perspective detection tasks. Specifically, prompt-based learning results in performance degradation when the distribution of inputs differs between the source (e.g., the general domain) and target domains (e.g., the political domain) [6]. Intuitively, the significant differences between the source and target domains in the terminologies used and their contextual meanings evoke this phenomenon. For example, \"abortion issue\" is not a commonly used term but is used relatively frequently in the political domain. Moreover, the expression \"right\" is generally used to indicate the meaning \"correct\" or a direction, but in the political domain, it is mainly used to indicate democracy. Therefore, language models pre-trained on a general corpus may suffer from prompt-based learning on political perspective detection tasks. To address these issues, we develop a multi-stage continuous prompt tuning framework for political perspective detection in low-resource settings. In the first stage, we inject political knowledge into short domain-specific prompts by post-training on an unlabeled political news corpus. We carefully design a post-training task called masked political phrase prediction (MP3). To this end, we construct and utilize background knowledge (i.e., political-issue-specific phrases). Our framework then masks the political-issue-specific phrases in an unlabeled political news corpus. Finally, we train the domain-specific prompts using a frozen language model to predict the contiguous masked tokens corresponding to the selected political phrases.