# Multi-Stage Prompt Tuning for Political Perspective Detection in Low-Resource Settings
## 3 Methodology
### Domain-Specific Prompt Tuning on MP3 Task
#### 3.1.2 Masked Political Phrase Prediction Task (MP3)  
Motivated by the success of masking contiguous spans, noun phrases, and entities [17; 18; 19], we introduce the MP3 task, which identifies political-issue-specific phrases and focuses on masking them. An example of MP3 is illustrated in Figure 2.  
Figure 1: Multi-stage prompt tuning strategy. **Constructing Political Lexicon** We first construct a lexicon of political terms to inject the political knowledge into domain-specific prompts with MP3. We collect plain news article texts from the politics sections of various news media (we crawled articles from New York Times, Washington Post, Daily Caller, and The Hill) and a benchmark news dataset (we excluded 645 articles with golden labels) [7]. We identify political terms in the corpus by calculating the phrase score (P-score) based on a simple data-driven approach [20] as follows:  
\[P-score(\mathrm{w}_{i},\mathrm{w}_{j})=\frac{count(\mathrm{w}_{i}\mathrm{w}_{ j})-\delta}{count(\mathrm{w}_{i})\times count(\mathrm{w}_{j})}, \tag{8}\]  
where \(\mathrm{w}_{i}\) and \(\mathrm{w}_{j}\) are adjacent words in the corpus. We use \(\delta\) to prevent excessive phrases consisting of very infrequent words from forming. We use bigrams with scores greater than the threshold as phrases (we empirically set the \(\delta\) value and threshold to 100). We then repeat the same approach once or twice to identify longer phrases (i.e., trigrams and 4-grams). Finally, the identified terms include 102,849 bigrams, 34,776 trigrams, and 10,161 4-grams. **Extracting Political-Issue Phrase** We may not need to use every phrase for post-training because some phrases are meaningless regarding ideological confrontation. Therefore, we extract political-issue-specific phrases from the political lexicon. Inspired by [21], we first collect 23 major and typical political issues in the United States (e.g., abortion, drugs, gun control, health care, and immigration) from OnTheIssues (www.ontheissues.org, accessed on 16 February 2021). Second, we use human political knowledge to extend typical political issues to fine-grained political events or issues. We asked a human expert to provide dozens of fine-grained political issues for each typical issue. We provided hotly debated political events and issue candidates from ISIDEWITH (www.isidewith.com/polls, accessed on 16 February 2021) and Ranker (www.ranker.com, accessed on 16 February 2021). For instance, the human expert can attach fine-grained issues such as \"skilled immigration\" or \"border wall\" to \"immigration\". Finally, we extract the political-issue-specific term by calculating the point-wise mutual information (PMI) [22] with fine-grained political issues. We compute the PMI for a pair of a phrase \(p\) and an issue \(i\) as follows:  
\[PMI(p,i)=log\frac{P(p,i)}{P(p)P(i)}, \tag{9}\]  
where \(P(i)\) is the probability that \(i\) or its fine-grained issues appear in the news articles. We discard phrases that appear in more than 10% of the unlabeled articles. Finally, our proposed model uses 500 top-PMI issue-specific phrases for each of the issues. In addition, we add the single words comprising the above phrases to the lexicon. Figure 2: The whole architecture of MP3. 1 means that the PMI score is not logged for readability. **Masked Phrase Prediction** After extracting the political phrases, we mask them in the new corpus to inject domain knowledge into the domain-specific prompts. First, we search for political phrases mentioned in the corpus. Given a sequence of tokens \(\mathbf{T}\) = (\(t_{1}\), \(t_{2}\),..., \(t_{n}\)), we detect political phrases \(\mathbf{P}\) and randomly mask them. We mask approximately 50% of the total occurring \(\mathbf{P}\). Because political phrases are not always sufficiently present in the sequence, we additionally sample random words until the masking budget is exhausted (approximately 15% of all tokens). We sample sequences of complete words (rather than subword tokens).