Cold-Start Data Selection for Better Few-shot Language Model Fine-tuning: A Prompt-based Uncertainty Propagation Approach (2023)

Abstract: We present PATRON, a prompt-based data selection method for pre-trained language model fine-tuning under cold-start scenarios, i.e., no initial labeled data are available. In PATRON, we design (1) a prompt-based uncertainty propagation approach to estimate the importance of data points and (2) a partition-then-rewrite (PTR) strategy to promote sample diversity when querying for annotations. Experiments on six text classification datasets show that PATRON outperforms the strongest cold-start data selection baselines by up to 6.9%. Besides, with 128 labels only, PATRON achieves 91.0% and 92.1% of the fully supervised performance based on vanilla fine-tuning and prompt-based learning respectively. Our implementation of PATRON will be published upon acceptance.

## introduction pre-trained language models (plms) @xcite @xcite have achieved competitive performance with limited labeled data @xcite for many natural language processing (nlp) tasks. however, there still exists a nonnegligible gap between the performance of fewshot and fully-supervised plms. besides, when the task-specific data for fine-tuning is small, the performance of plms can have high variance @xcite . as illustrated in figure 1 , when fine-tuning roberta-base @xcite on different subsets of ag news dataset with 32 labels, the performance on the test set varies up to 10% for vanilla fine-tuning and 5% for promptbased learning @xcite . such large variations demonstrate the crucial need for strategical selection of training data to improve plms' performance under low-data regimes. to solicit training data intelligently, active learning (al) @xcite has been proposed to adaptively annotate unlabeled data @xcite @xcite @xcite . despite their efficacy, most of these works assume there are hundreds, or even thousands of labels in the initial stage, and query similarly significant amounts of labeled data in each al round. in practice, however, we usually do not have any startup labels to initialize the al process, and the labeling budget can also be limited. this hinders the application of such techniques, as they often rely on a well-trained model with decent uncertainty @xcite , or gradient estimations @xcite to perform well. to facilitate training instance selection on such a challenging low-data regime, cold-start data selection (also known as cold-start al @xcite ) has been proposed, where we have only unlabeled data and zero initial labels, and need to design acquisition functions to effectively query samples for plm fine-tuning. however, cold-start data selection can be nontrivial for plms. due to the absence of labeled data, the estimated uncertainty for unlabeled data from the plm can be biased over classes @xcite . as a result, uncertainty-based approaches can underperform even the random selection strat-egy @xcite . moreover, cold-start data selection requires greater care to ensure the sample diversity compared to the traditional al, as fine-tuning plms on few redundant data will lead to poor generalization. existing approaches often first cluster the whole unlabeled data, and then greedily select samples from each cluster with predefined heuristics @xcite , which fails to control the distance between selected samples and thus cannot yield optimal sample diversity because they fail to control the distance between samples from different clusters. in addition, under cold-start scenarios, it is critical to harness the knowledge from plms for sample selection. while there are several methods that leverage pretrained embeddings @xcite or masked language modeling (mlm) loss @xcite to assist data selection, the mismatch between pre-training and fine-tuning tasks hurts their efficacy. to address the above challenges, we propose patron foot_0 , a prompt-based data-selection strategy tailored for plms. to estimate model uncertainty without access to any labeled data under the coldstart setting, patron leverages prompts @xcite , which convert the classification task into a cloze-style task with customized templates and verbalizers, to generate the task-aware pseudo labels for unlabeled data by predicting the surface name for the [mask] token. in this way, we also bridge the gap between pre-training and downsteam tasks, and distill task-specific knowledge from plms to facilitate data selection. however, one important issue for such pseudo labels is they can be inaccurate and biased even after calibration @xcite . to remedy this, we further propose uncertainty propagation to first measure the correlation between samples based on kernel similarity in the embedding space, and then propagate their prediction uncertainty to their neighbors. thus, a sample will have higher propagated uncertainty only when the predictive uncertainty for both itself and its neighbors are high, indicating the model is less certain for the local region around this sample. to select a batch of diverse samples, we go beyond existing techniques and propose a two stage method named partition-then-rewrite (ptr), which is initially proposed for combinatorial optimization @xcite , to dynamically adjust the selected sample within each cluster. concretely, we first use k-means clustering to partition the unlabeled data and select one sample from each cluster to initialize our solution. we then build a neighbor graph based on k-nearest-neighbor (knn) to encode the neighborhood relationships among selected data and explicitly control the distances between them. after that, we add an additional regularization term to prevent the selected sample in each cluster from being too close to samples in its neighbor clusters. we iterate the above process for several rounds to gradually refine our solution and promote diversity in data selection. we apply patron to various setups: vanilla finetuning, prompt-based learning, semi-supervised learning and standard multi-round al to improve the data efficiency for plm fine-tuning. our key contributions are as follows: (i) a cold-start data selection paradigm patron for addressing the label scarcity issue for few-shot plm fine-tuning; ## related work few-shot language model fine-tuning. our method is closely relevant to label-efficient learning paradigms in nlp such as cold-start finetuning @xcite , prompt-based learning foot_1 @xcite @xcite @xcite , semi-supervised learning @xcite @xcite . these works assume a small set of labeled data is given and focus on training strategies design. instead, we aim to select the most valuable instances from the unlabeled corpus, which is orthogonal to and can be combined with the above methods to enhance label efficiency, as shown in sec. 5.3 and 5.4. ## methodology in this section, we present our method, patron, that exploits prompts for cold-start data selection. we first introduce how to leverage prompts for uncertainty estimation under cold-start scenarios. with the estimated uncertainty, we then propose two key designs, namely uncertainty propagation and partition-then-rewrite (ptr) strategy to balance informativeness and diversity for sample selection. the overall procedure is shown in figure 2 . ## discussion connection to weakly-supervised learning. our method can also be considered as weaklysupervised data selection, where only classindicating keywords are provided. although such formulations have been adopted for nlp tasks @xcite @xcite (see @xcite for a detailed survey), how to effectively leverage such weak supervision signals for data selection has not been widely explored. in this study, we tackle this research problem to facilitate few-shot plm fine-tuning, and demonstrate such task-specific weak supervision is beneficial for downstream tasks. data selection under low and high budget. in this study, we mainly focus on cold-start setting to select data without any labeled data. this is different from traditional al pipelines, and we do not claim patron outperforms al methods under high-budget scenarios. however, experiments show our method shines under low-budget setting, and patron can also be leveraged in earlier rounds of standard al to improve the label efficiency. ## conclusion we developed patron, a data selection method for pre-trained language models (plms) under coldstart scenarios. by leveraging prompts, we can distill the task-specific knowledge from the frozen plm to guide data acquisition. moreover, we develop two techniques, namely uncertainty propagation and predict-then-rewrite (ptr) to achieve both sample representativeness and diversity. the experiments on six text classification tasks demonstrate the advantages of patron against baselines for few-shot plm fine-tuning.