Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation (2022)

Abstract: Prompt tuning, or the conditioning of a frozen pretrained language model (PLM) with soft prompts learned from data, has demonstrated impressive performance on a wide range of NLP tasks. However, prompt tuning requires a large training dataset to be effective and is outperformed by finetuning the entire PLM in data-scarce regimes. Previous work (Gu et al., 2022, Vu et al., 2022) proposed to transfer soft prompts pretrained on the source domain to the target domain. In this paper, we explore domain adaptation for prompt tuning, a problem setting where unlabeled data from the target domain are available during pretraining. We propose bOosting Prompt TunIng with doMain Adaptation (OPTIMA), which regularizes the decision boundary to be smooth around regions where source and target data distributions are similar. Extensive experiments demonstrate that OPTIMA significantly enhances the transferability and sample-efficiency of prompt tuning compared to strong baselines. Moreover, in few-shot settings, OPTIMA exceeds full-model tuning by a large margin.

## introduction prompt tuning @xcite @xcite is an effective method for adapting largescale pretrained language models for downstream tasks. while keeping the plm weights unchanged, prompt tuning trains input vectors, called soft prompts, that are input to the plm alongside the text embeddings. compared to other adaptation techniques for plms, such as adapter @xcite @xcite , @xcite , bitfit @xcite , lora @xcite , and ladder side-tuning @xcite , the advantage of prompt tuning is that it does not require addition or change of model parameters. as a result, with prompt tuning, we can easily specialize one neural network (possibly deployed on a large number source-domain classes target-domain classes of servers or as application-specific integrated circuits) to support many different tasks by simply switching out the soft prompt in the input, which greatly simplifies model deployment and maintenance. however, training effective soft prompts usually requires sufficient labeled training data @xcite . studies have shown that prompt tuning significantly underperforms full-model tuning on many few-shot classification tasks @xcite . our experiments corroborate this finding. in addition, we find that, in few-shot learning, prompt tuning is equally, if not more, sensitive to random seed choices compared to full-model tuning, despite having far fewer trainable parameters ( §3.4). @xcite address this by transferring prompts learned from a source domain to the target domain with limited training data. in this paper, we investigate a related but different scenario, unsupervised domain adaptation (uda) @xcite , where unlabeled data from the target domain are available. such situations are common when data are abundant but the labeling cost, including annotator recruitment, annotator training, and quality assurance, is high. utilizing unlabeled examples can be an effective approach toward enhancing the data efficiency of prompt tuning. we propose boosting prompt tuning with do-main adaptation (optima). employing regularization from adversarial perturbation, optima learns a smooth decision boundary that passes through regions of low data density. in addition, recognizing that the feature distributions in the two domains may overlap only partially, we propose to focus the regularization to regions where the target-domain and source-domain data exhibit high similarity. we illustrate the intuition in figure 1 . the popular domain-adversarial neural network (dann) technique @xcite encourages the network to learn domain-invariant features and optimizes for both a domain-specific task loss and a domain discrimination loss. however, the two losses could compete against each other, leading to optimization difficulties @xcite . empirically, dann exhibit low performance for prompt tuning. we hypothesize that the low capacity of prompts worsens the optimization problem. to solve this issue, in optima, we create input disturbance vectors that optimize for domain similarity, so that the prompt needs to optimize for only the task loss. the separation leads to excellent results. experiments shows that optima learns effective data representations that transfer well to the target domain under zero-shot and few-shot settings. optima outperforms eight baselines, including state-of-the-art transfer learning techniques such as spot @xcite . our contributions include the following. ## domain adaptation for prompt tuning in this section, we first introduce prompt tuning for text classification. then, we introduce how to enhance in-domain generalization performance of soft prompts by augmenting the input with virtual perturbations. next, we propose how to optimize the perturbations to reduce the domain gap and obtain soft prompts with domain-invariant knowledge. finally, we show how to use the soft prompts to boost few-shot learning in the target domain. ## experimental evaluation we evaluate the representations learned by op-tima under zero-shot and few-shot settings. ## conclusions in this paper, we propose optima to enhance soft prompt transfer performance by regularizing the training on source domain under perturbations generated with domain adaptation. we extensively evaluate the proposed method. compared to competitive baselines, soft prompt trained with op-tima generalizes better to the source domain and significantly boosts zero-shot and few-shot learning in the target domain. we observe that pre-training soft prompts on a similar dataset confers more benefits than pre-training on a disimilar dataset. we expect the current work to contribute to the wide deployment of plms.