PTP : Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer (2023)

Abstract: Recent studies show that prompt tuning can better leverage the power of large language models than fine-tuning on downstream natural language understanding tasks. However, the existing prompt tuning methods have training instability issues, as the variance of scores under different random seeds is quite large. To address this critical problem, we first investigate and find that the loss landscape of vanilla prompt tuning is precipitous when it is visualized, where a slight change of input data can cause a big fluctuation in the loss landscape. This is an essential factor that leads to the instability of prompt tuning. Based on this observation, we introduce perturbation-based regularizers, which can smooth the loss landscape, into prompt tuning. We propose a new algorithm, called Prompt Tuning with Perturbation-based regularizer (PTP), which can not only alleviate training instability dramatically but also boost the performance of prompt tuning. We design two kinds of perturbation-based regularizers, including random-noise-based and adversarial-based. In particular, our proposed perturbations are flexible on both text space and embedding space. Extensive experiments show the effectiveness of our proposed methods in stabilizing the training. Our new algorithms improve the state-of-the-art prompt tuning methods by 1.94% and 2.34% on SuperGLUE and FewGLUE benchmarks, respectively.

## introduction releasing the burden of training models from scratch while keeping the outstanding performance on downstream tasks, pretrained language model (lm) brought nlp to a new era @xcite @xcite . recently, inspired by the success of gpt-3 @xcite on few-shot and zero-shot learning with manually created prompts, there has been a surging interest in prompting that freezes input sequence 𝑠 pre-trained lm and wraps up the input sequence with natural language templates. however, natural language prompts are handcrafted by experts and the performance is not comparable with finetuning methods. to tackle this issue, @xcite ; @xcite proposed prompt tuning, which prepends the input sequence with continuous embeddings and only tunes these embeddings during training. @xcite verified the effectiveness of prompt tuning on natural language understanding (nlu) tasks under both few-shot learning and supervised learning settings, which is comparable to the fine-tuning methods but with much fewer (1000× less) task-specific tunable parameters. however, under different random seeds, we observe that the current prompt tuning methods suffer from a high variance of scores, which indicates they suffer from training instability issues. to investigate the factor that causes the instability of prompt tuning, we visualize the loss landscape of the vanilla prompt tuning and observe that there exist many sharp crests, as shown in figure 2 (a), which harms the training stability. motivated by the recent study @xcite which shows that perturbation-based regularizers are powerful tools to smooth the loss landscape and stabilize the training of machine learning systems, we introduce them into prompt tuning to address the lack of stability and generalization issues. to be specific, we propose prompt tuning with perturbation-based regularizer (ptp) algorithm to make the training stable and boost the performance of prompt tuning. specifically, we consider two kinds of perturbations in ptp, random-noise-based (ptp-rn) perturbation and adversarial-based (ptp-adv) perturbation. ptp-rn is motivated by randomized smoothing @xcite , which applies the neighborhood averaging and makes the neural network smoother while ptp-adv is motivated by adversarial training, a method proposed to make the predictor capable of resisting the adversarial examples @xcite as well as boosting the clean accuracy of the predictors @xcite . moreover, in order to bring more flexibility and make our exploration more comprehensive, we apply perturbations to both text (discrete) and embedding (continuous) space, as depicted in figure 1 . in the experiments, we conduct extensive experiments to evaluate our proposed algorithms, ptp-rn and ptp-adv, on superglue @xcite and fewglue @xcite ) benchmark. by applying the ptp algorithm on text or embedding space to the existing prompt tuning methods, we can boost the performance of prompt tuning on superglue and fewglue benchmarks by 1.94% and 2.34% as well as make prompt tuning more stable. we also present a comprehensive ablation study and analysis of our algorithms with different perturbations. our contributions can be summarized as: it also shows the great power of our algorithm in improving training stability. ## experiments we conducted empirical studies on two popular natural language understanding (nlu) benchmarks: superglue benchmark @xcite and fewglue benchmark @xcite . we tested the proposed framework in both fully-supervised and few-shot settings to verify the effectiveness of our proposed ptp-rn and ptp-adv algorithm with perturbations on both text and embedding space. to further verify the effectiveness of our algorithm, we also conduct experiments on text summarization tasks @xcite . ## ablation study in this section, we conduct comprehensive ablation studies of our perturbation methods under both fully-supervised and few-shot learning settings. ## conclusion in this paper, we first investigated the training instability issues on prompt tuning, which has a precipitous loss landscape in its visualization. to tackle the problem, we proposed ptp-rn and ptp-adv algorithms, which include four different perturbations (rg, rm, adv, a2t) on both discrete and continuous spaces, to smooth the loss landscape and make the training stable. furthermore, our algorithms are also capable of boosting the performance of prompt tuning. the extensive experiments validate the effectiveness of our proposed algorithms on nlu tasks under both fully-supervised and few-shot settings.