FPT : Improving Prompt Tuning Efficiency via Progressive Training (2022)

Abstract: Recently, prompt tuning (PT) has gained increasing attention as a parameter-efficient way of tuning pre-trained language models (PLMs). Despite extensively reducing the number of tunable parameters and achieving satisfying performance, PT is training-inefficient due to its slow convergence. To improve PT’s training efficiency, we first make some novel observations about the prompt transferability of “partial PLMs”, which are defined by compressing a PLM in depth or width. We observe that the soft prompts learned by different partial PLMs of various sizes are similar in the parameter space, implying that these soft prompts could potentially be transferred among partial PLMs. Inspired by these observations, we propose Fast Prompt Tuning (FPT), which starts by conducting PT using a small-scale partial PLM, and then progressively expands its depth and width until the full-model size. After each expansion, we recycle the previously learned soft prompts as initialization for the enlarged partial PLM and then proceed PT. We demonstrate the feasibility of FPT on 5 tasks and show that FPT could save over 30% training computations while achieving comparable performance. The codes are publicly available at https://github.com/thunlp/FastPromptTuning.

## introduction the emergence of pre-trained language models (plms) has broken the glass ceiling for various nlp tasks @xcite . versatile semantic and syntactic knowledge acquired during pre-training could be leveraged when plms are adapted towards a specific downstream task to boost performance. the de facto strategy for such an adaptation is full-parameter fine-tuning, which is computationally expensive and profligate since it requires tuning and storing all the parameters in the plm for each downstream task. to remedy this, several delta tuning @xcite (also known as parameter-efficient tuning) algorithms are proposed in place of the vanilla fine-tuning @xcite @xcite , among which prompt tuning (pt) @xcite has gained increasing attention recently. pt prepends a few virtual tokens to the input text, these tokens are tuned during training while all the other plm parameters remain frozen. despite its simple form, pt has been demonstrated to achieve remarkable performance in various nlp tasks. especially when the scale of the plm becomes extremely huge, pt could achieve comparable performance to finetuning @xcite . despite extensively reducing the number of tunable parameters and achieving satisfying performance, pt is criticized to be training-inefficient due to the slow convergence @xcite as illustrated in figure 1 , and such incompetence would limit the practical application of pt. hence in this paper, we explore how to improve pt's training efficiency. our motivation is based on novel observations about the prompt transferability among "partial plms". here a partial plm is defined by compressing a plm in depth or width, which is implemented by dropping several layers or masking part of the connections in the feed-forward network (ffn) in each transformer @xcite layer. we observe that the soft prompts of the same task learned by different partial plms of various sizes tend to be close in the parameter space, implying that these soft prompts could potentially be transferred among different partial plms. inspired by the above observations, we propose fast prompt tuning (fpt), which starts by conducting pt using a small-scale partial plm to obtain the corresponding soft prompts. after that, we progressively expand the partial plm's depth and width until the full-model size by rehabilitating the dropped layers and masked neurons. after each expansion, we recycle the previously learned soft prompts as initialization for the enlarged plm and then proceed pt. since the partial plm requires fewer computations for each step, keeping the total training steps unchanged, we could reduce the overall computations consumed, and in the meantime, achieve comparable pt performance. in experiments, we demonstrate the feasibility of fpt on 5 nlp tasks. the experimental results show that fpt could save around 30% training computations and achieve satisfying downstream performance. ## conclusion in this work, towards improving pt's training efficiency, we first make several insightful observations by conducting pt on partial plms, and then propose fpt based on the observations. the results on 5 datasets demonstrate the feasibility of fpt in saving the training computations. being the first attempt towards accelerating pt, we encourage future work to design more sophisticated algorithms to further improve pt's training efficiency.