Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning (2024)

Abstract: Parameter-efficient (PE) methods (like Prompts or Adapters) for adapting pre-trained language models (PLM) to downstream tasks have been popular recently. However, hindrances still prevent these methods from reaching their full potential. For example, two significant challenges are few-shot adaptation and cross-task generalization. To tackle these issues, we propose a general PE priming framework to enhance and explore the few-shot adaptation and generalization ability of PE methods. In this framework, PLMs are primed with PE methods for rapidly adapting to various target tasks. To evaluate the generalization ability of these PE methods, we conduct experiments on a few-shot cross-domain benchmark containing 160 diverse NLP tasks. Our experiment not only reveals the best priming strategy but also verifies that priming facilitates the adaptation to target tasks.

## introduction in recent years, pre-trained language models (plms) in natural language processing (nlp) are blooming everywhere @xcite @xcite @xcite @xcite . however, not only the number of plms but also their size is rapidly growing, making it harder to perform full fine-tuning. to address the issue, tons of parameter-efficient fine-tuning (peft) methods have bubbled up, such as adapters @xcite @xcite , or prompts @xcite . these methods have made it equitable for researchers with insufficient computational resources. however, there is still a long way to go for these pe methods to reach their full potential. because the pre-training objectives are not directly related to pe, it is foreseeable that there is a mismatch between the plm and pe methods, which may prevent pe methods from unleashing their full power. to address figure 1 : we propose a general framework to improve the performance of parameter-efficient fine-tuning. we prime the plm with source tasks for parameter-efficient methods. some recent studies explore how to bridge the gap between pre-training tasks and target tasks. @xcite pre-trains the soft prompt tokens with self-supervised tasks to give a better initialization. @xcite ; @xcite exploits optimization-based meta-learning to find an initialization for soft prompts to facilitate faster adaptation to new tasks. @xcite tweaks the meta-learning algorithm maml @xcite for priming and simulates the peft procedure in the inner loop. however, they only focus on priming for a single pe method with a specific algorithm. in contrast, our work views priming as a general method to boost peft from a higher per-1 spective. moreover, previous works explore only single-domain tasks, which lack the exploration of generalization ability. on the contrary, our work evaluate pe methods with diverse nlp tasks in various domains. on top of that, we conduct comprehensive experiments over well-known pe methods like adapters and prompt tuning under different settings. the experiment result reveals that priming by tuning only plm leads to the best adaptation result on target tasks. in addition, we shows priming does help the whole model to converge more easily, which validates the necessity of priming. ## analysis in this section, we emphasize the advantages of priming by examining training loss during downstream fine-tuning. we compare loss curves between models w/ or w/o priming (ft_p and mtl m_p, respectively) across three diverse tasks from the target task training set. these tasks, unseen by prompts/adapters, feature distinct evaluation metrics. figure 4 presents the results, where the blue curves represent ft_p (w/o priming) and the orange curves represent mtl m_p (w/ priming). the primed model not only converges faster but also achieves a superior final level of loss. furthermore, the orange curves exhibit steadier convergence compared to the fluctuating and glitch-prone blue curves. these findings underscore the benefits of priming for prompts/adapters, facilitating rapid adaptation to various target tasks. ## conclusion in this paper, we systematically analyze priming peft within a comprehensive framework. our framework not only incorporates existing priming approaches but also explores previously uncharted strategies. our experimental results demonstrate that the majority of priming strategies enhance the performance of pe methods. notably, "priming plm only" emerges as the top-performing strategy when used in conjunction with multi-task learning. crucially, our study provides concrete evidence that priming significantly facilitates the convergence of fine-tuning prompts/adapters on unseen tasks, underscoring the efficacy of priming.