Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning (2022)

Abstract: Prefix-tuning, or more generally continuous prompt tuning, has become an essential paradigm of parameter-efficient transfer learning. Using a large pre-trained language model (PLM), prefix-tuning can obtain strong performance by training only a small portion of parameters. In this paper, we propose to understand and further develop prefix-tuning through the kernel lens. Specifically, we make an analogy between prefixes and inducing variables in kernel methods and hypothesize that prefixes serving as inducing variables would improve their overall mechanism. From the kernel estimator perspective, we suggest a new variant of prefix-tuning— inducer-tuning , which shares the exact mechanism as prefix-tuning while leveraging the residual form found in adapter-tuning. This mitigates the initialization issue in prefix-tuning. Through comprehensive empirical experiments on natural language understanding and generation tasks, we demonstrate that inducer-tuning can close the performance gap between prefix-tuning and fine-tuning.

## introduction transfer learning from large pre-trained language models (plms) has been the de-facto method to tackle downstream natural language processing (nlp) tasks with proven performance and scalability @xcite . among all the adaption techniques, fine-tuning @xcite is predominant for plms and maintains the models' architecture while updating all the parameters within. though powerful, fine-tuning is considered parameter-inefficient since it results in separate copies of model parameters for each task/client after training. with the sizes of plms increasing to hundreds of millions @xcite or even up to tril-lion @xcite parameters, the trend motivates a range of parameter-efficient adaptation techniques, including adapter-tuning and prompting, as promising lightweight alternatives to finetuning to reduce computational consumption and storage space. adapter-tuning inserts bottlenecked multi-layer perception (mlp) modules between the pre-trained layers of plms and tunes only these new parameters for task adaptation @xcite . prompting, instead, aims to adapt the general-purpose plms through prompts, whose effectiveness has been shown on a frozen gpt-3 model @xcite ). an implicit drawback of the prompt-based adaptation is the difficulty of searching for the proper prompt. to avoid manually designing the prompts, @xcite propose a search algorithm to find the effective prompt over discrete space of vocabularies; prefix-tuning @xcite and other concurrent methods @xcite further extend the discrete search to continuous prompts, attaining performance close to fine-tuning in some tasks. despite the effort, there is still a performance gap between "prefixtuning" and "fine-tuning" in many tasks, especially when the model size is small @xcite . in addition, the mechanism of prefix-tuning is still poorly understood and underexplored. prefix-tuning is also similar to adaptertuning, since they both insert additional modules into each transformer layer (classical prompt-based methods @xcite only add prompts to the embedding layer). scrutinizing the evolution of prompt-based methods, we can observe they have gradually deviated from the concept of "prompts". compared to the manually designed prompts, the discrete search usually results in counter-intuitive prompt tokens, which vaguely match the topic but are not as sensible as the manual one; for continuous prompt tuning, it even breaks the limit of the existing vo-cabulary. all these pieces imply that the mechanism behind prompt-based tuning might be more complicated than guiding the output through hint prompts. to open the black box of "prompts", in this work, we propose to consider the prompts (either hard or soft) as "inducing variables" in kernel methods @xcite . this analogy is justified due to the close connection between attention modules in plms and kernel estimators @xcite @xcite . this kernel perspective explains the potential mechanism of prefix-tuning and motivates a new method, inducer-tuning. specifically, inducertuning freezes all the original parameters in the plms as other prompt-based methods; when computing the attention output for a certain input token in each layer, inducer-tuning utilizes a point close to the query vector as the "inducer". this unique "soft prompt" eases the search for appropriate prompts and builds a new connection between "prompting" and "adapter-tuning". in summary, the contribution of this work is three-fold: 1 we explain the underlying mechanism of prefix-tuning as the inducing variables in kernel learning. 2 we propose a new parameterefficient adaptation technique, inducer-tuning, to further improve prefix-tuning. 3 through comprehensive empirical studies, we verify our proposed method can close the gap between "prefix-tuning" and "fine-tuning" on relatively small plms, and provide a tighter lower bound on the potential of continuous prompt tuning. ## related work in this section, we briefly introduce the classical form of adapter-tuning and mainly focus on the different variants of prompting. adapter-tuning. compared to fine-tuning all the parameters in the plms, @xcite , @xcite propose to modulate the output of a transformer layer through inserting additional small-bottleneck mlp layers (adapters) @xcite prompting. prompting prepends task-specific instructions to the task input and was originally demonstrated in @xcite . as manual prompts rely on trial and error, jiang et al. (2020), shin et al. (2020) suggests search algorithms to specify the prompts among all the tokens in the vocabulary. prompt-tuning @xcite and p-tuning @xcite remove the vocabulary restriction on prompts by using trainable "soft prompts". the prompts in the aforementioned methods are only inserted into the bottom embedding layer of plms, while prefix-tuning @xcite adds soft prompts to all the transformer layers to further increase the capacity of prompting. though effective, proper initialization of the soft prompts remains challenging. to mitigate the issue, li and liang (2021) used an extra mlp to reparameterize the prompts in each layer, thus adding more parameters that need training; spot @xcite suggests performing pre-training for soft prompts using a wide range of nlp tasks, which requires additional computational resources. in contrast, though adapters have a similar expression form to prefix-tuning @xcite , adaptertuning only requires regular initialization. we speculate that the residual form of adapters mitigates the initialization issue since the output of each layer in the new model would be centered around the output in the frozen plms, and the residual form contributes to gradient back-propagation as in skip connection. we rely on this intuition and utilize the above-mentioned advantages of adapters to guide the design of our proposed inducer-tuning. ## preliminaries: transformer layers before discussing the mechanism of prompt-tuning, we introduce the structure of transformer layers and necessary notations in this section. a general transformer-based plm is mainly composed of l stacked layers. each layer contains a multi-headed self-attention and a fully connected feed-forward network (ffn) sub-layer, both followed by an "add & norm" module @xcite . 2 hereon, we shall focus on the structure of the attention sub-layer since prefix-tuning directly works on this sub-layer. passing a length-n input sequence @xmath0, we first perform linear transforms to the input x and obtain the query matrix (q), the key matrix (k), and the value matrix (v ) as: ## parameter-efficient inducer-tuning we describe the motivation and the mechanism of inducer-tuning in this section. we first revisit the connection between self-attention and kernel estimators in § 4.1, which interprets attention from another perspective by considering query, key, and value matrices as three separate sets of vectors rather than the related representations of the same input sequence. this special perspective motivates and justifies the inducer-tuning we propose in § 4.3. ## experiments while prefix-tuning has been shown comparable to fine-tuning on some natural language understanding (nlu) tasks @xcite , there is still a performance gap between prefix-tuning and finetuning on natural language generation (nlg) tasks, especially for those tasks with long input sequences. complete settings of the experiments below can be found in appendix a and appendix b. the code for our algorithms is publicly available at https://github.com/ychen-stat-ml/kerneladapters . ## conclusion in this work, we connect attention modules to nadaraya-watson kernel estimators and review prefix-tuning from a kernel estimator perspective. we speculate that continuous prompt tuning prompts serve as inducing variables in kernel methods. following this principle, we propose inducertuning, which customizes an "inducer" for each query vector from the input sequence and adopts a residual form to resolve the initialization issue in prefix-tuning. in addition, the perspective implies a potential limitation of prompt-based methods: the positional interactions in attention cannot adapt to the new tasks in prompt tuning. we empirically demonstrate that our proposed method performs better on nlu and nlg tasks.