Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning (2023)

Abstract: Fine-tuning large pre-trained language models on various downstream tasks with whole parameters is prohibitively expensive. Hence, Parameter-efficient fine-tuning has attracted attention that only optimizes a few task-specific parameters with the frozen pre-trained model. In this work, we focus on prefix tuning, which only optimizes continuous prefix vectors (i.e. pseudo tokens) inserted into Transformer layers. Based on the observation that the learned syntax and semantics representation varies a lot at different layers, we argue that the adaptive prefix will be further tailored to each layer than the fixed one, enabling the fine-tuning more effective and efficient. Thus, we propose Adaptive Prefix Tuning (APT) to adjust the prefix in terms of both fine-grained token level and coarse-grained layer level with a gate mechanism. Experiments on the SuperGLUE and NER datasets show the effectiveness of APT. In addition, taking the gate as a probing, we validate the efficiency and effectiveness of the variable prefix.

## introduction vanilla fine-tuning strategy usually adjusts all the parameters to adapt the pre-trained language model to downstream tasks. parameter-efficient learning @xcite @xcite @xcite is an emerging framework that freezes the pre-trained model and only tunes a few number of task-specific parameters for downstream tasks. for instance, prefix tuning @xcite prepends length-equivalent pseudo prefix tokens, i.e. continuous task-specific vectors to each layer of the pre-trained model, achieving comparable even superior performance with only 0.1-3% parameters. in previous works, the length of prefix tokens (or the number of trainable parameters) is usually the same at each layer. however, a potential observation lies in that the structure information and representational capacity embedded in each layer are prone to be inconsistent @xcite . it is generally considered that the bottom layers of the language model tend to capture concrete and shallow phrase-level features, while the top layers concerns more with abstract semantic information @xcite . based on the perspective, we assume adaptive prefix can grab the emphasis more flexibly to adapt to various downstream tasks. in light of above motivation, we investigate the adaptive prefix in this work. we propose adaptive prefix tuning (apt) with an adaptive gate mechanism at both fine-grained token level and coarsegrained layer level. specifically, as shown in figure 1 , for fine granularity, apt scores each individual prefix token via gated weight assignment. then, the scaled weight is utilized to balance the inserted task-specific prefix tokens and original input tokens for current layer at coarse-grained level. extensive experiments against prefix tuning on the sentence and token classification tasks in full data and low resources setting validate the effectiveness of apt. in addition, the gate learned from apt could be served as a probing for the number of necessary parameters in different layers, guiding us to directly apply variable prefix to the original prefix tuning. the probing experiment further demonstrates the effectiveness of adaptive prefix. ## related works since fine-tuning the whole model is prohibitively expensive, parameter-efficient language model finetuning becomes a lightweight alternative that only optimizes a small number of parameters while keeping most pre-trained parameters frozen @xcite . adapter tuning @xcite inserts two tunable task-specific modules after multihead attention and feed-forward network, achieving comparable performance with only 2-4% of the parameters. prompt tuning @xcite and prefix-tuning (li and liang, 2021) only train soft prompts by adding prefix tokens to the input or hidden states. recently, @xcite extend the prefix tuning to the natural language understanding tasks, which matches the performance of fine-tuning with only 0.1%-3% tuned parameters. furthermore, with an overlap of our motivations that each layer of the pre-trained language model focuses on different aspects of feature for various tasks @xcite and extra parameters are probably not necessary for certain tasks @xcite @xcite , adaptable adapters @xcite selects beneficial adapter layers and learns task-specific activation function for downstream tasks to make adaptor dynamic for each task and layer. in addition to different frameworks (adapter versa prefix tuning), our key difference from their work lies in that we aim to dynamically filter required information at each layer in a soft way, while they choose whether to add trainable modules at the layer level in a hard manner. ## conclusion in this paper, we investigate prefix tuning and assume that adaptive prefix is probably more efficient and effective than fixed prefix. firstly, we propose apt that leverages the token-level and the layerlevel gate mechanism which achieves an improvement of performance over original prefix tuning. then, we illustrate the weight distribution learned by apt and take it as a probe, which validates the variable prefix can work better than the fixed one. the above experiments and analysis demonstrate that the adaptive prefix can be served as a promising strategy for parameter-efficient fine-tuning.