Selective Prefix Tuning for Pre-trained Language Models (2024)

Abstract: The prevalent approach for optimizing pre-trained language models in downstream tasks is fine-tuning. However, it is both time-consuming and memory-inefficient. In response, a more efficient method called Prefix Tuning, which insert learnable vectors into each Transformer layers, has been proposed and proven effective. Recent investigations reveal that prefix tokens carry context-specific information, prompting the hypothesis that enhancing their specialization can improve model performance. To address this, we propose Selective Prefix Tuning (SPT), integrating a selective mechanism inspired by selective self-attention. Additionally, we introduce Selective Loss (SL) to encourage diversity in prefix tokens. Extensive experiments validate the effectiveness of SPT in sentence and token classification tasks. We contribute insight into understanding the role of prefix in model adaptation.

## introduction fine-tuning serves as a pivotal mechanism to adapt large pre-trained models for downstream tasks by adjusting all parameters but it is prohibitively expensive. parameter-efficient learning is an emerging framework that freezes pre-trained models and only tunes a few number of task-specific parameters. an exemplary illustration of this paradigm is prefix tuning and p-tuning-v2 @xcite , wherein a fixed-length of learnable vectors is concatenated in the transformer layer. studies have proven that prefix tuning can achieve comparable performance or even outperforms fine-tuning @xcite . recent investigations into soft tokens have unveiled that they harbor domain-or context-specific words rather than a general instruction for certain tasks @xcite . for instance, when tuning model on boolq @xcite benchmark, where approximately 20% of the questions in the training dataset pertain to the "nature/science" field, the learned prefix exhibits a notable frequency of vectors whose nearest neighbors are words like "science". @xcite assumes that a soft token is a combination of discrete tokens and surprisingly finds that a lot of tokens in the prefix is task-irrelevant, but serves to induce the model for the correct output. although these studies are based on tuning only the prefix tokens in the input layer, it is reasonable to generalize this to methods that insert prefix tokens into multiple transformer layers. it prompts the reasonable assumption that the role of prefix is to furnish the model with specific contextual cues. intuitively, making the prefix tokens more "specialized" can help model focus on the useful prefix cues and ignore unrelated ones. besides, the prefix should be diverse so as to provide the model with a richer bank of context. however, previous studies didn't fully exploit these attributes. so in this work, we posit that enhancing the specializa-tion of prefix tokens and forcing them to capture diversified information can lead to improved model performance. to fulfill the first objective of refining the specificity of the prefix, we guide the model to select a subset of useful prefix tokens. inspired by @xcite , which employs an auxiliary network to mask out less relevant tokens, we introduce a selective mechanism into prefix tuning, presenting our novel method called selective prefix tuning (spt). we strive to avoid adding new parameters, so we simplify the method by leveraging the original attention scores to effectively mask out irrelevant tokens in the prefix. the second objective is dedicated to augmenting the diversity of the prefix through the incorporation of selective loss (sl) as a regularization term. drawing inspiration from @xcite , where explicit terms are introduced to foster diversity among attention heads, our approach calculates the absolute value of average cosine similarities between prefix tokens to formulate the selective loss. through extensive experiments focusing on sentence and token classification tasks under full data settings, we validate our approach. besides, by adjusting newly introduced hyper-parameters, we further study the effect of the selective mask and find the selective loss can help the model learn more features and sometimes even achieve lower training loss. our code is available at https://github.com/potter-zhang/selectiveprefix-tuning . prompt-tuning methods prompt-tuning-related methods have achieved notable success. some researchers even use soft prompts for multi-modal learning @xcite . in order to incorporate soft tokens, various approaches have been explored. for instance, certain methods incorporate trainable tokens directly into the input layer @xcite . others add trainable bias to the hidden states of corresponding prefix tokens @xcite , and some opt to train the entire key-value (kv) cache of the transformer model @xcite . the baseline method employed in this paper is p-tuning-v2 @xcite , which focuses on training the kv cache. selective self-attention network with an overlap of our motivation, ssan @xcite uses a selector module to select a subset of tokens and apply self-attention afterwards, so the model can put more attention weight on the content words. @xcite introduces adaptive threshold into selective self-attention network when solving chinese ner problems. however, our method applies selective mechanism only in learnable prefix and it is used in every layer while ssan only masks out tokens in the first layer and introduces extra parameters for selecting. ## conclusions in this work, we propose a novel way to improve the performance of prefix tuning. without adding extra parameters, significant improvements are observed. experiments on natural language understanding (nlu) and named entity recognition (ner) tasks validate the effectiveness of our approach. in addition, selective prefix tuning provide us with a new perspective of the role and attributes of pseudo prefix. it can be served as a promising method for parameter-efficient learning.