AP rompt: Attention Prompt Tuning for Efficient Adaptation of Pre-trained Language Models (2023)

Abstract: With the continuous growth of large language models, the process of fine-tuning these models for new tasks has become increasingly parameter-intensive. Prompt tuning, a method that involves tuning a small set of soft prompts, has emerged as an effective and efficient approach for adapting large pre-trained language models. However, most existing prompt tuning approaches only introduce prompts at the input layer, limiting their performance and leaving large rooms for improvement. In this work, we propose a novel Attention Prompt tuning method, namely APrompt, for efficient adaptation of pre-trained language models. We first demonstrate that existing prompt tuning can be considered as a special case of attention prompt tuning. We then formally introduce APrompt, which incorporates query, key, and value prompts into the attention layer to guide the attention computation during fine-tuning. Experimental results on the SuperGLUE benchmark consistently demonstrate that our proposed approach outperforms state-of-the-art baselines and full fine-tuning method with pre-trained models at different scales. In addition, a comprehensive set of ablation studies validate the effectiveness of the prompt design, as well as the efficiency of our approach.

## introduction pre-trained language models (plms) have gained significant popularity in various natural language understanding tasks @xcite @xcite , exhibiting remarkable success under the pretrain-then-finetune paradigm. it has been consistently demonstrated in recent studies @xcite that scaling up the size of these models leads to improved performance. consequently, large language models such as llama 65b @xcite , @xcite , prompt tuning @xcite , and p-tuning v2 @xcite methods. our method consistently improves over prompt tuning methods and also outperforms fine-tuning method across tasks and model scales. and palm 540b @xcite are becoming increasingly prevalent. despite their compelling performance, fine-tuning large-scale plms is highly parameter-inefficient due to storing gradients and updating for all model parameters. this inefficiency also arises from the requirement to store and deploy a complete copy of the fine-tuned model for each individual task, resulting in computational expenses that hinder fast model adaptation. to tackle the challenges associated with full fine-tuning, researchers have proposed parameterefficient tuning approaches @xcite @xcite involving techniques such as partial tuning and extra module. partial tuning methods @xcite focus on fine-tuning only a portion of the backbone, such as the classifier head or the last few layers, while keeping the remaining layers frozen. on the other hand, extra module methods introduce learnable bias terms @xcite or additional adapters @xcite to the network for adaptation. these strategies operate within the pretrain-then-finetune paradigm and effectively reduce the number of learnable parameters. however, in general, these approaches tend to underperform the full fine-tuning models with large performance gaps. recently, prompt tuning approaches @xcite @xcite have been proposed, which utilize a set of learnable soft prompts prepended to the input. these soft prompts consist of continuous embeddings that are updated during the tuning process while keeping the backbone frozen. prompt tuning offers a conceptually simpler and more flexible method compared to other parameter-efficient tuning approaches. it has been demonstrated to perform closer to full model tuning, especially with large-scale plms @xcite . prompt tuning provides a promising parameter-efficient alternative to fine-tuning, as the soft prompts used in this approach are typically orders of magnitude smaller, constituting less than 0.5% of the total model parameters. however, most existing prompt tuning approaches have mainly focused on modifying the input layers and have not thoroughly explored the core architecture of the transformer's self-attention mechanism. therefore, they often underperform to full fine-tuning and leave substantial room for improvement. in this paper, we present a novel attention prompt tuning approach, aprompt, for efficient and effective large language model adaptation. we begin by reexamining the prompt tuning approach and establish, both theoretically and empirically, that its input prompts can be considered as specialized key-value prompts. we then formally introduce our aprompt. unlike previous prompt tuning methods, aprompt incorporates three sets of learnable prompts: query, key, and value prompts. these prompts are prepended to the respective matrices in the self-attention block within the transformer layer. during model tuning, these attention prompts are learned alongside the original input prompts, resulting in more effective guidance of attention computation for new tasks. evaluation on the superglue benchmark showcases the superior performance of aprompt compared to state-ofthe-art methods. the ablation study results provide strong evidence for the effectiveness and efficiency of the proposed attention prompts. we summarize the main contributions as follows: parameter-efficient tuning as the size of plms becomes larger, it is increasingly unaffordable to update and save full model copies for each downstream application. parameter-efficient tuning methods @xcite arise in the era of llm. depending on whether new parameters are introduced, we divide parameter-efficient tuning methods into categories of partial tuning and extra module. partial tuning methods simply update parts of the model such as the bias term @xcite or the last layers @xcite . extra module methods introduce task-specific parameters to various locations of the model including side-tuning @xcite and adaptors @xcite . there has also been effort unifying different parameter-efficient tuning methods for increased robustness and performance @xcite . prompt tuning prompt tuning @xcite inserts learnable parameters to the model as virtual tokens where the insertion can happen at the model input @xcite or each layer (li and liang, 2021). later variants improve prompt tuning methods for nlu @xcite and nlg @xcite tasks respectively through a series of optimization and adaptations. more recent studies add residual connections to improve the performance and stability of prompt tuning @xcite and extend prompt tuning to the continual learning setting @xcite . however, most of these methods only simply add prompts to input layers, which greatly limited their performances. most recently, mixture prompt tuning has been proposed in mixprompt @xcite ) and e 2 vpt @xcite , which combines the input prompts with key-value prompts. these methods can be treated as special cases of our attention prompt tuning approach. ## attention prompt tuning drawing inspiration from the insights presented in section 3, we propose an attention prompt tuning approach by introducing the attention prompts in the transformer layers to facilitate attention computation. the overall model architecture of aprompt is depicted in figure 3 . fundamentally, our model enables the training of only three components while keeping all other parameters frozen. these components are as follows: (1) input prompts, denoted as p i , which are inserted at the beginning of the input sequence for each transformer encoder layer. (2) attention prompts, represented by p q , p k , and p v , are incorporated into the query, key, and value matrices within the self-attention module, respectively. these prompts allow the model to learn new attention patterns from the finetuning data. (3) a task-specific head, which is a lightweight module dedicated to the specific task and can be trained efficiently. ## analysis and discussion results on low-resource scenario we conducted further evaluations to assess the performance of aprompt and other baseline models in a low-resource setting. following @xcite , we randomly selected 32 examples for each task as the new training set, using a fixed random seed. we fine-tuned the prompt model on this limited training set and reported the results on the full dev set using the best checkpoint in ## conclusions this work first connects existing prompt tuning with attention prompt tuning and show that input prompts can be considered as a special case of keyvalue prompts in the attention layer. inspired by the observation, we introduce three sets of new prompts, namely query, key, and value prompts, and incorporate them into the attention layer to guide the attention computation during fine-tuning. experimental results on superglue demonstrate the superior performance of our model over several state-of-the-art baselines.