ACCEPT : Adaptive Codebook for Composite and Efficient Prompt Tuning (2024)

Abstract: Prompt Tuning has been a popular Parameter-Efficient Fine-Tuning method attributed to its remarkable performance with few updated parameters on various large-scale pretrained Language Models (PLMs). Traditionally, each prompt has been considered indivisible and updated independently, leading the parameters increase proportionally as prompt length grows. To address this issue, we propose Adaptive Codebook for Composite and Efficient Prompt Tuning (ACCEPT). In our method, we refer to the concept of product quantization (PQ), allowing all soft prompts to share a set of learnable codebook vectors in each subspace, with each prompt differentiated by a set of adaptive weights. We achieve the superior performance on 17 diverse natural language tasks including natural language understanding (NLU) and question answering (QA) tasks by tuning only 0.3% of parameters of the PLMs. Our approach also excels in few-shot and large model settings, highlighting its significant potential.

## introduction with the blooming of large language models, parameter efficient fine-tuning becomes an effective solution to leverage the power of pretrained language models (lms). among various approaches, prompt tuning (pt) has been recognized for its simplicity and efficacy by adding tokens in front of the inputs. though prompting pretrained lms with specific or human-designed instructions makes model transferable to downstream tasks, additional effort is needed for elaborating the prompts as the output produced are often sensitive to them. to address this issue, learning the prompts becomes a solution. prompt tuning @xcite , prefix tuning @xcite and p-tuning @xcite replace explicit instructions with continuous prompt embeddings and provide flexibilities for the pretrained models to adapt themselves * equal contribution. with superior performance. following the concept, attempt @xcite , mpt @xcite , dept @xcite , and tpt @xcite demonstrate the capability of learnable pt in both single and multitask training scenarios. however, previous studies often treat the prompts as independent units in learning. though the learned prompts can be further clustered for noise filtering @xcite , the parameters needed for training are not reduced since learning occurs before clustering. in this work, we introduce a method that represents the prompt based on a set of learnable codewords. all prompts share a codebook with n codewords. compared with updating the prompts independently and thus preventing word embeddings from sharing information with each other, our codebooks are sharable across all prompts in a downstream task, making codebooks' parameters size independent of the prompt length. in addition, our approach does not follow the common practice of regarding each prompt as inseparable. when treating a prompt as an indivisible word embedding, we may overlook the possibility that, say, certain words may align with other words in the first half of the embedding and match different words in the second half. to tackle this issue, we adopt the idea of product quantization (pq) @xcite by dividing a prompt's word embedding into several subsections and construct a codebook for each subsection. in the past, pq is effective for approximate nearest-neighbor search @xcite and neural network compression @xcite . however, if we directly apply pq to the learned parameters, their amount will not be lowered for training. hence, we simply follow pq's concept where the codebooks are subsection-specific, and provide a set of learnable codewords for each subsection. this makes prompts share some identical subvectors, which allows part of tokens to have the same characteristics in a more fine-grained dimension. to ease the learning process and make it differentiable, we allow each subvector to be softly combined (via linear coefficients) with the codewords, rather than being assigned by only one of the codewords as in pq. this increases both diversity and flexibility of the representation. in the past, learning-based pt can be typically done by increasing the input length with prepended prompts @xcite @xcite , or further adding the original embedding of words with the same number of additional prompts @xcite . our method, referred to as accept, is generally applicable and works for both. we conduct experiments on 17 natural language tasks and show that our method consistently outperforms previous pt approaches. ## related work parameter-efficient fine-tuning enhances the capabilities of pretrained lms by updating a small set of parameters. the approach varies, such as training extra modules @xcite or modifying specific parts like biases or attention weights @xcite . among these, as prompt tuning (pt) is popular for its simplicity and effectiveness, we focus on pt. ## methodology we first give a preliminary of pt for downstream tasks and pq, and then present our method. ## experiments we present the experimental results and comparisons to other approaches in this section. ## conclusion in this paper, we present accept, a novel prompt tuning method based on product quantization. as compared with other pt methods, the proposed method allows versatile and efficient prompt learning by subdividing prompt embeddings and computing each subprompt with the linear combination of learnable codewords and weights. extensive experiments demonstrate that accept achieves outstanding performance across various nlp tasks. furthermore, we also show the proposed approach is capable of being effectively adapted to billionparameter models and achieves decent results. while we currently use all codewords for linear combination, we aim to explore sparse representations in the future work. besides, we plan to extend our research scope by applying accept to a wider range of tasks with a more diverse set of llms.