IAPT : Instance-Aware Prompt Tuning for Large Language Models (2024)

Abstract: Soft prompt tuning is a widely studied parameter-efficient fine-tuning method. However, it has a clear drawback: many soft tokens must be inserted into the input sequences to guarantee downstream performance. As a result, soft prompt tuning is less considered than Low-rank adaptation (LoRA) in the large language modeling (LLM) era. In this work, we propose a novel prompt tuning method, Instruction-Aware Prompt Tuning (IAPT), that requires only four soft tokens. First, we install a parameter-efficient soft prompt generator at each Transformer layer to generate idiosyncratic soft prompts for each input instruction. The generated soft prompts can be seen as a semantic summary of the input instructions and can effectively guide the output generation. Second, the soft prompt generators are modules with a bottleneck architecture consisting of a self-attention pooling operation, two linear projections, and an activation function. Pilot experiments show that prompt generators at different Transformer layers require different activation functions. Thus, we propose to learn the idiosyncratic activation functions for prompt generators automatically with the help of rational functions. We have conducted experiments on various tasks, and the experimental results demonstrate that (a) our IAPT method can outperform the recent baselines with comparable tunable parameters. (b) Our IAPT method is more efficient than LoRA under the single-backbone multi-tenant setting.

## introduction large language models (llms) have been emerging and achieving state-of-the-art (sota) results not only on a variety of natural language processing tasks @xcite but also many challenging evaluation tasks @xcite ) like question answering, reasoning, math, safety, instruction following. despite llms becoming general task solvers, fine-tuning still plays a vital role in efficient llm inference and controlling the style of the llms' generated contents. 1 fine-tuning such large models by full parameters is prohibitive since it requires a large amount of gpu memory and computations. thus, parameter-efficient fine-tuning (peft) @xcite has raised much attention in the research field since in peft, the tunable parameters are often less than 1% of the llms and the computation costs will be significantly decreased. many peft methods have been validated to be effective across various models and tasks, often yielding comparable results with full-parameter fine-tuning @xcite @xcite . among these peft methods, the reparameterization-based method low-rank adaptation (lora) @xcite is considered one of the most efficient and effective methods at present @xcite @xcite . although lora is effective and can bring stable performance with the original setting in @xcite , it still brings inconvenience under the multi-tenant setting @xcite : it has to add lora modules to multiple weights of the transformer layer and introducing significant additional latency in every generation steps under the multi-tenant setting. thus, it is of central importance to develop a novel peft method that introduces minimum latency during generation and still can perform competitively in downstream tasks. in this work, we propose a novel peft method called instruction-aware prompt tuning (iapt). we fine-tune the llms by inserting instruction aware soft prompts to each transformer layer (figure 1 ). to flexibly regulate the attentions of llms, we install a prompt generator before each transformer layer to generate the soft prompts by taking the input instruction's hidden states as input. the prompt generator is a lightweight module containing a down-projection layer, a self-attention based pooling layer, an activation function, and an upprojection layer. to enhance the expressiveness of prompt generators, we propose to automatically learn the idiosyncratic activation functions for different prompt generators with the help of rational functions. we conduct extensive experiments on a wide collection of tasks, including sentiment classification, natural language inference, question answering, constrained sentence generation, math reasoning, sql query generation, and instruction tuning, to demonstrate the effectiveness of our iapt method. notably, our method can consistently outperform strong peft baselines with comparable tunable parameter budgets, especially the recent lora variants @xcite and sota prompt tuning methods @xcite @xcite . we also use experiments and analysis to show that: (a) our method has significantly lower latency under the multi-tenant setting than the lora-based methods with comparable tunable parameters. (b) we proposed adding a self-attention pooling module in the prompt generator, which can help the different transformer layers share the projection layers, thus improving the parameter efficiency. (c) the activation functions are learned during fine-tuning, which improves the downstream performance. our contributions are summarized as follows: ## experiments in this section, we conduct a series of experiments and analysis to evaluate our iapt method. ## conclusion this work presents the instruction aware prompt tuning (iapt) method, an innovative method for the parameter-efficient fine-tuning of large language models. upon the hypothesis that different input instructions require different soft prompts, we propose to generate soft prompts from the input instructions. we propose three recipes for improving our framework's downstream performance: (a) selfattention pooling; (b) learning different activation functions during fine-tuning for different prompt generators of different depth; (c) cross-layer parameter sharing of prompt generators. our method is convenient to implement and off-the-shelf. experiments on various tasks demonstrate that our iapt method outperforms the baseline methods, while being efficient for inference.