PARA : Parameter-Efficient Fine-tuning with Prompt-Aware Representation Adjustment (2024)

Abstract: In the realm of parameter-efficient fine-tuning (PEFT) methods, while options like LoRA are available, there is a persistent demand in the industry for a PEFT approach that excels in both efficiency and performance within the context of single-backbone multi-tenant applications. This paper introduces a new and straightforward PEFT technique, termed Prompt Aware Representation Adjustment (PARA). The core of our proposal is to integrate a lightweight vector generator within each Transformer layer. This generator produces vectors that are responsive to input prompts, thereby adjusting the hidden representations accordingly. Our extensive experimentation across diverse tasks has yielded promising results. Firstly, the PARA method has been shown to surpass current PEFT benchmarks in terms of performance, despite having a similar number of adjustable parameters. Secondly, it has proven to be more efficient than LoRA in the single-backbone multi-tenant scenario, highlighting its significant potential for industrial adoption.

## introduction in industrial applications, large language models (llms) are frequently utilized in a single-instance, multi-tenant configuration, as highlighted in chen et al.'s 2023 study on punicaml @xcite ). an instance of this is when an llm vendor offers a model as a service (maas), as described by @xcite . in this arrangement, various clients can tailor the llm to their specific needs using their own parameterefficient fine-tuning (peft) modules. a locally installed llm is typically required to manage a variety of tasks for different tenants, each with their own set of peft parameters. however, while techniques like low-rank adaptation (lora) @xcite are adept at fine-tuning llms, they add considerable latency to each generation step because the low-rank components cannot be integrated into the main model structure. on the other hand, (ia) 3 @xcite , which relies solely on dot product operations, is a more efficient peft approach but may lack the necessary expressiveness. consequently, there is a pressing need in the industry for a peft method that strikes a balance between efficiency and effectiveness. in this work, we propose a novel peft method called prompt aware representation adjustment (para) (depicted in figure 1 ). our method finetuned the llms by directly modifying the hidden representations in the model by multiplying them by adjusting vectors and thus regulating the behaviors of llms. unlike the previous literature like @xcite or @xcite , we introduce a novel prompt-aware mechanism to the peft method. the adjusting vectors are not randomly initialized and fixed across different input prompts. instead, we install a vector generator (vg) before each transformer layer, taking the input prompts' hidden states as input and generating the adjusting vectors as outputs. vg is a lightweight bottleneck architecture consisting of a pooling layer, a down-projection layer, an activation function, and an up-projection layer. certainly! here's the revised version of your text with the latex formatting preserved: we perform a wide range of experiments across a diverse set of tasks to establish the efficacy of our para approach. it's important to note that our approach consistently surpasses robust peft benchmarks with similar adjustable parameter limits, particularly the latest lora iterations, (ia) 3 @xcite , and bitfit @xcite . we also demonstrate that our method exhibits substantially reduced latency in a multi-tenant environment compared to lora-based approaches, highlighting its suitability for real-world applications. our contributions can be encapsulated as follows: ## related works parameter-efficient fine-tuning (peft) entails selectively optimizing a subset of parameters within a large pre-trained model while leaving the core model architecture intact for adaptation purposes @xcite ). in contrast, addition-based techniques involve integrating extra neural components or parameters into the existing model framework. notable contributions in this domain include adapter @xcite @xcite , prefix tuning @xcite , prompt tuning @xcite , p-tuning v2 @xcite , (ia) 3 @xcite , and bitfit @xcite . conversely, specification-based methods involve the explicit designation of parameters that are either adjustable or subject to pruning @xcite @xcite . the reparameterization-based strategies have garnered significant interest @xcite . these approaches convert the parameters being optimized into a format that is both low-rank and parameterefficient. such peft methods are underpinned by the insight that the dimensionality intrinsic to finetuning is relatively low @xcite . lora @xcite , for instance, posits that the variation in weights during tuning is characterized by a low intrinsic rank, and thus focuses on optimizing the low-rank factorization of the weight matrix changes. peft techniques have found broad application, particularly with the rise of open-source large-scale language models @xcite and the trend of tailoring these models to specific use cases through instruction tuning @xcite . in this research, we introduce a novel framework known as para, which is designed for the parameter-efficient fine-tuning of large language models (llms). this approach not only enhances efficiency during llm inference but also delivers superior performance across various downstream applications. ## experiments in this section, we conduct experiments to evaluate our para method. ## conclusion this study introduces para, an innovative approach for the parameter-efficient fine-tuning of expansive language models. we integrate a vector generator within each transformer layer to produce adjustment vectors that modulate the functionality of the llm core. the vector generator utilizes the hidden states of the input prompts as inputs and features a lightweight bottleneck design. para offers greater efficiency in inference compared to lora, as it operates harmoniously with the kvcache system. our experiments across a range of tasks show that para surpasses the performance of standard methods while maintaining high inference efficiency. para is advantageous for industrial applications that leverage llms.