SIBO : A Simple Booster for Parameter-Efficient Fine-Tuning (2024)

Abstract: Fine-tuning all parameters of large language models (LLMs) necessitates substantial computational power and extended time. Latest advancements in parameter-efficient fine-tuning (PEFT) techniques, such as Adapter tuning and LoRA, allow for adjustments to only a minor fraction of the parameters of these LLMs. Concurrently, it has been noted that the issue of over-smoothing diminishes the effectiveness of these Transformer-based LLMs, resulting in suboptimal performances in downstream tasks. In this paper, we present SIBO, which is a SImple BOoster to enhance PEFT, by injecting an initial residual. SIBO is straightforward and readily extensible to a range of state-of-the-art PEFT techniques to alleviate over-smoothing and enhance performance. Extensive experiments on 22 benchmark datasets demonstrate that SIBO significantly enhances the performance of various strong baselines, achieving up to 15.7% and 23.5% improvement over existing PEFT methods on the arithmetic and commonsense reasoning tasks, respectively.

## introduction many transformer-based large language models (llms) exhibit significant depth, e.g., bert-large @xcite has 24 layers, llama-7b @xcite has 32 layers, and llama-65b has 80 layers. yet, this depth presents a challenge @xcite : deep transformers tend to encounter the over-smoothing problem. this issue, as detailed by @xcite , manifests in the deeper layers of transformers, where token representations increasingly converge toward uniformity. the over-smoothness not only impedes the scalability of transformer training, particularly in terms of depth, but also limits the efficacy of scaling up the model size. consequently, expanding the model often results in marginal enhancements or, in some cases, reduced accuracy @xcite . meanwhile, a significant drawback of full-model fine-tuning for llms is that it requires updating all the parameters of the original model. while this constitutes a relatively minor limitation for models like bert-large @xcite or robertalarge @xcite , it escalates into a major obstacle for larger models such as llama @xcite , which contain billions of trainable parameters. many approaches @xcite @xcite have been explored to address this issue by updating only a subset of the parameters or lightweight external modules tailored for new tasks. such strategies require storing and loading a relatively small number of task-specific parameters alongside the pre-trained model for each task. these compelling alternatives to full-model finetuning are called parameter-efficient fine-tuning (peft) @xcite , which significantly enhances the feasibility of deploying llms. although some approaches have been proposed to deal with the over-smoothing problem, such as adding specifically designed regularization to avoid "uniform tokens" @xcite and fusing the representations from all layers @xcite , no peft method has yet been proposed to alleviate the over-smoothing issue. in the era of llms, when internal modifications to models are infeasible for most use cases, addressing the over-smoothing issue through peft techniques becomes critical. challenge and present work. given that existing solutions to over-smoothing involve changes to the model architecture and are hence not parameterefficient, the question arises: how can oversmoothing be effectively addressed for peft techniques? two primary factors may contribute to the over-smoothing problem: 1) redundancy within the model's encoding layers, and 2) a suboptimal training process that hinders the effective optimization of the deeper layers. to address the first issue, a straightforward and logical solution is to reduce the number of layers in the encoder. however, this approach can result in a decline in performance @xcite . to address the second issue, previous approaches @xcite @xcite are not parameter-efficient, thereby limiting their application to llms. to devise a flexible yet simple plug-and-play framework for alleviating over-smoothing with existing peft techniques, our idea boils down to injecting an initial residual into the peft input. this initial residual connection ensures that the final representation of each token preserves at least a minimum portion of the input layer's features, aiming to reduce the uniformity of the final token representations. we name the novel framework sibo, a simple booster to enhance peft techniques designed for llms, most notably adapter @xcite and lora @xcite . empirically, on the arithmetic reasoning task, sibo outperforms adapter and lora by up to 15.7% and 13.6%, respectively. on the commonsense reasoning task, the improvement is up to 7.6% over adapter, and 23.5% over lora. ## preliminaries in the following, we present a summary of two popular lines of peft techniques: adapters and reparametrization-based methods. adapters. adapters fall into two distinct categories: parallel and serial adapters. parallel adapters @xcite integrate additional learnable modules alongside various layers of the core model. in contrast, series adapters @xcite insert these modules sequentially between specific layers, e.g., adding fully connected networks after both the attention and feed forward layers in the transformer model. in this work, we focus on the classical serial adapter, which has the following general formulation: reparameterization-based methods. these methods are designed to modify network weights through a low-rank strategy. this technique effectively reduces the number of tunable parameters without compromising performance. for example, low-rank adaptation (lora) approximates the update ∆w to a pre-trained weight matrix @xmath0: ## methodology in this section, we first analyze the over-smoothing issue in peft techniques, and subsequently present our proposed framework, sibo. ## related work we focus on the discussion of parameter-efficient fine-tuning (peft) of pre-trained language models. there exists three main categories of methods, including prompt-based learning, adapters, and reparametrization methods. however, none of the existing approaches is designed to alleviate the over-smoothing issue. prompt-based learning extends the identification of the ideal discrete (hard) prompt into the optimization of a continuous (soft) prompt instead. @xcite have introduced the idea of prompt tuning, which involves attaching a trainable tensor as a prefix to the input embeddings. similarly, @xcite have developed an independent method known as prefix tuning, which integrates soft prompts into the hidden states across all layers. another technique, intrinsic prompt tun-question matthew gave equal numbers of crackers and cakes to his 4 friends. if he had 32 crackers and 98 cakes initially. how many crackers did each person eat? to calculate the number of crackers each person ate, we need to find the number of crackers that each friend received initially and subtract it from the total number of crackers matthew had. here are the steps: 1. divide the total number of crackers by the number of friends (4) to find the number of crackers each friend received initially: 32 / 4 = 8 crackers per friend. 2. since we want to find the number of crackers each person ate, we need to subtract the initial number of crackers per friend from the total number of crackers: 32 -8 = 24 crackers. ## conclusion we present a novel framework sibo, a simple booster to enhance parameter-efficient fine-tuning (peft) techniques for large pre-trained language models. our core idea revolves around mitigating the over-smoothing issue, which is achieved by injecting an initial residual into various peft modules at specific positions within the pre-trained models. sibo is straightforward and readily extensible to various state-of-the-art peft methods including adapter and lora. extensive experiments on 22 benchmark datasets across three problem areas demonstrate that sibo effectively mitigate over-smoothing and significantly improves the performance of existing peft techniques.