DL o RA : Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model (2024)

Abstract: To enhance the performance of large language models (LLM) on downstream tasks, one solution is to fine-tune certain LLM parameters and make them better align with the characteristics of the training dataset. This process is commonly known as parameter-efficient fine-tuning (PEFT). Due to the scale of LLM, PEFT operations are usually executed in the public environment (e.g., cloud server). This necessitates sharing sensitive user data across public environments, thereby raising potential privacy concerns. To tackle these challenges, we propose a distributed PEFT framework called DLoRA. DLoRA enables scalable PEFT operations to be performed collaboratively between the cloud and user devices. Coupled with the proposed Kill and Revive algorithm, the evaluation results demonstrate that DLoRA can significantly reduce the computation and communication workload over user devices while achieving superior accuracy and privacy protection.

## introduction large language models (llms) have recently incited substantial public interest. their ability to grasp context and nuance enables them to handle natural language processing (nlp) tasks such as text generation @xcite , translation @xcite and summarization @xcite with remarkable proficiency. because of the extensive number of parameters in llms and the substantial computational workload during their operations, llms are usually implemented on nodes with rich compute resources such as cloud servers @xcite . during operation, users send their data to the cloud server for llm processing, after which the llm results are transmitted back to the user devices. previous studies @xcite ) has shown that llms can extend their learned knowledge to novel tasks not seen during the training phase, a phenomenon commonly referred to as zero-shot capability. however, fine-tuning still remains essential to enhance llm performance on unseen user datasets and tasks. due to its scale, a widely adopted strategy for fine-tuning llms involves adjusting a limited number of llm parameters while keeping the remainder unchanged. this approach, termed parameter-efficient-fine-tuning (peft), adds small modules of parameters to predefined positions of the pre-trained llm and only fine-tunes these modules @xcite @xcite over the downstream tasks to better adapt to the user data. while peft presents an efficient approach for improving llm performance, it also poses significant challenges for system deployment. to deploy peft, one potential solution involves transferring user input to the cloud, and the entire peft process is performed over the cloud servers. this scheme is referred to as cloud-only solution (figure 1 (a) ). however, this approach comes with several drawbacks. on one hand, keeping private user data in a shared cloud environment raises immediate privacy concerns. on the other hand, in order to deliver a personalized llm service, it is necessary to create and fine-tune a separate set of personal llm parameters using the training dataset from each user. this can result in significant scalability challenges as the user group expands in size. by contrast, another option is to offload the llm fine-tuning process completely to the user device, presented as edge-only solution in figure 1 (b), unfortunately, this approach is often impractical due to the limited computational resources available on user devices. to mitigate the aforementioned system problems, in this work we propose a distributed peft solution named dlora (figure 1 (c )) for collaborative peft operations between a cloud server and user device. dlora eliminates the need to deliver private user data for llm fine-tuning in the cloud thereby ensuring the personal llm parameters are stored completely within the user device, thereby minimizing the risk of privacy leakage. additionally, dlora offloads partial computational workload for llm fine-tuning to user devices, effectively mitigating the scalability issues. beyond that, our preliminary studies on conventional peft algorithms suggests that the majority of trainable parameters within llm remain fairly constant throughout the fine-tuning process, with only a small subset of parameters undergoing active changes. this group of changing parameters varies with the training dataset and downstream tasks. motivated by this observation, we introduced a halt and proceed (hp) algorithm for dlora that dynamically identifies and fine-tunes the set of llm parameters most responsive to the training data, resulting in a substantial reduction in computation and communication workloads on the user derives. overall, our contribution can be summarized as follows: in this section, we introduce llm computations in section 2.1. we then describe the computational flow of peft operations in section 2.2. subsequently, we introduce two well-known privacypreserving mechanisms, federated learning 2.3 and on-device machine learning, and compare our methods against them. ## halt and proceed mechanism in this section, we describe halt and proceed (hp) algorithm in detail, which aims to minimize both computational and communication burdens on user devices during peft computations. we will begin by discussing the early-halt mechanism in section 3.2, which is a simple yet effective approach to search for a minimal set of tunable parameters and eliminate redundant fine-tuning operations with negligible impact on accuracy. following that, we will introduce the parameter proceeding mechanism in section 3.3, which selects and reactivates a subset of previously frozen parameters, further enhancing the llm accuracy. ## evaluation in this section, we provide a comprehensive evaluation of the hp algorithm and dlora system. we begin by describing the evaluation setup in section 4.1. next, we assess the accuracy and system performance of the hp algorithm across multiple tasks and llms in section 4.2. subsequently, we perform multiple ablation studies in section 4.3. ## conclusion the paper introduces dlora, a novel distributed solution tailored for efficient peft operations spanning cloud and edge devices. dlora utilizes the halt and proceed algorithm to enhance efficiency during the fine-tuning process, resulting in substantial reductions in computational and communication workloads. experimental findings illustrate that compared to both cloud-only and edge-only solutions, our dlora system notably minimizes computation and communication while upholding accuracy and privacy.