## 3 Method Design
### Instruction Tuning  
LLMs possess a vast number of parameters, and fully fine-tuning an LLM is extremely resource-intensive, leading to significant computational costs that limit feasibility for clients with limited computational capabilities. Furthermore, using traditional FL algorithms (such as FedAvg) to aggregate and distribute LLM parameters increases the communication overhead of the FL system. The standard FL workflow is not applicable in this context, and both factors reduce the efficiency of LLM fine-tuning. Fine-tuning is a transfer learning technique that involves additional training on a pre-trained model to adapt it to a specific task or dataset. In this paper's experiments, the Low-Rank Adaptation (LoRA) fine-tuning method is adopted to optimize the LLama3 large language model. LoRA is one of the most effective parameter-efficient fine-tuning paradigms, offering notable advantages in cross-task generalization and privacy protection. QLoRA [26] quantizes the pre-trained model to 4 bits, significantly reducing memory usage while maintaining performance comparable to that of 16-bit fine-tuning. GLoRA [27] can dynamically adjust low-rank updates, enabling better model control and adaptability. AdaLoRA [28], on the other hand, automatically allocates the budget for fine-tunable parameters based on their importance, enhancing the model's performance across different tasks and datasets. The fundamental principle of LoRA is to freeze the pre-trained model parameters and introduce trainable bypass matrices (low-rank separable matrices) at each layer of the Transformer. These bypass outputs are then added to the initial path outputs before being fed into the network, and only the parameters of these newly added bypass matrices are trained. The low-rank separable matrices consist of two matrices: the first matrix reduces the dimensionality, and the second matrix increases it, with an intermediate dimension of r, simulating the intrinsic rank. These two low-rank matrices significantly reduce the number of parameters [29]. Given a dense neural network layer parameterized by \(W_{0}\in{}^{d\times k}\), to adapt it to a specific task, \(\Delta W\in{}^{d\times k}\) is updated to obtain a new layer parameterized by \(W=W_{0}+\Delta W\). In the context of LoRA fine-tuning, \(r\ll\min(d,k)\), where \(\Delta W=BA\), with \(A,B\in{}^{d\times k}\). During the training process, \(W_{0}\) is fixed and no longer updated via gradient descent; only A and B are trained. For an input \(x\), the forward propagation of the model is updated as follows:  
\[h=W_{0}x+\Delta Wx=W_{0}x+BAx \tag{1}\]  
When adapting LLMs to vertical domains such as healthcare, finance, tourism, and others, the available training data may be privately owned by multiple clients. In such scenarios, to achieve decentralized training data while maintaining data localization and fine-tuning LLMs, federated learning methods are commonly employed. In federated learning, clients typically compute weight updates locally and then share these updates with other clients to globally update the LLM. However, this approach can impose communication and computational costs on clients. Nevertheless, LoRA, with its parameter efficiency and pluggability, can reduce communication costs and lower the demand for computational resources [30]. LoRA enhances the overall efficiency and scalability of federated learning. The process of LoRA fine-tuning based on federated learning is illustrated in Figure 3.  
Step 1: On the server, the pre-trained large language model LLM is initialized as a global model, most of the parameters in the LLM's weight matrix are frozen, and the global model parameters \(w_{glo}\) are issued. Step 2: Each client uses the global LLM as the initial local LLM and introduces LoRA to achieve personalized local knowledge learning. During the fine-tuning process of LoRA, two low-rank matrices, \(A\) and \(B\), are introduced to adjust the weight matrix of the model.