## 3 LLM-Empowered IoT Architecture for Distributed Systems
### LLM on IoT: Deployment Strategies
#### 3.3.1 Edge Fine-Tuning  
Adapting pre-trained models locally using PEFT is key. To adapt large pre-trained models like LLMs without incurring the high computational and memory costs of full fine-tuning, PEFT methods can be employed. A prominent example is the popular LoRA [16]. Instead of updating the entire pre-trained weight matrix \(\mathbf{W}_{0}\in\mathbb{R}^{d\times k}\), LoRA introduces two smaller, low-rank matrices, \(\mathbf{A}\in\mathbb{R}^{d\times r}\) and \(\mathbf{B}\in\mathbb{R}^{r\times k}\), where the rank \(r\) is typically much smaller than \(d\) or \(k\) (i.e., \(r\ll\min(d,k)\)). The core idea is to represent the weight update \(\Delta\mathbf{W}\) as the product of these low-rank matrices (\(\Delta\mathbf{W}=\mathbf{B}\mathbf{A}\)). During fine-tuning, the original weights \(\mathbf{W}_{0}\) remain frozen, and only the parameters in \(\mathbf{A}\) and \(\mathbf{B}\) are trained. This mechanism is illustrated in Figure 4. The effective weight matrix used in the forward pass is then computed as  
\[\mathbf{W}=\mathbf{W}_{0}+\Delta\mathbf{W}=\mathbf{W}_{0}+\mathbf{B}\mathbf{A}. \tag{3}\]  
This approach drastically reduces the number of trainable parameters from \(d\times k\) for full fine-tuning down to only \(r\times(d+k)\) for LoRA [60]. This significant reduction in parameters, memory usage, and computation makes fine-tuning large models feasible even on resource-constrained edge devices and substantially decreases communication overhead in Federated Learning scenarios where only the small **A** and **B** matrices need to be exchanged [61].  
Recent advancements in LoRA have produced variants that further address challenges pertinent to resource-constrained IoT and FL settings. For instance, DoRA (Weight-Decomposed Low-Rank Adaptation) [63] extends LoRA by decomposing the pre-trained weights into magnitude and direction components for fine-tuning. This allows LoRA updates to refine the model with a different learning mechanism compared to full fine-tuning of magnitudes and directions, reportedly enabling more efficient and effective training by potentially better capturing nuanced parameter adjustments while maintaining or even improving performance over standard LoRA. Such an approach can be particularly valuable in Federated Learning rounds over bandwidth-constrained IoT environments where efficient yet impactful updates are crucial.  
Building upon such decomposed approaches, EDoRA (Efficient DoRA) [64] aims to further optimize training efficiency. The EDoRA methodology is designed to significantly reduce the computational burden and memory footprint during the fine-tuning process, potentially involving techniques like quantization or structured pruning tailored for the decomposed components. For example, EDoRA is reported to achieve substantial reductions in communication overhead compared to standard LoRA while maintaining comparable task performance by employing sparse updating mechanisms or more aggressive compression. These characteristics make variants like EDoRA highly suitable for IoT+FL scenarios where both computational and communication efficiency are critical for practical deployment on diverse and often limited edge devices.  
However, PEFT methods, including LoRA and its variants, have benefits and drawbacks. The choice of parameters, such as the rank \(r\) in LoRA, directly impacts the balance between efficiency and the model's adaptation capacity; a very low rank might limit the model's ability to capture complex task-specific nuances [65]. Furthermore, the generalization capability of PEFT methods, especially when adapting models to tasks significantly different from the pre-training data, compared to full fine-tuning, remains an active area of investigation [66].