## 6 Key Challenges and Mitigation Strategies
### Resource Constraints  
A primary obstacle when deploying LLMs within IoT ecosystems arises from the stark mismatch between the models' demands and the typically severe resource constraints of edge devices [3]. Edge units often provide limited processing power, small memory capacities (e.g., typically 1-4 GB of RAM), and must operate under strict power budgets (often \(\leq 10\) W) [24]. Yet, even moderately sized models, like a 7-billion-parameter LLM, can require approximately 4 GB of memory just for inference, making deployment challenging [10].  
To bridge this gap and enable on-device LLM adaptation and execution, several mitigation strategies focusing on efficiency can be employed. Model compression techniques, notably quantization (e.g., to 4-bit precision), can significantly slash memory usage by roughly 75% while often preserving a high percentage (e.g., 92-97%) of the original model's accuracy on tasks like text classification [62]. Another approach is split computing, particularly SFL, which partitions the model layers between the device and a more capable edge server. This can cut on-device memory requirements substantially (e.g., by 40-60%), though it introduces trade-offs such as increased round-trip latency (e.g., 150-300 ms) during operations like federated training iterations [18]. Furthermore, PEFT methods have emerged as a highly promising strategy. Techniques like LoRA drastically reduce the number of trainable parameters by updating only a small fraction (e.g., about 1-2%) of the model's weights, achieving massive reductions (up to 98%) in parameters needing training and storage [16]. Impressively, this efficiency often comes with only a modest decrease in performance, retaining substantial percentages (e.g., around 89%) of full fine-tuning performance on standard benchmarks.  
The trade-offs between parameter efficiency and task performance for various PEFT methods, including LoRA, adapter tuning, and prompt tuning compared to full fine-tuning, are clearly visualized in Figure 5. Full fine-tuning involves updating all model parameters, which leads to the highest performance but at a substantial computational and memory cost. In contrast, PEFT methods significantly reduce the number of trainable parameters--LoRA updates approximately 1% of parameters, adapters around 2%, and prompt tuning fewer than 0.1%--while still achieving competitive downstream task performance. As the figure illustrates, these methods strike different balances between efficiency and effectiveness, making them particularly attractive for resource-constrained IoT and Federated Learning settings where full fine-tuning is often impractical. This visual comparison underscores the growing importance of PEFT techniques in scaling LLM applications to diverse, decentralized edge environments.  
Finally, complementing these model-level optimizations, adaptive distribution techniques employing dynamic workload schedulers can monitor real-time device telemetry (available RAM, CPU load, network bandwidth) to adjust model partitioning or batch sizes on the fly, maximizing the utilization of available resources. Together, these diverse approaches--compression, splitting, parameter-efficient adaptation, and dynamic scheduling--make it increasingly practical to deploy and adapt sophisticated LLMs effectively on resource-constrained IoT hardware.