## 4 Results
### Communication Latency  
We conduct experiments to evaluate the interconnection communication latency associated with transferring intermediate parameters. In the experiment, we consider the communication latency \(l_{n}^{com}\) equals \(d_{n}^{p}\) and test its value.  
Initially, we consider the interconnect via an SSH connection. The measured average \(l_{n}^{com}\) is recorded at 0.1039 s. Despite SSH having certain latency, it still has the advantage of facilitating distributed computing, upon network availability.  
To reduce network latency, wireless networking emerges as a viable option for interconnection. In the test, the temperature is 290 K, and the \(B_{n}\) is 160 MHz. We measure an \(S_{n}\) of \(10^{-3.90}\) mW. According to Equation (2), we calculate an \(\omega_{n}\) of \(10^{-9.19}\) mW in wireless networks. According to Equation (2), the \(SNR_{n}\) can be computed as \(10^{5.29}\). After obtaining the \(SNR_{n}\), according to Equation (3), we can calculate \(C_{n}\) as \(2.796\times 10^{9}\) bit/s. Converting \(2.796\times 10^{9}\) bits per second to megabytes per second gives 349.5 MB/s. The \(D_{n}\) is 64 bytes, according to Equation (4), we can calculate the minimum \(d_{n}^{p}\) as \(1.748\times 10^{-7}\) s. Based on the results, we ascertain that the observed value of \(l_{n}^{com}\) is less than 0.001 s. The experiment meets expectations. Consequently, it can be inferred that the influence of using wireless on LLM is insignificant.  
After testing interconnection communication latency, we test the local communication latency on the local device. We record the value of \(s_{n}^{dc}\) as 2.135 GBps and record the value of \(s_{n}^{GS}\) as 12.46 GBps. For the quantized LLM weights in the LLM, each layer is approximately 0.55 GB.  
If the weight only loads from the CPU's RAM, according to Equation (5), the \(l_{n}^{GS}\) for each layer is approximately 0.044 s. However, if the weight loads from the disk, according to Equation (6), the \(l_{n}^{dc}\) for each layer is approximately 0.26 s. According to Equation (7), the \(l_{n}^{GS}\) for each layer is approximately 0.304 s. We assume that in the device, eight layers offload to CPU's RAM, and eight layers offload to disk, according to Equation (8), the \(l_{n}^{loc}\) can be calculated as 2.784 s. It is evident that the disk speed significantly influences the inference speed.  
We find that opting for high-performance disks can significantly reduce latency. After using the high-performance NVMe drive, we boost the \(s_{n}^{dc}\) to 3.166 GBps. According to Equation (6), the \(l_{n}^{dc}\) is approximately calculated as 0.17 s. To calculate performance  
Figure 4: This is the trend in changes in VRAM usage with the number of offloading experts. Our strategy requires less VRAM capacity than the strategy method across all number of offloading experts scenarios.  
improvement, we denote the improvement performance as _Improvement_, while the original latency is denoted as _Original_ and _Improved_ is the improved latency. The formula is  
\[Improvement=\left(\frac{Original-Improved}{Original}\right)\times 100\%. \tag{10}\]  
According to Equation (10), we can calculate that the performance improvement is approximately 32%.  
Similarly, loading the majority of the model weights into the CPU's RAM leads to a substantial reduction in latency. However, in terms of the model's performance, the impact is only on latency and does not affect its inference capabilities.  
Finally, we can calculate the _1 total_. We assume there are 32 IoMT devices similar to our device. Each device has one layer in the model. We use SSH connections and offload the weights to disk. According to Equation (9), we can approximately calculate the _1 total_ as 13.16 s. If we offload the model weights to the CPU's RAM and use SSH connections, we can approximately calculate the _1 total_ as 4.837 s. If we offload the model weights to the CPU's RAM and use the wireless connection, we can approximately calculate the _1 total_ as 1.408 s.