# Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning
## 5 Experimental Results and Analysis
### Efficiency of the LFP Method
This is because their optimizations prioritize latency in workload partitioning without considering the power consumption characteristics of different device types. As shown in Figure 8a,b, the LFP method saves 16% and 20% energy, respectively, on the VGG-16 and VGG-19 models compared to local inference. This is because the LFP method accounts for differences in computational and transmission power between different edge devices. It utilizes the workload partitioning module proposed in Section 3.4 to achieve the optimal partitioning strategy. In contrast, the BODP and FTP methods focus solely on latency without considering energy consumption, resulting in higher energy costs than local inference. As depicted in Figure 8c,d, the LFP method is less effective than local inference for ResNet, as the model's lower computational demands limit the optimization potential of the fused block strategy. Both FTP and BODP exhibit higher energy expenditures than local inference. This is because their optimizations do not account for the power characteristics of different device types, prioritizing latency in workload partitioning instead. Overall, the LFP method achieves the lowest energy expenditure compared to FTP and BODP. This is accomplished by jointly optimizing computation and balancing the computational capabilities and power characteristics of devices. In the experiments, we used 512 \(\times\) 512 resolution images as input data for the DNN model. After completing the inference of the convolutional layers, the feature map shape becomes \(C_{out}\times 16\times 16\). According to the proposed partitioning algorithm with high partitioning, theoretically, up to 16 edge devices can be supported for distributed inference. However, this would result in significant communication overhead between the master device and the worker devices, leading to transmission latency far exceeding computation latency. As the computational capability of the edge devices increases, the computation  
Figure 8: Comparison between the energy consumptions of different methods across four DNN models. latency during distributed inference decreases. While the overall inference latency reduces, it is still partially affected by the transmission latency. Figure 9 presents a comparison between the communication overheads of different algorithms across four edge devices. Both FTP and LFP use fused-block strategies to avoid data transmission between individual network layers, resulting in low communication overheads. However, LFP incurs a slightly higher overhead than FTP owing to the extra synchronization step that reduces redundancy between fused blocks. The BODP method, which involves layer-by-layer partitioning, incurs the highest communication overhead, considerably exceeding those of FTP and LFP. This can be attributed to the fact that the number of channels in the input feature map doubles every few layers during inference and the height and width shrink slowly. Consequently, each synchronization step transmits much larger amounts of data than the original inference task. To evaluate the robustness of the LFP method, we added new devices to the edge device cluster to measure latency and energy consumption. The DNN model selected for this experiment was VGG-19. Devices were added in the following sequence: V1, V1 and V2, V2, V3. Figure 10 presents the measurement results of the LFP method, with the devices added sequentially indicated at the top of the figure. As the number of devices increases, energy consumption initially decreases but then gradually rises because the computational energy consumption of the newly added devices exceeds the benefits gained from reduced latency. Latency gradually decreases as the number of devices grows, but the redundancy between fused blocks also increases. Eventually, latency stabilizes at a lower range and ceases to decrease further. Figure 11 shows the changes in latency and energy consumption of the LFP method over time.