# Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning
## 2 Related Work
### Distributed Inference  
To further reduce the latency of inference tasks on edge devices, certain methods [12; 13; 14; 15; 16; 17; 22; 23; 24] are employed to divide DNN inference tasks into multiple independently executable subtasks. These subtasks can be executed in parallel across multipleedge devices. In [12], a biased one-dimensional partitioning (BODP) method was proposed to divide the input of each network layer into strips and distribute more workload to devices with higher computational capacity. In [13], a greedy two-dimensional partitioning (GTDP) scheme was developed for layer partitioning; however, this approach significantly increased the synchronization overhead between devices with adjacent partitions. The methods in [12,13] executed distributed inference in a layer-by-layer manner, resulting in significant synchronization overhead between layers. Therefore, the methods in [12,13] are not well-suited for complex DNN models. In contrast, our approach avoids the layer-by-layer synchronization overhead, making it capable of handling models with a greater number of layers. In [14], the FTP method was proposed to vertically divide and fuse convolutional layers in a grid-like manner to reduce memory usage and communication overhead; however, this method is only applicable in scenarios with homogeneous devices. Based on [14], a fusion search strategy with dynamic programming that dynamically selects the optimal execution strategy based on the availability of computational resources and network conditions was introduced [17]. Unlike this work, the study in [14] does not consider scenarios involving heterogeneous edge devices. The study in [22] introduced DistErEdge, which leverages deep reinforcement learning to enable adaptability to various scenarios (e.g., different network conditions and diverse device types). DistErEdge identifies optimal split decisions for CNN models and executes inference tasks in parallel across edge devices to achieve accelerated inference. The study in [23] proposes SDPMP, which combines the principles of pipeline parallelism and the partial dependency characteristics within CNN layers to intelligently perform inter-layer and intra-layer partitioning, meeting the low latency requirements for single-task inference. While the studies in [17,22,23] all consider the heterogeneity of edge devices, unlike this work, they do not take energy consumption into account when reducing inference latency. Overall, the methods proposed in [12,13,14,15,16,17,22,23] focus on distributed inference across edge devices to achieve lower inference latency. Inspired by these works, we propose improvements to the FTP algorithm to make it suitable for the practical requirements of edge-computing systems. We explore DNN deployment in edge systems, including relevant models, algorithms, and methods for distributed DNN inference.  
In summary, while prior works have made significant progress in distributed inference for edge systems, they often fail to consider energy consumption as a critical optimization objective, limiting their applicability in resource-constrained environments. Furthermore, most existing approaches lack effective handling of residual blocks in deep neural networks. To address these limitations, our work introduces a novel approach that incorporates energy consumption into the optimization framework while explicitly accounting for the unique challenges posed by residual blocks. Experimental results demonstrate that our method reduces energy consumption to a certain extent while also improving inference latency.