# Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning
## 1 Introduction  
Deep neural networks (DNNs) are vital in computationally intensive and memory-intensive applications, such as computer vision, natural language processing, and speech recognition [1]. However, the computational complexity of DNNs is increasing because of the increasing demand for higher precision in these applications. This increasing computational complexity requires significant computational resources, which inevitably limit the applications of DNNs. This problem is conventionally mitigated by offloading computationally intensive tasks to cloud data centers, with edge devices receiving the processed results. However, this approach introduces network latency and network delays. Owing to the rapid development of artificial intelligence and the Internet of Things, edge computing has emerged as an important strategy for processing large-scale data. In edge computing, edge devices process data in real time at the source, thereby reducing latency and alleviating the burden on cloud data centers. Achieving efficient distributed inference on resource-constrained edge devices has become an important challenge in current research. In recent years, numerous studies [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] have focused on developing various inference methods for edge devices. These methods aim to alleviate the conflict between the resource requirements of DNN model deployment and the limited computational resources of edge devices. These methods can be categorized into three types: methods based on DNN model modification, methods of cloud-edge collaboration, and methods based on distributed inference. Certain studies [2, 3, 4, 5, 6, 7, 8] were primarily focused on modifying theDNN architecture. Model compression algorithms [3; 4; 5; 6] have been employed to eliminate unnecessary parameters for resource-constrained edge devices. Knowledge distillation methods have also been used to appropriately reduce model complexity [7; 8]. However, these methods [3; 4; 5; 6; 7; 8] are only applicable to homogeneous computing systems. When multiple types of edge devices are involved, the models must be modified separately for each type; thus, these approaches are ineffective in heterogeneous edge computing. Studies [9; 10; 11] have investigated cloud-edge collaboration methods to partition DNN models, offloading computationally intensive components to cloud data centers to alleviate the workload on edge devices. However, these approaches are constrained by the network bandwidth, leading to high latency and unreliable performance. Furthermore, they do not account for network fluctuations and, thus, are unsuitable for real-time inference in heterogeneous edge-computing systems. The distributed-inference-based approach is adopted to address these limitations and improve the real-time adaptability of DNN inference on edge devices [12; 13; 14; 15; 16; 17]. Experimental results show that this method leverages the collaboration between edge devices to enhance the overall inference performance. However, in [14; 15; 17], only applications involving homogeneous edge devices were considered; thus, the scalability of edge device inference was limited. In [12; 13; 16], complex DNN models for heterogeneous edge computing were not sufficiently explored; this can lead to high latency and energy consumption on edge devices. By contrast, in this study, the workload-partitioning problem for heterogeneous edge systems is addressed. This is achieved by comprehensively considering the computational capabilities and the processing and transmission powers of these devices. To achieve the optimization objective of reducing inference latency and energy consumption, the workload-partitioning problem is formulated by applying a resource-adaptive workload-partitioning optimization model. The fused tile partitioning (FTP) algorithm [14] exhibits good adaptability when partitioning inference tasks for different DNN models.