# Inference Acceleration with Adaptive Distributed DNN Partition over Dynamic Video Stream
## 1 Introduction  
Deep neural network (DNN)-based applications are developing rapidly and are widely used in intelligent services for IoT devices (e.g., video streaming auditing [1], wearable devices in medical care [2, 3]). However, DNN inference is computationally intensive, e.g., VGG-16 requires 15.5 G MACs (multiply-add computations) to classify a 224 \(\times\) 224 image [4]. However, due to production cost, mobility, and energy consumption limitations, IoT devices have poor computational power to perform computationally intensive tasks independently. To accomplish intelligent tasks on IoT devices, computing tasks are usually offloaded to devices with powerful computing power. Cloud servers are the main target for offloading computational tasks from IoT devices because of their abundant computational resources. However, with the rapid growth in the number of end devices, there is an explosion in the quantity of data to be processed that accompanies it, with nearly 79.4 eletabytes (ZB) of data to be generated and consumed in 2025. The approach of completely offloading computational tasks to the cloud imposes a strong computational burden on the cloud. Furthermore, for real-time tasks, the long-distance data transfer between end devices and cloud computing centers is vulnerable to network communication conditions, and the QoS is unstable [5, 6]. Moreover, uploading the data completely to the cloud can cause privacy concerns for users and does not have a high quality of user experience (QoE). For the above-mentioned problems of cloud computing, edge computing can effectively alleviate them. The edge server is closer to the device side than the cloud, with more stable data transmission between the end, and the edge also has a large number ofcomputing resources. Therefore, the edge can sink the cloud computing resources and use edge servers to respond to the intelligent application task requests of IoT devices, which can effectively alleviate the QoS-dependent network bandwidth problem of cloud computing and reduce the response latency. Under edge computing, DNN model inference can be performed in three ways: device inference, edge inference, and collaborative inference. With the wide application of IoT, the number of end devices is overgrowing, and the total number of installed IoT devices is expected to reach about 21 billion in 2025. As a common phenomenon, how the edge provides stable services for multiple end devices is an important research content. In the multiended unilateral scenario common in smart cities, many cameras are predeployed in the city to realize intelligent services such as smart transportation, smart security check, and traveler location. The cameras are unevenly distributed in the city and correlate with the density of human traffic. Based on these characteristics, we analyze the best way to provide intelligent services for smart city stations. Intelligent tasks are computationally intensive, and end-to-end reasoning requires a large computational overhead with a high time cost, which is not in line with the design concept of smart cities. In addition, there are many devices in smart cities, and the edge servers are not sufficient to complete real-time requests from each end device, which creates high-latency problems, which is not in line with the design concept of smart cities. As a method that can effectively utilize the computing resources of the devices and expand the total computing capacity while guaranteeing the inference delay, edge-end collaboration is the best technical means to process data in a smart city in real time. Specifically, the quantity of data collected by devices distributed near dense crowds is greater than that collected by devices in other locations, and the computing power of devices is fixed. The load of the balanced devices can effectively accelerate the inference time of model pushing.