# Inference Acceleration with Adaptive Distributed DNN Partition over Dynamic Video Stream
## 5 Performance Evaluation
### Experiment Settings  
We built a combined simulation and hardware testbed to validate the ADDP design and evaluate its performance for video-streaming object detection applications. We used four Cortex-A72-processor-equipped Raspberry Pi 4s with onboard cameras as end devices. We connect the Raspberry Pi to a common switch without limiting the network transmission speed to simulate the communication conditions between the end devices in a real application scenario. The edge server was simulated using a laptop equipped with a 2.6 GHz Intel Core i7 chip.  
In addition, limited by the hardware cost, we simulated the game process between one server and dozens of AI cameras regarding the model-partitioning strategy by numerical simulation. Three state-of-the-art DNN models that are often used as object detection backbone networks, namely Vgg16, ResNet18, and ResNet50 were considered in the experiments. We implemented the ADDP framework based on PyTorch and OpenCV, performed partitioning strategies, and ran deep inference on these models. We evaluated the cutting method by the inference time of feature extraction models commonly used in computer vision applications. The layer completion time was the time from the start of DNN inference to the completion of that layer, and the completion time of the last layer was the total completion time of that model.  
Three model-level partitioning methods were chosen as the baseline of comparison for FLP.  
* **Devices**: all layers of the DNN model were executed on the terminal cluster, i.e., the cut point was set to the maximum value of the corresponding model.
* **Edge**: DNN models of all access devices were executed on the edge server, i.e., the cut point was set to 0.  
* **ANS**[21]: a built-in online learning module was used to search for the best cut point based on a novel contextual slot machine algorithm to generate partitioning decisions dynamically. We selected three model vertical partitioning methods as the baseline for comparison of FSP.
* **MoDNN**[27]: based on the characteristics of convolutional layers with computational consumption and fully connected layers with storage consumption, a differential intralayer partitioning approach was performed for different types of DNN layers, the sliced feature map was assigned to multiple devices for collaborative computation, and finally the output on each computational node was collected by the master node for the next layer of inference; the process introduced a significant communication overhead.
* **DeepThings**[25]: end-to-end slicing of the chain structure of the model, using layer fusion, divided the feature map process into multiple independent parallelizable computational tasks, reducing the quantity of data that needed to be communicated, and avoiding overlapping computation between adjacent task partitions by fusion slicing (FTP) method and task assignment strategy.
* **DeepSlicing**[26]: for the shortage of MoDNN and Deepthings, the single feature map was sliced according to bars to reduce the part of repeated computations, balanced single-layer slicing and multilayer vertical slicing, and the number of compromises was used to reduce the duplicated region of feature map and not to divide the parallel computation tasks too much.