# Inference Acceleration with Adaptive Distributed DNN Partition over Dynamic Video Stream
## 5 Performance Evaluation
### Results
#### 5.2.5. ADDP on Video Streaming  
We tested the execution results of the object detection task under the condition of an unbalanced load of four intelligent cameras. The input video was captured by an onboard camera using OpenCV, and the video frames were captured at 1280 \(\times\) 720 pixels, filtered, and converted to the size required by the DNN model: 224 \(\times\) 224 \(\times\) 3. We set up four Raspberry Pis with different crowd densities for 10 minutes of traffic video and filtered key frames by the interframe difference method to input into the target detection model. Among them, the larger the difference between frames, the more it meant that the frames could contain important objects or events. The video with a higher crowd density contained more key frames and a higher device load. The video key frame sampling rate and FPS are shown in the Table 2, in which all data come from the DNN inference statistics on the same Raspberry Pi. The FPS of the low-load (i.e., low key frame sampling rate) device is up to 48.8, reaching about eight times that of the high-load device.  
Figure 8: Latency of FSP vs. baseline algorithm on device clusters.  
Figure 9: Time difference between layer padding and input paddingAfter applying ADDP, the frame processing rate of each camera was determined and is shown in Figure 10; the FPS is the same for all devices, although the FPS of the low-load devices decreases. This is because the FSP divides the computation required for each frame after inputting it into the DNN model into four parts evenly, according to the device's arithmetic power, so the load on each Raspberry Pi is the same, and it can be seen that the FSP module of ADDP ensures the consistency of inference time across IoT devices. In addition to FPS, the DNN models on different load devices have wildly different response times for key frames. We defined the key frame response time as the window size between acquiring key frames and getting the target frame. When the device is under high load, the pending frames are backed up into the input queue so that the key frame response time increases significantly. When FSP is applied, the response latency of all devices remains consistent and is lower than the result of a single device under minimum load. Each device is evenly responsible for a portion of the key frame area. The key frame response latency is further reduced to about 40% of the original one when FLP is combined with FSP because the model-partitioning module FLP arranges only the first three layers of ResNet18 to be executed on the four Raspberry Pi's only, and the subsequent computations are executed in the simulated edge server. The experimental results show that our proposed solution can achieve consistency in inference time among the four devices with uneven load and reduce the critical frame response latency from 24.89% to 62.41% of the original latency through data division. Overall, ADDP reduces single-frame response latency to 10-25% compared to the pure on-device processing. Although not tested on a larger scale with real devices, the available experiments enable us to conclude that our proposed solution is applicable to dynamically changing video streams.