# Inference Acceleration with Adaptive Distributed DNN Partition over Dynamic Video Stream
## 4 The Proposed ADDP Framework
### The Horizontal Partition between the Devices Cluster and the Edge
#### 4.2.2 Multidevice Single-Edge Cut-Point Solution  
In a multidevice single-edge smart city scenario, the cut-point configuration chosen by the master node of each terminal cluster affects the computing power allocation of the edge server. In this case, it is not beneficial for the terminal device clusters to monopolize the server resources. Due to the arithmetic capacity of different device clusters, there are differences in the quantity of data collected. Therefore, different device clusters have various pursuits for DNN model cutting points. Game theory is a powerful tool for designing distributed mechanisms that enable end devices to make decisions that satisfy other terminals and servers in the face of the DNN model and achieve the shortest average response time for all tasks. Game theory provides a practical framework to analyze the interactions between multiple clusters of devices that act in their interest and to design incentive-compatible mechanisms for remote execution of parts of the model so that no endpoint has an incentive to deviate unilaterally. Specifically, we modeled the model cut-point solution problem between camera sets and edge servers in a smart city scenario where an AI camera 1-to-N technology was deployed as a multiuser cut-point game. Based on this, a distributed cut-point updating algorithm was proposed to achieve the Nash equilibrium of the game. In Algorithm 1, we set the DNN model on device \(D_{i}\) to execute all remotely, i.e., \(partition(0)=end\). Each node cluster then interacted with the server to obtain the latest resource allocation information and made a cut-point update request, iteratively improving its own partitioning decision, which had impacts on all devices and could reach the optimal global solution in a finite number of times. The \(D_{i}\) cut point \(partition(t)=i\) at moment \(t\), and the three parts of DNN model time overhead returned by the server were \(T_{j}^{i}\), \(T_{e}^{i}\), \(T_{j,e}^{i}\). Based on the server inference time delay \(T_{e}^{i}\), \(partition(t)\) and the regression model, we could estimate the arithmetic power \(F_{e}\) that can be allocated to \(D_{i}\) by the time slot \(t\) server. Based on the computational resources \(F_{i}\) of \(D_{i}\), the communication bandwidth \(B_{i}\), the available server resources \(F_{e}\), and the model parameters, we found the optimal cut point \(partition(t+1)\) that applied to this case. \(D_{i}\) sends an update request to the server. If the server allows it, \(D_{i}\) adjusts the cut point. Otherwise, the cut policy remains unchanged. ```
1:each device cluster \(D_{i}\) chooses the partition decision \(partition(0)=end\)
2:end initialization:
3:for each device cluster and each decision slot \(t\) in parallel do
4: Execute the first \(partition_{i}(t)\) layer of the DNN model
5: Receive the inference time \(T_{e}^{i}\) and data transmission time \(T_{j,e}^{i}\)
6: Estimate the computing power \(F_{e}\) that the edge server can allocate to \(D_{i}\)
7: Compute the optimal updating \(partition_{i}(t+1)=\text{Func}(F_{i},B_{i},F_{e},cfg)\)
8:if\(\Delta_{i}(t,t+1)\
eq 0\)then
9: Send RTU request to the \(s\) for contending for the partition update opportunity
10:ifreceive the permission from \(s\)then
11: Update the configuration of the device \(D_{i}\) in the next time slot \(partition_{i}(t+1)\)
12:else
13: Keep the original partition configuration \(partition_{i}(t+1)=partition_{i}(t)\)
14:endif
15:else
16: Keep the original partition configuration \(partition_{i}(t+1)=partition_{i}(t)\)
17:endif
18:endfor
```  
**Algorithm 1** Distributed partition decision algorithm  
The server \(S_{i}\) executes the computation tasks for all end-device offloads as a task queue and returns the time from the start of the offload until the server gets the result \(\{T_{e}^{i},T_{j,e}^{i}\}\) for the corresponding task.