# Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning
## 3 System Model and Problem Description
### Estimation of Inference Latency and Energy Consumption of Edge Devices  
#### 3.2.1 Latency  
When the MD assigns inference tasks to WDs, the latency primarily consists of two components: the computation latency on the WD, and the communication latency between the WD and MD. According to [13], under the given computational hardware and DNN model configuration, the computation time is approximately proportional to the number of floating-point operations (FLOPs) required by the layers of the DNN model. Additionally, as indicated in [25], the FLOPs, \(F_{ij}\), for the convolutional kernel in network layer \(\mathcal{L}_{j}\) on edge device \(U_{i}\) can be calculated by applying Equation (3):  
\[F_{ij}=2 HW(C_{in}K^{2}+1)C_{out} \tag{3}\]  
where \(H\), \(W\), and \(K\) represent the height, width, and kernel size of the input feature map for layer \(\mathcal{L}_{j}\), respectively; and \(C_{in}\) and \(C_{out}\) denote the numbers of input and output channels in the feature map of \(\mathcal{L}_{j}\), respectively. (1) Computation Latency of WDs  
The computation latency of the inference model on a WD is composed of the latency generated by two multi-layer fused networks. The computation time of a single DNN layer is approximated by using a linear regression model for convolutional computation time, \(T_{comp,ij}^{pre}\)[16]. Typically, a higher computational capability of the edge device corresponds to a lower value of \(T_{comp,ij}^{pre}\). Thus, the computation latency for layer \(\mathcal{L}_{j}\) on edge device \(U_{i}\) is estimated as per Equation (4):  
\[T_{comp,ij}^{pre}=\alpha_{i}F_{ij}+\beta_{i} \tag{4}\]  
where \(\alpha_{i}\) and \(\beta_{i}\) are parameters obtained from the linear regression model. The latency generated by the two segments of the fused multi-layer network can be further estimated as follows. As the number of layers in the DNN increases, the inference precision increases; however, an excessively high number of layers leads to increased redundancy in the fused blocks. To address this, a partition point, \(p\), is selected within the network layers to divide the multi-layer network into two segments, and the resultant fused blocks help minimize redundancy. Based on this partitioning strategy, the computation latency on the WD is estimated by using Equation (5):  
\[T_{comp,i}^{pre}=\sum_{l_{1}}T_{comp,jl_{1}}^{pre}+\sum_{l_{2}}T_{comp,jl_{2}}^{ pre} \tag{5}\]  
where \(l_{1}\in[1,p],l_{2}\in[p+1,L]\). \begin{table}
\begin{tabular}{c c c c} \hline
**Variable Symbol** & **Meaning** & **Variable Symbol** & **Meaning** \\ \hline \(\mathcal{L}_{j}\) & The j-th layer of the network in \(\mathbb{G}\) & \(\mathbb{G}\) & DNN model \\ \(H\) & The height of the output feature map & \(P_{comm,i}\) & The computational power of \(U_{i}\) \\  & & of \(\mathcal{L}_{j}\) & \\ \(W\) & The width of the output feature map & \(P_{comp,i}\) & The transmission power of \(U_{i}\) \\  & & of \(\mathcal{L}_{j}\) & \\ \(K\) & The kernel size of the convolutional & \(B\) & The bandwidth between \(U_{i}\) and \(U_{j}\) \\  & & layer \(\mathcal{L}_{j}\) & \\ \(\alpha_{i}\) & The coefficients of the linear & \(\beta_{i}\) & The coefficients of the linear \\  & & regression model of \(U_{i}\) & regression model of \(U_{i}\) \\  & & The partition points of the & & The proportion covered by the \\ \(p\) & & network layer & & workload-partitioning decision of \(U_{i}\) \\ \hline \end{tabular}
\end{table} 
Table 1: _Cont._Equation (5) is used to split the DNN model into two fused blocks with reduced redundancy, accelerating the inference process. The communication latency, \(T^{pre}_{comm,ij}\), depends on the size of the feature maps being transmitted and the bandwidth, \(B\), of the communication link between the MD and WD.