## 5 Performance Evaluation
### Comparison of Inference Performance
#### 5.2.1 Comparison in Considering Partitioning Optimization and Deployment Optimization Separately  
To obtain an optimal partition scheme, the experiment adopts the DNN partitioning algorithm proposed in [13; 40] to optimally divide a given DNN into two parts and calculates the optimal deployment by exhaustive searching. The average value, maximum value, minimum value, mode, and standard deviation are achieved by running each algorithm ten times. In Table 2, method-1 and method-2 present partitioning DNN based on [13; 40], respectively.  
In Table 2, the average inferencing time by distributing inferencing according to the proposed algorithm is superior to other methods. In addition, the proposed algorithm results in smaller standard deviation values for partitioning most of these CNNs, which means the proposed algorithm is relatively more stable.  
5.2.2 Comparison in Considering Partitioning Optimization and Deployment Optimization Simultaneously  
As a genetic algorithm is an approximate algorithm, it cannot ensure obtaining the absolute optimal solution. This experiment adopts an exhaustive method to obtain the optimal inferencing delay and energy cost in any given setting as the baseline, and then compares the corresponding results by running basic GA and the improved GA in considering partitioning optimization and deployment optimization simultaneously. The following experimental results are all average values from running each algorithm ten times in every test case.  
Firstly, Figure 4 shows the comparison of inferencing delay in each test case. In this figure, each sub-picture refers to a different CNN where the horizontal axis represents the number of partitions and the vertical axis represents the average inferencing time in milliseconds. It demonstrates that the inferencing delay resulting from the proposed GA is closer to the optimal value than basic GA. In addition, the trends of the proposed GA and the optimal value are also more similar.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**DNN Model** & **Method** & **Avg** & **Max** & **Min** & **Mode** & **SD** \\ \hline \multirow{4}{*}{AlexNet} & Method-1 & 49.73 & 92.88 & 20.97 & 20.97 & 35.23 \\  & Method-2 & 42.54 & 92.88 & 20.97 & 20.97 & 32.95 \\  & Proposed method & **28.90** & 92.88 & 20.97 & 20.97 & **21.44** \\  & Method-1 & 36.13 & 51.81 & 20.45 & 51.81 & 15.68 \\ \multirow{2}{*}{SqueezeNet} & Method-2 & 32.99 & 51.81 & 20.45 & 20.45 & 15.36 \\  & Proposed method & **30.54** & 52.04 & 20.45 & 20.45 & **14.05** \\  & Method-1 & 96.20 & 114.10 & 84.27 & 84.27 & 14.61 \\ \multirow{2}{*}{MobileNet} & Method-2 & 93.22 & 114.10 & 84.27 & 84.27 & 13.67 \\  & Proposed method & **95.86** & 117.41 & 84.27 & 114.10 & **15.11** \\ \multirow{2}{*}{ResNet110} & Method-1 & 232.05 & 259.54 & 204.56 & 259.54 & 27.49 \\  & Method-2 & 226.55 & 259.54 & 204.56 & 204.56 & 26.93 \\ \multirow{2}{*}{ResNet110} & Proposed method & **220.03** & 254.99 & 204.56 & 233.89 & **16.77** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison on inferencing delay (ms).  
A similar conclusion can be achieved by observation of Figure 5, which compares device average energy costs in each test case.  
Figure 4: Average Inferencing Time Comparison.  
Figure 5: Device Average Energy Consumption Comparison.  
As a result, the proposed algorithm can produce better deployments under different scenarios compared to the basic genetic algorithm. The performance of some solutions is even close to the optimal deployment generated by the exhaustive method.