# NUDIF: A Non-Uniform Deployment Framework for Distributed Inference in Heterogeneous Edge Clusters
## 4 Experimental Results
### Experimental Results Analysis
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Number of Devices** & **Grouping** & **Sub-Model 0** & **Sub-Model 1** & **Sub-Model 2** & **Sub-Model 3** & \
\begin{tabular}{c} **Group** \\ **Throughput** \\ \end{tabular} &
\begin{tabular}{c} **Total** \\ **Throughput** \\ \end{tabular} \\ \hline
12 & Group 1 & Device 10 & Device 9 & Device 1 & Device 11 & 0.5905 & \\
12 & Group 2 & Device 8 & Device 2 & Device 7 & Device 3 & 0.5529 & 1.6563 \\  & Group 3 & Device 6 & Device 0 & Device 5 & Device 4 & 0.5129 & \\ \hline
13 & Group 1 & Device 11 & Device 9 & Device 10 & Device 12 & 0.8623 & \\
13 & Group 2 & Device 7 & Device 8 & Device 1 & Device 3 & 0.5681 & 1.9597 \\  & Group 3 & Device 6 & Device 2 & Device 5 & Device 0 & 0.5293 & \\ \hline
14 & Group 1 & Device 12 & Device 9 & Device 10 & Device 11 & 0.8623 & \\
14 & Group 2 & Device 13 & Device 6 & Device 1 & Device 3 & 0.5747 & 1.9775 \\  & Group 3 & Device 8 & Device 2 & Device 7 & Device 0 & 0.5405 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Deployment plan and throughput of EdgeShard-multi under N = 12, 13, 14. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Number of Devices** & **Stage** & **Used Devices** & **Stage Throughput** & **Total Throughput** \\ \hline
12 & Stage 0 & 0, 3, 4, 8 & 1.9162 & \\  & Stage 1 & 5, 6, 7 & 1.7295 & \\  & Stage 2 & 9, 10 & 1.7365 & \\  & Stage 3 & 1, 2, 11 & 1.8106 & \\ \hline
13 & Stage 0 & 7, 8, 9 & 2.0256 & \\  & Stage 1 & 2, 11, 12 & 2.0756 & \\  & Stage 2 & 1, 4, 5, 6 & 2.1257 To quantify the throughput improvement of NUDIF, we present its performance gains over EdgeShard-multi across all tested device counts, as illustrated in Figure 3. In all experiments, NUDIF consistently outperforms EdgeShard-multi, with throughput improvements ranging from 0.91% to 44.67%, averaging 9.95% with a standard deviation of 10.11%. Although the magnitude of improvement fluctuates with the number of devices, all experimental results indicate that NUDIF's throughput remains consistently higher than that of EdgeShard-multi. The 95% confidence interval is [5.34%, 14.55%], and the \(p\)-value is 0.00021, indicating that this improvement is statistically significant. NUDIF's higher throughput compared to EdgeShard-multi can be attributed to its superior resource utilization. To better explain the throughput improvement, we further analyzed the average resource utilization of devices in clusters with varying numbers of devices. The resource utilization of a device is defined as the ratio of the memory required by the deployed sub-model to the device's total memory capacity. If no sub-model is deployed, the resource utilization is zero. The average resource utilization of the cluster is given by:  
\[U_{\text{avg}}=\frac{1}{N}\sum_{i=1}^{N}\frac{M_{i}}{C_{i}} \tag{15}\]  
where \(N\) is the total number of devices, \(M_{i}\) is the memory required by the sub-model deployed on device \(i\), and \(C_{i}\) is the total memory capacity of device \(i\). The results depicted in Figure 4 indicate that as the number of devices increases, the average resource utilization of the EdgeShard-single cluster gradually decreases. This is because EdgeShard-single can only form one set of inference instances. As the number of devices increases, the number of idle devices correspondingly increases, leading to a decrease in the average resource utilization of the cluster. For EdgeShard-multi, when the number of devices is 8, 12, 16, 20, and 24, the average resource utilization of the cluster is enhanced. This is due to EdgeShard-multi's ability to utilize all devices in the cluster to form multiple sets of inference instances, thereby making full use of the cluster resources. However, when the number of devices falls within ranges such as 4-7, 8-11, 12-15, 16-19, and 20-23, the increase in the number of devices does not result in the formation of new inference instance sets, leading to an increase in the number of idle devices and consequently causing a decline in the average resource utilization of the cluster.