# Exploring Distributed Deep Learning Inference Using Raspberry Pi Spark Cluster
## 5 Discussion
### Performance Gap of Systems  
Focusing on the standalone Pi 4B (Configuration 1), it is unsurprising to see a huge performance gap between the Pi and the desktop (Configuration 4) in all experiments as per Table 10. When InceptionV3 was used for Experiment 1, the desktop was 14.41 times faster on average compared to the standalone Pi. The optimistic target was for the Pi cluster to perform closely to or outperform its VM counterpart (Configuration 3) in all experiments. However, results for cluster systems are not available, as they failed to finish inference run using InceptionV3. By using lightweight neural network models, the performance difference between the desktop and the Pi was reduced to 7.42 times. This may suggest that the Pi could remain competitive with modern hardware when a model with lightweight architecture is used. The VM cluster is operating as a de facto single VM because of the phenomenon explained in Section 5.1. It was only 29% faster than the Pi, which also highlights the fact that the performance difference between the two systems may not be as drastic as initially expected. In Experiment 2, using a lightweight model also reflects the same trend, as the desktop was only 7.96 times faster than the Pi in MTCNN compared to a difference of 12.24 times in YOLOface and 10.54 times in RetinaFace, respectively. Once again, the results for cluster systems are absent due to the issues mentioned in the previous section. For Experiment 3, the standalone Pi outperformed the VM cluster by 11% in MTCNN. The desktop PC still outperformed the Pi by a difference of sixfold. When using both YOLOface and RetinaFace, the average difference between the Pi and desktop PCs widened to close to tenfold, where it was 14.52 times faster in YOLOface and 9.95 times faster in RetinaFace, respectively. The same case is reflected in the VM cluster, where it was 1.76 times faster in YOLOface and 1.67 times faster in RetinaFace, respectively. Overall, the desktop PC still outperforms the Pi multiplicatively because of the hardware difference between the two systems. However, this discovery of the reduction in performance gap when using lightweight models for inference suggests that using lightweight neural network models for Pi is vital to maintaining competitiveness in most applications using DL against modern hardware. This is especially true when the performance difference between the desktop PC and the Pi was reduced when using lightweight DL models as opposed to standard DL models. While the Pi could barely keep up with the VM cluster, the performance difference between the VM cluster and the Pi was not drastic as initially expected. This shows that the Pi can rival the VM, which itself is a simulation of a low-end system created for this performance review. Referring to Table 11, using lightweight models for the Pi massively reduces execution time at the cost of accuracy. Execution time in ImageNetV2 was reduced from 6529.43 s to only 828.33 s. This helped in achieving a reduction in execution time by 7.88 times, while the accuracy was reduced only by 12%. In CIFAR10, the execution time was reduced from \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Experiment Tested**} & \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Mean Speedup against Configuration 1**} \\ \cline{3-5}  & & **Config 1** & **Config 2** & **Config 4** \\ \hline \multirow{2}{*}{Experiment 1} & InceptionV3 & & - & 14.41 \\  & MobileNetV3 & & 1.29 & 7.42 \\ \hline \multirow{3}{*}{Experiment 2} & MTCNN & & 0.19 & 7.96 \\  & YOLOface & & - & 12.24 \\  & RetinaFace & & & 10.54 \\ \hline \multirow{3}{*}{Experiment 3} & MTCNN & & 0.89 & 6.00 \\  & YOLOface & & 1.76 & 14.52 \\ \cline{1-1}  & RetinaFace & & 1.67 & 9.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Mean speedup against Configuration 1 in all experiments. 3810.57 s to just 20.67 s, which is an overwhelming reduction of 184.35 times at the cost of a 25% decrease in accuracy.