# CitiusSynapse: A Deep Learning Framework for Embedded Systems
## 5 Experiments
### Comparison of Inference Times
Table 7 presents a comparison of the inference time between our framework and AI benchmark. The inference time of the GPU, similar to the ARM Mali-G52 equipped in Odroid-N2+ used in the performance evaluation of this study, is summarized for comparison. In the AI benchmark, AI Benchmark 3.0.0 [76], operated in Android, was used for the performance evaluation. AI Benchmark 3.0.0 provides the benchmark functions to compare CPU and NNAPI-based execution results. NNAPI was used in the AI benchmark to evaluate the performance of mobile devices. The inference time of MobileNet-v2 was 51 ms when the GPU (Mali-G52 MP2), which was most similar to the performance evaluation of this study among various performance evaluations of the AI benchmark, was used. The inference time of MobileNet-v2 was 40 ms when the model and NDRange optimization of our framework were applied. Accordingly, these results verify that our framework is competitive with popular DNN acceleration techniques, such as NNAPI. Table 8 presents a comparison of the inference time between PyTorch and our framework. Our framework performed inference on an Odroid N2+, and PyTorch performed inference on a Jetson Nano. This is because, as mentioned earlier, Jetson Nano does not support OpenCL. The Jetson Nano has 128 CUDA cores and shows a performance of up to 472 GFLOPS (GPU FLoating point Operations Per Second) [65]. In comparison, the Odroid N2+ has six execution engines and shows a performance of approximately 163 GFLOPS [64,77]. In the case of ResNet-18, the difference in inference speed occurs as much as the difference in GFLOPS between Jetson Nano and Odroid N2+. In contrast, in the case of MobileNet-v2, our framework showed a faster inference speed than PyTorch. This is because, when the parameters of the model are small, the efficiency of in-place kernel optimization and asynchronous queue execution is relatively high, as well that as the parallel processing performance of OpenCL is maximized through the on-device optimizer. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{\begin{tabular}{c} **DNN** \\ **FameWork** \\ \end{tabular} } & \multicolumn{8}{c}{\begin{tabular}{c} **Our** \\ **FrameWork** \\ \end{tabular} } & \multicolumn{8}{c}{\begin{tabular}{c} **AI Benchmark** \\ **AI Benchmark 3.0.0 with NNAPI** \\ \end{tabular} } \\ \cline{2-10}  & \begin{tabular}{c} **MediaTek** \\ **Odroid N2+** \\ **(Mali-G52)** \\ **(Mali-G72)** \\ **MP40** \\ \end{tabular} & \begin{tabular}{c} **Exynos 9810** \\ **Odci 907** \\ **(Mali-G72)** \\ **MP18)** \\ \end{tabular} & \begin{tabular}{c} **Eynos 8895** \\ **Octa** \\ **(Mali-G72)** \\ **MP30)** \\ \end{tabular} & \begin{tabular}{c} **MediaTek** \\ **Helio P70** \\ **(Mali-G52)** \\ **MP20)** \\ \end{tabular} & \begin{tabular}{c} **MediaTek** \\ **Helio P65** \\ **(Mali-G72)** \\ **MP3)** \\ \end{tabular} & \begin{tabular}{c} **MediaTek** \\ **Helio P60** \\ **(Mali-G72)** \\ **MP20)** \\ \end{tabular} & \begin{tabular}{c} **MediaTek** \\ **Helio P70** \\ **(Mali-G52)** \\ **MP3)** \\ \end{tabular} & \begin{tabular}{c} **MediaTek** \\ **Helio P65** \\ **(Mali-G72)** \\ **MP3)** \\ \end{tabular} & \
\begin{tabular}{c} **MediaTek** \\ **Helio P60** \\ **(Mali-G72)** \\ **MP3)** \\ \end{tabular} \\ \hline \hline \multirow{2}{*}{
\begin{tabular}{c} Mobile \\ Net v2 \\ \end{tabular} } & \multirow{2}{*}{40\({}^{1}\)} & \multirow{2}{*}{37} & \multirow{2}{*}{72} & \multirow{2}{*}{63} & \multirow{2}{*}{66} & \multirow{2}{*}{51} & \multirow{2}{*}{68} & \multirow{2}{*}{61} & \multirow{2}{*}{77} \\ \cline{1-1} \cline{5-10}  & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of inference times (ms) of our framework and AI Benchmark.