# Optimized Deep Neural Networks for Real-Time Object Classification on Embedded GPUs +
## 6 Experiments and Results
### Performance Evaluation of Proposed Scheme on Mobile Tablet (For GPUs Having C.C. < 3.5)  
The second platform used for the performance evaluation of proposed scheme is Nvidia Shield K1 tablet. It has Kepler K1 GPU with 192 CUDA cores. Same experiments are performed to evaluate the proposed scheme on this second embedded GPU having compute capability less than 3.5.  
First of all, the GPU-only ConvMM layer is compared with its sequential, heterogeneous and memory-optimized version and results are listed in Table 6[15]. Figure 13 shows that the GPU-only ConvMM layer is 2\(\times\) faster than the heterogeneous version where the transformation step is performed sequentially using the CPU and the multiplication is performed using the GPU. It can also be noted that for smaller workload sizes, Unified memory-based GPU-only ConvMM layer is slower because of its lengthy context initialization time while for the larger workloads, it outperformed the other  
As mentioned previously, the compute capability of Nvidia Shield K1 tablet is 3.2 and cuBLAS-based matrix multiplication routine cannot be used on this platform. Therefore, shared memory-based accelerated matrix multiplication (SAMM) is proposed for the devices having compute capability less than 3.2. This shared memory-based ConvSAMM layer is further optimized using the unified memory scheme. Results are listed in Table 7. Results show that the accelerated ConvSAMM layer is faster than the GPU-only ConvMM layer for each case of workload and it can be further accelerated using unified memory for larger workloads.  
Tables 8 and 9 tabulate the performance of different versions of ConvMM layers over the same classification models. Results show that the proposed Unified ConvSAMM layer boosts the performance of deep classifiers on the mobile platform and outperforms the other versions for the same classification problems.  
By accelerating the matrix multiplication function and using optimized memory flow, proposed Unified ConvSAMM layer is 271\(\times\)-696\(\times\) faster than the sequential version and 4\(\times\)-11\(\times\) faster than the heterogeneous version.  
Furthermore, battery life is a critical performance metric of the mobile devices. Android phones are getting more powerful with time, and with that, energy consumption of them is gradually going up. Unacceptable energy consumption can reduce the life of the battery. For this reason, energy consumption is an important factor to consider when performing the real-time image classification using a mobile phone. The Nvidia Shield K1 tablet has a battery of 5192 mAh and 19.75 Wh. A software  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Size of Input Image** & **GPU-Only ConvMM** & **ConvSAMM** & **Unified ConvSAMM** \\ \hline \multicolumn{4}{c}{Single} \\ \hline \multicolumn{4}{c}{(Milliseconds)} \\ \hline Cifar (32 \(\times\) 32 \(\times\) 3) & 3.95 & **3.20** & 29.15 \\ \hline Imagenet (224 \(\times\) 224 \(\times\) 3) & 28.01 & **19.91** & 34.75 \\ \hline VGA (480 \(\times\) 640 \(\times\) 3) & 154.32 & 127.32 & **62.98** \\ \hline SVGA (600 \(\times\) 800 \(\times\) 3) & 231.95 & 143.45 & **102.69** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Comparison of different versions of accelerated and optimized ConvMM layer, best results are written in bold.
profiler is used to measure the actual energy consumed by the android application through hardware. Results are listed in Table 10 and represent the energy consumption per image frame.  
Results show that the proposed scheme has a significant impact on the total amount of consumed energy and heterogeneous approach is consuming up to 50% more energy than the proposed accelerated flow.