# Neuoplasticity-Based Pruning Method for Deep Convolutional Neural Networks
## 5 Experimental Results
### MobileNet
#### 5.2.1 Road Damage  
In this subsection, experimental results for recognition of road damages using the images of the dataset [41], classified by MobileNet sub-models generated using the neuroplasticity-based pruning method, are presented. For this experiment, instead of using a layered approach as the criterion to select candidates for pruning, it is proposed to use the depth-wise separable convolution blocks as the origin and destination sub-structures. In Figure 17, a total of 78 scatter points are plotted, where each point on the figure represents a generated sub-model after the one-shot pruning and retraining process. In the above figure, the horizontal-axis represents the params ratio, which is calculated as the relation between the number of params from a generated sub-model and the number of params from the base model. On the other hand, the vertical-axis represents the testing accuracy of the model after retraining. It is important to note that a dotted line has been included to show the accuracy reached using the original model of the MobileNet network (labeled as original accuracy). In addition, the size of the scatter points changes according to the parameters ratio (smaller points for smaller parameters ratio, bigger points for higher parameters ratio), and their color is assigned by the testing accuracy reached. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Params** & **FLOPs** & **Testing Acc.** & **Train Acc.** \\ \hline Road Damage & 3.2679 M & 1.14 G & 0.8328 & 0.8601 \\ Flowers & 3.2597 M & 1.14 G & 0.8973 & 0.9530 \\ Caltech 101 & 3.2707 M & 1.14 G & 0.9266 & 0.9993 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: MobileNet architectures, computed using the Hyperband algorithm. Similar to the VGG16 Road Damage experiment, most sub-models achieved an excellent performance. Therefore, even using different base models, our proposed pruning method shows an efficient way to obtain competitive accuracy levels and compression rates. Table 6 shows the performance and information on the top five generated sub-models, selected according to their accuracy reached and params ratio. In the first column of Table 6, the names of the models are shown. The first part of the name indicates the origin layer of the pruning, and the second part of the name stands for the destination layer of the pruning. As in the previous experiments, the top-accuracy generated sub-models have connections to the last sub-structures of the network. Thus, once again, the hypothesis of pruning networks by connecting the layers/blocks to the final sub-structures is supported. Then, this hypothesis can be used as a heuristic for determining the destination layers of our pruning method. For a better illustration of the experimental results produced by pruning and reconnecting different sub-structures, in Figure 18 two heatmaps are presented. On each heatmap, there is a grid where the performance of the sub-models is mapped over an indicator. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Re-Connection** & **Params** & \multirow{2}{*}{**FLOPs (\%)**} & **Testing** & **Training** \\
**(A to B)** & & & **Accuracy** & **Accuracy** \\ \hline DS-Conv 2 to DS-Conv 13 & **0.3826** & **0.2547 G (22.34\%)** & 0.8007 & 0.8028 \\ DS-Conv 3 to DS-Conv 13 & 0.3883 & 0.3690 G (32.36\%) & 0.8040 & 0.8208 \\ DS-Conv 5 to DS-Conv 13 & 0.4606 & 0.5426 G (47.59\%) & 0.8228 & 0.8786 \\ DS-Conv 6 to DS-Conv 13 & 0.5826 & 0.6208 G (54.45\%) & 0.8405 & **0.9061** \\ DS-Conv 6 to DS-Conv 12 & 0.6660 & 0.7241 G (63.51\%) & **0.8549** & 0.8938 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: MobileNet Top-5 Road Damage sub-models. Figure 17: MobileNet sub-models trained on the road damage dataset.