# Neuoplasticity-Based Pruning Method for Deep Convolutional Neural Networks
## 5 Experimental Results
### MobileNetV2
#### 5.3.1 Road Damage  
In this subsection, experimental results for recognition of road damages using the images of the dataset [41], classified by MobileNetV2 sub-models generated using the neuroplasticity-based pruning method, are presented.  
For this experiment, it is proposed to use the inverted bottlenecks as the origin and destination sub-structures. Therefore, in Figure 25, a total of 136 scatter points are plotted, where each point on the figure represents a generated sub-model after the one-shot pruning and retraining process.  
In the above figure, the horizontal-axis represents the params ratio, which is calculated as the relation between the number of params from a generated sub-model and the number of params from the base model. On the other hand, the vertical-axis represents the testing accuracy of the model after retraining. It is important to note that a dotted line has been included to show the accuracy reached using the original model of the MobileNetV2 network (labeled as original accuracy). In addition, the size of the scatter points changes according to the parameters ratio (smaller points for smaller parameters ratio, bigger points for higher parameters ratio), and their color is assigned by the testing accuracy reached.  
From the experimental results produced, it can be observed that this time the sub-models did not outperform the accuracy reached by the base model. However, there are different models with parameters reduction of almost 80% and accuracy levels nearby to the original from the base model.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Params** & **FLOPs** & **Testing Acc.** & **Train Acc.** \\ \hline Road Damage & 2.3067 M & 0.6134 G & 0.8361 & 0.8876 \\ Flowers & 2.2965 M & 0.6134 G & 0.8927 & 0.9646 \\ Caltech 101 & 2.3096 M & 0.6134 G & 0.9209 & 0.9974 \\ \hline \hline \end{tabular}
\end{table}
Table 9: MobileNetV2 architectures, computed using the Hyperband algorithm.  
Table 10, shows the performance and information of the top five generated sub-models, selected according to their accuracy reached and params ratio. In the first column of Table 10, the names of the models are shown. The first part of the name indicates the origin layer of the pruning, and the second part of the name stands for the destination layer of the pruning.  
As it can be seen in Table 10, the MobileNetV2 shows excellent performance as a feature extractor such as the MobileNet in the previous experiments. Furthermore, one of the top-accuracy sub-models achieves a remarkable parameters reduction, and their accuracy is barely different from the original. To visualize the network compression, this sub-model is displayed in Figure 26.  
For a better illustration of the experimental results produced, in Figure 27 the heatmaps of the experiment are presented.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Re-Connection** & **Params** & \multirow{2}{*}{**FLOPs (\%)**} & **Testing** & **Training** \\
**(A to B)** & **Ratio** & & **Accuracy** & **Accuracy** \\ \hline block6 to conv2 & **0.2450** & **0.2948 G (48.06\%)** & **0.8250** & 0.8592 \\ block7 to conv2 & 0.2693 & 0.3158 G (51.48\%) & 0.8140 & 0.8568 \\ block8 to conv2 & 0.2935 & 0.3367 G (54.89\%) & 0.8073 & 0.8587 \\ block9 to conv2 & 0.3178 & 0.3577 G (58.31\%) & 0.8184 & **0.8596** \\ block10 to conv2 & 0.3519 & 0.3845 G (62.68\%) & **0.8250** & 0.8559 \\ \hline \end{tabular}
\end{table}
Table 10: MobileNetV2 Top-5 Road Damage sub-models.  
Figure 26: Top sub-model generated from the MobileNetV2 architecture. Block 7 to Block 16 are pruned, thus, Block 6 and Conv 2 (the last layer) are reconnected by the translator block.  
Figure 25: MobileNetV2 sub-models trained on the road damage dataset.