# Neuoplasticity-Based Pruning Method for Deep Convolutional Neural Networks
## 5 Experimental Results
### Vgg16
#### 5.1.2 Flowers  
In this subsection, experimental results for recognition of flowers using the images of the dataset [42], classified by VGG16 sub-models generated using the neuroplasticity-based pruning method, are presented.  
In Figure 13, the sub-models generated from the VGG16 trained on the flowers dataset are displayed.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Re-Connection** & **Params** & **FLOPs (\%)** & **Testing** & **Training** \\
**(A to B)** & **Ratio** & & **Accuracy** & **Accuracy** \\ \hline conv4-1 to conv5-3 & **0.3772** & 21.57 G (70.24\%) & 0.7929 & 0.8113 \\ conv3-2 to conv5-2 & 0.4083 & **16.89 G (55.00\%)** & 0.7973 & 0.7819 \\ conv3-3 to conv5-2 & 0.4484 & 20.59 G (67.05\%) & 0.8073 & 0.8359 \\ conv4-2 to conv5-3 & 0.5374 & 25.27 G (82.29\%) & 0.8062 & 0.8502 \\ conv4-1 to conv5-2 & 0.5374 & 22.49 G (73.23\%) & **0.8128** & **0.8905** \\ \hline \hline \end{tabular}
\end{table}
Table 2: VGG16 Top-5 Road Damage sub-models.  
Figure 12: VGG16 Road Damage sub-models heatmaps.  
Once again, it can be observed that some sub-models achieved higher accuracy than the original network, which is illustrated by the dotted line in Figure 13.  
Table 3 shows the performance and information of the top five generated sub-models, selected according to their accuracy reached and params ratio. Once more, the models are named using the first part of the name as the origin layer, and the second part of the name refers to the destination layer of the pruning.  
From the top-accuracy generated sub-models, it can be observed that connections between the last layers of the network are relevant. In fact, from the heatmaps presented in Figure 14 for this experiment, it is evident that not only the models from the last two columns are those with low params ratio but also with low accuracy.  
In the testing accuracy heatmap, we can observe an accuracy drop from models with connections to the convolutional layer 5-2, which may suggest that up to the convolutional layer 5-1, there are substantial feature maps that were pruned from the network. In other words, relevant transformations to the input volume are lost by pruning specific sub-structures of the network, which may cause an undesired performance. Furthermore, the sub-models with connections to the convolutional layer 5-3 present a similar behavior, then multiple sub-models have an accuracy drop by skipping and pruning some sub-structures of the network.  
In Figures 12 and 14, there is consistent that sub-models with desired performances (i.e., accuracy values close to or above 0.8) and non-significant accuracy loss are those with params ratio close to 0.6, as the top-accuracy sub-models reported in Table 3.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Re-Connection** & **Params** & **FLOPs (\%)** & **Testing** & **Training** \\
**(a to B)** & **Ratio** & & **Accuracy** & **Accuracy** \\ \hline conv2-2 to conv5-1 & **0.5038** & **12.24 G (39.86\%)** & 0.7251 & 0.6863 \\ conv3-1 to conv5-1 & 0.5283 & 14.11 G (45.95\%) & 0.7815 & 0.7502 \\ conv3-3 to conv5-1 & 0.6084 & 21.52 G (70.07\%) & 0.7946 & 0.8120 \\ conv3-1 to conv4-3 & 0.6885 & 17.97 G (58.52\%) & 0.8062 & 0.8590 \\ conv4-3 to conv5-3 & 0.6974 & 28.97 G (94.33\%) & **0.9035** & **0.9428** \\ \hline \hline \end{tabular}
\end{table}
Table 3: VGG16 Top-5 Flowers sub-models.  
Figure 13: VGG16 sub-models trained on the flowers dataset.  
The above may suggest that experimentally and for this particular architecture, the compressed models with params ratio closer to 0.6 may lead to the balance between the accuracy levels and the compression rates. Furthermore, the models with parameters ratios lower than 0.6 are those that create connections between the last layers of the network. Therefore, an experimental criterion to prune networks while using our proposed method is to generate connections between the latter layers/block of the network, to create lightweight and efficient networks.