## 6 Evaluation
### Only Fully Connected Layers Pruning  
Figure 3 presents the distribution of the number of parameters for LeNet5 (top-panel), AlexNet (middle-panel), and VGGNet16 (bottom-panel). Between the different convolutional layers, we have Pooling and Batch-Normalization layers that have insignificant weights, as well as the Drop-out layer between the fully connected layers. The convolutional layers have less weight than the fully connected layers. This is mainly due to the weight sharing and local connectivity of the neurons. Unlike AlexNet and LeNet5, VGGNet16 still has a considerable number of weights in the convolutional layers. Although the kernels are very small the convolutional layers have a large number of filters in VGGNet16. Figure 4 shows a comparison of fully connected layers to the rest of the neural network. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**LeNet5** & **Original** & **80\% Pruning** & **85\% Pruning** & **90\% Pruning** & **85\% Pruning** \\ \hline \hline \multicolumn{5}{c}{**Pruning**} \\ \hline Size & 42.54 Mb & 12.02 Mb & 10.01 Mb & 7.91 Mb & 5.65 Mb \\ \hline Accuracy & 83.56 & **84.28** & 80.14 & 83.38 & 83.11 \\ \hline \multicolumn{5}{c}{**Pruning + postquantization**} \\ \hline Size & 9.91 Mb & 3.17 Mb & 2.49 Mb & 1.86 Mb & 1.10 Mb \\ \hline Accuracy & 83.55 & **84.36** & 82.65 & 83.55 & 84.09 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison in terms of size before and after compression on LeNet5 and the plant seedlings dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**AlexNet** & **Original** & **80\% Pruning** & **85\% Pruning** & **90\% Pruning** & **95\% Pruning** \\ \hline \hline \multicolumn{5}{c}{**Pruning**} \\ \hline Size & 131.25 Mb & 37.27 Mb & 30.95 Mb & 24.40 Mb & 17.40 Mb \\ \hline Accuracy & 86.70 & 87.24 & 87.87 & 86.88 & **87.96** \\ \hline \multicolumn{5}{c}{**Pruning + postquantization**} \\ \hline Size & 32.37 Mb & 10.18 Mb & 8.09 Mb & 5.89 Mb & 3.42 Mb \\ \hline Accuracy & 86.97 & 86.70 & **88.31** & 86.61 & 87.78 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison in terms of size before and after compression on AlexNet and the plant seedlings dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Models** & **Accuracy** & **80\% Pruning** & **85\% Pruning** & **90\% Pruning** & **95\% Pruning** \\ \hline \hline \multicolumn{5}{c}{**Pruning**} \\ \hline
**VGG16** & 91.36 & 87.70 & 90.58 & 90.88 & **91.49** \\ \hline
**AlexNet** & 96.07 & 96.86 & **97.9** & **97.9** & 95.81 \\ \hline
**LeNet5** & **95.03** & 94.76 & 92.41 & 94.76 & **95.03** \\ \hline \multicolumn{5}{c}{**Pruning + postquantization**} \\ \hline
**VGG16** & **91.49** & 84.28 & 80.14 & 83.38 & 83.11 \\ \hline
**AlexNet** & 96.59 & 96.59 & **97.9** & **97.9** & 95.54 \\ \hline
**LeNet5** & 95.02 & 94.50 & 92.67 & **95.81** & 95.02 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison in terms accuracy before and after compression on the Flavia database. Figure 3: Weights histogram. Between the different convolutional layers, we have Pooling and Batch-Normalization layers that have insignificant weights, as well as the Drop-out layer between the fully connected layers. The convolutional layers have less weight than the fully connected layers. This is mainly due to the weight sharing and local connectivity of the neurons. Unlike AlexNet and LeNet5, VGGNet16 still has a considerable number of weights in the convolutional layers. Although the kernels are very small the convolutional layers have a large number of filters in VGGNet16. Dense layers have an exceptionally large number of parameters, which greatly influence the size of the model. Here, the convolutional layers are not pruned during training. Tables 6-8 present the results obtained after 99% pruning of the weights of the fully connected (FC) layers of models VGGNet16, AlexNet, LeNet5, respectively. On VVGNet16, by using only 1% of weights in the fully connected layers, the model size is reduced by factor 2.5 with a drop of 7.99 in terms of accuracy.