# Block-Wisely Supervised Network Pruning with Knowledge Distillation and Markov Chain Monte Carlo
## 4 Experiments
### Results  
#### 4.2.1 Results on CIFAR-10  
VGGNet: We first evaluated the performance of BNP in compressing VGGNet, and the results are depicted in Table 1. Our method prunes 54% of FLOPs with a cost of 0.37% accuracy drop under \(\alpha=0.1\). The high rank of the feature maps (HRank) [15] reaches a similar FLOP reduction rate but sacrifices the accuracy by 0.53%. Generative adversarial learning (GAL) [6] can prune 45.2% of FLOPs with a cost of similar accuracy loss of 0.54%. Sparse structure selection (SSS) [26] is able to reduce 41.6% of FLOPs but results in a much-degraded model performance with 0.94% accuracy loss. Note that our method presents lower rates in pruning parameters than the competitors, which is attributed to the fact that BNP takes FLOP reduction as the objective, as shown in Equation (2). One can expect an elevated pruning rate of parameters by reformulating the efficiency score with the pruning rate of parameters. ResNet-56: We further examined the effectiveness of BNP in compressing the residual network ResNet-56. The performance of our method was compared with that of the state-of-the-art methods, and the results are shown in Table 2. With \(\alpha=0.1\), BNP performs much better than the RL-based method AMC [2] in reducing FLOPs (62.97% vs. 50%), as \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Pars\(\downarrow\)\%** & **FLOPs\(\downarrow\)\%** & **Baseline Acc** & **Acc\(\downarrow\)\%** \\ \hline SSS [26] & 66.70 & 36.30 & 93.96 & 0.33 \\ SSS [26] & 73.80 & 41.60 & 93.96 & 0.94 \\ GAL [6] & 77.60 & 39.60 & 93.96 & 0.19 \\ GAL [6] & 88.20 & 45.20 & 93.96 & 0.54 \\ HRank [15] & 82.90 & 53.50 & 93.96 & 0.53 \\ \hline BNP (\(\alpha=0.1\)) & 64.88 & 54.00 & 93.51 & 0.37 \\ BNP (\(\alpha=0.2\)) & 59.73 & 62.94 & 93.51 & 1.80 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Network pruning results of VGGNet on CIFAR-10. In all tables, Paras\(\downarrow\) denotes parameters reduction, FLOPs\(\downarrow\) represents FLOPs reduction, and Acc\(\downarrow\) means accuracy loss on test dataset. well as preserving model accuracy (0.16% vs. 0.9% accuracy loss). With a similar FLOP pruning rate, our method shows significantly better performance in preserving model capability than the adversarial learning-based method GAL (0.16% vs. 1.68% accuracy loss). In addition, our method also outperforms filter pruning via geometric median (FPGM) [27], learning filter pruning criteria (LFPC) [28], differentiable sparsity allocation (DSA) [29], and dynamic and progressive filter pruning (DPFPS) [30] by yielding more compact substructures and sacrificing less model accuracy. With \(\alpha=0.2\), BNP produces a compact subnet with 66.29% of FLOPs reduced and 0.66% accuracy drop. These results imply our method is effective in learning block-wise knowledge from baseline networks, and thus has advantages in delivering better substructures than the competitors. ResNet-110: We proceed to assess BNP in a much deeper neural network ResNet-110, and the comparison results between different methods are depicted in Table 3. Our method achieves consistently better pruning results than the SOTA methods. For instance, BNP improves model accuracy by 0.16% after removing 59.9% parameters and 61.24% FLOPs, while HRank [15] results in 0.14% accuracy loss under a similar pruning rate of parameters and FLOPs (59.2% and 58.2%). LFPC is also effective in improving model efficiency and capability with 60.3% FLOP removal and 0.11% increased accuracy. By delivering the same model capability, BNP removes more FLOPs than FPGM [27] (61.24% vs. 52.3%). In addition, our method can remove as high as 67.86% of parameters and 61.44% of FLOPs without attenuating model accuracy. Increasing the hyperparameter \(\alpha\) to 0.2 results in a more compact network by removing 69.21% of FLOPs without heavy damage to model accuracy. The enhanced ability of BNP builds on the effective feature representations via block-wise KD.