# Filter Pruning via Attention Consistency on Feature Maps
## 4 Experiment and Analysis
### Results and Analysis  
#### 4.2.1 Results on CIFAR-10  
To verify the generality of the proposed method, the pruning experiments are presented on three popular network structures including single-branch (VGG-16 [5]), multiple-branch (ResNet-56/110 [40] and DenseNet-40 [41]) and Inception (GoogLeNet [4]) structures. The proposed method is compared with the state-of-the-art on CIFAR-10 dataset. Note that, referring to the original paper, the initial accuracy of the baseline is different from FPAC in several comparison methods. **Single-branch structure:** To verify pruning efficiency on the single-branch network, the pruning experiments on VGG-16 are presented. The experimental results on VGG-16 are shown in Table 1. The results with different methods including weight-based methods and server feature map-based methods are compared. Firstly, the weight-based methods, e.g., L1 and FPGM, perform well with a low pruning ratio, especially the L1 method. However, with the guidance of data prior information, the proposed method achieves higher accuracy with a larger FLOPs and parameters pruning ratio. For example, compared with FPGM, FPAC achieves better accuracy (0.07% increase by FPAC vs. 0.04% decrease by FPGM) with a larger FLOPs reductions ratio (58.1% by FPAC vs. 34.2% by FPGM). Secondly,HRank, as a feature map-based method, makes full use of the distribution information of feature maps and shows good performance with a higher pruning ratio. Because of the introduction of the attention mechanism to the feature maps, the proposed method outperforms in all aspects (accuracy and pruning ratio). For example, FPAC achieves a higher accuracy (93.86% by FPAC vs. 93.43% by FPGM) with the larger FLOPs reductions ratio (66.6% by FPAC vs. 53.5% by HRank). Thirdly, compared with adaptive importance-based methods, e.g., GAL, the proposed method provides significantly better accuracy and a higher reduction in the number of parameters and FLOPs. **Multiple-branch structure:** The pruning efficiency is evaluated on three popular multiple-branch networks including ResNet-56, ResNet-110, and DenseNet-40. For ResNet-56 and ResNet-110, shortcuts are simply implemented using padding rather than 1\(\times\)1 convolution similar to HRank. For DenseNet-40, shortcuts are implemented by channel-wise concatenation. For simplicity, in cases where the feature maps generated by the convolution layer have the same resolution, the compression ratios are set to the same. Because of the above treatment of ResNet and DenseNet, the convolution layer can be compressed such as a single-branch network and keep a balance of information from the shortcut and convolution layer. Considering fairness and comprehensive comparison, the implementation of the proposed method with several pruning ratios is presented. Under similar FLOPs reductions with the existing methods, the proposed method obtains an excellent top-1 accuracy which is even much better than the baseline model at a low pruning ratio. For example, for ResNet-56, the accuracy of FPAC is increased by 0.45% with 47.4% FLOPs reductions, which is higher than other methods. It shows that the proposed method still works on a multiple-branch network. **Inception structure:** Except for single-branch and multiple-branch structures, inception structure is widely used in popular networks such as GoogLeNet. To verify the generality of the proposed method, the proposed method is tested on GoogLeNet. In the implementation, only the 3 \(\times\) 3 convolution layers are pruned and obtain an equal pruning ratio on the convolution layer of one inception module. Due to the complexity of the inception structure, using most of the existing pruning methods reduces the accuracy. The last column of Table 1 presents the performance of several existing methods which have an experiment on GoogLeNet including weight-based methods, e.g., L1, feature map-based methods, e.g., HRank and adaptive importance-based method GAL.