## 4 Experimental Results  
### Experimental Settings  
**Datasets and Baseline Architectures. ** To demonstrate the effectiveness and performance of our pruning technique we conducted experiments on two commonly used datasets, CIFAR-10 [39] dataset, and ImageNet [3] benchmark dataset. The CIFAR-10 dataset contains 50 k training images and 10k validation images of size \(32\times 32\) categorized into 10 classes. The ImageNet dataset contains 1.28 million training images and 50 k validation images categorized into 1k classes. We mainly focus on pruning the multi-branch residual network structures (ResNet [40]) and evaluated them on both CIFAR-10 and ImageNet datasets. Moreover, we have also pruned a relatively easy single-branch VGGNet [41] and evaluated it on the CIFAR-10 dataset. The VGGNets are based on the most essential features of CNN and contain sequential convolutions. For example, the VGG-16 network used in our experiment consists of 16 sequential convolutional layers and three fully connected layers. On the other hand, ResNet is designed with residual blocks. The concept of shortcut connection in residual blocks is the main strength of this type of network solving the vanishing gradient problem and allowing us to design deeper architectures. **Training Setting. ** For training ResNet and VGGNet on the CIFAR-10 dataset, we use the same training settings as in [42; 43] and [15; 24], respectively. Training ResNet on the ImageNet dataset we used the default parameter setting from [PERSON] et al. [40] and the same data augmentation strategies used in official PyTorch [44] examples. After pruning, while retraining the pruned network, the starting learning rate is set to 10% of the original learning rate, while keeping the other settings the same as in baseline training. Although the same number of training epochs are used in retraining the pruned network, we empirically found that pruned network converges in less number of epochs as compared to baseline training. **Pruning Setting. ** We use the deep learning framework PyTorch [44] along with a structural pruning tool developed by [PERSON] [45] for pruning and fine-tuning the neural network. In every pruning iteration, the pruned filters are permanently removed and the network is rearranged for the next pruning iteration. The maximum pruning rate for each layer is empirically set to \(\alpha_{max}=0.7\), i.e., in each layer we limit the maximum number of pruned filters to 70% of the original number of filters in that layer. This value makes sense because by allowing too many filters to be removed from a single layer it will be difficult to recover the baseline accuracy of the pruned model. Again, reducing the maximum limit to a smaller value will limit the freedom for filter selection from a suitable layer in the network as we will experiment with up to the 54% FLOPs reduction rate. We set \(\alpha_{s}=0.1\), which means, in each pruning iteration we empirically prune 10% of the filters from the selected pruning layer. During pruning, after every \(\beta_{P}=0.3\) (i.e., 3%) network FLOPs reduction rate, the pruned network is fine-tuned for a small number of epochs, which ranges from 1 to 3. This makes sure that network accuracy will not drop abruptly during pruning which also helps the loss-aware filter selection to work effectively. **Evaluation Matrices. ** The primary goal of a structured pruning algorithm is to directly reduce the computational requirement of the model without requiring any specialized hardware. This can be achieved by reducing the required FLOPs of the model. The proposed algorithm directly takes the FLOPs reduction rate as the target pruning rate. Given the target pruning rate, average top-1 accuracy along with standard deviation from three experiments on the CIFAR-10 dataset is used to evaluate the capability of the pruning. As suggested by [PERSON] et al. [46], we used both final accuracy and accuracy drop w.r.t. baseline accuracy for performance evaluation.