EXPLAINABLE AI-BASED DYNAMIC FILTER PRUNING OF CONVOLUTIONAL NEURAL NETWORKS
Authors: Muhammad Sabih
Frank Hannig
Jürgen Teich
Venue: ICLR 2022 Submitted (2022)
Keywords: XAI
Pruning
CNN
Explainable-AI

Abstract: Filter pruning is one of the most effective ways to accelerate Convolutional Neural Networks (CNNs). Most of the existing works are focused on the static pruning of CNN filters. In dynamic pruning of CNN filters, existing works are based on
the idea of switching between different branches of a CNN or exiting early based on the difficulty of a sample. These approaches can reduce the average latency of inference, but they cannot reduce the longest-path latency of inference. In contrast, we present a novel approach of dynamic filter pruning that utilizes explainable AI along with early coarse prediction in the intermediate layers of a CNN. This coarse prediction is performed using a simple branch that is trained to perform top-k classification. The branch either predicts the output class with high confidence, in which case, the rest of the computations are left out. Alternatively, the branch predicts the output class to be within a subset of possible output classes. After this coarse prediction, only those filters that are important for this subset of classes are utilized for further computations. The importances of filters for each output class are obtained using explainable AI. Using this architecture of dynamic pruning, we not only reduce the average latency of inference, but we can also reduce the longest-path latency of inference. Our proposed architecture for dynamic pruning can be deployed on different hardware platforms. We evaluate our approach using commonly used image classification models and datasets on CPU and GPU platforms and demonstrate speedup without significant overhead.

Under review as a conference paper at ICLR 2022
EXPLAINABLE AI-BASED DYNAMIC FILTER PRUNING OF CONVOLUTIONAL NEURAL NETWORKS
An Anonymous authors
Paper under double-blind review
ABSTRACT
Filter pruning one of the most effective ways to accelerate Convolutional Neural Networks (CNNs). Most of the existing works are focused on the static pruning of CNN filters. In dynamic pruning of CNN filters, existing works are based the idea of switching between different branches of CNN or exiting early based on the difficulty latency of inference, but they cannot reduce the longest-path latency of inference. In contrast, we present a novel approach of dynamic filter pruning that utilizes explainable AI along with early coarse prediction in the intermediate layers of a CNN. This coarse prediction is performed using a simple branch that is trained to perform top-k classification. The branch either predicts the output class with high confidence, in which case, the rest of the computations are left out. Alternatively, the branch predicts the output class be within a subset of possible output classes. After this coarse prediction. only those filters that are important for this subset of classes are utilized for further computations. The importances of filters for each output class are obtained using explainable AI. Using this architecture of dynamic pruning, we not only reduce the average latency of inference, but we can also reduce the longest-path latency of inference. Our proposed architecture for dynamic pruning can be deployed on different hardware platforms. We evaluate our approach using commonly used image classification models and datasets on CPU and GPU platforms and demonstrate speedup without significant overhead.
sample. These approaches can reduce the averuge
INTRODUCTION
Deep Neural Networks (DNNs) have achieved rapid success many image processing applications, including image classification [1, 2, 3] image segmentation [4, 5], object detection [6], etc. The key ingredients in this success of DNN have been the usage of deeper networks and a large amount of training data. However, as the network gets deeper, the model complexity also increases rapidly, The training of DNNs can be carried out on high-performance clusters with Graphics Processing Unit (GPU) acceleration; however, for implementing these networks hardware, the complexity of the DNNs needs to be reduced. This includes decreasing the memory requirements, energy consumption, latency, or throughput of the implementation of DNNs the hardware. Pruning is one of the main techniques that are utilized for reducing the complexity of DNNs. DNN Pruning refers to removing the undesired parameters of a DNN that have little influence on the output of the neural network [7. 8j. This leads to fewer Moutiply-AAcumulatt Operation (MAC) operations and fewer Number of Parameters (NPs). Pruning can be neuron pruning, filter pruning, weight pruning, and layer pruning. In neuron pruning, individual neurons are removed, i.e. all the incoming and outgoing connections neuron are also removed [9]. filter pruning, CNN filters are removed [10]. In layer pruning, some of the layers can also be pruned [11]. Weight pruning is used synonymously for unstructured pruning, where the redundant weights are set to zero. The two fundamental objectives for pruning the model are: (1) reducing the memory by lowering the NPs and reducing the latency and energy consumption of the computation by reducing MACs. These two objectives are often conflicting in DNN because the MACs concentrated in the lower layers of typical image classification network and the NPs are concentrated in the higher layers of a typical image classification network.
Pruning can be structured unstructured. In structured pruning, the filters and weights eliminated by removing all their input and output connections, and this means that no additional compilation techniques or hardware optimization is required to obtain the gain on hardware in terms of reduction

---

Under review as a conference paper at ICLR 2022
in the size of the model reduction in the inference time of sample. This is because the whole model structured is changed in unstructured pruning.
In unstructured pruning, the unimportant filters or weights are set zero, compiler utilizes these zeros to skip some computations, thereby decreasing the inference time. Unstructured pruning has an additional cost in terms of compilation effort or computational effort in order to exploit the irregllrr sparsity. Typically, unstructured pruning provides a more pruning ratio as compared to structure pruning.
Network Pruning can also be classified in terms of static pruming and dynamic pruming. In static pruning, the parameters of DNN removed permanently, while in dynamic pruning, the parameters of DNN are not removed permanently, instead, they are selectively used for computation based on the input to a DNN. DNN can be pruned using static pruning, thereby reducing MACs and NPs, and then the same DNN can be dynamically pruned as well.
In [12], the authors surveyed dynamic neural networks. They divided dynamic networks into three main categories: (1) instance-wil wiie dynamic models that process each input sample or instance with data-dependent architectures or parameters, (2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data, and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts.
Our work comes under the category of instance-wise dynamic models. Within this category, the two types of dynamic networks that are relevant to our work are (1) dynamic depth models and (2) dynamic width models. In dynamic depth models, the sample could be predicted earlier in a network. Works such [13] utilize this strategy. This strategy is based on the principle that easier sample in dataset can be predicted earlier within a network than a hard sample.
In contrast to dynamic depth models, the dynamic width models change the number of filters channels of a CNN, based on the sample. In [14], the authors utilized reinforcement learning for training an agent that judges the importance of each convolutional kernel and conducts channel-wise pruning conditioned on different samples such that the network is pruned more when the sample is easier. Their work utilizes hardness of sample for dynamic behavior in a DNN, whereas our work utilizes coarse prediction.
Similar to our work is [15]. in which the authors proposed Learning Kernel Activation Module (LKAM), which is able to dynamically activate or deactivate a subset of filter kernels depending upon the input image content during the inference phase. Their method requires bank of 1x1 convolutional kernels followed by average pooling and sigmoid function in order to choose which filter kernels a layer will be activated.
Our work is unique in this regard that in addition to classifying easy samples earlier, we utilize an intermediate branch to perform a coarse prediction. The lower layers of a CNN output simpler features, while the higher layers output more categorical features that correlate with specific class. In higher layers, different filters output features specific to various classes. The concept behind our proposed dynamic pruning is perform coarse prediction. Then based on this coarse prediction, we select the CNN filters only relevant to specific classes. explainable Al allows us to obtain filter importances relative to specific classes. Our approach allows to reduce the average latency as well as the longest-path latency of inference while keeping the overhead low and hardware implementation easy.
1.1 CONTRIBUTIONS
Our contribution can be summarized in the following points:
We propose novel method for dynamic pruning that utilizes early exit along with early coarse prediction and explainable AL.
The early coarse prediction branch is trained using deep top-k loss. If the branch predicts a sample with high confidence, the prediction is made. Otherwise, coarse prediction is obtained, thereby restricting the possible output to be within a subset of classes.
The coarse prediction is used dynamically select CNN filters relevant for those classes.
2

---

Under review as a conference paper at ICLR 2022
The filters relevant for all output classes obtained in prior using explainable AI. In run sun-time, only ranking of filters done.
The dynamically pruned model is trainable and easily deployable on the end hardware.
2
BACKGROUND
In this section, we briefly describe some of the concepts that are utilized in work.
2.1 RANKING CRITERIA FOR PRUNING
One of the most essential tasks in filter pruning is to rank the filters based on their importances. The importance of a filter can be obtained locally within a layer or it can be obtained globally within network. Some ranking criteria for pruning are as follows:
Magnitude-based metrics These methods utilize fo-mrrm and f--norm of the model weights and have been shown to work reasonably well for general cases in works such as [16] and [17].
Loss-preservation based metries These measures determine the effect of removing a set of pa- rameters model loss, for example, first-order Taylor decomposition has been used for this purpose by [18].
XAI-based metries As described in [19], explainable Al (XAI) seeks to explain why a neural network produces the output that it does for the input it gets. Explanations of DNNs can be of different types such as explaining which neurons are most sensitive (saliency methods), which neurons have the most effect output, which input excites which neurons (signal methods), etc. For example, explainable AI has been used to guide the quantization and pruning of DNNs [20].
Saliency methods explain the decision of neural network by assigning values that reflect the importance of input components in their contribution towards the output. These methods can be used to obtain the importance of both the features and the weights. Some of the methods in this category are for example: DeepLift [21]. Conductance [22], IntegratedGrrdients [23], etc. We choose DeepLiFT as our work is not aimed at comparing different explainable-AI methods specifically, rather it utilizes them in a novel pruning strategy. We choose DeepLIFT due to its robustness and less computational requirement. One of the most attractive aspects of explainable AI based algorithms is that they provide scores of different filters respective to each output class, which makes it possible for these methods to be utilized our dynamic pruning architecture.
DeepLIFT [21], the authors propose a method for decomposing the output prediction a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input.
DeepLIFT compares the activation of each neuron to its reference activation and assigns contribution scores according to the difference. The choice of reference activation is important for the algorithm's outcome, and it often requires domain-speciff knowledge. To specify reference activation, must understand the intuition behind the DeepLIFT algorithm. compares the effect the features baseline of what the model would predict when it cannot the features. Therefore, a good reference activation for MNIST [24] is an all-black image.
Mathematically, the DeepLIFT algorithm works as follows: Let represent some target output neuron of interest and let x2,22,....... Xx represent some neurons in some intermediate layer or set of layers that are necessary and sufficient compute Let l represent the reference activation of r. The quantity At is defined as the cetterennccfoom-rfeeeeeee that is A = 1 f.. DeepLIFT assigns contribution scores Cas,Ar to Ax; s.t.:
Σ CANA = Ar
(1)
CAr,Ai can be thought of as the amount of difference-frommreeeeeee frem--rfeerereee that is attributed to the difference- from-reference of x,. At is the DeepI.IFT score, which can be represented as follows:
3

---

Under review as a conference paper at ICLR 2022
DL(1,x)= = E Cax,Ar
(2)
2.2 OBTAINING FILTER IMPORTANCES FROM EXPLAINABLE AI ALGORITTMS
The explainable Al methods take sample of a dataset as input and output sensitivity maps instead of activations/feature maps. These sensitivity maps have the same dimensions as the feature maps. Let Ω be set of all indices of all feature map elements of all layers of neural network, (m, 92) be the importances of all feature map elements from nr" sample from the validation set, then we can define total feature map importance as the average of importances obtained from samples:
M-1
I(0)
= M Σ I(m 0)
(3)
where M is the number of samples used by the XAI method for obtaining the importances. M varies depending upon the dataset, its typical value is 1-2% of validation samples for CIFARIO. These sensitivity maps are converted into the importances of the filter weights using different methods, which are as follows:
-oomr criteria We can take -nomm of the sensitivity map. For example, for d x d convolutional
kernel, for each sample m E M, the explainable Al method outputs a sensitivity map O of height H and
width W, the sensitivity the weights of this convolutional kernel is given by: Σ Σ 10w>l / (H
W- H-1
W..
Max-Min criteria We can take also get importances by subtracting max from min of a importances, this is useful in DeepLIFT which also outputs negative importances to indicate pixels that negate
W W-1#-1
class. This will be given by Σ max (O,,)) min(Ow.h)) / x WW).
A=
2.3 EARLY EXIT OF CNNS
In early exit CNNs, a number of exit blocks are placed with convolutional layers [13, 25]. Each exit block consists confidence branch and a prediction branch. The prediction branch makes the prediction and the confidence branch outputs the confidence score of the prediction. The branches utilize average pooling followed by a linear layer, which keeps the computational overhead low. The prediction branch utilizes the softmax function while the confidence branch utilizes the sigmoid function to generate the output.
For neural network represented by parameters 0, the output vector of the exit branch be written as y Fxx 0), where x is the input image, is the input to a softmax function, and its output can be defined
exp(y))
softmax(y:)
(4)
Σ,exp(y))
where indicates the index of the predicted class and indicates the index of other output classes. The output of the confidence branch can be defined as:
a (y:) = + exp(yx)'
(5)
(yy) is more than a certain threshold, the model exits the early exit branch. The threshold 4, can be easily obtained using the validation dataset.
2.4 DEEP TOP-K K LOSS FUNCTION
In addition to the early exit, we also perform a coarse prediction, and performing this optimally is important for dynamic pruning approach. Formally speaking, we need to train the branch
4

---

Under review as a conference paper at ICLR 2022
using top-k classification. For this purpose, utilize the loss function proposed in [26]. As the authors formulate the problem. the top-k classification can be performed with DNNs trained with the cross-entropy loss. For example, the fate-of-the-an models trained with erosspyntropy loss yield successful results for top-5 error, even though the cross-entropy loss is not tailored for the top-5 enror minimization. However, in case of limited amount of data noisy data, minimizing with cross-entropy loss does not work as well.
Our use-case of performing top-k prediction in the intermediate layers of DNN is similar in the sense that the input to the coarse prediction branch is noisy, based on which the branch has to be trained perform top-k prediction. The authors take inspiration from multi-class Support Vector Machines (SVMs) and create margin between correct top-k prediction and incorrect predictions. Another contribution of the work in [26] to smoothen the loss with a temperature parameter. This allows smooth gradients and better training under the assumption of noisy data. The work can be followed for exact derivation of the deep top-k loss function.
PROPOSED ARCHITECTURE
schematic of our proposed dynamic pruning architecture is shown in Figure 1. We now describe the components of our architecture.
Pruned Conv
Pruned Conv
input image
Conv
Conv
output
EE Confidence
CP Branch
Dynamic Pruner
output
Figure 1: Schematic diagram of dynamic pruning architecture.
3.1 EARLY EXIT (EE) CONFIDENCE
The EE confidence block consists of average pooling layer followed linear layer with sigmoid function as activation. The goal of this branch is to provide confidence measure to the top-1 prediction of the coarse prediction branch. the top-1 prediction is made by the Coarse Prediction (CP) branch with high confidence, the sample exits the model and the prediction is made. As an example, for VGG11 on t the CIFARIO 10 dataset, 30% of the samples could predicted with higher accuracy than the overall accuracy at the 5th layer. So if the overall top-1 classification accuracy of the model is 90.8% and 30% of casier samples could be predicted at the 5th layer with higher top-1 classification accuracy, then the prediction is made.
3.2
CP BRANCH
At CP branch, if the confidence of predicting the sample obtained from EE branch is greater than a threshold, the model is exited, otherwise, coarse prediction performed. In order train this branch, all the model parameters are frozen except the coarse prediction branch. The goal is to obtain maximum top-k accuracy the coarse prediction branch, which can be used the successive layers for selecting filters that are important for the coarse prediction. For example, with VGG11 on CIFARIO dataset, we attach the coarse prediction branch after the 5th layer and measure top-3 and 5 accuracy. In our experiments, training the coarse prediction branch with deep-k loss function after setting & yields the best results. For VGG11 on CIFAR10, we train this branch for 10 epochs with learning rate of 0.01 using Stochastic Gradient Descent (SGD) [27]. As shown in Table 1,
5

---

Under review as a conference paper at ICLR 2022
utilizing deep top-k loss function is significantly better for coarse prediction than the cross-entropy loss.
Table 1: Utilizing deep top-k loss function while setting r = 1 has significantly better performance than 5.
method
top-5 accuracy
top-3 accuracy
deep top-k deep top-k (r 5)
97.50%
93.10%
72.50%
30.24%
cross entropy loss
73.50%
62.70%
Placing the coarse prediction branch Placing of the coarse prediction branch is dependent upon the constraints of accuracy, model, and the dataset. For example, for VGG11 model with CIFARIO dataset, we placed the branch at the 5th layer using a simple grid search. Branch is placed at each layer successively starting from the highest convolutional layer, trained for five epochs using deep top-k loss function and top-3 accuracy is obtained on the validation set. If the top-3 accuracy falls below a specific threshold, the branch is placed layer ahead of it. We show the different top-3 accuracies obtained for different branch placements in Table 2.
Table Top-3 accuracies for branch placement at different layers of VGGI1.
7th layer
6T layer
5th layer
4th layer
95.3%
94.4%
93.1%
85.7%
3.3 XAI DYNAMIC PRUNER
This block does not contain any neurons. It stores the XAI-based importances of pruned layers. The importances are obtained once in the start and during prediction of the sample, the ranking of filters done. Based on the coarse prediction, the XAI dynamic pruner selects the filter kernels pruned convolution layers that need to be used. This block does the dynamic pruning.
Once the coarse prediction branch is trained, the network needs to be trained with dynamic pruning enabled. The model can be trained for arbitrary amount of pruming ratio. For training purposes, the weights corresponding to the pruned filters are zeroes out, which allows the model to be differentiable and the training occur. We name this phase soft dynamic praning the model thinning is not carried out yet. Whereas for hardware deployment, the hard dynamic pruning is carried out, whereby the model is thinned.
Hardware Deployment and post-deployment fine-tuning After the model has been trained using dynamic pruning, proceed convert the model into hardware-deployahe model. For hardware deployment, all that is needed is instead of zeroing the pruned filters, we instantiate convolutional layers with fewer filter kernels and store the un-pruned kernels in buffer. Based on the coarse prediction, the XAI Dynamic Pruner selects a subset of un pruned kernels from the buffer are uses them as weights of the convolutional layers. In terms of overhead, our method does not reduce the memory requirement of the model as compared to the un pruned model. However, it reduces the latency by selectively utilizing the filter kernels. After this phase, the output layers of the model are fine-tuned.
4 EVALUATION
In this section, we evaluate our approach using standard datasets and commonly used deep learning models.
4.1 SOFTWARE AND HARDWARE
For all experiments, we used an NVIDIA Titan RTX GPU with 24 GB of memory and an Intel i7- 9700 processor. The experiments were implemented using Python [28] as programming language in combination with the following software: Py Porch [29], numpy [30], matplotlib [31], scikit-leam [32], Captum [33], PyTorcCVV [34].

---

Under review as a conference paper at ICLR 2022
4.2 METRICS
For evaluation, utilize classification accuracy, average inference time (Imean), trngest-pathl infer- ence time (4ong) and Floating Point Operations (FLOPs) (flops). Imean is calculated by measuring the average inference time over the entire test dataset, which includes samples that exit early. !oong the average inference time of the samples that do not exit early.
Discussion of complexity: We measure the additional complexity for implementing our dynam- ically pruned model in reference to the corresponding statically pruned model. We show that the computational complexity is negligible for most hardware platforms.
4.3 BASELINE AND REFERENCE
As reference, we use the un-pruned model in order to compare the speedup as well as accuracy. For the baseline, we train a statically pruned model with the same number of filters as the dynamically pruned model. The statically pruned model is pruned using one-shot pruning with &n-norm of DeepLIFT as the ranking criteria. The statically pruned model useful in giving us a measure of computational overhead due to the additional branches and processing that is incurred in dynamic pruning process.
4.4 VGGII WITH CIFARIO
Firstly, we evaluate VGG11 with CIFARIO dataset. VGG11 dataset consists of 11 layers, 8 of which are convolutional layers. The number of filters in these layers are 64, 128, 256, 256, 512, 512, 512, and 512, respectively. We obtain a VGG11 model trained CIFAR10 10 dataset. The pre-trained un-pruned reference model has a top-1 accuracy of 90.9% on the test dataset.
Then we proceed with our approach and first attach the early exit and coarse prediction branches and train these branches. In this configuration, the early exit and coarse prediction branch are placed after the 4th layer. The early exit branch and coarse prediction branches are trained initially with 15 epochs with a learning rate of 0.01 and a momentum of 0.9 using SGD. This allows us to obtain a top-5 accuracy of around 97%, while 30% of samples can exit the network with an accuracy of 92.3%. After training, the coarse prediction and early exit branch, the soft dynamic pruning is turned on and the network trained for 30 epochs. Then the network converted into hardware deployable form and hard dynamic pruning is used. After hard dynamic pruning, only the final output layers of the network need to be fine-tuned for 15 epochs. The number of filters in the last four layers are reduced from 512 to 128. One could also employ layer sensitivities and iterative pruning, however our goal is to demonstrate the utility of our approach, so we pick one configuration for that purpose. The results for CPU are shown in Table 3.
Table 3: This table shows the performance of our architecture CPU with VGGII-CIFARIO 10 the model-dataset pair.
method
accuracy
[oon
Imean
flops
Dynamic Pruning
91.30%
2.83ms
2.65ms
161 .M
Static Pruning Bascline
89.40%
2.70ms
2.70ms
159.5M
Reference Model
90.09%
5.60ms
5.60ms
171.9M
Results on CPU As can observe, the accuracy of our dynamically pruned model is even greater than the reference model, this happens when an paraametttee model is pruned. Additionally, the static pruning baseline useful in concluding that the computational overhead for introducing dynamic pruning is negligible. We can also observe that while the reduction in flops is not significant for pruned model, while the reduction in latency almost double. This could be due to the reason that memory transfers take more time and have more influence on latency as compared to flops.
We obtain the results for GPU as well, which are shown in Table 4. The latency on GPU is reduced by factor of 2, observed in the case of CPU well.
Results on GPU
7

---

Under review as a conference paper at ICLR 2022
Table This table shows the performance of our architecture GPU with VGGII-CIFARIO as the model-datasee pair.
method
accuracy
Stong
Imoon
flops
Dynamic Pruning
91.30%
0.227ms
0.21 ms
161.8M
Static Pruning Baseline
89,40%
0.220ms
0.220ms
159.5M
Reference Model
90.09%
0,447ms
0,447ms
171.9M
4.5 RESNET20 WITH CIFARIO 10
We test ResNet20 with CIFARIO dataset. This ResNet architecture consists of three different widths; six 3 x 3 convolutional layers consist of 16 filters, six other 3 x 3 convolutional layers consist of 32 filters and last six 3 3 convolutional layers consist of 64 filters, while two x convolutional layers are identity layers and one layer is fully connected output layer. In this experiment, we prune the last six layers consisting of 64 layers and reduce them to 48, The results for measurements on CPU are shown in Table 5. As
see, the overhead of dynamic pruning is low and the latency is close to the latency obtained for the static pruning baseline, where the accuracy for the dynamically pruned model is significantly higher.
Table 5: This table shows the performance comparison on CPU with ResNet20-CIFAR10 as the model-dataset pair.
method
accuracy
[lon
Imean
flops
Dynamic Pruning
86.5%
2. 2.17ms
2.13ms
34.5M
Static Pruning Baseline
78.5%
2. 2.12ms
2.12ms
34.0M
Reference Model
94.3%
2.45ms
2.45ms
40.5M
4.6 VGG16 WITH CIFARIO0
We test VGG16 with CIFAR100 100 dataset to show the application of our approach on a dataset with large number of classes the CIFARIO0 dataset contains hundred output classes. VGG16 model consists of 13 convolutional layyers, one with 64 filters, three with 128 filters, three with 256 filters and six with 512 filters. The exit branch is placed after the 7th layer and last six convolutional layers are dynamically pruned by reducing the number of filters to 128, the top-5 coarse prediction is performed at the coarse prediction branch. The results are shown in Table 6.
Table 6: This table shows the performance comparison on CPU with VGGI6-CIFAR 100 as the model-dataset pair.
method
accuracy
flony 8. 8.14ms
Imean
flops
Dynamic Pruning
71.2%
7.82ms
440.0M
Reference Model
71.4%
13.20ms
13.20ms
666.3M
4.7 COMPARISON WITH STAT-OF-THE-ARE STATIC PRUNING
Our work is not directly comparable with the state-of-the-ar static pruning approaches, our dynamic pruning method allows for further prunability after the static pruning has been done. In order demonstrate this, we utilize a VGG16 model that is statically pruned on CIFARIO dataset using gate-decorator method proposed in [35]. The statically pruned model contains 17, 46, 57, 71, 73, 60, 32 filters in initial seven convolutional layers respectively and filters each in the last six convolutional layers. We prune the last six layers dynamically and reduce the number of filters from 51 to 40. In terms of inference time, this gives us a speedup 1.33 on CPU.
Table 7: This table shows the performance comparison CPU between statically pruned VGG16 model for CIFARIO dataset and dynamic pruning applied on that statically pruned model.
method
accuracy
Itong
Ineon
flops
Dynamic Pruning
91.2%
2.67ms
2.48ms
199.9M
Reference Model
91.3%
3.33ms
3.33ms
209.75M
8

---

Under review as a conference paper at ICLR 2022
5 CONCLUSION AND FUTURE WORK
In conclusion, we propose a novel dynamic pruning architecture for pruning CNN filters that utilizes explainable AI and can be deployed on hardware with relative ease. provides an impressive gain in terms of inference latency on different types of hardware. To the best of our knowledge, the idea of performing coarse prediction in intermediate layers and utilizing explainable Al select filters important to those classes has not been proposed before. In comparison with existing static pruning approaches, our architecture has the advantage that can be used after static filter pruning to obtain further speedup. In comparison with existing dynamic pruning approaches, our approach has the advantage of being able reduce the longest-path inference time in addition the average inference time.
For future work, some of the things could be improved further, for example, design space exploration for placement of the early exit and coarse prediction branches well as efficient utilizing of multiple branches could be explored. The structure of the branches could also be changed from simpler linear layers to more complicated layers.
REFERENCES
[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton. "ImageNet Classification with Deep Convolu- tional Neural Networks". In: Proceedings the Conference on Advances in Neural Information Processing Systems (NIPS). Ed. by F. Pereira, J. C. Burges, L. Bottou, and K. Q. Weinberger.
Vol. 25. Curran Associates, Inc., 2012, pp. 1106-1114.
[2] K. He, X. Zhang, S. Ren, and J. Sun. "Deep Residual Learning for Image Recognition". In: 7 (Dec. 2015).
[3] K. He, X. ZZhang, S. Ren, and Sun. "Deep Residual Learning for Image Recognition". In:
The Computinn Research Repository (CoRR) (2015). arXiv: 1512.03385 [cs CV].
[4] O. Ronneberger, P. Fischer, and T. Brox. "U-Net: Convolutional Networks for Biomedical Image Segmentation". In: Medical Image Computing and Computer-Assisted Intervention MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceed- ings, Part III. Ed. by N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi. Cham: Springer, 2015, pp. 234-241. DOI: 10.
[5] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. "Deeplan Lab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs". In: The Computing Research Repository (CoRR) (2017), arXiv: 1606,00915 [cs. CV).
[6] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN", In: The Computing Research Repository (CoRR) (2017). arXiv: 1703.06870 [cs. CV].
[7] Y. LeCun, J. Denker, and S. Solla. "Optimal Brain Damage".. In: Proceedings of the Conference on Advances in Neural Information Processing Systems (NTPS). Ed. by D. Touretzky. Morgan- Kaufmann, 1989 pp. 598-605.
[8] S. Han, J. Pool, J. Tran, and W. Dally. "Learning both Weights and Connections for Efficient Neural Network". In: Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS). Ed. by C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett. Curran Associates, Inc., 2015, pp. 1135-1143.
[9] S. Srinivas and R. V. Babu. "Data-free parameter pruning for Deep Neural Networks." " In: The Computing Research Repository (CoRR) (2015). arXiv: 149.00619 [cs. CV].
[10] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. Graf, "Pruning Filters for Efficient ConvNets". In: The Computing Research Repository (CoRR) (2016). arXiv: 1608. 08710 CV).
[11] S. Chen and Q. Zhao. "Shallowing Deep Networks: Layer-Wise Pruning Based on Feature Representations" In: IEEE Transactions on Pattern Analysis and Machine Intelligence 41.12 (2019), pp. 2048-3056 DOI: 10. 109/TPAMI 8. 2874444.
[12] Y. Han, G. Huang, S. Song, L. Yang, H. Wang, and Y. Wang. "Dynamic Neural Networks: A Survey". In: The Computing Research Repository (CoRR) (2021), arXiv: 2102. 04906 [V.

---

Under review as a conference paper at ICLR 2022
[13] S. Teerapittayanon, B. McDanel, and H. T. Kung. "BranchyNet: Fast inference via early exiting from deep neural networks". In: Proceedings of the 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016, pp. 2464-2469, DOI: 10.1 10.1109/1CPR.201 7.7000000
[14] J. Lin, Y. Rao, J. Lu, and J. Zhou. "Runtime Neural Pruning". In: Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS). Ed. by I. Guyon, U. v. Luxburg,
S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett. Curran Associates, Inc.,
2017, pp. 2181-2191.
[15] F. Nikolaos, I. Theodorakopoulos, V. Pothos, and E. Vassalos. "Dynamic Pruning of CNN networks" In: 2019 10th International Conference on Information, Intelligence, Systems and Applications (ISA).. 2019, pp. 1-5. DOE: 10.1 A..//00........900000
[16] H. Li, A. Kadav, I. Durdanovie, H. Samet, and Graf. "Pruning Filters for Efficient ConvNets". In: The Computing Research Repository (CoRR) (2016). arXiv: 1608.087 00.100 10 [vs.
[17] Y. He, G. Kang, X. Dong, Y. Fu, and Y. Yang. "Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks". In: IJCAI'18. Stockholm, Sweden: AAAI Press, 2018,
pp. 2234-2240.. ISBN: 9780999241127.
[18] P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz. "Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning". In: The Computing Research Repository (CoRR) (2016). arXiv: 161 1.06440 [cs.LG].
[19] E. Tjoa and C. Guan. "A Survey Explainable Artificial Intelligence (XAI): Towards Medical XAI". In: The Computing Research Repository (CoRR) (2019). arXiv: 1907.07372 [cs LG], [20] M. Sabih, F. Hannig, and J. Teich. "Utilizing Explainable Al for Quantization and Pruning of Deep Neural Networks". In: The Computing Research Repository (CoRR) (2020). arXiv:
2008. 0008.09072 [cs.CV],
[21] A. Shrikumar, P. Greenside, and A. Kundaje. "Learning Important Features Through Propagat- ing Activation Differences". In: The Computing Research Repository (CoRR) (2017). arXiv: 1704.02685 [cs. CV].
[22] K. Dhamdhere, M. Sundararajan, and Q. Yan. "How Important Is a Neuron?" In: The Comput- ing Research Repository (CoRR) (2018). arXiv: 12233 [cs LG].
[23] M. Sundararajan, A. Taly, and Q. Yan. "Axiomatic Attribution for Deep Networks". In: The Computing Research Repository ( (CoRR) (2017). arviv: 1703.01365 [cs.LG)
[24] Y. LeCun and C. Cortes. MNIST Handwritten Digit Database. 2010. URL: http://yann.lecu. com/exdb/mnist/..
[25] I. Leontiadis, S. Laskaridis, S. I. Venieris, and N. D. Lane. "It's always personal: Using Early Exits for Efficient On- CNN Personalisation" In: The Computing Research Repository (CoRR) (2021). arviv: 2 202.01339
[cs.LG].
[26] L. Berrada, A. Zisserman, and M. P. Kummr. "Smooth Loss Functions for Deep Top-k Clas sification".. In: The Computing Research Repository (CoRR) (2018). arXiv: 1802. 07595
[.G].
[27] S. Ruder. "An overview of gradient descent optimization algorithms". In: The Computing Research Repository (CoRR) (2016). arXiv: 1609.04747 [cs . LG].
[28] G. Van Rossum and F. L. Drake. Python 3 Reference Manual. Scotts Valley, CA, USA:
SpacteSpace, 2009. ISBN: 978- 1441412690.
[29] A. Paszke et al. "PyTorch: An Imperative Style, High-Performanm Deep Learning Library". In:
Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS). 2019, pp. 8024-801 8035.
[30] S. van der Walt, S. C. Colbert, and G. Varoquaux. "The NumPy Array: A Structure for Efficient Numerical Computation". In: Computing in Science and Engineering 13.2 (2011), pp. 22-30.
DOT: 10.1 109/CCSE.201 1.3377
[31] J. D. Hunter. "Matplotlib: A 2D Graphies Environment". In: Computing in Science and Engineering 9.3 (2007), pp. 90-95. DOI: 10. 10.1109/MCE.22007.
[32] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and F. Duchesnay. "Scikit-learn: Machine Learning in Python". In: Journal of Machine Learning Research 12 (2011), pp. 2825-2830.
10

---

Under review as a conference paper at ICLR 2022
[33] N. Kokhlikyan, V. Miglani, M. Martin, E. Wang, J. Reynolds, A. Melnikov, N. Lunova, and
O. Reblitz-Richardsonn PyTorch Captum. 2019. URL: http:///gghuub.oo///yoooo//p//pp
[34] chenyaofo. Pretrained Models on CIFAR10/100 in PyTorch. 2019. URL: http:://gihub..cm/ chenyaofo/CIIARR-pceeee See----eerainddmmodeeee
[35] Z. You, K. Yan, J. Ye, M. Ma, and Wang. "Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks". In: Proceedings of the Conference Advances in Neural Information Processing Systems (NeurIPS). 2019, pp. 2130-2141.


Reviews:
[{"review_id": "sF3SKodhze", "reviewer": "Reviewer_PWoD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Incomplete experimental results\n1-1) Experimental results on more and more various datasets (e.g., ImageNet), networks (e.g., MobileNet, EfficientNet, ResNet50), and configurations (e.g., various FLOPs reduction ratio) should be included.\n1-2) It is necessary to show whether the proposed method is effective not only for classification but also for more practical detection and segmentation networks.\n1-3) As with other pruning-related SOTA papers, in order to prove the superiority of the proposed method, all experiments should be compared with various SOTA studies. Section 4.7 contains almost nothing.\n1-4) Recently, activation functions with negative values such as Swish and Mish are widely used. In general, pruning is vulnerable to activation functions including negative values due to the influence of shift parameters, so please show whether the proposed method has robust properties to these activation functions.\n1-5) As you know, fine-tuning requires a significant training cost, so many filter pruning techniques that achieve excellent performance without fine-tuning have been proposed. It is necessary to show whether good performance can be maintained even if fine-tuning is removed from the proposed method.\n\n2. Novelty of the proposed method: It is difficult to find a clear novelty in the proposed method because it is mostly based on existing methods and only a few changes have been made.\n\n3. It is difficult to agree with the argument of this paper because the motivation of this paper is not sufficiently presented. To compensate for the fact that the experimental results of this paper are not sufficient to prove the superiority of the proposed method, the motivation of this study needs to be clearly presented at the beginning for enhancing the novelty of this paper.\n\n4. To support the sentence, “Our proposed architecture for dynamic pruning can be deployed on different hardware platforms.”, analysis and experimental results for more diverse hardware (e.g., embedded GPU board, FPGA, ASIC) should be accompanied.\n\n5. Recently, many kernel pruning studies have been attempted, and therefore it would be good to explain the advantages of the proposed method compared to kernel pruning targeting the kernel in the filter.\n\n6. The relationship and role of explainable AI in the proposed method are not well explained, and it is not clear why the authors performed studies belonging to instance-wise dynamic models among the three categories.", "summary_of_the_paper": "This paper proposed a new dynamic filter pruning method that utilizes explainable AI along with early coarse prediction in the intermediate layers of a CNN. The early coarse prediction branch is trained using deep top-k loss, and the coarse prediction is used to dynamically select CNN filters relevant for those classes. The filters relevant for all output classes are obtained in prior using explainable AI. The dynamically pruned model by the proposed method is trainable and easily deployable on the various end hardware.", "summary_of_the_review": "The contribution needs to be made clearer. In addition, the experiment part that supports the contribution needs to be thoroughly supplemented. Please address my concerns in \"Main Review\" through the rebuttal process.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cDw08tbMpoD", "reviewer": "Reviewer_vKUc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Weakness:\n\n1. This work was evaluated on CIFAR 10 and 100. Without evaluation on a larger dataset (like ImageNet), the usefulness of this work is harder to evaluate?\n2. This work also does not compare against strong alternatives like prior work in dynamic neural network pruning (eg- See [A] and [B] for relevant related work) or compare against state of the art static pruning techniques (See [C]) or more recent efficient architectures like MobileNet or EfficientNet.  The one static pruning technique compared against is L1 pruning using a one-shot framework. This is a really weak benchmark to compare against. Without comparison against these alternatives, its really hard to evaluate the effectiveness of this technique.\n3. The motivations of using XAI for pruning criterion is also unclear, given that the added understanding of importance was not used anywhere during the work\n4. The authors also dont motivate the reason for importance of reduction of longest-path latency of inference via discussing and showing this element to be a limitation of various prior work. Further, not all dynamic inference techniques suffer from this issue (eg - See [A]). \n\n\nQuestions for the author:\n\n1. What is the importance of XAI based filter importance criterion? Wouldn't other type of filter pruning criterion also help with this objective?\n2. One of the arguments that the authors make is that this work reduces the longest-path latency of inference, however the author do not provide any data showing what the worst case scenario for prior works looks like and why is that a problem worth solving?\n\nSuggestions for improvement:\n1. The authors should strengthen the motivation of this work (See weakness 3 & 4). There are a lot of open questions regarding the choices for the architecture made in this work that are not backed by specific data showing that the reasons for the said decisions are reasonable.\n2. There is a huge body of work in the domain of dynamic filter pruning. See related work in [A] and [B]. Some of these work eliminate the need of optimizing for worst case latency and provide strong benchmarks to compare against\n\n[A] https://arxiv.org/abs/1810.05331\n[B] https://dl.acm.org/doi/10.1145/3417313.3429380\n[C] https://arxiv.org/abs/2003.03033", "summary_of_the_paper": "This paper focusses on a dynamic filter pruning technique that reduces the longest path latency of inference while using explainable AI (XAI) to help with determining pruning criterion. The approach uses an early coarse prediction branch that is used to perform a top-k classification. This branch is added to the middle section of the neural network. If this branch either predicts the output class with high prediction, then the rest of the computation is skipped. Else, this branch predicts the top-k classes for this input and executes the rest of the part of the NN. While executing this part, only filters useful for predicting these classes are loaded during execution. To determine the set of filters useful for a certain set of classes, the paper uses XAI to determine importance criterion for these filters for each class. The authors evaluate their work on CIFAR 10 and CIFAR 100 benchmarks on VGG11&16 and ResNet20 architecture while comparing against a dense benchmark and a simple static pruning technique.", "summary_of_the_review": "The motivations in this paper are weakly established. Further the results in this paper have not been compared against relevant benchmarks or on more established benchmarks. Given these issues, I recommend rejecting this paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "YhopupQUyjd", "reviewer": "Reviewer_nAuM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "strengths:\n\ngood readability except for the XAI section  (which is to brief)\nEasy to understand concept\nfaster on CIFAR-100\n\nweaknesses:\n\n-incremental novelty, it is a combination of existing ideas\n-no speed gains on CIFAR-10\n-experimental evidence is done using only two networks and only CIFAR datasets.\n\n-What is needed: add a second modern network (post-resnet)\nadd a more complex dataset than just the sketchy cifar-10 datasets, e.g. LSUN, COCO or the smaller Pascal VOC or similar whatever size fits the budget, as long as the images are of a higher resolution than 32x32 have a higher intraclass variability than CIFAR.\n\nTable 1 shows what ? top-k performance for the top-k prediction heads ?\n- analyze the impact of the k in top-k  on the performance of the whole model after pruning.\n\nThe paper would strongly benefit from more extensive experiments.\n\nSome missing details (weaknesses):\n\n-section on pruning XAI is somewhat too brief and needs a clearer explanation of the process\n\nThey store relevant filters precomputed for each class  ?\nOr on what outputs else are the deeplift scores initialized?\n\n-\"Based on the coarse prediction, the XAI dynamic pruner selects the filter kernels of pruned\nconvolution layers that need to be used.\"\n\nHow do they combine filters for the classes from the top-k prediction for k>1?\ndo they use only k=1?\n\n-How much do you prune ? How do you decide how much to prune?\nThis is even more so important because for some reason CIFAR-100 (same resolution) results in a much larger+slower model (Table 6 vs Table 7). Is it just because of a 10x increased number of output classes and thus retained filters in the oversized linear branches of VGG-16?\n\nlanguage (minor):\n- \" The\nimportances are obtained once in the start and during prediction of the sample, the ranking of filters\nis done. \"\nThis sentence can be misread as:\nThe importances are obtained once in the start and during prediction of the sample.\n\nIt needs an \",and,\" or, better, a rewrite.\n\nWhy table 7 lists reference model when the text states a statically pruned baseline?\n\nQuestion: with what loss and labels the confidence in the exit branch is trained?\n\n\nCitations:\nconsider to cite https://ieeexplore.ieee.org/document/9502445 instead of the arxiv for [20]\nalso please consider to cite:\nhttps://www.sciencedirect.com/science/article/pii/S0031320321000868\nhttps://arxiv.org/abs/2103.06460\n", "summary_of_the_paper": "The authors propose a set of steps in order to reduce the latency and the flops computation effort of neural networks. They add early exit prediction layers, a top-k prediction layer and dynamic pruning based on the set of classes from the top-k prediction layer. The pruning uses statistics precomputed from a validation dataset in order to decide which filters are to be used for which class. The statistics are obtained from deeplift scores.", "summary_of_the_review": "Its a paper which is a bit below the perceived acceptance level. It is an empirical paper, therefore for a high ranked conference it should come with a larger experimental evaluation and clearer details on how the XAI was used and how much was pruned. Most problems have higher complexity than CIFAR-10/100.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0buolkG7MNS", "reviewer": "Reviewer_kGTk", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I think it is an interesting approach, but the novelty is minimal. The authors combine multiples techniques that already exists in the literature. Although the combination is done in a clever way, all techniques are barely explained, making the reader to go to the original papers to properly reproduce their approach.\n\nRegarding to the experimental setup, too much work has to be done before publishing it. There is no comparison against other dynamic pruning algorithms like [1] or [2]. Furthermore, it is only tested on huge networks that have a bad accuracy vs. size ratio. I suggest the authors to include both more datasets and networks.\n\nFinally, the results obtained are not impressive if we compare them against the papers mentioned above.\n\n\n[1] Liu, Z., Xu, J., Peng, X., & Xiong, R. (2018, December). Frequency-domain dynamic pruning for convolutional neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (pp. 1051-1061).\n\n[2] Lin, S., Ji, R., Li, Y., Wu, Y., Huang, F., & Zhang, B. (2018, July). Accelerating Convolutional Networks via Global & Dynamic Filter Pruning. In IJCAI (Vol. 2, No. 7, p. 8).", "summary_of_the_paper": "The authors propose a dynamic pruning algorithm by combining multiple methods like early exit and a top-k loss function. The experimental results show a good performance over the vgg network, whereas the performance when using resnet architectures decays a little bit.", "summary_of_the_review": "Although the idea is interesting, the novelty is minimal. Besides that, more work has to be done in the experimental section to prove this new algorithm can achieve state-of-the-art results in dynamic pruning.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: ### Summary

This paper presents a technique to reduce the worst-case latency of inference. The key idea is to use a combination of early exit and filter selection to achieve its results. The filter selection predicts the top-k classes for the input and, using that indication, uses the filters that are the most relevant  (using DeepLIFT) to refine the result.

### Strengths (from Discussion)

- The idea is interesting. Early exit, mixtures of experts (one potential interpretation of the filter selection here), as well as pruning are interesting mechanisms for neural network efficiency. 
There may be new opportunities to find synergies in their combination.

### Weaknesses (from Discussion)

- The clarity of writing could be significantly improved, particularly in the description and illustration of the constituent techniques. Figures, such as those in https://arxiv.org/abs/2008.13006, that clearly present the constitution of various layers, in particular, would help.

- There are relevant and applicable baselines that a comparison would contextualize the strength of the approach (as per Reviewer vKUc examples)

- ImageNet experiments appear to be within reach of this experimental apparatus (i.e., without extreme cost). Hence, such experiments would validate the applicability of this approach to practice. 

- A small point arose that longest-path inference was not motivated. Work on optimizing tail latency (https://research.google/pubs/pub40801/) may be helpful contextualization here.

### Recommendation 

My recommendation is Reject. The work here is a very promising start for a new idea. Though requests for additional experimentation and baselines can be ill-defined recommendations. Here, scaling of the results to ImageNet as well as comparing against baselines in the literature (as per Reviewer vKUc's examples) would provide much stronger scoping for this work.
Decision: Reject