Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning
Authors: Jianda Chen
Shangyu Chen
Sinno Jialin Pan
Venue:  (2020)

Abstract: In this paper, we propose a deep reinforcement learning (DRL) based framework to efficiently perform runtime channel pruning on convolutional neural networks (CNNs). Our DRL-based framework aims to learn a pruning strategy to determine how many and which channels to be pruned in each convolutional layer, depending on each specific input instance in runtime. The learned policy optimizes the performance of the network by restricting the computational resource on layers under an overall computation budget. Furthermore, unlike other runtime pruning methods which require to store all channels parameters in inference, our framework can reduce parameters storage consumption at deployment by introducing a static pruning component. Comparison experimental results with existing runtime and static pruning methods on state-of-the-art CNNs demonstrate that our proposed framework is able to provide a tradeoff between dynamic flexibility and storage efficiency in runtime channel pruning. 

Under review as a conference paper at ICLR 2020
STORAGE EFFICIENT AND DYNAMIC FLEXIBLE RUN- TIME CHANNEL PRUNING VIA DEEP REINFORCEMENT LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, propose deep reinforcement learning (DRL) based framework to efficiently perform runtime channel pruning on convolutional neural networks (CNNs). Our DRL-based framework aims to learn pruning strategy to determine how many and which channels to be pruned in each convolutional layer, depend- ing each specific input instance runtime. The learned policy optimizes the performance of the network by restricting the computational resource on layers under overall computation budget. Furthermore, unlike other runtime pruning methods which require to store all channels parameters for inference, our frame- work can reduce parameters storage consumption for deployment by introducing a static pruning component. Comparison experimental results with existing run- time and static pruning methods on state-of-the -frtt CNNs demonstrate that our proposed framework is able to provide a tradeoff between dynamic flexibility and storage efficiency runtime channel pruning.
INTRODUCTION
In recent years, convolutional neural networks (CNNs) have been proven to be effective a wide range of computer vision tasks, such as image classification (Krizhevsky al.] 2012 Simonyan & Zisserman 2015; He et al.) 2016), objection detection (He et al.] 2017 Zhou et al. 2019 Law & Deng 2018), segmentation (He etal 2017 Zhu al. 2019). Therefore, nowadays, many computer- vision-based systems, such as automatic-driving cars, security surveillance cameras, and robotics, are built on the power of CNNs. However, since most state-of-the-art CNNs require expensive computation power for inference and huge storage space to store large amount of parameters, the limitation of energy, computation and storage on mobile or edge devices has become the major bot- tleneck real-world deployments of CNNs. Existing studies have been focused on speeding up the execution of CNNs for inference on edge devices by model compression using matrix decom- position (Denil et al.) 2013 Masana al. (017), network quantization (Courbariaux et al.) 2016), network pruning (Dong et al. 2017). 7). etc. Among these approaches, channel pruning has shown promising performance (He et al.) 2017]: [uo al. 2017 /Zhuang et al.) 2018 Peng et al.) 2019). Specifically, channel pruning discards an entire input or output channel and keep the rest of the model with structures.
Most channel pruning approaches can be categorized into two types: runtime approaches and static approaches. Static channel pruning approaches aim to design measurement to evaluate the im- portance of each channel over the whole training dataset and remove the least important channels to minimize the loss of performance after pruning. By permanently pruning number of chan- nels, the computation and storage cost of CNNs can be dramatically reduced when being deployed, and the inference execution can be accelerated consequently. Runtime channel pruning approaches have been recently proposed to achieve dynamic channel pruning each specific instance (Gao) ctt al. 2019 Luo & Wu 2018). To be specific, the goal of runtime approaches aims to evaluate the channel importance at runtime, which is assumed to be different on different input instances. By pruning channels dynamically, different pruned structures can be considered as different routing of data stream inside CNNs. This kind of approaches is able to significantly improve the representation capability of CNNs, and thus achieve better performance in terms prediction accuracy compared with static approaches. However, previous runtime approaches trade storage cost off dynamic flex-

---

Under review as a conference paper at ICLR 2020
ibility of pruning. To achieve dynamic pruning different specific instances, all parameters of kernels are required to be stored (or even more parameters are introduced). This makes runtime approaches not applicable on resource-lmitted edge devices. Moreover, most of previous runtime approaches only evaluate the importance among channels in each single layer independently, with- out considering the difference in efficiency among layers.
In this paper, to address the aforementioned issues of runtime channel pruning approaches, we pro- pose a deep reinforcement learning (DRL) based pruning framework. Basically, we aim to apply DRL to prune CNNs by maximizing received rewards, which are designed to satisfy the overall budget constraints along side with network's 's training accuracy. Note that automatic channel pruning by DRL is a difficult task because the action space is usually very huge. Specifically, the discrete action space for the DRL agent as large as the number of channels at each layer, and the action spaces may vary among layers since there are different numbers of channels in different layers. To facilitate pruning CNNs by DRL, for each layer, we first design novel prediction component to estimate the importance of channels, and then develop DRL-based component to learn the sparsity ratio the layer, i.e., how many channels should be pruned.
More specifically, different from previous runtime channel pruning approaches, which only learn runtime importance of each channel, we propose to learn both runtime importance and additionally static importance for each channel. While runtime importance maintains the saliency of specific channels for each given specific input, the static importance captures the overall saliency of the corresponding channel among the whole dataset. According each type of the channel importance, we further design different DRL agents (i.e., a runtime agent and a static agent) to learn a sparsity ratio in a layer-wise manner. The sparsity ratio learned by the runtime agent together with the estimated runtime importance of channels are used to generate runtime pruning structures, while the sparsity ratio learned by the static agent together with the estimated static importance of channels are used to generate static (permanent) pruning structures. By considering both the pruning structures, our framework able to provide a trade-off between storage efficiency and dynamic flexibility for runtime channel pruning-
In summary, our contributions are 2-fold. First, we propose to prune channels by taking both runtime and static information of the environment into consideration. Runtime information endows pruning with flexibility based on different input instances while static information reduces the number of parameters in deployment, leading to storage reduction, which cannot be achieved by conventional runtime pruming approaches. Second, we propose to use DRL to determine sparsity ratios, which is different from the previous pruning approaches which manually set sparsity ratios. Extensive experiments demonstrate the effectiveness of our method.
RELATED WORK AND PRELIMINARY
2.1 STRUCTURE PRUNING
Wen et al. (016)) pioneered structure pruning in deep neural network by imposing the L2 norm in training. Under the same framework, [iu et al. (2017) regarded parameters in batch normaliza- tion as channel selection signal, which is minimized to achieve pruning during training. [e et al. (217)) formulated channel pruning into two-step iterative process including LASSO regression based channel selection and least square reconstruction. [uo et al. (2017) formulated channel prun- ing as minimization of difference of output features, which solved by greedy selection. Zhuang al. (2018) further considered early prediction, reconstruction loss and final loss to select impor- tance channels. Overall, structure pruning methods accelerate inference by producing regular and compact model. However, this brought regularness requires preserving more parameters to ensure performance.
2.2 DYNAMIC PRUNING
Dynamic pruning provides different pruning strategies according to input data. Wang et al. (2018) proposed to reduce computation by skipping layers or channels based on the analysis of input fea- tures. Luo & Wu (2018) proposed to layer input to leam channel importance, which is then binarized for pruning. Gao et al. (2019) applied the same framework while extended features selec- tion in both input and output features. Similarly, Liu & Deng (2018) introduced multiple branches
2

---

Under review as a conference paper at ICLR 2020
Conv layer
Importance predictor
F
F Fou
Runlime agent (RNIN)
Solect
Trade-off
Static agent (RNN)
Bi
Figure Illustration of our proposed DRL-based runtime pruning framework.
for runtime inference according inputs. A gating module is learnt to guide the flow of feature maps. Bolukbasi et al. (2017) learned to choose the components of a deep network to be evalu- ated for each input adaptively. Early exit is introduced to accelerate computation. Dynamic pruning adaptively takes different actions for different inputs, which is able to accelerate the overall inference time. However, the original high-precision model needs to be stored, together with extra parameters for making specified pruning actions. [Rosenbaum et al. (2018) proposed to learn routers to route layers output to different next layers, in order to adjust a network multi-task learning.
2.3 DEEP REINFORCEMENT LEARNING IN PRUNING
Channel selection is on trial using deep reinforcement learning. Lin et al. (2077) trained a LSTM model to remember and provide channel pruning strategy for backbone CNN model, which is con- ducted using reinforcement learning techniques. He et al. (2018) proposed to determine the com- pression ratio in each layer by training an agent regarding the pruning-retraining process as an envi- ronment.
2.4 PRELIMINARY
Reinforcement Learning We consider standard setup of reinforcement learning: an agent se- quentially takes actions over a sequence of time steps in an environment, order maximize the cumulative reward (Sutton & Barto 1998). This problem can be formulated as a Markov Deci- sion Process (MDP) of a tuple (8, A. P. R. 7), where S is the state space, A the action space, P S x A x S + [0, 1] is transition probabilities, R S A - R is the reward function, and E 1) is the discount factor. The goal of reinforcement learning is to learn policy т ((a) s) that maximizes the objective of cumulative rewards over finite time steps,
T
R(s..a)).
where E S and E A are state and taken action at time step t respectively.
3 DRL-BASED RUNTIME PRUNING FRAMEWORK
The overview of our proposed framework is presented in Fig. To prune convolutional layer 4, we learn two types of learnable channel importance: runtime channel importance u, E Rx1 and static channel importance u, E RCx1, where C is the number of channels in layer t. The runtime channel importance u, is generated by subnetwork importance predictor f()), which takes the input feature map Fin input, while the static channel importance u, is randomly initialized and updated during training. Both u, and u, indicate the channel importance of the full precision output feature map F out through convolution layer. Channels are selected to be pruned according to the values of each element in u, and ug, and how many channels to be selected decided by the sparsity ratios d, and dg, respectively. To learn the sparsity ratios d, and d,, two DRL agents, the runtime agent and the static agent, are introduced, where actions of and at are defined to set values of d, and d,, respectively. The detail of the two DRL agents described in Sec. 3.3 Consequently,
3

---

Under review as a conference paper at ICLR 2020
trade-off pruner g(-) is performed to balance the runtime and static pruning results, and output a decision mask M binary values (1/0) to indicate which channels to be pruned (1: pruned. 0: preserved), well as unified channel importance vector ICX1 as follows,
M, gur, Us d,, d.).
The final output after pruning is constructed by multiplying the full precision output feature map F oout by 1 M and u as,
F Font = Fou & (1 M) Qu u.
where is the broadcast element multiplier, and is the matrix of the same size as M with all the elements being 1. In the following, we introduce how to learn the runtime channel importance vector u, and the static channel importance vector u, in Sec. 3.1 how to construct the trade-off' pruner g()) in Sec. 3.2. and how to design the two DRL agents in Sec. 3.3
3.1 LEARNABLE CHANNEL IMPORTANCE
We consider that a convolutional layer takes input of feature map Fin. E RC AgCin.Hia XH,,, XW,,. and gen- erates output feature map out E Co x Hour W out, where C., HI, and W, are the number of channels, width and height of the feature map F., respectively. Each element of the channel impor- tance vectors u, RCoue and u, Cost represents the importance value of the corresponding channel, respectively. In the following, we drop the subscript out for simplicity in presentation.
3.1.1 RUNTIME CHANNEL IMPORTANCE
As mentioned above, the runtime channel importance u, of output feature Fout predicted by a importance predictor f()), which takes Fin as input. Therefore, can be considered as a function of F... whose values vary over different input instances. In this paper, we design a subnetwork to approximate f()), which is expected to be of small size and computationally efficient. Similar to many existing dynamic network pruning methods (Gao et al. 2019 Hu al. 2018 Luo & Wul 2018).
global pooling layer the first layer in f()), because global pooling is computation- ally efficient and it can reduce the dimension of Fi, dramatically. We then feed the output of global pooling into fully-connected layer without any activation function. The output of fully-connected layer is the runtime channel importance vector u,,
Which channels be preserved pruned at runtime are determined according the values of u,- We denote by M, E {0, 13 a mask for pruming, where if the value is O, then the corresponding channel is preserved, otherwise pruned. For now, suppose sparsity ratio d, for runtime pruning has already been generated via the dynamic DRL agent, which will be introduced in Sec. 333 We then prune (C [1) channels with the smallest importance values in u, Accordingly, the value of an element in M, is set to be if the corresponding channel is pruned, otherwise 0.
3.1.2 STATIC CHANNEL IMPORTANCE
The static channel importance vector u, is to capture the global information for pruning, and thus learned from the whole dataset. It is randomly generated and learned through backpropagation. Similar to runtime channel pruning, given a sparsity ratio de learned by the static DRL agent, (C [d, channels with smallest importance values u, are pruned, and mask M, E {0, 1] is generated to indicate the static pruning results.
3.2 TRADE-OFF PRUNER
With the runtime and the static pruning decisions, M, and M., we now propose a trade-off pruner to generate unified channel pruning decision. The main idea behind the trade-off pruner is to 1) prune those channels which are agreed to be pruned by both decisions, and 2) prune portion of the rest channels by weighted votes from both decisions.
To be specific, we define the mask representing channels pruned by both decisions as
M. = M, M..
(3)
where A is element-wise logical AND and 1/0 in mask represents logical true or false. The channels indicated to be pruned by M, (i.e., the corresponding values are 1) are pruned in final. The channels
4

---

Under review as a conference paper at ICLR 2020
which are determined be pruned by M, but not by Mg can be represented by new mask M, M. - M,. Similarly, the channels which are determined to be pruned by M, but not by M, can be represented by another new mask M, M, M..
To control the trade-off between M, and M,, we define rate R, denoting how much we trust the pruning decision made by M,. while 1 - R is for M,. That means the channels selected by M, will be finally pruned with the rate R,, Specifically, the number of channels which are selected by M, and finally will be pruned is
LR, M.)J, (1)M),,
where 1 M, returns the number of channels selected by M,. We then select the first C, -mallees important channels which are recommended to be pruned by M, form a mask M,. Similarly, for static pruning, we select the first C--smallest important channels which are recommended to be pruned by M, to form another mask M,, where C. 111-1111p) R1 M.
The final trade-off pruning mask is defined as
M M. M, + M,,
(5)
Moreover, in this work, the unified channel importance is simply defined as follows,
u up X Ug
(6)
With the trade-off pruning mask M and the unified channel importance u, the pruned output feature F out can be generated by Eq. 2
3.3 DRL BASED PRUNING
In this section, we present how formulate the problems of learning the ratios d, and de for static pruning and runtime pruning, as MDP, and solve it via DRL, respectively.
3.3.1 DRL FOR RUNTIME PRUNING
In the MDP for runtime pruning, we consider the t-th layer of the network as the t-th timestamp. The details of the MDP are listed as follows.
Given an input feature map F, of layer we pass it to a global pooling layer to reduce its dimension to Cr, where Cin, is the number of input channel of layer t. Since On varies among layers, we feed the output of global pooling a layer-dependent encoder to project it to fix-length vector sf, which is considered as as state representation of DRL in the context of runtime pruning.
State
The action at is defined as the sparsity ratio at layer t, alternating d, in runtime pruning mentioned in Section 3.1.I Existing DRL-base pruning method RNP (Lin et (017)) uses a unified discrete actions space with k actions which are too coarse to achieve high accuracy. However, fine- grained discrete action space as large as number of channels suffers from exploration difficulty. Therefore, instead of using discrete action spaces, we propose a continuous action space with action af E (0, 1]. To avoid over-pruning the filters and crashing in training, we set minimum sparsity ratio +a such that of E (+a,1).
Action
Reward The reward function is proposed consider both network accuracy and computation budget. We define the accuracy relative reward based on the loss of pruned backbone network,
R LCNNN
where LONN is the loss in CNN, and it may vary in scale among different training stage, i.c. large at beginning of training and small near convergence. To avoid the instability brought by the reward scale, R is normalized by a moving average,
R R Bo,
(8)
Bb A + (1 R)
(9)
5

---

Under review as a conference paper at ICLR 2020
where B6 is the moving average at the b-th training batch and X is the moving weight.
To force computation of the pruned network under a given computation budget, we define expo- nential reward function of budget regarding reward Rud
[exp(ar (Beom Boom)) 1, B ceom B coo:
Ridd
(10)
0,
otherwise,
where Boom is the computation consumption, which calculated based on the current of pruned strategy, and B com is the given computation budget constraint. Finally we sum up the two rewards form sparse rewards, with being non-zero at terminated step 7 and zeros at other time step < T,
{ Ricce + Rind' 10
1=
R
(11)
<
To solve the continuous action space problem, we choose a commonly used actor-critic agent with a Gaussian policy. Actor-critic agent consists of two components: 1) actor outputs the mean and variance to form a Gaussian policy where the continuous action are sampled from; 2) critic outputs scalar predicting the future discounted accumulated reward and assists the policy training. Actor network and Critic network share one-layer RNN which takes state s as input. The output of RNN is fed into actor specific network constructed by two branches of fully- connected layers, leading to the mean and variance of the Gaussian policy. The action is sampled for the Gaussian distribution outputed by the actor:
Actor-Critic Agent
a ~
(12)
where u(s{;0") and (sT; (*) is the mean and variance outputed from actor network. The Critic specific network has one fully-comnected layer after the shared RNN, and outputs the predictive value (s[::"))
To optimize the actor-critic agent, Proximal Policy Optimization (PPO) (Schulman et al. 2017) is used. Note that we relax the action of to (-o0, PPO, and use truncate function to clip oj in (+a, 1] when perform pruning.
Besides, an additional regularizer is introduced to restrict the relaxed of staying in range (+a, 1].
1 La - 10 max(min(a), 1), +
(13)
2
3.3.2 DRL FOR STATIC PRUNING
Similar to runtime pruning, the MDP in static pruning is also formulated layer-by-layer. The differ- ence against runtime pruning is the definition of state and reward.
State The state sf in static pruning is defined as the full shape of F Fout. and does not depend on out and the current input data.
Action Action of is sampled from actor's 's outputed Gaussian policy, and it is to alternate the spar- sity de in static pruning mentioned in Section 3.1.2
The reward function takes both network accuracy and parameters budget into considera- tion. The accuracy relative is defined as the same as that in runtime pruning:
Reward
R
R
(14)
To reduce the number of parameters of network satisfy the parameters storage budget, the param- eters relative reward is defined in an exponential form as,
f 0,
B Bparam)) 1,
Bparom > B Bparam:
R Reeramm
(15)
otherwise,
where B Bperam is the number of preserved parameters after static pruning and B Bporam is the param- eters storage budget.

---

Under review as a conference paper at ICLR 2020
Method
Baseline acc.
Acc.
Aacc.
Speed-up
#Params
FBS
Gao et al. 2019
91.37
89.88
-1.49
3,93x
1.11x
RNP Lin al.
2017
92.07
84.93
-7.14
3,56x
1,00
ours (runtime only R 1)
92.07
91.333
-0.737
3.92x
1.31
ours RR 0.5)
92.07
91.066
-1.004
3.92x
0.78x
Table 1: Comparison state-of-the-ar runtime pruning methods on CIFAR-10 at sparsity 0.5.
Speed-up is calculated on MACs.
Method
Baseline acc.
Acc.
Aacc.
Speed-up
#Params
FBS (Gao al. 2019)
91.37
91.23
-0.14
2x
1.11x
(R, 1.0)
92.07
93.178
+1.108
1.99x
1.31x
ours (R, = 0.5)
92.07
92.502
+0.432
1.99x
0.97x
Table 2:
Comparison to state-of-the-ar runtime pruning methods on CIFAR-IO at sparsity 0.7. Speed-up is calculated on MACs.
Actor-Critic Agent This agent is similar to the one runtime pruning. It has the same archi- tecture runtime pruning but differs in introducing fully-connected layer the encoder before RNN. This agent is also optimized by PPO.
3.4 INFERENCE
In inference, the static agent is not required any more because the static pruning strategy does not depend on individual input data points but the full shape of Fn. Therefore, the output action az is fixed to each layer t. With the action af and the rate R,, decide which filters be pruned permanently. Specifically, channels with ((1 af) R,)))smalles static importance values are pruned permanently.
4 EXPERIMENT
We evaluate our DRL pruning framework on two popular datasets: CIFAR-10 (Krizhevsky) 2009) and ImageNet ILSVRC2012 (Russakovsky et al. 2015). to show the advantage over other chan- nel pruning methods. We analyze the effect of hyper-parameters and different sparsity settings on 100. For CIFAR-10 10, we use M-CifarNet (Zhao et al., 2018) as the backbone CNN. On Ima- geNet ILSVRC2012, ResNet- is used as the backbone CNN.
4.1 IMPLEMENTATION DETAILS
We start with a pretrained backbone CNN. Firstly we finetune the backbone CNN and train runtime importance predictor jointly, with sparsity d, 1 and fixed all static pruning importance u, 1. Then we remove the restriction on the static pruning importance u,, and train static pruning impor- tance, the backbone CNN and the runtime importance predictor, with sparsity d, = 1 and runtime pruning sparsity fixed as d, = 1. After finetuning, we use the DRL agents to predict the sparsity given computation and storage constraints. The DRL agents and the CNN with runtime/static im- portance are trained in alternating manner: We first fix the CNN as well runtime/static importance and train two DRL agents, regarding the CNN as environments. Then we fix two agents and finetune the CNN and runtime/static importance. We repeat these two steps until convergence is achieved. We use Adam optimizer for both DRL agent and CNN, and set learning rate at 10 for the DRL agents. For CNN finetuning and runtime/static importance training, the learning rate is set 10-3 on CIFAR-10. On ImageNet ILSVRC2012, the learning rate starts from 10-3 and is divided by 10 after 15 millions iterations.

---

Under review as a conference paper at ICLR 2020
2330000
05
10
10.30
1100000
301
60.00
1220000
140
3350000
09..0
835000
430000
09.00
85
Figure 2: Trade-off between runtime pruning and static pruning at sparsity 0.45. X-axis is the rate R.
Figure 3: Comparison accuracy drop for M- CifarNet on CIFAR-10 with computational budget.
Baseline top-1 acc.
Top-1
À top-1
Baseline top-5 acc.
Top-5 acc.
À top-5
Method
Speed-up
acc.
acc.
acc.
DCP (huang el al
2018
69.64 64
67.35
-229
88.98 89.63
88.86
-0.12
1.71x
FPGM [Ho al. 2019
70.28
68.41
-1.87
88.48
-1.15
1.71
Dynamic Sparse Craph
2019
69.48
64.8
-4.68
1.4
CGNN (Hua et al. 2018
69.02
67.95
-1.07
88.84
88.21
-0.63
1.63x
FBS (Gão et al. 2019
70.71
68.17
-2.54
89.68
88,22
-1.46
1,98x
Ours (H, 0.3)
69.758
68.79
-0.968
89.078
88.534
-0.544
1.94x
Table 3: Comparison with the state-of-the-an channel pruning ResNet- 18 on ImageNet. Speed-up calculated on MACs.
4.2 EXPERIMENTAL RESULTS ON CIFAR-10
We compare our proposed method with the following state-of-the-art runtime pruning methods: FBS (Gao et al. 2019) RNP (Lin al. 2017) on CIFAR-10. The comparison results sparsity 0.5 and 0.7 are shown in Table I and Table respectively. Note that for fair comparison with other methods, the computation and storage budget constraints in our method is calculated according to the sparsity of other methods. Under these constraints, our method does not necessarily lead to the same sparsity as other methods in each layer. RNP cannot set exact sparsity ratio. Instead, its average sparsity ratio is accessible only during testing, which is 0.537 in Table The result of FBS reproduced using the released code The column #Parums represents the number of parameters compared to the backbone CNN.
Table shows that our method outperforms other state-of-the-art methods, achieving highest aceu- racy at overall sparsity ratio of 0.5, Our method has very close computation speed-up compared to FBS, but outperforms FBS around 0.48% to 0.76%. When the runtime pruning strategy is solely considered by setting R, 1, method surpasses other comparison methods, indicating that Dased framework improves the performance of channel runtime pruning. By balancing run- time and static pruning via setting R, 0.5, method reduces the number of the overall stored parameters and achieves lower accuracy drop than other methods. Table shows that our method outperforms FBS at sparsity of 0.7. When R2 0.5, our method achieves better performance than the baseline CNN with 2 speed-up and contains less parameters.
We also study the relation between R2 and network compactness in our framework. Fig. 2 demon- strates the impact of R, when sparsity is 0.45. The hyperr-paramete R, determines how much we trust about runtime pruning. With R, close dynamic network flexibility but the space of the parameters storage also increases. When R, dimin- ishes, the network accuracy decreases but the parameter storage is reduced.
1, the accuracy becomes higher due to the more
Fig. 3 shows the performance of various sparsity ratios in our method. Again, our method does not prune with one single sparsity ratio for all layers, but uses the sparsity ratio to calculate computation and storage constraints, with which the sparsity ratio is learned for each layer. Fig. demonstrates
8

---

Under review as a conference paper at ICLR 2020
that method holds the accuracy when sparsity is larger than about 0.5, which corresponds to about 4x computational acceleration.
4.3 EXPERIMENTAL RESULTS ON IMAGENET ILSVRC2012
We compare our method with state-of-the channel pruning methods on ImageNet ILSVRC2012
as shown in 3 In this experiment, we use ResNet- 18 as the backbone CNN. Among the state- of-the-art pruning methods for comparison, FBS (Gao et al. 2019) and CGNN (Hua et al. 2018) are runtime pruning methods. The overall sparsity ratio of our method is 0.7, which is under the same setting of FBS. Our method with R, 0.5 achieves the smallest top-1 accuracy drop com- pared with other methods, and also achieves the highest top-1 accuracy after pruning. Overall, our proposed method achieves comparable or better performance compared to other methods with more acceleration. Our method has very close MACs to FBS, while the number of preserved parameters reduced to 81.2% of the baseline.
5 CONCLUSION
In this paper, we present a deep reinforcement learning based framework for deep neural network channel pruning in both runtime and static sheme. Specially, channels are pruned according input feature runtime pruning, and based on entire training dataset as static pruning, with 2 reinforce- ment agents determine the corresponding sparsity. Our method combines the merits of runtime and static pruning, and provides trade-off between storage and dynamic flexibility. Extensive exper- iments demonstrate the effectiveness of our proposed method.
REFERENCES
Tolga Bolukbasi, Joseph Wang, Ofer Dekel, and Venkatesh Saligrama. Adaptive neural networks for efficient inference. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 527-536. JMLR. org, 2017.
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El- Yaniv,, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1. arXiv preprint arXiv: /et.::0022..220 2016.
Misha Denil, Babak Shakibi, Laurent Dinh, Marc' Aurelio Ranzato, and Nando de Freitas. Predicting parameters in deep learning. In Advances in Neural Information Processing Systems 26, pp. 2148- 2156. Curran Associates, Inc., 2013.
Xin Dong, Shangyu Chen, and Sinno Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. In Advances in Neural Information Processing Systems, pp. 4857 2017.
Xitong Gao, Yiren Zhao, ukasz Dudziak, Robert Mullins, and Cheng zhong Xu. Dynamic channel pruning: Feature boosting and suppression. In International Conference Learning Represen- tations, 2019. URL / / forum 1o
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, June 2016. doi: 10. N0.1109/CVPR.2016333
K. He, G. Gkioxari, P. Dollr, and R. Girshick. Mask r-cnn. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988, Oct 2017. doi: 10. 109//CCV.2017.322
Yang He. Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang. Filter pruning via geometric median for deep convolutional neural networks acceleration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4340-4349, 2019.
Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural net- works. Proceedings of the IEEE International Conference on Computer Vision, pp. 1389-1397, 2017.

---

Under review as a conference paper at ICLR 2020
Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Ame: Automl for model compression and acceleration on mobile devices. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 784-800, 2018.
Jie Hu, Li Shen, and Gang Sun. gguuezz--ad-excitttin networks. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2018.
Weizhe Hua, Christopher De Sa, Zhiru Zhang, and G Edward Suh. Channel gating neural networks. 2018.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 25, pp. 1097- wtwpperr.niiipoi. 4824 pwageeettccaaseee fiattfiiiati....... oo- ti-wiit otw.:ttdeeeeerrooooooooooooooaaa networks.
1105. Curran Associates, Inc., 2012. URL
pdf
Hei Law and Jia Deng. Comernet:
Detecting objects as paired keypoints. In The European Confer- ence on Computer Vision (ECCV), September 2018.
Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou Runtime neural pruning. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 2181-2191. Curran Associates, Inc., 2017. URL 6813 runt imme neural-pruning: pdf
http://papers. wwwprs.ppppppepapppeepeaaaaeeeap http::sssipnpp.ppppppeeeeee
Lanlan Liu and Jia Deng. Dynamic deep neural networks: Optimizing accuracy-efficienccy trade-off's by selective execution. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Liu Liu, Lei Deng. Xing Hu, Maohua Zhu, Guoqi Li, Yufei Ding, and Yuan Xie. Dynamic sparse graph for efficient deep learning. In International Conference on Learning Representations, 2019.
URL https::///// /attps://openreviiie
net/ / forum? Hd-hundo---111111
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learn- ing efficient convolutional networks through network slimming. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2736-2262222 7736-4444, 2017.
Jia Luo and Jianxin Wu. Autopruner: An end-to-end trainable filter pruning method for efficient deep model inference. CoRR, abs/1805.08941, 2018. URL http::/aaxiv..eeett
abs/1805.0893 08941
Jaan-Hao Luo, Jianxin Wu, and Weiyao Lin. Thinet: A filter level pruning method for deep neural network compression. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Mare Masana, Joost van de Weijer, Luis Herranz, Andrew D. Bagdanov, and Jose M. Alvarez. Domain-adaptive deep network compression. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Hanyu Peng, Jiaxiang Wu, Shifeng Chen, and Junzhou Huang. Collaborative channel pruning for deep networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume of Proceedings of Machine Learning Research, pp. 5113-5122, Long Beach, California, USA, 09-15 Jun 2019. PMLR.
Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing networks: Adaptive selection of non-linear functions for multi-task learning. In International Conference on Learning Represen- tations. 2018. URL htttstt::: http:://ppeeeeieeeeee net lttee//orrmmmmd-yyytoeo
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei- Fei. Imagenet large scale visual recognition challenge. International Journal Computer Vi- sion, 115(3):211 1 1-25, Dec 2015. ISSN 1573-1405. doi: 10.10077/112233-05-000166 URL -10-330------55
https://do: org. 00. 000000000201130111111111
10

---

Under review as a conference paper at ICLR 2020
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1 717770677777 2017. URL http:// http::///axvv..oo org.//oorr///.. abs/
1107006347
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations (ICLR), 2015.
Richard S. Sutton and Andrew G. Barto. Introduction Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998. ISBN 0262193981.
Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph Gonzalez. Skipnet: Learning dy- namic routing in convolutional networks. In Proceedings of the European Conference Com- puter Vision (ECCV), pp. 409-424,20 2018.
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In Advances in neural information processing systems, pp. 2074-2082, 2016.
Yiren Zhao, Xitong Gao. Robert Mullins, and Chengzhong Xu. Mayo: A framework for auto- generating hardware friendly deep neural networks. In Proceedings of the 2Nd International Workshop on Embedded and Mobile Deep Learning, EMDL' 18, pp. 25-30, New York. NY, USA, 2018. ACM. ISBN 978 6506--5844-66 doi: 10.1 0..1445/2122755321277 acm. www.orgg....... 100..1111/222225..... 3212726
UtLhttp://doo
Xingyi Zhou, Dequan Wang, and Philipp Krähenbühh. Objects points. In arXiv preprint 2019.
Yi Zhu, Karan Sapra, Fitsum A. Reda, Kevin J. Shih, Shawn Newsam, Andrew Tao, and Bryan Catanzaro. Improving semantic segmentation via video propagation and label relaxation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang, Jing Liu, Yong Guo, Qingyao Wu, Junzhou Huang, and Jinhui Zhu. Discrimination-aveni channel pruning for deep neural networks. In Advances in Neural Information Processing Systems, pp. 875-886, 2018
A ADDITIONAL EXPERIMENTAL RESULTS
A.1 COMPARISON TO SEPARATELY STATIC AND RUNTIME PRUNING
In this section, we compare our method with two additional baseline methods. One is a variation of method by separately training static pruning and the training runtime pruning. In this method, we start from a pretrained backbone CNN, f()) and Then we add the static DRL agent to prune channels statically by learning the static policy and u,. Finally, we add the runtime DRL agent to prune channels dynamically, by fixing the static DLR agent and u, and updating the runtime DLR agent and f() only. Another method is combine state-of-the-arr static and runtime pruning methods. We start from pretrain backbone CNN, and then prune channels with the static pruning method FPGM (He et al. 2019),, and finally prune channels with the runtime method FBS (Gao et al. 2019). The experimental results are shown in Table4
Method
Baseline acc.
Acc.
Aacc.
ours (runtime only R, = 1)
92.07
91.333
-0.737
ours (R, 0.5) ours (separate static and runtime)
92.07
91.066
-1.004
92.07
90.965
-1.0105
FPGM+FBS
92.07
90.456
-1.614
Table Comparison to methods with separately static and runtime pruning on Cifar-10 sparsity 0.5.
11

---

Under review as a conference paper at ICLR 2020
A.2 STORAGE/ACCURACY TRADE-OFF
Besides Fig. to further illustrate the trade-off between storage and accuracy, we show additional results in Fig.
23.00
30.00
89.75
89.50
89:25-
89:00
1800000
1600000
24000co
1200000
2000000
800003
500000
Maram
Figure 4: Trade-off between runtime pruning and statie pruning sparsity 0.45, X-axis is stor- age(number of parameters), Y-axis is accuracy.
B PSEUDOCODE OF TRAINING PROCESS
Algorithm 1: Training process
INPUT: pretrained backbone CNN, computation budget Beoms storage budget Bparam
OUTPUT: backbone CNN, importance predictor f()), static pruning importance runtime and static DRL agents
Add runtime importance predictor f(-) and static pruning importance us.
U f 1;
d. < 1; d, 1:
while not converge do fix us update f(-) and backbone CNN;
end
while not converge
update u,, f(-) and backbone CNN; end
add runtime DRL agent and static DRL agent to predict actions of and at, and alternate d, and d,
at each layer t;
while not converge do for i - 1 to N₁ do
Forward entired model;
Compute rewards I2 and Rt using budget Boom and B Bparam ;
Fix u. f(.) and backbone CNN, update runtime and static DRL agents by PPO loss;
end for
<- 1 to N₂ do Fix runtime and static DRL agents, update us, f(-) and backbone CNN by cross entropy
loss
end
end
12

---

Under review as a conference paper at ICLR 2020
300
TDIn SccuSrcy
19sl accuuity
Acsuuicy
120
spech
Figure 5: Training curve of our method on CIFAR-10 at sparsity 0.5 with R, 0.5.
C TRAINING CURVE
We show the training curve of our method in Fig. 5 We train our method on CIFAR-10 at sparsity 0.5 with R, 0.5. The figure includes the accuracy curves evaluated on training set and test set, showing that our methods stable during training.
13


Reviews:
[{"review_id": "rJg3jfxlcH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to learn static and dynamic channel pruning policies for convolutional neural networks. Static pruning depends only on the training dataset and is computed once before the model is deployed. Dynamic pruning is input-dependent. The policies are obtained with deep reinforcement learning on the training dataset using a combination of the loss function and storage/computation resource budgets as a reward signal. The key novelty in this paper is to combine static and dynamic pruning which can obtain the benefits from both worlds. Experimentally, the learned pruning policies are competitive with recent dynamic pruning approaches on CIFAR-10 and ILSVRC2012, in terms of both final test accuracy and number of parameters/inference time.\n\nOverall, I do like the idea of combining static and dynamic pruning, the DRL approach is reasonable and it seems to do well in practice. However, there are some key issues that must be addressed by the authors. In summary, these are:\n\n1- Additional baseline methods: why not compare against a simple combination of the best published static pruning method, and one of the best published dynamic pruning methods (e.g. FBS or RNP)? I'd like to understand what the added value of jointly learning the static and dynamic pruning policies is. If a simple combination of existing static and dynamic methods works well, you will need to justify the need for the more complicated DRL approach you propose in this paper.\n\n2- Writing: it has to be substantially improved; please see the sections Writing and Minor below.\n\n3- Training cost: how stable is the training process described in Section 4.1? Does it much longer to train compared to pure dynamic pruning methods? \n\nRelated work:\n- Discuss connections to routing networks which adaptively route layer outputs to the next layer's modules: Rosenbaum, Clemens, Tim Klinger, and Matthew Riemer. \"Routing networks: Adaptive selection of non-linear functions for multi-task learning.\" arXiv preprint arXiv:1711.01239 (2017).\n\nWriting:\n- Sections 3.3 up to Section 5 need lots of rewriting; I suggest some changes in \"Minor\" below but please make a full pass as the paper is difficult to read at the moment.\n- Section 4.1 is extremely difficult to read and so I don't really understand how you train the pruning policies. Please improve the writing and summarize the process in pseudocode or illustrate it. Also, I believe 4.1 should be a section of 3 rather than in experiments. It is extremely important to understand how the network+DRL agents are trained!\n\nClarification:\n- Number of pruned channels, runtime vs static: \\ceil{d_r C} vs (C - \\ceil{d_s C}); why are these different in form? Seems like the static formula prunes \\ceil{(1-d_s) C}. Why is that?\n- Tables 1-2: what is the \"Baseline acc.\" and why is it different for each method? Isn't this the accuracy of the same network before any pruning?\n\nMinor:\n- Title is too long: particularly the expression \"DYNAMIC FLEXIBLE RUNTIME CHANNEL\". Perhaps you can think of shorter titles.\n- \"We consider a standard form of reinforcement learning an agent\" --> \"We consider a standard form of reinforcement learning: an agent\"\n- \"are the the number of channels\" --> \"are the number of channels\"\n- \"output feature F_out u_r is predicted\" --> \"output feature F_out, u_r, is predicted\"\n- \"many existed dynamic\" --> \"many existing dynamic\"\n- \"Since C_in is various among layers\" --> \"Since C_in varies among layers\"\n- \"To avoid over-prune the filters and crashed in training, we set a minimum sparsity ratio +α, then the action space change to a_t^r ∈ (+α, 1].\" --> \"To avoid over-pruning the filters and crashing the training, we set a minimum sparsity ratio +α, such that the action space becomes a_t^r ∈ (+α, 1].\"\n- \"The reward function is proposed to consider both of network accuracy and computation budget.\" --> \"The reward function is considers both the network accuracy and computation budget.\"\n- \"and it may be various in scale\" --> \"and it may vary in scale\"\n- \"large at begin of training and small near converge\" --> \"large at the beginning of training and small near convergence\"\n- \"which filters can be to prune permanently\" --> \"which filters can be pruned permanently\"\n- \"on two popular dataset\" --> \"on two popular datasets\"\n- \"4.1 IMPLEMENT DETAILS\" --> 4.1 IMPLEMENTATION DETAILS\n- \"For CNN fintuning\" --> \"For CNN finetuning\"\n- \"Noted that for fair\" --> \"Note that for fair\"\n- Tables 1-2: \"Compare to state-of-the-art\" --> \"Comparison to state-of-the-art\"\n- \"Our methods has\" --> \"Our method has\"", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "S1eQmrWAqr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work introduces a Reinforcement Learning based framework that simultaneously learns both a static and dynamic pruning strategy. The combination allows the static pruner to decrease the required storage while the dynamic pruning can optimize the required compute using input-dependent pruned weights. The RL agent can dynamically learn the optimal sparsity distribution among the different layer of the networks while staying under a resource constraint as opposed to other methods which often enforce a layer level sparsity ratio. It demonstrates the efficacy of the algorithm on CIFAR10 and ILSVRC2012 and showed the effect of the tradeoff between static and dynamic pruning.\n\nOverall I believe this paper is a borderline accept. It proposes a unified framework to manage the trade-off between static pruning to decrease storage requirements and network flexibility for dynamic pruning to decrease runtime costs. The empirical results demonstrate the capability of the framework but would benefit from some clarification and additional ablations.\n\nPros:\nProposed an RL formulation of a unified framework for static and dynamic channel pruning.\n\nThe empirical results demonstrate the ability of the model to achieve high accuracy while sparsifying the compute and balancing storage consumption and accuracy. Strong results are shown for CIFAR10 and ILSVRC2012.\n\nDemonstrated a tradeoff between static pruning and runtime-pruning through ablation of R_r.\n\nCons:\nIt is unclear how speed-up calculated. Is it wall clock and benchmarked on what device? What was the cost of running the RL agent during runtime?\nIf you are computing MACs, that should be reported as such unless a strong correlation can be proven with the particular pruning scheme. MAC reduction does not translate directly to speedup in hardware.\n\nIt could be clearer if the ablation of R_r also demonstrated a storage/accuracy tradeoff.\n \nIt is unclear if similar results may be achieved by first running a static pruning and then separately training a dynamic pruning algorithm on the already statically pruned network?\nIt might benefit from an ablation study with and without simultaneously training static pruning?\n\nSome of the algorithmic details could benefit from some clarification.\nIn section 3.2, it is unclear to me the effect of R_r during training. It seems that the agent could learn to over select channels to prune to adapt to R_r. Section 3.4 seems to lack inclusion of R_r in the number of statically pruned values. The treatment of M_0 in section 3.2 separate from  M_r and M_s seems to make the number of statically pruned filters during training depend on the layers chosen by dynamic pruning. It seems like it may cause differing sparsity between training and inference time.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "H1lVKaN15B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an offline and online pruning method for CNNs where RL is used for tuning the sparsity.\n\nPruning methods are important for real-time applications of CNNs on low resourced devices, so the paper addresses a genuine need. However, the paper is poorly written and the experimental results are not as convincing.\n\n1) Majority of the description of the models and architecture is written in text and is very difficult to parse, while this could've been avoided by usage of mathematical notations for operations. This is specifically evident for early parts of sec 3 which makes parsing and reading very difficult.\n\n2) It's not clear why the baseline performances in the experimental section are different across different methods. If I understand correctly, all the original non-pruned models should be the same for this experiment to make sense. This is specially important for Tab 2 where the closest competing algorithms (FBS and CGNN). Due to using different baselines, the numbers are not comparable.\n\n3) One thing that is not clear in the text is the computation taken by the runtime pruner architecture. Are these rolled in the estimations on the speedup?", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Main content: Proposes a deep RL unified framework to manage the trade-off between static pruning to decrease storage requirements and network flexibility for dynamic pruning to decrease runtime costs
Summary of discussion:
reviewer1: Reviewer likes the proposed DRL approach, but writing and algorithmic details are lacking
reviewer2: Pruning methods are certainly imortant, but there are details missing wrt the algorithm in the paper. 
reviewer3: Presents a novel RL algorithm, showing good results on CIFAR10 and ISLVRC2012. Algorithmic details and parameters are not clearly explained. 
Recommendation: All reviewers liked the work but the writing/algorithmic details are lacking. I recommend Reject. 
Decision: Reject