Title: STORAGE EFFICIENT AND DYNAMIC FLEXIBLE RUN-TIME CHANNEL PRUNING VIA DEEP REINFORCEMENT LEARNING

Abstract: In this paper, we propose a deep reinforcement learning (DRL) based framework to efficiently perform runtime channel pruning on convolutional neural networks (CNNs). Our DRL-based framework aims to learn a pruning strategy to determine how many and which channels to be pruned in each convolutional layer, depending on each specific input instance at runtime. The learned policy optimizes the performance of the network by restricting the computational resource on layers under an overall computation budget. Furthermore, unlike other runtime pruning methods which require to store all channels parameters for inference, our framework can reduce parameters storage consumption for deployment by introducing a static pruning component. Comparison experimental results with existing runtime and static pruning methods on state-of-the-art CNNs demonstrate that our proposed framework is able to provide a tradeoff between dynamic flexibility and storage efficiency in runtime channel pruning.

1. INTRODUCTION:

In recent years, convolutional neural networks (CNNs) have been proven to be effective in a wide range of computer vision tasks, such as image classification (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015; He et al., 2016) , objection detection (He et al., 2017; Zhou et al., 2019; Law & Deng, 2018) , segmentation (He et al., 2017; Zhu et al., 2019) . Therefore, nowadays, many computervision-based systems, such as automatic-driving cars, security surveillance cameras, and robotics, are built on the power of CNNs. However, since most state-of-the-art CNNs require expensive computation power for inference and huge storage space to store large amount of parameters, the limitation of energy, computation and storage on mobile or edge devices has become the major bottleneck on real-world deployments of CNNs. Existing studies have been focused on speeding up the execution of CNNs for inference on edge devices by model compression using matrix decomposition (Denil et al., 2013; Masana et al., 2017) , network quantization (Courbariaux et al., 2016) , network pruning (Dong et al., 2017) , etc. Among these approaches, channel pruning has shown promising performance (He et al., 2017; Luo et al., 2017; Zhuang et al., 2018; Peng et al., 2019) . Specifically, channel pruning discards an entire input or output channel and keep the rest of the model with structures.

Most channel pruning approaches can be categorized into two types: runtime approaches and static approaches. Static channel pruning approaches aim to design a measurement to evaluate the importance of each channel over the whole training dataset and remove the least important channels to minimize the loss of performance after pruning. By permanently pruning a number of channels, the computation and storage cost of CNNs can be dramatically reduced when being deployed, and the inference execution can be accelerated consequently. Runtime channel pruning approaches have been recently proposed to achieve dynamic channel pruning on each specific instance (Gao et al., 2019; Luo & Wu, 2018) . To be specific, the goal of runtime approaches aims to evaluate the channel importance at runtime, which is assumed to be different on different input instances. By pruning channels dynamically, different pruned structures can be considered as different routing of data stream inside CNNs. This kind of approaches is able to significantly improve the representation capability of CNNs, and thus achieve better performance in terms of prediction accuracy compared with static approaches. However, previous runtime approaches trade storage cost off dynamic flex-ibility of pruning. To achieve dynamic pruning on different specific instances, all parameters of kernels are required to be stored (or even more parameters are introduced). This makes runtime approaches not applicable on resource-limited edge devices. Moreover, most of previous runtime approaches only evaluate the importance among channels in each single layer independently, without considering the difference in efficiency among layers.

In this paper, to address the aforementioned issues of runtime channel pruning approaches, we propose a deep reinforcement learning (DRL) based pruning framework. Basically, we aim to apply DRL to prune CNNs by maximizing received rewards, which are designed to satisfy the overall budget constraints along side with network's training accuracy. Note that automatic channel pruning by DRL is a difficult task because the action space is usually very huge. Specifically, the discrete action space for the DRL agent is as large as the number of channels at each layer, and the action spaces may vary among layers since there are different numbers of channels in different layers. To facilitate pruning CNNs by DRL, for each layer, we first design a novel prediction component to estimate the importance of channels, and then develop a DRL-based component to learn the sparsity ratio of the layer, i.e., how many channels should be pruned.

More specifically, different from previous runtime channel pruning approaches, which only learn runtime importance of each channel, we propose to learn both runtime importance and additionally static importance for each channel. While runtime importance maintains the saliency of specific channels for each given specific input, the static importance captures the overall saliency of the corresponding channel among the whole dataset. According to each type of the channel importance, we further design different DRL agents (i.e., a runtime agent and a static agent) to learn a sparsity ratio in a layer-wise manner. The sparsity ratio learned by the runtime agent together with the estimated runtime importance of channels are used to generate runtime pruning structures, while the sparsity ratio learned by the static agent together with the estimated static importance of channels are used to generate static (permanent) pruning structures. By considering both the pruning structures, our framework is able to provide a trade-off between storage efficiency and dynamic flexibility for runtime channel pruning.

In summary, our contributions are 2-fold. First, we propose to prune channels by taking both runtime and static information of the environment into consideration. Runtime information endows pruning with flexibility based on different input instances while static information reduces the number of parameters in deployment, leading to storage reduction, which cannot be achieved by conventional runtime pruning approaches. Second, we propose to use DRL to determine sparsity ratios, which is different from the previous pruning approaches which manually set sparsity ratios. Extensive experiments demonstrate the effectiveness of our method.

2. RELATED WORK AND PRELIMINARY:

2.1 STRUCTURE PRUNING Wen et al. (2016) pioneered structure pruning in deep neural network by imposing the L 2,1 norm in training. Under the same framework, Liu et al. (2017) regarded parameters in batch normalization as channel selection signal, which is minimized to achieve pruning during training. He et al. (2017) formulated channel pruning into a two-step iterative process including LASSO regression based channel selection and least square reconstruction. Luo et al. (2017) formulated channel pruning as minimization of difference of output features, which is solved by greedy selection. Zhuang et al. (2018) further considered early prediction, reconstruction loss and final loss to select importance channels. Overall, structure pruning methods accelerate inference by producing regular and compact model. However, this brought regularness requires preserving more parameters to ensure performance.

2.2. DYNAMIC PRUNING:

Dynamic pruning provides different pruning strategies according to input data. Wang et al. (2018) proposed to reduce computation by skipping layers or channels based on the analysis of input features. Luo & Wu (2018) proposed to use layer input to learn channel importance, which is then binarized for pruning. Gao et al. (2019) applied the same framework while extended features selection in both input and output features. Similarly, Liu & Deng (2018) for runtime inference according to inputs. A gating module is learnt to guide the flow of feature maps. Bolukbasi et al. (2017) learned to choose the components of a deep network to be evaluated for each input adaptively. Early exit is introduced to accelerate computation. Dynamic pruning adaptively takes different actions for different inputs, which is able to accelerate the overall inference time. However, the original high-precision model needs to be stored, together with extra parameters for making specified pruning actions. Rosenbaum et al. (2018) proposed to learn routers to route layers output to different next layers, in order to adjust a network to multi-task learning.

introduced multiple branches:



2.3. DEEP REINFORCEMENT LEARNING IN PRUNING:

Channel selection is on trial using deep reinforcement learning. Lin et al. ( 2017) trained a LSTM model to remember and provide channel pruning strategy for backbone CNN model, which is conducted using reinforcement learning techniques. He et al. (2018) proposed to determine the compression ratio in each layer by training an agent regarding the pruning-retraining process as an environment.

2.4. PRELIMINARY:

Reinforcement Learning We consider a standard setup of reinforcement learning: an agent sequentially takes actions over a sequence of time steps in an environment, in order to maximize the cumulative reward (Sutton & Barto, 1998) . This problem can be formulated as a Markov Decision Process (MDP) of a tuple (S, A, P, R, γ), where S is the state space, A is the action space, P : S × A × S → [0, 1] is transition probabilities, R : S × A → R is the reward function, and γ ∈ [0, 1) is the discount factor. The goal of reinforcement learning is to learn a policy π(a|s) that maximizes the objective of cumulative rewards over finite time steps,

max π T t=0 R(s t , a t ),

where s t ∈ S and a t ∈ A are state and taken action at time step t respectively.

3. DRL-BASED RUNTIME PRUNING FRAMEWORK:

The overview of our proposed framework is presented in Fig. 1 . To prune convolutional layer t, we learn two types of learnable channel importance: runtime channel importance u r ∈ R C×1 and static channel importance u s ∈ R C×1 , where C is the number of channels in layer t. The runtime channel importance u r is generated by a subnetwork importance predictor f (•), which takes the input feature map F in as input, while the static channel importance u s is randomly initialized and updated during training. Both u r and u s indicate the channel importance of the full precision output feature map F out through a convolution layer. Channels are selected to be pruned according to the values of each element in u r and u s , and how many channels to be selected is decided by the sparsity ratios d r and d s , respectively. To learn the sparsity ratios d r and d s , two DRL agents, the runtime agent and the static agent, are introduced, where actions a r t and a s t are defined to set values of d r and d s , respectively. The detail of the two DRL agents are described in Sec. 3.3. Consequently, a trade-off pruner g(•) is performed to balance the runtime and static pruning results, and output a decision mask M of binary values (1/0) to indicate which channels to be pruned (1: pruned, 0: preserved), as well as a unified channel importance vector u ∈ R C×1 as follows,

[M, u] = g(u r , u s , d r , d s ).

(1)

The final output after pruning is constructed by multiplying the full precision output feature map F out , by 1 -M and u as,

Fout = F out ⊗ (1 -M) ⊗ u,

(2) where ⊗ is the broadcast element-wise multiplier, and 1 is the matrix of the same size as M with all the elements being 1. In the following, we introduce how to learn the runtime channel importance vector u r and the static channel importance vector u s in Sec. 3.1, how to construct the trade-off pruner g(•) in Sec. 3.2, and how to design the two DRL agents in Sec. 3.3.

3.1. LEARNABLE CHANNEL IMPORTANCE:

We consider that a convolutional layer takes input of feature map F in ∈ R Cin×Hin×Win and generates an output feature map F out ∈ R Cout×Hout×Wout , where C * , H * and W * are the number of channels, width and height of the feature map F * , respectively. Each element of the channel importance vectors u r ∈ R Cout and u s ∈ R Cout represents the importance value of the corresponding channel, respectively. In the following, we drop the subscript out for simplicity in presentation.

3.1.1. RUNTIME CHANNEL IMPORTANCE:

As mentioned above, the runtime channel importance u r of output feature F out is predicted by a importance predictor f (•), which takes F in as input. Therefore, u r can be considered as a function of F in , whose values vary over different input instances. In this paper, we design a subnetwork to approximate f (•), which is expected to be of a small size and computationally efficient. Similar to many existing dynamic network pruning methods (Gao et al., 2019; Hu et al., 2018; Luo & Wu, 2018) , we use global pooling layer as the first layer in f (•), because global pooling is computationally efficient and it can reduce the dimension of F in dramatically. We then feed the output of global pooling into a fully-connected layer without any activation function. The output of fully-connected layer is the runtime channel importance vector u r .

Which channels to be preserved / pruned at runtime are determined according to the values of u r . We denote by M r ∈ {0, 1} C a mask for pruning, where if the value is 0, then the corresponding channel is preserved, otherwise pruned. For now, suppose a sparsity ratio d r for runtime pruning has already been generated via the dynamic DRL agent, which will be introduced in Sec. 3.3. We then prune (C -d r C ) channels with the smallest importance values in u r . Accordingly, the value of an element in M r is set to be 1 if the corresponding channel is pruned, otherwise 0.

3.1.2. STATIC CHANNEL IMPORTANCE:

The static channel importance vector u s is to capture the global information for pruning, and thus is learned from the whole dataset. It is randomly generated and learned through backpropagation. Similar to runtime channel pruning, given a sparsity ratio d s learned by the static DRL agent, (Cd s C ) channels with smallest importance values in u s are pruned, and a mask M s ∈ {0, 1} C is generated to indicate the static pruning results.

3.2. TRADE-OFF PRUNER:

With the runtime and the static pruning decisions, M r and M s , we now propose a trade-off pruner to generate a unified channel pruning decision. The main idea behind the trade-off pruner is to 1) prune those channels which are agreed to be pruned by both decisions, and 2) prune a portion of the rest channels by weighted votes from both decisions.

To be specific, we define the mask representing channels pruned by both decisions as

EQUATION

where ∧ is element-wise logical AND and 1/0 in mask represents logical true or false. The channels indicated to be pruned by M o (i.e., the corresponding values are 1) are pruned in final. The channels which are determined to be pruned by M r but not by M s can be represented by a new mask M r = M r -M o . Similarly, the channels which are determined to be pruned by M r but not by M s can be represented by another new mask M s = M s -M o .

To control the trade-off between M r and M s , we define a rate R r denoting how much we trust the pruning decision made by M r , while 1 -R r is for M s . That means the channels selected by M r will be finally pruned with the rate R r . Specifically, the number of channels which are selected by M r and finally will be pruned is

EQUATION

where 1 M r returns the number of channels selected by M r . We then select the first C r -smallest important channels which are recommended to be pruned by M r to form a mask M r . Similarly, for static pruning, we select the first C s -smallest important channels which are recommended to be pruned by M s to form another mask M s , where

C s = (1 -R r )(1 M s ) .

The final trade-off pruning mask is defined as

EQUATION

Moreover, in this work, the unified channel importance is simply defined as follows,

EQUATION

With the trade-off pruning mask M and the unified channel importance u, the pruned output feature Fout can be generated by Eq. 2.

3.3. DRL BASED PRUNING:

In this section, we present how to formulate the problems of learning the ratios d s and d t for static pruning and runtime pruning, as a MDP, and solve it via DRL, respectively.

3.3.1. DRL FOR RUNTIME PRUNING:

In the MDP for runtime pruning, we consider the t-th layer of the network as the t-th timestamp.

The details of the MDP are listed as follows.

State Given an input feature map F in of layer t, we pass it to a global pooling layer to reduce its dimension to R Cin , where C in is the number of input channel of layer t. Since C in varies among layers, we feed the output of global pooling to a layer-dependent encoder to project it to a fix-length vector s r t , which is considered as as a state representation of DRL in the context of runtime pruning.

Action The action a r t is defined as the sparsity ratio at layer t, alternating d r in runtime pruning mentioned in Section 3.1.1. Existing DRL-base pruning method RNP (Lin et al., 2017) uses a unified discrete actions space with k actions which are too coarse to achieve high accuracy. However, finegrained discrete action space as large as number of channels suffers from exploration difficulty. Therefore, instead of using discrete action spaces, we propose a continuous action space with action a r t ∈ (0, 1]. To avoid over-pruning the filters and crashing in training, we set a minimum sparsity ratio +α such that a r t ∈ (+α, 1].

Reward The reward function is proposed to consider both network accuracy and computation budget. We define the accuracy relative reward based on the loss of pruned backbone network,

EQUATION

where L CN N is the loss in CNN, and it may vary in scale among different training stage, i.e. large at beginning of training and small near convergence. To avoid the instability brought by the reward scale, R r acc is normalized by a moving average,

EQUATION

EQUATION

where β b is the moving average at the b-th training batch and λ is the moving weight.

To force computation of the pruned network under a given computation budget, we define a exponential reward function of budget regarding reward R r bud :

EQUATION

)

where B com is the computation consumption, which is calculated based on the current of pruned strategy, and B com is the given computation budget constraint. Finally we sum up the two rewards to form sparse rewards, with being non-zero at terminated step T and zeros at other time step t < T ,

EQUATION

)

Actor-Critic Agent To solve the continuous action space problem, we choose a commonly used actor-critic agent with a Gaussian policy. Actor-critic agent consists of two components: 1) actor outputs the mean and variance to form a Gaussian policy where the continuous action are sampled from; 2) critic outputs a scalar predicting the future discounted accumulated reward and assists the policy training. Actor network and Critic network share one-layer RNN which takes state s r t as input. The output of RNN is fed into actor specific network constructed by two branches of fullyconnected layers, leading to the mean and variance of the Gaussian policy. The action is sampled for the Gaussian distribution outputed by the actor:

EQUATION

where µ(s r t ; θ r ) and σ(s r t ; θ r ) is the mean and variance outputed from actor network. The Critic specific network has one fully-connected layer after the shared RNN, and outputs the predictive value V (s r t ; θ r ). To optimize the actor-critic agent, Proximal Policy Optimization (PPO) (Schulman et al., 2017) is used. Note that we relax the action a r t to (-∞, +∞) in PPO, and use truncate function to clip a r t in (+α, 1] when perform pruning.

Besides, an additional regularizer is introduced to restrict the relaxed a r t staying in range (+α, 1],

L a = 1 2 ||a r tmax(min(a r t , 1), +α)|| 2 2 .

(13)

3.3.2. DRL FOR STATIC PRUNING:

Similar to runtime pruning, the MDP in static pruning is also formulated layer-by-layer. The difference against runtime pruning is the definition of state and reward.

State The state s s t in static pruning is defined as the full shape of F out , and does not depend on F out and the current input data. Action Action a s t is sampled from actor's outputed Gaussian policy, and it is to alternate the sparsity d s in static pruning mentioned in Section 3.1.2.

Reward The reward function takes both network accuracy and parameters budget into consideration. The accuracy relative is defined as the same as that in runtime pruning:

EQUATION

To reduce the number of parameters of network to satisfy the parameters storage budget, the parameters relative reward is defined in an exponential form as,

EQUATION

where B param is the number of preserved parameters after static pruning and B param is the parameters storage budget.

Method:

Baseline acc. Acc. ∆acc. Speed-up #Params FBS (Gao et al., 2019) 91.37 89.88 -1.49 3.93× 1.11× RNP (Lin et al., 2017) 92.07 84.93 -7.14 3.56× 1.00× ours (runtime only R r = 1) 92.07 91.333 -0.737 3.92× 1.31× ours (R r = 0.5) 92.07 91.066 -1.004 3.92× 0.78× Actor-Critic Agent This agent is similar to the one in runtime pruning. It has the same architecture as runtime pruning but differs in introducing a fully-connected layer as the encoder before RNN. This agent is also optimized by PPO.

3.4. INFERENCE:

In inference, the static agent is not required any more because the static pruning strategy does not depend on individual input data points but the full shape of F in . Therefore, the output action a s t is fixed to each layer t. With the action a s t and the rate R r , we can decide which filters can be pruned permanently. Specifically, channels with ((1 -a s t )(1 -R r ))-smallest static importance values are pruned permanently.

4. EXPERIMENT:

We evaluate our DRL pruning framework on two popular datasets: CIFAR-10 ( Krizhevsky, 2009) and ImageNet ILSVRC2012 (Russakovsky et al., 2015) , to show the advantage over other channel pruning methods. We analyze the effect of hyper-parameters and different sparsity settings on CIFAR-10. For CIFAR-10, we use M-CifarNet (Zhao et al., 2018) as the backbone CNN. On Ima-geNet ILSVRC2012, ResNet-18 is used as the backbone CNN.

4.1. IMPLEMENTATION DETAILS:

We start with a pretrained backbone CNN. Firstly we finetune the backbone CNN and train runtime importance predictor jointly, with sparsity d r = 1 and fixed all static pruning importance u s to 1. Then we remove the restriction on the static pruning importance u s , and train static pruning importance, the backbone CNN and the runtime importance predictor, with sparsity d s = 1 and runtime pruning sparsity fixed as d r = 1. After finetuning, we use the DRL agents to predict the sparsity given computation and storage constraints. The DRL agents and the CNN with runtime/static importance are trained in alternating manner: We first fix the CNN as well as runtime/static importance and train two DRL agents, regarding the CNN as environments. Then we fix two agents and finetune the CNN and runtime/static importance. We repeat these two steps until convergence is achieved. We use Adam optimizer for both DRL agent and CNN, and set learning rate at 10 -6 for the DRL agents. For CNN finetuning and runtime/static importance training, the learning rate is set to 10 -3 on CIFAR-10. On ImageNet ILSVRC2012, the learning rate starts from 10 -3 and is divided by 10 after 15 millions iterations. Top-1 acc.

∆ top-1 acc.

Baseline top-5 acc.

Top-5 acc.

∆ 

4.2. EXPERIMENTAL RESULTS ON CIFAR-10:

We compare our proposed method with the following state-of-the-art runtime pruning methods: FBS (Gao et al., 2019) , RNP (Lin et al., 2017) on CIFAR-10. The comparison results at sparsity 0.5 and 0.7 are shown in Table 1 and Table 2 respectively. Note that for fair comparison with other methods, the computation and storage budget constraints in our method is calculated according to the sparsity of other methods. Under these constraints, our method does not necessarily lead to the same sparsity as other methods in each layer. RNP cannot set exact sparsity ratio. Instead, its average sparsity ratio is accessible only during testing, which is 0.537 in Table 1 . The result of FBS is reproduced using the released code1 . The column #Params represents the number of parameters compared to the backbone CNN.

Table 1 shows that our method outperforms other state-of-the-art methods, achieving highest accuracy at an overall sparsity ratio of 0.5. Our method has very close computation speed-up compared to FBS, but outperforms FBS around 0.48% to 0.76%. When the runtime pruning strategy is solely considered by setting R r = 1, our method surpasses other comparison methods, indicating that our DRL-based framework improves the performance of channel runtime pruning. By balancing runtime and static pruning via setting R r = 0.5, our method reduces the number of the overall stored parameters and achieves lower accuracy drop than other methods. Table 2 shows that our method outperforms FBS at sparsity of 0.7. When R r = 0.5, our method achieves better performance than the baseline CNN with 2× speed-up and contains less parameters.

We also study the relation between R r and network compactness in our framework. Fig. 2 demonstrates the impact of R r when sparsity is 0.45. The hyper-parameter R r determines how much we trust about runtime pruning. With R r close to 1, the accuracy becomes higher due to the more dynamic network flexibility but the space of the parameters storage also increases. When R r diminishes, the network accuracy decreases but the parameter storage is reduced.

Fig. 3 shows the performance of various sparsity ratios in our method. Again, our method does not prune with one single sparsity ratio for all layers, but uses the sparsity ratio to calculate computation and storage constraints, with which the sparsity ratio is learned for each layer. Fig. 3 demonstrates that our method holds the accuracy when sparsity is larger than about 0.5, which corresponds to about 4× computational acceleration.

4.3. EXPERIMENTAL RESULTS ON IMAGENET ILSVRC2012:

We compare our method with state-of-the-art channel pruning methods on ImageNet ILSVRC2012 as shown in Table 3 . In this experiment, we use ResNet-18 as the backbone CNN. Among the stateof-the-art pruning methods for comparison, FBS (Gao et al., 2019) and CGNN (Hua et al., 2018) are runtime pruning methods. The overall sparsity ratio of our method is 0.7, which is under the same setting of FBS. Our method with R r = 0.5 achieves the smallest top-1 accuracy drop compared with other methods, and also achieves the highest top-1 accuracy after pruning. Overall, our proposed method achieves comparable or better performance compared to other methods with more acceleration. Our method has very close MACs FBS, while the number of preserved parameters is reduced to 81.2% of the baseline.

5. CONCLUSION:

In this paper, we present a deep reinforcement learning based framework for deep neural network channel pruning in both runtime and static sheme. Specially, channels are pruned according to input feature as runtime pruning, and based on entire training dataset as static pruning, with 2 reinforcement agents to determine the corresponding sparsity. Our method combines the merits of runtime and static pruning, and provides trade-off between storage and dynamic flexibility. Extensive experiments demonstrate the effectiveness of our proposed method.

A ADDITIONAL EXPERIMENTAL RESULTS:

A.1 COMPARISON TO SEPARATELY STATIC AND RUNTIME PRUNING

In this section, we compare our method with two additional baseline methods. One is a variation of our method by separately training static pruning and the training runtime pruning. In this method, we start from a pretrained backbone CNN, f (•) and u s . Then we add the static DRL agent to prune channels statically by learning the static policy and u s . Finally, we add the runtime DRL agent to prune channels dynamically, by fixing the static DLR agent and u s , and updating the runtime DLR agent and f (•) only. Another method is to combine state-of-the-art static and runtime pruning methods. We start from a pretrain backbone CNN, and then prune channels with the static pruning method FPGM (He et al., 2019) , and finally prune channels with the runtime method FBS (Gao et al., 2019) 

C TRAINING CURVE:

We show the training curve of our method in Fig. 5 . We train our method on CIFAR-10 at sparsity 0.5 with R r = 0.5. The figure includes the accuracy curves evaluated on training set and test set, showing that our methods is stable during training.

:

https://github.com/deep-fry/mayo

Figures and Tables:

Figure

Figure 2: Trade-off between runtime pruning and static pruning at sparsity 0.45. X-axis is the rate R r

Figure 3: Comparison accuracy drop for M-CifarNet on CIFAR-10 with computational budget.

Figure 5: Training curve of our method on CIFAR-10 at sparsity 0.5 with R r = 0.5.

Comparison to state-of-the-art runtime pruning methods on CIFAR-10 at sparsity 0.5. Speed-up is calculated on MACs.

Table content: <table><tr><td>Method</td><td colspan="2">Baseline acc. Acc.</td><td>∆acc.</td><td colspan="2">Speed-up #Params</td></tr><tr><td colspan="2">FBS (Gao et al., 2019) 91.37</td><td>91.23</td><td>-0.14</td><td>2×</td><td>1.11×</td></tr><tr><td>ours (R r = 1.0)</td><td>92.07</td><td colspan="3">93.178 +1.108 1.99×</td><td>1.31×</td></tr><tr><td>ours (R r = 0.5)</td><td>92.07</td><td colspan="3">92.502 +0.432 1.99×</td><td>0.97×</td></tr></table>

Comparison to state-of-the-art runtime pruning methods on CIFAR-10 at sparsity 0.7. Speed-up is calculated on MACs.

Table content: <table/>

Comparison with the state-of-the-art channel pruning ResNet-18 on ImageNet. Speed-up is calculated on MACs.

Table content: <table/>

. The experimental results are shown in Table4. Comparison to methods with separately static and runtime pruning on Cifar-10 at sparsity 0.5.

Table content: <table><tr><td>Method</td><td colspan="2">Baseline acc. Acc.</td><td>∆acc.</td></tr><tr><td>ours (runtime only R r = 1)</td><td>92.07</td><td colspan="2">91.333 -0.737</td></tr><tr><td>ours (R r = 0.5)</td><td>92.07</td><td colspan="2">91.066 -1.004</td></tr><tr><td colspan="2">ours (separate static and runtime) 92.07</td><td colspan="2">90.965 -1.0105</td></tr><tr><td>FPGM+FBS</td><td>92.07</td><td colspan="2">90.456 -1.614</td></tr></table>

Peer review (iclr2020):
This paper proposes to learn static and dynamic channel pruning policies for convolutional neural networks. Static pruning depends only on the training dataset and is computed once before the model is deployed. Dynamic pruning is input-dependent. The policies are obtained with deep reinforcement learning on the training dataset using a combination of the loss function and storage/computation resource budgets as a reward signal. The key novelty in this paper is to combine static and dynamic pruning which can obtain the benefits from both worlds. Experimentally, the learned pruning policies are competitive with recent dynamic pruning approaches on CIFAR-10 and ILSVRC2012, in terms of both final test accuracy and number of parameters/inference time. Overall, I do like the idea of combining static and dynamic pruning, the DRL approach is reasonable and it seems to do well in practice. However, there are some key issues that must be addressed by the authors. In summary, these are: 1- Additional baseline methods: why not compare against a simple combination of the best published static pruning method, and one of the best published dynamic pruning methods (e.g. FBS or RNP)? I*d like to understand what the added value of jointly learning the static and dynamic pruning policies is. If a simple combination of existing static and dynamic methods works well, you will need to justify the need for the more complicated DRL approach you propose in this paper. 2- Writing: it has to be substantially improved; please see the sections Writing and Minor below. 3- Training cost: how stable is the training process described in Section 4.1? Does it much longer to train compared to pure dynamic pruning methods? Related work: - Discuss connections to routing networks which adaptively route layer outputs to the next layer*s modules: Rosenbaum, Clemens, Tim Klinger, and Matthew Riemer. *Routing networks: Adaptive selection of non-linear functions for multi-task learning.* arXiv preprint arXiv:1711.01239 (2017). Writing: - Sections 3.3 up to Section 5 need lots of rewriting; I suggest some changes in *Minor* below but please make a full pass as the paper is difficult to read at the moment. - Section 4.1 is extremely difficult to read and so I don*t really understand how you train the pruning policies. Please improve the writing and summarize the process in pseudocode or illustrate it. Also, I believe 4.1 should be a section of 3 rather than in experiments. It is extremely important to understand how the network+DRL agents are trained! Clarification: - Number of pruned channels, runtime vs static: ceil{d_r C} vs (C - ceil{d_s C}); why are these different in form? Seems like the static formula prunes ceil{(1-d_s) C}. Why is that? - Tables 1-2: what is the *Baseline acc.* and why is it different for each method? Isn*t this the accuracy of the same network before any pruning? Minor: - Title is too long: particularly the expression *DYNAMIC FLEXIBLE RUNTIME CHANNEL*. Perhaps you can think of shorter titles. - *We consider a standard form of reinforcement learning an agent* --> *We consider a standard form of reinforcement learning: an agent* - *are the the number of channels* --> *are the number of channels* - *output feature F_out u_r is predicted* --> *output feature F_out, u_r, is predicted* - *many existed dynamic* --> *many existing dynamic* - *Since C_in is various among layers* --> *Since C_in varies among layers* - *To avoid over-prune the filters and crashed in training, we set a minimum sparsity ratio +?, then the action space change to a_t^r ? (+?, 1].* --> *To avoid over-pruning the filters and crashing the training, we set a minimum sparsity ratio +?, such that the action space becomes a_t^r ? (+?, 1].* - *The reward function is proposed to consider both of network accuracy and computation budget.* --> *The reward function is considers both the network accuracy and computation budget.* - *and it may be various in scale* --> *and it may vary in scale* - *large at begin of training and small near converge* --> *large at the beginning of training and small near convergence* - *which filters can be to prune permanently* --> *which filters can be pruned permanently* - *on two popular dataset* --> *on two popular datasets* - *4.1 IMPLEMENT DETAILS* --> 4.1 IMPLEMENTATION DETAILS - *For CNN fintuning* --> *For CNN finetuning* - *Noted that for fair* --> *Note that for fair* - Tables 1-2: *Compare to state-of-the-art* --> *Comparison to state-of-the-art* - *Our methods has* --> *Our method has*