Year: 2019

Abstract:
In this paper, we propose a deep reinforcement learning (DRL) based framework to efficiently perform runtime channel pruning on convolutional neural networks (CNNs). Our DRL-based framework aims to learn a pruning strategy to determine how many and which channels to be pruned in each convolutional layer, depending on each specific input instance at runtime. The learned policy optimizes the performance of the network by restricting the computational resource on layers under an overall computation budget. Furthermore, unlike other runtime pruning methods which require to store all channels parameters for inference, our framework can reduce parameters storage consumption for deployment by introducing a static pruning component. Comparison experimental results with existing runtime and static pruning methods on state-of-the-art CNNs demonstrate that our proposed framework is able to provide a tradeoff between dynamic flexibility and storage efficiency in runtime channel pruning.

heading: 1 INTRODUCTION
text: In recent years, convolutional neural networks (CNNs) have been proven to be effective in a wide range of computer vision tasks, such as image classification (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015; He et al., 2016), objection detection (He et al., 2017; Zhou et al., 2019; Law & Deng, 2018), segmentation (He et al., 2017; Zhu et al., 2019). Therefore, nowadays, many computervision-based systems, such as automatic-driving cars, security surveillance cameras, and robotics, are built on the power of CNNs. However, since most state-of-the-art CNNs require expensive computation power for inference and huge storage space to store large amount of parameters, the limitation of energy, computation and storage on mobile or edge devices has become the major bottleneck on real-world deployments of CNNs. Existing studies have been focused on speeding up the execution of CNNs for inference on edge devices by model compression using matrix decomposition (Denil et al., 2013; Masana et al., 2017), network quantization (Courbariaux et al., 2016), network pruning (Dong et al., 2017), etc. Among these approaches, channel pruning has shown promising performance (He et al., 2017; Luo et al., 2017; Zhuang et al., 2018; Peng et al., 2019). Specifically, channel pruning discards an entire input or output channel and keep the rest of the model with structures.
Most channel pruning approaches can be categorized into two types: runtime approaches and static approaches. Static channel pruning approaches aim to design a measurement to evaluate the importance of each channel over the whole training dataset and remove the least important channels to minimize the loss of performance after pruning. By permanently pruning a number of channels, the computation and storage cost of CNNs can be dramatically reduced when being deployed, and the inference execution can be accelerated consequently. Runtime channel pruning approaches have been recently proposed to achieve dynamic channel pruning on each specific instance (Gao et al., 2019; Luo & Wu, 2018). To be specific, the goal of runtime approaches aims to evaluate the channel importance at runtime, which is assumed to be different on different input instances. By pruning channels dynamically, different pruned structures can be considered as different routing of data stream inside CNNs. This kind of approaches is able to significantly improve the representation capability of CNNs, and thus achieve better performance in terms of prediction accuracy compared with static approaches. However, previous runtime approaches trade storage cost off dynamic flex-
ibility of pruning. To achieve dynamic pruning on different specific instances, all parameters of kernels are required to be stored (or even more parameters are introduced). This makes runtime approaches not applicable on resource-limited edge devices. Moreover, most of previous runtime approaches only evaluate the importance among channels in each single layer independently, without considering the difference in efficiency among layers.
In this paper, to address the aforementioned issues of runtime channel pruning approaches, we propose a deep reinforcement learning (DRL) based pruning framework. Basically, we aim to apply DRL to prune CNNs by maximizing received rewards, which are designed to satisfy the overall budget constraints along side with network’s training accuracy. Note that automatic channel pruning by DRL is a difficult task because the action space is usually very huge. Specifically, the discrete action space for the DRL agent is as large as the number of channels at each layer, and the action spaces may vary among layers since there are different numbers of channels in different layers. To facilitate pruning CNNs by DRL, for each layer, we first design a novel prediction component to estimate the importance of channels, and then develop a DRL-based component to learn the sparsity ratio of the layer, i.e., how many channels should be pruned.
More specifically, different from previous runtime channel pruning approaches, which only learn runtime importance of each channel, we propose to learn both runtime importance and additionally static importance for each channel. While runtime importance maintains the saliency of specific channels for each given specific input, the static importance captures the overall saliency of the corresponding channel among the whole dataset. According to each type of the channel importance, we further design different DRL agents (i.e., a runtime agent and a static agent) to learn a sparsity ratio in a layer-wise manner. The sparsity ratio learned by the runtime agent together with the estimated runtime importance of channels are used to generate runtime pruning structures, while the sparsity ratio learned by the static agent together with the estimated static importance of channels are used to generate static (permanent) pruning structures. By considering both the pruning structures, our framework is able to provide a trade-off between storage efficiency and dynamic flexibility for runtime channel pruning.
In summary, our contributions are 2-fold. First, we propose to prune channels by taking both runtime and static information of the environment into consideration. Runtime information endows pruning with flexibility based on different input instances while static information reduces the number of parameters in deployment, leading to storage reduction, which cannot be achieved by conventional runtime pruning approaches. Second, we propose to use DRL to determine sparsity ratios, which is different from the previous pruning approaches which manually set sparsity ratios. Extensive experiments demonstrate the effectiveness of our method.

heading: 2 RELATED WORK AND PRELIMINARY

heading: 2.1 STRUCTURE PRUNING
text: Wen et al. (2016) pioneered structure pruning in deep neural network by imposing the L2,1 norm in training. Under the same framework, Liu et al. (2017) regarded parameters in batch normalization as channel selection signal, which is minimized to achieve pruning during training. He et al. (2017) formulated channel pruning into a two-step iterative process including LASSO regression based channel selection and least square reconstruction. Luo et al. (2017) formulated channel pruning as minimization of difference of output features, which is solved by greedy selection. Zhuang et al. (2018) further considered early prediction, reconstruction loss and final loss to select importance channels. Overall, structure pruning methods accelerate inference by producing regular and compact model. However, this brought regularness requires preserving more parameters to ensure performance.

heading: 2.2 DYNAMIC PRUNING
text: Dynamic pruning provides different pruning strategies according to input data. Wang et al. (2018) proposed to reduce computation by skipping layers or channels based on the analysis of input features. Luo & Wu (2018) proposed to use layer input to learn channel importance, which is then binarized for pruning. Gao et al. (2019) applied the same framework while extended features selection in both input and output features. Similarly, Liu & Deng (2018) introduced multiple branches
for runtime inference according to inputs. A gating module is learnt to guide the flow of feature maps. Bolukbasi et al. (2017) learned to choose the components of a deep network to be evaluated for each input adaptively. Early exit is introduced to accelerate computation. Dynamic pruning adaptively takes different actions for different inputs, which is able to accelerate the overall inference time. However, the original high-precision model needs to be stored, together with extra parameters for making specified pruning actions. Rosenbaum et al. (2018) proposed to learn routers to route layers output to different next layers, in order to adjust a network to multi-task learning.

heading: 2.3 DEEP REINFORCEMENT LEARNING IN PRUNING
text: Channel selection is on trial using deep reinforcement learning. Lin et al. (2017) trained a LSTM model to remember and provide channel pruning strategy for backbone CNN model, which is conducted using reinforcement learning techniques. He et al. (2018) proposed to determine the compression ratio in each layer by training an agent regarding the pruning-retraining process as an environment.

heading: 2.4 PRELIMINARY
text: Reinforcement Learning We consider a standard setup of reinforcement learning: an agent sequentially takes actions over a sequence of time steps in an environment, in order to maximize the cumulative reward (Sutton & Barto, 1998). This problem can be formulated as a Markov Decision Process (MDP) of a tuple (S , A, P , R, γ), where S is the state space, A is the action space, P : S × A × S → [0, 1] is transition probabilities, R : S × A → R is the reward function, and γ ∈ [0, 1) is the discount factor. The goal of reinforcement learning is to learn a policy π(a|s) that maximizes the objective of cumulative rewards over finite time steps,
max π T∑ t=0 R(st, at),
where st ∈ S and at ∈ A are state and taken action at time step t respectively.

heading: 3 DRL-BASED RUNTIME PRUNING FRAMEWORK
text: The overview of our proposed framework is presented in Fig. 1. To prune convolutional layer t, we learn two types of learnable channel importance: runtime channel importance ur ∈ RC×1 and static channel importance us ∈ RC×1, where C is the number of channels in layer t. The runtime channel importance ur is generated by a subnetwork importance predictor f(·), which takes the input feature map Fin as input, while the static channel importance us is randomly initialized and updated during training. Both ur and us indicate the channel importance of the full precision output feature map Fout through a convolution layer. Channels are selected to be pruned according to the values of each element in ur and us, and how many channels to be selected is decided by the sparsity ratios dr and ds, respectively. To learn the sparsity ratios dr and ds, two DRL agents, the runtime agent and the static agent, are introduced, where actions art and a s t are defined to set values of dr and ds, respectively. The detail of the two DRL agents are described in Sec. 3.3. Consequently,
a trade-off pruner g(·) is performed to balance the runtime and static pruning results, and output a decision mask M of binary values (1/0) to indicate which channels to be pruned (1: pruned, 0: preserved), as well as a unified channel importance vector u ∈ RC×1 as follows,
[M,u] = g(ur,us, dr, ds). (1)
The final output after pruning is constructed by multiplying the full precision output feature map Fout, by 1−M and u as, F̂out = Fout ⊗ (1−M)⊗ u, (2) where⊗ is the broadcast element-wise multiplier, and 1 is the matrix of the same size as M with all the elements being 1. In the following, we introduce how to learn the runtime channel importance vector ur and the static channel importance vector us in Sec. 3.1, how to construct the trade-off pruner g(·) in Sec. 3.2, and how to design the two DRL agents in Sec. 3.3.

heading: 3.1 LEARNABLE CHANNEL IMPORTANCE
text: We consider that a convolutional layer takes input of feature map Fin ∈ RCin×Hin×Win and generates an output feature map Fout ∈ RCout×Hout×Wout , where C∗, H∗ and W∗ are the number of channels, width and height of the feature map F∗, respectively. Each element of the channel importance vectors ur ∈ RCout and us ∈ RCout represents the importance value of the corresponding channel, respectively. In the following, we drop the subscript out for simplicity in presentation.

heading: 3.1.1 RUNTIME CHANNEL IMPORTANCE
text: As mentioned above, the runtime channel importance ur of output feature Fout is predicted by a importance predictor f(·), which takes Fin as input. Therefore, ur can be considered as a function of Fin, whose values vary over different input instances. In this paper, we design a subnetwork to approximate f(·), which is expected to be of a small size and computationally efficient. Similar to many existing dynamic network pruning methods (Gao et al., 2019; Hu et al., 2018; Luo & Wu, 2018), we use global pooling layer as the first layer in f(·), because global pooling is computationally efficient and it can reduce the dimension of Fin dramatically. We then feed the output of global pooling into a fully-connected layer without any activation function. The output of fully-connected layer is the runtime channel importance vector ur.
Which channels to be preserved / pruned at runtime are determined according to the values of ur. We denote by Mr ∈ {0, 1}C a mask for pruning, where if the value is 0, then the corresponding channel is preserved, otherwise pruned. For now, suppose a sparsity ratio dr for runtime pruning has already been generated via the dynamic DRL agent, which will be introduced in Sec. 3.3. We then prune (C − ddrCe) channels with the smallest importance values in ur. Accordingly, the value of an element in Mr is set to be 1 if the corresponding channel is pruned, otherwise 0.

heading: 3.1.2 STATIC CHANNEL IMPORTANCE
text: The static channel importance vector us is to capture the global information for pruning, and thus is learned from the whole dataset. It is randomly generated and learned through backpropagation. Similar to runtime channel pruning, given a sparsity ratio ds learned by the static DRL agent, (C − ddsCe) channels with smallest importance values in us are pruned, and a mask Ms ∈ {0, 1}C is generated to indicate the static pruning results.

heading: 3.2 TRADE-OFF PRUNER
text: With the runtime and the static pruning decisions, Mr and Ms, we now propose a trade-off pruner to generate a unified channel pruning decision. The main idea behind the trade-off pruner is to 1) prune those channels which are agreed to be pruned by both decisions, and 2) prune a portion of the rest channels by weighted votes from both decisions.
To be specific, we define the mask representing channels pruned by both decisions as
Mo = Ms ∧Mr, (3) where ∧ is element-wise logical AND and 1/0 in mask represents logical true or false. The channels indicated to be pruned by Mo (i.e., the corresponding values are 1) are pruned in final. The channels
which are determined to be pruned by Mr but not by Ms can be represented by a new mask Mr = Mr −Mo. Similarly, the channels which are determined to be pruned by Mr but not by Ms can be represented by another new mask Ms = Ms −Mo. To control the trade-off between Mr and Ms, we define a rate Rr denoting how much we trust the pruning decision made by Mr, while 1 − Rr is for Ms. That means the channels selected by Mr will be finally pruned with the rate Rr. Specifically, the number of channels which are selected by Mr and finally will be pruned is
C ′ r = bRr(1>Mr)c, (4)
where 1>Mr returns the number of channels selected by Mr. We then select the first C ′
r-smallest important channels which are recommended to be pruned by Mr to form a mask M̂r. Similarly, for static pruning, we select the first C ′
s-smallest important channels which are recommended to be pruned by Ms to form another mask M̂s, where C ′
s = b(1−Rr)(1>Ms)c. The final trade-off pruning mask is defined as
M = Mo + M̂r + M̂s. (5)
Moreover, in this work, the unified channel importance is simply defined as follows,
u = ur ⊗ us. (6)
With the trade-off pruning mask M and the unified channel importance u, the pruned output feature F̂out can be generated by Eq. 2.

heading: 3.3 DRL BASED PRUNING
text: In this section, we present how to formulate the problems of learning the ratios ds and dt for static pruning and runtime pruning, as a MDP, and solve it via DRL, respectively.

heading: 3.3.1 DRL FOR RUNTIME PRUNING
text: In the MDP for runtime pruning, we consider the t-th layer of the network as the t-th timestamp. The details of the MDP are listed as follows.
State Given an input feature map Fin of layer t, we pass it to a global pooling layer to reduce its dimension to RCin , where Cin is the number of input channel of layer t. Since Cin varies among layers, we feed the output of global pooling to a layer-dependent encoder to project it to a fix-length vector srt , which is considered as as a state representation of DRL in the context of runtime pruning.
Action The action art is defined as the sparsity ratio at layer t, alternating dr in runtime pruning mentioned in Section 3.1.1. Existing DRL-base pruning method RNP (Lin et al., 2017) uses a unified discrete actions space with k actions which are too coarse to achieve high accuracy. However, finegrained discrete action space as large as number of channels suffers from exploration difficulty. Therefore, instead of using discrete action spaces, we propose a continuous action space with action art ∈ (0, 1]. To avoid over-pruning the filters and crashing in training, we set a minimum sparsity ratio +α such that art ∈ (+α, 1].
Reward The reward function is proposed to consider both network accuracy and computation budget. We define the accuracy relative reward based on the loss of pruned backbone network,
Rracc = −LCNN , (7)
where LCNN is the loss in CNN, and it may vary in scale among different training stage, i.e. large at beginning of training and small near convergence. To avoid the instability brought by the reward scale, Rracc is normalized by a moving average,
Rr ′
acc = R r acc/βb, (8)
βb = λβb−1 + (1− λ)Rracc, (9)
where βb is the moving average at the b-th training batch and λ is the moving weight.
To force computation of the pruned network under a given computation budget, we define a exponential reward function of budget regarding reward Rrbud:
Rrbud = { exp(α1(Bcom −Bcom))− 1, Bcom > Bcom, 0, otherwise,
(10)
where Bcom is the computation consumption, which is calculated based on the current of pruned strategy, and Bcom is the given computation budget constraint. Finally we sum up the two rewards to form sparse rewards, with being non-zero at terminated step T and zeros at other time step t < T ,
Rrt =
{ Rr ′
acc +R r bud, t = T,
0, t < T. (11)
Actor-Critic Agent To solve the continuous action space problem, we choose a commonly used actor-critic agent with a Gaussian policy. Actor-critic agent consists of two components: 1) actor outputs the mean and variance to form a Gaussian policy where the continuous action are sampled from; 2) critic outputs a scalar predicting the future discounted accumulated reward and assists the policy training. Actor network and Critic network share one-layer RNN which takes state srt as input. The output of RNN is fed into actor specific network constructed by two branches of fullyconnected layers, leading to the mean and variance of the Gaussian policy. The action is sampled for the Gaussian distribution outputed by the actor:
art ∼ N (µ(srt ; θr), σ(srt ; θr)), (12)
where µ(srt ; θ r) and σ(srt ; θ r) is the mean and variance outputed from actor network. The Critic specific network has one fully-connected layer after the shared RNN, and outputs the predictive value V (srt ; θ r).
To optimize the actor-critic agent, Proximal Policy Optimization (PPO) (Schulman et al., 2017) is used. Note that we relax the action art to (−∞,+∞) in PPO, and use truncate function to clip art in (+α, 1] when perform pruning.
Besides, an additional regularizer is introduced to restrict the relaxed art staying in range (+α, 1],
La = 1
2 ||art −max(min(art , 1),+α)||22. (13)

heading: 3.3.2 DRL FOR STATIC PRUNING
text: Similar to runtime pruning, the MDP in static pruning is also formulated layer-by-layer. The difference against runtime pruning is the definition of state and reward.
State The state sst in static pruning is defined as the full shape of Fout, and does not depend on Fout and the current input data.
Action Action ast is sampled from actor’s outputed Gaussian policy, and it is to alternate the sparsity ds in static pruning mentioned in Section 3.1.2.
Reward The reward function takes both network accuracy and parameters budget into consideration. The accuracy relative is defined as the same as that in runtime pruning:
Rsacc = R r′ acc. (14)
To reduce the number of parameters of network to satisfy the parameters storage budget, the parameters relative reward is defined in an exponential form as,
Rsparam = { exp(α2(Bparam −Bparam))− 1, Bparam > Bparam, 0, otherwise,
(15)
where Bparam is the number of preserved parameters after static pruning and Bparam is the parameters storage budget.
Actor-Critic Agent This agent is similar to the one in runtime pruning. It has the same architecture as runtime pruning but differs in introducing a fully-connected layer as the encoder before RNN. This agent is also optimized by PPO.

heading: 3.4 INFERENCE
text: In inference, the static agent is not required any more because the static pruning strategy does not depend on individual input data points but the full shape of Fin. Therefore, the output action ast is fixed to each layer t. With the action ast and the rate Rr, we can decide which filters can be pruned permanently. Specifically, channels with ((1 − ast )(1 − Rr))-smallest static importance values are pruned permanently.

heading: 4 EXPERIMENT
text: We evaluate our DRL pruning framework on two popular datasets: CIFAR-10 (Krizhevsky, 2009) and ImageNet ILSVRC2012 (Russakovsky et al., 2015), to show the advantage over other channel pruning methods. We analyze the effect of hyper-parameters and different sparsity settings on CIFAR-10. For CIFAR-10, we use M-CifarNet (Zhao et al., 2018) as the backbone CNN. On ImageNet ILSVRC2012, ResNet-18 is used as the backbone CNN.

heading: 4.1 IMPLEMENTATION DETAILS
text: We start with a pretrained backbone CNN. Firstly we finetune the backbone CNN and train runtime importance predictor jointly, with sparsity dr = 1 and fixed all static pruning importance us to 1. Then we remove the restriction on the static pruning importance us, and train static pruning importance, the backbone CNN and the runtime importance predictor, with sparsity ds = 1 and runtime pruning sparsity fixed as dr = 1. After finetuning, we use the DRL agents to predict the sparsity given computation and storage constraints. The DRL agents and the CNN with runtime/static importance are trained in alternating manner: We first fix the CNN as well as runtime/static importance and train two DRL agents, regarding the CNN as environments. Then we fix two agents and finetune the CNN and runtime/static importance. We repeat these two steps until convergence is achieved. We use Adam optimizer for both DRL agent and CNN, and set learning rate at 10−6 for the DRL agents. For CNN finetuning and runtime/static importance training, the learning rate is set to 10−3 on CIFAR-10. On ImageNet ILSVRC2012, the learning rate starts from 10−3 and is divided by 10 after 15 millions iterations.

heading: 4.2 EXPERIMENTAL RESULTS ON CIFAR-10
text: We compare our proposed method with the following state-of-the-art runtime pruning methods: FBS (Gao et al., 2019), RNP (Lin et al., 2017) on CIFAR-10. The comparison results at sparsity 0.5 and 0.7 are shown in Table 1 and Table 2 respectively. Note that for fair comparison with other methods, the computation and storage budget constraints in our method is calculated according to the sparsity of other methods. Under these constraints, our method does not necessarily lead to the same sparsity as other methods in each layer. RNP cannot set exact sparsity ratio. Instead, its average sparsity ratio is accessible only during testing, which is 0.537 in Table 1. The result of FBS is reproduced using the released code1. The column #Params represents the number of parameters compared to the backbone CNN.
Table 1 shows that our method outperforms other state-of-the-art methods, achieving highest accuracy at an overall sparsity ratio of 0.5. Our method has very close computation speed-up compared to FBS, but outperforms FBS around 0.48% to 0.76%. When the runtime pruning strategy is solely considered by setting Rr = 1, our method surpasses other comparison methods, indicating that our DRL-based framework improves the performance of channel runtime pruning. By balancing runtime and static pruning via setting Rr = 0.5, our method reduces the number of the overall stored parameters and achieves lower accuracy drop than other methods. Table 2 shows that our method outperforms FBS at sparsity of 0.7. When Rr = 0.5, our method achieves better performance than the baseline CNN with 2× speed-up and contains less parameters. We also study the relation between Rr and network compactness in our framework. Fig. 2 demonstrates the impact of Rr when sparsity is 0.45. The hyper-parameter Rr determines how much we trust about runtime pruning. With Rr close to 1, the accuracy becomes higher due to the more dynamic network flexibility but the space of the parameters storage also increases. When Rr diminishes, the network accuracy decreases but the parameter storage is reduced.
Fig. 3 shows the performance of various sparsity ratios in our method. Again, our method does not prune with one single sparsity ratio for all layers, but uses the sparsity ratio to calculate computation and storage constraints, with which the sparsity ratio is learned for each layer. Fig. 3 demonstrates
1https://github.com/deep-fry/mayo
that our method holds the accuracy when sparsity is larger than about 0.5, which corresponds to about 4× computational acceleration.

heading: 4.3 EXPERIMENTAL RESULTS ON IMAGENET ILSVRC2012
text: We compare our method with state-of-the-art channel pruning methods on ImageNet ILSVRC2012 as shown in Table 3. In this experiment, we use ResNet-18 as the backbone CNN. Among the stateof-the-art pruning methods for comparison, FBS (Gao et al., 2019) and CGNN (Hua et al., 2018) are runtime pruning methods. The overall sparsity ratio of our method is 0.7, which is under the same setting of FBS. Our method with Rr = 0.5 achieves the smallest top-1 accuracy drop compared with other methods, and also achieves the highest top-1 accuracy after pruning. Overall, our proposed method achieves comparable or better performance compared to other methods with more acceleration. Our method has very close MACs to FBS, while the number of preserved parameters is reduced to 81.2% of the baseline.

heading: 5 CONCLUSION
text: In this paper, we present a deep reinforcement learning based framework for deep neural network channel pruning in both runtime and static sheme. Specially, channels are pruned according to input feature as runtime pruning, and based on entire training dataset as static pruning, with 2 reinforcement agents to determine the corresponding sparsity. Our method combines the merits of runtime and static pruning, and provides trade-off between storage and dynamic flexibility. Extensive experiments demonstrate the effectiveness of our proposed method.

heading: A ADDITIONAL EXPERIMENTAL RESULTS
text: A.1 COMPARISON TO SEPARATELY STATIC AND RUNTIME PRUNING
In this section, we compare our method with two additional baseline methods. One is a variation of our method by separately training static pruning and the training runtime pruning. In this method, we start from a pretrained backbone CNN, f(·) and us. Then we add the static DRL agent to prune channels statically by learning the static policy and us. Finally, we add the runtime DRL agent to prune channels dynamically, by fixing the static DLR agent and us, and updating the runtime DLR agent and f(·) only. Another method is to combine state-of-the-art static and runtime pruning methods. We start from a pretrain backbone CNN, and then prune channels with the static pruning method FPGM (He et al., 2019), and finally prune channels with the runtime method FBS (Gao et al., 2019). The experimental results are shown in Table4.
A.2 STORAGE/ACCURACY TRADE-OFF
Besides Fig. 2, to further illustrate the trade-off between storage and accuracy, we show additional results in Fig. 4.

heading: B PSEUDOCODE OF TRAINING PROCESS
text: Algorithm 1: Training process INPUT: pretrained backbone CNN, computation budget B̄com, storage budget B̄param OUTPUT: backbone CNN, importance predictor f(·), static pruning importance us, runtime and static DRL agents
Add runtime importance predictor f(·) and static pruning importance us.; us ← 1; ds ← 1; dr ← 1; while not converge do
fix us, update f(·) and backbone CNN; end while not converge do
update us, f(·) and backbone CNN; end add runtime DRL agent and static DRL agent to predict actions art and a s t , and alternate dr and ds at each layer t ; while not converge do
for i← 1 to N1 do Forward entired model; Compute rewards Rrt and R s t using budget B̄com and B̄param ;
Fix us, f(·) and backbone CNN, update runtime and static DRL agents by PPO loss; end for i← 1 to N2 do
Fix runtime and static DRL agents, update us, f(·) and backbone CNN by cross entropy loss
end end

heading: C TRAINING CURVE
text: We show the training curve of our method in Fig. 5. We train our method on CIFAR-10 at sparsity 0.5 with Rr = 0.5. The figure includes the accuracy curves evaluated on training set and test set, showing that our methods is stable during training.

Review summaries:
This paper proposes to learn static and dynamic channel pruning policies for convolutional neural networks. Static pruning depends only on the training dataset and is computed once before the model is deployed. Dynamic pruning is input-dependent. The policies are obtained with deep reinforcement learning on the training dataset using a combination of the loss function and storage/computation resource budgets as a reward signal. The key novelty in this paper is to combine static and dynamic pruning which can obtain the benefits from both worlds. Experimentally, the learned pruning policies are competitive with recent dynamic pruning approaches on CIFAR-10 and ILSVRC2012, in terms of both final test accuracy and number of parameters/inference time.

This work introduces a Reinforcement Learning based framework that simultaneously learns both a static and dynamic pruning strategy. The combination allows the static pruner to decrease the required storage while the dynamic pruning can optimize the required compute using input-dependent pruned weights. The RL agent can dynamically learn the optimal sparsity distribution among the different layer of the networks while staying under a resource constraint as opposed to other methods which often enforce a layer level sparsity ratio. It demonstrates the efficacy of the algorithm on CIFAR10 and ILSVRC2012 and showed the effect of the tradeoff between static and dynamic pruning.