Structural Pruning via Latency-Saliency Knapsack
Anonymous Author(s)
AffiliationAddress email
Abstract
Structural pruning can simplify network architecture and improve inference speed.
We propose Latency-Aware Structural Pruning (LASP) that formulates structuralpruning as a global resource allocation optimization problem, aiming at maximizingthe accuracy while constraining latency under a predefined budget.
For filterimportance ranking, LASP leverages latency lookup table to track latency reductionpotential and global saliency score to gauge accuracy drop.
Both metrics canbe evaluated very efficiently during pruning, allowing us to reformulate globalstructural pruning under a reward maximization problem given target constraint.
This makes the problem solvable via our augmented knapsack solver, enabling
LASP to surpass prior work in pruning efficacy and accuracy-efficiency tradeoff.
We examine LASP on both classification and detection tasks, over varyingnetworks, on ImageNet and VOC datasets, on different platforms.
In particular,for ResNet-50/-101 pruning on ImageNet, LASP improves network throughput by1 .
60 × / 1 . 90 × with +0 .
3% / − 0 .
2% top-1 accuracy changes, respectively.
For SSDpruning on VOC, LASP improves throughput by 1 . 94 × with only a 0 .
56 mAPdrop.
LASP consistently outperforms prior art, sometimes by large margins.
Introduction
Convolutional Neural Networks (CNNs) act as the central tenet behind the rapid development incomputer vision tasks such as classification, detection, segmentation, image synthesis, among others.
As performance boosts, so do model size, computation, and latency.
With millions, sometimes billionsof parameters ( e g ., [-GPT-
 [4]),-]GPT-3 [3]), modern neural networks face increasing challenges upon ubiquitousdeployment, that mostly faces stringent constraints such as energy and latency [8, 38, 37].
In certaincases like autonomous driving, a breach of real-time constraint not only undermines user experience,but also imposes critical safety concerns.
Even for cloud service, speeding up the inference directlytranslates into higher throughput, allowing more clients and users to benefit from the service.
One effective and efficient method to reduce model complexity is network pruning.
The primarygoal of pruning is to remove the parameters, along with their computation, that are deemed leastimportant for inference [2, 18, 31, 37].
Compatible with other compression streams of work such asquantization [4, 53, 60] and distillation [21, 39, 57], pruning enables a flexible tuning of model complexity towards varying constraints, while requiring much less design efforts by neural architecturesearch [48, 50, 54] and architecture re-designs [22, 36, 49].
Thus, in this work, we study pruning, inparticular structured pruning that reduces channels to benefit off-the-shelf platforms, e . g ., GPUs.
As the pruning literature develops, the pruning criteria also evolves to better reflect final efficiency.
Theearly phase of the field focuses on maximum parameter removal in seek for minimum representationsof the pretrained model.
This leads to a flourish of approaches that rank neurons effectively tomeasure their importance [38, 51].
As each neuron/filter possesses intrinsically different computation,following works explore proxy to enhance redundancy removal, FLOPs being one of the most widelyadopted metrics [26, 55, 59] to reflect how many multiplication and addition computes needed for
[Figure]
the model.
However, for models with very similar FLOPs, their latency can vary significantly [48].
Recently, more and more works start directly working on reducing latency [5, 56].
However, notmuch was done in the field of GPU friendly pruning methods due to non-trivial latency-architecturetrade-off.
For example, as recently observed in [43], GPU usually imposes staircase-shaped latencypatterns for convolutional operators with varying channels, which inevitably occur per varyingpruning rate, see the latency surface in Fig.
This imposes a constraint that pruning needs to bedone in groups to achieve latency improvement.
Moreover, getting the exact look-up table of layersunder different pruning configuration will benefit maximizing performance while minimizing latency.
Pruning different layers in the deep neural network will result in different accuracy-latency trade-off.
Typically, removing channels from the latter layers has smaller impact on accuracy and smaller impacton latency versus removing channels from the early layers.
We ask the question, if it is better toremove more neurons from latter layer or less from early layer to achieve the same accuracy-latencytrade-off.
By nature, the problem is combinatorial and requires the appropriate solution.
In this paper, we propose latency-aware structural pruning (LASP) that formulates pruning as aresource allocation optimization problem to maximize the accuracy while maintaining a latencybudget.
The overall workflow is shown in Fig.
For latency estimate per pruned architecture,we pre-analyze the operator-level latency values by creating a look-up table for every layer of themodel on the target hardware.
Then we introduce an additional score for each neuron group toreflect and encourage latency reduction.
To this end, we first rank the neurons according to theirimportance estimates, and then dynamically adjust their latency contributions.
With neurons recalibrated towards the hardware-aware latency curve, we now select remaining neurons to maximizethe gradient-based importance estimates for accuracy, within the total latency constraint.
This makesthe entire neuron ranking solvable under the knapsack paradigm.
To enforce the neuron selectionorder in a layer to be from the most important to the least, we have enhanced the knapsack solver so that the calculated latency contributions of the remaining neurons would hold.
LASP surpasses priorart in pruning efficacy, see Fig.
2 and the more detailed analysis in Section 4.
Our main contributionsare summarized as follows:
• We propose a latency-driven structured pruning algorithm that exploits hardware latency traits
to yield direct inference speedups.
• We orient the pruning process around a quick yet highly effective knapsack scheme that seeksfor a combination of remaining neuron groups to maximize importance while constraining tothe target latency.•
We introduce a group size adjustment scheme for knapsack solver amid varying latency contributions across layers, hence allowing full exploitation of the latency landscape of the underlyinghardware.• We compare to prior art when pruning ResNet, MobileNet, VGG architectures on ImageNet,PASCAL VOC and demonstrate that our method yields consistent latency and accuracy improvements over state-of-the-art methods.
Our ImageNet pruning results present a viable 1 .
6 × to1 .
9 × speedup while preserving very similar original accuracy of the ResNets.
Related work
Pruning methods .
Depending on when to perform pruning, current methods can generally bedivided into three groups [13]: i) prune pretrained models [18, 34, 27, 20, 38, 37, 15], ii) prune atinitialization [14, 25, 9], and iii) prune during training [2, 16, 35].
Despite notable progresses in thelater two streams, pruning pretrained models remains as the most popular paradigm, with structuralsparsity favored by off-the-shelf inference platforms such as GPU.
To improve on inference efficiency, many previous pruning methods trim down the neural networkaiming to achieve a high compression rate while maintaining an acceptable accuracy.
The estimationof neuron importance has been widely studied in literature [23, 34, 38].
For example, [37] proposesto use Taylor expansion to measure the importance of neurons and prunes the least-ranked ones untila desired number of neurons are pruned.
However, a compression ratio does not directly translate intocomputation reduction ratio, amid the fact that each neuron/filter possesses different computation.
There are recent methods that focus primarily on reducing FLOPs.
Some of them take FLOPs intoconsideration when calculating the neuron importance to encourage penalizing neurons that inducehigh computations [55].
An alternative line of work propose to select the best pruned network from aset of candidates [26, 56].
However, it would take a long time for candidate selection due to the largeamount of candidates.
In addition, these methods use FLOPs as a proxy of latency, which is usuallyinaccurate as networks with similar FLOPs might have significantly different latencies [48].
Latency-aware compression.
Emerging compression techniques shift attention to directly pruneto cut down on latency.
One popular stream is Neural Architecture Search (NAS) methods [8,11, 48, 54] that adaptively adjusts the architecture of the network for a given latency requirement.
They incorporate the platform constraints into the optimization process in both the architecture andparameter space to jointly optimize the model size and accuracy.
Despite remarkable insights, NASmethods remain computationally expensive in general compared to their pruning counterparts.
Latency-oriented pruning has also gained a growing amount of attention.
[5] presents a frameworkfor network compression under operational constraints, using Bayesian optimization to iterativelyobtain compression hyperparameters that satisfies the constraints.
Along the same line, NetAdapt [56]iteratively prunes neurons across layers under the guidance of the empirical latency measurements onthe targeting platform.
While these methods push the frontier of latency constrained pruning, thehardware-incurred latency surface in fact offers much more potential under our enhanced pruningpolicy - as we show later, large rooms for improvements remain un-exploited and realizable.
Method
In this section, we first formulate the pruning process as an optimization process, before diving deepinto the importance estimation for accuracy and latency.
Then, we elaborate on how to solve theoptimization via knapsack regime, augmented by dynamic grouping of neurons.
We finalize themethod by combining these key steps under one realm of LASP.
3.1 Objective function
Consider a neural network that consists of L layers performing linear operations on their inputs,together with non-linear activation layers and potentially pooling layers.
Suppose there are N l neurons(output channels) in the l th layer and each neuron is encoded by parameters W nl ∈ R C inl × K l × K l ,where C in is the number of input channels and K is the kernel size.
By putting all the neurons acrossthe network together, we get the neuron parameter set W = {{ W nl } N l n =1 } Ll =1 , where N = (cid:80) Ll
=1 N lis the total number of neurons in the network.
Given a training set D
= { ( x i , y i ) }
Mi =1 , the problem of network pruning with a given constraint Ccan be generally formulated as the following optimization problem:
[Equation]
where ˆ W ⊂ W is the remaining parameters after pruning and L is the loss of the task.
f ( · ) encodes thenetwork function, and Φ( · ) maps the network to the constraint C , such as latency, FLOPs, or memory.
We primarily focus on latency in this work while the method easily scales to other constraints.
The key to solving the aforementioned problem relies on identifying the portion of the network thatsatisfies the constraint while incurring minimum performance disruption:
[Equation]
where p l denotes the number of kept neurons at layer l , I l
( p l ) signals the maximum importance tothe final accuracy with p l neurons, and T l ( p l − 1 , p l ) checks on the associated latency contribution oflayer l with p l − 1 input channels and p l output channels.
p 0 denotes a fixed input channel number forthe first convolutional block, e .
g ., 3 for RGB images.
We next elaborate on I ( · ) and T ( · ) in detail.
Importance score.
To get the importance score of a layer to final accuracy, namely I l ( p l ) in Eq.
2,we take it as the accumulated score from individual neurons (cid:80) p
l j =
I jl .
We first approximate theimportance of neurons using the Taylor expansion of the loss change [37].
Specifically, we prune onbatch normalization layers and the importance of the n -th neuron in the l -th layer is calculated as
[Equation]
where g denotes the gradient of the weight, γ nl and β nl are the corresponding weight and bias fromthe batch normalization layer, respectively.
Unlike a squared loss in [37], we use absolute differenceas we observe slight improvements.
In order to maximize the total importance, we keep the most important neurons at a higher priority.
To this end, we rank the neurons in the l th layer according to their importance score in a descendingorder and denote the importance score of the j th -ranked neuron as I jl , thus we have
[Equation]
Latency contribution.
We empirically obtain the layer latency T l ( p l − 1 , p l ) in Eq. 2 by pre-buildinga layer-wise look-up table with pre-measured latencies.
This layer latency corresponds to theaggregation of the neuron latency contribution of each neuron in the layer, c jl :
[Equation]
The latency contribution of the j -th neuron in the l -th layer can also be computed using the entries inthe look up table as:
[Equation]
In practice, we first rank globally neurons by importance and then consider their latency contribution.
Thus, we can draw the following properties.
If we remove the least important neuron in layer l , thenthe number of neurons will change from p l to p l − 1 , leading to a latency reduction c
p l l as this neuron’slatency contribution score.
We assign the potential latency reduction to neurons in the layer by theimportance order.
The most important neuron in that layer would always have a latency contributionc 1 l .
At this stage, finding the right combination of neurons to keep imposes a combinatorial problem,and in the next section we tackle it via reward maximization considering latency and accuracy traits.
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
3.2 Augmented knapsack solver
Given both importance and latency estimates [Figure] we now aim at solving Eq. 2.
By plugging back in thelayer importance Eq. 4 and layer latency Eq.
5, we come tomax
L (cid:88) l = p l (cid:88)
j =
I jl , s.t.
L (cid:88) l = p l (cid:88)
j = c [Equation] C, [Equation]
p l
[Equation]
N l , I 1 l [Equation] l
[Equation]
I N l l .
This simplifies the overall pruning process into a knapsack problem only with additional precedingconstraints.
The preceding enforcement originates from the fact that for a neuron with rank j in thel th layer, the neuron latency contribution only holds when all the neurons with rank r = 1 , . . .
, j − 1are kept in the l th layer and the rest of the neurons with rank r = j + 1 , j + 2 , · · · , N l are removed.
Yet the problem is solvable by specifying each item with a list of preceding items that need to beselected before its inclusion.
We augment the knapsack solver to consider the reordered neurons with descending importance scoreso that all the preceding neurons will removed before it.
A description of the pseudo code of theaugmented knapsack solver is provided in Algo.
1 (a detailed explanation is provided in Appendix A).
The augmented solver is required to make sure that the latency cost is correct.
3.3 Neuron grouping
Considering each neuron individually results in burdensome computation during pruning.
We nextexplore grouping neurons so that a number of them can be jointly considered and removed enablingfaster pruning [57].
Neuron grouping helps exploit hardware-incurred channel granularity guidedby the latency, speeds up knapsack solving of Eq. 7, and yields structures that maximize the GPUutilization, keeping as many parameters as possible under the similar latency.
In addition, neurongrouping simplifies the knapsack problem that scales linearly with the number of candidates underconsideration (see Line 3 of Alg. 1), thus speedups the solver as we’ll discuss later.
We refer to the difference of neuron counts between two latency cliffs of the staircase-patternedlatency as the latency step size.
In our method, we group s channels in a layer as an entirety, wherethe value of s is equal to the latency step size.
The neurons are grouped by the order of importance.
Then we aggregate the importance score and latency contribution for the grouped entity.
For skipconnections in ResNet and group convolutions in MobileNet, we not only group neurons within alayer, we also group the neurons sharing the same channel index from the connected layers [10, 33].
For cross-layer grouping, as the latency step size for different layers might be different, we use thelargest group size among the layers.
Latency-aware grouping enables additional performance benefitswhen compared to a heuristic universal grouping, as we will later show in the experiments.
3.4 Final LASP regime
With all aforementioned steps, we formulate the final LASP as follows.
The pruning process takesa trained network as input and prunes it iteratively to satisfy the requirement of a given latencybudget C .
We perform one pruning every r minibatches and repeat it k pruning steps in total.
Inparticular, we set k milestones gradually decreasing the total latency to reach the goal via exponentialscheduler [9], with C 1 > C 2 > · · · > C k , C k = C .
The algorithm gradually trims down neuronsusing steps below:
• Step 1 .
For each minibatch, we get the gradients of the weights and update the weights as duringthe normal training.
We also calculate each neuron’s importance score as Eq.
3.• Step 2 .
Over multiple minibatches we calculate the average importance score for each neuronand rank them accordingly.
Then we count the number of neurons remaining in each layer and
dynamically adjust the latency contribution as in Eq. 6.
• Step 3 .
We group neurons as described in Sec.
3.3 and calculate group’s importance and latencyreduction.
Then we get the nearest preceding group for each layer.• Step 4 .
We execute the Algo.
1 to select the neurons being remained with current latencymilestone.
Repeat starting from the Step 1 until k milestones are reached.
Once pruning finishes we fine-tune the network to recover accuracy.
Experiments
We demonstrate the efficacy and feasibility of the proposed LASP method in this section.
Weuse ImageNet ILSVRC
[45] for classification.
We first study five architectures (ResNet50,
ResNet101, VGG16 and MobileNet-V1/V2) on different platforms and compare our pruning resultswith the state-of-the-art methods on classification task.
We then study the impact of grouping size son the pruned top-1 accuracy and the inference time to show the effectiveness of our grouping scheme.
Finally, we further show the generalization ability of our algorithm by testing with object detectiontask.
We introduce the details of experimental setting in appendix Sec.
B. We apply LASP targetinglatency reduction on multiple platforms to show the scalability of our method: NVIDIA TITAN V GPU, Jetson TX2, Jetson Xavier and Intel CPU.
All the latencies are measured with batchedimages to take advantage of computation capacities.
We include experimental results for batch size 1inference in the appendix, which also show the efficacy of our algorithm.
4.1 Results on ImageNet
ResNets.
We start by pruning ResNet50 and ResNet101 and compare our results with state-of-the-artmethods in Tab. 1 on TITAN V.
In order to have a fair comparison of the latency, for all the othermethods, we recreate pruned networks according to the pruned structures they published and measurethe latency.
Those methods showing ‘-’ in the table do not have pruned structures published so weare unable to measure the latency.
For our method, by setting the percentage of latency to remainafter pruning to be X , we get the final pruned model and refer to it as LASP- X % .
We report FPS(frames per second) in the table and calculate the speedup of a pruned network as the ratio of FPSbetween pruned and unpruned models.
From the results comparison in Table.
1 we can find that for pruned networks with similar FLOPsusing different methods, our method achieves the highest accuracy and also the fastest inferencespeed.
This also shows that FLOPs do not correspond 1 : 1 to the latency.
Among these methods for
ResNet50 comparison, EagleEye [26] yields the closest accuracy to ours, but the speedup is lowerthan ours.
In Table.
1, for the pruned ResNet50 network with 3G FLOPs remaining, our methodachieves a . 4% higher top1 accuracy and slightly ( . 04 × ) faster inference.
It is expected that theadvantage of our method for accelerating the inference is more obvious when it comes to a morecompact pruned network, which is 14% (or . 20 × ) additionally faster for a 2G-FLOPs network whileincreasing accuracy by .
2% , and .
17 × faster with .
3% higher accuracy compared to EagleEye-1G.
We analyze the pruned network structure in detail in the supplementary material (Appendix Sec. Q).
We plot the results comparison in Fig.
2, where we also add the results of training our pruned networkwith a teacher model RegNetY-16GF (top1 82 .
With knowledge distillation, our model is2 .
70 × faster than the original model at 1% accuracy drop.
[Table]
[Figure]
Scalability to other networks.
We next experiment with three other models: VGG [47], MobileNetV1 [22] and MobileNetV2 [46].
Same as pruning on ResNets, in Tab. 1, we perform pruningwith different latency constraints and compare with prior art.
As shown, among these methods, theproposed LASP performs significantly better with higher accuracy and larger inference speedup.
Scalability to other platforms.
Our approach is not limited to a single platform.
In this section, weconduct the same ResNet50 experiments on three new platforms: NVIDIA Jetson TX2, NVIDIA
Xavier, and Intel CPU Xeon E5, and compare the results in Fig.3.
As shown, our approach consistentlyoutperforms the other methods with more speedup and higher accuracy, as LASP leverages the latencycharacteristics of the platform to achieve a better accuracy-efficiency trade-off.
Specifically, on theseplatforms, LASP yields up to relative 1 .
27 × speedup with slightly increment in top-1 accuracycompared to EagleEye [26], and 1 . 72 × faster than the original model.
From these results, we canconclude that our method generalizes well to different platforms.
[Table]
4.2 LASP acceleration on GPUs with TensorRT
To make it closer to the real application in production, we also export the models into onnx formatand test the inference speed with TensorRT.
We run the inference of the model with
FP32, FP
andalso INT8.
For INT8, we quantize the model using entropy calibration with 2560 randomly selected
ImageNet training images.
Since the INT8 TensorCore speedup is not supported in TITAN V GPU,we only report the quantized results on RTX3080 GPU.
The accelerations and the corresponding top1accuracy drop (compared to PyTorch baseline model) are listed in Tab. 2.
We include more pruningresults specifically for INT8 quantization in Appendix.
N.
4.3 Design effort for pruning
Our approach is around 4.3 × faster than the next best method.
Sub-network selection timing is approximated as running on same device (a NVIDIA V100)
[Table]
In addition to noticeable performance boosts,
LASP in fact requires less design effort compared to prior art, as summarized in Tab. 3 (details in Appendix O).
NetAdapt [56] and AutoSlim [59] generate many proposals during iterative pruning.
Then evaluations of the proposals are needed to select the best candidatefor the next pruning iteration.
EagleEye [26]pre-obtains 1000 candidates before pruning andevaluates all of them in order to get the best one.
Such pruning candidate selection is intuitive butcauses a lot of additional time costs.
The computation cost for MetaPruning [30] and AMC [20]can be even higher because they need to train auxiliary network to generate the pruned structure.
Compared to these methods, our method does not require auxiliary network training nor sub-networkevaluation.
The latency contribution in our method can be quickly obtained during pruning by thepre-generated latency lookup table.
Although creating the table for the target platform might costtime, we only do it once for all pruning ratios.
Solving the augmented knapsack problem brings extracomputation, however, after neuron grouping, it only takes around additional 30 minutes of CPU timein total for ResNet50 pruning and less than 1 minute for MobileNetV1, which is negligible comparedto the fine-tuning process or training additional models.
Moreover, this is significantly lower thanother methods, for example the fastest of them EagleEye [26] requires 30 GPU hours.
4.4 Efficacy of neuron grouping
We then show the benefits of latency-aware neuron grouping and compare the performance underdifferent group size settings.
Performance comparison.
As described in Sec. 3.3, we group s neurons in a layer as an entirety sothat they are removed or kept together.
Choosing different group sizes leads to different performances,and also different computation cost on the augmented knapsack problem solving.
In our method, weset an individual group size for each layer according to each layer’s latency step size in the look-uptable.
We name the grouping in our method as latency-aware grouping (LG).
For instance, for a
ResNet50, using this approach we set the individual group size of 23 layers to 32, of 20 layers to 64,and 10 layers to 128.
Layers closer to the input tend to use a smaller group size.
Another option forneuron grouping is to heuristically set a fixed group size for all layers as literature does [57].
Fig.
4 shows the performance of our grouping approach compared to various fixed group sizes for a
ResNet50 pruned with different constraints.
[Figure]
As shown, using small group sizes yields theworst performance independently of the latencyconstraint.
At the same time, a very large groupsuch as 256 do also harm the final performance.
Intuitively, a large group size averages the contribution of many neurons and therefore is notdiscriminative enough to select the most important ones.
Besides, large groups might promotepruning the entire layer in a single pruning step,leading to performance drop.
On the other hand,small group sizes such as 2 promote removingunimportant groups of neurons.
These groupsdo not significantly improve the latency, but cancontribute to the final performance.
In contrast, our latency-aware grouping performs the best, showing the efficacy of our grouping scheme.
Algorithm efficiency improvement.
Setting the group size according to the latency step size notonly improves the performance, but also reduces computation cost on knapsack problem solving forneuron selection since it reduces the total number of object N to a smaller value.
In our ResNet experiment, except for the first convolution layer, the group size of other layers varies from 32 to128 .
By neuron grouping, the value of N can be reduced to 215 , which takes around one minute onaverage at each pruning step to solve the knapsack problem on CPU.
We have 30 pruning steps intotal in our experiments, thus the time spent on neuron selection is around 30 minutes in total, whichcan be negligible compared to training time.
4.5 Generalization to object detection
[Figure]
To show the generalization ability of our proposed LASP algorithm, we also apply the algorithm to the object detection task.
In thisexperiment we take the popular architecture Single Shot Detector (SSD) [29] on the PASCAL
VOC dataset [12].
Following the “07+12” setting in [29], we use the union of VOC2007 and
VOC2012 trainval as our training set and use the
VOC2007 test as test set.
We pretrain a SSD512detector with ResNet50 as the backbone.
Thedetails of the SSD structure are elaborated in theappendix.
Same to classification task, we prunethe trained detector and finetune afterwards.
Weonly prune the backbone network in the detector.
The results in Fig.
4 show that the pruned detector maintain the similar final mAP but reduce the FLOPs and improve the inference speed greatly,with 77% FLOPs reduction and around 1 .
94 × speedup at the cost of only 0 .
56% mAP drop.
Wecompare the pruned detector to some other commonly-used detectors in the table.
The results showthat pruning a detector using LASP improves performance in almost all aspects.
Conclusion
We proposed latency-aware structural pruning (LASP) that focuses on pruning for underlying hardware towards latency budgets.
We formulated pruning as a resource allocation optimization problemto achieve maximum accuracy within a given latency budget.
We further proposed a latency-awareneuron grouping scheme to improve latency reduction.
Over multiple neural network architectures,classification and detection tasks, and changing datasets, we have shown the efficiency and efficacyof LASP by showing consistent improvements over state-of-the-art methods.
Our work effectivelyreduces the latency while maintaining the accuracy, which could significantly impact applications inresource-constrained environments, such as autonomous vehicles or other mobile devices.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data are
proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] Please see the Section M about
the discussion of latency lookup table.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] Please see Section 4 of main paper and Section I of supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] Please see Section 4 and more details in appendix Sec.
B.
(c) Did you report error bars (e.g., with respect to the random seed after running ex-
periments multiple times)?
[No] We observed quite stable final performance of the
pruning.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Please see Section 4 and more details in appendix Sec.
B.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Please see Section 4.
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
We will release the code upon the acceptance.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
The ImageNet [45] and PASCAL VOC [12] dataset are opensource and available for non-commercial academic research.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
The ImageNet [45] and PASCAL VOC [12]
dataset are open source and available for non-commercial academic research.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]