  Network pruning can significantly reduce the computation and memory footprint
of large neural networks. To achieve a good trade-off between model size and
performance, popular pruning techniques usually rely on hand-crafted heuristics
and require manually setting the compression ratio for each layer. This process
is typically time-consuming and requires expert knowledge to achieve good
results. In this paper, we propose NAP, a unified and automatic pruning
framework for both fine-grained and structured pruning. It can find out
unimportant components of a network and automatically decide appropriate
compression ratios for different layers, based on a theoretically sound
criterion. Towards this goal, NAP uses an efficient approximation of the
Hessian for evaluating the importances of components, based on a
Kronecker-factored Approximate Curvature method. Despite its simpleness to use,
NAP outperforms previous pruning methods by large margins. For fine-grained
pruning, NAP can compress AlexNet and VGG16 by 25x, and ResNet-50 by 6.7x
without loss in accuracy on ImageNet. For structured pruning (e.g. channel
pruning), it can reduce flops of VGG16 by 5.4x and ResNet-50 by 2.3x with only
1% accuracy drop. More importantly, this method is almost free from
hyper-parameter tuning and requires no expert knowledge. You can start NAP and
then take a nap!
