  The concept of compressing deep Convolutional Neural Networks (CNNs) is
essential to use limited computation, power, and memory resources on embedded
devices. However, existing methods achieve this objective at the cost of a drop
in inference accuracy in computer vision tasks. To address such a drawback, we
propose a framework that leverages knowledge distillation along with
customizable block-wise optimization to learn a lightweight CNN structure while
preserving better control over the compression-performance tradeoff.
Considering specific resource constraints, e.g., floating-point operations per
inference (FLOPs) or model-parameters, our method results in a state of the art
network compression while being capable of achieving better inference accuracy.
In a comprehensive evaluation, we demonstrate that our method is effective,
robust, and consistent with results over a variety of network architectures and
datasets, at negligible training overhead. In particular, for the already
compact network MobileNet_v2, our method offers up to 2x and 5.2x better model
compression in terms of FLOPs and model-parameters, respectively, while getting
1.05% better model performance than the baseline network.
