Learning Sparse Patterns in Deep Neural Networks Acceleration in deeper neural networks is hindered by high demand for computation and memory storage in resource constrained devices. In this paper, we propose a novel regularization method to learn hardwarefriendly sparse structures for deep convolutional neural networks. A limited number of fixed sparse patterns are learned through the regularization and the filters are represented by these fixed sparse patterns. Our regularization method can produce compact models with higher degree of sparsity than the stateoftheart structured sparsity learning method. The limited number of fixed sparse patterns also facilitate the hardware implementation. Our experiments on several deep neural networks with image classification tasks show that our proposed approach can achieve remarkably higher degree of sparsity with acceptable loss of accuracy.

Identifying and Pruning Redundant Structures for Deep Neural Networks Deep convolutional neural networks have achieved considerable success in the field of computer vision. However, it is difficult to deploy stateoftheart models on resourceconstrained platforms due to their high storage, memory bandwidth, and computational costs. In this paper, we propose a structured pruning method which employs a threestep process to reduce the resource consumption of neural networks. First, we train an initial network on the training set and evaluate it on the validation set. Next, we introduce an iterative pruning and finetuning algorithm to identify and prune redundant structures, which results in a pruned network with a compact architecture. Finally, we train the pruned network from scratch on both the training set and validation set to obtain the final accuracy on the test set. In the experiments, our pruning method significantly reduces the model size (by 87.2% on CIFAR10), saves inference time (53.3% on CIFAR10), and achieves better performance as compared to recent stateoftheart methods.