  Although 3D Convolutional Neural Networks are essential for most learning
based applications involving dense 3D data, their applicability is limited due
to excessive memory and computational requirements. Compressing such networks
by pruning therefore becomes highly desirable. However, pruning 3D CNNs is
largely unexplored possibly because of the complex nature of typical pruning
algorithms that embeds pruning into an iterative optimization paradigm. In this
work, we introduce a Resource Aware Neuron Pruning (RANP) algorithm that prunes
3D CNNs at initialization to high sparsity levels. Specifically, the core idea
is to obtain an importance score for each neuron based on their sensitivity to
the loss function. This neuron importance is then reweighted according to the
neuron resource consumption related to FLOPs or memory. We demonstrate the
effectiveness of our pruning method on 3D semantic segmentation with widely
used 3D-UNets on ShapeNet and BraTS'18 datasets, video classification with
MobileNetV2 and I3D on UCF101 dataset, and two-view stereo matching with
Pyramid Stereo Matching (PSM) network on SceneFlow dataset. In these
experiments, our RANP leads to roughly 50%-95% reduction in FLOPs and 35%-80%
reduction in memory with negligible loss in accuracy compared to the unpruned
networks. This significantly reduces the computational resources required to
train 3D CNNs. The pruned network obtained by our algorithm can also be easily
scaled up and transferred to another dataset for training.
