  Neural network pruning is an essential approach for reducing the
computational complexity of deep models so that they can be well deployed on
resource-limited devices. Compared with conventional methods, the recently
developed dynamic pruning methods determine redundant filters variant to each
input instance which achieves higher acceleration. Most of the existing methods
discover effective sub-networks for each instance independently and do not
utilize the relationship between different inputs. To maximally excavate
redundancy in the given network architecture, this paper proposes a new
paradigm that dynamically removes redundant filters by embedding the manifold
information of all instances into the space of pruned networks (dubbed as
ManiDP). We first investigate the recognition complexity and feature similarity
between images in the training set. Then, the manifold relationship between
instances and the pruned sub-networks will be aligned in the training
procedure. The effectiveness of the proposed method is verified on several
benchmarks, which shows better performance in terms of both accuracy and
computational cost compared to the state-of-the-art methods. For example, our
method can reduce 55.3% FLOPs of ResNet-34 with only 0.57% top-1 accuracy
degradation on ImageNet.
