  Frame sampling is a fundamental problem in video action recognition due to
the essential redundancy in time and limited computation resources. The
existing sampling strategy often employs a fixed frame selection and lacks the
flexibility to deal with complex variations in videos. In this paper, we
present an explainable, adaptive, and effective frame sampler, called
Motion-guided Sampler (MGSampler). Our basic motivation is that motion is an
important and universal signal that can drive us to select frames from videos
adaptively. Accordingly, we propose two important properties in our MGSampler
design: motion sensitive and motion uniform. First, we present two different
motion representations to enable us to efficiently distinguish the motion
salient frames from the background. Then, we devise a motion-uniform sampling
strategy based on the cumulative motion distribution to ensure the sampled
frames evenly cover all the important frames with high motion saliency. Our
MGSampler yields a new principled and holistic sample scheme, that could be
incorporated into any existing video architecture. Experiments on five
benchmarks demonstrate the effectiveness of our MGSampler over previously fixed
sampling strategies, and also its generalization power across different
backbones, video models, and datasets.
