Recent state-of-the-art methods adopt a 3D CNN stream to learn spatiotemporal features and another flow stream to learn motion features. In this work, we aim to efficiently encode these two features in a unified 2D framework. To this end, we first propose an STM block, which contains a Channel-wise SpatioTemporal Module (CSTM) to present the spatiotemporal features and a Channel-wise Motion Module (CMM) to efficiently encode motion features. We then replace original residual blocks in the ResNet architecture with STM blcoks to form a simple yet effective STM network by introducing very limited extra computation cost. Extensive experiments demonstrate that the proposed STM network outperforms the state-of-the-art methods on both temporal-related datasets (i.e., with the help of encoding spatiotemporal and motion features together.
STM (Spatio-Temporal and Motion Encoding) REF designs two module, specifically, a channel-wise spatio-temporal module to encode spatiotemporal feature and a channel-wise motion module to learn motion feature.
STM: SpatioTemporal and Motion Encoding for Action Recognition
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Computer Science