ResFlow: Multitasking of Sequentially Pooling Spatiotemporal Features for Action Recognition and Optical Flow Estimation Since deeplearningbased method has been widelyused and is capable of generating generic model, most existing methods about action recognition use either twostream structure, considering spatial and temporal features separately, or C3D, costing lots of prices in memory and time. We aim to design a robust system to extract spatiotemporal features with aggregation mechanism to integrate local features in temporal order. In light of this, we propose ResFlow to estimate optical flow and predict action recognition simultaneously. Leveraging the characteristic of optical flow estimation, we extract spatiotemporal feature via an autoencoder. Via a novel Sequentially Pooling Mechanism which literally pool global spatiotemporal feature sequentially, we extract spatiotemporal feature at each time and aggregate these local features into global feature. This design use only RGB images as input with temporal information encoded, pretrained by optical flow, and sequentially aggregate spatiotemporal features in high efficiency. We evaluate our ability of estimating optical flow on FlyingChairs dataset and show the promising results of action recognition on UCF101 dataset through a series of experiments.

A Spatiotemporal Hybrid Network for Action Recognition Convolutional Neural Networks (CNNs) are powerful in learning spatial information for static images, while they appear to lose their abilities for action recognition in videos because of the neglecting of longterm motion information. Traditional 3D convolution has high computation complexity and the used Global Average Pooling (GAP) on the bottom of network can also lead to unwanted content loss or distortion. To address above problems, we propose a novel action recognition algorithm by effectively fusing 2D and Pseudo3D CNN to learn spatiotemporal features of video. First, we use Pseudo3D CNN with proposed Multilevel pooling module to learn spatiotemporal features. Second, the features output by multilevel pooling module are passed through our proposed processing module to make full use of the rich features. Third, a 2D CNN fed with motion vectors is designed to extract motion patterns, which can be regarded as a supplement of Pseudo3D CNN to make up for the information lost by RGB images. Fourth, a dependencybased fusion method is proposed to fuse the multistream features. Finally, the effectiveness of our proposed action recognition algorithm is demonstrated on public UCF101 and HMDB51 datasets.