# Multi-Scale Receptive Fields Convolutional Network for Action Recognition
## 2 Related Work  
Video action recognition is one of the representative tasks for video understanding, which has drawn significant attention from researchers over the last decade. The existing methods of action recognition can be summarized into two kinds of methods: the methods based on hand-crafted features and the methods based on deep learning models. Although deep learning methods have become the standard for action recognition, hand-crafted features dominated the video understanding literature before 2015 due to their high accuracy and good robustness. Hand-crafted features can be divided into global features and local features. Global features provide a holistic description of the motion target. For example, [PERSON] et al. [20] first used contour information as global features to describe human action, and proposed two temporal template features, motion energy image (MEI) and motion history image (MHI). The former indicates the part of the action that occurs, while the latter reflects the sequence of the action. Unlike global features, local features describe only part of the regional information of the motion target and are more widely used. [PERSON] et al. [21] extended the two-dimensional Harris corner point detector to three-dimensional space, thus detecting the most drastically changing space-time interest points (STIP) in the spatio-temporal dimension, which promotes the development of local features. Later, various feature descriptors were successively proposed to describe the action features in the vicinity of STIP, such as histogram of oriented gradients (HoG) [1], histogram of oriented optical flow (HoF) [2], and motion boundary histograms (MBH) [22]. The state-of-the-art hand-crafted feature is based on dense trajectory [3; 4], particularly improved dense trajectories (IDT) [4], which obtains the action trajectory by calculating dense optical flow and then extracting local features along the trajectory. However, hand-crafted features have a heavy computational cost and are hard to scale and deploy. In recent years, deep learning has made significant breakthroughs in image classification [23], semantic segmentation [24], object detection [25], and other fields. Action recognition methods based on deep learning have gradually become the focus of research. Unlike hand-crafted features, deep learning models can automatically learn feature representations from video data, and the learned features are more general. The common deep learning method currently used in the field of vision is ConvNet. The seminal work was proposed by [PERSON] et al. [26], which uses a single 2D ConvNet on each video frame independently and investigates late fusion, early fusion, and slow fusion to find the best temporal connectivity patterns. However, its performance on UCF101 is 20% lower than IDT. To learn better spatio-temporal features, [PERSON] et al. [5] proposed two-stream networks for action recognition. This method includes two independent 2D ConvNets. One takes RGB frames as input to capture appearance information, called the spatial stream, while the other takes a stack of optical flow images as input to capture motion information between video frames, called the temporal stream. The prediction scores from the two streams are then averaged to obtain the final prediction. By feeding the extra optical flow information to ConvNets, ConvNet-based approaches achieved similar performance to the previous best hand-crafted feature IDT. However, optical flow cannot capture long-range temporal information due to its characteristics. To tackle this problem, [PERSON] et al. [14] proposed the temporal segment network (TSN). TSN divides a whole video into several segments, and all the segments share the same two-stream networks. Then, a segmental consensus, such as average pooling and max pooling, is performed to aggregate information from the sampled segments. Thanks to this sparse sampling strategy, TSN is able to see the content from the entire video, thus modeling long-range temporal information.