# Human Action Recognition Based on Foreground Trajectory and Motion Difference Descriptors
## 1 Introduction  
Human behavior recognition is an important research area in intelligent video analysis. It is widely used in human-computer interactions, video surveillance, motion analysis, virtual reality, and other fields. Usually, the realization of video human behavior mainly includes two steps: one is feature extraction based on video information, the other is video classification based on feature vectors; however, because of the problems of background interference, partial occlusion, and different viewpoints, it is still a challenging task to obtain efficient and accurate motion representation [1; 2; 3].  
In recent years, the trajectory-based method has achieved great success in the field of behavior recognition [4; 5; 6]. Unlike the method of directly extracting local features, the trajectory-based method extracts space-time trajectories by matching feature points between adjacent frames and then representing human behavior [7; 8; 9; 10]. [PERSON] [11] used scale-invariant feature transform (SIFT) to match and track spatiotemporal context information between adjacent frames, and [PERSON] [12; 13] used the Kanade-Lucas-Tomasi (KLT) optical flow method to track feature points between adjacent frames and extract trajectories. Although these methods can capture the movement trends of behavior, there are still some problems. These are all sparse tracking methods, and sparse trajectories cannot adequately represent complex human behavior. In order to solve this problem, [PERSON] [14] used dense optical flow to sample dense feature points on regular grids, forming dense trajectory (DT). Four descriptors, histogram of oriented gradients (HOGs), optical flow (HOF), motion boundary histograms (MBHs), and trajectory shapes (TSs), are used to represent human behavior [15; 16]. However, [PERSON] [17] foundthat accuracy of human behavior recognition was maintained only by randomly selecting 30%-40% of descriptors in dense trajectory-based methods. This means that although dense trajectory descriptors can obtain a lot of information, most of them are redundant or even irrelevant.  
In view of the above analysis, a human behavior recognition method is proposed based on foreground trajectory and motion differential descriptors. Firstly, the optical flow method is used to estimate the motion amplitude of each frame, and the foreground area is determined according to the motion amplitude of the pixels. Based on the extracted foreground area, the initial sampling point of the trajectory is placed, which reduces interference information caused by the background compared with the global dense trajectory. Secondly, in order to better describe the relative time information between different behaviors, a motion difference descriptor (MDD) is introduced to quantify the foreground trajectory. A direction histogram of motion difference is constructed by calculating direction information of the motion difference per unit time of the trajectory point. Finally, a behavior descriptor based on motion difference is constructed. A Fisher vector (FV) is used to encode the local features to the global features at the video level, and the features are inputted into a support vector machine (SVM) to determine the behavior category. Experimental results show that the proposed method can extract the trajectory more accurately than the traditional optical flow method and, thus, achieve higher recognition accuracy.