Recently, Convolutional Neural Networks (ConvNets) have shown promising performances in many computer vision tasks, especially image-based recognition. How to effectively use ConvNets for video-based recognition is still an open problem. In this paper, we propose a compact, effective yet simple method to encode spatiotemporal information carried in 3D skeleton sequences into multiple 2D images, referred to as Joint Trajectory Maps (JTM), and ConvNets are adopted to exploit the discriminative features for realtime human action recognition. The proposed method has been evaluated on three public benchmarks, i.e., MSRC-12 Kinect gesture dataset (MSRC-12), G3D dataset and UTD multimodal human action dataset (UTD-MHAD) and achieved the state-of-the-art results.
An effective method was proposed in REF for encoding a joint's trajectories to JTMs (joint trajectory maps), where the motion information can be encoded into texture patterns, and the convolutional neural networks are used to exploit the discriminative features for real-time hand gesture recognition for the MSRC-12 Kinect gesture dataset (MSRC-12).
Action Recognition Based on Joint Trajectory Maps Using Convolutional Neural Networks
MM '16
Computer Science