Action Recognition Based on 3D Skeleton and RGB Frame Fusion Action recognition has wide applications in assisted living, health monitoring, surveillance, and humancomputer interaction. In traditional action recognition methods, RGB videobased ones are effective but computationally inefficient, while skeletonbased ones are computationally efficient but do not make use of lowlevel detail information. This work considers action recognition based on a multimodal fusion between the 3D skeleton and the RGB image. We design a neural network that uses a 3D skeleton sequence and a single middle frame from an RGB video as input. Specifically, our method picks up one frame in a video and extracts spatial features from it using two attention modules, a selfattention module and a skeletonattention module. Further, temporal features are extracted from the skeleton sequence via a BILSTM subnetwork. Finally, the spatial features and the temporal features are combined via a feature fusion network for action classification. A distinct feature of our method is that it uses only a single RGB frame rather than an RGB video. Accordingly, it has a lightweighted architecture and is more efficient than RGB videobased methods. Comparative evaluation on two public datasets, NTURGBD and SYSU, demonstrates that, our method can achieve competitive performance compared with stateoftheart methods.

Data Augmentation For CNNBased 3D Action Recognition on SmallScale Datasets Videobased human action recognition recently plays a vital role in many industrial applications thanks to the popularity of depth sensors. A large number of conventional approaches, which have combined handcrafted features and traditional classifiers, cannot deal with various challenges in the field such as the complexity of human actions in the realistic environment. In order to improve recognition performance by exploiting more highlevel discriminative features, an efficient skeletonbased action recognition method using deep convolutional neural networks (CNNs) is studied with an image encoder to transform skeleton coordinate data to imageformed data. Since deep learning techniques are fundamentally designed for efficiently working with large datasets, the network overfitting usually occurs if training CNNs on smallscale datasets. To address this issue, a novel data augmentation technique is proposed for both the informative enrichment and overfitting prevention, wherein a skeleton sequence is depicted by manifold action images based on randomly adding some skeleton frames during the data transformation and preparation for the training set. Experimental results on several smallscale challenging datasets demonstrate that the proposed method outperforms stateoftheart approaches in terms of action recognition accuracy.