Multipath Convolutional Neural Network based on Rectangular Kernel with Path Signature Features for Gesture Recognition Skeleton based gesture recognition has gained more attention due to its wide application and largescale databases availability. Recent methods designed for skeleton sequence data mainly pay attention to network architecture but ignore an essential characteristic of skeleton sequences that the temporal dimensionality of skeleton sequences is usually higher than its spatial dimensionality. Directly applying CNNs designed for image classification to skeletonbased data can not capture this unique property. Considering this fact, we propose the rectangular convolution and pooling to skeleton sequence data. Temporal features are crucial for gesture action recognition. Further, we introduce path signature features (PSF) to represent temporal variation characteristics of each joint. Moreover, there only exist a few minor distinctions between some gestures. To classify them more accurately, we add two subnetworks to extract discriminative features from two hands respectively. We evaluate our method on three major benchmark gesture datasets, i.e., ChaLearn 2013, ChaLearn 2016 and MSRC12, and reach the stateoftheart performance.

Gesture Correctness Estimation with Deep Neural Networks and Rough Path Descriptors One of the classical problems in computer vision is the automatic identification of different gestures or motions from a sequence of frames, captured via cameras or sensors, that usually contain angles or positions of many body joints. A large body of research addressing this problem exists in the literature. However, an interesting variant is not just to identify the gesture, but to decide whether a certain gesture is correctly or incorrectly executed. This can be in the context of physical exercises for rehabilitation in healthcare, robotics, sports, sign language and more. In this work we present our research on the automatic classification of gestures into correct or incorrect categories using the following three different but powerful techniques: convolutional neural networks (CNNs), long shortterm memory (LSTM) neural networks and a technique based on rough path theory to extract unique sequences (i.e. path signatures) from gesture data and to use them as inputs to classical classifiers such as RandomForest (RF) and kNearest Neighbor (kNN). We collected the data we used as part of a collaborative project from Perkeso Tun Razak Rehabilitation Centre, Malacca, Malaysia. We recorded 17 patients performing nine types of gestures. For controls, we recorded 14 healthy individuals performing the same gestures. Our cross subject evaluation yields high accuracy for several subjects and gestures. CNNs and LSTMs achieve accuracy values up to 100% for several subjects in many gestures with the highest average accuracy for some gestures reaching 92%. The best overall classifier when using the rough paths representation was kNN as its average accuracy can reach 96.74% (100% accuracy for a subset of subjects in some gestures). For the three methods, the average accuracy across all gestures is 85.24%, 85.63% and 94.67% respectively. Overall, our research shows a great potential in identifying the right learning algorithm in motion correctness identification for real data.