Finger Gesture Spotting from Long Sequences Based on MultiStream Recurrent Neural Networks Gesture spotting is an essential task for recognizing finger gestures used to control incar touchless interfaces. Automated methods to achieve this task require to detect video segments where gestures are observed, to discard natural behaviors of usersxe2x80x99 hands that may look as target gestures, and be able to work online. In this paper, we address these challenges with a recurrent neural architecture for online finger gesture spotting. We propose a multistream network merging hand and handlocation features, which help to discriminate target gestures from natural movements of the hand, since these may not happen in the same 3D spatial location. Our multistream recurrent neural network (RNN) recurrently learns semantic information, allowing to spot gestures online in long untrimmed video sequences. In order to validate our method, we collect a finger gesture dataset in an invehicle scenario of an autonomous car. 226 videos with more than 2100 continuous instances were captured with a depth sensor. On this dataset, our gesture spotting approach outperforms stateoftheart methods with an improvement of about 10% and 15% of recall and precision, respectively. Furthermore, we demonstrated that by combining with an existing gesture classifier (a 3D Convolutional Neural Network), our proposal achieves better performance than previous hand gesture recognition methods.

Multipath Convolutional Neural Network based on Rectangular Kernel with Path Signature Features for Gesture Recognition Skeleton based gesture recognition has gained more attention due to its wide application and largescale databases availability. Recent methods designed for skeleton sequence data mainly pay attention to network architecture but ignore an essential characteristic of skeleton sequences that the temporal dimensionality of skeleton sequences is usually higher than its spatial dimensionality. Directly applying CNNs designed for image classification to skeletonbased data can not capture this unique property. Considering this fact, we propose the rectangular convolution and pooling to skeleton sequence data. Temporal features are crucial for gesture action recognition. Further, we introduce path signature features (PSF) to represent temporal variation characteristics of each joint. Moreover, there only exist a few minor distinctions between some gestures. To classify them more accurately, we add two subnetworks to extract discriminative features from two hands respectively. We evaluate our method on three major benchmark gesture datasets, i.e., ChaLearn 2013, ChaLearn 2016 and MSRC12, and reach the stateoftheart performance.