Dynamic gesture recognition and human-computer interaction

In this paper, the recognition algorithm of dynamic gestures, which based on depth information, is proposed. A new human-computer interaction controlled by dynamic gestures is also achieved. The proposed algorithm extracts Histogram of Oriented Gradients (HOG) and weighted Hu invariant moments from RGB and depth map of the interested region of gestures respectively, and fuses weighted Hu invariant moments into HOG. Then, we establish Support Vector Machine model (SVM) for static hand gestures recognition. Base on this, an HMM toolbox supporting multi-dimensional continuous data input is adopted to do the training and dynamic gesture recognition. Finally, the visual simulation software Vega Prime is applied to simulate a virtual UAV flight controlled by dynamic gestures. Experimental results demonstrate that the proposed algorithm is effective and accurate in complex background and practical real-time applications. Introduction Recently, with the rapid development of computer vision technology, the growing popularity of the human-computer interaction has led to a revolution in gesture recognition, and Human-computer interaction based on dynamic gesture recognition is also becoming more popular. In this paper, the recognition algorithm of dynamic gestures, which based on depth information, is proposed. The algorithm mainly consists of feature extraction, fusion and HMM classifier. Finally, the three-dimensional modeling software Creator is applied to build 3D models and visual simulation software Vega Prime to simulate, achieving a visual simulation of flight system controlled by dynamic gestures based on the proposed algorithm. Dynamic gesture recognition In this paper, two features combined with the SVM classifier is applied to identify the different types of static gestures, and the dynamic gesture recognition is achieved by HMM training. The two features are HOG and weighted Hu invariant moments extracted from RGB and depth map of the interested region of gestures. Gesture feature extraction and fusion. Unlike other image geometric features, HOG does not consider the image characteristics from a whole, but subdivides image into several small cell units, and then calculates the gradient or edge direction histogram of each pixel in all cells. To improve performance, a number of cells making up a block, thus the image becomes a connected graph consisting of many blocks, then normalizes the gradient of all cells in these blocks and obtains the final direction of gradient vector. A brief flowchart of extracting the HOG is given in Fig.1. Input image HOG feature vector Calculate the gradient of the horizontal and vertical direction The gradient direction histogram statistics within the Cell The gradient direction histogram statistics within the Block Fig.1 A brief flowchart of extracting the HOG The basic idea in Ref. [1] is used in this paper, and makes improvements according to gestures itself characteristics. As shown in Fig. 2, considering the differences mainly in the finger part, RGB image of gesture obtained through kinect is divided into five regions, this system only extracts HOG feature in (2), (3), (4) area shown in Fig. 2. International Industrial Informatics and Computer Engineering Conference (IIICEC 2015) © 2015. The authors Published by Atlantis Press 1836 Fig.2 Gesture area images In order to get the overall shape and contour characteristic, the Hu invariant moments of the binary image is selected as a gesture feature parameter in this page. Hu invariant moment is a good solution to solve the scale changes, image translation changes, rotation of the coordinate transformation in the process. Due to different Hu moment's contribution to the behavior criterion value is different in the feature extraction, so should take full account of the contribution values of different moment to construct a weighted Hu moment. In this paper, nine extended Hu moments are modified in linear programming to find the optimal weights for each Hu moment, and then construct a weighted modified Hu moments. Set a new weighted fixed Hu structure as: 9 '' '' '' 1 1 2 2 9 9 1 , 1,0 1 i i i σ σ σ σ σ = Φ = Φ + Φ + + Φ = < < ∑  (1) The specific solving process of the weighted Hu moments is presented in Ref. [2]. In order to better describe the gesture feature, this page refers to Ref. [3,4], and propose fusing HOG features and weighted Hu moment feature to describe gesture image. Before the feature fusion, HOG vectors HOG F and weighted Hu moments vector Hu F are normalized to ' HOG F and ' Hu F with the largest-minimum principle. After normalization, we fuse two kinds of features with corresponding weights represented as F in ' ' 1 2 1 2 [ , ], 1. HOG Hu F w F w F w w = + = (2) A whole process of the proposed algorithm is shown in Fig. 3. Fig.3 The overall process of feature fusion Gesture recognition and tracking. Feature after fusion is regarded as the input of SVM. SMO (Sequential Minimal Optimiz-ation) algorithm is selected for training in SVM, and Gaussian kernel function for inner product operations. Then the results of SVM training regarded as the HMM state vector , and select the center of gravity of adjacent palm direction vector as an HMM 's output observation variable, to train and recognize dynamic gestures. Human-Computer Interaction With the advanced human-computer interaction technology proposal and development, human-computer interaction using gesture recognition technology has become a hot research topic. In this paper, three-dimensional modeling software Creator builds all entity object models and large terrain model required by visual simulation of flight system. Visual simulation software Vega prime