# Deep Learning for Intelligent Human-Computer Interaction
## 5 Human Gesture Recognition Based on Deep Learning
(2020) [72] proposed a skeleton-guided multimodal network for human behavior recognition. The proposed method makes full use of the complementarity of the two modes at the level of semantic features. From a technical point of view, a bootstrap block is introduced, which is a key component of the skeleton bootstrap multimodal network, and two related operation schemes are discussed. Experimental results show that the proposed method achieves the most advanced performance compared with the existing methods. The same conclusion appeared in the work of [PERSON] et al. (2020) [73]. Here, an action recognition technology based on feature fusion and optimal feature selection was implemented. First, the color transformation of hue-saturation-intensity (HIS) was carried out to improve the contrast of video frames. Then, the motion features were extracted by an optical flow algorithm. Furthermore, a new parallel method named the length control feature was extracted and fused with shape and texture features. The new weighted entropy variance was applied to combination vectors, and the best vector was selected for classification. The multimodal skeleton guidance network is better than single-modal recognition. Most existing methods using convolutional neural networks and long short-term memory have achieved promising performance for skeleton-based action recognition. However, these methods are limited in their ability to explore rich information about spatiotemporal relationships. Graph convolutional networks achieve the latest results of skeleton-based behavior recognition by extending convolutional neural networks to graphs. However, due to the lack of effective feature aggregation methods, such as maximum pooling in CNNs, existing graph convolutional network-based methods can only learn local information between adjacent joints. Moreover, it is difficult to obtain high-level interaction features, such as the interaction between the five parts of the human body. Moreover, subtle differences in confounding actions are often hidden in specific channels of key joint features, and this discriminative information has rarely been exploited in previous methods. [PERSON] et al. (2020) [74] proposed a graph convolution network based on a structural graph pool scheme and joint channel attention module. The scheme for the pool of structural maps aggregates human skeletal maps based on prior knowledge of human typology. This pooling scheme not only leads to more global representations but also reduces the number of parameters and the computational cost. The joint channel attention module learns to selectively focus on discriminative joints of the bone and give different degrees of attention to different channels. This novel attention mechanism enhances the ability of the model to classify confounding behaviors. [PERSON] et al. (2020) [75] proposed a new spatiotemporal model with an end-to-end bidirectional LSTM-CNN, which uses a hierarchical spatiotemporal dependence model to explore the rich spatiotemporal information in skeleton data. Although convolutional neural networks have achieved great success in object recognition in still images, the improvement of convolutional neural networks over traditional methods for identifying actions in videos is slight because raw videos usually have more redundant or irrelevant information than still images. This point was proven by [PERSON] et al. (2020) [76], where a spatial-temporal attentive convolutional neural network (STA-CNN) was proposed, which selects the discriminative time period and automatically pays attention to the information space region. The STA-CNN model integrates the temporal attention mechanism and spatial attention mechanism into a unified CNN to identify actions in videos. The novel time attention mechanism automatically mines distinguishable time fragments from long and noisy videos. First, the spatial attention mechanism uses instantaneous motion information in optical flow characteristics to locate the significant regions of motion.