# HandFormer: A Dynamic Hand Gesture Recognition Method Based on Attention Mechanism
## 1 Introduction  
With the increasing prevalence of artificial intelligence in both the manufacturing and service industries, intelligent manufacturing and intelligent services have emerged as the prevailing trends of future development. Within human-machine interaction, gestures have become a vital means of information communication, and are therefore a critical aspect to consider. Driven by the sensory inputs of sound and vision, the use of gestures is considered to be the most effective and powerful method in this context [1]. Gesture recognition has a wide range of practical applications, including remote automation equipment operation, intelligent vehicles, remote medical care, remote teaching guidance, and VR/AR (Virtual Reality/Augmented Reality) game interaction [2]. In addition, the gesture recognition community pays special attention to understanding the ways to improve the quality of life of hearing-impaired people, and gesture recognition can be used in sign language recognition and translation tasks for hearing-impaired people [3]. Numerous studies on gesture recognition have been conducted, and significant progress has been made in this field. Gesture recognition can be categorized into static gesture recognition and dynamic gesture recognition. Static gesture recognition only requires learning and classifying thespatial characteristics of the gesture, without considering the temporal characteristics. In contrast, dynamic gesture recognition requires taking into account both the spatial and temporal characteristics of the gesture as it changes over time. As a result, dynamic gesture recognition is significantly more complex than static gesture recognition, but the application of dynamic gestures is more extensive. This paper provides a lightweight gesture action recognition network for real-time human-computer interaction and control such as smart car interaction. Despite the great practical potential of effective gesture recognition, it is still an unsolved problem due to the significant differences in the semantic and syntactic structure of gestures. Therefore, for sign language recognition technology for hearing-impaired people, explicit translation from sign language to textual representation is currently an almost unsolvable problem. As of now, there is no fully automatic model or method for the recognition system of many dynamic gestures. Developing such a model requires deep semantic analysis, which can only be carried out at a superficial level due to the imperfection of current text analysis algorithms and knowledge bases, and can only achieve word-level recognition, so the proposed method also has some reference significance for the sign language recognition of hearing-impaired people. In order to address the above problems, we propose a lightweight CNN and Transformer [4] fusion model based on spatial attention and temporal attention mechanisms. The model is mainly divided into three parts. The first part is the fast down-sampling PoolFormer model [5] to roughly extract the spatial features of gestures and compress them. The second part is the Spatial Attention Former Block based on a spatial attention mechanism to extract the spatial features of gestures. The third part is the Temporal Attention Former Block based on temporal attention to extract the temporal features of gestures. In summary, the main contributions of this paper are as follows:  
1. This paper proposes a deep neural network that fuses a CNN and Transformer model by embedding spatial attention convolution and temporal attention convolution into the Transformer model. 2. The spatial features of each frame of the input gesture are extracted, and the temporal features are fused into the channel information in the channel dimension; then, the temporal and spatial features of the temporal features are extracted. 3. The proposed method is tested on three datasets.