# Content-Adaptive and Attention-Based Network for Hand Gesture Recognition
## 1 Introduction  
Hand gestures are a form of non-vocal communication that communicate particular messages via visible hand movements. Compared to other physical input devices, hand gestures provide a more natural and convenient way for humans to interact with devices. Computer vision-based gesture recognition is a form of technology that uses a computing module to read and interpret hand movements as commands [1; 2; 3; 4; 5; 6]. Computer vision-based gesture recognition technology can be used in several industries, such as interactive entertainment, smart home, VR/AR, and sign language machine translation [7; 8; 9; 10; 11; 12; 13]. Due to its wide application in several industries, hand gesture recognition based on computer vision technology has received great attention from researchers in the computer vision community in recent years. Gestures in a video consist of spatial information in each frame and temporal information in neighboring frames. Capturing motion information between neighboring frames from sequence data is a key challenge for gesture recognition tasks. For example, the operation of zooming in and out of the screen with fingers has similar features in spatial domains, but the temporal information is reversed. [PERSON] et al. proposed a method using a two-stream network [14], which combines a color image frame and multiple stacked optical flow frames to predict action classes. As an extension to the two-stream method, [PERSON] et al. presented a temporal segment network (TSN) [15] based on long-range temporal structure modeling. The main idea of TSN is that we can divide the long video into several equal segments, process each segment individually, and then obtain a segment consensus from each segment and perform the final prediction. Despite the performance gains obtained by TSN, this still relies on optical flow features that need to be precomputed. Somework [16; 17; 18] has attempted to use a 3D convolution approach to learn spatial-temporal features in place of optical flow. In these methods, continuous frames from a video clip are stacked and then fed into a network for spatial-temporal modeling. Three-dimensional convolution neural networks (CNNs) have sufficient capacity to encode spatial-temporal information in densely sampled frames, but the huge number of parameters to be optimized leads to expensive computational consumption. Recurrent neural networks (RNNs), which consist of recurrent units, are another kind of neural network used to process sequential data [19; 20]. For any moment in the sequence, there is a recurrent unit corresponding to it, which will fuse the input of the current moment and output of the recurrent unit of the previous moment to calculate the output of the current moment. Each recurrent unit of an RNN has forward dependency, i.e., the output of the network at the current moment is jointly determined by the previous sequences. This property allows the past information of the sequence to be continuously transmitted, but also causes a decrease in the efficiency of model operation. Assuming that the length of the input sequence is \(n\), the minimum sequential operation required by the recurrent neural network layer is \(O(n)\). Since there are \(O(n)\) sequential operations, RNNs cannot be operated in parallel, and this sequential modeling approach leads to information loss during long-distance passing. Since 2017, a new model architecture called transformer [21] has received significant attention due to its performance beyond RNNs in many natural language processing (NLP) tasks. Transformers use a mechanism called self-attention to build sequence models that do not rely on any convolution or recurrent layers. Table 1 compares the minimum number of sequential operations, maximum path length, and complexity for CNNs, RNNs, and transformer layers [21]. As shown in Table 1, self-attention has both an advantage of shortest maximum path length and minimum sequential operation.