# HandFormer: A Dynamic Hand Gesture Recognition Method Based on Attention Mechanism
## 3 Method
### MetaFormer  
PoolFormer has demonstrated that the high performance of Transformer-like models mainly originates from the powerful ability of the MetaFormer structure [5]. The MetaFormer structure is shown in Figure 2a, where the input of MetaFormer is first passed through a patch embedding:  
\[F_{pe}=PatchEmbedding(I). \tag{1}\]  
Figure 1: Framework of the proposed method.  
Figure 2: Comparison of MetaFormer block, Spatial Attention Former Block, and Temporal Attention Former Block. (**a**) MetaFormer Block. (**b**) Spatial Attention Former Block (SAFB). (**c**) Temporal Attention Former Block (TAFB).  
Here, the formula shows that the tokenized feature map \(F_{pe}\) is obtained after the input data are passed through the patch embedding, where I \(\in R^{H\times W\times C}\) is the input of MetaFormer, which is a feature map with size \(H\times W\) and C channels. The patch embedding layer converts the feature map into a tokenized feature map \(F_{pe}\). The patch embedding technique is typically implemented by applying a \(3\times 3\) convolutional operation with a padding size of 1 and a stride size of 1. This approach ensures that the size of the feature map remains unchanged before and after the operation. The tokenized feature map is then passed through the MetaFormer block, which consists of two residual connection blocks. The first residual connection block includes a normalization layer and a token mixer layer that can learn features, as well as a skip connection, which can be expressed as  
\[F_{tf}=F_{pe}+TokenMixer(Norm(F_{pe})). \tag{2}\]  
The above equation represents the first residual connection block in the MetaFormer architecture, where \(Norm(\cdot)\) represents a normalization layer, such as layer normalization or batch normalization. \(TokenMixer(\cdot)\) is the core block in MetaFormer that can learn fine-grained features from the input image. It can be implemented using a convolutional neural network or an MLP (Multilayer Perceptron) mixer. The second residual connection block consists of a Norm layer, an MLP layer with a learnable channel expansion, and an activation function such as Gelu or Relu, as well as a skip connection:  
\[F_{mf}=F_{tf}+\sigma(MLP(Norm(F_{tf}))). \tag{3}\]  
The above equation represents the second residual connection block in the MetaFormer architecture, where \(MLP(\cdot)\) is a learnable channel expansion layer that serves as an activation function to enhance the non-linear expressive capacity of the model, such as Gelu or Relu.  
Inspired by MetaFoormer, we designed a network model for dynamic gesture recognition based on a spatial attention mechanism and temporal attention mechanism. Each stacked block in this network model is designed based on the MetaFormer block, in which the first stage of the network is replaced with a PoolFormer block that uses the simplest non-learnable parameter pooling operation to replace the token mixer in MetaFormer. The second and third stages are replaced with the Spatial Attention Former Block (SAFB) that extracts spatial feature information using the proposed convolutional operations based on the spatial attention mechanism, and the final stage is replaced with the Temporal Attention Former Block (TAFB) that extracts temporal feature information using the proposed convolutional operations based on the temporal attention mechanism, both of which replace the token mixer in MetaFormer.