# Multi-View Gait Recognition Based on a Siamese Vision Transformer
## 3 SMViT and the Gradually Moving-View Training Method  
### Model Structure  
In order to solve the problem of multi-perspective situations, this paper uses the Siamese neural network as a design basis and calculates the correlation between the characteristics of different views and uses this as the basis for the conversion of the characteristics of the view. When there are few specimens, a Siamese neural network can extract and learn the links between two groups of photos [23]. ViT is advantageous for the extraction of multi-scale features because of its robust strength and resistance to interference from mistaken samples [24,25]. In this paper, a two-channel Siamese module (Conv and MViT, CM Block) of convolution is constructed to extract the characteristics of multi-view gait contour features. The specific model structure is shown in Figure 1.  
Figure 1: Structure diagram of SMViT. In order to extract the gait information from various viewpoints, two feature extraction networks are used in the Siamese network module described in this paper. Convolution channels are used inside each module to obtain the contour's high-dimensional local features. Furthermore, we utilize the Mobile ViT channel to create high-dimensional states that are indicative long-distance attention characteristics of the current view. In addition, the mobile view transformation (MVT) module is used to extract the view characteristics and tensors, meaning that the advantages of convolution and ViT are retained in the extraction of the gait contour features. This module is based on the Mobile ViT model, and incorporates the convolution with the transformer. Local processing is replaced with deeper global treatment in the convolution. In an effective receiving domain, we model long-distance non-local dependencies. The model has smaller parameters and produces ideal experimental results. The specific details of the module are shown in Figure 2.  
As shown in Figure 2, there are two types of the conv block module (Conv Block), \(N\times N\) convolution and point-by-point convolution, according to the difference in the convolution kernel size. This module consists of a convolution layer and a batch normalization layer. Mobile Block takes MobileNet as the essential conceptual basis [26; 27; 28] and controls the network's depth and the number of parameters by constructing depth-separable convolution. In addition, due to the differences in the processing methods and content of the feature extraction between the convolution and the ViT, the format conversion needs to be carried out before and after the transformer module in order to control the data processing format. _SiLU_ is used as the activation function in each module, as shown in Equation (1), and the global average pooling layer is used in the pooling layer, as shown in Equation (2), where \(x\) means the input matrix and \(x_{w}\) represents the operation area of the pooling layer:  
\[SiLU(x)=x\cdot Sigmoid(x) \tag{1}\]  
\[Pooling(x_{w})=Avg(x_{w}) \tag{2}\]  
The transformer module absorbs some of the advantages of the convolutional computing and retains its characteristic processing capabilities in the space-perception domain. By dividing a large global receptive field into different patches in a non-overlapping way, \(P=Wh\), where \(w\) and \(h\) are the width and height of the patches, respectively. Then, the transformer is used to encode the relationship between the patches. Specifically, the self-attention mechanism calculates the scaled dot-product attention by constructing the query vector \(Q\), the value vector \(V\), and the key vector \(K\), as shown in Equations (3)-(5):  
Figure 2: Details of the module. \[f(Q,K)=\frac{QK^{T}}{\sqrt{d_{k}}} \tag{3}\]  
\[X=\textit{softmax}(f(Q,K)) \tag{4}\]  
\[Attention(X,V)=X\times V \tag{5}\]  
In this case, the module's computation cost of multi-head self-attention is \(O\big{(}N^{2}Pd\big{)}\).