## 3 Correlation Algorithm
### Vision Transformer  
The Google team introduced the vision transformer (ViT) model in 2021, aiming to leverage the transformer's capabilities for image classification tasks [29]. While previous research had explored applying transformers to visual tasks, ViT stands out as a significant milestone in the field of computer vision due to its simplicity, impressive performance, and scalability (larger models lead to better results). The release of ViT has triggered a surge of interest in transformer-based approaches for various visual tasks, including object detection and image generation. Figure 4 shows a visual representation of the ViT architecture. Based on the flowchart of ViT, a ViT block can be partitioned into the following stages:  
1. Patch Embedding: The input image is partitioned into fixed-size patches, with each patch having dimensions of 16 \(\times\) 16 pixels. Each patch is mapped to a fixed-dimensional vector using a linear projection layer. This way, the image is represented as a sequence where each patch becomes a token with a dimension of 768. An additional special token called CLS is added as the starting marker of the sequence, resulting in a final sequence dimension of 197 \(\times\) 768. 2. Positional Encoding: To capture the positional information of the patches in the image, ViT introduces positional encoding. Positional encoding is a table with the same dimension as the input sequence embedding, where each row represents a position'svector. By adding the positional encoding to the input sequence embedding, the positional information is fused into the sequence. Thus, the sequence's dimensions remain 197 \(\times\) 768. 3. Layer Normalization and Multi-Head Attention: ViT employs a multi-head self-attention mechanism to process the sequence. First, the input sequence is mapped to queries (q), keys (k), and values (v). If there is only one attention head, the dimensions of q, k, and v are all 197 \(\times\) 768. If there are multiple attention heads (e.g., 12 heads with each head having a dimension of 64), the dimensions of q, k, and v are 197 \(\times\) 64, and there are 12 sets of q, k, and v. These sets of q, k, and v are then concatenated together, resulting in an output dimension of 197 \(\times\) 768. The output is then layer-normalized, ensuring that each feature dimension has a similar distribution across different positions in the sequence. 4. MLP: The sequence is further processed using a multi-layer perceptron (MLP) [30]. The sequence undergoes a linear transformation layer, expanding the dimension to 197 \(\times\) 3072. Then, an activation function and another linear transformation layer are applied to reduce the dimension back to 197 \(\times\) 768. This MLP structure introduces non-linear relationships and performs more complex feature transformations. \[z_{0}=\left[X_{class};X_{p}^{1}E;X_{p}^{2}E;\cdots;X_{p}^{N}E\right]+E_{pos},E \in\mathbb{R}^{(P^{2}\text{-}\text{C})\times D},E_{pos}\in\mathbb{R}^{(N+1) \times D} \tag{1}\]  
\[z_{l}^{\prime}=MSA(LN(z_{l-1}))+z_{l-1},\ \ \ l=1\ldots L \tag{2}\]  
\[z_{l}=MLP\big{(}LN\big{(}z_{l}^{\prime}\big{)}\big{)}+z_{l}^{\prime},\ \ \ \ \ \ \ l=1\ldots L \tag{3}\]  
\[\text{y}=LN\Big{(}z_{L}^{0}\Big{)} \tag{4}\]  
The above-described steps are the fundamental procedures outlined in the ViT, where the dimension after each block remains the same as the input, i.e., 197 \(\times\) 768. The depth of the model can be increased by stacking multiple blocks. In ViT, the encoder's final output, which corresponds to the special token \"CLS\", serves as the image's representation. For image classification tasks, this output can be passed to an MLP for further classification. Figure 4: Architecture diagram of vision transformer.