## 4 Methodology
### Overview of the Deep Learning Models Used for Classification
#### 4.2.6 Vision Transformer (ViT)  
The Vision Transformer (ViT) represents a significant shift in computer vision, introducing the Transformer architecture to image classification challenges, previously dominated by CNNs [58]. By employing the attention mechanism, ViT can concentrate on various image segments, enhancing its focus and interpretability. An image is segmented into patches, encoded as tokens, and processed through Transformer layers, allowing the model to capture both local and global contexts efficiently. Additionally, the architecture of ViT is depicted in Figure 8.  
ViT begins by dividing an image into fixed-size patches. Each patch is then flattened and linearly projected into an embedding space, resulting in a sequence of image patch embeddings. Mathematically, given an input image \(I\in\mathbb{R}^{H\times W\times\mathbb{C}}\) (height, width, and channels), it is reshaped into a sequence of flattened 2D patches \(x_{p}\in\mathbb{R}^{N\times(P^{2}\cdot\mathbb{C})}\), where \(P\) is the patch size and \(N=\frac{HW}{P^{2}}\) is the number of patches. Each patch is then linearly transformed into a vector of size \(D\) using a trainable linear projection \(E\):  
\[E=[E_{1},E_{2},\dots,E_{N}], \tag{8}\]  
where \(E_{i}\in\mathbb{R}^{D}\) is the embedding of the \(i\)-th patch.  
The core component of ViT is the multi-head self-attention mechanism. For each attention head, the queries \(Q\), keys \(K\), and values \(V\) are computed as linear transformations of the input embeddings. The attention mechanism is defined as:  
Figure 7: DenseNet121 architecture.  
\[\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V, \tag{9}\]  
where \(Q,K,V\in\mathbb{R}^{N\times d_{k}}\) are the query, key, and value matrices and \(d_{k}\) is the dimension of the keys. The scaled dot-product attention computes the similarity between queries and keys, followed by a softmax operation to obtain the attention weights, which are then used to weight the values.  
The Transformer layer consists of multi-head self-attention and feed-forward neural networks, applied to the sequence of patch embeddings. Positional encodings are added to the patch embeddings to retain positional information:  
\[Z=\text{MultiHead}(E+P), \tag{10}\]  
where \(Z\) is the output of the Transformer layer, MultiHead denotes the multi-head attention mechanism, and \(P\) represents the positional encodings.  
ViT demonstrates its versatility across different scenarios by employing the _ViTForImageClassification_ model. In this model, the final representation from the Transformer encoder is fed into a classification head for binary or multi-class classification tasks. The ability of ViT to segment images into patches and apply an attention mechanism allows it to adeptly manage various scales of image features, capturing intricate details and broad contexts simultaneously. ViTs mark a promising direction in leveraging Transformer models beyond text, harnessing their power for intricate image-recognition tasks, and setting a new standard in computer vision.  
Figure 8: ViT architecture [65].