## 3 Materials and Methods
### Proposed Vision Transformer Model for LU/LC Classification  
A transformer is a deep-learning model that has emerged through the self-attention mechanism. The transformer follows the encoder-decoder architecture by processing the sequential data parallelly without depending on any recurrent network. It has been widely used in the scientific fields of NLP and computer vision. The Vision Transformer architecture has attracted an interesting view from researchers in recent years by showing good performance in the area of machine- and deep-learning applications. The Vision Transformer has been used in the area of image classification for providing state-of-the-art performance and to outperform the standard classification models. The Vision Transformer develops the encoder module of the transformer for performing the image classification by representing the sequence of image patches to the classified label. The attention mechanism of the Vision Transformer goes through all areas of the image and integrates the information into the full-sized image [47; 48; 49; 50; 51]. The end-to-end Vision Transformer model for the classification of satellite images is shown in Figure 6. The Vision Transformer classification model has experimented with the preprocessed LISS-III satellite image of Javadi Hills for the years 2012 and 2015. The Vision Transformer architecture is composed of an embedding, encoder, and classifier layer. Equations (1) and (2) represent the first step of analyzing and dividing the training images into a sequence of patches. Figure 4: Preprocessed Landsat RED bands of Javadi Hill for the years (**a**) 2012 and (**b**) 2015. Figure 5: Preprocessed Landsat NIR bands of Javadi Hill for the years (**a**) 2012 and (**b**) 2015. Let \(S_{i}\) represent a set of training satellite images, \(r\), where \(X_{i}\) is a satellite image; \(y_{i}\) represents the class labels \(\{y_{i}\in 1,\ 2,\ \dots\dots,\ m\}\) associated with the \(X_{i}\), and m denotes the number of defined LU/LC classes for that set. \[S_{i}=\{X_{i},y_{i}\}_{i=1}^{r} \tag{1}\]  
Figure 6: Proposed Vision Transformer model for LU/LC classification. In the first step of the Vision Transformer model, an image \(X_{i}\) from the training, the set is divided into non-overlapping patches of fixed size. Each patch is observed by the Vision Transformer as an individual token. Thus, from the size \(h\ *\ w\ *\ c\) (where \(h\) is the height, \(c\) is the number of channels, and \(w\) is the width) of an image \(X_{i}\), we extracted the patches of dimension \(c\ *\ p\ *\ p\) (\(p\) is the patch size) from it. The extracted patches are converted to a sequence of images \((x_{1},\ x_{2},\ x_{3},\ \ldots\ldots\ldots,\ x_{n})\) of length \(n\) through flattening. \[n=hw/p^{2} \tag{2}\]  
The image patches are linearly projected into a vector setup of model dimension, \(d\), using the known embedding matrix, \(E\). The concatenation of embedded representations is processed along with the trained classification token \(vclass\) for performing the classification task. The positional information, \(E_{pos}\), is programmed and attached to the patch representation. The spatial arrangements of the trained image patches were processed through positional embedding. The resulting sequence of image patches from positional embedding with token \(z_{0}\) is given in Equation (3). \[z_{0}=[vclass;x1E;x2E;\ldots\ldots,xnE]+E_{pos},\ E\in\ \mathbb{R}^{(p^{2}c)*d},\ E_{pos}\in\ \mathbb{R}^{(n+1)*d} \tag{3}\]  
The resulting sequence of embedded image patches, \(z_{0}\), is passed into the transformer encoder with \(L\) identical layers. It has a multi-head self-attention block (\(MSA\)) and fully connected feed-forward \(MLP\) (Multilayer Perceptron) block with the GeLU activation function between them. The two subcomponents of the encoder work with the residual skip connections through the normalization layer \((LN)\). The representation of the two main components of the encoder is given in Equations (4) and (5).