# Person Identification and Gender Classification Based on Vision Transformers for Periocular Images
## 3 Materials and Methods
### Transformers in Vision  
ViT model was introduced in a research work designated \"An image is worth \(16\times 16\) words: Transformers for Image Recognition at Scale\" at ICLR 2021 [6]. Transformers can be considered a comprehensive training method that appropriates various types of data for better performance measures. The ViT model treats the whole image (input) as an inherent image-patches collection, similar to how word embeddings are represented when text processing is done with transformers. Figure 4 illustrates the process followed for image classification [6].  
Figure 3: (i) Scaled dot product attention and (ii) Multi-head attention [25].  
For the processing of images based on the ViT Model, they have been initially broken down into fixed-size patches. If an input image is given as H \(\times\) W \(\times\) C, wherein H indicates height, W indicates width, C indicates channels, and P indicates patch size, then the new patch image size is P \(\times\) P \(\times\) C. The total count of patches (N) is the entire image size divided by the individual patches, i.e., N = (H/P) \(\times\) (W/P). For example, if the input imagesize is 224 \(\times\) 224 along with a patch size of 16, then [(224/16) \(\times\) (224/16)] = [14 \(\times\) 14] = 196 patches or sequence tokens are constituted.  
The next step is flattening the image patches into one dimension, which can be given as H \(\times\) W \(\times\) C = N \(\times\) P \(\times\) C = N \(\times\) P \(\times\) C = N \(\times\) P\({}^{2}\)C. In general, the value of C is considered as 1 if the original vector is 1-D. The patches are projected via a linear layer that outputs the latent vector of size D. For classification, we include a trainable 'categorization token' in the process. The categorization token refers to a class token. The class token is appended to every linearly mapped 1-D vector. This additional information will preserve the classification label. Such a token is placed at the beginning of the patch sequence. After being sent through the transformer, a learnable embedding of size D will be utilized for classification. This approach is followed in BERT for NLP [30].  
To maintain positional coordinates, position clusters are attached to patch embeddings. At the initial time, the position embeddings carry no information about the 2D locations of the patches; thus, all spatial interactions amidst them must be learned from scratch. The encoder receives the resultant sequence of embedding vectors. After passing through the transformer encoder, the class token is permitted to pay attention to meaningful representations of the patches to learn an embedding for classification. The class token is to be extracted from the encoder output. The dot product of the class token and the MLP output gives a vector of 1 \(\times\) NumClasses, followed by layer normalization and SoftMax to help us with image classification tasks. The model versions involving tiny, small, and base architectures add a new depth to understanding ViT. Several characteristics control the shape of ViT models. The patch size, the dimensionality of patch embeddings and self-attention (width), the size of encoder units (depth), the size of attention heads, and also the MLP block's latent dimensions are all factors to consider (MLP-width) [6]. The specifics are illustrated in Table 1.  
Figure 4: ViT model for the processing of images [6].  
To express the model and input patch sizes, we employed simple notation: ViT-S/16, for example, denotes the \"small\" variation having a 16 \(\times\) 16 input patch size. Similarly, B denotes \"base\" and T denotes \"tiny\". The models are pretrained based on the timm PyTorch library [31].