# Person Identification and Gender Classification Based on Vision Transformers for Periocular Images
###### Abstract
Recently, vision transformers [6] have been utilized by computer vision researchers for image classification. The generic steps followed by vision transformers [6] are as follows:  
1. Perform image resizing and normalization. For resizing, the size of the images will be the same as the image size used during the development of the pre-trained model. 2. The motivation of vision transformers comes from sentence transformers in NLP (natural language processing). The sentence transformers expect sequences of words as input. However, in the image processing context, sequences of words are not available. Hence, the entire image is split into patches of equal size so that those  
Figure 1: (**a**) Person’s Image with a mask; (**b**) Person’s image without a mask to highlight the challenges in identification. patches could be further arranged as sequences to be fed to the encoder of the vision transformer. It can be noted that even after splitting the original image, the patches are still in 2-D form. 3. Each patch is subsequently flattened to form a 1-D vector from a 2-D matrix. 4. The 1-D vectors are further subjected to linear mapping. This assigns an intermediate representation to the original 1-D vector. It may be a more reduced dimension than the original 1-D vector's dimensionality. 5. A classification token is appended to every linearly mapped 1-D vector. This additional information will preserve the classification label. 6. Using a positional encoding function, the vectors are mapped onto a set of values that typically follow frequency patterns. 7. The vectors are further subjected to an encoding process. This encoding is a two-step procedure that involves the normalization of layers and the multi-head self-attention algorithm. During normalization, a layer-specific mean subtraction and division by standard deviation is performed. For multi-head self-attention, the following process is performed: For every processed 1-D vector pertaining to its 2-D patch, a dot product is computed using query (Q) and key (K). This will yield a cosine similarity between Q and K. The product is normalized with the dimensions of the key vector (division operation). The result is subjected to the softmax function, which will yield a matrix whose values are in the range of 0 to 1. This matrix is multiplied with a value matrix. This process will assign the similarity amongst patches, which is a final output of the multi-head self-attention layer. At the end of this process, the linear mapping is applied to reduce the dimensionality of the concatenated multi-head self-attention matrix. 8. The MLP (multi-layer perceptron) head defines intermediate layers that finally yield the probability of predefined categories. The main contributions of this work are (i) competitive performance metrics for person identification as well as gender classification using periocular images and (ii) introduction of the use of concatenated average pool and max pool layers in the vision transformer architecture for periocular image classification. It can be inferred that the proposed technique yields higher accuracies for person identification as well as gender classification. The goal of this paper is to propose a technique for obtaining enhanced performance metrics for periocular person identification and gender classification. The remainder of the article is organized in this manner. Section 2 presents a literature review of techniques related to periocular biometrics. Section 3 illustrates the resources and techniques employed, which include the dataset used and transformers in NLP and vision, followed by the proposed model, ROI (region of interest) for pre-processing, and a step-by-step understanding of the proposed approach to obtain better accuracy. In Section 4, we exhibit the results of the experimental work in a tabular form compared with previous results obtained and graphical analysis. Finally, in Section 5 discussion on the results is presented followed by the conclusion in Section 6.