# COMSATS Face: A Dataset of Face Images with Pose Variations, Its Design, and Aspects

Authors: Mahmood Ul Haq; Muhammad Athar Javed Sethi; Rehmat Ullah; Aamir Shazhad; Laiq Hasan; Ghulam Mohammad Karami
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Due to the three-dimensional formation and flexibility, a human face may appear different in numerous events. Researchers are developing robust and efficient algorithms for face detection, face recognition, and face expression analysis, causing several difficulties due to face poses, illumination, face expression, head orientation, occlusion, hairstyle, etc. To determine the effectiveness of the algorithms, it needs to be tested using a specific benchmark of face images/databases. Face pose is an important factor that severely reduces the recognition ability. In this paper, two contributions are made: (i) a dataset of face images with multiple poses is introduced. The dataset includes 850 images of 50 individuals under 17 different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, -5°, -10°, -15°, -20°, -25°, -30°, -35°, -55°). These images were captured closed to real-world conditions in the time span of five months in COMSATS University, Abbottabad Campus. Face images included in this dataset can reveal the efficiency and robustness of future face detection and face recognition algorithms. (ii) A comparative analysis of three face recognition algorithms such as PAL, PCA, and LDA is presented based on the proposed face database.


---

## Body

## 1. Introduction

Face recognition (FR) is an essential biometric technique that compares the face features of two different images to determine the similarity between these images. FR is a rapidly growing and the most popular research area. Face appearance depends on various factors such as illumination variations, face pose variations, and occlusion [1]. To examine the robustness of the face recognition algorithm to these multiple aspects, a database of significant size and diversity is required. Two main classes of face recognition are (a) face detection and (b) face verification [2]. Face recognition is legally and commercially used in applications, with high collectability and acceptability [3]. By using new imaging sensors, a new range of possibilities is open to boost the performance of face recognition systems [4].Much has been published about FR in previous literature [5–8]. These conventional methods are successful, but their robustness is being challenged by different factors such as bad lighting conditions and the low resolution of face images [8]. Currently, there are existing techniques that show considerable accuracy if the images face being processed is of sufficient resolution. Below we briefly illustrate the latest face recognition developments.Researchers in [9] developed a Laplacian face approach (LFA). In this approach, through the process of optimal linear approximation of eigenfaces, Laplacian faces are obtained. Based on the published results, the LFA based face recognition approach attains much lower error rates. Simonyan et al. [10] use Fisher vectors on densely sampled Scaled Invariant Feature Transform (SIFT) features to classify faces. The overall accuracy of the proposed algorithm for the standard Labelled Faces in the Wild (LFW) dataset is 87.7%. Wavelets based facial recognition systems have been proposed in [11]. The wavelets transform provides insensitivity towards illumination changes and pose variations. The proposed algorithm shows acceptable results on FERET facial database. The overall false rejection ratio is 0.12%. However, the author has not provided information about the occlusion in the paper. In [12], the authors used PCA based face recognition to recognize criminal faces through CCTV cameras installed in public and private areas. Their algorithm achieved an accuracy of 80%. But the proposed algorithm was not tested by real-time criminal face images.In [13], authors have proposed MFDD and RMFRD datasets for training and testing deep learning-based face recognition algorithms. MFDD dataset was designed for the detection of face wearing while RMFRD dataset was collected for validation or fine-tuning dataset in training and testing datasets in real situations. Authors in [14] presented a new face recognition algorithm MagFace to learn unified features. The proposed algorithm achieved 95.97% accuracy as compared to other algorithms. Recently researchers in [15–20] have presented different features extraction schemes that can be investigated for developing a robust FR algorithm. The main contribution of this paper is as follows:(i)
This paper briefs the creation of the COMSATS face database developed in COMSATS University Abbottabad Campus. This database contains 850 face images of 50 individuals with seventeen different poses.(ii)
The image acquisition process has been repeated with a very comparable setup in the two labs. For each subject, three separate sessions were performed with time of four months.(iii)
The proposed dataset has been tested on three baseline algorithms such as PCA, PAL, and LDA by changing the face poses and image resolution.(iv)
We cover a large range of face pose and image resolution in simulation to test the three baseline face recognition algorithms. We investigated the images of having resolution of 144 × 256, 140 × 140, 70 × 70, 40 × 40, 20 × 20, 10 × 10, and 5 × 5 pixels.(v)
The comparison has been presented under a very challenging situation, when there is only one test image.The rest of the paper is organized as follows. Section2 provides a general idea of available face databases. Section 3 describes the acquisition setup of database. Section 4 reports the three baseline face recognition algorithms tested on the COMSATS face database. Finally, Section 5 lists the results of FR algorithms and Section 6 presents some final remarks. For each section, Table 1 shows the nomenclature used in this paper.Table 1
Nomenclature.
NotationDescriptionFRFace recognitionAdaBoostAdaptive boostingPCAPrincipal component analysisLDALinear discriminant analysisPALPrincipal component analysis with adaptive boosting of linear discriminant analysisI croppedCropped face imageLBPLocal binary patternImsMean image of S subject


## 2. Available Face Databases

A huge number of databases are available in the face recognition community, and the face recognition algorithms perform differently on different datasets. Researchers' teams gathered these databases, which varied in scope, purpose, and size. Here, we briefly review the key features of these available face recognition databases such as number of subjects and images, condition, image resolution, and type. But due to the inaccessibility of information, these databases are not discussed with the same level of detail. AT&T database contains 400 images with 40 distinct subjects collected by Cambridge University. Each subject has 40 different images. Images were taken with different facial expressions (closed/open eyes, smiling/not smiling), varying lighting conditions, and facial details (with glasses, without glasses).Face recognition data contain 395 subjects including males and females having 20 images per subject. Most subjects of this database are 18–20 years old, with some older subjects. Some subjects have beards and glasses. The images format of this database is a colour JPEG image of 24-bit.Facial recognition technology (FERET) [21] was started in 1993. A total of 14051 face images of 1209 people have been included in this database covering a large range of variations in facial expressions, illuminations, viewpoints, and acquisition time. AR database [22] was collected by Alex Martinez and Robert Benavente. This database contains 4,000 colour face images of 126 subjects including 70 men and 56 women. The dataset included images with frontal view, illumination, facial expressions, and occlusions like glasses and scarves. JAFFE database is also called the Japanese female face database containing 213 images of 10 Japanese models with seven facial expressions (neutral and basic facial expression). Indian face database [23] was collected in IIT Kanpur Campus during February 2002 in JPEG format. This dataset contains 40 subjects including males and females with eleven images of each subject. The size of the images is 640 × 480 pixels, an 8-bit image. These images contain faces looking upwards left, looking down, and looking upwards right. Available expressions of this dataset are neutral, smiling, laughter, and sadness. Georgia Tech Face database was collected by the Georgia Institute of Technology in the time span of five months. This database contains 50 subjects and 15 colour images per subject in JPEG format with different scales and locations. Various images were captured in two sessions to consider the variations in expression, illumination conditions, and appearance. PUT face database was created by CIE biometrics containing 10000 images of 100 subjects. These images were captured in a controlled environment. This database includes additional data such as rectangles containing eyes, mouth, nose, landmarks positions, and face and is accessible for research work. CMU PIE database [24] contains 68 subjects with 41,368 images having 43 illumination conditions and 13 poses, with four different expressions. This database is also called the CMU Pose, Illumination, and Expression database. This database was collected from November 2000 up to December 2000. CMU Multi-PIE [25] database includes a vast collection of images captured with different pose angles. CMU Multi-PIE database was collected in five months having more than 750000 facial images of 337 subjects taken at several viewpoints displaying a range of expressions and poses.LFW [26] dataset has images with different poses, expressions, illumination variations, and occlusion. LFW database contains more than 13000 images with 1680 subjects. Yale face database [27] contains 15 subjects with 165 images. This database includes 11 images per person with different facial expressions, lighting conditions and occlusion such as glasses. Yale face B database contains 5760 images of 10 persons, 576 with 9 poses, and 64 illumination conditions per subject. The Basel Face model is collected by the University of Basel and is available on their website. The Morphable model has registered 100 male and 100 female 3D scans faces. The chokePoint dataset is a video dataset of 48 videos including 64,204 face images. This dataset includes person reidentification, image set matching, clustering, 3D face reconstruction, face tracking, background subtraction, and estimation. In Table 2, we present a review of face recognition dataset which can help the development and validation of new FR algorithms.Table 2
Available datasets along with their features.
DatabaseImagesFeaturesAT&T 29 LFW400 images with 40 peopleLight and expression variations with glassesAR face 214000 images with 126 peopleExpression, occlusion, illumination, and frontal poseFace recognition data395 subjects and 20 images/subject18–20 years old with some older subjects. Some subjects have beards and glassesFERET 23Containing 14126 images with 199 subjectsPose, expression and time variations, colour imagesYale face 1515 subjects with 165 imagesEyeglasses, expressions, and lighteningYale face B 1010 subjects with 5760 imagesIllumination and pose variationThe Extended M2VTS database, University of Surrey, UKFour recordings of 295 subjectsSpeaking headshot, rotating headshot, high-quality colour imagesJAFFE database213 images of 10 female modelsSeven facial expressionsPIE database, CMU 6841368 images with 68 subjectsIllumination expression and poseCMU Multi-PIE750000 facial images of 337 subjectsNineteen poses and different viewpointsLFW databaseMore than 13000 images with 1680 subjectsPose, illumination, expression background variation, and occlusion


## 3. COMSATS Face Dataset

### 3.1. Equipment

Different instruments are used to collect dataset like total station (Trimble M3 DR5), theodolite (DT-5), staff road, stand, permanent marker, background sheet, and a digital camera. The angles were measured using theodolite and total station. The staff road of 5 meters is used to find the elevation of angles. Theodolite is used to measure the angles of the vertical axis and horizontal axis. Theodolite and the total station were used because of their fine accuracy. The images were captured by a professional photographer with cannon EOS6D in the lightening of fluorescent lamps as shown in Figure1(a). The optic was a Canon 85 mm, f1.8 with an aperture of f5.6 and shutter speed of 1/60th.Figure 1
(a) Angle measurement process and predefined structure, (b) measured angles.
(a)(b)


### 3.2. Observation

An organized indoor atmosphere was set up with fluorescent lamps and natural light. The participant was asked to sit at the predefined point in front of the camera at 0.5 to 0.8 meters and follow the predefined structure as shown in Figure1(b). A white sheet was placed behind the background to produce uniformity. The camera operator observed the participant face angle for the desirable results before taking the images.


### 3.3. Image Acquisition

Fifty volunteers participated in the collection of the dataset, and all belonged to the same gender (Male) with different ages, weight, colour, and cast. Their ages limits were from 18 to 35 years. Most of them were students of COMSATS University Islamabad Abbottabad (Campus) with few alumni. The database collection work was performed in the survey lab of civil engineering department of COMSATS University Islamabad Abbottabad (Campus). The dataset was completed in the duration of five months. These images were captured in two separate sessions at a lab explicitly prepared for purpose of the dataset. Samples were sited in front of a white sheet. Two of the image processing experts were selected to provide the mental state term and to set the face of an actor according to the corresponding face angle. To prepare himself for the interpretation of the related face angle, the participator was given time as needed. When the participator provided a thumb gesture to the photographer, the picture was taken in the desired view angle. Importantly, for a guarantee and natural interpretation of a given face angle, the participators were restricted not to tilt the head. The participant then immediately turned to the next face angle as advised by the instructors from the camera, and a second picture was taken. The camera operator collected the dataset images at the end of the experiment. This database consists of 850 images of 50 subjects under 17 different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, −55°) with each subject having different age, weight, height, and facial colour.A consent form has been signed by every individual, which ensures that their face images will be used for research purposes. Specifications of the dataset are presented in Table3.Table 3
Dataset specifications table.
Subject areaElectronics and computer engineeringSpecific subject areaComputer vision, image processing, face recognition, electronicsType of dataImagesHow dataset was acquiredThe dataset was acquired by using the following instruments1- Camera (Cannon 20.6 mega pixel)2- Theodolite (DT-5)3- Total station (Trimble M3 DR5)4- Fluorescent lamp (for lightening)Data formatJPGExperimental factorsImages with 17 different poses (-55 to +55) of a total of 50 individuals were captured with the help of a photographer, lab assistant, and lab engineerExperimental featuresThe database consists of 850 images of fifty subjects with seventeen different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, -5°, -10°, -15°, -20°, -25°, -30°, -35°, -55°)Data source locationCOMSATS University Islamabad, Abbottabad (Campus), PakistanData availabilityData is available on http://cuiatd.edu.pk/COMSATSFacePoseVarProj.html


### 3.4. Image Specification

The database contains 850 jpg image files with a resolution of 2988 × 5312 pixels (colour images) with the built-in flash of the camera. Each image was then preprocessed, and their resolution has been changed to 144 × 256. The size of each preprocessed image is less than 1 MB. Properties of images, i.e., dimensions and pixels before and after preprocessing, are presented in Figure2. In the database preprocessing step, all the images of each individual were renamed by their face angles. These images were resized by MATLAB using nearest neighbor interpolation algorithm and the dimensions of images were changed to get relevant results. These images were cropped manually to get the specific (important) portion of an image. Raw images can be obtained upon request from the authors. Researchers can use these images for face detection, face recognition, age estimation, facial expression recognition, and face pose recognition.Figure 2
Properties of a single image: (a) original image, (b) image after preprocessing.
(a)(b)


### 3.5. Dataset Structure

The database consists of 850 images of fifty subjects under seventeen different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, −55°). The images of individuals are presented in Figure 3. These images were captured close to real-world conditions for a duration of five months. Figure 4 shows 17 different poses of each individual. Face images involved in this dataset can reveal the effectiveness and robustness of different face detection and recognition algorithms. These images were cropped in preprocessing step to get the specific (face) portion of an image. However, for research purposes, raw images can be obtained upon request from the authors.Figure 3
Example images of subjects.Figure 4
Seventeen different poses of individual subject.


### 3.6. Data Records

(i)
This dataset will be used for the evaluation of the performance of different algorithms proposed for security and attendance purposes.(ii)
This data will be a source for different algorithms like LDA [28], Local Binary Pattern [29], eigenfaces [30], and Deep Learning and will be a challenge for recently published face recognition algorithms [31–33].(iii)
It includes the poses in the range of −55 to +55 of all the subjects. These poses are 0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, and −55°. This dataset includes fifty subjects having different ages of people; the age range is from 18 to 25 years.


## 3.1. Equipment

Different instruments are used to collect dataset like total station (Trimble M3 DR5), theodolite (DT-5), staff road, stand, permanent marker, background sheet, and a digital camera. The angles were measured using theodolite and total station. The staff road of 5 meters is used to find the elevation of angles. Theodolite is used to measure the angles of the vertical axis and horizontal axis. Theodolite and the total station were used because of their fine accuracy. The images were captured by a professional photographer with cannon EOS6D in the lightening of fluorescent lamps as shown in Figure1(a). The optic was a Canon 85 mm, f1.8 with an aperture of f5.6 and shutter speed of 1/60th.Figure 1
(a) Angle measurement process and predefined structure, (b) measured angles.
(a)(b)


## 3.2. Observation

An organized indoor atmosphere was set up with fluorescent lamps and natural light. The participant was asked to sit at the predefined point in front of the camera at 0.5 to 0.8 meters and follow the predefined structure as shown in Figure1(b). A white sheet was placed behind the background to produce uniformity. The camera operator observed the participant face angle for the desirable results before taking the images.


## 3.3. Image Acquisition

Fifty volunteers participated in the collection of the dataset, and all belonged to the same gender (Male) with different ages, weight, colour, and cast. Their ages limits were from 18 to 35 years. Most of them were students of COMSATS University Islamabad Abbottabad (Campus) with few alumni. The database collection work was performed in the survey lab of civil engineering department of COMSATS University Islamabad Abbottabad (Campus). The dataset was completed in the duration of five months. These images were captured in two separate sessions at a lab explicitly prepared for purpose of the dataset. Samples were sited in front of a white sheet. Two of the image processing experts were selected to provide the mental state term and to set the face of an actor according to the corresponding face angle. To prepare himself for the interpretation of the related face angle, the participator was given time as needed. When the participator provided a thumb gesture to the photographer, the picture was taken in the desired view angle. Importantly, for a guarantee and natural interpretation of a given face angle, the participators were restricted not to tilt the head. The participant then immediately turned to the next face angle as advised by the instructors from the camera, and a second picture was taken. The camera operator collected the dataset images at the end of the experiment. This database consists of 850 images of 50 subjects under 17 different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, −55°) with each subject having different age, weight, height, and facial colour.A consent form has been signed by every individual, which ensures that their face images will be used for research purposes. Specifications of the dataset are presented in Table3.Table 3
Dataset specifications table.
Subject areaElectronics and computer engineeringSpecific subject areaComputer vision, image processing, face recognition, electronicsType of dataImagesHow dataset was acquiredThe dataset was acquired by using the following instruments1- Camera (Cannon 20.6 mega pixel)2- Theodolite (DT-5)3- Total station (Trimble M3 DR5)4- Fluorescent lamp (for lightening)Data formatJPGExperimental factorsImages with 17 different poses (-55 to +55) of a total of 50 individuals were captured with the help of a photographer, lab assistant, and lab engineerExperimental featuresThe database consists of 850 images of fifty subjects with seventeen different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, -5°, -10°, -15°, -20°, -25°, -30°, -35°, -55°)Data source locationCOMSATS University Islamabad, Abbottabad (Campus), PakistanData availabilityData is available on http://cuiatd.edu.pk/COMSATSFacePoseVarProj.html


## 3.4. Image Specification

The database contains 850 jpg image files with a resolution of 2988 × 5312 pixels (colour images) with the built-in flash of the camera. Each image was then preprocessed, and their resolution has been changed to 144 × 256. The size of each preprocessed image is less than 1 MB. Properties of images, i.e., dimensions and pixels before and after preprocessing, are presented in Figure2. In the database preprocessing step, all the images of each individual were renamed by their face angles. These images were resized by MATLAB using nearest neighbor interpolation algorithm and the dimensions of images were changed to get relevant results. These images were cropped manually to get the specific (important) portion of an image. Raw images can be obtained upon request from the authors. Researchers can use these images for face detection, face recognition, age estimation, facial expression recognition, and face pose recognition.Figure 2
Properties of a single image: (a) original image, (b) image after preprocessing.
(a)(b)


## 3.5. Dataset Structure

The database consists of 850 images of fifty subjects under seventeen different poses (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, −55°). The images of individuals are presented in Figure 3. These images were captured close to real-world conditions for a duration of five months. Figure 4 shows 17 different poses of each individual. Face images involved in this dataset can reveal the effectiveness and robustness of different face detection and recognition algorithms. These images were cropped in preprocessing step to get the specific (face) portion of an image. However, for research purposes, raw images can be obtained upon request from the authors.Figure 3
Example images of subjects.Figure 4
Seventeen different poses of individual subject.


## 3.6. Data Records

(i)
This dataset will be used for the evaluation of the performance of different algorithms proposed for security and attendance purposes.(ii)
This data will be a source for different algorithms like LDA [28], Local Binary Pattern [29], eigenfaces [30], and Deep Learning and will be a challenge for recently published face recognition algorithms [31–33].(iii)
It includes the poses in the range of −55 to +55 of all the subjects. These poses are 0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, and −55°. This dataset includes fifty subjects having different ages of people; the age range is from 18 to 25 years.


## 4. Face Recognition Algorithms

This section describes the comparison of face recognition algorithms based on the abovementioned database. The study was performed using different sizes of images. Images of fifty subjects (three images per person) were chosen for training (gallery), whereas these algorithms were tested on seventeen images per subject of different sizes such as 144 × 256, 140 × 140, 70 × 70, 40×40, 20 × 20, 10 × 10, and 5 × 5 pixels.


### 4.1. PCA Based (Eigenfaces) Face Recognition Algorithm

Principal Component Analysis is a statistical procedure in which transformation is used as set of observed possible correlated variables into linearly noncorrelated variables which are called the principal of components. In the face recognition system, PCA plays a vital role as it is a very efficient method for face recognition. As in PCA all images of the training set are represented as a combination of weighted eigenfaces and calculate covariance matrices. By the covariance matrix of a training set of images, eigenvectors are obtained. Weights of eigenvectors are found by the set of eigenfaces that are most relevant. Recognition of faces is done by projecting a test image on the subspace of eigenfaces. The distance between the test image and training images is calculated using(1)dbibj=∑a=1nxki−xkj2,where bi and bj represent two matrices for training and test samples, respectively, and xki−xkj2 is the Euclidean distance (ED) between two image components Xki and Xkj. Test image must have minimum Euclidean distance with a recognized image that exists in the training images. There are three possible scenarios in PCA based face recognition algorithm when the test image is tested with the face database as described below.Scenarios:(i)
If the test face image is far away from face space, it is not a face image.(ii)
If the test face image is near face space and far away from face class, then the image is not recognized by the algorithm.(iii)
If the test image is close to both face class and face space, then the face image is correctly recognized in the face database. For implementation and detail of the PCA based FR algorithm, readers are referred to [34].


### 4.2. Linear Discriminant Analysis (LDA) and Fisher's Face

The LDA is proposed as an enhancement to Principal Component Analysis (PCA). LDA constructs a discriminant subspace that reduces the scatter between the same class images and maximizes the scatter between images of different classes. Letc = [X1, X2,…, Xc] be the face classes in the database and let each face class Xi has face images xj, where j = 1,2,..…, k. Within class, variance can be calculated using with in-class scatter matrix.(2)Sw=∑i=1c∑j=1sxk−μixk−μiT,where, for all classes (c), xk denotes the jth sample, while μi represents mean of ith class and can be calculated by(3)μi=∑j=1kxjk.Similarly, the between-class scatter matrix (Sb) can be defined as(4)Sb=∑i=1cNiμi−μμi−μT,where μ represents the mean of all classes and can be calculated as(5)μ=∑i=1cμic.After computingSb and Sw, find the product of SW−1 and Sb and compute the eigenvectors of the product (SW−1Sb). To reduce the scatter matrix dimensionality, use the same approach as eigenfaces (PCA). The last step is to project each face image to face space(6)Si=∪Txj−μ.For a detailed study, readers are referred to [35].


### 4.3. PAL Face Recognition Algorithm

In the PAL FR algorithm, initially 68 specific points on training and testing faces are detected after face detection using a machine learning algorithm. In next step, all these faces are cropped according to these 68 landmarks. The mean and standard deviation of each face image are calculated and updated according to the relation given in (7) to reduce the error due to lighting variations(7)In=Icropped−X¯×σdefσi+X¯def,where X¯ represents the mean and σi represents the standard deviation of each input image while X¯def and σdef are predefined mean and standard deviation suggested for all input images to reduce light variations. In this technique mean image of each class is taken to reduce time complexity, memory requirements, and errors due to pose variations. Mean image can be calculated as(8)Ims=∑j=1JInjsJ,where Injs is the jth training image (normalized) of subject 's' and J represents a total number of training images of 's' subject.Furthermore, these images are fed to AdaBoost combined with LDA for recognition. A scoring value of the test image with each class is attained using the final classifier and the maximum scoring value achieved with the class will be considered as recognized image with desired class. For detailed study, readers are referred to [36]. The pseudocode of the proposed algorithm is presented in Table 4.Table 4
Proposed PAL approach.
Input: A set of input imagesA=ai=1Ji=1I with I = {1,2, ……., I} classes and J images of each class.Do fori = 1, …, I(2) convert RGB images to grey,(3) estimate and crop faceIcropped.(4) update mean and standard deviation of each image,In=Icroped−X¯×σdef/σi+X¯def(5) calculate mean image of each class,tri=∑j=1Jaji/J.Final training images of each class,Tr=tr1,……tr1.Initialize mislabelled distribution overm, D1i=1/m=1/N−1Do fort=1,…T:(1) ift = 1, choose i samples per class for the learner.(2) train LDA feature extractor.(3) build ag classifier ht.(4) calculate pseudo loss,et(5) calculateβt=et/1−et(6) ifβt=0, abort the loop(7) update the distributionFinalg classifier of training image, hf z=argmax∑log1/βthtz,y.Generate a matching score.Output: Maximum matching score (Mscore), I recog = argmax (Mscore).The proposed face database is tested on three baseline techniques such as PAL, PCA, and LDA. Tables5 and 6 show the overall accuracy of the above-mentioned algorithms on the proposed face database.Table 5
Recognition accuracy, precision, and recall for different image resolutions.
Image resolution (in pixel)AlgorithmRecognition accuracy (%)PrecisionRecall144 × 256PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.9069140 × 140PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.906970 × 70PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.906940 × 40PCA algorithm73.22073220.7881LDA algorithm58.30.58330.6290PAL algorithm86.660.86660.911220 × 20PCA algorithm53.460.53460.5721LDA algorithm41.490.41490.4523PAL algorithm81.430.81430.849110 × 10PCA algorithm45.050.45050.4732LDA algorithm37.340.37340.3972PAL algorithm77.750.77750.81335 × 5PCA algorithm32.70.32770.423LDA algorithm26.20.26230.3477PAL algorithm71.30.71330.7889Table 6
Comparison of classification accuracy algorithms for pose variations.
Image resolutionFR algorithmsRecognition accuracy%±55°±35°±30°±25°±20°±15°±10°±5°0°144 × 256 pixelPCA [34]304947535364100100100LDA [35]436166717788100100100PAL technique [36]64747984879210010010070 × 70 pixelPCA [34]304947535364100100100LDA [35]436166717788100100100PAL technique [36]6474798487921001001005 × 5 pixelPCA [34]61717243036395571LDA [35]0814132533385154PAL technique [36]324857627172100100100


## 4.1. PCA Based (Eigenfaces) Face Recognition Algorithm

Principal Component Analysis is a statistical procedure in which transformation is used as set of observed possible correlated variables into linearly noncorrelated variables which are called the principal of components. In the face recognition system, PCA plays a vital role as it is a very efficient method for face recognition. As in PCA all images of the training set are represented as a combination of weighted eigenfaces and calculate covariance matrices. By the covariance matrix of a training set of images, eigenvectors are obtained. Weights of eigenvectors are found by the set of eigenfaces that are most relevant. Recognition of faces is done by projecting a test image on the subspace of eigenfaces. The distance between the test image and training images is calculated using(1)dbibj=∑a=1nxki−xkj2,where bi and bj represent two matrices for training and test samples, respectively, and xki−xkj2 is the Euclidean distance (ED) between two image components Xki and Xkj. Test image must have minimum Euclidean distance with a recognized image that exists in the training images. There are three possible scenarios in PCA based face recognition algorithm when the test image is tested with the face database as described below.Scenarios:(i)
If the test face image is far away from face space, it is not a face image.(ii)
If the test face image is near face space and far away from face class, then the image is not recognized by the algorithm.(iii)
If the test image is close to both face class and face space, then the face image is correctly recognized in the face database. For implementation and detail of the PCA based FR algorithm, readers are referred to [34].


## 4.2. Linear Discriminant Analysis (LDA) and Fisher's Face

The LDA is proposed as an enhancement to Principal Component Analysis (PCA). LDA constructs a discriminant subspace that reduces the scatter between the same class images and maximizes the scatter between images of different classes. Letc = [X1, X2,…, Xc] be the face classes in the database and let each face class Xi has face images xj, where j = 1,2,..…, k. Within class, variance can be calculated using with in-class scatter matrix.(2)Sw=∑i=1c∑j=1sxk−μixk−μiT,where, for all classes (c), xk denotes the jth sample, while μi represents mean of ith class and can be calculated by(3)μi=∑j=1kxjk.Similarly, the between-class scatter matrix (Sb) can be defined as(4)Sb=∑i=1cNiμi−μμi−μT,where μ represents the mean of all classes and can be calculated as(5)μ=∑i=1cμic.After computingSb and Sw, find the product of SW−1 and Sb and compute the eigenvectors of the product (SW−1Sb). To reduce the scatter matrix dimensionality, use the same approach as eigenfaces (PCA). The last step is to project each face image to face space(6)Si=∪Txj−μ.For a detailed study, readers are referred to [35].


## 4.3. PAL Face Recognition Algorithm

In the PAL FR algorithm, initially 68 specific points on training and testing faces are detected after face detection using a machine learning algorithm. In next step, all these faces are cropped according to these 68 landmarks. The mean and standard deviation of each face image are calculated and updated according to the relation given in (7) to reduce the error due to lighting variations(7)In=Icropped−X¯×σdefσi+X¯def,where X¯ represents the mean and σi represents the standard deviation of each input image while X¯def and σdef are predefined mean and standard deviation suggested for all input images to reduce light variations. In this technique mean image of each class is taken to reduce time complexity, memory requirements, and errors due to pose variations. Mean image can be calculated as(8)Ims=∑j=1JInjsJ,where Injs is the jth training image (normalized) of subject 's' and J represents a total number of training images of 's' subject.Furthermore, these images are fed to AdaBoost combined with LDA for recognition. A scoring value of the test image with each class is attained using the final classifier and the maximum scoring value achieved with the class will be considered as recognized image with desired class. For detailed study, readers are referred to [36]. The pseudocode of the proposed algorithm is presented in Table 4.Table 4
Proposed PAL approach.
Input: A set of input imagesA=ai=1Ji=1I with I = {1,2, ……., I} classes and J images of each class.Do fori = 1, …, I(2) convert RGB images to grey,(3) estimate and crop faceIcropped.(4) update mean and standard deviation of each image,In=Icroped−X¯×σdef/σi+X¯def(5) calculate mean image of each class,tri=∑j=1Jaji/J.Final training images of each class,Tr=tr1,……tr1.Initialize mislabelled distribution overm, D1i=1/m=1/N−1Do fort=1,…T:(1) ift = 1, choose i samples per class for the learner.(2) train LDA feature extractor.(3) build ag classifier ht.(4) calculate pseudo loss,et(5) calculateβt=et/1−et(6) ifβt=0, abort the loop(7) update the distributionFinalg classifier of training image, hf z=argmax∑log1/βthtz,y.Generate a matching score.Output: Maximum matching score (Mscore), I recog = argmax (Mscore).The proposed face database is tested on three baseline techniques such as PAL, PCA, and LDA. Tables5 and 6 show the overall accuracy of the above-mentioned algorithms on the proposed face database.Table 5
Recognition accuracy, precision, and recall for different image resolutions.
Image resolution (in pixel)AlgorithmRecognition accuracy (%)PrecisionRecall144 × 256PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.9069140 × 140PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.906970 × 70PCA algorithm78.40.78440.8451LDA algorithm62.20.66220.7462PAL algorithm86.660.8660.906940 × 40PCA algorithm73.22073220.7881LDA algorithm58.30.58330.6290PAL algorithm86.660.86660.911220 × 20PCA algorithm53.460.53460.5721LDA algorithm41.490.41490.4523PAL algorithm81.430.81430.849110 × 10PCA algorithm45.050.45050.4732LDA algorithm37.340.37340.3972PAL algorithm77.750.77750.81335 × 5PCA algorithm32.70.32770.423LDA algorithm26.20.26230.3477PAL algorithm71.30.71330.7889Table 6
Comparison of classification accuracy algorithms for pose variations.
Image resolutionFR algorithmsRecognition accuracy%±55°±35°±30°±25°±20°±15°±10°±5°0°144 × 256 pixelPCA [34]304947535364100100100LDA [35]436166717788100100100PAL technique [36]64747984879210010010070 × 70 pixelPCA [34]304947535364100100100LDA [35]436166717788100100100PAL technique [36]6474798487921001001005 × 5 pixelPCA [34]61717243036395571LDA [35]0814132533385154PAL technique [36]324857627172100100100


## 5. Simulation Results

The experiments were performed using a Super-Server 7047 GR machine having 92 GB of RAM with MATLAB 2019 as a simulation tool. To test the above-mentioned FR algorithms, numerous tests were carried out on the proposed database which has several face images with two different conditions, such as face poses and image resolutions.For each algorithm, three frontal images (0°, 5°, −5°) were chosen for training as shown in Figure 5(a) and seventeen different test images with a variation in pose (0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 55°, −5°, −10°, −15°, −20°, −25°, −30°, −35°, −55°) as shown in Figure 5(b)Figure 5
(a) Training images, (b) testing images.
(a)(b)


### 5.1. Face Image Resolution Analysis

In this study, face images of 144×256, 140×140, 70×70, 40×40, 20×20, 10×10, and 5×5 pixels were reinvestigated. Table5 details the results of the three FR algorithms. From Table 5, essential explanations are as follows.(i)
For face images having resolution of 70×70 pixels and above, PAL yields the highest recognition rate of 86.66% followed by PCA having an accuracy of 78.4% and LDA having an accuracy of 66.2%(ii)
For face resolution of 5×5, the accuracy of PCA and LDA has been decreased to 32.7% and 26.2% while the recognition rate of the PAL algorithm is 71.3%(iii)
PAL algorithm is most effective across all aforementioned ranges of image resolutions


### 5.2. Face Pose Analysis

Some features of an individual’s face are occluded due to variations in the facial pose. A good FR algorithm should be robust to pose variations and should be able to recognize a face with different viewing angles. In this study seventeen face poses of 50 subjects are investigated.As shown in Table6, the PAL method, the PCA, and LDA based face recognition algorithms yield 100% recognition accuracy for frontal face images of 144 × 256 and 70 × 70 pixels.(i)
The PAL method comprehensively outperforms other face recognition algorithms from frontal to ± 55° of pose variation.(ii)
We observed the LDA based face recognition algorithm is less effective under low resolution by achieving the maximum accuracy of 47% for frontal facial images. For  ± 55° of face pose, the LDA barely yields any recognition results.


### 5.3. Computational Complexity

Figure6 presents the execution times of algorithms for different image resolution face images.Figure 6
Computational complexity of the compared algorithms.From Figure6, important observations are as follows.(i)
For each face image resolution category, PAL algorithm consumes over 9 seconds and is most computationally complex as compared to PCA and LDA.(ii)
For image resolution of 40×40 pixels and below, the compared algorithms consume less than 4 seconds. The LDA is unable to recognize face image resolution of 10×10 pixels and below.(iii)
For the different face image resolutions, on average 3.23 seconds is consumed by PAL while executing even on a high-performance Super-Server machine. From the above analysis, it can be concluded that the compared algorithms are near real time.


### 5.4. Discussion

To develop a robust FR algorithm that can mimic the human vision system, continuous efforts are in progress. Table7 further highlights the importance of the FR algorithms.(i)
For extremely low-resolution frontal images, such as 20×20 pixels, PAL and PCA algorithms can be used.(ii)
For low-resolution nonfrontal images, such as crime scenes, only PAL should be used.(iii)
For less computational complexity, face poses, and average accuracy, readers are suggested to use the PCA algorithm.Table 7
Selection of the FR algorithms based on performance.
DescriptionAlgorithmLow-resolution frontal imagesPAL [36] and PCA [34]Low-resolution images with face posesPAL [36]Time efficient with average accuracyPCA [34]Time efficientPCA [34] and LDA [35]


## 5.1. Face Image Resolution Analysis

In this study, face images of 144×256, 140×140, 70×70, 40×40, 20×20, 10×10, and 5×5 pixels were reinvestigated. Table5 details the results of the three FR algorithms. From Table 5, essential explanations are as follows.(i)
For face images having resolution of 70×70 pixels and above, PAL yields the highest recognition rate of 86.66% followed by PCA having an accuracy of 78.4% and LDA having an accuracy of 66.2%(ii)
For face resolution of 5×5, the accuracy of PCA and LDA has been decreased to 32.7% and 26.2% while the recognition rate of the PAL algorithm is 71.3%(iii)
PAL algorithm is most effective across all aforementioned ranges of image resolutions


## 5.2. Face Pose Analysis

Some features of an individual’s face are occluded due to variations in the facial pose. A good FR algorithm should be robust to pose variations and should be able to recognize a face with different viewing angles. In this study seventeen face poses of 50 subjects are investigated.As shown in Table6, the PAL method, the PCA, and LDA based face recognition algorithms yield 100% recognition accuracy for frontal face images of 144 × 256 and 70 × 70 pixels.(i)
The PAL method comprehensively outperforms other face recognition algorithms from frontal to ± 55° of pose variation.(ii)
We observed the LDA based face recognition algorithm is less effective under low resolution by achieving the maximum accuracy of 47% for frontal facial images. For  ± 55° of face pose, the LDA barely yields any recognition results.


## 5.3. Computational Complexity

Figure6 presents the execution times of algorithms for different image resolution face images.Figure 6
Computational complexity of the compared algorithms.From Figure6, important observations are as follows.(i)
For each face image resolution category, PAL algorithm consumes over 9 seconds and is most computationally complex as compared to PCA and LDA.(ii)
For image resolution of 40×40 pixels and below, the compared algorithms consume less than 4 seconds. The LDA is unable to recognize face image resolution of 10×10 pixels and below.(iii)
For the different face image resolutions, on average 3.23 seconds is consumed by PAL while executing even on a high-performance Super-Server machine. From the above analysis, it can be concluded that the compared algorithms are near real time.


## 5.4. Discussion

To develop a robust FR algorithm that can mimic the human vision system, continuous efforts are in progress. Table7 further highlights the importance of the FR algorithms.(i)
For extremely low-resolution frontal images, such as 20×20 pixels, PAL and PCA algorithms can be used.(ii)
For low-resolution nonfrontal images, such as crime scenes, only PAL should be used.(iii)
For less computational complexity, face poses, and average accuracy, readers are suggested to use the PCA algorithm.Table 7
Selection of the FR algorithms based on performance.
DescriptionAlgorithmLow-resolution frontal imagesPAL [36] and PCA [34]Low-resolution images with face posesPAL [36]Time efficient with average accuracyPCA [34]Time efficientPCA [34] and LDA [35]


## 6. Conclusion and Future Work

This paper presents a dataset of face images with multiple poses (COMSATS face database). These images were captured close to real-world conditions in the time span of five months in COMSATS University, Abbottabad (Campus). Face images included in this dataset can reveal the efficiency and robustness of future face detection and recognition algorithms. This database can be used for other research areas such as gender classification, age estimation, emotion recognition, face pose recognition, age estimation, and face modelling.In the next step, a comparison of three well-known face recognition algorithms based on the proposed dataset is presented which are (i) PCA based face recognition (eigenfaces), (ii) LDA based face recognition, and (iii) PAL face recognition algorithm. Simulation results on the proposed database show that PAL face recognition algorithm can be reliably used for low resolution up to 5 × 5-pixel images and from frontal (0°) ranges to ± 55° of face pose variation near real time.In our future work, we intend to develop a new face recognition algorithm that can recognize low-resolution face images up to 5 × 5-pixel images and pose variation of ± 90°



---

*Source: 4589057-2022-05-23.xml*