# Recognition of Reconstructed Frontal Face Images Using FFT-PCA/SVD Algorithm

Authors: Louis Asiedu; Felix O. Mettle; Joseph A. Mensah
Journal: Journal of Applied Mathematics
 (2020)

---

## Abstract

Face recognition has gained prominence among the various biometric-based methods (such as fingerprint and iris) due to its noninvasive characteristics. Modern face recognition modules/algorithms have been successful in many application areas (access control, entertainment/leisure, security system based on biometric data, and user-friendly human-machine interfaces). In spite of these achievements, the performance of current face recognition algorithms/modules is still inhibited by varying environmental constraints such as occlusions, expressions, varying poses, illumination, and ageing. This study assessed the performance of Principal Component Analysis with singular value decomposition using Fast Fourier Transform (FFT-PCA/SVD) for preprocessing the face recognition algorithm on left and right reconstructed face images. The study found that average recognition rates for the FFT-PCA/SVD algorithm were 95% and 90% when the left and right reconstructed face images are used as test images, respectively. The result of the paired samplet-test revealed that the average recognition distances for the left and right reconstructed face images are not significantly different when FFT-PCA/SVD is used for recognition. FFT-PCA/SVD is recommended as a viable algorithm for recognition of left and right reconstructed face images.


---

## Body

## 1. Introduction

Recognizing people using face images has gained prominence among the various biometric-based methods (such as fingerprint and iris) due to its comparative advantage of being nonintrusive and less cooperative (of subjects under study). This task is easily carried out by humans. The design of machine-based face recognition systems (that mimic humans’ recognition prowess) and their underlying algorithms that give optimal classification or recognition rates, however, have been and continue to be challenging, especially when face images are obtained under uncontrolled environments (poor illumination conditions, varying poses, expressions, and occlusions) [1]. Thus, there is a growing interest in this field of research.In the case of partially occluded faces (resulting from the wearing of mask and sunglasses, blockage by external objects, captured angle images, etc.), occlusion insensitive, local matching, and reconstruction methods have been used for recognition [2]. Performing face recognition using half-face images can be considered a special case of partially occluded faces where either the left or right face is occluded and segmented and the remaining half (nonoccluded) face is used for recognition [3]. Bilateral symmetry is a property of many natural objects including the human face [4]. Leveraging this property, the performances of holistic face representation-based algorithms have been evaluated on the left, right, and average half faces based on symmetry scores [5, 6].Singh and Nandi [6] applied PCA on the full and left and right half faces and measured the performance of their algorithm in terms of the recognition rate and accuracy. Their results showed no difference in recognition rates between the left and right half faces but achieved higher accuracy for the left half face. They also reported no difference in accuracy rates between the full face and half faces. However, the recognition rate for half faces was half that for the full faces.Harguess and Aggarwal [5] evaluated the recognition rate of full faces and average half faces using eigenfaces and the nearest neighbor as a classifier. They reported a significantly higher recognition rate using the average half face for both men and women compared to the full face. Asiedu et al. [7] applied the PCA/SVD algorithm on full faces under varying facial expressions. They concluded that the algorithm was most consistent and efficient under varying expressions and achieved appreciable performance with an average recognition rate of 92.86%.Avuglah et al. [8] also applied the FFT-PCA/SVD algorithm on face images under angular constraints and statistically evaluated the performance of the algorithm. They found that the algorithm perfectly recognizes head tilts that are 24° or less. They concluded that the algorithm has an appreciable performance with an average recognition rate of 92.5% in recognition of face images under angular constraints. The question of whether this algorithm performs well on partial and reconstructed frontal face images, however, has not been explored. This paper, therefore, intends to assess the performance of the FFT-PCA/SVD algorithm on partial and reconstructed frontal face images based on bilateral symmetry.


## 2. Materials and Methods

### 2.1. Source of Data

Frontal face images from the Massachusetts Institute of Technology (MIT) (20032005) and Japanese Female Facial Expressions (JAFFE) database were used to benchmark the face recognition algorithm. The train-image database contains twenty frontal face images. Ten of these images were 0° straight pose from the MIT (2003-2005) database, and ten were neutral pose face images from JAFFE. The images captured into the train-image database are denoted as train images and are used to train the algorithm.Twenty images reconstructed from the half-face images (created through vertical segmentation) of the train images were captured into the test-image database. The images captured into the test-image database are called the test images and are used for testing the recognition algorithm.To keep the data uniform, captured images were digitized into gray-scale precision and resized into200×200 dimensions, and the data types were changed into double precision for preprocessing. This made the images (matrices) conformable and enhanced easy computations. Figures 1 and 2 show subjects in the train image database.Figure 1
Subjects in the MIT (2003-2005) database.Figure 2
Subjects in the JAFFE database.


### 2.2. Image Reconstruction

The left segmented half-face images were reconstructed using the following steps:(1)
Rotate left segmented half-face image through 270°, and denote it asNl1(2)
Rotate the left segmented half-face image through 180°, and denote it asNl2(3)
ConcatenateNl1 and Nl2 as
(1)Tl=Nl1∣Nl2The right segmented half-face images were reconstructed using the following steps:(1)
Rotate right segmented half-face image through 270°, and denote it asNr1(2)
Rotate the right segmented half-face image through 180°, and denote it asNr2(3)
ConcatenateNr2 and Nr1 as
(2)Tr=Nr2∣Nr1Figure3 contains the original full images, left and right half-images, and their reconstructed images used in the test-image database.Figure 3
Reconstructed images from left and right segmented half images.


### 2.3. Research Design

Figure4 shows a design of the recognition module.Figure 4
Research design.


### 2.4. Preprocessing

Preprocessing is an important phase of image processing where the quality of the images is enhanced. The image acquisition process comes with various forms of noise. The preprocessing techniques are used to denoise the images making them better conditioned for recognition.


#### 2.4.1. Fast Fourier Transform

Fast Fourier Transform (FFT) was adopted as a noise reduction mechanism. According to Glynn [9], the FFT algorithm reduces the computational burden to ONlogN arithmetic operations.The DFT of a column vectorajk is represented mathematically as
(3)ajk∗=DFTajk=∑r=0p−1ajke−i2πsr/p,where s=0,1,⋯,p−1, j=1,2,⋯,n. ajk is the kth column of the image matrix, Aj [8].Due to the Gaussian nature of illumination variations, the Gaussian filter is adopted for filtering the face images after the Discrete Fourier transformation. After filtering, the inverse Discrete Fourier transformation (IDFT) was performed to reconstruct images into their original forms. The inverse Discrete Fourier transformation (IDFT) is given by(4)ajk=IDFTajk∗=1p∑r=0p−1ajk∗ei2πsr/p,s=0,1,⋯,p−1,j=1,2,⋯,n.The real components of the inverse transformations are extracted for the feature extraction stage whereas the imaginary component is discarded as noise. Figure5 shows the stages in FFT preprocessing of an image (Avuglah et al. [8]).Figure 5
FFT preprocessing cycle.


### 2.5. Feature Extraction

Face image space is very large, and its storage requires reduction of the dimensions of original images using feature extraction methods. This study adopted Principal Component Analysis (PCA) proposed by Kirby and Sirovich [10] for feature extraction. PCA extracts the most significant components or those components which are more informative and less redundant, from the original data. According to Shlens [11], PCA computes the most meaningful basis to reexpress a noisy garbled dataset. The rationale behind this new basis is to filter out the noise and reveal hidden dynamics in the dataset.


### 2.6. Implementation of FFT-PCA/SVD Algorithm

As noted earlier, used images were extracted from the Massachusetts Institute of Technology (MIT) (2003-2005) database and Japanese Female Facial Expressions (JAFFE). The study considered subjects captured under 0° pose from the MIT database and neutral facial expressions from JAFFE. Twenty face images, ten each from the MIT database and JAFFE, were trained, and their reconstructed images from left and right segmented half images were used for testing the recognition algorithm. To keep the data uniform, captured images were digitized into gray-scale precision and resized into200×200 dimensions and the data types changed into double precision for preprocessing.The FFT-PCA/SVD algorithm was used to train the image database. Unique face features of the training set are extracted and stored in memory during the training phase.Now, given the sampleX=X1,X2,⋯,Xn, whose elements are the vectorised form of the individual images in the study. The mean centering preprocessing mechanism is performed by subtracting the mean image from the individual images under study. The mean is given by
(5)a¯j=EXj,j=1,2,⋯,n,a¯j=1N∑i=1NXji=1N∑i=1p∑k=1pajik,j=1,2,3,⋯,n,where N=p×p, length (=rowsofimage×columnsofimage) of the image data, Xj.According to Avuglah et al. [8], the variance-covariance matrix C of the image space is given as
(6)C=1nWWT,where the mean centered matrix is W=w1,w2,⋯,wn.The eigenvalues and their corresponding eigenvectors are extracted from singular value decomposition (SVD) of the matrix,C=UΣVT.This decomposes the covariance matrixC into two orthogonal matrices U and V and a diagonal matrix Σ:
(7)zj=∑j=1nwjuj,where uj is the jth column vector of U.From the training set, the principal components are extracted as(8)γj=zjTxj−a¯,and ΓT=γ1,γ2,⋯,γn.When a new face (test image) is passed through the recognition module, its unique features are extracted as(9)γj∗=zjTxr−a¯,and Γr∗T=γ1∗,γ2∗,⋯,γn∗.The recognition distances (Euclidean distances) are computed as(10)ψ=Γ−Γr∗.The minimum Euclidean distancedji=minψ, j=1,2,⋯,n, i=1,2, is chosen as the recognition distance.


## 2.1. Source of Data

Frontal face images from the Massachusetts Institute of Technology (MIT) (20032005) and Japanese Female Facial Expressions (JAFFE) database were used to benchmark the face recognition algorithm. The train-image database contains twenty frontal face images. Ten of these images were 0° straight pose from the MIT (2003-2005) database, and ten were neutral pose face images from JAFFE. The images captured into the train-image database are denoted as train images and are used to train the algorithm.Twenty images reconstructed from the half-face images (created through vertical segmentation) of the train images were captured into the test-image database. The images captured into the test-image database are called the test images and are used for testing the recognition algorithm.To keep the data uniform, captured images were digitized into gray-scale precision and resized into200×200 dimensions, and the data types were changed into double precision for preprocessing. This made the images (matrices) conformable and enhanced easy computations. Figures 1 and 2 show subjects in the train image database.Figure 1
Subjects in the MIT (2003-2005) database.Figure 2
Subjects in the JAFFE database.


## 2.2. Image Reconstruction

The left segmented half-face images were reconstructed using the following steps:(1)
Rotate left segmented half-face image through 270°, and denote it asNl1(2)
Rotate the left segmented half-face image through 180°, and denote it asNl2(3)
ConcatenateNl1 and Nl2 as
(1)Tl=Nl1∣Nl2The right segmented half-face images were reconstructed using the following steps:(1)
Rotate right segmented half-face image through 270°, and denote it asNr1(2)
Rotate the right segmented half-face image through 180°, and denote it asNr2(3)
ConcatenateNr2 and Nr1 as
(2)Tr=Nr2∣Nr1Figure3 contains the original full images, left and right half-images, and their reconstructed images used in the test-image database.Figure 3
Reconstructed images from left and right segmented half images.


## 2.3. Research Design

Figure4 shows a design of the recognition module.Figure 4
Research design.


## 2.4. Preprocessing

Preprocessing is an important phase of image processing where the quality of the images is enhanced. The image acquisition process comes with various forms of noise. The preprocessing techniques are used to denoise the images making them better conditioned for recognition.


### 2.4.1. Fast Fourier Transform

Fast Fourier Transform (FFT) was adopted as a noise reduction mechanism. According to Glynn [9], the FFT algorithm reduces the computational burden to ONlogN arithmetic operations.The DFT of a column vectorajk is represented mathematically as
(3)ajk∗=DFTajk=∑r=0p−1ajke−i2πsr/p,where s=0,1,⋯,p−1, j=1,2,⋯,n. ajk is the kth column of the image matrix, Aj [8].Due to the Gaussian nature of illumination variations, the Gaussian filter is adopted for filtering the face images after the Discrete Fourier transformation. After filtering, the inverse Discrete Fourier transformation (IDFT) was performed to reconstruct images into their original forms. The inverse Discrete Fourier transformation (IDFT) is given by(4)ajk=IDFTajk∗=1p∑r=0p−1ajk∗ei2πsr/p,s=0,1,⋯,p−1,j=1,2,⋯,n.The real components of the inverse transformations are extracted for the feature extraction stage whereas the imaginary component is discarded as noise. Figure5 shows the stages in FFT preprocessing of an image (Avuglah et al. [8]).Figure 5
FFT preprocessing cycle.


## 2.4.1. Fast Fourier Transform

Fast Fourier Transform (FFT) was adopted as a noise reduction mechanism. According to Glynn [9], the FFT algorithm reduces the computational burden to ONlogN arithmetic operations.The DFT of a column vectorajk is represented mathematically as
(3)ajk∗=DFTajk=∑r=0p−1ajke−i2πsr/p,where s=0,1,⋯,p−1, j=1,2,⋯,n. ajk is the kth column of the image matrix, Aj [8].Due to the Gaussian nature of illumination variations, the Gaussian filter is adopted for filtering the face images after the Discrete Fourier transformation. After filtering, the inverse Discrete Fourier transformation (IDFT) was performed to reconstruct images into their original forms. The inverse Discrete Fourier transformation (IDFT) is given by(4)ajk=IDFTajk∗=1p∑r=0p−1ajk∗ei2πsr/p,s=0,1,⋯,p−1,j=1,2,⋯,n.The real components of the inverse transformations are extracted for the feature extraction stage whereas the imaginary component is discarded as noise. Figure5 shows the stages in FFT preprocessing of an image (Avuglah et al. [8]).Figure 5
FFT preprocessing cycle.


## 2.5. Feature Extraction

Face image space is very large, and its storage requires reduction of the dimensions of original images using feature extraction methods. This study adopted Principal Component Analysis (PCA) proposed by Kirby and Sirovich [10] for feature extraction. PCA extracts the most significant components or those components which are more informative and less redundant, from the original data. According to Shlens [11], PCA computes the most meaningful basis to reexpress a noisy garbled dataset. The rationale behind this new basis is to filter out the noise and reveal hidden dynamics in the dataset.


## 2.6. Implementation of FFT-PCA/SVD Algorithm

As noted earlier, used images were extracted from the Massachusetts Institute of Technology (MIT) (2003-2005) database and Japanese Female Facial Expressions (JAFFE). The study considered subjects captured under 0° pose from the MIT database and neutral facial expressions from JAFFE. Twenty face images, ten each from the MIT database and JAFFE, were trained, and their reconstructed images from left and right segmented half images were used for testing the recognition algorithm. To keep the data uniform, captured images were digitized into gray-scale precision and resized into200×200 dimensions and the data types changed into double precision for preprocessing.The FFT-PCA/SVD algorithm was used to train the image database. Unique face features of the training set are extracted and stored in memory during the training phase.Now, given the sampleX=X1,X2,⋯,Xn, whose elements are the vectorised form of the individual images in the study. The mean centering preprocessing mechanism is performed by subtracting the mean image from the individual images under study. The mean is given by
(5)a¯j=EXj,j=1,2,⋯,n,a¯j=1N∑i=1NXji=1N∑i=1p∑k=1pajik,j=1,2,3,⋯,n,where N=p×p, length (=rowsofimage×columnsofimage) of the image data, Xj.According to Avuglah et al. [8], the variance-covariance matrix C of the image space is given as
(6)C=1nWWT,where the mean centered matrix is W=w1,w2,⋯,wn.The eigenvalues and their corresponding eigenvectors are extracted from singular value decomposition (SVD) of the matrix,C=UΣVT.This decomposes the covariance matrixC into two orthogonal matrices U and V and a diagonal matrix Σ:
(7)zj=∑j=1nwjuj,where uj is the jth column vector of U.From the training set, the principal components are extracted as(8)γj=zjTxj−a¯,and ΓT=γ1,γ2,⋯,γn.When a new face (test image) is passed through the recognition module, its unique features are extracted as(9)γj∗=zjTxr−a¯,and Γr∗T=γ1∗,γ2∗,⋯,γn∗.The recognition distances (Euclidean distances) are computed as(10)ψ=Γ−Γr∗.The minimum Euclidean distancedji=minψ, j=1,2,⋯,n, i=1,2, is chosen as the recognition distance.


## 3. Results and Discussion

Figures6 and 7 present the recognition matches and distances of the left and right reconstructed face images. It can be seen in Figure 6 that there were two mismatches when the right reconstructed images are used as test images for recognition. Also, there was one mismatch when the left reconstructed images are used as test images for recognition. All recognitions on the MIT (2003-2005) database gave correct matches. This is evident from Figure 7.Figure 6
Recognition of the reconstructed images (JAFFE).Figure 7
Recognition of the reconstructed images (MIT 2003-2005 database).


### 3.1. Numerical Evaluations

The average recognition rate was the numerical assessment criteria used in this study. The average recognition rate,Ravg, of an algorithm is defined as
(11)Ravg=∑i=1trunncritrun×ntot×100,where trun is the number of experimental runs, ncri is the number of correct recognitions in the ith run, and ntot is the total number of faces being tested in each run [7]. The average error rate, Eavg, is defined as 100−Ravg.The total number of correct recognitions∑i−110ncri for the study algorithm is 19.The total number of experimental runstrun=10, and the total number of images in a single experimental run ntot=2.Now, using the left reconstructed face images as test images, the average recognition rate of the study algorithm (FFT-PCA/SVD) is(12)Ravg=19102×100=95%.The average error rate(13)Eavg=100−Ravg=100−95=5%.Using the right reconstructed half-face images as test images, the average recognition rate of the study algorithm (FFT-PCA/SVD) is(14)Ravg=18102×100=90%.The average error rate(15)Eavg=100−Ravg=100−90=10%.


### 3.2. Statistical Evaluation

The paired samplet-test is usually used to evaluate measurements of the same individuals/units recorded under different environmental conditions. The paired responses may then be analysed by computing their differences, thereby eliminating much of the influence of the extraneous unit to unit variation (Johnson and Wichern [12]). Let dj1 denote the recognition distance recorded using the left reconstructed images as test images and dj2 denote the recognition distance using the right reconstructed half-face images as test images for the jth individual, then the paired differences
(16)Dj=dj1−dj2,j=1,2,⋯,n,should reflect the differential effects of the treatments. Given that the difference Dj, j=1,2,⋯,n, are independent observations from Nδ,σd2 distribution, the statistic
(17)t=D¯−δsd/n,where
(18)D¯=1n∑j=1nDj,sd2=1n−1∑j=1nDj−D¯2has a t-distribution with n−1 degrees of freedom. Consequently, an α-level test of the hypothesisH0:δ=0 (mean difference of recognition distances from left and right reconstructed images is zero) againstH1:δ≠0is conducted by comparing∣t∣ with tn−1α/2, the upper 100α/2 percentile of the t-distribution with n−1 degree of freedom. A 1001−α confidence interval for the mean difference in recognition distance δ=Edj1−dj2 is constructed as
(19)d¯±tn−1α2sdn.This test is sensitive to the assumption that the paired difference should come from a univariate normal population. The Shapiro-Wilk test on the observed difference gave a value of 0.980 with ap value of 0.933. This shows that the distribution of the observed difference is the same as the expected distribution (normal). The Q‐Q plot shown in Figure 8 confirms that the observed difference is normally distributed.Figure 8
NormalQ‐Q plot of observed difference.The paired sample correlations between the distance for the left and right reconstructed face images are 0.858 with ap value of 0.000. This indicates that there exists a strong positive linear relationship between the Euclidean distance for the left and right reconstructed face images. The p value of 0.000 shows that this relationship is significant. The results of the paired sample t-test are shown in Table 1.Table 1
Paired samplet-test.
RRI-LRIStd. Dev.SE95% CItdfSig.MeanLowerUpper42.58565205.254445.89628-53.47637138.647670.928190.365From Table1, the average difference between the left and right reconstructed face images (LRI and RRI, respectively) is 42.5865. The test statistic value from the paired sample test is 0.928 corresponding to a p value of 0.365. It can be inferred from the p value that there is no significant difference between the average recognition distance for the left and right reconstructed face images. This means that the reconstructed images have the same average recognition distances at 5% level of significance when used as test images.


## 3.1. Numerical Evaluations

The average recognition rate was the numerical assessment criteria used in this study. The average recognition rate,Ravg, of an algorithm is defined as
(11)Ravg=∑i=1trunncritrun×ntot×100,where trun is the number of experimental runs, ncri is the number of correct recognitions in the ith run, and ntot is the total number of faces being tested in each run [7]. The average error rate, Eavg, is defined as 100−Ravg.The total number of correct recognitions∑i−110ncri for the study algorithm is 19.The total number of experimental runstrun=10, and the total number of images in a single experimental run ntot=2.Now, using the left reconstructed face images as test images, the average recognition rate of the study algorithm (FFT-PCA/SVD) is(12)Ravg=19102×100=95%.The average error rate(13)Eavg=100−Ravg=100−95=5%.Using the right reconstructed half-face images as test images, the average recognition rate of the study algorithm (FFT-PCA/SVD) is(14)Ravg=18102×100=90%.The average error rate(15)Eavg=100−Ravg=100−90=10%.


## 3.2. Statistical Evaluation

The paired samplet-test is usually used to evaluate measurements of the same individuals/units recorded under different environmental conditions. The paired responses may then be analysed by computing their differences, thereby eliminating much of the influence of the extraneous unit to unit variation (Johnson and Wichern [12]). Let dj1 denote the recognition distance recorded using the left reconstructed images as test images and dj2 denote the recognition distance using the right reconstructed half-face images as test images for the jth individual, then the paired differences
(16)Dj=dj1−dj2,j=1,2,⋯,n,should reflect the differential effects of the treatments. Given that the difference Dj, j=1,2,⋯,n, are independent observations from Nδ,σd2 distribution, the statistic
(17)t=D¯−δsd/n,where
(18)D¯=1n∑j=1nDj,sd2=1n−1∑j=1nDj−D¯2has a t-distribution with n−1 degrees of freedom. Consequently, an α-level test of the hypothesisH0:δ=0 (mean difference of recognition distances from left and right reconstructed images is zero) againstH1:δ≠0is conducted by comparing∣t∣ with tn−1α/2, the upper 100α/2 percentile of the t-distribution with n−1 degree of freedom. A 1001−α confidence interval for the mean difference in recognition distance δ=Edj1−dj2 is constructed as
(19)d¯±tn−1α2sdn.This test is sensitive to the assumption that the paired difference should come from a univariate normal population. The Shapiro-Wilk test on the observed difference gave a value of 0.980 with ap value of 0.933. This shows that the distribution of the observed difference is the same as the expected distribution (normal). The Q‐Q plot shown in Figure 8 confirms that the observed difference is normally distributed.Figure 8
NormalQ‐Q plot of observed difference.The paired sample correlations between the distance for the left and right reconstructed face images are 0.858 with ap value of 0.000. This indicates that there exists a strong positive linear relationship between the Euclidean distance for the left and right reconstructed face images. The p value of 0.000 shows that this relationship is significant. The results of the paired sample t-test are shown in Table 1.Table 1
Paired samplet-test.
RRI-LRIStd. Dev.SE95% CItdfSig.MeanLowerUpper42.58565205.254445.89628-53.47637138.647670.928190.365From Table1, the average difference between the left and right reconstructed face images (LRI and RRI, respectively) is 42.5865. The test statistic value from the paired sample test is 0.928 corresponding to a p value of 0.365. It can be inferred from the p value that there is no significant difference between the average recognition distance for the left and right reconstructed face images. This means that the reconstructed images have the same average recognition distances at 5% level of significance when used as test images.


## 4. Conclusion and Recommendation

The average recognition rates for the FFT-PCA/SVD algorithm were 95% and 90% when left and right reconstructed face images are used as test images, respectively. It could be inferred from the above results that the recognition algorithm has relatively higher performance when left reconstructed images are used as test images. The statistical assessment revealed that there is no significant difference between the average recognition distances for the left and right reconstructed images.It can therefore be concluded that the average recognition distance for left reconstructed images is not significantly different from the average recognition distance for right reconstructed images when FFT-PCA/SVD is used for recognition. These results are consistent with the findings of Singh and Nandi [6]. It is worthy of note that apart from the numerical evaluations which are mostly adopted in literature, this study used a more data-driven approach or a statistical approach to also assess the performance of the recognition algorithm on left and right reconstructed face image databases. The performances of FFT-PCA/SVD on both databases are viable. FFT-PCA/SVD is therefore recommended for recognition of left and right reconstructed face images.



---

*Source: 9127465-2020-10-05.xml*