# An Improved Metric Learning Approach for Degraded Face Recognition

Authors: Guofeng Zou; Yuanyuan Zhang; Kejun Wang; Shuming Jiang; Huisong Wan; Guixia Fu
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

To solve the matching problem of the elements in different data collections, an improved coupled metric learning approach is proposed. First, we improved the supervised locality preserving projection algorithm and added the within-class and between-class information of the improved algorithm to coupled metric learning, so a novel coupled metric learning method is proposed. Furthermore, we extended this algorithm to nonlinear space, and the kernel coupled metric learning method based on supervised locality preserving projection is proposed. In kernel coupled metric learning approach, two elements of different collections are mapped to the unified high dimensional feature space by kernel function, and then generalized metric learning is performed in this space. Experiments based on Yale and CAS-PEAL-R1 face databases demonstrate that the proposed kernel coupled approach performs better in low-resolution and fuzzy face recognition and can reduce the computing time; it is an effective metric method.


---

## Body

## 1. Introduction

The metric is a function which gives the scalar distance between two patterns. Distance metric is an important basis for similarity measure between samples, and it is one of the core issues in pattern recognition. The aim of distance metric learning is to find a distance metric matrix; its essence is to obtain another representation method with better class separability by linear or nonlinear transformation.In recent years, some researches about distance metric have been done by researchers [1–7]. They learn a distance metric by introducing sample similarity constraint or category information; the distance metric is used to improve the data clustering or classification. These researches can be concluded to two categories: linear distance metric learning and nonlinear distance metric learning. The linear distance metric learning is equivalent to learning a linear transformation in sample space, including a variety of common linear dimensionality reduction methods, such as principal component analysis [8], linear discriminant analysis [9], and independent component analysis method [10]. The nonlinear distance metric learning is equivalent to learning a nonlinear transformation in sample space; the locally linear embedding [11], isometric mapping [12], and Laplace mapping [13] are the traditional nonlinear methods. Recently, some new nonlinear distance metric methods have been proposed. Baghshah and Shouraki [14] proposed the nonlinear metric learning method based on pair-wise similarity and dissimilarity constraints and the geometrical structure of data. Babagholami-Mohamadabadi et al. [15] proposed the probabilistic nonlinear distance metric learning. The deep nonlinear metric learning method [16] based on neural networks is a new nonlinear metric learning method. In addition, there are some more flexible distance metric learning algorithms, which are based on kernel matrix [7, 17, 18].These traditional distance metric learning methods are defined on the set of single attribute. If the elements belong to different sets with different attribute, these distance measurement methods are incapable for the distance metric. For example, for two images with different resolution, which can be considered to belong to different sets, obviously, the traditional distance metric method will not be able to directly calculate the distance. The general approach is normalized operation before recognition by using interpolation algorithm or sampling algorithm. But the interpolation inevitably introduced false information, and sampling may miss some useful information, so it is difficult to get high recognition rate.Aiming at the shortage of traditional distance metric, Li et al. proposed the coupled metric learning (CML) [19–21]. The goal of coupled metric learning is to find a coupled distance function to meet the specific requirement in given task. The essential idea is that, firstly, the data in different collections are projected to the unified coupled space, and the data should be as close as possible in this new space. Then, the generalized metric learning is performed in this unified coupled space. Obliviously, this new metric method will have broader application scope and better recognition effect. However, the proposed coupled metric learning methods are based on linear transformation, which can be called linear coupled metric learning (LCML). These methods have two shortages in dealing with practical problems. First, the practical problems usually are nonlinear, and the linear transformation does not represent the features effectively. Secondly, the image needs to be converted into one-dimensional vector in the LCML algorithm; it is easy to cause the increasing of dimensions of autocorrelation matrix.Based on the idea of coupled metric learning, we improved the supervised locality preserving projection algorithm and added supervised locality preserving information to coupled metric learning. So the improved coupled metric learning approach based on supervised locality preserving projection (SLPP-CML) was proposed. Introducing kernel technology into the coupled metric learning, we proposed the kernel coupled metric learning approach based on supervised locality preserving projection (SLPP-KCML). The SLPP-KCML realized the extension from linear coupled metric to nonlinear coupled metric. To verify the effectiveness of the proposed method, the experiments based on two face databases were performed. The experimental results show that a higher recognition rate can be achieved in the SLPP-KCML algorithm, and the operation time is reduced greatly.


## 2. Related Works

The traditional distance metric learning algorithm is to learn a distance functiond(xi,xj) between the data points expressed as follows:
(1)d(xi,xj)=∥xi-xj∥A=(xi-xj)TA(xi-xj).Distance metric learning aims to find a distance metric matrixA; it is required that A is a real symmetric and positive semidefinite matrix; namely, A=PTP, where P is a transformation matrix
(2)dA(xi,xj)=dP(xi,xj)=(Pxi-Pxj)T(Pxi-Pxj).Obviously, the distance metric learning is realized by learning a transformation matrixP, so the process of distance metric learning is equivalent to the process of obtaining other representation forms with better separability through linear or nonlinear transformation of the samples.IfX⊂RDx, Y⊂RDy represent two different collections, respectively, the function d(x,y) is the distance metric between data x∈X and data y∈Y. If Dx≠Dy, the traditional method does not work for distance metric. Even if Dx=Dy, because the data x∈X and data y∈Y, which belong to different attribute collections, the distance metric has no physical meaning.The coupled distance metric is a distance function for the data elements of different kinds of collections. The elements of collectionsX and Y are mapped from the original space to a common coupled space RDC by using the mapping functions fx and fy. Then, the distance metric is performed in the coupled space. The measured distance can be represented mathematically as
(3)dc(x,y)=dA(x~,y~)=dA(fx(x),fy(y))=∥fx(x)-fy(y)∥A=(fx(x)-fy(y))TA(fx(x)-fy(y)),
where matrix A is a real symmetric and positive semidefinite matrix. Letting A=WaWaT, we can get
(4)dc(x,y)=(fx(x)-fy(y))TWaWaT(fx(x)-fy(y))=(WaTfx(x)-WaTfy(y))T(WaTfx(x)-WaTfy(y))=∥WaT(fx(x)-fy(y))∥.The goal of coupled metric learning can be achieved by minimizing the distance function; the objective function is as follows:(5)min⁡J=min⁡∑(i,j)∈C∥WaT(fx(xi)-fy(yj))∥2,
where C is a correlation matrix of elements in collections  X and Y. According to different supervised information, we can obtain different matrix C, so as to realize the different coupled metric learning.


## 3. The Coupled Metric Learning Based on Supervised Locality Preserving Projection (SLPP-CML)

The coupled distance metric learning must be used under the constraints of supervised information. In this paper, we improved the supervised locality preserving projection (SLPP) algorithm [22]. Based on the improved SLPP algorithm, we proposed the coupled metric learning method based on supervised locality preserving projection.In order to better illustrate the coupled metric learning algorithm based on supervised locality preserving projection, we provide a theorem about the matrix norm.Theorem 1.
LettingA∈Rn×n, then the Frobenius norm has the following properties:(1)
∥A∥2=Tr⁡(ATA)=∑i=1nλi(ATA), where λi is the eigenvalue of the matrix ATA;(2)
Tr⁡(ATA)=Tr⁡(AAT), Tr⁡(·) represents the trace operation.The coupled metric learning based on supervised locality preserving projection includes the following steps.Step  1 (building the neighborhood relation in the same collection). We use the k nearest neighbor method. First, building within-class adjacency graph in the same collection, if the data point xi(yi) is one of the k within-class nearest neighbors of data point xj(yj), we connect these two data points; and then, building between-class adjacency graph in same collection; if the data point xi(yi) is one of the k between-class nearest neighbors of data point xj(yj), these two data points are connected.Step  2 (building the connected relation between two collections). If the data points xi and yj in two different collections belong to the same class, then these two points are connected, otherwise not connected.Step  3 (constructing the relation matrix in the same collection). According to the neighborhood relations, the relation matrixes (similarity matrixes) of within-class and between-class are constructed, respectively.Within-class similarity matrix isW corresponding to within-class adjacency graph and the within-class similarity value is Wij. The definition is as follows:
(6)Wij={exp⁡(-∥xi-xj∥2t)ifxiconnectedxj,0otherwise.Between-class similarity matrix isB corresponding to between-class adjacency graph and the between-class similarity value is Bij. It can be defined as follows:
(7)Bij={exp⁡(-∥xi-xj∥2t)ifxiconnectedxj,0otherwise,
where parameter t is the average distance between all sample points.Step 4 (constructing the relation matrix S between two collections). The similarity value is as follows:
(8)Sij={1ifxiconnectedyj,0otherwise.Step  5 (calculating the final similarity matrix C between two collections). As shown in Figure 1, the similarity relations between element x1∈X and elements of collection Y include the following several situations.(a)
The similarity betweenx1 and y1: these two data points in different collections belong to the same class and they are connected to each other, so the similarity of which is C11=S11=1.(b)
The similarity betweenx1 and y5: these two data points belong to different class, but the relationship between y5 and y3 is the between-class neighborhood relation in same collection, and the similarity B35 is the maximum similarity value, so similarity between x1 and y5 is C15=B35.(c)
The similarity betweenx1 and y6: these two data points belong to different class and the y6 does not have between-class neighborhood relation with any element of collection Y of class 1. But there is a between-class neighborhood relation of same collection between y5 and y3, and within-class neighborhood relation between y5 and y6. So the similarity between x1 and y6 is defined as the product of between-class similarity B35 and within-class similarity W56, which is the maximum similarity between y6 and y3; that is, C16=B35·W56.(d)
The similarity betweenx1 and y9: these two data points belong to different class; there are not any between-class neighborhood relations between the elements of class 1 and class 3 in collection Y; namely, C19=0.Figure 1
The relationship between the elements of different collections.Step  6 (constructing the optimal objective function). Cosider the following:
(9)J=∑(i,j)∈C∥WaT(fx(xi)-fy(yj))∥2=∑i∑j∥WaT(fx(xi)-fy(yj))∥2·Cij,
where the functions fx and fy are considered to be linear; that is, fx(x)=WxTx, fy(y)=WyTy. The optimal objective function can be rewritten as follow:
(10)J=∑i∑j∥WaT(fx(xi)-fy(yj))∥2·Cij=∑i∑j∥WaTWxTxi-WaTWyTyj∥2·Cij.LettingPx=WxWa, Py=WyWa, we can get
(11)J=∑i∑j∥WaTWxTxi-WaTWyTyj∥2·Cij=∑i∑j∥PxTxi-PyTyj∥2·Cij.
Therefore, our method aims to learn two linear transformations Px and Py.According to Theorem1, (12) is an alternate matrix expression of (11):
(12)J=Tr⁡[PxTXF1(C)XTPx+PyTYF2(C)YTPy-PxTXCYTPy-PyTYCTXTPx]=Tr⁡([PxPy]T[XY][F1(C)-C-CTF2(C)][XY]T[PxPy]),
where Tr⁡(X) represent computing the trace of matrix X, F1(C) and F2(C) are diagonal matrixes, and their diagonal elements are the row or column sums of similarity matrix C, respectively.Assuming thatP=[PxPy], Z=[XY], Γ=[F1(C)-C-CTF2(C)], (12) can be rewritten as follow:
(13)J(P)=Tr⁡[PTZΓZTP].To make the equation have a unique solution,PTZZTP=I and PTZe=0 as constraints are added, where e=[1,1,…1]T is a vector with dimensions of (Nx+Ny)×1, Nx and Ny are the numbers of samples in collections  X and Y. The solution to make (13) minimized is obtained by generalized eigendecomposition of (ZΓZT)p=λ(ZZT)p and taking eigenvectors p2,p3,…pm+1 corresponding to the second to (m+1)th smallest eigenvalues λ2,λ3,…λm+1. Assuming that P=[p2,p3,…pm+1], its dimension is (Dx+Dy)×m, Dx and Dy are the dimensions of samples in collections X and Y, so the transformation matrix Px corresponds to the 1st to Dxth rows of P and Py corresponds to the (Dx+1)th to Dyth rows of P.Step  7. Bringing the matrix Px and Py to (11), the distance metric of the elements belonging to different collections can be realized.


## 4. The Kernel Coupled Metric Learning Based on Supervised Locality Preserving Projection (SLPP-KCML)

In practical dimension reduction and measurement process, the linear model is not well to represent the features, and it is difficult to map two complex collections to the same space using the linear transformation. So combining the kernel method, we extend the SLPP-CML algorithm; a nonlinear coupled metric learning methods based on the supervised locality preserving projection is proposed.Assuming that the mapping functionsfx and fy are nonlinear functions, namely, fx=ϕx(x), fy=ϕy(y), using the nonlinear mapping ϕ:Rn→F, x→ϕx(x), y→ϕy(y), the sample data can be mapped to the high dimensional Hilbert space. The criterion can be defined by
(14)J=∑(i,j)∈C∥WaTϕx(xi)-WaTϕy(yj)∥2.An alternate matrix expression is as follow:(15)J=Tr⁡(WaTΦx(x)GxΦx(x)TWa+WaTΦy(y)GyΦy(y)TWa-WaTΦx(x)CΦy(y)TWa-WaTΦy(y)CTΦx(x)TWaWaT)=Tr⁡([WaWa]T[Φx(x)Φy(y)][Gx-C-CTGy]×[Φx(x)Φy(y)]T[WaWa]),
where Tr⁡ represent computing the trace of matrix, Gx and Gy are diagonal matrixes, and their diagonal elements are the row or column sums of similarity matrix C, respectively.LettingWa=Φx(x)Ax, Wa=Φy(y)Ay, we can get
(16)J=Tr⁡([Φx(x)AxΦy(y)Ay]T[Φx(x)Φy(y)][Gx-C-CTGy]×[Φx(x)Φy(y)]T[Φx(x)AxΦy(y)Ay]).The kernel functionKi,jx=(ϕ(xi)·ϕ(xj)), Ki,jy=(ϕ(yi)·ϕ(yj)); the kernel matrixes Kx and Ky are real symmetric matrices. Equation (17) is an alternative expression of (16):
(17)J(Ax,Ay)=Tr⁡([AxAy]T[KxKy][Gx-C-CTGy]×[KxKy]T[AxAy]).Obviously, the coupled metrics learning in kernel space is a process of calculating the transformation matrixAx and Ay.Assuming thatA=[AxAy], K=[KxKy], Γ=[Gx-C-CTGy], similar to the SLPP-CML algorithm, solving the optimal solution can be transformed into the generalized eigenvalue problem. The generalized characteristic equation is Ea=λFa, and E=KΓKT, F=KKT; a is the eigenvector corresponding to eigenvaule λ. The eigenvectors corresponding to the minimum to the Dcth smallest eigenvalues construct the feature matrix A; the size of matrix A is (Nx+Ny)×Dc, where Nx and Ny are the numbers of training samples of collections  X and Y. Finally, we can get the feature matrix Ax corresponding to the data matrix X and the feature matrix Ay corresponding to the data matrix Y.In addition, the samples mapped to the high dimensional space need centering processing. In the linear coupled metric learning, the centering can be realized by abandoning the eigenvector corresponding to eigenvalue of “zero.” However, the centering of nonlinear coupled metric learning in kernel space can be realized by centering the kernel matrixKx and Ky(18)K~=K-1nK1nn-1n1nnK+1n21nnK1nn,
where n is the dimension of kernel matrix K. 1nn is a matrix with size of n×n and all elements are one.


## 5. Experiment and Analysis

### 5.1. Introduction of the Face Database

The proposed coupled metric learning approach is used for face recognition. It is tested on Yale face database [23] and CAS-PEAL-R1 face database [24]. The Yale face database contains 165 pictures of 15 people with the size of 100×100 and 256 gray levels. These images were taken in different expression and illumination conditions. In experiment, we used the former 6 images per person as training samples, a total of 90, and the other images were used as test samples.The CAS-PEAL-R1 face database contains 30863 face images, which was divided into two parts:(1) the frontal face image subset; (2) the nonfrontal face image subset. In the experiment, we used the accessory data set of the frontal face image subsets (CAS-PEAL-R1-FRONTAL-Accessory). The face images per person in CAS-PEAL-R1-FRONTAL-Accessory contain 6 different appendages; there are 3 images with different glasses and 3 images with different hats. We selected 300 images of 50 people with the size of 360×480 and 256 gray levels in the experiment; the odd-numbered images were used as training samples and even-numbered images were used as test samples, respectively. Some training images are shown in Figure 2 and some test images are shown in Figure 3.Some original training face images.
(a)
Some images in Yale face database
(b)
Some images in CAS-PEAL-R1-FRONTAL-AccessorySome original test face images.
(a)
Some images in Yale face database
(b)
Some images in CAS-PEAL-R1-FRONTAL-Accessory


### 5.2. The Low-Resolution Face Recognition

Due to the differences between the different resolution cameras and the uncertainty of distance between camera and face, the resolution of face image that we collected is not uniform. Obviously, traditional measurement method can not be used to calculate the distance between two images with different resolution. The general handling method is interpolation operation, but the interpolation operation is easy to introduce false information. With the increase in false information, the distortion degree increases, as shown in Figure4. Aiming at the problem of recognition rate declining because of image distortion, the researchers realized the low-resolution image compensation by increasing the image restoration preprocessing. But the image restoration algorithm is more complex, and the quality of image restoration has great impact on final recognition results.(a) (A), (B), and (C) are the normal face, low-resolution face, and face after restoration, respectively, in Yale face database. (b) (A), (B), and (C) are the normal face, low-resolution face, and face after restoration, respectively, in CAS-PEAL-R1-FRONTAL-Accessory.
(a)
(b)However, the proposed coupled metric learning method can directly realize the feature extraction and measurement of different resolution images. This method not only saves computing time, but also avoids the negative impact of image restoration on recognition performance. To better illustrate the experimental processes, Figure5 gives the flow of the degraded face recognition.Figure 5
Degraded face recognition process. In training process, we obtained the matrixAD for degraded face and the matrix AN for clear face image. In test process, matrices AD and AN transformed the degraded test face and clear test face to the coupled metric space, calculating the distance.In experiment, the training samples include clear face and degraded face images. The size of original normal training face image is adjusted to64×64 pixel, and these adjusted faces are used as clear face images. However, there are not original low-resolution face images in public face database, so we obtained the low-resolution training face image through blurring and sampling original normal training face image, and the size of low-resolution face image is 16×16.The test samples are the low-resolution face images, which were generated through blurring and sampling original normal test face image introduced in Section5.1.


#### 5.2.1. Experiment 1: The Low-Resolution Face Recognition Based on SLPP-CML

Through the theoretical analysis, the SLPP-CML algorithm has two influence factors:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature. Therefore, the recognition results based on different parameters should be discussed and analyzed. Figure 6 shows the change of recognition rate with the change of feature dimensions, when the number of neighbors takes different values.The recognition rate under different dimensions and different numbers of the nearest neighbors.
(a)
The experiment result in Yale face database
(b)
The experiment result in CAS-PEAL-R1 face databaseThese recognition rate curves are in two different face databases. These curves have a general change law, with the increase in feature dimensions; the recognition rate kept a decreasing trend after increasing, and the best recognition results​​can be achieved only in the optimal feature dimensions.In Yale face database, the recognition rate kept a higher trend when feature dimensions remain 10–20. The optimal recognition rate is 86.67% when feature dimension is 10 and the number of neighbors is 5. In CAS-PEAL-R1 face database, the recognition rate can reach the maximum value 86.67%, when feature dimension is 40 and the number of neighbors is 2.The experimental data show that the number of training samples of each class is 6 in Yale face database and the recognition effect is optimal when the number of neighbors is 5. In CAS-PEAL-R1 face database, we can obtain optimal recognition rate when the number of training samples of each class is 3 and the number of neighbors is 2. Obviously, the number of neighbors isT-1; T is the number of training samples of each class.In addition, in order to illustrate the effectiveness of SLPP-CML method, the comparative experiments were carried out. The experiment results are shown in Table1.Table 1
Experimental comparison of this method with other methods.
Method
Yale face database
CAS-PEAL-R1 face database
Image restoration [25] + PCA
61.33
55.33
CML [19]
77.33
74.67
CLPM [21]
82.67
80.67
SLPP-CML
86.67
86.67The experimental data illustrated that the recognition results of feature extraction after restoration are not satisfactory. The coupled metric learning in [19] can not overcome the influences of within-class multiple modes, so the identification effect is not good. The coupled metric learning in [21] is conducive to resolving within-class multiple modes; the recognition effects have been greatly improved, but it does not fully consider the between-class relationships of training samples. The proposed SLPP-CML takes advantage of the supervisory of category information, while the within-class and between-class relationship information of training samples have been considered into the metric learning, so we can get better recognition results.


#### 5.2.2. Experiment 2: The Low-Resolution Face Recognition Based on SLPP-KCML

The SLPP-KCML algorithm is a nonlinear coupled metric learning algorithm. Through the analysis, there are three factors which affect this algorithm:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature; (3) the kernel function. Based on the experiment result, the number of the nearest neighbors is the same as that of SLPP-CML. For the kernel function, we choose the Gauss function k(x,y)=exp⁡(-∥x-y∥2/α); the value of adjustable factor α affects the function performance. So, in this paper, the experiments were carried out according to the different adjustable factors and the change of feature dimensions on recognition rate; the experimental results are shown in Figure 7.The recognition rate under different dimensions and different adjustable factors.
(a)
The experiment result in Yale face database
(b)
The experiment result in Yale CAS-PEAL-R1 face databaseThe curves indicated that, in Yale face database, the optimal recognition rate is 89.33% when the value ofα is 0.5 and the feature dimension is 20; compared with the SLPP-CML algorithm, the recognition rate increased by 2.66%. In CAS-PEAL-R1 face database, when α=0.7 and the feature dimensions Dc=40, recognition rate is 91.33%; compared with the SLPP-CML algorithm, the recognition rate increased by 4.66%. Obviously, the nonlinear coupled metric learning method can effectively extract the classification information of face image and obtain a high recognition rate.Considering the training time, the SLPP-CML algorithm requires calculating the eigenvalue and eigenvector of the image covariance matrix. The resolution of clear face image is64×64 pixel and low-resolution face image is 16×16 pixel. So the dimensions of image covariance matrix are 4352×4352 and the average training time is about 553.905 seconds.However, the matrix of SLPP-KCML is related to the number of classes and the number of training samples of each class, so the dimension of the covariance matrix of Yale face database is90×90, the size of covariance matrix of CAS-PEAL-R1 face database is 150×150, and the average training time is about 6.687 seconds. Obviously, the training speed of the SLPP-KCML algorithm is faster than SLPP-CML algorithm, and the recognition time of these two algorithms is about 0.0225 seconds. Based on the above analysis, the efficiency of SLPP-KCML algorithm is better than SLPP-CML algorithm.


### 5.3. The Fuzzy Face Recognition

Besides the low-resolution face image, the blurring image usually makes the performance of face recognition system decrease. The fuzzy face image is shown in Figure8. Obviously, it is difficult to identify the fuzzy face image; a part of face details can be restored by deblurring algorithm, but it still cannot provide enough information in identification.(a) (A), (B), and (C) are the normal face, blurred face image, and deblurring results, respectively, in Yale face database. (b) (A), (B), and (C) are the normal face, blurred face image, and deblurring face, respectively, in CAS-PEAL-R1-FRONTAL-Accessory.
(a)
(b)Similar to Section5.2, we carried out the recognition experiments of SLPP-CML algorithm based on different face databases, and the comparative experiments with other methods were made. In the experiment, the clear images with size of 64×64 are the same as those used in Section 5.1, and the fuzzy face images were generated by convolution of corresponding clear face image. The training samples are composed by clear training face images and generated fuzzy face images. The test samples are the fuzzy face images by convolution of the clear test face images and the size is 64×64 pixels. The number of training and test samples has been introduced in Section 5.1. The experiment results are shown in Table 2.Table 2
Experimental comparison of this method with other methods.
Method
Yale face database
CAS-PEAL-R1 face database
Deblurring [26] + PCA
64.00
68.67
CML [19]
77.33
79.33
CLPM [21]
85.67
83.33
SLPP-CML
86.67
88.67
SLPP-KCML
90.00
92.66The experimental data is the best recognition rate of each algorithm; the number of neighbors of SLPP-CML and SLPP-KCML algorithm isT-1, where T is the number of training samples of each class. The optimal value of adjustable factor of Gaussian kernel function in SLPP-KCML algorithm is 0.7. Table 3 gives the feature dimensions of training samples in Yale and CAS-PEAL-R1 face database of SLPP-CML and SLPP-KCML algorithm.Table 3
The feature dimensions in our proposed algorithm.
Method
Yale face database
CAS-PEAL-R1 face database
SLPP-CML
15
39
SLPP-KCML
18
40


## 5.1. Introduction of the Face Database

The proposed coupled metric learning approach is used for face recognition. It is tested on Yale face database [23] and CAS-PEAL-R1 face database [24]. The Yale face database contains 165 pictures of 15 people with the size of 100×100 and 256 gray levels. These images were taken in different expression and illumination conditions. In experiment, we used the former 6 images per person as training samples, a total of 90, and the other images were used as test samples.The CAS-PEAL-R1 face database contains 30863 face images, which was divided into two parts:(1) the frontal face image subset; (2) the nonfrontal face image subset. In the experiment, we used the accessory data set of the frontal face image subsets (CAS-PEAL-R1-FRONTAL-Accessory). The face images per person in CAS-PEAL-R1-FRONTAL-Accessory contain 6 different appendages; there are 3 images with different glasses and 3 images with different hats. We selected 300 images of 50 people with the size of 360×480 and 256 gray levels in the experiment; the odd-numbered images were used as training samples and even-numbered images were used as test samples, respectively. Some training images are shown in Figure 2 and some test images are shown in Figure 3.Some original training face images.
(a)
Some images in Yale face database
(b)
Some images in CAS-PEAL-R1-FRONTAL-AccessorySome original test face images.
(a)
Some images in Yale face database
(b)
Some images in CAS-PEAL-R1-FRONTAL-Accessory


## 5.2. The Low-Resolution Face Recognition

Due to the differences between the different resolution cameras and the uncertainty of distance between camera and face, the resolution of face image that we collected is not uniform. Obviously, traditional measurement method can not be used to calculate the distance between two images with different resolution. The general handling method is interpolation operation, but the interpolation operation is easy to introduce false information. With the increase in false information, the distortion degree increases, as shown in Figure4. Aiming at the problem of recognition rate declining because of image distortion, the researchers realized the low-resolution image compensation by increasing the image restoration preprocessing. But the image restoration algorithm is more complex, and the quality of image restoration has great impact on final recognition results.(a) (A), (B), and (C) are the normal face, low-resolution face, and face after restoration, respectively, in Yale face database. (b) (A), (B), and (C) are the normal face, low-resolution face, and face after restoration, respectively, in CAS-PEAL-R1-FRONTAL-Accessory.
(a)
(b)However, the proposed coupled metric learning method can directly realize the feature extraction and measurement of different resolution images. This method not only saves computing time, but also avoids the negative impact of image restoration on recognition performance. To better illustrate the experimental processes, Figure5 gives the flow of the degraded face recognition.Figure 5
Degraded face recognition process. In training process, we obtained the matrixAD for degraded face and the matrix AN for clear face image. In test process, matrices AD and AN transformed the degraded test face and clear test face to the coupled metric space, calculating the distance.In experiment, the training samples include clear face and degraded face images. The size of original normal training face image is adjusted to64×64 pixel, and these adjusted faces are used as clear face images. However, there are not original low-resolution face images in public face database, so we obtained the low-resolution training face image through blurring and sampling original normal training face image, and the size of low-resolution face image is 16×16.The test samples are the low-resolution face images, which were generated through blurring and sampling original normal test face image introduced in Section5.1.


### 5.2.1. Experiment 1: The Low-Resolution Face Recognition Based on SLPP-CML

Through the theoretical analysis, the SLPP-CML algorithm has two influence factors:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature. Therefore, the recognition results based on different parameters should be discussed and analyzed. Figure 6 shows the change of recognition rate with the change of feature dimensions, when the number of neighbors takes different values.The recognition rate under different dimensions and different numbers of the nearest neighbors.
(a)
The experiment result in Yale face database
(b)
The experiment result in CAS-PEAL-R1 face databaseThese recognition rate curves are in two different face databases. These curves have a general change law, with the increase in feature dimensions; the recognition rate kept a decreasing trend after increasing, and the best recognition results​​can be achieved only in the optimal feature dimensions.In Yale face database, the recognition rate kept a higher trend when feature dimensions remain 10–20. The optimal recognition rate is 86.67% when feature dimension is 10 and the number of neighbors is 5. In CAS-PEAL-R1 face database, the recognition rate can reach the maximum value 86.67%, when feature dimension is 40 and the number of neighbors is 2.The experimental data show that the number of training samples of each class is 6 in Yale face database and the recognition effect is optimal when the number of neighbors is 5. In CAS-PEAL-R1 face database, we can obtain optimal recognition rate when the number of training samples of each class is 3 and the number of neighbors is 2. Obviously, the number of neighbors isT-1; T is the number of training samples of each class.In addition, in order to illustrate the effectiveness of SLPP-CML method, the comparative experiments were carried out. The experiment results are shown in Table1.Table 1
Experimental comparison of this method with other methods.
Method
Yale face database
CAS-PEAL-R1 face database
Image restoration [25] + PCA
61.33
55.33
CML [19]
77.33
74.67
CLPM [21]
82.67
80.67
SLPP-CML
86.67
86.67The experimental data illustrated that the recognition results of feature extraction after restoration are not satisfactory. The coupled metric learning in [19] can not overcome the influences of within-class multiple modes, so the identification effect is not good. The coupled metric learning in [21] is conducive to resolving within-class multiple modes; the recognition effects have been greatly improved, but it does not fully consider the between-class relationships of training samples. The proposed SLPP-CML takes advantage of the supervisory of category information, while the within-class and between-class relationship information of training samples have been considered into the metric learning, so we can get better recognition results.


### 5.2.2. Experiment 2: The Low-Resolution Face Recognition Based on SLPP-KCML

The SLPP-KCML algorithm is a nonlinear coupled metric learning algorithm. Through the analysis, there are three factors which affect this algorithm:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature; (3) the kernel function. Based on the experiment result, the number of the nearest neighbors is the same as that of SLPP-CML. For the kernel function, we choose the Gauss function k(x,y)=exp⁡(-∥x-y∥2/α); the value of adjustable factor α affects the function performance. So, in this paper, the experiments were carried out according to the different adjustable factors and the change of feature dimensions on recognition rate; the experimental results are shown in Figure 7.The recognition rate under different dimensions and different adjustable factors.
(a)
The experiment result in Yale face database
(b)
The experiment result in Yale CAS-PEAL-R1 face databaseThe curves indicated that, in Yale face database, the optimal recognition rate is 89.33% when the value ofα is 0.5 and the feature dimension is 20; compared with the SLPP-CML algorithm, the recognition rate increased by 2.66%. In CAS-PEAL-R1 face database, when α=0.7 and the feature dimensions Dc=40, recognition rate is 91.33%; compared with the SLPP-CML algorithm, the recognition rate increased by 4.66%. Obviously, the nonlinear coupled metric learning method can effectively extract the classification information of face image and obtain a high recognition rate.Considering the training time, the SLPP-CML algorithm requires calculating the eigenvalue and eigenvector of the image covariance matrix. The resolution of clear face image is64×64 pixel and low-resolution face image is 16×16 pixel. So the dimensions of image covariance matrix are 4352×4352 and the average training time is about 553.905 seconds.However, the matrix of SLPP-KCML is related to the number of classes and the number of training samples of each class, so the dimension of the covariance matrix of Yale face database is90×90, the size of covariance matrix of CAS-PEAL-R1 face database is 150×150, and the average training time is about 6.687 seconds. Obviously, the training speed of the SLPP-KCML algorithm is faster than SLPP-CML algorithm, and the recognition time of these two algorithms is about 0.0225 seconds. Based on the above analysis, the efficiency of SLPP-KCML algorithm is better than SLPP-CML algorithm.


## 5.2.1. Experiment 1: The Low-Resolution Face Recognition Based on SLPP-CML

Through the theoretical analysis, the SLPP-CML algorithm has two influence factors:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature. Therefore, the recognition results based on different parameters should be discussed and analyzed. Figure 6 shows the change of recognition rate with the change of feature dimensions, when the number of neighbors takes different values.The recognition rate under different dimensions and different numbers of the nearest neighbors.
(a)
The experiment result in Yale face database
(b)
The experiment result in CAS-PEAL-R1 face databaseThese recognition rate curves are in two different face databases. These curves have a general change law, with the increase in feature dimensions; the recognition rate kept a decreasing trend after increasing, and the best recognition results​​can be achieved only in the optimal feature dimensions.In Yale face database, the recognition rate kept a higher trend when feature dimensions remain 10–20. The optimal recognition rate is 86.67% when feature dimension is 10 and the number of neighbors is 5. In CAS-PEAL-R1 face database, the recognition rate can reach the maximum value 86.67%, when feature dimension is 40 and the number of neighbors is 2.The experimental data show that the number of training samples of each class is 6 in Yale face database and the recognition effect is optimal when the number of neighbors is 5. In CAS-PEAL-R1 face database, we can obtain optimal recognition rate when the number of training samples of each class is 3 and the number of neighbors is 2. Obviously, the number of neighbors isT-1; T is the number of training samples of each class.In addition, in order to illustrate the effectiveness of SLPP-CML method, the comparative experiments were carried out. The experiment results are shown in Table1.Table 1
Experimental comparison of this method with other methods.
Method
Yale face database
CAS-PEAL-R1 face database
Image restoration [25] + PCA
61.33
55.33
CML [19]
77.33
74.67
CLPM [21]
82.67
80.67
SLPP-CML
86.67
86.67The experimental data illustrated that the recognition results of feature extraction after restoration are not satisfactory. The coupled metric learning in [19] can not overcome the influences of within-class multiple modes, so the identification effect is not good. The coupled metric learning in [21] is conducive to resolving within-class multiple modes; the recognition effects have been greatly improved, but it does not fully consider the between-class relationships of training samples. The proposed SLPP-CML takes advantage of the supervisory of category information, while the within-class and between-class relationship information of training samples have been considered into the metric learning, so we can get better recognition results.


## 5.2.2. Experiment 2: The Low-Resolution Face Recognition Based on SLPP-KCML

The SLPP-KCML algorithm is a nonlinear coupled metric learning algorithm. Through the analysis, there are three factors which affect this algorithm:(1) the number of neighbors k of supervised locality preserving projection; (2) the reserved dimensions Dc of the feature; (3) the kernel function. Based on the experiment result, the number of the nearest neighbors is the same as that of SLPP-CML. For the kernel function, we choose the Gauss function k(x,y)=exp⁡(-∥x-y∥2/α); the value of adjustable factor α affects the function performance. So, in this paper, the experiments were carried out according to the different adjustable factors and the change of feature dimensions on recognition rate; the experimental results are shown in Figure 7.The recognition rate under different dimensions and different adjustable factors.
(a)
The experiment result in Yale face database
(b)
The experiment result in Yale CAS-PEAL-R1 face databaseThe curves indicated that, in Yale face database, the optimal recognition rate is 89.33% when the value ofα is 0.5 and the feature dimension is 20; compared with the SLPP-CML algorithm, the recognition rate increased by 2.66%. In CAS-PEAL-R1 face database, when α=0.7 and the feature dimensions Dc=40, recognition rate is 91.33%; compared with the SLPP-CML algorithm, the recognition rate increased by 4.66%. Obviously, the nonlinear coupled metric learning method can effectively extract the classification information of face image and obtain a high recognition rate.Considering the training time, the SLPP-CML algorithm requires calculating the eigenvalue and eigenvector of the image covariance matrix. The resolution of clear face image is64×64 pixel and low-resolution face image is 16×16 pixel. So the dimensions of image covariance matrix are 4352×4352 and the average training time is about 553.905 seconds.However, the matrix of SLPP-KCML is related to the number of classes and the number of training samples of each class, so the dimension of the covariance matrix of Yale face database is90×90, the size of covariance matrix of CAS-PEAL-R1 face database is 150×150, and the average training time is about 6.687 seconds. Obviously, the training speed of the SLPP-KCML algorithm is faster than SLPP-CML algorithm, and the recognition time of these two algorithms is about 0.0225 seconds. Based on the above analysis, the efficiency of SLPP-KCML algorithm is better than SLPP-CML algorithm.


## 5.3. The Fuzzy Face Recognition

Besides the low-resolution face image, the blurring image usually makes the performance of face recognition system decrease. The fuzzy face image is shown in Figure8. Obviously, it is difficult to identify the fuzzy face image; a part of face details can be restored by deblurring algorithm, but it still cannot provide enough information in identification.(a) (A), (B), and (C) are the normal face, blurred face image, and deblurring results, respectively, in Yale face database. (b) (A), (B), and (C) are the normal face, blurred face image, and deblurring face, respectively, in CAS-PEAL-R1-FRONTAL-Accessory.
(a)
(b)Similar to Section5.2, we carried out the recognition experiments of SLPP-CML algorithm based on different face databases, and the comparative experiments with other methods were made. In the experiment, the clear images with size of 64×64 are the same as those used in Section 5.1, and the fuzzy face images were generated by convolution of corresponding clear face image. The training samples are composed by clear training face images and generated fuzzy face images. The test samples are the fuzzy face images by convolution of the clear test face images and the size is 64×64 pixels. The number of training and test samples has been introduced in Section 5.1. The experiment results are shown in Table 2.Table 2
Experimental comparison of this method with other methods.
Method
Yale face database
CAS-PEAL-R1 face database
Deblurring [26] + PCA
64.00
68.67
CML [19]
77.33
79.33
CLPM [21]
85.67
83.33
SLPP-CML
86.67
88.67
SLPP-KCML
90.00
92.66The experimental data is the best recognition rate of each algorithm; the number of neighbors of SLPP-CML and SLPP-KCML algorithm isT-1, where T is the number of training samples of each class. The optimal value of adjustable factor of Gaussian kernel function in SLPP-KCML algorithm is 0.7. Table 3 gives the feature dimensions of training samples in Yale and CAS-PEAL-R1 face database of SLPP-CML and SLPP-KCML algorithm.Table 3
The feature dimensions in our proposed algorithm.
Method
Yale face database
CAS-PEAL-R1 face database
SLPP-CML
15
39
SLPP-KCML
18
40


## 6. Conclusions

Aiming at the problem that the traditional metric methods can not calculate the distance of the elements in different data sets, we proposed the coupled metric learning method based on supervised locality preserving projection. First, the elements of different sets are mapped to the coupled space combined with the within-class and between-class information, and then the metric matrix learning is performed. Furthermore, we extended this algorithm to nonlinear space, and the kernel coupled metric learning method based on supervised locality preserving projection is proposed. In kernel coupled metric learning approach, two elements of different collections are mapped to the unified high dimensional feature space by kernel function, and then the traditional metric learning is performed in this space. In order to verify the effectiveness of the proposed algorithm, we have done a lot of experiments on two face databases. This algorithm can effectively extract the face nonlinear features, and the operation is simple. Low-resolution and fuzzy face recognition experiments show that the proposed method can obtain a higher recognition rate and has a high computational efficiency.



---

*Source: 724978-2014-02-11.xml*