# Sparsity Preserving Discriminant Projections with Applications to Face Recognition

Authors: Yingchun Ren; Zhicheng Wang; Yufei Chen; Weidong Zhao
Journal: Mathematical Problems in Engineering
 (2016)

---

## Abstract

Dimensionality reduction is extremely important for understanding the intrinsic structure hidden in high-dimensional data. In recent years, sparse representation models have been widely used in dimensionality reduction. In this paper, a novel supervised learning method, called Sparsity Preserving Discriminant Projections (SPDP), is proposed. SPDP, which attempts to preserve the sparse representation structure of the data and maximize the between-class separability simultaneously, can be regarded as a combiner of manifold learning and sparse representation. Specifically, SPDP first creates a concatenated dictionary by classwise PCA decompositions and learns the sparse representation structure of each sample under the constructed dictionary using the least square method. Secondly, a local between-class separability function is defined to characterize the scatter of the samples in the different submanifolds. Then, SPDP integrates the learned sparse representation information with the local between-class relationship to construct a discriminant function. Finally, the proposed method is transformed into a generalized eigenvalue problem. Extensive experimental results on several popular face databases demonstrate the feasibility and effectiveness of the proposed approach.


---

## Body

## 1. Introduction

In many fields such as object recognition [1, 2], text categorization [3], and information retrieval [4], the data are usually provided in high-dimensional form; this makes it difficult to describe, understand, and recognize these data. As an effective method, dimensionality reduction has been widely used in practice to handle these problems [5–8]. Up to now, a variety of dimensionality reduction algorithms have been designed. Based on the data structure they utilize, these methods fall into three categories: global structure-based methods, local neighborhood-based methods, and sparse representation-based methods.Principal Component Analysis (PCA) [9], Linear Discriminant Analysis (LDA) [10], and their kernelized versions are typical global structure-based methods [11, 12]. Owing to its simplicity and effectiveness, PCA, which aims at maximizing the variance of the projected data, has extensive applications in the fields of science and engineering. PCA is a good dimensionality reduction method; however, it does not employ the label information of the samples, leading to inefficiency of the classification. Unlike PCA, LDA is a supervised method that attempts to identify an optimal projection by maximizing the between-class scatter and as such minimizing the within-class scatter. Because the label information is fully exploited, LDA has been proven more efficient than PCA in classification [13]. However, LDA can extract at best K-1 features (K is the number of categories), which is unacceptable in many situations. Moreover, both PCA and LDA are based on the hypothesis that samples from each class lie on a linear subspace [14, 15]; that is, neither of them can identify the local submanifold structure hidden in high-dimensional data.Recently, manifold learning methods, which are especially useful for the analysis of the data that lie on a submanifold of the original space, have been proposed [16–26]. Representative manifold learning methods include Isomap [16], Laplacian Eigenmaps (LE) [17], and Locally Linear Embedding (LLE) [18]. All these nonlinear methods are able to discover the optimal feature subspace by solving an optimization problem based on the weight graph question; however, none of them can overcome the “out-of-sample” problem [19]. That is, they yield maps that are characterized only on the training data points but how to evaluate the maps on new test data points is still unclear. In order to address this problem, Cai et al., respectively, developed the linear visions of the above manifold learning methods such as isometric projection [20], Locality Preserving Projections (LPP) [21], and Neighborhood Preserving Embedding (NPE) [22]. However, these methods suffer from a limitation that they do not encode discriminant information, which is very important for recognition tasks. Recently, Gui et al. proposed a new supervised learning algorithm called Locality Preserving Discriminant Projections (LPDP) to improve the classification performance of LPP and applied it to face recognition [26]. Experimental results show that LPDP is more suitable for recognition tasks than LPP.Sparse representation, as a new branch of the state-of-the-art techniques for signal representation, has attracted considerable research interests [27–38]. It attempts to preserve the sparse representation structure of the samples in a low-dimensional embedding subspace. The representative dimensionality reduction algorithms based on sparse representation include Sparsity Preserving Projections (SPP) [39], Sparsity Preserving Discriminant Analysis (SPDA) [40], Discriminative Learning by Sparse Representation Projections (DLSP) [41], Sparse Tensor Discriminant Analysis (STDA) [42], and sparse nonnegative matrix factorization [43]. It is worthwhile to note that a sparse model also depends on the subspace assumption: each sample can be linearly expressed by other samples from the same class; that is, each sample can be sparsely recovered by samples from all classes. In general, these sparse learning algorithms provide superior recognition accuracy compared with the conditional methods. However, all these dimensionality reduction methods based on sparse coding mentioned above are required to solve the l1 norm minimization problem to construct the sparse weight matrix. Therefore, they are computationally prohibitive for large-scale problems. For example, SPP attempts to preserve the sparse reconstructive relationship of the data [39], which is an effective and powerful technique for dimensionality reduction. However, the computational complexity of SPP is excessively high and hence, it cannot be used extensively for large-scale data processing (in fact, the time cost for constructing the sparse weight graph is O(n4), where n indicates the total number of training samples). Moreover, SPP does not absorb the label information. Thus, the algorithm is unsupervised.Motivated by the above works, a novel supervised learning method, called Sparsity Preserving Discriminant Projection (SPDP), is proposed in this paper. By integrating SPP with local discriminant information for dimensionality reduction, SPDP can be viewed as a combiner of sparse representation and manifold learning. Because sparse representation can implicitly discover the local structure of the data owing to the sparsity prior, this property can be used to describe the local structure. However, differing from the existing SPP, which is time-consuming in sparse reconstruction for each test sample, SPDP first creates a concatenated dictionary using classwise PCA decompositions and learns the sparse representation structure of each sample under the constructed dictionary quickly with the least square method. Then, a local between-class separability function is defined to characterize the scatter of the samples in the different submanifolds. Subsequently, by integrating the sparse representation information with the local between-class relationship, SPDP attempts to preserve the sparse representation structure of the data and maximize the local between-class separability simultaneously. Finally, the proposed method is converted into a generalized eigenvalue problem.It is worth emphasizing some merits of SPDP and the main contributions of this paper:(1)
SPDP is a supervised dimensionality reduction method that attempts to identify a discriminating subspace where the sparse representation structure of the data and the label information are maintained. Meanwhile, the separability of different submanifolds is maximized; that is, different submanifolds can be distinguished more clearly.(2)
SPDP is able to explore the local submanifold structure hidden in high-dimensional data because the manifold learning is employed to characterize the local between-class separability.(3)
The time required for extracting discriminant vectors in SPDP is significantly less than many algorithms based on sparse representation. Therefore, the proposed method can be widely applied for large-scale problems.(4)
Label information is employed twice in SPDP. First, it is absorbed in constructing the dictionary for sparse representation and calculating the sparse coefficient vector, which may contribute to a more discriminating sparse representation structure. Further, it is utilized in computing the local between-class separability, which is more conducive for classification.The rest of this paper is organized as follows: Section2 briefly reviews the existing SPP algorithm. The SPDP algorithm is described in detail in Section 3. The experimental results and analysis are presented in Section 4 and the paper ends with concluding remarks in Section 5.


## 2. Brief Review of Sparsity Preserving Projections (SPP)

SPP aims to preserve the sparse reconstruction relationship of the samples [39]. Given a set of training samples xii=1n, where xi∈Rm and n is the number of training samples, let X=[x1,x2,…,xn]∈Rm×n be the data matrix consisting of all the training samples. SPP first seeks the sparse reconstruction coefficient vector si for each sample xi through the following modified l1 minimization problem:(1)minsisi1,s.t.xi=Xsi,1=1Tsi,where si=[si1,…,si,i-1,0,si,i+1,…,si,n]T is an n-dimensional column vector in which the ith element is equal to zero, implying xi is removed from X, and the element sij, j≠i, denotes the contribution of xj for reconstructing xi. Then, the sparse reconstructive weight matrix S is given as follows:(2)S=s1,s2,…,sn,where si is the optimal solution of (1). The final optimal projection vector w is obtained through the following maximization problem:(3)maxw⁡wTXSβXTwwTXXTw,with Sβ=S+ST-STS. This problem transforms to a generalized eigenvalue problem.It follows that SPP must resolven time-consuming l1 norm minimization problems to obtain the sparse weight matrix S. Thus, the computational complexity of SPP is excessively high and therefore not widely applicable to large-scale data processing. Moreover, SPP does not exploit the prior knowledge of class information, which is valuable for classification and recognition problems such as face recognition.


## 3. Sparsity Preserving Discriminative Learning

In this section, the proposed SPDP algorithm is described in more detail. To reduce the disadvantage that is inevitable for SPP to resolven time-consuming l1 norm minimization problems to obtain the sparse weight matrix S, SPDP first constructs a concatenated dictionary through classwise PCA decompositions and learns the sparse representation structure of each sample under the constructed dictionary quickly using the least square method. To enhance the discriminant performance, it defines a local between-class separability function to characterize the scatter of the samples in the different submanifolds. Then, by integrating the sparse representation information with the local interclass relationship, SPDP aims to maximize the separation between the submanifolds (or intrinsic clusters) without destroying localities and meanwhile preserve the sparse representation structure of the data. Hence, the proposed algorithm is expected to preserve the intrinsic geometry structure and have superior discriminant abilities.


### 3.1. Constructing the Concatenated Dictionary

For convenience, we first provide some notations used in this paper. Assume thatX={x1,x2,…,xn} is a set of training samples, where xi∈Rm. We can categorize the training samples as X=[X1,X2,…,XK], where Xi=[xi1,xi2,…,xini]∈Rm×ni (i=1,2,…,K) consists of samples from class i. Suppose that samples from a single class lie on a linear subspace. Thus, each sample can be sparse linearly represented by samples from all classes. The subspace model is a powerful tool to capture the underlying information in real data sets [44]. For the convenience of PCA decomposition and relevant calculations, we first center the samples from each class at the origin, X~i=[xi1-μi,xi2-μi,…,xini-μi] (i=1,2,…,K), where μi denotes the mean of class i; that is, μi=∑i=1ni‍xi/ni. Therefore, the training sample can be recast as X~=[X~1,X~2,…,X~K]. Afterwards, PCA decomposition is conducted for every X~i (i=1,2,…,K), whose objective function is(4)maxd=1⁡dT∑id,where ∑i‍ is the sample covariance matrix of X~i. For every class i, the first li principal components are selected to construct Di=[d1,d2,…,dli] (in fact, li is automatically selected by the value of the PCA ratio from the system). Thus, a sample x from class i can be simply represented as(5)x=Dis~i=D1,D2,…,Di-1,Di,Di+1,…,DKs=Ds,with D=[D1,D2,…,DK] and s=[0T,0T,…,0T,s~iT,0T,…,0T]T. Di is the dictionary of class i by the PCA decomposition above, D is the concatenated dictionary composed of all Di (i=1,2,…,K), s is the sparse representation of a sample x under the concatenated dictionary D, and s~i is the coefficient vector under the dictionary Di. In fact, s~i can be quickly computed from the least square method as(6)s~i=DiTDi-1DiTx=DiTx.The orthogonality of each principal component of PCA decomposition of the same class is utilized in the reduction of the above formula. The process of constructing the concatenated dictionary is presented in Figure1.Figure 1
The process of constructing the concatenated dictionary.According to the preceding procedure, each training sample corresponds to a sparse representation under the concatenated dictionaryD and the sparse coefficient vector s of any training sample from class i can be quickly computed from the least square method (in fact, it is the primary reason that the proposed approach is significantly faster than SPP, which will be explained in detail in Section 4.4) because the computational process of s involves only Di, which is column orthogonal in view of (5) and (6).


### 3.2. Preserving Sparse Representation Structure

As can be seen in Section3.1, to some extent, the dictionary D describes the intrinsic geometric properties of the data and the sparse coefficient vectors explicitly encode the discriminant information of the training samples. Thus, it is hoped that this valued property in the original high-dimensional space can be preserved in the low-dimensional embedding subspace. Therefore, the objective function is expected to look for an optimal projection that can best preserve the sparse representation structure:(7)Jsw=minw⁡∑i=1nwTxi-wTDsi22,where si is the sparse reconstruction vector corresponding to xi.Using algebraic operations, (7) can be arranged as(8)∑i=1nwTxi-wTDsi22=wT∑i=1nxi-Dsixi-DsiTw=wT∑i=1nxixiT-xisiTDT-DsixiT+DsiDsiTw=wTXXT-XSTDT-DSXT+DSSTDTw,where S=[s1,s2,…,sn], and therefore, (7) can be simply recast as(9)Jsw=minw⁡wTXXT-XSTDT-DSXT+DSSTDTw.


### 3.3. Characterization of the Local Interclass Separability

To effectively discover the discriminant structure embedded in high-dimensional data and improve the classification performance, in this subsection, we construct a local interclass weight graph. Because data in the same class lie on one or more submanifolds and data belonging to different classes are distributed on different submanifolds, it is important for classification problems to distinguish one submanifold from another. Therefore, a local between-class separability function is defined in this section to characterize the separability of the samples in different submanifolds. The aim of SPDP is that different submanifolds can be distinguished more clearly after being projected; hence, the local between-class separability of different submanifolds should be maximized. Thus, we can construct a label matrixB to describe the local and interclass relationships of each point as follows:(10)Bij=1+exp⁡-xi-xj22σ,ifi∈NK-jorj∈NK-i;0,otherwise,where xi-xj22 denotes the geodesic distance between points xi and xj, σ is a parameter which is often set to be as the standard deviation of the samples, NK-(i) denotes the index in the K nearest neighbors of the sample xi, however with a different class label, and B is called the local between-class weight matrix (or local interclass weight graph). As can be seen in the above definition, if two distant points xi and xj belong to different submanifolds, the scatter of them is big and vice versa. That is, the points belonging to different submanifolds should be located farther after projection. Therefore, the local interclass separability can be characterized as the following equation:(11)Jbw=12∑i∑jyi-yj22Bij,where yi=wTxi (i=1,2,…,n) is the low-dimensional representation of the original data, which can be obtained by projecting each xi onto the direction vector w∈Rm. With algebraic simplifications, (11) can be rewritten as(12)Jb=12∑i,jnBijyi-yj22=12wT∑i,jnBijxi-xjTxi-xjw=wT12∑i=1n∑j=1nBijxi-xjTxi-xjw=wT12∑i=1n∑j=1nBijxixiT-2∑i=1n∑j=1nBijxixjT+∑i=1n∑j=1nBijxjxjTw=wT∑i=1nD′iixixiT-∑i=1n∑j=1nBijxixjTw=wTXD′-BXTw=wTXLXTw,where L is Laplacian matrix with definition L=D′-B and D′ is a diagonal matrix [45]; that is, D′ii=∑j‍Bij. Equation (12) characterizes the separability (or scatter) of the data set in different submanifolds. Therefore, each manifold can be separated clearly, as long as the optimal projection w∗ is adopted.


### 3.4. Sparsity Preserving Discriminant Projections

To achieve improved recognition results, we explicitly integrate the sparsity preserving constraint as indicated in (7) with the local between-class separability as illustrated in (12). The novel supervised algorithm SPDP, which not only preserves the sparse representation structure but also separates each submanifold as distant as possible, is defined as(13)maxw⁡Jw=JbwJsw=wTXLXTwwTXXT-XSTDT-DSXT+DSSTDTw,where the denominator term Js(w) measures the quality of preserving the sparse representation structure and the numerator term Jb(w) measures the separability of different submanifolds. It is well known that the criterion of LDA is to maximize the between-class scatter and, meanwhile, minimize the within-class scatter. Similar to LDA, the aim of SPDP is to maximize the ratio of the local between-class separability to the sparse representation weight scatter.Letting(14)M=XXT-XSTDT-DSXT+DSSTDT,the objective function can be recast as the following optimization problem:(15)maxw⁡wTXLXTwwTMw.Then, the optimal w’s are the eigenvectors corresponding to the largest d eigenvalues of the following generalized eigenvalue problem:(16)XLXTw=λMw.It is worth noting that since the training sample size is much smaller than the feature dimensions for those high-dimensional data,M might be singular. This problem can be tackled by projecting the training set X onto a PCA subspace spanned by the leading eigenvectors to get X′ and replacing X by X′.Based on the above discussion, the proposed SPDP is summarized in Algorithm1.Algorithm 1 (Sparsity Preserving Discriminant Projections (SPDP)).
We have the following steps.
Step  1. Execute PCA decomposition for each Xi (i=1,2,…,K) using (4) to obtain the concatenated dictionary D.
Step  2. Calculate the coefficient vector s~i under the dictionary Di for each sample based on (6) to obtain the sparse coefficient vector s and then calculate S.
Step  3. Calculate B and L by (10) and (12), respectively.
Step  4. Calculate the projecting vectors by the generalized eigenvalue problem in (16).


## 3.1. Constructing the Concatenated Dictionary

For convenience, we first provide some notations used in this paper. Assume thatX={x1,x2,…,xn} is a set of training samples, where xi∈Rm. We can categorize the training samples as X=[X1,X2,…,XK], where Xi=[xi1,xi2,…,xini]∈Rm×ni (i=1,2,…,K) consists of samples from class i. Suppose that samples from a single class lie on a linear subspace. Thus, each sample can be sparse linearly represented by samples from all classes. The subspace model is a powerful tool to capture the underlying information in real data sets [44]. For the convenience of PCA decomposition and relevant calculations, we first center the samples from each class at the origin, X~i=[xi1-μi,xi2-μi,…,xini-μi] (i=1,2,…,K), where μi denotes the mean of class i; that is, μi=∑i=1ni‍xi/ni. Therefore, the training sample can be recast as X~=[X~1,X~2,…,X~K]. Afterwards, PCA decomposition is conducted for every X~i (i=1,2,…,K), whose objective function is(4)maxd=1⁡dT∑id,where ∑i‍ is the sample covariance matrix of X~i. For every class i, the first li principal components are selected to construct Di=[d1,d2,…,dli] (in fact, li is automatically selected by the value of the PCA ratio from the system). Thus, a sample x from class i can be simply represented as(5)x=Dis~i=D1,D2,…,Di-1,Di,Di+1,…,DKs=Ds,with D=[D1,D2,…,DK] and s=[0T,0T,…,0T,s~iT,0T,…,0T]T. Di is the dictionary of class i by the PCA decomposition above, D is the concatenated dictionary composed of all Di (i=1,2,…,K), s is the sparse representation of a sample x under the concatenated dictionary D, and s~i is the coefficient vector under the dictionary Di. In fact, s~i can be quickly computed from the least square method as(6)s~i=DiTDi-1DiTx=DiTx.The orthogonality of each principal component of PCA decomposition of the same class is utilized in the reduction of the above formula. The process of constructing the concatenated dictionary is presented in Figure1.Figure 1
The process of constructing the concatenated dictionary.According to the preceding procedure, each training sample corresponds to a sparse representation under the concatenated dictionaryD and the sparse coefficient vector s of any training sample from class i can be quickly computed from the least square method (in fact, it is the primary reason that the proposed approach is significantly faster than SPP, which will be explained in detail in Section 4.4) because the computational process of s involves only Di, which is column orthogonal in view of (5) and (6).


## 3.2. Preserving Sparse Representation Structure

As can be seen in Section3.1, to some extent, the dictionary D describes the intrinsic geometric properties of the data and the sparse coefficient vectors explicitly encode the discriminant information of the training samples. Thus, it is hoped that this valued property in the original high-dimensional space can be preserved in the low-dimensional embedding subspace. Therefore, the objective function is expected to look for an optimal projection that can best preserve the sparse representation structure:(7)Jsw=minw⁡∑i=1nwTxi-wTDsi22,where si is the sparse reconstruction vector corresponding to xi.Using algebraic operations, (7) can be arranged as(8)∑i=1nwTxi-wTDsi22=wT∑i=1nxi-Dsixi-DsiTw=wT∑i=1nxixiT-xisiTDT-DsixiT+DsiDsiTw=wTXXT-XSTDT-DSXT+DSSTDTw,where S=[s1,s2,…,sn], and therefore, (7) can be simply recast as(9)Jsw=minw⁡wTXXT-XSTDT-DSXT+DSSTDTw.


## 3.3. Characterization of the Local Interclass Separability

To effectively discover the discriminant structure embedded in high-dimensional data and improve the classification performance, in this subsection, we construct a local interclass weight graph. Because data in the same class lie on one or more submanifolds and data belonging to different classes are distributed on different submanifolds, it is important for classification problems to distinguish one submanifold from another. Therefore, a local between-class separability function is defined in this section to characterize the separability of the samples in different submanifolds. The aim of SPDP is that different submanifolds can be distinguished more clearly after being projected; hence, the local between-class separability of different submanifolds should be maximized. Thus, we can construct a label matrixB to describe the local and interclass relationships of each point as follows:(10)Bij=1+exp⁡-xi-xj22σ,ifi∈NK-jorj∈NK-i;0,otherwise,where xi-xj22 denotes the geodesic distance between points xi and xj, σ is a parameter which is often set to be as the standard deviation of the samples, NK-(i) denotes the index in the K nearest neighbors of the sample xi, however with a different class label, and B is called the local between-class weight matrix (or local interclass weight graph). As can be seen in the above definition, if two distant points xi and xj belong to different submanifolds, the scatter of them is big and vice versa. That is, the points belonging to different submanifolds should be located farther after projection. Therefore, the local interclass separability can be characterized as the following equation:(11)Jbw=12∑i∑jyi-yj22Bij,where yi=wTxi (i=1,2,…,n) is the low-dimensional representation of the original data, which can be obtained by projecting each xi onto the direction vector w∈Rm. With algebraic simplifications, (11) can be rewritten as(12)Jb=12∑i,jnBijyi-yj22=12wT∑i,jnBijxi-xjTxi-xjw=wT12∑i=1n∑j=1nBijxi-xjTxi-xjw=wT12∑i=1n∑j=1nBijxixiT-2∑i=1n∑j=1nBijxixjT+∑i=1n∑j=1nBijxjxjTw=wT∑i=1nD′iixixiT-∑i=1n∑j=1nBijxixjTw=wTXD′-BXTw=wTXLXTw,where L is Laplacian matrix with definition L=D′-B and D′ is a diagonal matrix [45]; that is, D′ii=∑j‍Bij. Equation (12) characterizes the separability (or scatter) of the data set in different submanifolds. Therefore, each manifold can be separated clearly, as long as the optimal projection w∗ is adopted.


## 3.4. Sparsity Preserving Discriminant Projections

To achieve improved recognition results, we explicitly integrate the sparsity preserving constraint as indicated in (7) with the local between-class separability as illustrated in (12). The novel supervised algorithm SPDP, which not only preserves the sparse representation structure but also separates each submanifold as distant as possible, is defined as(13)maxw⁡Jw=JbwJsw=wTXLXTwwTXXT-XSTDT-DSXT+DSSTDTw,where the denominator term Js(w) measures the quality of preserving the sparse representation structure and the numerator term Jb(w) measures the separability of different submanifolds. It is well known that the criterion of LDA is to maximize the between-class scatter and, meanwhile, minimize the within-class scatter. Similar to LDA, the aim of SPDP is to maximize the ratio of the local between-class separability to the sparse representation weight scatter.Letting(14)M=XXT-XSTDT-DSXT+DSSTDT,the objective function can be recast as the following optimization problem:(15)maxw⁡wTXLXTwwTMw.Then, the optimal w’s are the eigenvectors corresponding to the largest d eigenvalues of the following generalized eigenvalue problem:(16)XLXTw=λMw.It is worth noting that since the training sample size is much smaller than the feature dimensions for those high-dimensional data,M might be singular. This problem can be tackled by projecting the training set X onto a PCA subspace spanned by the leading eigenvectors to get X′ and replacing X by X′.Based on the above discussion, the proposed SPDP is summarized in Algorithm1.Algorithm 1 (Sparsity Preserving Discriminant Projections (SPDP)).
We have the following steps.
Step  1. Execute PCA decomposition for each Xi (i=1,2,…,K) using (4) to obtain the concatenated dictionary D.
Step  2. Calculate the coefficient vector s~i under the dictionary Di for each sample based on (6) to obtain the sparse coefficient vector s and then calculate S.
Step  3. Calculate B and L by (10) and (12), respectively.
Step  4. Calculate the projecting vectors by the generalized eigenvalue problem in (16).


## 4. Experiments

In this section, the proposed SPDP algorithm is tested on three publicly available face databases (Yale [13], ORL [46], and CMU PIE [47]) and compared with six popular dimensionality reduction methods—PCA, LDA, LPP, NPE, LPDP, and SPP. For PCA, the only model parameter is the subspace dimension and for LDA, the performance is directly influenced by the energy of the eigenvalues kept in the PCA preprocessing phase. For LPP and NPE, the supervised versions are adopted. In particular, the neighbor mode in LPP and NPE is set to be “supervised”; the weight mode in LPP is set to be “Cosine.” The empirically determined parameter α in LPDP is taken to be 1 [26], ε in SPP is set to be 0.05 as indicated in [39], and σ in SPDP is set to be the standard deviation of the samples. The nearest neighbor classifier (1-NN) is employed to predict the classes of the test data. All experiments are accomplished with MATLAB R2013a on a personal computer with Intel(R) Core i7-4770 K 3.50 GHz CPU, 16.0 GB main memory, and the Windows 7 operating system.


### 4.1. Experiment on Yale Face Database

The Yale face database contains 165 face images of 15 individuals. There are 11 images per individual. These images were collected under different facial expressions (normal, happy, sad, surprised, sleepy, and wink) and configurations (left-light, center-light, and right-light) and with or without glasses. All the images are cropped to a size of32×32 and then normalized to have a unit norm. Some samples from this database are presented in Figure 2. For each person, k (k varies from 2 to 8) images are randomly selected as the training samples and the remaining 11-k for the test. For each k, the results are averaged over 50 random splits. Table 1 presents the best recognition rate and the associated standard deviation of the seven algorithms under the different sizes of the training set. Figure 3(a) presents the best recognition rate versus the variation of the size of the training set. Figure 3(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as six. The fact that the upper bound for the dimensionality of LDA is K-1 (K is the number of categories) because there are at most K-1 generalized nonzero eigenvalues [13] deserves to be noted; similar situations will occur in other experiments in this paper. Hence, one can see that the SPDP algorithm significantly outperforms the other methods.Table 1
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on Yale (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPDP
0.7137 (±0.031)
0.8333 (±0.036)
0.9125 (±0.027)
0.9458 (±0.031)
0.9619 (±0.025)
0.9786 (±0.029)
0.9916 (±0.033)
LPDP
0.5674 (±0.057)
0.7175 (±0.045)
0.7807 (±0.037)
0.8186 (±0.039)
0.8679 (±0.035)
0.8816 (±0.029)
0.9066 (±0.036)
PCA
0.4389 (±0.027)
0.4895 (±0.035)
0.5514 (±0.037)
0.5838 (±0.048)
0.6241 (±0.038)
0.6561 (±0.043)
0.6727 (±0.046)
LDA
0.5354 (±0.061)
0.6486 (±0.052)
0.7222 (±0.036)
0.7792 (±0.047)
0.8132 (±0.037)
0.8375 (±0.040)
0.8613 (±0.044)
LPP
0.5783 (±0.041)
0.6814 (±0.044)
0.7469 (±0.036)
0.8025 (±0.035)
0.8139 (±0.027)
0.8244 (±0.014)
0.8392 (±0.018)
NPE
0.5635 (±0.025)
0.6811 (±0.019)
0.7455 (±0.027)
0.7593 (±0.023)
0.8112 (±0.017)
0.8284 (±0.025)
0.8463 (±0.023)
SPP
0.5202 (±0.038)
0.6425 (±0.027)
0.7098 (±0.033)
0.7471 (±0.033)
0.7653 (±0.026)
0.7827 (±0.032)
0.8037 (±0.035)Figure 2
Some face samples from the Yale database.Figure 3
Recognition rates of the seven algorithms on the Yale database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as six.
(a)
(b)


### 4.2. Experiment on ORL Face Database

There are 400 images of 40 people in the ORL face data set, where each one has 10 different pictures. The images were collected at different time points, under different lighting conditions, varying facial expressions. In our experiment, each image is cropped to a resolution of32×32 as shown in Figure 4. We randomly select k (k varies from 2 to 8) pictures from each person for training; the remainder are used for testing. We repeat these splits 50 times and report the average results. Table 2 displays the best classification accuracy of the seven algorithms under the different sizes of the training set; the number in parentheses is the corresponding standard deviation. Figure 5(a) presents the best recognition rate versus the variation of the size of the training set. Figure 5(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as five. It can be seen that SPDP and LPDP are superior to other compared methods (their performances on the ORL database are quite similar), especially when the size of the training set is small. The reason may be that both SPDP and LPDP consider the discriminant information and local structure of the data.Table 2
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on ORL (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPDP
0.8283 (±0.023)
0.8939 (±0.019)
0.9266 (±0.022)
0.9523 (±0.015)
0.9686 (±0.021)
0.9802 (±0.015)
0.9839 (±0.017)
LPDP
0.8109 (±0.032)
0.9006 (±0.017)
0.9305 (±0.015)
0.9579 (±0.010)
0.9772 (±0.009)
0.9863 (±0.013)
0.9906 (±0.015)
PCA
0.6709 (±0.026)
0.7576 (±0.035)
0.8204 (±0.036)
0.8626 (±0.023)
0.8866 (±0.027)
0.9045 (±0.033)
0.9116 (±0.026)
LDA
0.7241 (±0.021)
0.8276 (±0.022)
0.8958 (±0.031)
0.9231 (±0.027)
0.9360 (±0.033)
0.9465 (±0.041)
0.9563 (±0.044)
LPP
0.7833 (±0.023)
0.8657 (±0.019)
0.9060 (±0.016)
0.9289 (±0.022)
0.9432 (±0.027)
0.9527 (±0.025)
0.9546 (±0.026)
NPE
0.7869 (±0.015)
0.8689 (±0.017)
0.9047 (±0.022)
0.9331 (±0.023)
0.9469 (±0.021)
0.9565 (±0.028)
0.9584 (±0.023)
SPP
0.7324 (±0.028)
0.8055 (±0.022)
0.8442 (±0.025)
0.8704 (±0.031)
0.8935 (±0.026)
0.9162 (±0.035)
0.9397 (±0.034)Figure 4
Some face samples from the ORL database.Figure 5
Recognition rates of the seven algorithms on the ORL database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as five.
(a)
(b)


### 4.3. Experiment on CMU PIE Face Database

In this subsection, it is verified that the proposed algorithm achieves higher classification accuracy than the other dimensionality reduction methods under varying illumination, pose, and expression. The CMU PIE face database contains over 41,368 face images of 68 subjects that were captured by 13 synchronized cameras and 21 flashes under varying poses, illumination, and expression. In our experiments, we choose the five frontal poses (C05, C07, C09, C27, and C29). This leaves 170 face images per subject; all the images are cropped to32×32. Figure 6 shows some pictures of one subject. A random subset with k(=5,10,15,20) pictures per subject is selected with labels to form the training set; the remainder are used for testing. For each given k, we average the classification accuracies over 50 random splits. Table 3 presents the best recognition rate and the associated standard deviation in brackets of the seven algorithms under the different size of the training set. Figure 7(a) presents the best recognition rate versus the variation of the size of the training set. Figure 7(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as ten. We can observe that the proposed SPDP outperforms the other dimensionality reduction methods such as PCA, LDA, LPP, NPE, LPDP, and SPP about pose, illumination, and expression variations.Table 3
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on CMU PIE (k is the training sample size).
Methods
k
=
5
k
=
10
k
=
15
k
=
20
SPDP
0.7882 (±0.012)
0.9015 (±0.009)
0.9356 (±0.017)
0.9517 (±0.022)
LPDP
0.7653 (±0.012)
0.8759 (±0.015)
0.9127 (±0.009)
0.9405 (±0.011)
PCA
0.2817 (±0.024)
0.4260 (±0.032)
0.5345 (±0.021)
0.6028 (±0.028)
LDA
0.7250 (±0.011)
0.8625 (±0.016)
0.9175 (±0.008)
0.9337 (±0.015)
LPP
0.7253 (±0.033)
0.8659 (±0.028)
0.9005 (±0.039)
0.9342 (±0.023)
NPE
0.7148 (±0.035)
0.8601 (±0.029)
0.8905 (±0.021)
0.9231 (±0.028)
SPP
0.6391 (±0.026)
0.7720 (±0.031)
0.8285 (±0.018)
0.8607 (±0.034)Figure 6
Some face samples from the CMU PIE database.Figure 7
Recognition rates of the seven algorithms on the CMU PIE database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as ten.
(a)
(b)


### 4.4. Comparison of Time Cost for Acquiring the Discriminant Vectors of SPP with SPDP

In this subsection, the time cost for acquiring the discriminant vectors of SPDP is compared with that of SPP. Tables4, 5, and 6 list the average time costs for acquiring the discriminant vectors of SPP and SPDP versus the different sizes of the training set on the three face data sets. It is demonstrated that SPDP is significantly faster than SPP in acquiring the embedding functions in our experiments, especially in the large-scale problems such as CMU PIE.Table 4
Time (s) for acquiring the discriminant vectors of SPP and SPDP on Yale (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPP
0.3729
0.6387
1.0335
1.5506
2.1609
2.9087
4.0471
SPDP
0.2475
0.4016
0.4835
0.5352
0.6827
0.7036
0.8263Table 5
Time (s) for acquiring the discriminant vectors of SPP and SPDP on ORL (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPP
1.1933
2.5641
5.1679
8.4467
13.0688
19.7787
29.4638
SPDP
0.2672
0.3526
0.5128
0.5933
0.7032
0.8158
1.2875Table 6
Time (s) for acquiring the discriminant vectors of SPP and SPDP on CMU PIE (k is the training sample size).
Methods
k
=
5
k
=
10
k
=
15
k
=
20
SPP
40.9725
67.6737
104.9756
178.6327
SPDP
0.2736
0.8135
1.9037
2.7218The critical factor of the above phenomenon is that the approaches of SPP and SPDP to obtain the sparse representation structure are entirely different. In SPP,n time-consuming l1 norm minimization problems are required to be solved to construct the sparse weight matrix, whose computational cost is O(n4) [48, 49], whereas SPDP can achieve this significantly faster through only K PCA decompositions and n least square methods. Because K PCA decompositions can be completed in O(m2∑i=1K‍li) according to the more efficient algorithm [50], the time cost for learning the sparse coefficient vector of each sample, which only involves the least square method, is O(mli) and the sparse weight matrix S can be calculated with O(m∑i=1K‍nili); the computational complexity of SPDP to learn the sparse representation structure is O(m2∑i=1K‍li+m∑i=1K‍nili). In general, ni≪n, li≪n, and K≪n; hence, SPDP performs considerably faster than SPP as indicated in Tables 4, 5, and 6.


### 4.5. Overall Observations and Discussions

Several observations and analysis can be achieved from the above experimental results.(1)
From Tables1, 2, and 3 and Figures 3(a), 5(a), and 7(a), we can draw a conclusion that the proposed algorithm consistently outperforms the other compared methods, especially when the number of the training data is particularly small. The reason is that SPDP simultaneously considers both the sparse representation structure and the separability of the different submanifolds. Further, this indicates that SPDP can capture more inherent information that is hidden in the data compared to the other compared methods.(2)
From Figures3(b), 5(b), and 7(b), it can be observed that the reduction dimensions for SPDP to achieve the best recognition rate are less than those of the other compared algorithms. This saves a considerable amount of time and storage space after obtaining the optimal embedding functions.(3)
From Tables4, 5, and 6, it is indicated that SPDP is considerably faster than SPP in obtaining the discriminant vectors. This is because the method SPDP uses to learn the sparse representation structure which is more effective than that of SPP as analyzed in Section 4.4.


## 4.1. Experiment on Yale Face Database

The Yale face database contains 165 face images of 15 individuals. There are 11 images per individual. These images were collected under different facial expressions (normal, happy, sad, surprised, sleepy, and wink) and configurations (left-light, center-light, and right-light) and with or without glasses. All the images are cropped to a size of32×32 and then normalized to have a unit norm. Some samples from this database are presented in Figure 2. For each person, k (k varies from 2 to 8) images are randomly selected as the training samples and the remaining 11-k for the test. For each k, the results are averaged over 50 random splits. Table 1 presents the best recognition rate and the associated standard deviation of the seven algorithms under the different sizes of the training set. Figure 3(a) presents the best recognition rate versus the variation of the size of the training set. Figure 3(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as six. The fact that the upper bound for the dimensionality of LDA is K-1 (K is the number of categories) because there are at most K-1 generalized nonzero eigenvalues [13] deserves to be noted; similar situations will occur in other experiments in this paper. Hence, one can see that the SPDP algorithm significantly outperforms the other methods.Table 1
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on Yale (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPDP
0.7137 (±0.031)
0.8333 (±0.036)
0.9125 (±0.027)
0.9458 (±0.031)
0.9619 (±0.025)
0.9786 (±0.029)
0.9916 (±0.033)
LPDP
0.5674 (±0.057)
0.7175 (±0.045)
0.7807 (±0.037)
0.8186 (±0.039)
0.8679 (±0.035)
0.8816 (±0.029)
0.9066 (±0.036)
PCA
0.4389 (±0.027)
0.4895 (±0.035)
0.5514 (±0.037)
0.5838 (±0.048)
0.6241 (±0.038)
0.6561 (±0.043)
0.6727 (±0.046)
LDA
0.5354 (±0.061)
0.6486 (±0.052)
0.7222 (±0.036)
0.7792 (±0.047)
0.8132 (±0.037)
0.8375 (±0.040)
0.8613 (±0.044)
LPP
0.5783 (±0.041)
0.6814 (±0.044)
0.7469 (±0.036)
0.8025 (±0.035)
0.8139 (±0.027)
0.8244 (±0.014)
0.8392 (±0.018)
NPE
0.5635 (±0.025)
0.6811 (±0.019)
0.7455 (±0.027)
0.7593 (±0.023)
0.8112 (±0.017)
0.8284 (±0.025)
0.8463 (±0.023)
SPP
0.5202 (±0.038)
0.6425 (±0.027)
0.7098 (±0.033)
0.7471 (±0.033)
0.7653 (±0.026)
0.7827 (±0.032)
0.8037 (±0.035)Figure 2
Some face samples from the Yale database.Figure 3
Recognition rates of the seven algorithms on the Yale database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as six.
(a)
(b)


## 4.2. Experiment on ORL Face Database

There are 400 images of 40 people in the ORL face data set, where each one has 10 different pictures. The images were collected at different time points, under different lighting conditions, varying facial expressions. In our experiment, each image is cropped to a resolution of32×32 as shown in Figure 4. We randomly select k (k varies from 2 to 8) pictures from each person for training; the remainder are used for testing. We repeat these splits 50 times and report the average results. Table 2 displays the best classification accuracy of the seven algorithms under the different sizes of the training set; the number in parentheses is the corresponding standard deviation. Figure 5(a) presents the best recognition rate versus the variation of the size of the training set. Figure 5(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as five. It can be seen that SPDP and LPDP are superior to other compared methods (their performances on the ORL database are quite similar), especially when the size of the training set is small. The reason may be that both SPDP and LPDP consider the discriminant information and local structure of the data.Table 2
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on ORL (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPDP
0.8283 (±0.023)
0.8939 (±0.019)
0.9266 (±0.022)
0.9523 (±0.015)
0.9686 (±0.021)
0.9802 (±0.015)
0.9839 (±0.017)
LPDP
0.8109 (±0.032)
0.9006 (±0.017)
0.9305 (±0.015)
0.9579 (±0.010)
0.9772 (±0.009)
0.9863 (±0.013)
0.9906 (±0.015)
PCA
0.6709 (±0.026)
0.7576 (±0.035)
0.8204 (±0.036)
0.8626 (±0.023)
0.8866 (±0.027)
0.9045 (±0.033)
0.9116 (±0.026)
LDA
0.7241 (±0.021)
0.8276 (±0.022)
0.8958 (±0.031)
0.9231 (±0.027)
0.9360 (±0.033)
0.9465 (±0.041)
0.9563 (±0.044)
LPP
0.7833 (±0.023)
0.8657 (±0.019)
0.9060 (±0.016)
0.9289 (±0.022)
0.9432 (±0.027)
0.9527 (±0.025)
0.9546 (±0.026)
NPE
0.7869 (±0.015)
0.8689 (±0.017)
0.9047 (±0.022)
0.9331 (±0.023)
0.9469 (±0.021)
0.9565 (±0.028)
0.9584 (±0.023)
SPP
0.7324 (±0.028)
0.8055 (±0.022)
0.8442 (±0.025)
0.8704 (±0.031)
0.8935 (±0.026)
0.9162 (±0.035)
0.9397 (±0.034)Figure 4
Some face samples from the ORL database.Figure 5
Recognition rates of the seven algorithms on the ORL database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as five.
(a)
(b)


## 4.3. Experiment on CMU PIE Face Database

In this subsection, it is verified that the proposed algorithm achieves higher classification accuracy than the other dimensionality reduction methods under varying illumination, pose, and expression. The CMU PIE face database contains over 41,368 face images of 68 subjects that were captured by 13 synchronized cameras and 21 flashes under varying poses, illumination, and expression. In our experiments, we choose the five frontal poses (C05, C07, C09, C27, and C29). This leaves 170 face images per subject; all the images are cropped to32×32. Figure 6 shows some pictures of one subject. A random subset with k(=5,10,15,20) pictures per subject is selected with labels to form the training set; the remainder are used for testing. For each given k, we average the classification accuracies over 50 random splits. Table 3 presents the best recognition rate and the associated standard deviation in brackets of the seven algorithms under the different size of the training set. Figure 7(a) presents the best recognition rate versus the variation of the size of the training set. Figure 7(b) is the variation rules of the recognition rates of the seven algorithms under different reduced dimensions when the size of the training samples from each class is fixed as ten. We can observe that the proposed SPDP outperforms the other dimensionality reduction methods such as PCA, LDA, LPP, NPE, LPDP, and SPP about pose, illumination, and expression variations.Table 3
The best recognition rate and the corresponding standard deviation of the seven algorithms under the different size of the training set on CMU PIE (k is the training sample size).
Methods
k
=
5
k
=
10
k
=
15
k
=
20
SPDP
0.7882 (±0.012)
0.9015 (±0.009)
0.9356 (±0.017)
0.9517 (±0.022)
LPDP
0.7653 (±0.012)
0.8759 (±0.015)
0.9127 (±0.009)
0.9405 (±0.011)
PCA
0.2817 (±0.024)
0.4260 (±0.032)
0.5345 (±0.021)
0.6028 (±0.028)
LDA
0.7250 (±0.011)
0.8625 (±0.016)
0.9175 (±0.008)
0.9337 (±0.015)
LPP
0.7253 (±0.033)
0.8659 (±0.028)
0.9005 (±0.039)
0.9342 (±0.023)
NPE
0.7148 (±0.035)
0.8601 (±0.029)
0.8905 (±0.021)
0.9231 (±0.028)
SPP
0.6391 (±0.026)
0.7720 (±0.031)
0.8285 (±0.018)
0.8607 (±0.034)Figure 6
Some face samples from the CMU PIE database.Figure 7
Recognition rates of the seven algorithms on the CMU PIE database: (a) the best recognition rates versus the different size of the training set and (b) the average recognition rates versus the variation of dimensions when the size per class is fixed as ten.
(a)
(b)


## 4.4. Comparison of Time Cost for Acquiring the Discriminant Vectors of SPP with SPDP

In this subsection, the time cost for acquiring the discriminant vectors of SPDP is compared with that of SPP. Tables4, 5, and 6 list the average time costs for acquiring the discriminant vectors of SPP and SPDP versus the different sizes of the training set on the three face data sets. It is demonstrated that SPDP is significantly faster than SPP in acquiring the embedding functions in our experiments, especially in the large-scale problems such as CMU PIE.Table 4
Time (s) for acquiring the discriminant vectors of SPP and SPDP on Yale (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPP
0.3729
0.6387
1.0335
1.5506
2.1609
2.9087
4.0471
SPDP
0.2475
0.4016
0.4835
0.5352
0.6827
0.7036
0.8263Table 5
Time (s) for acquiring the discriminant vectors of SPP and SPDP on ORL (k is the training sample size).
Methods
k
=
2
k
=
3
k
=
4
k
=
5
k
=
6
k
=
7
k
=
8
SPP
1.1933
2.5641
5.1679
8.4467
13.0688
19.7787
29.4638
SPDP
0.2672
0.3526
0.5128
0.5933
0.7032
0.8158
1.2875Table 6
Time (s) for acquiring the discriminant vectors of SPP and SPDP on CMU PIE (k is the training sample size).
Methods
k
=
5
k
=
10
k
=
15
k
=
20
SPP
40.9725
67.6737
104.9756
178.6327
SPDP
0.2736
0.8135
1.9037
2.7218The critical factor of the above phenomenon is that the approaches of SPP and SPDP to obtain the sparse representation structure are entirely different. In SPP,n time-consuming l1 norm minimization problems are required to be solved to construct the sparse weight matrix, whose computational cost is O(n4) [48, 49], whereas SPDP can achieve this significantly faster through only K PCA decompositions and n least square methods. Because K PCA decompositions can be completed in O(m2∑i=1K‍li) according to the more efficient algorithm [50], the time cost for learning the sparse coefficient vector of each sample, which only involves the least square method, is O(mli) and the sparse weight matrix S can be calculated with O(m∑i=1K‍nili); the computational complexity of SPDP to learn the sparse representation structure is O(m2∑i=1K‍li+m∑i=1K‍nili). In general, ni≪n, li≪n, and K≪n; hence, SPDP performs considerably faster than SPP as indicated in Tables 4, 5, and 6.


## 4.5. Overall Observations and Discussions

Several observations and analysis can be achieved from the above experimental results.(1)
From Tables1, 2, and 3 and Figures 3(a), 5(a), and 7(a), we can draw a conclusion that the proposed algorithm consistently outperforms the other compared methods, especially when the number of the training data is particularly small. The reason is that SPDP simultaneously considers both the sparse representation structure and the separability of the different submanifolds. Further, this indicates that SPDP can capture more inherent information that is hidden in the data compared to the other compared methods.(2)
From Figures3(b), 5(b), and 7(b), it can be observed that the reduction dimensions for SPDP to achieve the best recognition rate are less than those of the other compared algorithms. This saves a considerable amount of time and storage space after obtaining the optimal embedding functions.(3)
From Tables4, 5, and 6, it is indicated that SPDP is considerably faster than SPP in obtaining the discriminant vectors. This is because the method SPDP uses to learn the sparse representation structure which is more effective than that of SPP as analyzed in Section 4.4.


## 5. Conclusions

This paper proposed a new supervised learning method, called Sparsity Preserving Discriminative Projections (SPDP), by combining manifold learning and sparse representation. Specifically, SPDP first constructs a concatenated dictionary by means of classwise PCA decompositions and learns the sparse representation structure of each sample under the constructed dictionary quickly using the least square method. Then, it defines a local between-class separability function to characterize the separability of the samples in different submanifolds. Subsequently, SPDP integrates the sparse representation information with the local between-class relationship. Thus, SPDP preserves the sparse representation structure of the data and maximizes the local between-class separability simultaneously. Finally, the proposed method is transformed into a generalized eigenvalue problem. Extensive experiments on three publicly available face data sets confirmed the promising performance of the proposed SPDP approach.



---

*Source: 5269236-2016-01-20.xml*