# Face Recognition Using Double Sparse Local Fisher Discriminant Analysis

Authors: Zhan Wang; Qiuqi Ruan; Gaoyun An
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Local Fisher discriminant analysis (LFDA) was proposed for dealing with the multimodal problem. It not only combines the idea of locality preserving projections (LPP) for preserving the local structure of the high-dimensional data but also combines the idea of Fisher discriminant analysis (FDA) for obtaining the discriminant power. However, LFDA also suffers from the undersampled problem as well as many dimensionality reduction methods. Meanwhile, the projection matrix is not sparse. In this paper, we propose double sparse local Fisher discriminant analysis (DSLFDA) for face recognition. The proposed method firstly constructs a sparse and data-adaptive graph with nonnegative constraint. Then, DSLFDA reformulates the objective function as a regression-type optimization problem. The undersampled problem is avoided naturally and the sparse solution can be obtained by adding the regression-type problem to al
1 penalty. Experiments on Yale, ORL, and CMU PIE face databases are implemented to demonstrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

Dimensionality reduction tries to transform the high-dimensional data into lower-dimensional space in order to preserve the useful information as much as possible. It has a wide range of applications in pattern recognition, machine learning, and computer vision. A well-known approach for supervised dimensionality reduction is linear discriminant analysis (LDA) [1]. It tries to find a projection transformation by maximizing the between-class distance and minimizing the within-class distance simultaneously. In practical applications, LDA usually suffers from some limitations. First, LDA usually suffers from the undersampled problem [2]; that is, the dimension of data is larger than the number of training samples. Second, LDA can only uncover the global Euclidean structure. Third, the solution of LDA is not sparse, which cannot give the physical interpretation.To deal with the first problem, many methods have been proposed. Belhumeur et al. [3] proposed a two-stage principal component analysis (PCA) [4] + LDA method, which utilizes PCA to reduce dimensionality so as to make the within-class scatter matrix nonsingular, followed by LDA for recognition. However, some useful information may be compromised in the PCA stage. Chen et al. [5] extracted the most discriminant information from the null space of within-class scatter matrix. However, the discriminant information in the nonnull space of within-class scatter matrix would be discarded. Huang et al. [6] proposed an efficient null-space approach, which first removes the null space of total scatter matrix. This method is based on the observation that the null space of total scatter matrix is the intersection of the null space of between-class scatter matrix and the null space of within-class scatter matrix. Qin et al. [7] proposed a generalized null space uncorrelated Fisher discriminant analysis technique that integrates the uncorrelated discriminant analysis and weighted pairwise Fisher criterion for solving the undersampled problem. Yu and Yang [8] proposed direct LDA (DLDA) to overcome the undersampled problem. It removes the null space of between-class scatter matrix and extracts the discriminant information that corresponds to the smallest eigenvalues of the within-class scatter matrix. Zhang et al. [9] proposed an exponential discriminant analysis (EDA) method to extract the most discriminant information which is contained in the null space of the within-class scatter matrix.To deal with the second problem, many methods have been developed for dimensionality reduction. These methods focus on finding the local structure of the original data space. Locality preserving projections (LPP) [10] was proposed to find an embedding subspace that preserves local information. One limitation of LPP is that it is an unsupervised method. Because the discriminant information is important to the classification tasks, some locality preserving discriminant methods have been proposed. Discriminant locality preserving projection (DLPP) [11] was proposed to improve the performance of LPP. Laplacian linear discriminant analysis (LapLDA) [12] tries to capture the global and local structure of the data simultaneously by integrating LDA with a locality preserving regularizer. Local Fisher discriminant analysis (LFDA) [13] was proposed to deal with the multimodal problem. It combines the ideas of Fisher discriminant analysis (FDA) [1] and LPP and maximizes between-class separability and preserves within-class local structure simultaneously. In LDA, the dimension of the embedding space should be less than the number of classes. This limitation can be solved by using the LFDA algorithm.To deal with the third problem, many dimensionality reduction methods integrating the sparse representation theory have been proposed. These methods can be classified into two categories. The first category focuses on finding a subspace spanned by sparse vectors. The sparse projection vectors reveal which element or region of the patterns is important for recognition tasks. Sparse PCA (SPCA) [14] was proposed by using the least angle regression and elastic net to produce sparse principal components. Sparse discriminant analysis (SDA) [15] and sparse linear discriminant analysis (SLDA) [16] were proposed to learn a sparse discriminant subspace for feature extraction and classification in biological and medical data analysis. Both methods try to transform the original objective into a regression-type problem and add a lasso penalty to obtain the sparse projection axes. One disadvantage of these methods is that the number of sparse vectors is at most C
-
1. C is the number of class. The second category focuses on the sparse reconstructive weight among the training samples. Graph embedding framework views many dimensionality reduction methods as the graph construction [17]. The k-nearest neighbor and the ε-ball based methods are two popular ways for graph construction. Instead of them, Cheng et al. built the l
1-graph based on sparse representation [18]. The l
1-graph has proved that it is efficient and robust to data noise. l
1-graph based subspace learning methods include sparse preserving projections (SPP) [19] and discriminant sparse neighborhood preserving embedding (DSNPE) [20].Motivated byl
1-graph and sparse subspace learning, in this paper, we proposed double sparse local Fisher discriminant analysis (DSLFDA) for multimodal problem. It measures the similarity on the graph by integrating the sparse representation and nonnegative constraint. To obtain sparse projection vectors, the objective function can be transformed into a regression-type problem. Furthermore, the space spanned by the solution of regression-type problem is identical to that spanned by the solution of original problem. The proposed DSLFDA has two advantages: (1) it remains the sparse characteristic of l
1-graph; (2) to enhance the discriminant power of DSLFDA, the label information is used in the definition of local scatter matrices. Meanwhile, the projection vectors are sparse, which can make the physical meaning of the patterns clear. The proposed method is applied to face recognition and is examined using the Yale, ORL, and PIE face databases. Experimental results show that it can enhance the performance of LFDA effectively.The rest of this paper is organized as follows. In Section2, the LFDA algorithm is presented. The double sparse local Fisher discriminant analysis algorithm is proposed in Section 3. In Section 4, experiments are implemented to evaluate our proposed algorithm. The conclusions are given in Section 5.


## 2. Related Work

In this section, we give a brief of LDA and LFDA. Given a data setX
=
x
1
,
x
2
,
…
,
x
n
∈
R
m
×
n with each column corresponding to a data sample, x
i
∈
R
m
(
1
≤
i
≤
n
), the class label of x
i is set to y
i
∈
{
1,2
,
…
,
c
}, and c is the number of classes. We denote n
i as the number of samples in the ith class. Dimensionality reduction tries to map the point x
i
∈
R
m into z
i
∈
R
d (d
≪
m) by the linear transformation:(1)
z
i
=
V
T
x
i
i
=
1,2
,
…
,
n
,
where
V
∈
R
m
×
d
.The above transformation can be written as matrix form:(2)
Z
=
V
T
X
,where Z
=
[
z
1
,
z
2
,
…
,
z
n
]
∈
R
d
×
n.


### 2.1. Linear Discriminant Analysis

Linear discriminant analysis tries to find the discriminant vectors by the Fisher criterion, that is, the within-class distance is minimized and the between-class distance is maximized simultaneously. The within-class scatter matrixS
w and between-class scatter matrix S
b are, respectively, defined as follows:(3)
S
w
=
∑
i
=
1
c
∑
x
∈
X
i
(
x
-
m
i
)
x
-
m
i
T
,
S
b
=
∑
i
=
1
c
n
i
m
i
-
m
m
i
-
m
T
,where X
i is the data set of class i. m
i is the mean of the samples in class i and m is the mean of the total data. LDA seeks the optimal projection matrix by maximizing the following Fisher criterion:(4)
V
opt
=
arg
⁡
max
⁡
V
V
T
S
b
V
V
T
S
w
V
.The above optimization is equivalent to solving the following generalized eigenvalue problem: (5)
S
b
v
i
=
λ
i
S
w
v
i
;
V
=
[
v
1
,
v
2
,
…
,
v
d
] consists of the eigenvectors of S
w
-
1
S
b corresponding to the first d largest eigenvalues.


### 2.2. Local Fisher Discriminant Analysis

Local Fisher discriminant analysis (LFDA) is also a discriminant analysis method. It aims to deal with the multimodal problem. The local within-class scatter matrixS
lw and the between-class scatter matrix S
lb are defined as(6)
S
lw
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
i
j
lw
(
x
i
-
x
j
)
x
i
-
x
j
T
,
S
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
i
j
lb
(
x
i
-
x
j
)
x
i
-
x
j
T
,where(7)
W
i
j
lw
=
A
i
j
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
0
,
otherwise
,
W
i
j
lb
=
A
i
j
1
n
-
1
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
1
n
,
otherwise
;
A is the affinity matrix. A
i
j
=
exp
⁡
(
-
x
i
-
x
j
2
/
η
i
η
j
), and η
i is the local scaling of x
i defined by η
i
=
x
i
-
x
i
(
k
) where x
i
(
k
) is the kth nearest neighbor of x
i.The objection function of LFDA is formulated as(8)
V
opt
=
arg
⁡
max
⁡
V
tr
⁡
(
V
T
S
lb
V
)
tr
⁡
(
V
T
S
lw
V
)
,where tr
⁡
(
·
) is the trace of a matrix. The projection matrix can be obtained by calculating the eigenvectors of the following generalized eigenvalue problem:(9)
S
lb
v
i
=
λ
S
lw
v
i
,
i
=
1,2
,
…
,
d
;
V
=
[
v
1
,
v
2
,
…
,
v
d
]. Because of the definition of matrix A, LFDA can effectively preserve the local structure of the data.


## 2.1. Linear Discriminant Analysis

Linear discriminant analysis tries to find the discriminant vectors by the Fisher criterion, that is, the within-class distance is minimized and the between-class distance is maximized simultaneously. The within-class scatter matrixS
w and between-class scatter matrix S
b are, respectively, defined as follows:(3)
S
w
=
∑
i
=
1
c
∑
x
∈
X
i
(
x
-
m
i
)
x
-
m
i
T
,
S
b
=
∑
i
=
1
c
n
i
m
i
-
m
m
i
-
m
T
,where X
i is the data set of class i. m
i is the mean of the samples in class i and m is the mean of the total data. LDA seeks the optimal projection matrix by maximizing the following Fisher criterion:(4)
V
opt
=
arg
⁡
max
⁡
V
V
T
S
b
V
V
T
S
w
V
.The above optimization is equivalent to solving the following generalized eigenvalue problem: (5)
S
b
v
i
=
λ
i
S
w
v
i
;
V
=
[
v
1
,
v
2
,
…
,
v
d
] consists of the eigenvectors of S
w
-
1
S
b corresponding to the first d largest eigenvalues.


## 2.2. Local Fisher Discriminant Analysis

Local Fisher discriminant analysis (LFDA) is also a discriminant analysis method. It aims to deal with the multimodal problem. The local within-class scatter matrixS
lw and the between-class scatter matrix S
lb are defined as(6)
S
lw
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
i
j
lw
(
x
i
-
x
j
)
x
i
-
x
j
T
,
S
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
i
j
lb
(
x
i
-
x
j
)
x
i
-
x
j
T
,where(7)
W
i
j
lw
=
A
i
j
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
0
,
otherwise
,
W
i
j
lb
=
A
i
j
1
n
-
1
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
1
n
,
otherwise
;
A is the affinity matrix. A
i
j
=
exp
⁡
(
-
x
i
-
x
j
2
/
η
i
η
j
), and η
i is the local scaling of x
i defined by η
i
=
x
i
-
x
i
(
k
) where x
i
(
k
) is the kth nearest neighbor of x
i.The objection function of LFDA is formulated as(8)
V
opt
=
arg
⁡
max
⁡
V
tr
⁡
(
V
T
S
lb
V
)
tr
⁡
(
V
T
S
lw
V
)
,where tr
⁡
(
·
) is the trace of a matrix. The projection matrix can be obtained by calculating the eigenvectors of the following generalized eigenvalue problem:(9)
S
lb
v
i
=
λ
S
lw
v
i
,
i
=
1,2
,
…
,
d
;
V
=
[
v
1
,
v
2
,
…
,
v
d
]. Because of the definition of matrix A, LFDA can effectively preserve the local structure of the data.


## 3. Double Sparse Local Fisher Discriminant Analysis

### 3.1. Graph Construction by Sparse Representation

In LFDA, the affinity matrices are defined by the local scaling method. This method can be regarded as an extension ofk-nearest neighbor method. Recent research shows that the l
1-graph is robust to data noise and efficient for finding the underlying manifold structure. Therefore, we defined the new affinity matrix A
¯ by sparse representation theory. Let A
¯
=
[
a
1
,
a
2
,
…
,
a
n
]; each a
i is a sparse vector and obtained by the following l
1-minimization problem:(10)
min
⁡
a
i
1
s
.
t
lh
x
i
=
X
a
i
s
.
t
hl
1
=
1
T
a
i
s
.
t
hl
a
i
j
≥
0
,
∀
j
,where a
i
=
[
a
i
1
,
…
,
a
i
,
i
-
1
,
0
,
a
i
,
i
+
1
,
…
,
a
i
,
n
]
T is a n-dimensional vector in which the ith element is equal to zero. 1
∈
R
n is a vector of all ones. The l
1-minimization problem (10) can be solved by many efficient numerical algorithms. In this paper, the LARS algorithm [21] is used for solving problem (10). The matrix A
¯ can be seen as the similarity measurement by setting the matrix A
¯
=
(
A
¯
+
A
¯
T
)
/
2. Therefore, the new local scatter matrices can be defined as follows:(11)
S
¯
lw
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lw
(
x
i
-
x
j
)
x
i
-
x
j
T
,
(12)
S
¯
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lb
(
x
i
-
x
j
)
x
i
-
x
j
T
,where W
¯
lw and W
¯
lb are the weight matrices and defined as(13)
W
¯
i
j
lw
=
A
¯
i
j
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
0
,
otherwise
,
W
¯
i
j
lb
=
A
¯
i
j
1
n
-
1
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
1
n
,
otherwise
.The final objective function is described as follows:(14)
V
opt
=
arg
⁡
max
⁡
V
V
T
S
¯
lb
V
V
T
S
¯
lw
V
.The optimal projection V can be obtained by solving the following generalized eigenvalue problem:(15)
S
¯
lb
v
i
=
λ
S
¯
lw
v
i
,
i
=
1,2
,
…
,
d
.When the matrix S
¯
lw is nonsingular, the eigenvectors are obtained by the eigendecomposition of matrix (
S
¯
lw
)
-
1
S
¯
lb. However, the projection matrix is not sparse.


### 3.2. Finding the Sparse Solution

We first reformulate formulas (11) and (12) in matrix form. Consider(16)
S
¯
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
(
lb
)
(
x
i
-
x
j
)
x
i
-
x
j
T
S
¯
lb
=
1
2
∑
i
,
j
=
1
n
W
¯
i
j
lb
x
i
x
i
T
+
x
j
x
j
T
-
x
i
x
j
T
-
x
j
x
i
T
S
¯
lb
=
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lb
x
i
x
i
T
-
∑
i
,
j
=
1
n
W
¯
i
j
(
lb
)
x
i
x
j
T
S
¯
lb
=
X
D
¯
lb
-
W
¯
lb
X
T
=
X
L
¯
lb
X
T
,where D
¯
(
lb
) is the diagonal matrix and the ith diagonal element is D
¯
i
,
i
(
lb
)
=
∑
j
=
1
n
W
¯
i
j
(
lb
), L
¯
(
lb
)
=
D
¯
(
lb
)
-
W
¯
(
lb
). Similarly, formula (12) can be expressed as (17)
S
¯
lw
=
X
L
¯
(
lw
)
X
T
,where L
¯
(
lw
)
=
D
¯
(
lw
)
-
W
¯
(
lw
), D
¯
(
w
) is the diagonal matrix, and the ith diagonal element is D
¯
i
,
i
(
lw
)
=
∑
j
=
1
n
W
¯
i
j
(
lw
).MatricesL
¯
(
lb
) and L
¯
(
lw
) are always symmetric and positive semidefinite; therefore, the eigendecomposition of L
¯
(
lb
) and L
¯
(
lw
) can be expression as follows:(18)
L
¯
(
lb
)
=
U
b
Σ
b
U
b
T
,
L
¯
(
lw
)
=
U
w
Σ
w
U
w
T
,where Σ
b and Σ
w are the diagonal matrices. Their diagonal elements are the eigenvalues of matrices L
¯
(
lb
) and L
¯
(
lw
), respectively. So S
¯
lb and S
¯
lw can be rewritten as(19)
S
¯
lb
=
X
L
¯
(
lb
)
X
T
=
X
U
b
Σ
b
U
b
T
X
T
=
X
U
b
Σ
b
1
/
2
X
U
b
Σ
b
1
/
2
T
S
¯
lb
=
H
b
H
b
T
,
S
¯
lw
=
X
L
¯
(
lw
)
X
T
=
X
U
w
Σ
w
U
w
T
X
T
=
X
U
w
Σ
w
1
/
2
X
U
w
Σ
w
1
/
2
T
S
¯
lw
=
H
w
H
w
T
,where H
b
=
X
U
b
Σ
b
1
/
2 and H
w
=
X
U
w
Σ
w
1
/
2.The following result which was inspired by [14, 16] gives the relationship between problem (10) and the regression-type problem.Theorem 1.
Suppose thatS
¯
l
w is positive definite; its Cholesky decomposition can be expressed as S
¯
l
w
=
F
w
F
w
T, where F
w
∈
R
m
×
m is a lower triangular matrix. Let V
=
[
v
1
,
v
2
,
…
,
v
d
] be the eigenvector of problem (15) associated with the first d largest eigenvalues. Let P
=
[
p
1
,
p
2
,
…
,
p
d
] and Q
=
[
q
1
,
q
2
,
…
,
q
d
] be the optimal solution to the following problem:(20)
J
P
,
Q
=
arg
⁡
min
P
,
Q
∑
i
=
1
n
F
w
-
1
H
b
:
,
i
-
P
Q
T
H
b
:
,
i
2
+
α
∑
j
=
1
d
q
j
T
S
l
w
q
j
s
.
t
.
P
T
P
=
I
,where α
>
0 and H
b
(
:
,
i
) is the ith column of H
b. Then the columns of Q span the same linear space as well as those of V.To obtain sparse projection vectors, we add al
1 penalty to the objective function (20):(21)
J
1
P
,
Q
=
arg
⁡
min
⁡
P
,
Q
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
+
α
∑
j
=
1
d
q
j
T
S
lw
q
j
+
ζ
∑
j
=
1
d
q
j
1
s
.
t
.
P
T
P
=
I
.Generally speaking, it is difficult to compute the optimalP and Q simultaneously. An iterative algorithm was usually used for solving problem (21). For a fixed P, there exists an orthogonal matrix P
^ such that [
P
,
P
^
] is m
×
m column orthogonal matrix. Then the first term of (21) can be rewritten as(22)
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
=
H
b
T
F
w
-
T
-
H
b
T
Q
P
T
2
=
H
b
T
F
w
-
T
P
,
P
^
T
-
H
b
T
Q
P
T
P
,
P
^
T
2
=
H
b
T
F
w
-
T
P
-
H
b
T
Q
2
+
H
b
T
F
w
-
T
P
^
2
=
∑
j
=
1
d
H
b
T
F
w
-
T
p
j
-
H
b
T
q
j
2
+
H
b
T
F
w
-
T
P
^
2
.If P is fixed, then problem (21) is transformed into(23)
arg
⁡
min
⁡
Q
∑
i
=
1
d
H
b
T
F
w
-
T
p
j
-
H
b
T
q
j
2
+
α
∑
j
=
1
d
q
j
T
S
lw
q
j
hhhh
+
ζ
∑
j
=
1
d
q
j
1which is equivalent to d independent LASSO problem.For a fixedQ, problem (21) is equivalent to minimizing the following problem with ignoring the constant terms:(24)
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
=
H
b
T
G
w
-
T
-
H
b
T
Q
P
T
2
,which is subject to P
T
P
=
I. The optimal solution can be obtained by computing the singular value decomposition (25)
F
w
-
1
H
b
H
b
T
Q
=
U
P
Σ
P
V
Pand P
=
U
p
V
p
T.The algorithm procedure of DSLFDA is summarized as follows.Input: the data matrix X.Output: the sparse projection matrix Q.(1)
Calculate affinity matrixA
¯ by l
1-minimization problem  (10).
(2)
Calculate matrixH
b
=
X
U
b
Σ
b
1
/
2 by (18) and matrix F
w by the Cholesky decomposition of S
¯
lw.
(3)
Initialize matrixP as an arbitrary column orthogonal matrix.
(4)
For givenP, solve l
1-minimization problem (23) which is equivalent to d independent LASSO problem.
(5)
Calculate the SVD ofF
w
-
1
H
b
H
b
T
Q
=
U
P
Σ
P
V
P and update P
=
U
p
V
p
T.
(6)
Repeat steps 4 and 5 until converges.


## 3.1. Graph Construction by Sparse Representation

In LFDA, the affinity matrices are defined by the local scaling method. This method can be regarded as an extension ofk-nearest neighbor method. Recent research shows that the l
1-graph is robust to data noise and efficient for finding the underlying manifold structure. Therefore, we defined the new affinity matrix A
¯ by sparse representation theory. Let A
¯
=
[
a
1
,
a
2
,
…
,
a
n
]; each a
i is a sparse vector and obtained by the following l
1-minimization problem:(10)
min
⁡
a
i
1
s
.
t
lh
x
i
=
X
a
i
s
.
t
hl
1
=
1
T
a
i
s
.
t
hl
a
i
j
≥
0
,
∀
j
,where a
i
=
[
a
i
1
,
…
,
a
i
,
i
-
1
,
0
,
a
i
,
i
+
1
,
…
,
a
i
,
n
]
T is a n-dimensional vector in which the ith element is equal to zero. 1
∈
R
n is a vector of all ones. The l
1-minimization problem (10) can be solved by many efficient numerical algorithms. In this paper, the LARS algorithm [21] is used for solving problem (10). The matrix A
¯ can be seen as the similarity measurement by setting the matrix A
¯
=
(
A
¯
+
A
¯
T
)
/
2. Therefore, the new local scatter matrices can be defined as follows:(11)
S
¯
lw
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lw
(
x
i
-
x
j
)
x
i
-
x
j
T
,
(12)
S
¯
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lb
(
x
i
-
x
j
)
x
i
-
x
j
T
,where W
¯
lw and W
¯
lb are the weight matrices and defined as(13)
W
¯
i
j
lw
=
A
¯
i
j
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
0
,
otherwise
,
W
¯
i
j
lb
=
A
¯
i
j
1
n
-
1
n
k
,
x
i
,
x
j
belong
to
k
th
class
,
1
n
,
otherwise
.The final objective function is described as follows:(14)
V
opt
=
arg
⁡
max
⁡
V
V
T
S
¯
lb
V
V
T
S
¯
lw
V
.The optimal projection V can be obtained by solving the following generalized eigenvalue problem:(15)
S
¯
lb
v
i
=
λ
S
¯
lw
v
i
,
i
=
1,2
,
…
,
d
.When the matrix S
¯
lw is nonsingular, the eigenvectors are obtained by the eigendecomposition of matrix (
S
¯
lw
)
-
1
S
¯
lb. However, the projection matrix is not sparse.


## 3.2. Finding the Sparse Solution

We first reformulate formulas (11) and (12) in matrix form. Consider(16)
S
¯
lb
=
1
2
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
(
lb
)
(
x
i
-
x
j
)
x
i
-
x
j
T
S
¯
lb
=
1
2
∑
i
,
j
=
1
n
W
¯
i
j
lb
x
i
x
i
T
+
x
j
x
j
T
-
x
i
x
j
T
-
x
j
x
i
T
S
¯
lb
=
∑
i
=
1
n
∑
j
=
1
n
W
¯
i
j
lb
x
i
x
i
T
-
∑
i
,
j
=
1
n
W
¯
i
j
(
lb
)
x
i
x
j
T
S
¯
lb
=
X
D
¯
lb
-
W
¯
lb
X
T
=
X
L
¯
lb
X
T
,where D
¯
(
lb
) is the diagonal matrix and the ith diagonal element is D
¯
i
,
i
(
lb
)
=
∑
j
=
1
n
W
¯
i
j
(
lb
), L
¯
(
lb
)
=
D
¯
(
lb
)
-
W
¯
(
lb
). Similarly, formula (12) can be expressed as (17)
S
¯
lw
=
X
L
¯
(
lw
)
X
T
,where L
¯
(
lw
)
=
D
¯
(
lw
)
-
W
¯
(
lw
), D
¯
(
w
) is the diagonal matrix, and the ith diagonal element is D
¯
i
,
i
(
lw
)
=
∑
j
=
1
n
W
¯
i
j
(
lw
).MatricesL
¯
(
lb
) and L
¯
(
lw
) are always symmetric and positive semidefinite; therefore, the eigendecomposition of L
¯
(
lb
) and L
¯
(
lw
) can be expression as follows:(18)
L
¯
(
lb
)
=
U
b
Σ
b
U
b
T
,
L
¯
(
lw
)
=
U
w
Σ
w
U
w
T
,where Σ
b and Σ
w are the diagonal matrices. Their diagonal elements are the eigenvalues of matrices L
¯
(
lb
) and L
¯
(
lw
), respectively. So S
¯
lb and S
¯
lw can be rewritten as(19)
S
¯
lb
=
X
L
¯
(
lb
)
X
T
=
X
U
b
Σ
b
U
b
T
X
T
=
X
U
b
Σ
b
1
/
2
X
U
b
Σ
b
1
/
2
T
S
¯
lb
=
H
b
H
b
T
,
S
¯
lw
=
X
L
¯
(
lw
)
X
T
=
X
U
w
Σ
w
U
w
T
X
T
=
X
U
w
Σ
w
1
/
2
X
U
w
Σ
w
1
/
2
T
S
¯
lw
=
H
w
H
w
T
,where H
b
=
X
U
b
Σ
b
1
/
2 and H
w
=
X
U
w
Σ
w
1
/
2.The following result which was inspired by [14, 16] gives the relationship between problem (10) and the regression-type problem.Theorem 1.
Suppose thatS
¯
l
w is positive definite; its Cholesky decomposition can be expressed as S
¯
l
w
=
F
w
F
w
T, where F
w
∈
R
m
×
m is a lower triangular matrix. Let V
=
[
v
1
,
v
2
,
…
,
v
d
] be the eigenvector of problem (15) associated with the first d largest eigenvalues. Let P
=
[
p
1
,
p
2
,
…
,
p
d
] and Q
=
[
q
1
,
q
2
,
…
,
q
d
] be the optimal solution to the following problem:(20)
J
P
,
Q
=
arg
⁡
min
P
,
Q
∑
i
=
1
n
F
w
-
1
H
b
:
,
i
-
P
Q
T
H
b
:
,
i
2
+
α
∑
j
=
1
d
q
j
T
S
l
w
q
j
s
.
t
.
P
T
P
=
I
,where α
>
0 and H
b
(
:
,
i
) is the ith column of H
b. Then the columns of Q span the same linear space as well as those of V.To obtain sparse projection vectors, we add al
1 penalty to the objective function (20):(21)
J
1
P
,
Q
=
arg
⁡
min
⁡
P
,
Q
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
+
α
∑
j
=
1
d
q
j
T
S
lw
q
j
+
ζ
∑
j
=
1
d
q
j
1
s
.
t
.
P
T
P
=
I
.Generally speaking, it is difficult to compute the optimalP and Q simultaneously. An iterative algorithm was usually used for solving problem (21). For a fixed P, there exists an orthogonal matrix P
^ such that [
P
,
P
^
] is m
×
m column orthogonal matrix. Then the first term of (21) can be rewritten as(22)
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
=
H
b
T
F
w
-
T
-
H
b
T
Q
P
T
2
=
H
b
T
F
w
-
T
P
,
P
^
T
-
H
b
T
Q
P
T
P
,
P
^
T
2
=
H
b
T
F
w
-
T
P
-
H
b
T
Q
2
+
H
b
T
F
w
-
T
P
^
2
=
∑
j
=
1
d
H
b
T
F
w
-
T
p
j
-
H
b
T
q
j
2
+
H
b
T
F
w
-
T
P
^
2
.If P is fixed, then problem (21) is transformed into(23)
arg
⁡
min
⁡
Q
∑
i
=
1
d
H
b
T
F
w
-
T
p
j
-
H
b
T
q
j
2
+
α
∑
j
=
1
d
q
j
T
S
lw
q
j
hhhh
+
ζ
∑
j
=
1
d
q
j
1which is equivalent to d independent LASSO problem.For a fixedQ, problem (21) is equivalent to minimizing the following problem with ignoring the constant terms:(24)
∑
i
=
1
n
F
w
-
1
H
b
(
:
,
i
)
-
P
Q
T
H
b
(
:
,
i
)
=
H
b
T
G
w
-
T
-
H
b
T
Q
P
T
2
,which is subject to P
T
P
=
I. The optimal solution can be obtained by computing the singular value decomposition (25)
F
w
-
1
H
b
H
b
T
Q
=
U
P
Σ
P
V
Pand P
=
U
p
V
p
T.The algorithm procedure of DSLFDA is summarized as follows.Input: the data matrix X.Output: the sparse projection matrix Q.(1)
Calculate affinity matrixA
¯ by l
1-minimization problem  (10).
(2)
Calculate matrixH
b
=
X
U
b
Σ
b
1
/
2 by (18) and matrix F
w by the Cholesky decomposition of S
¯
lw.
(3)
Initialize matrixP as an arbitrary column orthogonal matrix.
(4)
For givenP, solve l
1-minimization problem (23) which is equivalent to d independent LASSO problem.
(5)
Calculate the SVD ofF
w
-
1
H
b
H
b
T
Q
=
U
P
Σ
P
V
P and update P
=
U
p
V
p
T.
(6)
Repeat steps 4 and 5 until converges.


## 4. Experimental Results

In this section, we use the proposed DSLFDA method for face recognition. Three face image databases, that is, Yale [22], ORL [23], and PIE [24], are used in the experiments. We compare our proposed algorithm with PCA, LDA, LPP, LFDA, SPCA, SPP, DSNPE, and SLDA. For simplicity, we use nearest neighbor classifier for classification task and the Euclidean metric is used as the distance measure.


### 4.1. Experiment on the Yale Face Database

The Yale face database contains 165 grayscale images of 15 individuals. Each individual has 11 images. These images were captured under lighting conditions (left-light, center-light, and right-light), with various facial expressions (normal, happy, sad, sleepy, surprised, and wink), and with facial details (with glasses or without). The original size of the images is243
×
320 pixels. In our experiments, the face region of each original image was cropped based on the location of eyes. Each cropped image was resized to 32
×
32 pixels. Figure 1 shows the cropped sample images of two individuals from the Yale database.Figure 1
Sample images of two individuals from Yale database.In the first experiment, we randomly selectl (l
=
2,3
,
4,5
,
6) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. For LFDA, the parameter is set to l
-
1 for simplicity. LPP is implemented in supervised model. For SPCA, we manually choose the sparse principal component in order to obtain the best performance. Table 1 shows the recognition accuracies of different methods with the corresponding dimension.Table 1
The top recognition rates (%) and the corresponding dimensions on Yale database by different methods (mean ± std).
2 trains
3 trains
4 trains
5 trains
6 trains
PCA
46.30 ± 3.26(30)
51.58 ± 4.00(44)
56.19 ± 4.16(56)
57.33 ± 4.60(74)
62.40 ± 4.06(84)
LDA
45.11 ± 3.46(10)
62.08 ± 4.31(14)
70.86 ± 4.88(14)
71.44 ± 5.19(14)
77.22 ± 3.47(14)
LPP
45.26 ± 3.52(10)
62.83 ± 4.22(14)
70.57 ± 4.59(14)
72.22 ± 3.81(16)
78.00 ± 3.46(20)
LFDA
45.11 ± 3.46(10)
62.50 ± 5.43(14)
71.33 ± 5.07(14)
72.33 ± 5.09(18)
78.27 ± 3.72(20)
SPCA
43.19 ± 3.22(28)
49.83 ± 4.08(44)
54.95 ± 3.70(56)
56.78 ± 3.33(44)
61.87 ± 5.11(30)
SLDA
51.19 ± 5.78(14)
63.85 ± 3.47(14)
72.00 ± 4.76(14)
72.56 ± 2.29(14)
78.40 ± 2.42(14)
SPP
46.59 ± 5.36(16)
52.92 ± 3.63(28)
57.67 ± 3.54(48)
58.48 ± 3.76(68)
64.53 ± 4.63(72)
DSNPE
50.74 ± 5.26(12)
63.58 ± 3.64(30)
73.62 ± 5.04(46)
75.89 ± 2.82(54)
80.08 ± 2.61(64)
DSLFDA
53.11 ± 5.36(30)
65.17 ± 3.68(36)
73.24 ± 5.03(40)
74.44 ± 3.51(42)
81.47 ± 2.77(56)In the second experiment, we experiment with different dimensionalities of the projected space. Five images per individual were randomly selected for training, and the remaining images were used for testing. Figure2 shows the performance of different methods.Figure 2
The recognition performance versus different dimensions on the Yale database.


### 4.2. Experiment on the ORL Face Database

The ORL database contains 400 images of 40 individuals. Each individual has 10 images. The images were captured at different times, under various light conditions, and with different facial expressions. The original size of the images is243
×
320 pixels. The images were manually cropped and resized to 32
×
32 pixels. Figure 3 shows the cropped sample images of two individuals from the ORL database.Figure 3
Sample images of two individuals from ORL database.In the first experiment, we randomly selectl (l
=
2,3
,
4,5
,
6) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. The experimental parameters were set as in Section 4.1.In the second experiment, we experiment with different dimensionalities of the projected space. Five images per individual were randomly selected for training, and the remaining images were randomly selected for testing. Figure4 shows the performance of different methods (Table 2).Table 2
The top recognition rates (%) and the corresponding dimensions on ORL database by different methods (mean ± std).
2 trains
3 trains
4 trains
5 trains
6 trains
PCA
71.63 ± 3.06(79)
79.14 ± 1.91(113)
83.87 ± 1.77(159)
86.85 ± 2.01(189)
89.94 ± 2.72(203)
LDA
75.87 ± 3.42(27)
85.96 ± 2.61(39)
91.54 ± 1.51(39)
93.85 ± 1.63(39)
95.63 ± 1.89(39)
LPP
78.78 ± 2.80(39)
86.25 ± 2.93(39)
90.46 ± 1.83(39)
94.40 ± 1.71(39)
95.94 ± 1.96(41)
LFDA
82.13 ± 3.19(39)
87.00 ± 2.97(37)
90.83 ± 1.73(37)
94.50 ± 1.90(39)
96.06 ± 2.74(39)
SPCA
71.16 ± 2.80(79)
78.50 ± 2.42(117)
84.04 ± 1.82(131)
86.80 ± 2.28(183)
90.56 ± 2.03(73)
SLDA
82.03 ± 2.82(39)
90.21 ± 1.85(39)
93.79 ± 1.26(39)
96.05 ± 1.83(39)
97.56 ± 1.12(39)
SPP
79.53 ± 2.22(57)
83.50 ± 2.08(63)
86.58 ± 2.10(99)
87.40 ± 2.63(95)
88.19 ± 3.19(93)
DSNPE
78.97 ± 3.47(39)
86.43 ± 1.92(55)
92.92 ± 1.59(73)
94.60 ± 1.96(71)
95.94 ± 2.29(89)
DSLFDA
81.16 ± 2.88(69)
90.25 ± 2.51(67)
94.38 ± 1.54(81)
96.65 ± 1.20(87)
97.88 ± 1.03(95)Figure 4
The recognition performance versus different dimensions on the ORL database.


### 4.3. Experiment on the PIE Face Database

The CMU PIE face database contains 41368 images of 68 individuals. The images were captured under 13 different poses, under 43 different illumination conditions, and with 4 different expressions. In our experiments, we choose a subset (C29) that contains 1632 images of 68 individuals. These were manually cropped and resized to32
×
32 pixels. Figure 5 shows the cropped sample images of two individuals from CMU PIE database.Figure 5
Sample images of two individuals from CMU PIE database.In the first experiment, we randomly selectl (l
=
3,6
,
9,12,15) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. The experimental parameters were set as in Section 4.1. Table 3 shows the recognition accuracies of different methods with the corresponding dimension.Table 3
The top recognition rates (%) and the corresponding dimensions on CMU PIE database by different methods (mean ± std).
3 trains
6 trains
9 trains
12 trains
15 trains
PCA
36.11 ± 0.85(155)
55.05 ± 0.94(179)
66.81 ± 2.08(387)
76.80 ± 1.66(415)
83.45 ± 1.77(431)
LDA
78.40 ± 1.43(67)
87.70 ± 1.30(67)
90.17 ± 1.04(67)
91.67 ± 0.46(67)
92.37 ± 0.56(67)
LPP
78.72 ± 1.19(79)
89.40 ± 0.96(187)
91.08 ± 0.86(163)
92.22 ± 0.52(179)
92.79 ± 0.65(179)
LFDA
78.68 ± 1.49(99)
88.88 ± 1.09(139)
90.70 ± 0.68(143)
91.83 ± 0.51(83)
92.43 ± 0.58(127)
SPCA
33.25 ± 0.60(63)
52.51 ± 1.05(347)
65.03 ± 2.14(499)
75.40 ± 1.97(695)
83.32 ± 2.02(751)
SLDA
78.17 ± 1.38(67)
88.46 ± 1.23(67)
92.15 ± 1.09(67)
94.68 ± 0.36(67)
96.36 ± 0.73(67)
SPP
75.11 ± 0.97(67)
87.26 ± 1.04(87)
90.59 ± 1.35(87)
93.42 ± 0.56(99)
95.36 ± 0.80(99)
DSNPE
78.93 ± 1.11(119)
88.35 ± 1.22(79)
91.77 ± 1.28(83)
94.09 ± 0.60(67)
96.05 ± 0.49(51)
DSLFDA
78.35 ± 1.24(147)
88.99 ± 1.20(263)
92.31 ± 1.10(211)
95.10 ± 0.47(191)
96.63 ± 0.62(251)In the second experiment, we experiment with different dimensionalities of the projected space. Fifteen images per individual were randomly selected for training, and the remaining images were randomly selected for testing. Figure6 shows the performance of different methods.Figure 6
The recognition performance versus different dimensions on the CMU PIE database.


## 4.1. Experiment on the Yale Face Database

The Yale face database contains 165 grayscale images of 15 individuals. Each individual has 11 images. These images were captured under lighting conditions (left-light, center-light, and right-light), with various facial expressions (normal, happy, sad, sleepy, surprised, and wink), and with facial details (with glasses or without). The original size of the images is243
×
320 pixels. In our experiments, the face region of each original image was cropped based on the location of eyes. Each cropped image was resized to 32
×
32 pixels. Figure 1 shows the cropped sample images of two individuals from the Yale database.Figure 1
Sample images of two individuals from Yale database.In the first experiment, we randomly selectl (l
=
2,3
,
4,5
,
6) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. For LFDA, the parameter is set to l
-
1 for simplicity. LPP is implemented in supervised model. For SPCA, we manually choose the sparse principal component in order to obtain the best performance. Table 1 shows the recognition accuracies of different methods with the corresponding dimension.Table 1
The top recognition rates (%) and the corresponding dimensions on Yale database by different methods (mean ± std).
2 trains
3 trains
4 trains
5 trains
6 trains
PCA
46.30 ± 3.26(30)
51.58 ± 4.00(44)
56.19 ± 4.16(56)
57.33 ± 4.60(74)
62.40 ± 4.06(84)
LDA
45.11 ± 3.46(10)
62.08 ± 4.31(14)
70.86 ± 4.88(14)
71.44 ± 5.19(14)
77.22 ± 3.47(14)
LPP
45.26 ± 3.52(10)
62.83 ± 4.22(14)
70.57 ± 4.59(14)
72.22 ± 3.81(16)
78.00 ± 3.46(20)
LFDA
45.11 ± 3.46(10)
62.50 ± 5.43(14)
71.33 ± 5.07(14)
72.33 ± 5.09(18)
78.27 ± 3.72(20)
SPCA
43.19 ± 3.22(28)
49.83 ± 4.08(44)
54.95 ± 3.70(56)
56.78 ± 3.33(44)
61.87 ± 5.11(30)
SLDA
51.19 ± 5.78(14)
63.85 ± 3.47(14)
72.00 ± 4.76(14)
72.56 ± 2.29(14)
78.40 ± 2.42(14)
SPP
46.59 ± 5.36(16)
52.92 ± 3.63(28)
57.67 ± 3.54(48)
58.48 ± 3.76(68)
64.53 ± 4.63(72)
DSNPE
50.74 ± 5.26(12)
63.58 ± 3.64(30)
73.62 ± 5.04(46)
75.89 ± 2.82(54)
80.08 ± 2.61(64)
DSLFDA
53.11 ± 5.36(30)
65.17 ± 3.68(36)
73.24 ± 5.03(40)
74.44 ± 3.51(42)
81.47 ± 2.77(56)In the second experiment, we experiment with different dimensionalities of the projected space. Five images per individual were randomly selected for training, and the remaining images were used for testing. Figure2 shows the performance of different methods.Figure 2
The recognition performance versus different dimensions on the Yale database.


## 4.2. Experiment on the ORL Face Database

The ORL database contains 400 images of 40 individuals. Each individual has 10 images. The images were captured at different times, under various light conditions, and with different facial expressions. The original size of the images is243
×
320 pixels. The images were manually cropped and resized to 32
×
32 pixels. Figure 3 shows the cropped sample images of two individuals from the ORL database.Figure 3
Sample images of two individuals from ORL database.In the first experiment, we randomly selectl (l
=
2,3
,
4,5
,
6) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. The experimental parameters were set as in Section 4.1.In the second experiment, we experiment with different dimensionalities of the projected space. Five images per individual were randomly selected for training, and the remaining images were randomly selected for testing. Figure4 shows the performance of different methods (Table 2).Table 2
The top recognition rates (%) and the corresponding dimensions on ORL database by different methods (mean ± std).
2 trains
3 trains
4 trains
5 trains
6 trains
PCA
71.63 ± 3.06(79)
79.14 ± 1.91(113)
83.87 ± 1.77(159)
86.85 ± 2.01(189)
89.94 ± 2.72(203)
LDA
75.87 ± 3.42(27)
85.96 ± 2.61(39)
91.54 ± 1.51(39)
93.85 ± 1.63(39)
95.63 ± 1.89(39)
LPP
78.78 ± 2.80(39)
86.25 ± 2.93(39)
90.46 ± 1.83(39)
94.40 ± 1.71(39)
95.94 ± 1.96(41)
LFDA
82.13 ± 3.19(39)
87.00 ± 2.97(37)
90.83 ± 1.73(37)
94.50 ± 1.90(39)
96.06 ± 2.74(39)
SPCA
71.16 ± 2.80(79)
78.50 ± 2.42(117)
84.04 ± 1.82(131)
86.80 ± 2.28(183)
90.56 ± 2.03(73)
SLDA
82.03 ± 2.82(39)
90.21 ± 1.85(39)
93.79 ± 1.26(39)
96.05 ± 1.83(39)
97.56 ± 1.12(39)
SPP
79.53 ± 2.22(57)
83.50 ± 2.08(63)
86.58 ± 2.10(99)
87.40 ± 2.63(95)
88.19 ± 3.19(93)
DSNPE
78.97 ± 3.47(39)
86.43 ± 1.92(55)
92.92 ± 1.59(73)
94.60 ± 1.96(71)
95.94 ± 2.29(89)
DSLFDA
81.16 ± 2.88(69)
90.25 ± 2.51(67)
94.38 ± 1.54(81)
96.65 ± 1.20(87)
97.88 ± 1.03(95)Figure 4
The recognition performance versus different dimensions on the ORL database.


## 4.3. Experiment on the PIE Face Database

The CMU PIE face database contains 41368 images of 68 individuals. The images were captured under 13 different poses, under 43 different illumination conditions, and with 4 different expressions. In our experiments, we choose a subset (C29) that contains 1632 images of 68 individuals. These were manually cropped and resized to32
×
32 pixels. Figure 5 shows the cropped sample images of two individuals from CMU PIE database.Figure 5
Sample images of two individuals from CMU PIE database.In the first experiment, we randomly selectl (l
=
3,6
,
9,12,15) images per subject for training and the remaining images are for testing. 10 time runs were implemented for stable performance. The average rates are used as the final recognition accuracies. The experimental parameters were set as in Section 4.1. Table 3 shows the recognition accuracies of different methods with the corresponding dimension.Table 3
The top recognition rates (%) and the corresponding dimensions on CMU PIE database by different methods (mean ± std).
3 trains
6 trains
9 trains
12 trains
15 trains
PCA
36.11 ± 0.85(155)
55.05 ± 0.94(179)
66.81 ± 2.08(387)
76.80 ± 1.66(415)
83.45 ± 1.77(431)
LDA
78.40 ± 1.43(67)
87.70 ± 1.30(67)
90.17 ± 1.04(67)
91.67 ± 0.46(67)
92.37 ± 0.56(67)
LPP
78.72 ± 1.19(79)
89.40 ± 0.96(187)
91.08 ± 0.86(163)
92.22 ± 0.52(179)
92.79 ± 0.65(179)
LFDA
78.68 ± 1.49(99)
88.88 ± 1.09(139)
90.70 ± 0.68(143)
91.83 ± 0.51(83)
92.43 ± 0.58(127)
SPCA
33.25 ± 0.60(63)
52.51 ± 1.05(347)
65.03 ± 2.14(499)
75.40 ± 1.97(695)
83.32 ± 2.02(751)
SLDA
78.17 ± 1.38(67)
88.46 ± 1.23(67)
92.15 ± 1.09(67)
94.68 ± 0.36(67)
96.36 ± 0.73(67)
SPP
75.11 ± 0.97(67)
87.26 ± 1.04(87)
90.59 ± 1.35(87)
93.42 ± 0.56(99)
95.36 ± 0.80(99)
DSNPE
78.93 ± 1.11(119)
88.35 ± 1.22(79)
91.77 ± 1.28(83)
94.09 ± 0.60(67)
96.05 ± 0.49(51)
DSLFDA
78.35 ± 1.24(147)
88.99 ± 1.20(263)
92.31 ± 1.10(211)
95.10 ± 0.47(191)
96.63 ± 0.62(251)In the second experiment, we experiment with different dimensionalities of the projected space. Fifteen images per individual were randomly selected for training, and the remaining images were randomly selected for testing. Figure6 shows the performance of different methods.Figure 6
The recognition performance versus different dimensions on the CMU PIE database.


## 5. Discussion and Conclusion

### 5.1. Discussion

Based on the above experimental results, we can conclude the following observations.(1)
For each method, the recognition rate increases with the increase of training sample sizes. The supervised extension of LPP can effectively improve the performance. PCA and SPCA achieve the worst results in all experiments; meanwhile, the performance of SPCA is inferior to that of PCA on all face databases. The reason may be that the number of nonzero variables for each component is selected equally.
(2)
For LDA and SLDA, the dimensionalities of projected subspace areC
-
1 at most; C is the number of classes. LFDA and DSLFDA can overcome this limitation; hence, we may project the original high-dimensional data into a low-dimensional subspace whose dimensionality is larger than the number of classes.
(3)
From Table3, LPP and SLDA outperform LFDA on the CMU PIE database. However, DSLFDA can achieve better performance than other methods. This point shows that DSLFDA improves not only the performance of LFDA but also the performance of sparse-based method, such as SLDA. The proposed DSLFDA algorithm constructs the graph on the original data and obtains the nonnegative similarity measurement. This is different from SPP and DSNPE.
(4)
From the experimental results, we obtain that SPP can get competitive performance on CMU PIE database, rather than ORL and Yale databases. The reason may be that the sparse representation needs abundant training samples. Conversely, the nonnegative similarity measurement in DSLFDA is adaptive and can overcome the drawback of sparse representation.
(5)
DSNPE can be regarded as an extension of SPP. It can extract the discriminant information and perform better than SPP. On the Yale database, DSNPE can achieve the best recognition performance when the training samples per individual are four and five.


### 5.2. Conclusion

In this paper, we proposed a sparse projection method, called DSLFDA, for face recognition. It defines a novel affinity matrix that describes the relationships of points on the original high-dimensional data. The sparse projection vectors are obtained by solving thel
1-optimization problem. Experiments on Yale, ORL, and CMU PIE face databases indicate that DSLFDA can get competitive performance compared to other dimensionality reduction methods.We only focus on supervised learning in this paper. Because a large amount of unlabeled data is available in practical applications, semisupervised learning has attracted much attention in recent years [25–27]. One of our future works is to extend our approach under the semisupervised learning framework. On the other hand, DSLFDA needs the local within-class scatter matrix positive definite. We add an identity matrix to the local within-class scatter matrix for regularization. This may motivate us to find the regularization method to approximate the local within-class scatter matrix well.


## 5.1. Discussion

Based on the above experimental results, we can conclude the following observations.(1)
For each method, the recognition rate increases with the increase of training sample sizes. The supervised extension of LPP can effectively improve the performance. PCA and SPCA achieve the worst results in all experiments; meanwhile, the performance of SPCA is inferior to that of PCA on all face databases. The reason may be that the number of nonzero variables for each component is selected equally.
(2)
For LDA and SLDA, the dimensionalities of projected subspace areC
-
1 at most; C is the number of classes. LFDA and DSLFDA can overcome this limitation; hence, we may project the original high-dimensional data into a low-dimensional subspace whose dimensionality is larger than the number of classes.
(3)
From Table3, LPP and SLDA outperform LFDA on the CMU PIE database. However, DSLFDA can achieve better performance than other methods. This point shows that DSLFDA improves not only the performance of LFDA but also the performance of sparse-based method, such as SLDA. The proposed DSLFDA algorithm constructs the graph on the original data and obtains the nonnegative similarity measurement. This is different from SPP and DSNPE.
(4)
From the experimental results, we obtain that SPP can get competitive performance on CMU PIE database, rather than ORL and Yale databases. The reason may be that the sparse representation needs abundant training samples. Conversely, the nonnegative similarity measurement in DSLFDA is adaptive and can overcome the drawback of sparse representation.
(5)
DSNPE can be regarded as an extension of SPP. It can extract the discriminant information and perform better than SPP. On the Yale database, DSNPE can achieve the best recognition performance when the training samples per individual are four and five.


## 5.2. Conclusion

In this paper, we proposed a sparse projection method, called DSLFDA, for face recognition. It defines a novel affinity matrix that describes the relationships of points on the original high-dimensional data. The sparse projection vectors are obtained by solving thel
1-optimization problem. Experiments on Yale, ORL, and CMU PIE face databases indicate that DSLFDA can get competitive performance compared to other dimensionality reduction methods.We only focus on supervised learning in this paper. Because a large amount of unlabeled data is available in practical applications, semisupervised learning has attracted much attention in recent years [25–27]. One of our future works is to extend our approach under the semisupervised learning framework. On the other hand, DSLFDA needs the local within-class scatter matrix positive definite. We add an identity matrix to the local within-class scatter matrix for regularization. This may motivate us to find the regularization method to approximate the local within-class scatter matrix well.



---

*Source: 636928-2015-03-26.xml*