# Robust Face Recognition via Block Sparse Bayesian Learning

Authors: Taiyong Li; Zhilin Zhang
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

Face recognition (FR) is an important task in pattern recognition and computer vision. Sparse
representation (SR) has been demonstrated to be a powerful framework for FR. In general, an
SR algorithm treats each face in a training dataset as a basis function and tries to find a sparse
representation of a test face under these basis functions. The sparse representation coefficients
then provide a recognition hint. Early SR algorithms are based on a basic sparse model. Recently,
it has been found that algorithms based on a block sparse model can achieve better recognition
rates. Based on this model, in this study, we use block sparse Bayesian learning (BSBL) to find
a sparse representation of a test face for recognition. BSBL is a recently proposed framework,
which has many advantages over existing block-sparse-model-based algorithms. Experimental
results on the Extended Yale B, the AR, and the CMU PIE face databases show that using BSBL
can achieve better recognition rates and higher robustness than state-of-the-art algorithms in most
cases.


---

## Body

## 1. Introduction

Owing to the rapid development of network and computer technologies, face recognition (FR) plays an important role in many applications, such as video surveillance, man-machine interface, and digital entertainment. Many methods of FR have been developed over the past two decades [1–5]. Basically, FR is a typical problem of classification.In a typical FR system, besides the face detection and face alignment, there are two main stages in the process of FR. One is feature extraction, which obtains a set of relevant information from a face image for further classification. Because of the huge size of face images, it is desired to extract features from each face image, which have lower dimensions and facilitate recognition. Lots of feature extraction methods have been proposed, such as PCA [3, 6], LPP [5], and LDA [7]. Another stage is classification, which builds a classification model and assigns a label to a test face image. There are many classification algorithms. Typical algorithms include nearest neighbor (NN) [8], nearest subspace (NS) [9], and support vector machine (SVM) [10].Recently, Wright et al. proposed a novel FR method called sparse representation classification (SRC) [11]. In this method, face images in the training set form a dictionary matrix (each face image is vectorized and forms a column of the dictionary matrix), and then a vectorized test face image is represented under this dictionary matrix. The representation coefficients provide hints for recognition. For example, if a test face image and a training face image belong to the same subject, then the representation coefficients of the vectorized test face image under the dictionary matrix are sparse (or compressive); that is, most coefficients are zero (or close to zero). For each class (i.e., the columns in the dictionary matrix which are associated with a subject), one can calculate the reconstruction error of the vectorized test face image using these columns and the associated representation coefficients. The class with the minimum reconstruction error suggests that test face image belongs to this class. More frequently, one uses a feature vector extracted from a face image, instead of the original vectorized face image, in this method. SRC is so robust that it can achieve good performance in occlusion and noise environments.Following the idea of SRC, a number of SRC-related recognition methods have been proposed. Gao et al. extended the basic SRC method to a kernel version [12]. Yang et al. proposed a face recognition method via sparse coding which is much more robust than SRC in occlusion, corruption, and disguise environments [13]. Some other works improved the basic SRC method using weighted sparse representations [14], Gabor feature-based sparse representations [15], dimensionality reduction [16], locally adaptive sparse representations [17], and supervised sparse representations [18].Recently, it is found that using algorithms based on a block sparse model [19], instead of the algorithms based on the basic sparse representation model, can achieve higher recognition rates in face recognition [20]. However, these algorithms ignore intrablock correlation in representation coefficients. The existence of intrablock correlation in representation coefficients results from the fact that training face images with the same class as a test face image are all correlated with the test face image, and thus the representation coefficients associated with the training face images are not independent. In sparse reconstruction scenarios, it is shown that exploiting the intrablock correlation can significantly improve algorithmic performance [21].In this study, we use block sparse Bayesian learning (BSBL) [21] to estimate the representation coefficients. BSBL has many advantages over existing block-sparse-model-based algorithms, especially the fact that it has the ability to exploit the intrablock correlation in representation coefficients for better algorithmic performance. Experimental results on the Extended Yale B, the AR, and the CMU PIE databases show that BSBL achieves better results than state-of-the-art SRC algorithms in most cases.The rest of this paper is organized as follows. Section2 gives a brief review of the original face recognition via sparse representation. Section 3 introduces sparse Bayesian learning. The block sparse Bayesian learning approach for face recognition is proposed in Section 4. Experimental results are reported in Section 5. Conclusion is drawn in the last section.


## 2. Related Work

### 2.1. Face Recognition via Sparse Representation

We first describe the basic SRC method [11] for face recognition. Given training faces of all K subjects, a dictionary matrix is formed as follows:
(1)Φ≜[Φ1,Φ2,…,ΦK],
where Φi=[vi,1,vi,2,…,vi,ni]∈ℝm×ni and vi,j is the jth face of the ith subject. (For simplicity, we describe vi,j as a vectorized face image. But in practice, vi,j is a feature vector extracted from the face image, as done in our experiments.) Then, a vectorized test face y∈ℝm×1 is represented under the dictionary matrix as follows:
(2)y=Φx=v1,1x1,1+⋯+v1,n1x1,n1+⋯+vi-1,ni-1xi-1,ni-1+vi,1xi,1+vi,2xi,2+⋯+vi,nixi,ni+vi+1,1xi+1,1+⋯+vK,nKxK,nK,
where x≜[x1,1,…,xi,1,…,xi,ni,…,xK,nK]T is the representation coefficient vector. In the basic SRC method, it is suggested that if the new test face y belongs to a subject in the training set, say the ith subject, then, under a sparsity constraint on x, only some of the coefficients xi,1,xi,2,…,xi,ni are significantly nonzero, while other coefficients, that is, xj,k(j≠i,∀k), are zero or close to zero.Mathematically, the previous idea can be described as the following sparse representation problem:(3)x^0=argminx∥x∥0s.t.y=Φx,
where ∥x∥0 counts the number of nonzero elements in the vector x. Once we have obtained the solution x^0, the class label of y can be found by
(4)i=argminj∥y-Φδj(x^0)∥2,
where δj(x):ℝn→ℝn is the characteristic function which maintains the elements of x associated with the jth class, while sets other elements of x to zero.However, finding the solution to (3) is NP-hard [22]. Recent theories in compressed sensing [23, 24] show that if the true solution is sparse enough, under some mild conditions, the solution can be found by solving the following convex-relaxation problem:
(5)x^1=argminx∥x∥1s.t.y=Φx.
Further, to deal with small dense model noise, the problem (5) can be changed to the following one:
(6)x^1=argminx∥x∥1s.t.∥y-Φx∥2≤ϵ,
where ϵ is a noise-tolerance constant. Many ℓ1-minimization algorithms can be used to find the solution to (5) or to (6), such as LASSO [25] and Basis Pursuit Denoising [26].In a practical face recognition problem, the coefficient vectorx^1 (or x^0) not only is sparse but also is block sparse. To see this, we can rewrite the sparse representation problem (2) as follows:
(7)y=Φx=[Φ1,Φ2,…,ΦK]x=∑j=1K‍Φjxj,
where xj∈ℝnj×1 is the coefficient vector associated with the jth class and x≜[x1T,…,xKT]T. When a test face y belongs to the jth class, ideally only elements in xj are significantly nonzero. In other words, only the block xj has significantly nonzero norm. Clearly, this is a canonical block sparse model [19, 27]. Many algorithms for the block sparse model can be used here. For example, in [20], it is suggested to use the following algorithm:
(8)x^2,1=argminx∑j=1K‍∥xj∥2s.t.∥y-Φx∥2≤ϵ.
This is a natural extension of basic ℓ1-minimization algorithms, which imposes ℓ2 norm on block elements and then ℓ1 norm over blocks. It has been shown that exploiting the block structure can largely improve the estimation quality of x^0 [27–29].However, one should note that when the test face belongs to thejth class, not only the representation coefficient block xj is a nonzero block, but also its elements are correlated in amplitude. The correlation arises because the faces of the jth class in the training set are all correlated with the test face, and thus the elements in xj are mutually dependent. It is shown that exploiting the correlation within blocks can further improve the estimation quality of x^0 [21, 30] than only exploiting the block structure.Therefore, in this study, we propose to use block sparse Bayesian learning (BSBL) [21] to estimate x^0 by exploiting the block structure and the correlation within blocks. In the next section, we first briefly introduce sparse Bayesian learning (SBL) and then introduce BSBL.


## 2.1. Face Recognition via Sparse Representation

We first describe the basic SRC method [11] for face recognition. Given training faces of all K subjects, a dictionary matrix is formed as follows:
(1)Φ≜[Φ1,Φ2,…,ΦK],
where Φi=[vi,1,vi,2,…,vi,ni]∈ℝm×ni and vi,j is the jth face of the ith subject. (For simplicity, we describe vi,j as a vectorized face image. But in practice, vi,j is a feature vector extracted from the face image, as done in our experiments.) Then, a vectorized test face y∈ℝm×1 is represented under the dictionary matrix as follows:
(2)y=Φx=v1,1x1,1+⋯+v1,n1x1,n1+⋯+vi-1,ni-1xi-1,ni-1+vi,1xi,1+vi,2xi,2+⋯+vi,nixi,ni+vi+1,1xi+1,1+⋯+vK,nKxK,nK,
where x≜[x1,1,…,xi,1,…,xi,ni,…,xK,nK]T is the representation coefficient vector. In the basic SRC method, it is suggested that if the new test face y belongs to a subject in the training set, say the ith subject, then, under a sparsity constraint on x, only some of the coefficients xi,1,xi,2,…,xi,ni are significantly nonzero, while other coefficients, that is, xj,k(j≠i,∀k), are zero or close to zero.Mathematically, the previous idea can be described as the following sparse representation problem:(3)x^0=argminx∥x∥0s.t.y=Φx,
where ∥x∥0 counts the number of nonzero elements in the vector x. Once we have obtained the solution x^0, the class label of y can be found by
(4)i=argminj∥y-Φδj(x^0)∥2,
where δj(x):ℝn→ℝn is the characteristic function which maintains the elements of x associated with the jth class, while sets other elements of x to zero.However, finding the solution to (3) is NP-hard [22]. Recent theories in compressed sensing [23, 24] show that if the true solution is sparse enough, under some mild conditions, the solution can be found by solving the following convex-relaxation problem:
(5)x^1=argminx∥x∥1s.t.y=Φx.
Further, to deal with small dense model noise, the problem (5) can be changed to the following one:
(6)x^1=argminx∥x∥1s.t.∥y-Φx∥2≤ϵ,
where ϵ is a noise-tolerance constant. Many ℓ1-minimization algorithms can be used to find the solution to (5) or to (6), such as LASSO [25] and Basis Pursuit Denoising [26].In a practical face recognition problem, the coefficient vectorx^1 (or x^0) not only is sparse but also is block sparse. To see this, we can rewrite the sparse representation problem (2) as follows:
(7)y=Φx=[Φ1,Φ2,…,ΦK]x=∑j=1K‍Φjxj,
where xj∈ℝnj×1 is the coefficient vector associated with the jth class and x≜[x1T,…,xKT]T. When a test face y belongs to the jth class, ideally only elements in xj are significantly nonzero. In other words, only the block xj has significantly nonzero norm. Clearly, this is a canonical block sparse model [19, 27]. Many algorithms for the block sparse model can be used here. For example, in [20], it is suggested to use the following algorithm:
(8)x^2,1=argminx∑j=1K‍∥xj∥2s.t.∥y-Φx∥2≤ϵ.
This is a natural extension of basic ℓ1-minimization algorithms, which imposes ℓ2 norm on block elements and then ℓ1 norm over blocks. It has been shown that exploiting the block structure can largely improve the estimation quality of x^0 [27–29].However, one should note that when the test face belongs to thejth class, not only the representation coefficient block xj is a nonzero block, but also its elements are correlated in amplitude. The correlation arises because the faces of the jth class in the training set are all correlated with the test face, and thus the elements in xj are mutually dependent. It is shown that exploiting the correlation within blocks can further improve the estimation quality of x^0 [21, 30] than only exploiting the block structure.Therefore, in this study, we propose to use block sparse Bayesian learning (BSBL) [21] to estimate x^0 by exploiting the block structure and the correlation within blocks. In the next section, we first briefly introduce sparse Bayesian learning (SBL) and then introduce BSBL.


## 3. SBL and BSBL

SBL [31] was initially proposed as a machine learning method. But later it has been shown to be a powerful method for sparse representation, sparse signal recovery, and compressed sensing.


### 3.1. Advantages of SBL

Compared to LASSO-type algorithms (such as the original LASSO algorithm, Basis Pursuit Denoising, Group Lasso, Group Basis Pursuit, and other algorithms based onℓ1-minimization), SBL has the following advantages [32, 33].(1)
Its recovery performance is robust to the characteristics of the matrixΦ, while most other algorithms are not. For example, it has been shown that when columns of Φ are highly coherent, SBL still maintains good performance, while algorithms such as LASSO have seriously degraded performance [34]. This advantage is very attractive to sparse representation and other applications, since in these applications the matrix Φ is not a random matrix and its columns are highly coherent.(2)
SBL has a number of desired advantages over many popular algorithms in terms of local and global convergence. It can be shown that SBL provides a sparser solution than Lasso-type algorithms. In particular, in noiseless situations and under certain conditions, the global minimum of SBL cost function is unique and corresponds to the true sparsest solution, while the global minimum of the cost function of LASSO-type algorithms is not necessarily the true sparsest solution [35]. These advantages imply that SBL is a better choice in feature selection via sparse representation [36].(3)
Recent works in SBL [21, 37] provide robust learning rules for automatically estimating values of its regularizer (related to noise variance) such that SBL algorithms can achieve good performance. In contrast, LASSO-type algorithms generally need users to choose values for such regularizer, which is often obtained by cross-validation. However, this takes lots of time for large-scale datasets, which is not convenient and even impossible in some scenarios.


### 3.2. Introduction to BSBL

BSBL [21] is an extension of the basic SBL framework, which exploits a block structure and intrablock correlation in the coefficient vector x. It is based on the assumption that x can be partitioned into K nonoverlapping blocks:
(9)x=[x1,1,…,x1,n1︸x1T,…,xK,1,…,xK,nK︸xKT]T.
Among these blocks, few blocks are nonzero. Then, each block xi∈ℝni×1 is assumed to satisfy a parameterized multivariate Gaussian distribution:
(10)p(xi;γi,Bi)~𝒩(0,γiBi),i=1,…,K
with the unknown parameters γi and Bi. Here, γi is a nonnegative parameter controlling the block-sparsity of x. When γi=0, the ith block becomes zero. During the learning procedure most γi tend to be zero, due to the mechanism of automatic relevance determination [31]. Thus, sparsity at the block level is encouraged. Bi∈ℝdi×di is a positive definite and symmetrical matrix, capturing the intrablock correlation of the ith block. Under the assumption that blocks are mutually uncorrelated, the prior of x is p(x;{γi,Bi}i)~𝒩(0,Σ0), where Σ0=diag{γ1B1,…,γKBK}. (Here Σ0=diag{γ1B1,…,γKBK} means that Σ0 is a block diagonal matrix, and its principal diagonal blocks are γ1B1,…,γKBK.) To avoid overfitting, all the Bi will be imposed by some constraints and their estimates will be further regularized. The model noise n≜y-Φx is assumed to satisfy p(n;λ)~𝒩(0,λI), where λ is a positive scalar to be estimated. Based on the previous probability models, one can obtain a closed-form posterior distribution. Therefore, the estimate of x can be obtained by using the Maximum-A-Posteriori (MAP) estimation, providing all the parameters λ,{γi,Bi}i=1K are estimated.To estimate the parametersλ,{γi,Bi}i=1K, one can use the type II maximum likelihood method [31, 38]. This is equivalent to minimizing the following cost function:
(11)ℒ(Θ)≜-2log∫‍p(y∣x;λ)p(x;{γi,Bi}i)dx=log|λI+ΦΣ0ΦT|+yT(λI+ΦΣ0ΦT)-1y,
where Θ denotes all the parameters; that is, Θ≜{λ,{γi,Bi}i=1K}. There are several optimization methods to minimize the cost function, such as the expectation-maximum method, and the bound-optimization method, and the duality method. This framework is called the BSBL framework.BSBL not only has the advantages of the basic SBL listed in Section3.1, but also has another two advantages.(1)
BSBL provides large flexibility to model and exploit correlation structure in signals, such as intrablock correlation [21, 30]. By exploiting the correlation structures, recovery performance is significantly improved.(2)
BSBL has the unique ability to find less-sparse [39] and nonsparse [30] true solutions with very small errors. (Note that for an underdetermined inverse problem, that is, y=Φx, where Φ∈ℝm×P is one matrix or a product of a sensing matrix and a dictionary matrix as used in compressed sensing, one cannot find the true solution without any error, if the true solution x is non-sparse (i.e., ∥x∥0>m).) This is attractive for practical use, since in practice the true solutions may not be very sparse, and existing sparse signal recovery algorithms generally fail in this case.Therefore, BSBL is promising for pattern recognition. In the following, we use BSBL for face recognition. Among a number of BSBL algorithms, we choose the bound-optimization-based BSBL algorithm [21], denoted by BSBL-BO. (The BSBL-BO code can be downloaded at http://dsp.ucsd.edu/~zhilin/BSBL.html.)


## 3.1. Advantages of SBL

Compared to LASSO-type algorithms (such as the original LASSO algorithm, Basis Pursuit Denoising, Group Lasso, Group Basis Pursuit, and other algorithms based onℓ1-minimization), SBL has the following advantages [32, 33].(1)
Its recovery performance is robust to the characteristics of the matrixΦ, while most other algorithms are not. For example, it has been shown that when columns of Φ are highly coherent, SBL still maintains good performance, while algorithms such as LASSO have seriously degraded performance [34]. This advantage is very attractive to sparse representation and other applications, since in these applications the matrix Φ is not a random matrix and its columns are highly coherent.(2)
SBL has a number of desired advantages over many popular algorithms in terms of local and global convergence. It can be shown that SBL provides a sparser solution than Lasso-type algorithms. In particular, in noiseless situations and under certain conditions, the global minimum of SBL cost function is unique and corresponds to the true sparsest solution, while the global minimum of the cost function of LASSO-type algorithms is not necessarily the true sparsest solution [35]. These advantages imply that SBL is a better choice in feature selection via sparse representation [36].(3)
Recent works in SBL [21, 37] provide robust learning rules for automatically estimating values of its regularizer (related to noise variance) such that SBL algorithms can achieve good performance. In contrast, LASSO-type algorithms generally need users to choose values for such regularizer, which is often obtained by cross-validation. However, this takes lots of time for large-scale datasets, which is not convenient and even impossible in some scenarios.


## 3.2. Introduction to BSBL

BSBL [21] is an extension of the basic SBL framework, which exploits a block structure and intrablock correlation in the coefficient vector x. It is based on the assumption that x can be partitioned into K nonoverlapping blocks:
(9)x=[x1,1,…,x1,n1︸x1T,…,xK,1,…,xK,nK︸xKT]T.
Among these blocks, few blocks are nonzero. Then, each block xi∈ℝni×1 is assumed to satisfy a parameterized multivariate Gaussian distribution:
(10)p(xi;γi,Bi)~𝒩(0,γiBi),i=1,…,K
with the unknown parameters γi and Bi. Here, γi is a nonnegative parameter controlling the block-sparsity of x. When γi=0, the ith block becomes zero. During the learning procedure most γi tend to be zero, due to the mechanism of automatic relevance determination [31]. Thus, sparsity at the block level is encouraged. Bi∈ℝdi×di is a positive definite and symmetrical matrix, capturing the intrablock correlation of the ith block. Under the assumption that blocks are mutually uncorrelated, the prior of x is p(x;{γi,Bi}i)~𝒩(0,Σ0), where Σ0=diag{γ1B1,…,γKBK}. (Here Σ0=diag{γ1B1,…,γKBK} means that Σ0 is a block diagonal matrix, and its principal diagonal blocks are γ1B1,…,γKBK.) To avoid overfitting, all the Bi will be imposed by some constraints and their estimates will be further regularized. The model noise n≜y-Φx is assumed to satisfy p(n;λ)~𝒩(0,λI), where λ is a positive scalar to be estimated. Based on the previous probability models, one can obtain a closed-form posterior distribution. Therefore, the estimate of x can be obtained by using the Maximum-A-Posteriori (MAP) estimation, providing all the parameters λ,{γi,Bi}i=1K are estimated.To estimate the parametersλ,{γi,Bi}i=1K, one can use the type II maximum likelihood method [31, 38]. This is equivalent to minimizing the following cost function:
(11)ℒ(Θ)≜-2log∫‍p(y∣x;λ)p(x;{γi,Bi}i)dx=log|λI+ΦΣ0ΦT|+yT(λI+ΦΣ0ΦT)-1y,
where Θ denotes all the parameters; that is, Θ≜{λ,{γi,Bi}i=1K}. There are several optimization methods to minimize the cost function, such as the expectation-maximum method, and the bound-optimization method, and the duality method. This framework is called the BSBL framework.BSBL not only has the advantages of the basic SBL listed in Section3.1, but also has another two advantages.(1)
BSBL provides large flexibility to model and exploit correlation structure in signals, such as intrablock correlation [21, 30]. By exploiting the correlation structures, recovery performance is significantly improved.(2)
BSBL has the unique ability to find less-sparse [39] and nonsparse [30] true solutions with very small errors. (Note that for an underdetermined inverse problem, that is, y=Φx, where Φ∈ℝm×P is one matrix or a product of a sensing matrix and a dictionary matrix as used in compressed sensing, one cannot find the true solution without any error, if the true solution x is non-sparse (i.e., ∥x∥0>m).) This is attractive for practical use, since in practice the true solutions may not be very sparse, and existing sparse signal recovery algorithms generally fail in this case.Therefore, BSBL is promising for pattern recognition. In the following, we use BSBL for face recognition. Among a number of BSBL algorithms, we choose the bound-optimization-based BSBL algorithm [21], denoted by BSBL-BO. (The BSBL-BO code can be downloaded at http://dsp.ucsd.edu/~zhilin/BSBL.html.)


## 4. Face Recognition via BSBL

As stated in Section2, we use BSBL-BO to estimate x^0, denoted by x^BSBL, and then use rule (4) to assign a test face y to a class.In practice, a test facey may contain some outliers; that is, y=y0+ϵ, where y0 is the outlier-free face image and ϵ is a vector whose each entry is an outlier. Generally, the number of outliers is small, and thus ϵ is sparse. Addressing the outlier issue is important to a practical face recognition system. In [11], an augmented sparse model was used to deal with this issue. We now extend this method to our block sparse model and use BSBL-BO to estimate the solution. In particular, we adopt the following augmented block sparse model:
(12)y=y0+ϵ=Φx+n+ϵ=[Φ,I][xT,ϵT]T+n=Φ¯x¯+n,
where n is a vector modeling dense Gaussian noise, Φ¯≜[Φ,I], and x¯≜[xT,ϵT]T. Here, I is an identity matrix of the dimension m×m. Clearly, x¯ is also a block sparse vector, whose first K blocks are the blocks of x and last m elements are m blocks with the block size being 1. (In experiments we found that treating the m elements as one big block resulted in similar performance, while significantly sped up the algorithm.) Thus, (12) is still a block sparse model and can be solved by BSBL-BO. Once BSBL-BO obtains the solution, denoted by x^BSBLϵ, its first K blocks (denoted by x^BSBL) and its last m elements (denoted by ϵ^) are used to assign y to a class according to
(13)i=argminj∥y-ϵ^-Φδj(x^BSBL)∥2.We now take the Extended Yale B database [40] as an example to show how our method works. As shown in SRC [11], we randomly select half of the total 2414 faces (i.e., 1207 faces) as the training set and the rest as the testing set. Each face is downsampled from 192 × 168 to 24 × 21 = 504. The training set contains 38 subjects. Each subject has about 32 faces. Therefore, in our model K=38 and n1≈⋯≈nK≈32. The matrix Φ has the size 504×1207, and thus the matrix Φ¯ has the size 504×1711.The procedure is illustrated in Figure1. Figure 1(a) shows that a test face (belonging to Subject 4) can be linearly combined by a few training faces. Most of the coefficients estimated by BSBL-BO (i.e., x^BSBL) are zero or near zero and only those associated with the test face are significantly nonzero. Figure 1(b) shows the residuals ∥y-Φδj(x^BSBL)∥2 for j=1,…,38. The residual at j=4 is 0.0008, while the residuals at j≠4 are all close to 1, which makes it easy to assign the test face to Subject 4. See Section 5.1.1 for more details.Face recognition via BSBL. (a) Recognition with 24× 21 downsampled faces as features. The left picture shows a test face image and the downsampled face. The right picture shows the estimated coefficients x^BSBL. The test face belongs to Subject 4, and thus the representation coefficients associated with the downsampled training faces of the subject, that is, Φ4, have large values. Two training faces (and their downsampled faces) associated with the two largest coefficients are plotted. The bars near the top and the bottom of the box indicate the blocks in the coefficient vector. (b) The residuals ∥y-Φδj(x^BSBL)∥2 for j=1,…,38.
(a)
(b)


## 5. Experimental Results

To demonstrate the superior performance of BSBL, we performed experiments on three widely used face databases: Extended Yale B [40], AR [41], and CMU PIE [42] face databases. The face images of these three databases were captured under varying lighting, pose, or facial expression. The AR database also has occluded face images for the test of robustness of face recognition algorithms. Section 5.1 shows experimental results on face images without occlusion, and Section 5.2 shows experimental results on face images with three kinds of occlusion.


### 5.1. Face Recognition without Occlusion

For the experiments on face images without occlusion, we used downsampling, Eigenfaces [3, 6], and Laplacianfaces [5] to reduce the dimensionality of original faces. We compared our method with three classical methods, including nearest neighbor (NN) [8], nearest subspace (NS) [9], and support vector machine (SVM) [10]. We also compared our method with recently proposed sparse-representation-based classification methods, including the basic sparse-representation classifier (SRC) [11] and the block-sparse recovery algorithm via convex optimization (BSCO) [20]. For NS, the subspace dimension was fixed to 9. For BSCO, we used the Pℓ2/ℓ1′ algorithm [20] which has been shown to be the best one among all the structured sparsity-based classifiers proposed in that work.


#### 5.1.1. Extended Yale B Database

The Extended Yale B database [40] consists of 2414 frontal-face images of 38 subjects (each subject has about 64 images). In the experiment, we used the cropped 192×168 face images which were captured under various lighting conditions [43]. Two subjects are shown in Figure 2 for illustration (for each subject, only 10 face images are shown). We randomly selected half face images of each subject as the training set and the rest as the testing set. We used downsampling, Eigenfaces, and Laplacianfaces to extract features from face images. The dimensions of extracted features were 30, 56, 120, and 504, respectively.Figure 2
Sample face images of 2 individuals from the Extended Yale B database. First row: ten sample face images of the first subject. Second row: ten sample face images of the third subject.Experimental results are shown in Figure3, where we can see our method uniformly outperformed other algorithms regardless of used features. Particularly, our method had better performance when using Laplacianfaces. The superiority of our method was much clearer when the feature dimension was smaller and Laplacianfaces were used. For example, when the feature dimension was 56, our method achieved the highest rate of 98.9%, while NN, NS, SVM, SRC, and BSCO achieved the rates of 83.5%, 90.4%, 85.0%, 91.7%, and 79.4%, respectively. Higher performance using low dimensional features is attractive for recognition, since lower feature dimension generally implies that the computational load is accordingly lower.Comparison of recognition rates on Extended Yale B database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)


#### 5.1.2. AR Database

The AR database [41] consists of more than 4000 front-face images of 126 human subjects. Each subject has 26 images in two separated sessions, as shown in Figure 4. This database includes more facial expression and facial disguise. We chose 100 subjects (50 males and 50 females) in this experiment. For each subject, seven face images with different illumination and facial expression (i.e., the first 7 images of each subject) in Session 1 were selected for training and the first 7 images of each subject in Session 2 for testing. All the images were converted to gray mode and were resized to 165×120. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with the dimensions of 30, 54, 130, and 540. Experimental results are shown in Figure 5.Figure 4
Face images of two individuals from AR database. The first row: face images of the first male subject in Session 1. The second row: face images of the first male subject in Session 2. The third row: face images of the first female subject in Session 1. The forth row: face images of the first female subject in Session 2.Comparison of recognition rates on AR database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure5(a), we can see that our algorithm significantly outperformed other classifiers when using downsampled features. However, our method did not achieve the highest rate when using Eigenfaces and Laplacianfaces. This might be due to the small block size in this experiment (n1=n2=⋯=n100=7). Although our method did not uniformly outperform other algorithms when using different face features, the recognition rate achieved by our method using downsampled faces (96.7%) was not exceeded by other algorithms using any face features.


#### 5.1.3. CMU PIE Database

The CMU PIE database [42] consists of 41368 front-face images of 68 human subjects under different poses, illumination, and expressions. We chose one subset (C29) which included 1632 face images of 68 subjects (24 images for each subject) for this experiment. The first subject in this subset is shown in Figure 6, which varies in pose, illumination, and expression. All the images were cropped and resized to 64×64. For each subject, we randomly selected 10 images for training and the rest (14 images for each subject) for testing. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with four dimensions, that is, 36, 64, 144, and 256. Experimental results are shown in Figure 7.Figure 6
Face images of the first individual from CMU PIE (C29) database.Comparison of recognition rates on CMU PIE database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure7(a), we can see that sparse-representation-based classifiers usually outperformed classical ones in this dataset. Among the sparse-representation-based classifiers, BSBL and BSCO achieved higher recognition rates than SRC. For BSBL and BSCO, BSBL slightly outperformed BSCO with downsampled faces and Laplacianfaces, while BSCO outperformed BSBL with Eigenfaces. Specifically, for each feature space, BSBL achieved the highest recognition rate of 95.80% with downsampled faces, and 94.12% with Laplacianfaces, while BSCO achieved 98.42% with Eigenfaces, which was the highest one in this experiment. Nevertheless, BSBL outperformed BSCO in 8 out of 12 different combinations of dimensions and features.


### 5.2. Face Recognition with Occlusion

For the experiments on face images with occlusion, we used downsampling to reduce the size of face images and compared our method with NN [8], SRC [11], and BSCO [20].


#### 5.2.1. Face Recognition with Pixel Corruption

We tested face recognition with pixel corruption on 3 subsets of the Extended Yale B database: 719 face images with normal-to-moderate lighting conditions from Subsets 1 and 2 for training and 455 face images with more extreme lighting conditions from Subset 3 for testing. For each test image, we first replaced a certain percentage (0%–50%) of its original pixels with uniformly distributed gray values in[0,255]. Both the gray values and the locations were random and hence unknown to the algorithms. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two corrupted face images were shown in Figures 8(a)-8(b).Corrupted and occluded face images. (a) 10% of pixels were corrupted. (b) 50% of pixels were corrupted. (c) 10% of block was occluded. (d) 50% of block was occluded.
(a)
(b)
(c)
(d)Results are shown in Table1. It can be seen that in almost all combinational situations of dimension and corruption, BSBL achieved the highest recognition rate when compared with NN and SRC, and the performance gap between our algorithm and the compared algorithms was very large. For example, when the dimension was 12×10 and 50% pixels were corrupted, BSBL achieved the recognition rate of 67.25%, while SRC only had a recognition rate of 46.37%. Meanwhile, BSBL outperformed BSCO in 17 out of 24 combinational situations of dimension and corruption. Figure 9(a) shows the recognition rates of the four algorithms at different pixel corruption levels with the dimension of 12×10.Table 1
Recognition rate on faces with pixel corruption (%).
Method
Dimension
Percent corrupted (%)
0
10
20
30
40
50
NN
6
×
5
36.92
42.42
49.67
46.15
28.35
14.95
8
×
7
48.79
54.95
60.00
59.34
40.88
20.22
12
×
10
67.25
75.17
79.56
74.73
58.02
35.39
24
×
21
87.25
93.19
94.95
92.53
76.48
56.04
SRC
6
×
5
54.51
44.62
50.55
46.59
32.75
21.76
8
×
7
82.64
61.32
66.59
63.52
49.23
28.79
12
×
10
98.02
85.06
85.28
83.96
71.87
46.37
24
×
21
100.00
98.24
98.24
97.14
92.09
73.63
BSCO
6
×
5
87.48
83.52
63.30
39.12
23.30
14.29
8
×
7
98.02
96.04
90.99
75.60
49.45
30.33
12
×
10
99.34
99.12
97.58
93.85
82.20
58.68
24
×
21
100.00
100.00
99.12
98.68
97.14
96.70
BSBL
6
×
5
87.25
85.71
68.79
51.43
30.99
19.56
8
×
7
94.29
92.97
86.15
72.53
59.12
39.34
12
×
10
99.56
99.34
97.80
92.31
84.18
67.25
24
×
21
100.00
100.00
99.78
99.12
97.58
89.01Comparison of recognition rates on faces with different percentage of (a) pixel corruption and (b) block occlusion when face dimension was12×20.
(a)
(b)


#### 5.2.2. Face Recognition with Block Occlusion

In this experiment, we used the same training and testing images as those in the previous pixel corruption experiment. For each test image, we replaced a randomly located square block with an unrelated image (the baboon image in SRC [11]), which occluded 0%–50% of the original testing image. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two occluded face images were shown in Figures 8(c)-8(d).Table2 shows the recognition rates of NN, SRC, BSCO, and BSBL on different dimensions and percentages of occlusion. Again, BSBL outperformed the compared algorithms in most cases. For example, when the occlusion percentage ranged from 10% to 50% and the face dimension was 12×10, BSBL achieved about 8.35%–13.19% higher recognition rate than BSCO, as shown in Figure 9(b).Table 2
Recognition rate on faces with block occlusion (%).
Method
Dimension
Percent occluded (%)
0
10
20
30
40
50
NN
6
×
5
36.92
34.29
27.69
24.40
20.44
15.17
8
×
7
48.79
44.84
38.68
32.09
21.54
18.46
12
×
10
67.25
64.18
52.09
45.71
30.33
22.64
24
×
21
87.25
85.50
76.92
67.25
52.31
37.14
SRC
6
×
5
54.51
36.26
28.13
22.64
17.36
14.29
8
×
7
82.64
50.99
39.56
31.65
20.66
17.36
12
×
10
98.02
75.39
59.78
48.57
30.33
20.88
24
×
21
100.00
96.48
89.23
72.31
54.29
35.17
BSCO
6
×
5
87.48
30.11
14.51
8.57
5.49
3.96
8
×
7
98.02
51.65
32.53
18.02
13.41
8.35
12
×
10
99.34
71.87
49.23
32.09
20.88
14.73
24
×
21
100.00
99.56
92.97
80.88
63.74
45.93
BSBL
6
×
5
87.25
46.59
28.35
18.68
11.65
10.55
8
×
7
94.29
66.59
40.88
26.59
20.22
12.53
12
×
10
99.56
83.30
60.00
45.28
33.19
23.08
24
×
21
100.00
96.92
92.31
75.60
56.48
42.64


#### 5.2.3. Face Recognition with Real Face Disguise

We used a subset of AR database to test the performance of our method on face recognition with disguise. We chose 799 images of various facial expressions without occlusion (i.e., the first 4 face images in each session except a corrupted image named “W-027-14.bmp”) for training. We formed two separate testing sets of 200 images. The images in the first set were from the neutral expression with sunglasses (the 8th image in each session) which cover roughly 20% of the face, while the ones in the second set were from the neutral expression with scarves (the 11th image in each session) which cover roughly 40% of the face. All the images were resized to9×6, 13×10, 27×20, and 42×30, respectively.Results are shown in Table3. In the case of neutral expression with sunglasses, both SRC and NN achieved higher recognition rates than BSCO and BSBL. However, in the case of neutral expression with scarves, BSBL outperformed NN, SRC, and BSCO significantly. Totally, BSBL achieved the highest recognition rates 72.50% and 74.5% with the dimensions of 27×20 and 42×30, respectively, for the two testing sets, while SRC achieved the highest recognition rates 28.25% and 44.00% with the other two dimensions.Table 3
Recognition rate on faces with disguise (%).
Method
Dimension
Sunglasses
Scarves
Total
NN
9
×
6
35.00
6.50
20.75
13
×
10
48.00
7.00
27.50
27
×
20
65.50
9.50
37.50
42
×
30
68.00
11.50
39.75
SRC
9
×
6
46.50
10.00
28.25
13
×
10
72.00
16.00
44.00
27
×
20
83.00
21.50
52.25
42
×
30
89.00
37.00
63.00
BSCO
9
×
6
14.50
9.50
12.00
13
×
10
35.00
19.50
27.25
27
×
20
68.00
44.00
56.00
42
×
30
76.00
50.00
63.00
BSBL
9
×
6
22.00
23.00
22.50
13
×
10
40.50
46.00
43.25
27
×
20
64.00
81.00
72.50
42
×
30
65.50
83.50
74.50


## 5.1. Face Recognition without Occlusion

For the experiments on face images without occlusion, we used downsampling, Eigenfaces [3, 6], and Laplacianfaces [5] to reduce the dimensionality of original faces. We compared our method with three classical methods, including nearest neighbor (NN) [8], nearest subspace (NS) [9], and support vector machine (SVM) [10]. We also compared our method with recently proposed sparse-representation-based classification methods, including the basic sparse-representation classifier (SRC) [11] and the block-sparse recovery algorithm via convex optimization (BSCO) [20]. For NS, the subspace dimension was fixed to 9. For BSCO, we used the Pℓ2/ℓ1′ algorithm [20] which has been shown to be the best one among all the structured sparsity-based classifiers proposed in that work.


### 5.1.1. Extended Yale B Database

The Extended Yale B database [40] consists of 2414 frontal-face images of 38 subjects (each subject has about 64 images). In the experiment, we used the cropped 192×168 face images which were captured under various lighting conditions [43]. Two subjects are shown in Figure 2 for illustration (for each subject, only 10 face images are shown). We randomly selected half face images of each subject as the training set and the rest as the testing set. We used downsampling, Eigenfaces, and Laplacianfaces to extract features from face images. The dimensions of extracted features were 30, 56, 120, and 504, respectively.Figure 2
Sample face images of 2 individuals from the Extended Yale B database. First row: ten sample face images of the first subject. Second row: ten sample face images of the third subject.Experimental results are shown in Figure3, where we can see our method uniformly outperformed other algorithms regardless of used features. Particularly, our method had better performance when using Laplacianfaces. The superiority of our method was much clearer when the feature dimension was smaller and Laplacianfaces were used. For example, when the feature dimension was 56, our method achieved the highest rate of 98.9%, while NN, NS, SVM, SRC, and BSCO achieved the rates of 83.5%, 90.4%, 85.0%, 91.7%, and 79.4%, respectively. Higher performance using low dimensional features is attractive for recognition, since lower feature dimension generally implies that the computational load is accordingly lower.Comparison of recognition rates on Extended Yale B database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)


### 5.1.2. AR Database

The AR database [41] consists of more than 4000 front-face images of 126 human subjects. Each subject has 26 images in two separated sessions, as shown in Figure 4. This database includes more facial expression and facial disguise. We chose 100 subjects (50 males and 50 females) in this experiment. For each subject, seven face images with different illumination and facial expression (i.e., the first 7 images of each subject) in Session 1 were selected for training and the first 7 images of each subject in Session 2 for testing. All the images were converted to gray mode and were resized to 165×120. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with the dimensions of 30, 54, 130, and 540. Experimental results are shown in Figure 5.Figure 4
Face images of two individuals from AR database. The first row: face images of the first male subject in Session 1. The second row: face images of the first male subject in Session 2. The third row: face images of the first female subject in Session 1. The forth row: face images of the first female subject in Session 2.Comparison of recognition rates on AR database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure5(a), we can see that our algorithm significantly outperformed other classifiers when using downsampled features. However, our method did not achieve the highest rate when using Eigenfaces and Laplacianfaces. This might be due to the small block size in this experiment (n1=n2=⋯=n100=7). Although our method did not uniformly outperform other algorithms when using different face features, the recognition rate achieved by our method using downsampled faces (96.7%) was not exceeded by other algorithms using any face features.


### 5.1.3. CMU PIE Database

The CMU PIE database [42] consists of 41368 front-face images of 68 human subjects under different poses, illumination, and expressions. We chose one subset (C29) which included 1632 face images of 68 subjects (24 images for each subject) for this experiment. The first subject in this subset is shown in Figure 6, which varies in pose, illumination, and expression. All the images were cropped and resized to 64×64. For each subject, we randomly selected 10 images for training and the rest (14 images for each subject) for testing. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with four dimensions, that is, 36, 64, 144, and 256. Experimental results are shown in Figure 7.Figure 6
Face images of the first individual from CMU PIE (C29) database.Comparison of recognition rates on CMU PIE database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure7(a), we can see that sparse-representation-based classifiers usually outperformed classical ones in this dataset. Among the sparse-representation-based classifiers, BSBL and BSCO achieved higher recognition rates than SRC. For BSBL and BSCO, BSBL slightly outperformed BSCO with downsampled faces and Laplacianfaces, while BSCO outperformed BSBL with Eigenfaces. Specifically, for each feature space, BSBL achieved the highest recognition rate of 95.80% with downsampled faces, and 94.12% with Laplacianfaces, while BSCO achieved 98.42% with Eigenfaces, which was the highest one in this experiment. Nevertheless, BSBL outperformed BSCO in 8 out of 12 different combinations of dimensions and features.


## 5.1.1. Extended Yale B Database

The Extended Yale B database [40] consists of 2414 frontal-face images of 38 subjects (each subject has about 64 images). In the experiment, we used the cropped 192×168 face images which were captured under various lighting conditions [43]. Two subjects are shown in Figure 2 for illustration (for each subject, only 10 face images are shown). We randomly selected half face images of each subject as the training set and the rest as the testing set. We used downsampling, Eigenfaces, and Laplacianfaces to extract features from face images. The dimensions of extracted features were 30, 56, 120, and 504, respectively.Figure 2
Sample face images of 2 individuals from the Extended Yale B database. First row: ten sample face images of the first subject. Second row: ten sample face images of the third subject.Experimental results are shown in Figure3, where we can see our method uniformly outperformed other algorithms regardless of used features. Particularly, our method had better performance when using Laplacianfaces. The superiority of our method was much clearer when the feature dimension was smaller and Laplacianfaces were used. For example, when the feature dimension was 56, our method achieved the highest rate of 98.9%, while NN, NS, SVM, SRC, and BSCO achieved the rates of 83.5%, 90.4%, 85.0%, 91.7%, and 79.4%, respectively. Higher performance using low dimensional features is attractive for recognition, since lower feature dimension generally implies that the computational load is accordingly lower.Comparison of recognition rates on Extended Yale B database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)


## 5.1.2. AR Database

The AR database [41] consists of more than 4000 front-face images of 126 human subjects. Each subject has 26 images in two separated sessions, as shown in Figure 4. This database includes more facial expression and facial disguise. We chose 100 subjects (50 males and 50 females) in this experiment. For each subject, seven face images with different illumination and facial expression (i.e., the first 7 images of each subject) in Session 1 were selected for training and the first 7 images of each subject in Session 2 for testing. All the images were converted to gray mode and were resized to 165×120. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with the dimensions of 30, 54, 130, and 540. Experimental results are shown in Figure 5.Figure 4
Face images of two individuals from AR database. The first row: face images of the first male subject in Session 1. The second row: face images of the first male subject in Session 2. The third row: face images of the first female subject in Session 1. The forth row: face images of the first female subject in Session 2.Comparison of recognition rates on AR database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure5(a), we can see that our algorithm significantly outperformed other classifiers when using downsampled features. However, our method did not achieve the highest rate when using Eigenfaces and Laplacianfaces. This might be due to the small block size in this experiment (n1=n2=⋯=n100=7). Although our method did not uniformly outperform other algorithms when using different face features, the recognition rate achieved by our method using downsampled faces (96.7%) was not exceeded by other algorithms using any face features.


## 5.1.3. CMU PIE Database

The CMU PIE database [42] consists of 41368 front-face images of 68 human subjects under different poses, illumination, and expressions. We chose one subset (C29) which included 1632 face images of 68 subjects (24 images for each subject) for this experiment. The first subject in this subset is shown in Figure 6, which varies in pose, illumination, and expression. All the images were cropped and resized to 64×64. For each subject, we randomly selected 10 images for training and the rest (14 images for each subject) for testing. Downsampled faces, Eigenfaces, and Laplacianfaces were applied with four dimensions, that is, 36, 64, 144, and 256. Experimental results are shown in Figure 7.Figure 6
Face images of the first individual from CMU PIE (C29) database.Comparison of recognition rates on CMU PIE database when using different face features. (a) Downsampling faces. (b) Eigenfaces. (c) Laplacianfaces.
(a)
(b)
(c)From Figure7(a), we can see that sparse-representation-based classifiers usually outperformed classical ones in this dataset. Among the sparse-representation-based classifiers, BSBL and BSCO achieved higher recognition rates than SRC. For BSBL and BSCO, BSBL slightly outperformed BSCO with downsampled faces and Laplacianfaces, while BSCO outperformed BSBL with Eigenfaces. Specifically, for each feature space, BSBL achieved the highest recognition rate of 95.80% with downsampled faces, and 94.12% with Laplacianfaces, while BSCO achieved 98.42% with Eigenfaces, which was the highest one in this experiment. Nevertheless, BSBL outperformed BSCO in 8 out of 12 different combinations of dimensions and features.


## 5.2. Face Recognition with Occlusion

For the experiments on face images with occlusion, we used downsampling to reduce the size of face images and compared our method with NN [8], SRC [11], and BSCO [20].


### 5.2.1. Face Recognition with Pixel Corruption

We tested face recognition with pixel corruption on 3 subsets of the Extended Yale B database: 719 face images with normal-to-moderate lighting conditions from Subsets 1 and 2 for training and 455 face images with more extreme lighting conditions from Subset 3 for testing. For each test image, we first replaced a certain percentage (0%–50%) of its original pixels with uniformly distributed gray values in[0,255]. Both the gray values and the locations were random and hence unknown to the algorithms. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two corrupted face images were shown in Figures 8(a)-8(b).Corrupted and occluded face images. (a) 10% of pixels were corrupted. (b) 50% of pixels were corrupted. (c) 10% of block was occluded. (d) 50% of block was occluded.
(a)
(b)
(c)
(d)Results are shown in Table1. It can be seen that in almost all combinational situations of dimension and corruption, BSBL achieved the highest recognition rate when compared with NN and SRC, and the performance gap between our algorithm and the compared algorithms was very large. For example, when the dimension was 12×10 and 50% pixels were corrupted, BSBL achieved the recognition rate of 67.25%, while SRC only had a recognition rate of 46.37%. Meanwhile, BSBL outperformed BSCO in 17 out of 24 combinational situations of dimension and corruption. Figure 9(a) shows the recognition rates of the four algorithms at different pixel corruption levels with the dimension of 12×10.Table 1
Recognition rate on faces with pixel corruption (%).
Method
Dimension
Percent corrupted (%)
0
10
20
30
40
50
NN
6
×
5
36.92
42.42
49.67
46.15
28.35
14.95
8
×
7
48.79
54.95
60.00
59.34
40.88
20.22
12
×
10
67.25
75.17
79.56
74.73
58.02
35.39
24
×
21
87.25
93.19
94.95
92.53
76.48
56.04
SRC
6
×
5
54.51
44.62
50.55
46.59
32.75
21.76
8
×
7
82.64
61.32
66.59
63.52
49.23
28.79
12
×
10
98.02
85.06
85.28
83.96
71.87
46.37
24
×
21
100.00
98.24
98.24
97.14
92.09
73.63
BSCO
6
×
5
87.48
83.52
63.30
39.12
23.30
14.29
8
×
7
98.02
96.04
90.99
75.60
49.45
30.33
12
×
10
99.34
99.12
97.58
93.85
82.20
58.68
24
×
21
100.00
100.00
99.12
98.68
97.14
96.70
BSBL
6
×
5
87.25
85.71
68.79
51.43
30.99
19.56
8
×
7
94.29
92.97
86.15
72.53
59.12
39.34
12
×
10
99.56
99.34
97.80
92.31
84.18
67.25
24
×
21
100.00
100.00
99.78
99.12
97.58
89.01Comparison of recognition rates on faces with different percentage of (a) pixel corruption and (b) block occlusion when face dimension was12×20.
(a)
(b)


### 5.2.2. Face Recognition with Block Occlusion

In this experiment, we used the same training and testing images as those in the previous pixel corruption experiment. For each test image, we replaced a randomly located square block with an unrelated image (the baboon image in SRC [11]), which occluded 0%–50% of the original testing image. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two occluded face images were shown in Figures 8(c)-8(d).Table2 shows the recognition rates of NN, SRC, BSCO, and BSBL on different dimensions and percentages of occlusion. Again, BSBL outperformed the compared algorithms in most cases. For example, when the occlusion percentage ranged from 10% to 50% and the face dimension was 12×10, BSBL achieved about 8.35%–13.19% higher recognition rate than BSCO, as shown in Figure 9(b).Table 2
Recognition rate on faces with block occlusion (%).
Method
Dimension
Percent occluded (%)
0
10
20
30
40
50
NN
6
×
5
36.92
34.29
27.69
24.40
20.44
15.17
8
×
7
48.79
44.84
38.68
32.09
21.54
18.46
12
×
10
67.25
64.18
52.09
45.71
30.33
22.64
24
×
21
87.25
85.50
76.92
67.25
52.31
37.14
SRC
6
×
5
54.51
36.26
28.13
22.64
17.36
14.29
8
×
7
82.64
50.99
39.56
31.65
20.66
17.36
12
×
10
98.02
75.39
59.78
48.57
30.33
20.88
24
×
21
100.00
96.48
89.23
72.31
54.29
35.17
BSCO
6
×
5
87.48
30.11
14.51
8.57
5.49
3.96
8
×
7
98.02
51.65
32.53
18.02
13.41
8.35
12
×
10
99.34
71.87
49.23
32.09
20.88
14.73
24
×
21
100.00
99.56
92.97
80.88
63.74
45.93
BSBL
6
×
5
87.25
46.59
28.35
18.68
11.65
10.55
8
×
7
94.29
66.59
40.88
26.59
20.22
12.53
12
×
10
99.56
83.30
60.00
45.28
33.19
23.08
24
×
21
100.00
96.92
92.31
75.60
56.48
42.64


### 5.2.3. Face Recognition with Real Face Disguise

We used a subset of AR database to test the performance of our method on face recognition with disguise. We chose 799 images of various facial expressions without occlusion (i.e., the first 4 face images in each session except a corrupted image named “W-027-14.bmp”) for training. We formed two separate testing sets of 200 images. The images in the first set were from the neutral expression with sunglasses (the 8th image in each session) which cover roughly 20% of the face, while the ones in the second set were from the neutral expression with scarves (the 11th image in each session) which cover roughly 40% of the face. All the images were resized to9×6, 13×10, 27×20, and 42×30, respectively.Results are shown in Table3. In the case of neutral expression with sunglasses, both SRC and NN achieved higher recognition rates than BSCO and BSBL. However, in the case of neutral expression with scarves, BSBL outperformed NN, SRC, and BSCO significantly. Totally, BSBL achieved the highest recognition rates 72.50% and 74.5% with the dimensions of 27×20 and 42×30, respectively, for the two testing sets, while SRC achieved the highest recognition rates 28.25% and 44.00% with the other two dimensions.Table 3
Recognition rate on faces with disguise (%).
Method
Dimension
Sunglasses
Scarves
Total
NN
9
×
6
35.00
6.50
20.75
13
×
10
48.00
7.00
27.50
27
×
20
65.50
9.50
37.50
42
×
30
68.00
11.50
39.75
SRC
9
×
6
46.50
10.00
28.25
13
×
10
72.00
16.00
44.00
27
×
20
83.00
21.50
52.25
42
×
30
89.00
37.00
63.00
BSCO
9
×
6
14.50
9.50
12.00
13
×
10
35.00
19.50
27.25
27
×
20
68.00
44.00
56.00
42
×
30
76.00
50.00
63.00
BSBL
9
×
6
22.00
23.00
22.50
13
×
10
40.50
46.00
43.25
27
×
20
64.00
81.00
72.50
42
×
30
65.50
83.50
74.50


## 5.2.1. Face Recognition with Pixel Corruption

We tested face recognition with pixel corruption on 3 subsets of the Extended Yale B database: 719 face images with normal-to-moderate lighting conditions from Subsets 1 and 2 for training and 455 face images with more extreme lighting conditions from Subset 3 for testing. For each test image, we first replaced a certain percentage (0%–50%) of its original pixels with uniformly distributed gray values in[0,255]. Both the gray values and the locations were random and hence unknown to the algorithms. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two corrupted face images were shown in Figures 8(a)-8(b).Corrupted and occluded face images. (a) 10% of pixels were corrupted. (b) 50% of pixels were corrupted. (c) 10% of block was occluded. (d) 50% of block was occluded.
(a)
(b)
(c)
(d)Results are shown in Table1. It can be seen that in almost all combinational situations of dimension and corruption, BSBL achieved the highest recognition rate when compared with NN and SRC, and the performance gap between our algorithm and the compared algorithms was very large. For example, when the dimension was 12×10 and 50% pixels were corrupted, BSBL achieved the recognition rate of 67.25%, while SRC only had a recognition rate of 46.37%. Meanwhile, BSBL outperformed BSCO in 17 out of 24 combinational situations of dimension and corruption. Figure 9(a) shows the recognition rates of the four algorithms at different pixel corruption levels with the dimension of 12×10.Table 1
Recognition rate on faces with pixel corruption (%).
Method
Dimension
Percent corrupted (%)
0
10
20
30
40
50
NN
6
×
5
36.92
42.42
49.67
46.15
28.35
14.95
8
×
7
48.79
54.95
60.00
59.34
40.88
20.22
12
×
10
67.25
75.17
79.56
74.73
58.02
35.39
24
×
21
87.25
93.19
94.95
92.53
76.48
56.04
SRC
6
×
5
54.51
44.62
50.55
46.59
32.75
21.76
8
×
7
82.64
61.32
66.59
63.52
49.23
28.79
12
×
10
98.02
85.06
85.28
83.96
71.87
46.37
24
×
21
100.00
98.24
98.24
97.14
92.09
73.63
BSCO
6
×
5
87.48
83.52
63.30
39.12
23.30
14.29
8
×
7
98.02
96.04
90.99
75.60
49.45
30.33
12
×
10
99.34
99.12
97.58
93.85
82.20
58.68
24
×
21
100.00
100.00
99.12
98.68
97.14
96.70
BSBL
6
×
5
87.25
85.71
68.79
51.43
30.99
19.56
8
×
7
94.29
92.97
86.15
72.53
59.12
39.34
12
×
10
99.56
99.34
97.80
92.31
84.18
67.25
24
×
21
100.00
100.00
99.78
99.12
97.58
89.01Comparison of recognition rates on faces with different percentage of (a) pixel corruption and (b) block occlusion when face dimension was12×20.
(a)
(b)


## 5.2.2. Face Recognition with Block Occlusion

In this experiment, we used the same training and testing images as those in the previous pixel corruption experiment. For each test image, we replaced a randomly located square block with an unrelated image (the baboon image in SRC [11]), which occluded 0%–50% of the original testing image. We then downsampled all the images to the sizes of 6×5, 8×7, 12×10, and 24×21, respectively. Two occluded face images were shown in Figures 8(c)-8(d).Table2 shows the recognition rates of NN, SRC, BSCO, and BSBL on different dimensions and percentages of occlusion. Again, BSBL outperformed the compared algorithms in most cases. For example, when the occlusion percentage ranged from 10% to 50% and the face dimension was 12×10, BSBL achieved about 8.35%–13.19% higher recognition rate than BSCO, as shown in Figure 9(b).Table 2
Recognition rate on faces with block occlusion (%).
Method
Dimension
Percent occluded (%)
0
10
20
30
40
50
NN
6
×
5
36.92
34.29
27.69
24.40
20.44
15.17
8
×
7
48.79
44.84
38.68
32.09
21.54
18.46
12
×
10
67.25
64.18
52.09
45.71
30.33
22.64
24
×
21
87.25
85.50
76.92
67.25
52.31
37.14
SRC
6
×
5
54.51
36.26
28.13
22.64
17.36
14.29
8
×
7
82.64
50.99
39.56
31.65
20.66
17.36
12
×
10
98.02
75.39
59.78
48.57
30.33
20.88
24
×
21
100.00
96.48
89.23
72.31
54.29
35.17
BSCO
6
×
5
87.48
30.11
14.51
8.57
5.49
3.96
8
×
7
98.02
51.65
32.53
18.02
13.41
8.35
12
×
10
99.34
71.87
49.23
32.09
20.88
14.73
24
×
21
100.00
99.56
92.97
80.88
63.74
45.93
BSBL
6
×
5
87.25
46.59
28.35
18.68
11.65
10.55
8
×
7
94.29
66.59
40.88
26.59
20.22
12.53
12
×
10
99.56
83.30
60.00
45.28
33.19
23.08
24
×
21
100.00
96.92
92.31
75.60
56.48
42.64


## 5.2.3. Face Recognition with Real Face Disguise

We used a subset of AR database to test the performance of our method on face recognition with disguise. We chose 799 images of various facial expressions without occlusion (i.e., the first 4 face images in each session except a corrupted image named “W-027-14.bmp”) for training. We formed two separate testing sets of 200 images. The images in the first set were from the neutral expression with sunglasses (the 8th image in each session) which cover roughly 20% of the face, while the ones in the second set were from the neutral expression with scarves (the 11th image in each session) which cover roughly 40% of the face. All the images were resized to9×6, 13×10, 27×20, and 42×30, respectively.Results are shown in Table3. In the case of neutral expression with sunglasses, both SRC and NN achieved higher recognition rates than BSCO and BSBL. However, in the case of neutral expression with scarves, BSBL outperformed NN, SRC, and BSCO significantly. Totally, BSBL achieved the highest recognition rates 72.50% and 74.5% with the dimensions of 27×20 and 42×30, respectively, for the two testing sets, while SRC achieved the highest recognition rates 28.25% and 44.00% with the other two dimensions.Table 3
Recognition rate on faces with disguise (%).
Method
Dimension
Sunglasses
Scarves
Total
NN
9
×
6
35.00
6.50
20.75
13
×
10
48.00
7.00
27.50
27
×
20
65.50
9.50
37.50
42
×
30
68.00
11.50
39.75
SRC
9
×
6
46.50
10.00
28.25
13
×
10
72.00
16.00
44.00
27
×
20
83.00
21.50
52.25
42
×
30
89.00
37.00
63.00
BSCO
9
×
6
14.50
9.50
12.00
13
×
10
35.00
19.50
27.25
27
×
20
68.00
44.00
56.00
42
×
30
76.00
50.00
63.00
BSBL
9
×
6
22.00
23.00
22.50
13
×
10
40.50
46.00
43.25
27
×
20
64.00
81.00
72.50
42
×
30
65.50
83.50
74.50


## 6. Conclusions

Classification via sparse representation is a popular methodology in face recognition and other classification tasks. Recently, it was found that using block-sparse representation, instead of the basic sparse representation, can yield better classification performance. In this paper, by introducing a recently proposed block sparse Bayesian learning (BSBL) algorithm, we showed that the BSBL is a better framework than the basic block-sparse representation framework, due to its various advantages over the latter. Experiments on common face databases confirmed that the BSBL is a promising sparse-representation-based classifier.



---

*Source: 695976-2013-11-27.xml*