# Incremental Nonnegative Matrix Factorization  for Face Recognition

Authors: Wen-Sheng Chen; Binbin Pan; Bin Fang; Ming Li; Jianliang Tang
Journal: Mathematical Problems in Engineering
 (2008)

---

## Abstract

Nonnegative matrix factorization (NMF) is a promising approach for local feature extraction in face recognition tasks. However, there are two major drawbacks in almost all existing NMF-based methods. One shortcoming is that the computational cost is expensive for large matrix decomposition. The other is that it must conduct repetitive learning, when the training samples or classes are updated. To overcome these two limitations, this paper proposes a novel incremental nonnegative matrix factorization (INMF) for face representation and recognition. The proposed INMF approach is based on a novel constraint criterion and our previous block strategy. It thus has some good properties, such as low computational complexity, sparse coefficient matrix. Also, the coefficient column vectors between different classes are orthogonal. In particular, it can be applied to incremental learning. Two face databases, namely FERET and CMU PIE face databases, are selected for evaluation. Compared with PCA and some state-of-the-art NMF-based methods, our INMF approach gives the best performance.


---

## Body

## 1. Introduction

Face recognition has been one of the most challenging problems in
computer science and information technology since 1990 [1, 2]. The approaches of
face recognition can be mainly categorized into two groups, namely geometric
feature-based and appearance-based [3]. The geometric features
are based on the short range phenomena of face images such as eyes, eyebrows,
nose, and mouth. The facial local features are learnt to form a face geometric
feature vector for face recognition. The appearance-based approach relies on
the global facial features, which generate an entire facial feature vector for
face classification. Nonnegative matrix factorization (NMF) [4, 5] belongs to geometric
feature-based category, while principle component analysis (PCA) [6] is based on
the whole facial features. Both NMF and PCA are unsupervised learning methods
for face recognition. The basic ideas of these two approaches are to find the
basis images using different criterions. All face images can be reconstructed by
the basis images. The basis images of PCA are called eigenfaces, which are the eigenvectors
corresponding to large eigenvalues of total scatter matrix. NMF aims to perform
nonnegative matrix decomposition on the training image matrix V such that V≈WH, where W and H are the basis
image matrix and the coefficient matrix, respectively. The local image features
are learnt and contained in W as
column vectors. Follow the success of applying NMF in learning the parts of
objects [4], many researchers have conducted in-depth investigation on NMF and
different NMF-based approaches have been developed [7–19]. Li et al. proposed a local NMF method [7]
by adding some spatial constraints. Wild et al. [8] utilized spherical K-means clustering to produce a structured
initialization for NMF. Buciu and Pitas [9] presented a DNMF method for learning facial expressions in a supervised
manner. However, DNMF does not guarantee convergence to a stationary limit
point. Kotsia et al. [15] thus
presented a modified DNMF method using projected gradients. Some similar supervised
methods incorporated into NMF were developed to enhance the classification power
of NMF [11–13, 19]. Hoyer [10] added sparseness constraints to NMF to find
solutions with desired degrees of sparseness. Lin [16, 17] modified traditional NMF
updates using projected gradient method and discussed their convergences. Recently,
Zhang et al. [18] proposed a topology
structure preservation constraint in NMF to improve the NMF performance.However, to the best of
our knowledge, almost all existing NMF-based approaches encounter two major problems,
namely time-consuming problem and incremental learning problem. In most cases,
the training image matrixV is very
large and it leads to expensive computational cost for NMF-based schemes. Also,
when the training samples or classes are updated, NMF must implement repetitive
learning. These drawbacks greatly restrict the practical applications of NMF-based
methods to face recognition. To avoid the above two problems, this paper, motivated
by our previous work on incremental learning [19], proposes a supervised
incremental NMF (INMF) approach under a novel constraint NMF criterion, which aims
to cluster within class samples tightly and augment the between-class distance simultaneously.
Our incremental strategy utilizes the supervised local features, which are
considered as the short-range phenomena of face images, for face
classifications. Two public available face databases, namely FERET and CMU PIE face
databases, are selected for evaluation. Experimental results show that our INMF
method outperforms PCA [6], NMF [4], and BNMF [19] approaches in both
nonincremental learning and incremental learning of face recognition.The rest of this paper is
organized as follows: Section2 briefly reviews the related works. Theoretical analysis
and INMF algorithm design are given in Section 3. Experimental results are
reported in Section 4. Finally, Section 5 draws the conclusions.


## 2. Related Work

This
section briefly introduces PCA [6], NMF [4], and BNMF [19] methods.
Details are as follows.


### 2.1. PCA

Principal component analysis (PCA), also called eigenface method,
is a popular statistic appearance-based linear method for dimensionality
reduction in face recognition. The theory used in PCA is based on Karhunen-Loeve
transform. It performs the eigenvalue decomposition on the total scatter matrixSt and then selects the large
principal components (eigenfaces) to account for most distributions. All face
images can be expressed by the linear combinations of these basis images
(eigenfaces). However, PCA is not able to exploit all of the feature classification
information and how to choose the principal component elements is still a
problem. Therefore, PCA cannot give satisfactory performance in pattern
recognition tasks.


### 2.2. NMF

NMF aims to find
nonnegative matricesW and H such that (2.1)Vm×n≈NMFWm×rHr×n, where matrix V is also a nonnegative matrix generated
by total n training images. Each
column of W is called basis image,
while H is the coefficient matrix.
The basis number r is usually chosen
less than n for dimensionality
reduction. The divergence between V and WH is defined as (2.2)F=∑ij(Vijlog⁡Vij(WH)ij−Vij+(WH)ij).NMF (2.1) is equivalent to
the following optimization problem: (2.3)min⁡W,H⁢F,s.t.W≥0,H≥0,∑iWik=1,∀k.The minimization problem
(2.3) can be solved using the following iterative formulae, which converge to a
local minimum: (2.4)Wij⟵Wij∑kVik(WH)ikHjk,Wij⟵Wij∑kWkj,Hij⟵Hij∑kWkiVkj(WH)kj.


### 2.3. BNMF

The
basic idea of BNMF is to perform NMF onc small matrices V(i)∈ℝm×n0 (i=1,2,…,c), namely (2.5)(V(i))m×n0≈NMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c, where V(i) contains n0 training images of the ith class, and c is the
number of classes. BNMF is yielded from (2.5) as follows: (2.6)Vm×n≈BNMFWm×rHr×n, where r=cr0,Vm×n=[V(1)V(2)⋯V(c)],Wm×r=[W(1)W(2)⋯W(c)],Hr×n=diag(H(1),H(2),…,H(c)),
and n(=cn0) is the total number of training images.


## 2.1. PCA

Principal component analysis (PCA), also called eigenface method,
is a popular statistic appearance-based linear method for dimensionality
reduction in face recognition. The theory used in PCA is based on Karhunen-Loeve
transform. It performs the eigenvalue decomposition on the total scatter matrixSt and then selects the large
principal components (eigenfaces) to account for most distributions. All face
images can be expressed by the linear combinations of these basis images
(eigenfaces). However, PCA is not able to exploit all of the feature classification
information and how to choose the principal component elements is still a
problem. Therefore, PCA cannot give satisfactory performance in pattern
recognition tasks.


## 2.2. NMF

NMF aims to find
nonnegative matricesW and H such that (2.1)Vm×n≈NMFWm×rHr×n, where matrix V is also a nonnegative matrix generated
by total n training images. Each
column of W is called basis image,
while H is the coefficient matrix.
The basis number r is usually chosen
less than n for dimensionality
reduction. The divergence between V and WH is defined as (2.2)F=∑ij(Vijlog⁡Vij(WH)ij−Vij+(WH)ij).NMF (2.1) is equivalent to
the following optimization problem: (2.3)min⁡W,H⁢F,s.t.W≥0,H≥0,∑iWik=1,∀k.The minimization problem
(2.3) can be solved using the following iterative formulae, which converge to a
local minimum: (2.4)Wij⟵Wij∑kVik(WH)ikHjk,Wij⟵Wij∑kWkj,Hij⟵Hij∑kWkiVkj(WH)kj.


## 2.3. BNMF

The
basic idea of BNMF is to perform NMF onc small matrices V(i)∈ℝm×n0 (i=1,2,…,c), namely (2.5)(V(i))m×n0≈NMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c, where V(i) contains n0 training images of the ith class, and c is the
number of classes. BNMF is yielded from (2.5) as follows: (2.6)Vm×n≈BNMFWm×rHr×n, where r=cr0,Vm×n=[V(1)V(2)⋯V(c)],Wm×r=[W(1)W(2)⋯W(c)],Hr×n=diag(H(1),H(2),…,H(c)),
and n(=cn0) is the total number of training images.


## 3. Proposed INMF

To overcome the drawbacks
of existing NMF-based methods, this section proposes a novel incremental NMF (INMF)
approach, which is based on a new constraint NMF criterion and our previous
block technique [19]. Details are discussed below.


### 3.1. Constraint NMF Criterion

The objective of our INMF
is to impose supervised class information on NMF such that between-class
distances increase, while the within-class distances simultaneously decrease.
To this end, we define the within-class scatter matrixSw(i) of the ith
coefficient matrix H(i)∈ℝr0×n0 as (3.1)Sw(i)=1n0∑j=1n0(Hj(i)−U(i))(Hj(i)−U(i))T, where U(i)=(1/n0)∑j=1n0Hj(i) is the mean column vector of the ith
class. The within-classsamples of the kth
class will cluster tightly as tr(Sw(k)) becomes small.AssumeU˜(i) is an enlarging vector of U(i),
that is, U˜(i)=(1+t)U(i) with t>0.
Then we have (3.2)∥U(i)−U(j)∥<(1+t)∥U(i)−U(j)∥=∥U˜(i)−U˜(j)∥. Inequality (3.2) implies that between-class distances are increased
as the mean vectors of classes in H are enlarged.Based on above analysis, we define a constraint divergence
criterion function for thekth class
as follows: (3.3)F(k)=∑ij(Vij(k)log⁡Vij(k)(WH)ij(k)−Vij(k)+(WH)ij(k))+α⁢tr(Sw(k))−β∥U(k)∥22, where parameters α,β>0 and k=1,2,…,c.Our entire INMF criterion
function is then designed as below:(3.4)F=∑k=1cF(k)=∑ijk(Vij(k)log⁡Vij(k)(WH)ij(k)−Vij(k)+(WH)ij(k))+∑k(α⁢tr(Sw(k))−β∥U(k)∥22).Based on criterion (3.4), the following constraint NMF (CNMF) update
rules (3.5)–(3.7) will be
derived in the next subsection. We can show that the iterative formulae (3.5)–(3.7) converge to a
local minimum as well: (3.5)Wij(k)⟵Wij(k)∑lVil(k)(WH)il(k)Hjl(k),(3.6)Wij(k)⟵Wij(k)∑lWlj(k),(3.7)Hij(k)⟵−b+b2−4ad2a, where a=2α−β/nk2,b=−(2α+β/nk)Ui(k)+1,d=−Hij(k)∑lVlj(k)Wli(k)/(W(k)H(k))lj, and Ui(k) is the ith
entry of vector U(k), k=1,2,…,c.So, our entire INMF is performed as follows:(3.8)[V(1)V(2)⋯V(c)]≈INMF[W(1)W(2)⋯W(c)][H(1)H(2)⋱H(c)], where (3.9)(V(i))m×n0≈CNMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c.


### 3.2. Convergence of Proposed Constraint NMF

This subsection reports
how to derive the iterative formulae (3.5)–(3.7) and discusses
their convergences under constraint NMF criterion (3.3).Deffinition 3.1 (see [5]).
J(Q,Q˜) is called an auxiliary function for E(Q), if J(Q,Q˜) satisfies (3.10)J(Q,Q˜)≥E(Q),J(Q,Q)=E(Q), where Q,Q˜ are matrices with the same size.Lemma 3.2 (see [5]).
IfJ(Q,Q˜) is an auxiliary function for E(Q), then E(Q) is a nonincreasing function under the update
rule (3.11)Qi+1=arg⁡min⁡Q⁢J(Q,Qi). To obtain iterative rule (3.7)
and prove its convergence, one first constructs
an auxiliary function for F with fixed W.Theorem 3.3.
IfF(k)(H(k)) is the value of criterion function (3.3) with fixed W(k), then G(k)(H(k),H˜(k)) is an auxiliary function for F(k)(H(k)), where (3.12)G(k)(H(k),H˜(k))=∑ij(Vij(k)logVij(k)−Vij(k)+(W(k)H(k))ij)−∑ijlVij(k)Wil(k)H˜lj(k)∑lWil(k)H˜lj(k)(log⁡(Wil(k)Hlj(k))−log⁡Wil(k)H˜lj(k)∑lWil(k)H˜lj(k))+α⁢tr(Sw(k))−β∥U(k)∥22.Proof.
It can be directly verified
thatG(k)(H(k),H(k))=F(k)(H(k)). So we just need show
the inequality G(k)(H(k),H˜(k))≥F(k)(H(k)). To this end, we will
use the convex function y=log⁡x.
For all i,j, and ∑lσijl=1, it holds that (3.13)−log⁡(∑lWil(k)Hlj(k))≤−∑lσijl⁢log⁢⁡Wil(k)Hlj(k)σijl. Substituting σijl=Wil(k)H˜lj(k)/∑lWil(k)H˜lj(k) into the above inequality, we have (3.14)−log⁡(∑lWil(k)Hlj(k))≤−∑lWil(k)H˜lj(k)∑lWil(k)H˜lj(k)(log⁡Wil(k)Hlj(k)−log⁡Wil(k)H˜lj(k)∑lWil(k)H˜lj(k)). Therefore, G(k)(H(k),H˜(k))≥F(k)(H(k)). This concludes the theorem immediately.Obviously, the functionG(H,H˜)=∑kG(k)(H(k),H˜(k)) is also an auxiliary function for the entire constraint
NMF criterion F(H)=∑kF(k)(H(k)).
Lemma 3.2 indicates that F(H)
is nonincreasing under the update rule (3.11). Let ∂G(H,H˜)/∂Hij(k)=0 and we have (3.15)∂G(H,H˜)∂Hij(k)=∂G(k)(H(k),H˜(k))∂Hij(k)=−∑lVlj(k)Wli(k)H˜ij(k)∑lWli(k)H˜ij(k)1Hij(k)+∑lWli(k)+2α(Hij(k)−Ui(k))−βnk(1nkHij(k)+Ui(k))=0. From the above equation, it directly induces the iterative formula (3.7), and
lemma 3.2 demonstrates that (3.7) converges to a local minimum. For update rule (3.5)-(3.6),
the proof is similar to that of update rule (3.7) using the following auxiliary
function with fixed H: (3.16)G(W,W˜)=∑kG(k)(W(k),W˜(k))=∑ijk(Vij(k)logVij(k)−Vij(k)+(W(k)H(k))ij)−∑ijklVij(k)W˜il(k)Hlj(k)∑lW˜il(k)Hlj(k)(log⁡(Wil(k)Hlj(k))−log⁡W˜il(k)Hlj(k)∑lW˜il(k)Hlj(k))+∑k(α⁢tr(Sw(k))−β∥U(k)∥22).


### 3.3. Incremental Learning

From the  above analysis, our
incremental learning algorithm is designed as follows:(i)Sample incremental learning. As a new training sample x0
of the ith class is added to training set, we
denote that V˜(i)=[V(i),x0]. Thus the training image
matrix becomes (3.17)V˜=[V(1)⋯V˜(i)⋯V(c)].In this case, it only
needs to perform CNMF on matrixV˜(i), that is, V˜(i)≈CNMFW˜(i)H˜(i). The rest decompositions
such as V(k)≈CNMFW(k)H(k)(k⁢≠i) need not implement repetitive computation. So, sample incremental
learning can be performed as follows: (3.18)V˜≈INMFW˜H˜=[W(1)⋯W˜(i)⋯W(c)][H(1)⋱H˜(i)⋱H(c)].(ii)Class incremental learning. As a new
class, denoted by matrix V(c+1), is added to the current
training set, it forms a new training image matrix as (3.19)V˜=[V(1)⋯V(c)∣V(c+1)].The incremental learning
settings are similar to the first item (i) that all decompositionsV(k)≈CNMFW(k)H(k)⁢(k=1,2,…,c) need not compute again. We only need perform CNMF on the matrix V(c+1), that is, V(c+1)≈CNMFW(c+1)H(c+1). Hence, class incremental
learning can be implemented as below: (3.20)V˜≈INMFW˜H˜=[W(1)⋯W(c)∣W(c+1)][H(1)⋱H(c)H(c+1)].


### 3.4. INMF Algorithm Design

Based on the above discussions, this subsection will give a detail design on our INMFalgorithm for face recognition. The algorithm involves two stages, namely training stage and
testing stage. Details are as follows.Training stage
Step 1.
Perform CNMF (3.9) on matrices (V(i))m×n0,i=1,2,…,c, namely, (3.21)(V(i))m×n0≈CNMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c.
Step 2.
INMF is obtained as(3.22)Vm×n≈INMFWm×rHr×n, where r=cr0,n=cn0,  and (3.23)Wm×r=[W(1)W(2)⋯W(c)],Hr×n=diag(H(1),H(2),…,H(c)).
If there is a new
training sample or class added to current training set, then the incremental
learning algorithm presented in Section3.4 is applied  to this stage.Recognition stage
Step 3.
Calculate the
coordinates of a testing samplev^ in the feature space span{W1,W2,…,Wr} by h^=W+v^, where W+ is the Moore-Penrose inverse of W.
Step 4.
Compute the mean column vectorv¯i of class i and its coordinates vector hi=W+v¯i⁢(i=1,2,…,c). The testing image v^ is classified to class k, if d(h^,hk)=min⁡1≤i≤c⁢d(h^,hi), where d(h^,hi) denotes the Euclidean distance between vectors h^ and hi.


### 3.5. Sparseness of Coefficient MatrixH

Leth∈ℝn, define sparseness
function with L1
and L2 norms [7] by (3.24)fsparse(h)=n−∥h∥1/∥h∥2n−1. It can be seen that sparseness function fsparse:ℝn→ℝ with range [0, 1].For INMF method, we have the following theorem for each
columnhi of H.Theorem 3.4.
Sparseness of each columnhi of H in INMF
has the following estimation: (3.25)cr0−r0cr0−1≤fsparse(hi)≤1.Proof.
Let(3.26)hi=(0,…,0,hi1(j),…,hir0(j),0,…,0)T∈ℝr,h˜i=(hi1(j),⋯,hir0(j))T∈ℝr0, where hi belongs to class i in H.
Obviously,(3.27)∥hi∥1=∥h˜i∥1,∥hi∥2=∥h˜i∥2. Moreover, (3.28)1≤∥h˜∥1∥h˜∥2≤r0. So, we have (3.29)r−r0r−1≤r−∥hi∥1/∥hi∥2r−1≤1. It concludes
for r=cr0 that (3.30)cr0−r0cr0−1≤fsparse(hi)≤1.In the experimental
section, the parameters are selected asr0=4
and c=120 using INMF on FERET
database. It can be calculated that (3.31)0.9522≤fsparse(hi)≤1.While on CMU PIE
database, we selectr0=4
and c=68 and calculate that (3.32)0.9355≤fsparse(hi)≤1.These demonstrate that each column ofH in INMF is highly sparse. Apparently, the coefficient column
vectors between different classes in H are automatically orthogonal.


### 3.6. Computational Complexity

This section discusses
the computational complexity of our proposed INMF approach. Theith iterative procedure of proposed INMF
includes two parts, namely W(i) and H(i). For each matrix V(i)， the iteration for W(i) needs mr0(n0r0+2n0+2) multiple times. While for H(i), it needs n0r0(mr0+2m+10) multiple times. Therefore, the total running multiple
times of our INMF are (3.33)TINMF=(2mn0r02+4mn0r0+2mr0+10n0r0)c=2mnr2c2+4mnrc+10nrc+2mr. Similar to
INMF, we can obtain the running multiple times of NMF approach as TNMF=2mnr2+4mnr+2mr+2nr. It can be seen that the
computational complexity of our INMF method is greatly lower than that of NMF.


## 3.1. Constraint NMF Criterion

The objective of our INMF
is to impose supervised class information on NMF such that between-class
distances increase, while the within-class distances simultaneously decrease.
To this end, we define the within-class scatter matrixSw(i) of the ith
coefficient matrix H(i)∈ℝr0×n0 as (3.1)Sw(i)=1n0∑j=1n0(Hj(i)−U(i))(Hj(i)−U(i))T, where U(i)=(1/n0)∑j=1n0Hj(i) is the mean column vector of the ith
class. The within-classsamples of the kth
class will cluster tightly as tr(Sw(k)) becomes small.AssumeU˜(i) is an enlarging vector of U(i),
that is, U˜(i)=(1+t)U(i) with t>0.
Then we have (3.2)∥U(i)−U(j)∥<(1+t)∥U(i)−U(j)∥=∥U˜(i)−U˜(j)∥. Inequality (3.2) implies that between-class distances are increased
as the mean vectors of classes in H are enlarged.Based on above analysis, we define a constraint divergence
criterion function for thekth class
as follows: (3.3)F(k)=∑ij(Vij(k)log⁡Vij(k)(WH)ij(k)−Vij(k)+(WH)ij(k))+α⁢tr(Sw(k))−β∥U(k)∥22, where parameters α,β>0 and k=1,2,…,c.Our entire INMF criterion
function is then designed as below:(3.4)F=∑k=1cF(k)=∑ijk(Vij(k)log⁡Vij(k)(WH)ij(k)−Vij(k)+(WH)ij(k))+∑k(α⁢tr(Sw(k))−β∥U(k)∥22).Based on criterion (3.4), the following constraint NMF (CNMF) update
rules (3.5)–(3.7) will be
derived in the next subsection. We can show that the iterative formulae (3.5)–(3.7) converge to a
local minimum as well: (3.5)Wij(k)⟵Wij(k)∑lVil(k)(WH)il(k)Hjl(k),(3.6)Wij(k)⟵Wij(k)∑lWlj(k),(3.7)Hij(k)⟵−b+b2−4ad2a, where a=2α−β/nk2,b=−(2α+β/nk)Ui(k)+1,d=−Hij(k)∑lVlj(k)Wli(k)/(W(k)H(k))lj, and Ui(k) is the ith
entry of vector U(k), k=1,2,…,c.So, our entire INMF is performed as follows:(3.8)[V(1)V(2)⋯V(c)]≈INMF[W(1)W(2)⋯W(c)][H(1)H(2)⋱H(c)], where (3.9)(V(i))m×n0≈CNMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c.


## 3.2. Convergence of Proposed Constraint NMF

This subsection reports
how to derive the iterative formulae (3.5)–(3.7) and discusses
their convergences under constraint NMF criterion (3.3).Deffinition 3.1 (see [5]).
J(Q,Q˜) is called an auxiliary function for E(Q), if J(Q,Q˜) satisfies (3.10)J(Q,Q˜)≥E(Q),J(Q,Q)=E(Q), where Q,Q˜ are matrices with the same size.Lemma 3.2 (see [5]).
IfJ(Q,Q˜) is an auxiliary function for E(Q), then E(Q) is a nonincreasing function under the update
rule (3.11)Qi+1=arg⁡min⁡Q⁢J(Q,Qi). To obtain iterative rule (3.7)
and prove its convergence, one first constructs
an auxiliary function for F with fixed W.Theorem 3.3.
IfF(k)(H(k)) is the value of criterion function (3.3) with fixed W(k), then G(k)(H(k),H˜(k)) is an auxiliary function for F(k)(H(k)), where (3.12)G(k)(H(k),H˜(k))=∑ij(Vij(k)logVij(k)−Vij(k)+(W(k)H(k))ij)−∑ijlVij(k)Wil(k)H˜lj(k)∑lWil(k)H˜lj(k)(log⁡(Wil(k)Hlj(k))−log⁡Wil(k)H˜lj(k)∑lWil(k)H˜lj(k))+α⁢tr(Sw(k))−β∥U(k)∥22.Proof.
It can be directly verified
thatG(k)(H(k),H(k))=F(k)(H(k)). So we just need show
the inequality G(k)(H(k),H˜(k))≥F(k)(H(k)). To this end, we will
use the convex function y=log⁡x.
For all i,j, and ∑lσijl=1, it holds that (3.13)−log⁡(∑lWil(k)Hlj(k))≤−∑lσijl⁢log⁢⁡Wil(k)Hlj(k)σijl. Substituting σijl=Wil(k)H˜lj(k)/∑lWil(k)H˜lj(k) into the above inequality, we have (3.14)−log⁡(∑lWil(k)Hlj(k))≤−∑lWil(k)H˜lj(k)∑lWil(k)H˜lj(k)(log⁡Wil(k)Hlj(k)−log⁡Wil(k)H˜lj(k)∑lWil(k)H˜lj(k)). Therefore, G(k)(H(k),H˜(k))≥F(k)(H(k)). This concludes the theorem immediately.Obviously, the functionG(H,H˜)=∑kG(k)(H(k),H˜(k)) is also an auxiliary function for the entire constraint
NMF criterion F(H)=∑kF(k)(H(k)).
Lemma 3.2 indicates that F(H)
is nonincreasing under the update rule (3.11). Let ∂G(H,H˜)/∂Hij(k)=0 and we have (3.15)∂G(H,H˜)∂Hij(k)=∂G(k)(H(k),H˜(k))∂Hij(k)=−∑lVlj(k)Wli(k)H˜ij(k)∑lWli(k)H˜ij(k)1Hij(k)+∑lWli(k)+2α(Hij(k)−Ui(k))−βnk(1nkHij(k)+Ui(k))=0. From the above equation, it directly induces the iterative formula (3.7), and
lemma 3.2 demonstrates that (3.7) converges to a local minimum. For update rule (3.5)-(3.6),
the proof is similar to that of update rule (3.7) using the following auxiliary
function with fixed H: (3.16)G(W,W˜)=∑kG(k)(W(k),W˜(k))=∑ijk(Vij(k)logVij(k)−Vij(k)+(W(k)H(k))ij)−∑ijklVij(k)W˜il(k)Hlj(k)∑lW˜il(k)Hlj(k)(log⁡(Wil(k)Hlj(k))−log⁡W˜il(k)Hlj(k)∑lW˜il(k)Hlj(k))+∑k(α⁢tr(Sw(k))−β∥U(k)∥22).


## 3.3. Incremental Learning

From the  above analysis, our
incremental learning algorithm is designed as follows:(i)Sample incremental learning. As a new training sample x0
of the ith class is added to training set, we
denote that V˜(i)=[V(i),x0]. Thus the training image
matrix becomes (3.17)V˜=[V(1)⋯V˜(i)⋯V(c)].In this case, it only
needs to perform CNMF on matrixV˜(i), that is, V˜(i)≈CNMFW˜(i)H˜(i). The rest decompositions
such as V(k)≈CNMFW(k)H(k)(k⁢≠i) need not implement repetitive computation. So, sample incremental
learning can be performed as follows: (3.18)V˜≈INMFW˜H˜=[W(1)⋯W˜(i)⋯W(c)][H(1)⋱H˜(i)⋱H(c)].(ii)Class incremental learning. As a new
class, denoted by matrix V(c+1), is added to the current
training set, it forms a new training image matrix as (3.19)V˜=[V(1)⋯V(c)∣V(c+1)].The incremental learning
settings are similar to the first item (i) that all decompositionsV(k)≈CNMFW(k)H(k)⁢(k=1,2,…,c) need not compute again. We only need perform CNMF on the matrix V(c+1), that is, V(c+1)≈CNMFW(c+1)H(c+1). Hence, class incremental
learning can be implemented as below: (3.20)V˜≈INMFW˜H˜=[W(1)⋯W(c)∣W(c+1)][H(1)⋱H(c)H(c+1)].


## 3.4. INMF Algorithm Design

Based on the above discussions, this subsection will give a detail design on our INMFalgorithm for face recognition. The algorithm involves two stages, namely training stage and
testing stage. Details are as follows.Training stage
Step 1.
Perform CNMF (3.9) on matrices (V(i))m×n0,i=1,2,…,c, namely, (3.21)(V(i))m×n0≈CNMF(W(i))m×r0(H(i))r0×n0,i=1,2,…,c.
Step 2.
INMF is obtained as(3.22)Vm×n≈INMFWm×rHr×n, where r=cr0,n=cn0,  and (3.23)Wm×r=[W(1)W(2)⋯W(c)],Hr×n=diag(H(1),H(2),…,H(c)).
If there is a new
training sample or class added to current training set, then the incremental
learning algorithm presented in Section3.4 is applied  to this stage.Recognition stage
Step 3.
Calculate the
coordinates of a testing samplev^ in the feature space span{W1,W2,…,Wr} by h^=W+v^, where W+ is the Moore-Penrose inverse of W.
Step 4.
Compute the mean column vectorv¯i of class i and its coordinates vector hi=W+v¯i⁢(i=1,2,…,c). The testing image v^ is classified to class k, if d(h^,hk)=min⁡1≤i≤c⁢d(h^,hi), where d(h^,hi) denotes the Euclidean distance between vectors h^ and hi.


## 3.5. Sparseness of Coefficient MatrixH

Leth∈ℝn, define sparseness
function with L1
and L2 norms [7] by (3.24)fsparse(h)=n−∥h∥1/∥h∥2n−1. It can be seen that sparseness function fsparse:ℝn→ℝ with range [0, 1].For INMF method, we have the following theorem for each
columnhi of H.Theorem 3.4.
Sparseness of each columnhi of H in INMF
has the following estimation: (3.25)cr0−r0cr0−1≤fsparse(hi)≤1.Proof.
Let(3.26)hi=(0,…,0,hi1(j),…,hir0(j),0,…,0)T∈ℝr,h˜i=(hi1(j),⋯,hir0(j))T∈ℝr0, where hi belongs to class i in H.
Obviously,(3.27)∥hi∥1=∥h˜i∥1,∥hi∥2=∥h˜i∥2. Moreover, (3.28)1≤∥h˜∥1∥h˜∥2≤r0. So, we have (3.29)r−r0r−1≤r−∥hi∥1/∥hi∥2r−1≤1. It concludes
for r=cr0 that (3.30)cr0−r0cr0−1≤fsparse(hi)≤1.In the experimental
section, the parameters are selected asr0=4
and c=120 using INMF on FERET
database. It can be calculated that (3.31)0.9522≤fsparse(hi)≤1.While on CMU PIE
database, we selectr0=4
and c=68 and calculate that (3.32)0.9355≤fsparse(hi)≤1.These demonstrate that each column ofH in INMF is highly sparse. Apparently, the coefficient column
vectors between different classes in H are automatically orthogonal.


## 3.6. Computational Complexity

This section discusses
the computational complexity of our proposed INMF approach. Theith iterative procedure of proposed INMF
includes two parts, namely W(i) and H(i). For each matrix V(i)， the iteration for W(i) needs mr0(n0r0+2n0+2) multiple times. While for H(i), it needs n0r0(mr0+2m+10) multiple times. Therefore, the total running multiple
times of our INMF are (3.33)TINMF=(2mn0r02+4mn0r0+2mr0+10n0r0)c=2mnr2c2+4mnrc+10nrc+2mr. Similar to
INMF, we can obtain the running multiple times of NMF approach as TNMF=2mnr2+4mnr+2mr+2nr. It can be seen that the
computational complexity of our INMF method is greatly lower than that of NMF.


## 4. Experimental Results

In this section, FERET and CMU PIE databases are selected to
evaluate the performance of our INMF method along with BNMF, NMF, and PCA
methods. All images in two databases are aligned by the centers of eyes and mouth
and then normalized with resolution112×92. The original images with resolution
112×92 are reduced to wavelet feature face with resolution 30×25 after
two-level D4 wavelet decomposition.
If there are negative pixels in the wavelet faces, we will transform them into
nonnegative faces with simple translations. The nearest neighbor classifier using Euclidean
distance is exploited here. In the following experiments, the parameters are set
to r=120 for NMF, r0=4 for BNMF and INMF, α=10−4,β=10−3 for INMF. The stopping condition of iterative
update is (4.1)F(n−1)−F(n)F(n)≤δ, where F(n) is the nth update criterion function defined in (3.3), the threshold δ is set to 10−12. We stop the iteration if
stopping condition (4.1) is met or if exceeding 1000 times iteration.


### 4.1. Face Databases

In FERET database, we
select 120 people, 6 images for each individual. The six images are extracted
from 4 different sets, namely Fa, Fb, Fc, and duplicate. Fa and Fb are sets of
images taken with the same camera at the same day but with different facial
expressions. Fc is a set of images taken with different camera at the same day.
Duplicate is a set of images taken around 6–12 months after
the day taking the Fa and Fb photos. Details of the characteristics of each set
can be found in [3]. Images from one individual are shown in Figure
1.Figure 1
Images of one person from FERET database.CMU PIE database includes totally 68 people. There are 13 pose
variations ranging from full right-profile image to full left-profile image and
43 different lighting conditions, 21 flashes with ambient light on or off. In our experiment, for each person, we select 56 images
including 13 poses with neutral expression and 43 different lighting conditions
in frontal view. Part images of one person are shown in Figure2.Figure 2
Part images of one person from CMU PIE database.


### 4.2. Basis Face Images

This section shows the basis images of the training set learnt by PCA,
NMF, BNMF, and INMF approaches. Figure3 shows 25 basis images of each approach
on CMU PIE database. It can be seen that the bases of all methods are additive
except for PCA. PCA extracts the holistic facial features. INMF learns more local
features than NMF and BNMF. Moreover, the greater number of basis image is, the
more localization is learnt in all NMF-based approaches.Comparisons on basis images of PCA, NMF,
BNMF, and INMF (from left to right), respectively, on CMU PIE database results.
(a)(b)(c)(d)


### 4.3. Results on FERET Database

This section reports the
experimental results with nonincremental learning and incremental learning on
FERET database. All methods use the same training and testing face images. The
experiments are repeated 10 times; and the average accuracies under different
training number, along with the mean running times, are recorded.


#### 4.3.1. Nonincremental Learning

We randomly selectn(n=2,3,4,5) images from each person
for training, while the rest of (6−n)
images of each individual for testing. The average accuracies of training
samples ranging from 2 to 5 are recorded in Table 1 and plotted in Figure 4(a).
The recognition accuracies of INMF, BNMF, NMF, and PCA are 66.73%, 66.07%, 64.44%,
and 34.33%, respectively, with 2 training images. The performance for each
method is improved when the number of training images increases. When the
number of training images is equal to 5, the recognition accuracies of INMF,
BNMF, NMF, and PCA are 83.08%, 81.67%, 80.25%, and 37.58%, respectively. In addition,
Table 2 gives the comparisons on average time-consuming in three NMF-based
approaches. It can be seen that our INMF method gives the best performance for
all cases of nonincremental learning on FERET database.Table 1
Accuracy
comparisons on FERET database.
Training numberPCANMFBNMFINMF234.33%64.44%66.07%66.73%336.00%69.72%72.81%74.39%434.29%76.25%78.04%78.92%537.58%80.25%81.67%83.08%Table 2
Running times (s) on FERET database.
Training numberNMF(s)BNMF(s)INMF(s)251.1430.6921.12374.5857.0233.684100.1275.8145.345122.6189.3456.30Accuracy comparisons
on (a) FERET and (b) CMU PIE databases.
(a)
Training number(b)
Training number


#### 4.3.2. Class Incremental Learning

For 119 people, we
randomly select 3 images from each individual for training and then add a new
class to the training set. NMF must conduct repeated learning while BNMF and INMF
need merely perform incremental training on the new added class. The average accuracies
and the mean running times are recorded in Table3 (plotted in Figure 6(a)) and
Table 4, respectively.Table 3
Accuracy comparisons
on class incremental learning.
Number of classNMFBNMFINMF11969.72%72.94%74.59%12069.45%72.88%74.44%Table 4
Running times (s)
on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)11972.5756.1232.3412074.580.860.34Compared with the NMF and BNMF approaches, the proposed method gives
around 5% and 1.5% accuracy improvements, respectively. The running time of INMF
is around 2 times and 219 times faster than that of NMF with 119 and 120 individuals
for training and class-incremental learning, respectively. Above all, our INMF
gives the best performance on FERET database.


### 4.4. Results on CMU PIE Database

The experimental setting
on CMU PIE database is similar to that of FERET database. It also includes two
parts, namely nonincremental training and incremental learning. The experiments
are repeated 10 times and the average accuracies under different training
number, along with the mean running times, are recorded for comparisons. Details
are as follows.


#### 4.4.1. Nonincremental Learning

For each individual,n(n=7,14,21,28) images are randomly selected for training, while the rest (56−n) images for testing. The average
recognition rates and mean running times are tabulated in Table 5 (plotted in
Figure 4(b)) and Table 6, respectively. It can be seen that the recognition
accuracies of INMF, BNMF, NMF, and PCA are 68.91%, 68.58%, 66.21%, and 23.94%,
respectively with training number 7. When the number of training images is equal
to 28, the recognition accuracies of INMF, BNMF, NMF, and PCA are 77.18%, 76.64%,
71.77%, and 27.51%, respectively. Compared with the PCA and NMF methods, the
proposed method gives around 49% and 5% accuracy improvements, respectively. The
performance of INMF is slightly better than that of BNMF. However, the
computational efficiency of INMF greatly outperforms BNMF.Table 5
Accuracy
comparisons on CMU PIE database.
Training numberPCANMFBNMFINMF723.94%66.21%68.58%68.91%1426.24%69.82%73.79%74.07%2127.15%71.33%75.93%76.55%2827.51%71.77%76.64%77.18%Table 6
Running times
(s) on CMU PIE database.
Training numberNMF(s)BNMF(s)INMF(s)779.5857.3537.2414144.52114.6189.5021208.20164.18124.4528267.97215.94176.23


#### 4.4.2. Sample Incremental Learning

We randomly select 7
images from each person for training, and the rest 49 images for testing. In
the sample-incremental learning stage, 7, 14, and 21 images of the first
individual are added to the training set, respectively, while the training
images from the rest individuals are kept unchanged. Table7 (Figure 5) and Table 8 show the average recognition accuracies and the mean running times,
respectively. Experimental results show that our INMF method gives the best
performance for all cases.Table 7
Accuracy comparisons
on sample incremental learning.
Incremental training numberNMFBNMFINMF066.21%68.58%68.91%766.35%68.66%68.97%1466.51%68.71%69.05%2166.68%68.78%69.14%Table 8
Running times on
sample incremental learning.
Incremental training numberNMF (s)BNMF (s)INMF (s)079.5857.3537.24779.772.341.201480.023.122.362180.234.533.21Figure 5
Comparisons on
sample incremental learning.Class incremental
learning comparisons on (a) FERET and (b) CUM PIE databases.
(a)
Number of class(b)
Number of class


#### 4.4.3. Class Incremental Learning

For 67 people, we randomly
select 7 images from each individual for training and then add a new class to
the training set. NMF should conduct repetitive learning. BNMF and INMF need
merely to perform incremental learning on the new added class. The average
recognition rates and the mean running times are recorded in Table9 (plotted
in Figure 6(b)) and Table 10, respectively. Experimental results show that INMF
outperforms BNMF and NMF in both recognition rates and computational
efficiency.Table 9
Comparisons on
class incremental learning.
Number of classNMFBNMFINMF6766.35%68.73%69.02%6866.21%68.46%68.89%Table 10
Running time
(s) on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)6772.4555.1934.286879.581.360.70


## 4.1. Face Databases

In FERET database, we
select 120 people, 6 images for each individual. The six images are extracted
from 4 different sets, namely Fa, Fb, Fc, and duplicate. Fa and Fb are sets of
images taken with the same camera at the same day but with different facial
expressions. Fc is a set of images taken with different camera at the same day.
Duplicate is a set of images taken around 6–12 months after
the day taking the Fa and Fb photos. Details of the characteristics of each set
can be found in [3]. Images from one individual are shown in Figure
1.Figure 1
Images of one person from FERET database.CMU PIE database includes totally 68 people. There are 13 pose
variations ranging from full right-profile image to full left-profile image and
43 different lighting conditions, 21 flashes with ambient light on or off. In our experiment, for each person, we select 56 images
including 13 poses with neutral expression and 43 different lighting conditions
in frontal view. Part images of one person are shown in Figure2.Figure 2
Part images of one person from CMU PIE database.


## 4.2. Basis Face Images

This section shows the basis images of the training set learnt by PCA,
NMF, BNMF, and INMF approaches. Figure3 shows 25 basis images of each approach
on CMU PIE database. It can be seen that the bases of all methods are additive
except for PCA. PCA extracts the holistic facial features. INMF learns more local
features than NMF and BNMF. Moreover, the greater number of basis image is, the
more localization is learnt in all NMF-based approaches.Comparisons on basis images of PCA, NMF,
BNMF, and INMF (from left to right), respectively, on CMU PIE database results.
(a)(b)(c)(d)


## 4.3. Results on FERET Database

This section reports the
experimental results with nonincremental learning and incremental learning on
FERET database. All methods use the same training and testing face images. The
experiments are repeated 10 times; and the average accuracies under different
training number, along with the mean running times, are recorded.


### 4.3.1. Nonincremental Learning

We randomly selectn(n=2,3,4,5) images from each person
for training, while the rest of (6−n)
images of each individual for testing. The average accuracies of training
samples ranging from 2 to 5 are recorded in Table 1 and plotted in Figure 4(a).
The recognition accuracies of INMF, BNMF, NMF, and PCA are 66.73%, 66.07%, 64.44%,
and 34.33%, respectively, with 2 training images. The performance for each
method is improved when the number of training images increases. When the
number of training images is equal to 5, the recognition accuracies of INMF,
BNMF, NMF, and PCA are 83.08%, 81.67%, 80.25%, and 37.58%, respectively. In addition,
Table 2 gives the comparisons on average time-consuming in three NMF-based
approaches. It can be seen that our INMF method gives the best performance for
all cases of nonincremental learning on FERET database.Table 1
Accuracy
comparisons on FERET database.
Training numberPCANMFBNMFINMF234.33%64.44%66.07%66.73%336.00%69.72%72.81%74.39%434.29%76.25%78.04%78.92%537.58%80.25%81.67%83.08%Table 2
Running times (s) on FERET database.
Training numberNMF(s)BNMF(s)INMF(s)251.1430.6921.12374.5857.0233.684100.1275.8145.345122.6189.3456.30Accuracy comparisons
on (a) FERET and (b) CMU PIE databases.
(a)
Training number(b)
Training number


### 4.3.2. Class Incremental Learning

For 119 people, we
randomly select 3 images from each individual for training and then add a new
class to the training set. NMF must conduct repeated learning while BNMF and INMF
need merely perform incremental training on the new added class. The average accuracies
and the mean running times are recorded in Table3 (plotted in Figure 6(a)) and
Table 4, respectively.Table 3
Accuracy comparisons
on class incremental learning.
Number of classNMFBNMFINMF11969.72%72.94%74.59%12069.45%72.88%74.44%Table 4
Running times (s)
on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)11972.5756.1232.3412074.580.860.34Compared with the NMF and BNMF approaches, the proposed method gives
around 5% and 1.5% accuracy improvements, respectively. The running time of INMF
is around 2 times and 219 times faster than that of NMF with 119 and 120 individuals
for training and class-incremental learning, respectively. Above all, our INMF
gives the best performance on FERET database.


## 4.3.1. Nonincremental Learning

We randomly selectn(n=2,3,4,5) images from each person
for training, while the rest of (6−n)
images of each individual for testing. The average accuracies of training
samples ranging from 2 to 5 are recorded in Table 1 and plotted in Figure 4(a).
The recognition accuracies of INMF, BNMF, NMF, and PCA are 66.73%, 66.07%, 64.44%,
and 34.33%, respectively, with 2 training images. The performance for each
method is improved when the number of training images increases. When the
number of training images is equal to 5, the recognition accuracies of INMF,
BNMF, NMF, and PCA are 83.08%, 81.67%, 80.25%, and 37.58%, respectively. In addition,
Table 2 gives the comparisons on average time-consuming in three NMF-based
approaches. It can be seen that our INMF method gives the best performance for
all cases of nonincremental learning on FERET database.Table 1
Accuracy
comparisons on FERET database.
Training numberPCANMFBNMFINMF234.33%64.44%66.07%66.73%336.00%69.72%72.81%74.39%434.29%76.25%78.04%78.92%537.58%80.25%81.67%83.08%Table 2
Running times (s) on FERET database.
Training numberNMF(s)BNMF(s)INMF(s)251.1430.6921.12374.5857.0233.684100.1275.8145.345122.6189.3456.30Accuracy comparisons
on (a) FERET and (b) CMU PIE databases.
(a)
Training number(b)
Training number


## 4.3.2. Class Incremental Learning

For 119 people, we
randomly select 3 images from each individual for training and then add a new
class to the training set. NMF must conduct repeated learning while BNMF and INMF
need merely perform incremental training on the new added class. The average accuracies
and the mean running times are recorded in Table3 (plotted in Figure 6(a)) and
Table 4, respectively.Table 3
Accuracy comparisons
on class incremental learning.
Number of classNMFBNMFINMF11969.72%72.94%74.59%12069.45%72.88%74.44%Table 4
Running times (s)
on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)11972.5756.1232.3412074.580.860.34Compared with the NMF and BNMF approaches, the proposed method gives
around 5% and 1.5% accuracy improvements, respectively. The running time of INMF
is around 2 times and 219 times faster than that of NMF with 119 and 120 individuals
for training and class-incremental learning, respectively. Above all, our INMF
gives the best performance on FERET database.


## 4.4. Results on CMU PIE Database

The experimental setting
on CMU PIE database is similar to that of FERET database. It also includes two
parts, namely nonincremental training and incremental learning. The experiments
are repeated 10 times and the average accuracies under different training
number, along with the mean running times, are recorded for comparisons. Details
are as follows.


### 4.4.1. Nonincremental Learning

For each individual,n(n=7,14,21,28) images are randomly selected for training, while the rest (56−n) images for testing. The average
recognition rates and mean running times are tabulated in Table 5 (plotted in
Figure 4(b)) and Table 6, respectively. It can be seen that the recognition
accuracies of INMF, BNMF, NMF, and PCA are 68.91%, 68.58%, 66.21%, and 23.94%,
respectively with training number 7. When the number of training images is equal
to 28, the recognition accuracies of INMF, BNMF, NMF, and PCA are 77.18%, 76.64%,
71.77%, and 27.51%, respectively. Compared with the PCA and NMF methods, the
proposed method gives around 49% and 5% accuracy improvements, respectively. The
performance of INMF is slightly better than that of BNMF. However, the
computational efficiency of INMF greatly outperforms BNMF.Table 5
Accuracy
comparisons on CMU PIE database.
Training numberPCANMFBNMFINMF723.94%66.21%68.58%68.91%1426.24%69.82%73.79%74.07%2127.15%71.33%75.93%76.55%2827.51%71.77%76.64%77.18%Table 6
Running times
(s) on CMU PIE database.
Training numberNMF(s)BNMF(s)INMF(s)779.5857.3537.2414144.52114.6189.5021208.20164.18124.4528267.97215.94176.23


### 4.4.2. Sample Incremental Learning

We randomly select 7
images from each person for training, and the rest 49 images for testing. In
the sample-incremental learning stage, 7, 14, and 21 images of the first
individual are added to the training set, respectively, while the training
images from the rest individuals are kept unchanged. Table7 (Figure 5) and Table 8 show the average recognition accuracies and the mean running times,
respectively. Experimental results show that our INMF method gives the best
performance for all cases.Table 7
Accuracy comparisons
on sample incremental learning.
Incremental training numberNMFBNMFINMF066.21%68.58%68.91%766.35%68.66%68.97%1466.51%68.71%69.05%2166.68%68.78%69.14%Table 8
Running times on
sample incremental learning.
Incremental training numberNMF (s)BNMF (s)INMF (s)079.5857.3537.24779.772.341.201480.023.122.362180.234.533.21Figure 5
Comparisons on
sample incremental learning.Class incremental
learning comparisons on (a) FERET and (b) CUM PIE databases.
(a)
Number of class(b)
Number of class


### 4.4.3. Class Incremental Learning

For 67 people, we randomly
select 7 images from each individual for training and then add a new class to
the training set. NMF should conduct repetitive learning. BNMF and INMF need
merely to perform incremental learning on the new added class. The average
recognition rates and the mean running times are recorded in Table9 (plotted
in Figure 6(b)) and Table 10, respectively. Experimental results show that INMF
outperforms BNMF and NMF in both recognition rates and computational
efficiency.Table 9
Comparisons on
class incremental learning.
Number of classNMFBNMFINMF6766.35%68.73%69.02%6866.21%68.46%68.89%Table 10
Running time
(s) on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)6772.4555.1934.286879.581.360.70


## 4.4.1. Nonincremental Learning

For each individual,n(n=7,14,21,28) images are randomly selected for training, while the rest (56−n) images for testing. The average
recognition rates and mean running times are tabulated in Table 5 (plotted in
Figure 4(b)) and Table 6, respectively. It can be seen that the recognition
accuracies of INMF, BNMF, NMF, and PCA are 68.91%, 68.58%, 66.21%, and 23.94%,
respectively with training number 7. When the number of training images is equal
to 28, the recognition accuracies of INMF, BNMF, NMF, and PCA are 77.18%, 76.64%,
71.77%, and 27.51%, respectively. Compared with the PCA and NMF methods, the
proposed method gives around 49% and 5% accuracy improvements, respectively. The
performance of INMF is slightly better than that of BNMF. However, the
computational efficiency of INMF greatly outperforms BNMF.Table 5
Accuracy
comparisons on CMU PIE database.
Training numberPCANMFBNMFINMF723.94%66.21%68.58%68.91%1426.24%69.82%73.79%74.07%2127.15%71.33%75.93%76.55%2827.51%71.77%76.64%77.18%Table 6
Running times
(s) on CMU PIE database.
Training numberNMF(s)BNMF(s)INMF(s)779.5857.3537.2414144.52114.6189.5021208.20164.18124.4528267.97215.94176.23


## 4.4.2. Sample Incremental Learning

We randomly select 7
images from each person for training, and the rest 49 images for testing. In
the sample-incremental learning stage, 7, 14, and 21 images of the first
individual are added to the training set, respectively, while the training
images from the rest individuals are kept unchanged. Table7 (Figure 5) and Table 8 show the average recognition accuracies and the mean running times,
respectively. Experimental results show that our INMF method gives the best
performance for all cases.Table 7
Accuracy comparisons
on sample incremental learning.
Incremental training numberNMFBNMFINMF066.21%68.58%68.91%766.35%68.66%68.97%1466.51%68.71%69.05%2166.68%68.78%69.14%Table 8
Running times on
sample incremental learning.
Incremental training numberNMF (s)BNMF (s)INMF (s)079.5857.3537.24779.772.341.201480.023.122.362180.234.533.21Figure 5
Comparisons on
sample incremental learning.Class incremental
learning comparisons on (a) FERET and (b) CUM PIE databases.
(a)
Number of class(b)
Number of class


## 4.4.3. Class Incremental Learning

For 67 people, we randomly
select 7 images from each individual for training and then add a new class to
the training set. NMF should conduct repetitive learning. BNMF and INMF need
merely to perform incremental learning on the new added class. The average
recognition rates and the mean running times are recorded in Table9 (plotted
in Figure 6(b)) and Table 10, respectively. Experimental results show that INMF
outperforms BNMF and NMF in both recognition rates and computational
efficiency.Table 9
Comparisons on
class incremental learning.
Number of classNMFBNMFINMF6766.35%68.73%69.02%6866.21%68.46%68.89%Table 10
Running time
(s) on class incremental learning.
Number of classNMF(s)BNMF(s)INMF(s)6772.4555.1934.286879.581.360.70


## 5. Conclusions

This paper proposed a novel constraint INMF method to address
the time-consuming problem and incremental learning problem of existing NMF-based
approaches for face recognition. INMF has some good properties, such as low
computational complexity; sparse coefficient matrix; orthogonal coefficient
column vectors between different classes in coefficient matrixH; especially for incremental learning,
and so on. Experimental results on FERET and CMU PIE face database show that INMF
outperforms PCA, NMF, and BNMF approaches in nonincremental learning and
incremental learning.



---

*Source: 410674-2008-09-01.xml*