# A Constrained Algorithm Based NMFα for Image Representation

Authors: Chenxue Yang; Tao Li; Mao Ye; Zijian Liu; Jiao Bao
Journal: Discrete Dynamics in Nature and Society
 (2014)

---

## Abstract

Nonnegative matrix factorization (NMF) is a useful tool in learning a basic representation of image data. However, its performance and applicability in real scenarios are limited because of the lack of image information. In this paper, we propose a constrained matrix decomposition algorithm for image representation which contains parameters associated with the characteristics of image data sets. Particularly, we impose label information as additional hard constraints to theα-divergence-NMF unsupervised learning algorithm. The resulted algorithm is derived by using Karush-Kuhn-Tucker (KKT) conditions as well as the projected gradient and its monotonic local convergence is proved by using auxiliary functions. In addition, we provide a method to select the parameters to our semisupervised matrix decomposition algorithm in the experiment. Compared with the state-of-the-art approaches, our method with the parameters has the best classification accuracy on three image data sets.


---

## Body

## 1. Introduction

Learning an efficient representation of image information is a key problem in machine learning and computer vision. Efficiency of the representation refers to the ability to capture significant information from a high dimensional image space. Such a high dimensional problem is difficult to manipulate and compute; therefore dimension reduction becomes the crucial method to cope with this problem. Fortunately, matrix factorization is a valid approach to solve the dimension reduction problem, and it has a long and successful history in dealing with image representation [1–3]. Some methods of matrix factorization can be referred to as principal component analysis (PCA) [4], singular value decomposition (SVD) [5], vector quantization (VQ) [6], and nonnegative matrix factorization (NMF) [7].Among all techniques for matrix factorization, NMF is distinguished from others by its use of nonnegative constraints in learning a basis representation of image data [8] and has been applied in face recognition [9–11], medical imaging [12, 13], electroencephalogram (EEG) classification for brain computer interface [14], and many other areas. However, NMF is an unsupervised learning algorithm and inapplicable to learning a basic representation from limited image information. Thus, to make up for this deficiency, extra constraints are implicitly or explicitly incorporated into NMF to derive some semisupervised matrix decomposition algorithms. In [15], the authors impose label information as additional hard constraints to NMF based on the squared Euclidean distance and Kulback-Leibler divergence. Such a representation encodes the data points from the same class using the indicator matrix in a new representation space, where the obtained part-based representation is more discriminating.However, none of the semisupervised NMF algorithms mentioned above contain parameters associated with the characteristics of image data sets. In this paper, we introduceα-divergence-NMF algorithm [16], where α is a parameter. We impose the labeled constraints to the α-divergence-NMF algorithm to derive a generic constraint matrix decomposition algorithm which includes some existing algorithms as their special cases: one of them is CNMFKL [15] with α=1. Then, we obtain the proposed algorithm using Karush-Kuhn-Tucker (KKT) method as well as the projected gradient method and prove its monotonic local convergence using an auxiliary function. Comparing to the current semisupervised NMF algorithms, we analyze the classification accuracy for two fixed values of α (α=0.5,2) on three image data sets.TheCNMFα algorithm does not work well for a fixed value of α. Since the parameter α is associated with the characteristics of a learning machine, the model distribution is more inclusive when α goes to +∞ and is more exclusive when α approaches to -∞. The selection of the optimal value of α plays a critical role in determining the discriminative basis vectors. In this paper, we provide a method to select parameters α for our semisupervised CNMFα algorithm. The variation of α is associated with the characteristics of image data sets. Compared with the algorithms in [15, 16], our algorithm is more complete and systemic.The rest of the paper is organized as follows. In Section2, we make a brief overview on standard NMF algorithm and constraint NMF algorithm. The detailed algorithms with labeled constraints and theoretical proof of the convergence of the algorithms are provided in Sections 3 and 4 separately. Section 5 presents some experimental results to show the advantages of our algorithm. Finally, a conclusion is given in Section 6.


## 2. Related Work

NMF, proposed by Lee and Seung [7], is considered to provide a part-based representation and applied in diverse examples of nonnegative data [17–21] including text data mining, subsystem identification, spectral data analysis, audio and sound processing, and document clustering.SupposeX=[x1,…,xn]∈Rm×n is a set of n training images, where {xt}t=1n is a column vector and consists of m nonnegative pixel values of a training image. NMF is to find two nonnegative matrix factors W∈Rm×r and H∈Rn×r to approximate the original image matrix
(1)X≈WHT,
where the positive integer r is smaller than m or n.NMF uses nonnegative constraints to make the representation purely adapted to an unsupervised way. It is inapplicable to learn a basis representation to the limited image information. To make up for this deficiency, extra constraints such as locality [22], sparseness [9], and orthogonality [23] were implicitly or explicitly incorporated into NMF to identify better local features or provide more sparse representation.In [15], the authors impose label information as additional hard constraints to NMF unsupervised learning algorithm to derive a semisupervised matrix decomposition algorithm, which makes the obtained representation more discriminating. The label information is incorporated as follows.SupposeX={xi}i=1n is a data set, which consists of n training images. Set that the first s images {x1,…,xs}(s≤n) are represented by the label information, and the remaining n-s images {xs+1,…,xn}(s≤n) are represented by the unlabeled. Assume there exist c classes and each image from {x1,…,xs} is designated one class. Then we have an s×c indicator matrix S, which can be represented as
(2)sij={1,ifxiisdesignatedthejthclass,0,otherwise.
From the indicator matrix S, a label constraint matrix U can be defined as
(3)U=(Ss×c00In-s),
where In-s denotes an (n-s)×(n-s) identity matrix.Imposing the label information as additional hard constraint by an auxiliary matrixV, there is H=UV. It verifies that hi=hj if xi and xj have the same labels. With the label constraints, the standard NMF is transformed into factorizing a large-size matrix X into the product of three small-size matrices W, U, and V:
(4)X≈W(UV)T.
Such a representation encodes the data points from the same class using the indicator matrix in a new representation space.


## 3. A Constrained Algorithm BasedNMFα

The exact form of the error measure of (1) is as crucial as the nonnegative constraints in the success of the NMF algorithm in learning a useful representation of image data. In the researches on NMF, there are quite a large number of investigations for error measure, such as Csiszár’s f-divergences [24], Amari’s α-divergence [25], and Bregman divergences [26]. Here, we introduce a genetic multiplicative updating algorithm [16] which iteratively minimizes the α-divergence between X and WHT. We define the α-divergence as
(5)Dα[X∥WHT]=1α(1-α)∑i=1m‍∑j=1n‍{αXij+(1-α)[WHT]ij-Xijα[WHT]ij1-α},
where α is a positive parameter. We combine the labeled constraints with (4) to derive the following objective function, which is based on the α-divergence between X and W(UV)T,
(6)Dα[X∥W(UV)T]=1α(1-α)×∑i=1m‍∑j=1n‍{αXij+(1-α)[WVTUT]ij-Xijα[WVTUT]ij1-α}.
With the constraints Wij≥0, Uij≥0, and Vij≥0, the minimization of Dα[X∥W(UV)T] can be formulated as a constrained minimization problem with inequality constraints. In the following, we will present two methods to find a local minimum of (6).


### 3.1. KKT Method

Letλij≥0 and μij≥0 be the Lagrangian multipliers associated with constraints Vij≥0 and Wij≥0, respectively. The Karush-Kuhn-Tucker conditions require that both the optimality conditions
(7)∂Dα[X∥W(UV)T]∂Vij=1α{∑k[UTW]ki-∑k[UTW]ki(Xkj[WVTUT]kj)α}=λij,(8)∂Dα[X∥W(UV)T]∂Wij=1α{∑k[VTUT]jk-∑k[VTUT]jk(Xik[WVTUT]ik)α}=μij
and the complementary slackness conditions
(9)λijVij=0,μijWij=0
are satisfied. If Vij=0 and Wij=0, then either λijμij can have any values. At the same time, if λij=0 and μij=0, then Vij≥0 and Wij≥0. Hence we need both λij≠0μij≠0 and Vij≠0Wij≠0. It follows from (9) that
(10)λijVijα=0,μijWijα=0.
We multiply both sides of (7) and (8) by Vij and Wij, respectively, and incorporate with (10), and then we obtain the following updating rules:
(11)Vij⟵Vij[∑k‍[UTW]ki(Xkj/[WVTUT]kj)α∑l‍[UTW]li]1/α,(12)Wij⟵Wij[∑k‍[VTUT]jk(Xik/[WVTUT]ik)α∑l‍[VTUT]jl]1/α.


### 3.2. Projected Gradient Method

Considering the gradient descent algorithm [24, 25], the updating rules for the objective function (6) can be also derived by using the projected gradient method [27] and have the form
(13)ϕ(Vij)⟵ϕ(Vij)+δij∂Dα[X∥W(UV)T]∂Vij,ϕ(Wij)⟵ϕ(Wij)+γij∂Dα[X∥W(UV)T]∂Wij,
where ϕ(·) is a suitably chosen function and δij and γij are two parameters to control the step size of gradient descent. Then, we have
(14)Vij⟵ϕ-1(ϕ(Vij)+δij∂Dα[X∥W(UV)T]∂Vij),Wij⟵ϕ-1(ϕ(Wij)+γij∂Dα[X∥W(UV)T]∂Wij).
Setting ϕ(Ω)=Ωα, to guarantee that the updating rules (11) and (12) hold, we need
(15)δij=-αVijα∑k‍[UTW]ki,γij=-αWijα∑k‍[VTUT]jk.
From (7) and (8), the updating rules become
(16)Vijα⟵Vijα+δij∂Dα[X∥W(UV)T]∂Vij=Vijα∑k‍[UTW]ki(Xkj/[WVTUT]kj)α∑l‍[UTW]li,Wijα⟵Wijα+γij∂Dα[X∥W(UV)T]∂Wij=Wijα∑k[VTUT]jk‍(Xik/[WVTUT]ik)α∑l‍[VTUT]jl,
which are the same as the updating rules (11) and (12). We have shown that the algorithm can be derived using Karush-Kuhn-Tucker conditions and presented alternative of the algorithm using the projected gradient. The use of the two methods guarantees the correctness of the algorithm theoretically.In the following, we will give a theorem to guarantee the convergence of the iterations in updates (11) and (12).Theorem 1.
For the objective function (6), Dα[X∥W(UV)T] is nonincreasing under the updating rules (11) and (12). The objective function Dα is invariant under these updates if and only if V and W are at a stationary point.Multiplicative updates for our constrained algorithm basedNMFα are given in (11) and (12). These updates find a local minimum of Dα[X∥W(UV)T], which is the final solution of (11) and (12). Note that, when α=1, the updates (11) and (12) are the same with CNMFKL algorithm [15], which is a special case included in our generic constraint matrix decomposition algorithm. In the following, we will give the proof of Theorem 1.


## 3.1. KKT Method

Letλij≥0 and μij≥0 be the Lagrangian multipliers associated with constraints Vij≥0 and Wij≥0, respectively. The Karush-Kuhn-Tucker conditions require that both the optimality conditions
(7)∂Dα[X∥W(UV)T]∂Vij=1α{∑k[UTW]ki-∑k[UTW]ki(Xkj[WVTUT]kj)α}=λij,(8)∂Dα[X∥W(UV)T]∂Wij=1α{∑k[VTUT]jk-∑k[VTUT]jk(Xik[WVTUT]ik)α}=μij
and the complementary slackness conditions
(9)λijVij=0,μijWij=0
are satisfied. If Vij=0 and Wij=0, then either λijμij can have any values. At the same time, if λij=0 and μij=0, then Vij≥0 and Wij≥0. Hence we need both λij≠0μij≠0 and Vij≠0Wij≠0. It follows from (9) that
(10)λijVijα=0,μijWijα=0.
We multiply both sides of (7) and (8) by Vij and Wij, respectively, and incorporate with (10), and then we obtain the following updating rules:
(11)Vij⟵Vij[∑k‍[UTW]ki(Xkj/[WVTUT]kj)α∑l‍[UTW]li]1/α,(12)Wij⟵Wij[∑k‍[VTUT]jk(Xik/[WVTUT]ik)α∑l‍[VTUT]jl]1/α.


## 3.2. Projected Gradient Method

Considering the gradient descent algorithm [24, 25], the updating rules for the objective function (6) can be also derived by using the projected gradient method [27] and have the form
(13)ϕ(Vij)⟵ϕ(Vij)+δij∂Dα[X∥W(UV)T]∂Vij,ϕ(Wij)⟵ϕ(Wij)+γij∂Dα[X∥W(UV)T]∂Wij,
where ϕ(·) is a suitably chosen function and δij and γij are two parameters to control the step size of gradient descent. Then, we have
(14)Vij⟵ϕ-1(ϕ(Vij)+δij∂Dα[X∥W(UV)T]∂Vij),Wij⟵ϕ-1(ϕ(Wij)+γij∂Dα[X∥W(UV)T]∂Wij).
Setting ϕ(Ω)=Ωα, to guarantee that the updating rules (11) and (12) hold, we need
(15)δij=-αVijα∑k‍[UTW]ki,γij=-αWijα∑k‍[VTUT]jk.
From (7) and (8), the updating rules become
(16)Vijα⟵Vijα+δij∂Dα[X∥W(UV)T]∂Vij=Vijα∑k‍[UTW]ki(Xkj/[WVTUT]kj)α∑l‍[UTW]li,Wijα⟵Wijα+γij∂Dα[X∥W(UV)T]∂Wij=Wijα∑k[VTUT]jk‍(Xik/[WVTUT]ik)α∑l‍[VTUT]jl,
which are the same as the updating rules (11) and (12). We have shown that the algorithm can be derived using Karush-Kuhn-Tucker conditions and presented alternative of the algorithm using the projected gradient. The use of the two methods guarantees the correctness of the algorithm theoretically.In the following, we will give a theorem to guarantee the convergence of the iterations in updates (11) and (12).Theorem 1.
For the objective function (6), Dα[X∥W(UV)T] is nonincreasing under the updating rules (11) and (12). The objective function Dα is invariant under these updates if and only if V and W are at a stationary point.Multiplicative updates for our constrained algorithm basedNMFα are given in (11) and (12). These updates find a local minimum of Dα[X∥W(UV)T], which is the final solution of (11) and (12). Note that, when α=1, the updates (11) and (12) are the same with CNMFKL algorithm [15], which is a special case included in our generic constraint matrix decomposition algorithm. In the following, we will give the proof of Theorem 1.


## 4. Convergence Analysis

To prove Theorem1, we will make use of an auxiliary function that was used in the expectation-maximization algorithm [28, 29].Definition 2.
A functionG(x,x^) is defined as an auxiliary function for F(x) if the following two conditions are both satisfied:
(17)G(x,x)=F(x),G(x,x^)≥F(x).Lemma 3.
Assume that the functionG(x,x^) is an auxiliary function for F(x); then F(x) is nonincreasing under the update
(18)xt+1=argmin⁡xG(x,xt).Proof .
ConsiderF(xt+1)≤G(xt+1,xt)≤G(xt,xt)=F(xt).It can be observed that the equalityF(xt+1)=F(xt) holds only if xt is a local minimum of G(x,x^). We iterate the update in (18) to obtain a sequence of estimates that converge to a local minimum xmin⁡=argmin⁡xF(x) of the objective function given by
(19)F(xmin⁡)≤⋯≤F(xt+1)≤F(xt)≤⋯≤F(x2)≤F(x1)≤F(x0).
In the following, we will show that the objective function (6) is nonincreasing under the updating rules (11) and (12) by defining the appropriate auxiliary functions with respect to Vij and Wij.Lemma 4.
Function(20)G(V,V^)=1α(1-α)∑i,j,k‍Xijξijk(V^)×{α+(1-α)Wik[VTUT]kjXijξijk(V^)-(Wik[VTUT]kjXijξijk(V^))1-α},
where ξijk(V^)=Wik[V^TUT]kj/∑l‍Wil[V^TUT]lj, is an auxiliary function for
(21)F(V)=1α(1-α)×∑i,j‍αXij+(1-α)[WVTUT]ij-Xijα[WVTUT]ij1-α.Proof.
Obviously,G(V,V)=F(V). According to the definition of auxiliary function, we only need to prove G(V,V^)≥F(V). To do this, we use the convex function f(·) for positive α to rewrite the α-divergence function F(V) as
(22)F(V)=∑i,j‍Xijf(∑k‍Wik[VTUT]kjXij),
where
(23)f(x)=1α(1-α)[α-(1-α)x-x1-α].
Note that ∑k‍ξijk(V^)=1 and ξijk(V^)≥0 from the definition of ξijk(V^). Applying Jensen’s inequality [30], it leads to
(24)f(∑k‍Wik[VTUT]kj)≤∑k‍ξijk(V^)f(Wik[VTUT]kjξijk(V^)).
From the above inequality, it follows that
(25)F(V)=∑i,j‍Xijf(∑k‍Wik[VTUT]kjXij)≤∑i,j,k‍Xijξijk(V^)f(Wik[VTUT]kjXijξijk(V^))=G(V,V^),
which satisfies the condition of auxiliary function.Reversing the rules ofVij and Wij in Lemma 4, we define an auxiliary function for the update (12).Lemma 5.
Function(26)G(W,W^)=1α(1-α)∑i,j,k‍Xijηijk(W^)×{α+(1-α)Wik[VTUT]kjXijηijk(W^)-(Wik[VTUT]kjXijηijk(W^))1-α},
where ηijk(W^)=W^ik[VTUT]kj/∑l‍W^il[VTUT]lj, is an auxiliary function for
(27)F(W)=1α(1-α)×∑i,j‍αXij+(1-α)[WVTUT]ij-Xijα[WVTUT]ij1-α.This can be easily proved in the same way as Lemma4. From Lemmas 4 and 5, now we can demonstrate the convergence of Theorem 1.Proof.
To guarantee the stability ofF(V), from Lemma 3, we just need to obtain the minimum of G(V,V^) with respect to Vij. Set the gradient of (20) to zero; there is
(28)∂G(V,V^)∂Vij=1α∑k‍[UTW]ki{1-(Wki[VTUT]ijXkjξkji(V^))-α}=0.
Then, it follows that
(29)Vij=V^ij[∑k‍[UTW]ki(Xkj/∑l‍Wkl[V^TUT]lj)α∑k‍[UTW]ki]1/α,
which is similar to the form of the updating rule (11). Similarly, to guarantee the updating rule (12) holds, the minimum of G(W,W^), which can be determined by setting the gradient of (26) to zero, must exist.SinceG is an auxiliary function, according to Lemma 4, F in (21) is nonincreasing under the update (11). Multiplying updates (11) and (12), we can find a local minimum of Dα[X∥W(UV)T].


## 5. Experiments

In this section, theCNMFα algorithm is systematically compared with the current constrained NMF algorithms on three image data sets, named ORL Database [31], Yale Database [32], and Caltech 101 Database [33]. The details of the above three databases will be described individually later. We introduce the evaluated algorithms firstly.(i)
Constrained nonnegative matrix factorization algorithm in [15] aims to minimize the F-norm cost.(ii)
Constrained nonnegative matrix factorization algorithm with parameterα=0.5 in this paper aims to minimize the Hellinger divergence cost.(iii)
Constrained nonnegative matrix factorization algorithm with parameterα=1 in [15], aiming at minimizing the KL-divergence cost, is the best reported algorithm in image representation.(iv)
Constrained nonnegative matrix factorization algorithm with parameterα=2 in this paper aims to minimize the χ2-divergence cost.(v)
Constrained nonnegative matrix factorization algorithm with parameterα=αk in this paper aims to minimize the α-divergence cost, where the parameters αk are associated with the characteristics of the image database and designed by the presented method. CNMFKL algorithm is a special case of our CNMFαk algorithm with α=αk=1.We apply these algorithms to a problem of classification and evaluate their performance on three image data sets which contain a number of different categories of image. For each date set, the evaluations are conducted with different numbers of clusters; here the number of clustersk varies from 2 to 10. We randomly choose k categories from one image data set and mix the images of these k categories as the collection X. Then, for the semisupervised algorithms, we randomly pick up 10 percent images from each category in X and record their category number as the available label information to obtain the label matrix U. For some special data sets, the label process is different and we will describe the details later.Suppose a data set hasN categories C1,C2…,CN, and the cardinalities of these labeled images are L1,L2…,LN, respectively. Since the label constraint matrix U is composed of the indicator matrix S and the identity matrix I, the indicator matrix S plays a critical role in classification performance for different categories in X. To determine the effectiveness of S, we define a measure to represent the relationship between the cardinalities of labeled samples and the total samples,
(30)r(k)=Lmax⁡(k)-Lmin⁡(k)∑i=1kCik,
where Lmax⁡(k) and Lmin⁡(k) denote the maximum and minimum labeled cardinalities of k categories. For the fixed cluster number k in the data set, r(k) is different if the number of samples in each category is different. Then, we compute α as follows:
(31)αk=αNr(N)(+|Lmin⁡(k)k∑i=1kCi-∑i=1NLi∑i=1NCi|)-1|r(k)-r(N)|×(|Lmax⁡(k)k∑i=1kCi-∑i=1NLi∑i=1NCi|+|Lmin⁡(k)k∑i=1kCi-∑i=1NLi∑i=1NCi|)-1),
where
(32)αN=(Lmax⁡(N)-Lmin⁡(N))×(|Lmax⁡(N)-∑i=1NLi∑i=1NCi|+|Lmin⁡(N)-∑i=1NLi∑i=1NCi|)-1.
The value of αk computed by (31) is associated with the characteristics of image data sets, since its variation is caused by both the cardinalities of labeled samples in each category and the total samples. We can obtain both the cardinalities of labeled samples and the total samples in our semisupervised algorithms. However, we can not get the cardinalities of labeled images exactly in many real-word applications. Moreover, the value of α varies depending on data sets. It is still an open problem how to select the optimal α [16].To evaluate the classification performance, we define classification accuracy as the first measure. OurCNMFα algorithm described in (11) and (12) provides a classification label of each sample, marked sili. Suppose X={xi}i=1n is a data set, which consists of n training images. For each sample, let li be the true class label provided by the image data set. More specifically, if the image xi is designated the lith class, we evaluate it as a correct label and set sili=1. Otherwise, it is counted as a false label and noted sili=0. Eventually, we compute the percentage of correct labels obtained by defining the accuracy measure as
(33)Ac(sili)=∑i=1nsilin.To evaluate the classification performance, we carry out computation about the normalized mutual information, which is used to measure how similar two sets of clusters are, as the second measure. Given two data sets of clustersCi and Cj, their normalized mutual information is defined as
(34)NMI(Ci,Cj)=∑ci∈Ci,cj∈Cjp(ci,cj)·log⁡{p(ci,cj)/[p(ci)p(cj)]}max⁡(H(Ci),H(Cj)),
which takes values between 0 and 1. Where p(ci), p(cj) denote the probabilities that an image arbitrarily chosen from the data set belongs to the clusters Ci and Cj, respectively, and p(ci,cj) denotes the joint probability that this arbitrarily selected image belongs to the cluster Ci as well as Cj at the same time.  H(Ci) and H(Cj) are the entropies of Ci and Cj, respectively.Experimental results on each data set will be presented as classification accuracy and the normalized mutual information is in Tables1 and 2.Table 1
The comparison of classification accuracy rates on the ORL Database.
k
Classification accuracy Ac(sili) (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
93.90
92.95
93.51 ± 0.09
92.14 ± 0.18
93.89 ± 0.91
3
86.20
84.33
85.71 ± 0.83
82.26 ± 1.23
86.21 ± 2.07
4
84.55
84.05
84.74 ± 0.56
83.02 ± 0.47
84.78 ± 0.36
5
80.16
78.38
80.10 ± 0.22
80.15 ± 0.18
80.16 ± 0.29
6
76.72
76.73
77.13 ± 0.28
77.60 ± 0.19
78.09 ± 0.40
7
81.14
79.04
80.34 ± 0.81
78.07 ± 0.88
82.15 ± 1.95
8
78.08
77.02
78.66 ± 0.69
78.77 ± 0.94
79.45 ± 1.33
9
83.19
82.23
83.57 ± 0.36
81.22 ± 0.21
85.19 ± 1.21
10
77.03
76.88
77.69 ± 0.12
77.75 ± 0.19
78.96 ± 0.73
Avg.
82.33
81.29
82.38
81.22
83.21Table 2
The comparison of normalized mutual information on the ORL Database.
k
Normalized mutual information (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
78.24
75.82
76.01 ± 0.14
73.94 ± 0.11
77.67 ± 0.58
3
76.61
75.36
75.38 ± 0.41
74.50 ± 0.53
75.92 ± 1.55
4
78.69
77.82
77.96 ± 0.47
77.04 ± 0.54
79.27 ± 1.26
5
76.69
74.27
75.32 ± 0.80
72.62 ± 0.87
75.35 ± 1.67
6
76.52
76.09
76.21 ± 0.12
76.09 ± 0.67
77.60 ± 1.82
7
82.45
81.35
82.36 ± 0.42
81.35 ± 0.26
83.08 ± 1.59
8
80.57
79.92
80.11 ± 0.73
79.91 ± 0.68
82.49 ± 1.97
9
86.03
85.67
86.73 ± 0.70
85.67 ± 0.37
88.05 ± 1.15
10
81.77
81.07
82.07 ± 0.12
81.07 ± 0.12
82.97 ± 1.71
Avg.
79.73
78.60
79.13
78.02
80.27


### 5.1. ORL Database

The Cambridge ORL Face Database has 400 images for 40 different people, 10 images per person. The images of some people are taken at different times, varying lighting slightly, facial expressions (open/closed eyes, smiling/nonsmiling), and facial details (glasses/no glasses). All the images are taken against a dark homogeneous background with the subjects in an upright, frontal position and slight left-right out-of-plane rotation. To locate the faces, the input images are preprocessed. They are resized to32×32 pixels with 256 gray levels per pixel and normalized in orientation so that two eyes in the facial areas are aligned at the same position.There are 10 images for each category in ORL and 10 percent is just one image. For the fixed parameterα (α=0.5,1,2), we randomly choose two images from each category to provide the label information. Note that the same label is meaningless for (30). To obtain r(k), we divide the 40 categories into 3 groups: 10 categories, 20 categories, and 10 categories. In the first 10 categories, pick up 1 image from each category to provide the label information; pick up 2 images from each category in the second 20 categories; and pick up 3 from each category in the remaining categories. The dividing process is repeated 10 times and the obtained average classification accuracy is recorded as the final result.Figure1 shows the graphical classification accuracy rates and normalized mutual information on the ORL Database. Note that if the samples in the collection X come from the same group, we set αk=0.67. Because of the same number of samples in each category, the variation of the α is small even though we label different cardinalities of samples. Compared to the constrained nonnegative matrix factorization algorithms with fixed parameters, our CNMFαk algorithm gives the best performance since the selection of αk is suitable to the collection X. Table 1 summarizes the detailed classification accuracy and error bars of CNMFα=0.5, CNMFα=2, and CNMFαk. It shows that our CNMFαk algorithm achieves 1.92 percent improvement compared to the best reported CNMFKL algorithm (α=1) [15] in average classification accuracy. For normalized mutual information, the details and the error bars of our constrained algorithms with α=0.5, α=2, and αk are listed in Table 2. Comparing to the best algorithm CNMF, our CNMFαk algorithm achieves 0.54 percent improvement.Classification performance on the ORL Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


### 5.2. Yale Database

The Yale Database consists of 165 grayscale images for 15 individuals, 11 images per person. One per image is taken from different facial expression or configuration: center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink. We preprocess all the images of Yale Database in the same way as the ORL Database. Each image is linearly stretched to a 1,024-dimensional vector in image space.The Yale Database also has the same number of samples in each category. To obtain an appropriateα, we do similar label processing to the ORL Database. Divide the 15 individuals into 3 groups averagely, choose 1 image from each category in the first group, choose 2 from each category in the second, and choose 3 from each category in the remaining group. We repeat the process 10 times and record the average classification accuracy as the final result.Figure2 shows the classification accuracy and normalized mutual information on the Yale Database. Set αk=0.55 when r(k)-r(N)=0. It indicates that the samples in the collection X just come from two groups; that is, choose 15 images from 10 categories in the first and second group. CNMFαk achieves an extraordinary performance for all the cases and CNMFα=0.5 follows. This suggests that the constrained nonnegative matrix factorization algorithm has a higher classification accuracy when the value of α is close to αk.  Comparing to the best reported CNMFKL algorithm, CNMFαk achieves 2.42 percent improvement in average classification accuracy. For normalized mutual information, CNMFαk achieves 7.2 percent improvement compared to CNMF. The details of classification accuracy and normalized mutual information are provided in Tables 3 and 4, which contain the error bars of CNMFα=0.5, CNMFα=2, and CNMFαk.Table 3
The comparison of classification accuracy rates on the Yale Database.
k
Ac(sili) (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
81.64
85.23
85.93 ± 0.65
84.18 ± 0.81
86.27 ± 2.46
3
68.48
76.91
77.14 ± 0.86
77.26 ± 0.42
79.01 ± 1.09
4
64.25
66.70
68.11 ± 0.67
66.12 ± 0.16
69.33 ± 2.23
5
65.82
67.71
69.14 ± 1.21
66.54 ± 1.01
70.57 ± 2.22
6
60.12
64.26
65.62 ± 0.94
63.70 ± 0.94
66.92 ± 1.41
7
59.25
62.47
63.97 ± 0.72
59.83 ± 0.38
65.23 ± 1.77
8
54.40
59.26
60.12 ± 0.31
56.76 ± 0.07
60.89 ± 1.28
9
54.85
57.30
58.08 ± 0.12
54.88 ± 0.14
58.98 ± 0.62
10
52.68
56.20
57.56 ± 0.03
53.83 ± 0.01
57.56 ± 0.73
Avg.
62.39
65.89
67.30
64.79
68.31Table 4
The comparison of normalized mutual information on the Yale Database.
k
Normalized mutual information (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
48.99
59.66
59.73 ± 0.39
58.17 ± 0.17
61.11 ± 1.82
3
42.90
53.52
53.58 ± 0.73
52.75 ± 0.93
54.49 ± 1.74
4
47.80
53.60
53.66 ± 0.18
52.99 ± 0.65
54.58 ± 1.64
5
53.99
57.49
57.56 ± 0.07
56.21 ± 0.06
58.58 ± 2.32
6
50.63
55.88
56.15 ± 0.07
55.40 ± 0.08
57.62 ± 1.09
7
53.03
57.60
57.88 ± 0.22
57.27 ± 0.16
59.05 ± 1.89
8
50.83
56.52
56.93 ± 0.13
56.20 ± 0.13
58.98 ± 1.46
9
53.08
55.30
55.98 ± 0.73
54.81 ± 0.81
56.15 ± 1.20
10
52.84
56.73
57.43 ± 0.16
56.41 ± 0.41
58.30 ± 1.11
Avg.
50.45
56.26
56.54
55.58
57.65Classification performance on the Yale Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


### 5.3. Caltech 101 Database

The Caltech 101 Database created by Caltech University has images of 101 different object categories. Each category contains about 31 to 800 images with a total of 9,144 samples of size roughly300×300 pixels. This database is particularly challenging for learning a basis representation of image information, because the number of training samples per category is exceedingly small. In our experiment, we select the 10 largest categories (3,044 images in total), except the background category. To represent the input images, we do the preprocessing by using the codewords generated by SIFT features [34]. Then we obtain 555,292 SIFT descriptors and generate 500 codewords. By assigning the descriptors to the closest codewords, each image in Caltech 101 database is represented by a 500-dimensional frequency histogram.We randomly selectk categories from Faces-Easy category in Caltech 101 database and convert them to gray-scale of 32×32. The label process is repeated 10 times and the obtained values of αk computed by (31) are listed in Table 7. The variation of the α in the same k categories is great. That is, selecting an appropriate α plays a critical role for the mixture of different categories in one image data set, especially in the case that the number of samples in each category is different. The choice of αk can fully reflect the effectiveness of the indicator matrix S.Figure3 shows the effect that derived from the using of proposed algorithm with αk. The upper part of figure is the original samples which contain 26 images, the middle part is their gray images, and the lower is the combination of the basis vectors learned by CNMFαk.Sample images used in YaleB Database. As shown in the three2×13 montages, (a) is the original images, (b) is the gray images, and (c) is the basis vectors learned by CNMFαk. Positive values are illustrated with white pixels.
(a)
(b)
(c)The classification accuracy results and normalized mutual information for Faces-Easy category in Caltech 101 database are detailed in Tables5 and 6, which contain the error bars of CNMFα=0.5, CNMFα=2, and CNMFαk. The graphical results of classification performance are shown in Figure 4. The best performance in this experiment is achieved when the parameters αk listed in Table 7 were selected. In general, our method demonstrates much better effectiveness in classification by choosing αk. Comparing to the best reported algorithm other than our CNMFα algorithm, CNMFαk achieves 2.4 percent improvement in average classification accuracy, and comparing to the CNMF algorithm [15] other than CNMFα algorithm, CNMFαk achieves 8.77 percent improvement in average classification accuracy. For normalized mutual information, CNMFαk achieves 2.29 percent improvement and consistently outperforms the other algorithms.Table 5
The comparison of classification accuracy rates on the Caltech 101 Database.
k
Ac(sili) (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
66.39
78.80
78.31 ± 1.69
80.10 ± 2.07
82.76 ± 3.56
3
60.91
69.98
68.77 ± 1.57
72.94 ± 1.51
73.59 ± 3.51
4
50.82
57.50
58.19 ± 1.98
56.28 ± 1.96
59.89 ± 2.82
5
50.83
53.67
53.71 ± 1.02
54.12 ± 1.44
55.06 ± 2.63
6
50.49
55.05
55.63 ± 0.89
58.04 ± 1.47
58.97 ± 1.97
7
46.09
51.12
50.04 ± 0.11
51.73 ± 0.25
52.36 ± 1.81
8
44.21
50.50
51.72 ± 0.51
48.37 ± 1.59
52.64 ± 1.52
9
41.34
46.58
47.85 ± 0.28
45.87 ± 0.30
49.54 ± 1.23
10
40.80
46.02
46.02 ± 0.05
45.76 ± 0.08
46.02 ± 0.26
Avg.
50.21
56.58
56.69
57.02
58.98Table 6
The comparison of normalized mutual information on the Caltech 101 Database.
k
Normalized mutual information (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
21.05
38.17
38.20 ± 0.18
37.48 ± 0.21
40.08 ± 2.19
3
31.62
40.63
40.68 ± 0.15
40.05 ± 0.14
41.88 ± 1.24
4
27.60
34.36
34.41 ± 0.79
34.12 ± 0.91
36.41 ± 1.17
5
34.46
35.82
36.00 ± 0.99
35.42 ± 1.01
37.20 ± 1.78
6
34.85
38.97
39.16 ± 0.86
38.53 ± 1.11
39.64 ± 1.83
7
34.88
37.67
37.86 ± 0.47
37.47 ± 0.49
39.86 ± 2.58
8
32.63
37.43
37.71 ± 0.39
37.22 ± 0.42
37.91 ± 2.18
9
31.20
35.97
36.21 ± 0.60
35.86 ± 0.63
37.73 ± 0.67
10
31.21
35.71
36.06 ± 0.02
35.68 ± 0.03
37.48 ± 0.22
Avg.
31.05
37.19
37.37
36.87
38.69Table 7
The value ofα in k categories.
k
α
1
α
2
α
3
α
4
α
5
α
6
α
7
α
8
α
9
α
10
2
2.07
2.08
2.22
2.27
2.57
3.52
3.97
3.98
4.37
8.96
3
0.45
0.55
0.66
1.07
1.95
2.31
2.31
3.54
4.24
7.54
4
0.30
0.32
0.35
0.86
0.98
1.27
1.65
2.10
3.27
5.80
5
0.45
0.49
0.51
0.71
0.80
1.13
2.05
2.55
2.82
3.89
6
0.07
0.56
0.73
1.08
1.26
1.46
2.01
2.12
3.91
8.52
7
0.71
0.94
1.12
1.14
1.16
1.26
1.27
1.37
1.57
8.97
8
0.12
0.30
0.50
0.64
0.66
0.69
0.78
0.79
0.97
1.11
9
0.78
0.79
2.01
0.10
0.17
0.28
0.29
0.30
0.31
0.39
10
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00Classification performance on the Caltech 101 Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


## 5.1. ORL Database

The Cambridge ORL Face Database has 400 images for 40 different people, 10 images per person. The images of some people are taken at different times, varying lighting slightly, facial expressions (open/closed eyes, smiling/nonsmiling), and facial details (glasses/no glasses). All the images are taken against a dark homogeneous background with the subjects in an upright, frontal position and slight left-right out-of-plane rotation. To locate the faces, the input images are preprocessed. They are resized to32×32 pixels with 256 gray levels per pixel and normalized in orientation so that two eyes in the facial areas are aligned at the same position.There are 10 images for each category in ORL and 10 percent is just one image. For the fixed parameterα (α=0.5,1,2), we randomly choose two images from each category to provide the label information. Note that the same label is meaningless for (30). To obtain r(k), we divide the 40 categories into 3 groups: 10 categories, 20 categories, and 10 categories. In the first 10 categories, pick up 1 image from each category to provide the label information; pick up 2 images from each category in the second 20 categories; and pick up 3 from each category in the remaining categories. The dividing process is repeated 10 times and the obtained average classification accuracy is recorded as the final result.Figure1 shows the graphical classification accuracy rates and normalized mutual information on the ORL Database. Note that if the samples in the collection X come from the same group, we set αk=0.67. Because of the same number of samples in each category, the variation of the α is small even though we label different cardinalities of samples. Compared to the constrained nonnegative matrix factorization algorithms with fixed parameters, our CNMFαk algorithm gives the best performance since the selection of αk is suitable to the collection X. Table 1 summarizes the detailed classification accuracy and error bars of CNMFα=0.5, CNMFα=2, and CNMFαk. It shows that our CNMFαk algorithm achieves 1.92 percent improvement compared to the best reported CNMFKL algorithm (α=1) [15] in average classification accuracy. For normalized mutual information, the details and the error bars of our constrained algorithms with α=0.5, α=2, and αk are listed in Table 2. Comparing to the best algorithm CNMF, our CNMFαk algorithm achieves 0.54 percent improvement.Classification performance on the ORL Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


## 5.2. Yale Database

The Yale Database consists of 165 grayscale images for 15 individuals, 11 images per person. One per image is taken from different facial expression or configuration: center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink. We preprocess all the images of Yale Database in the same way as the ORL Database. Each image is linearly stretched to a 1,024-dimensional vector in image space.The Yale Database also has the same number of samples in each category. To obtain an appropriateα, we do similar label processing to the ORL Database. Divide the 15 individuals into 3 groups averagely, choose 1 image from each category in the first group, choose 2 from each category in the second, and choose 3 from each category in the remaining group. We repeat the process 10 times and record the average classification accuracy as the final result.Figure2 shows the classification accuracy and normalized mutual information on the Yale Database. Set αk=0.55 when r(k)-r(N)=0. It indicates that the samples in the collection X just come from two groups; that is, choose 15 images from 10 categories in the first and second group. CNMFαk achieves an extraordinary performance for all the cases and CNMFα=0.5 follows. This suggests that the constrained nonnegative matrix factorization algorithm has a higher classification accuracy when the value of α is close to αk.  Comparing to the best reported CNMFKL algorithm, CNMFαk achieves 2.42 percent improvement in average classification accuracy. For normalized mutual information, CNMFαk achieves 7.2 percent improvement compared to CNMF. The details of classification accuracy and normalized mutual information are provided in Tables 3 and 4, which contain the error bars of CNMFα=0.5, CNMFα=2, and CNMFαk.Table 3
The comparison of classification accuracy rates on the Yale Database.
k
Ac(sili) (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
81.64
85.23
85.93 ± 0.65
84.18 ± 0.81
86.27 ± 2.46
3
68.48
76.91
77.14 ± 0.86
77.26 ± 0.42
79.01 ± 1.09
4
64.25
66.70
68.11 ± 0.67
66.12 ± 0.16
69.33 ± 2.23
5
65.82
67.71
69.14 ± 1.21
66.54 ± 1.01
70.57 ± 2.22
6
60.12
64.26
65.62 ± 0.94
63.70 ± 0.94
66.92 ± 1.41
7
59.25
62.47
63.97 ± 0.72
59.83 ± 0.38
65.23 ± 1.77
8
54.40
59.26
60.12 ± 0.31
56.76 ± 0.07
60.89 ± 1.28
9
54.85
57.30
58.08 ± 0.12
54.88 ± 0.14
58.98 ± 0.62
10
52.68
56.20
57.56 ± 0.03
53.83 ± 0.01
57.56 ± 0.73
Avg.
62.39
65.89
67.30
64.79
68.31Table 4
The comparison of normalized mutual information on the Yale Database.
k
Normalized mutual information (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
48.99
59.66
59.73 ± 0.39
58.17 ± 0.17
61.11 ± 1.82
3
42.90
53.52
53.58 ± 0.73
52.75 ± 0.93
54.49 ± 1.74
4
47.80
53.60
53.66 ± 0.18
52.99 ± 0.65
54.58 ± 1.64
5
53.99
57.49
57.56 ± 0.07
56.21 ± 0.06
58.58 ± 2.32
6
50.63
55.88
56.15 ± 0.07
55.40 ± 0.08
57.62 ± 1.09
7
53.03
57.60
57.88 ± 0.22
57.27 ± 0.16
59.05 ± 1.89
8
50.83
56.52
56.93 ± 0.13
56.20 ± 0.13
58.98 ± 1.46
9
53.08
55.30
55.98 ± 0.73
54.81 ± 0.81
56.15 ± 1.20
10
52.84
56.73
57.43 ± 0.16
56.41 ± 0.41
58.30 ± 1.11
Avg.
50.45
56.26
56.54
55.58
57.65Classification performance on the Yale Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


## 5.3. Caltech 101 Database

The Caltech 101 Database created by Caltech University has images of 101 different object categories. Each category contains about 31 to 800 images with a total of 9,144 samples of size roughly300×300 pixels. This database is particularly challenging for learning a basis representation of image information, because the number of training samples per category is exceedingly small. In our experiment, we select the 10 largest categories (3,044 images in total), except the background category. To represent the input images, we do the preprocessing by using the codewords generated by SIFT features [34]. Then we obtain 555,292 SIFT descriptors and generate 500 codewords. By assigning the descriptors to the closest codewords, each image in Caltech 101 database is represented by a 500-dimensional frequency histogram.We randomly selectk categories from Faces-Easy category in Caltech 101 database and convert them to gray-scale of 32×32. The label process is repeated 10 times and the obtained values of αk computed by (31) are listed in Table 7. The variation of the α in the same k categories is great. That is, selecting an appropriate α plays a critical role for the mixture of different categories in one image data set, especially in the case that the number of samples in each category is different. The choice of αk can fully reflect the effectiveness of the indicator matrix S.Figure3 shows the effect that derived from the using of proposed algorithm with αk. The upper part of figure is the original samples which contain 26 images, the middle part is their gray images, and the lower is the combination of the basis vectors learned by CNMFαk.Sample images used in YaleB Database. As shown in the three2×13 montages, (a) is the original images, (b) is the gray images, and (c) is the basis vectors learned by CNMFαk. Positive values are illustrated with white pixels.
(a)
(b)
(c)The classification accuracy results and normalized mutual information for Faces-Easy category in Caltech 101 database are detailed in Tables5 and 6, which contain the error bars of CNMFα=0.5, CNMFα=2, and CNMFαk. The graphical results of classification performance are shown in Figure 4. The best performance in this experiment is achieved when the parameters αk listed in Table 7 were selected. In general, our method demonstrates much better effectiveness in classification by choosing αk. Comparing to the best reported algorithm other than our CNMFα algorithm, CNMFαk achieves 2.4 percent improvement in average classification accuracy, and comparing to the CNMF algorithm [15] other than CNMFα algorithm, CNMFαk achieves 8.77 percent improvement in average classification accuracy. For normalized mutual information, CNMFαk achieves 2.29 percent improvement and consistently outperforms the other algorithms.Table 5
The comparison of classification accuracy rates on the Caltech 101 Database.
k
Ac(sili) (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
66.39
78.80
78.31 ± 1.69
80.10 ± 2.07
82.76 ± 3.56
3
60.91
69.98
68.77 ± 1.57
72.94 ± 1.51
73.59 ± 3.51
4
50.82
57.50
58.19 ± 1.98
56.28 ± 1.96
59.89 ± 2.82
5
50.83
53.67
53.71 ± 1.02
54.12 ± 1.44
55.06 ± 2.63
6
50.49
55.05
55.63 ± 0.89
58.04 ± 1.47
58.97 ± 1.97
7
46.09
51.12
50.04 ± 0.11
51.73 ± 0.25
52.36 ± 1.81
8
44.21
50.50
51.72 ± 0.51
48.37 ± 1.59
52.64 ± 1.52
9
41.34
46.58
47.85 ± 0.28
45.87 ± 0.30
49.54 ± 1.23
10
40.80
46.02
46.02 ± 0.05
45.76 ± 0.08
46.02 ± 0.26
Avg.
50.21
56.58
56.69
57.02
58.98Table 6
The comparison of normalized mutual information on the Caltech 101 Database.
k
Normalized mutual information (%)
CNMF
CNMF
α
=
1
CNMF
α
=
0.5
CNMF
α
=
2
CNMF
α
k
2
21.05
38.17
38.20 ± 0.18
37.48 ± 0.21
40.08 ± 2.19
3
31.62
40.63
40.68 ± 0.15
40.05 ± 0.14
41.88 ± 1.24
4
27.60
34.36
34.41 ± 0.79
34.12 ± 0.91
36.41 ± 1.17
5
34.46
35.82
36.00 ± 0.99
35.42 ± 1.01
37.20 ± 1.78
6
34.85
38.97
39.16 ± 0.86
38.53 ± 1.11
39.64 ± 1.83
7
34.88
37.67
37.86 ± 0.47
37.47 ± 0.49
39.86 ± 2.58
8
32.63
37.43
37.71 ± 0.39
37.22 ± 0.42
37.91 ± 2.18
9
31.20
35.97
36.21 ± 0.60
35.86 ± 0.63
37.73 ± 0.67
10
31.21
35.71
36.06 ± 0.02
35.68 ± 0.03
37.48 ± 0.22
Avg.
31.05
37.19
37.37
36.87
38.69Table 7
The value ofα in k categories.
k
α
1
α
2
α
3
α
4
α
5
α
6
α
7
α
8
α
9
α
10
2
2.07
2.08
2.22
2.27
2.57
3.52
3.97
3.98
4.37
8.96
3
0.45
0.55
0.66
1.07
1.95
2.31
2.31
3.54
4.24
7.54
4
0.30
0.32
0.35
0.86
0.98
1.27
1.65
2.10
3.27
5.80
5
0.45
0.49
0.51
0.71
0.80
1.13
2.05
2.55
2.82
3.89
6
0.07
0.56
0.73
1.08
1.26
1.46
2.01
2.12
3.91
8.52
7
0.71
0.94
1.12
1.14
1.16
1.26
1.27
1.37
1.57
8.97
8
0.12
0.30
0.50
0.64
0.66
0.69
0.78
0.79
0.97
1.11
9
0.78
0.79
2.01
0.10
0.17
0.28
0.29
0.30
0.31
0.39
10
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00Classification performance on the Caltech 101 Database.
(a)
Classification accuracy rates
(b)
Normalized mutual information


## 6. Conclusion

In this paper, we present a generic constraint nonnegative matrix factorization algorithm by imposing label information as additional hard constraint to theα-divergence-NMF algorithm. The proposed algorithm can be derived using Karush-Kuhn-Tucker conditions and presented alternative of the algorithm using the projected gradient. The use of the two methods guarantees the correctness of the algorithm theoretically. The image representation learned by our algorithm contains a parameter α. Since α-divergence is a parametric discrepancy measure and the parameter α is associated with the characteristics of a learning machine, the model distribution is more inclusive when α goes to +∞ and is more exclusive when α approaches -∞. The selection of the optimal value of α plays a critical role in determining the discriminative basis vectors. We provide a method to select the parameters α for our semisupervised CNMFα algorithm. The variation of the α is caused by both the cardinalities of labeled samples in each category and the total samples. In the experiments, we apply the fixed parameters α and αk to analyze the classification accuracy on three image databases. The experimental results have demonstrated that the CNMFαk algorithm has best classification accuracy. However, we can not get the cardinalities of labeled images exactly in many real-word applications. Moreover, the value of α varies depending on data sets. It is still an open problem how to select the optimal α for a specific image data set.



---

*Source: 179129-2014-10-22.xml*