# Gene Feature Extraction Based on Nonnegative Dual Graph Regularized Latent Low-Rank Representation

Authors: Guoliang Yang; Zhengwei Hu
Journal: BioMed Research International
 (2017)

---

## Abstract

Aiming at the problem of gene expression profile’s high redundancy and heavy noise, a new feature extraction model based on nonnegative dual graph regularized latent low-rank representation (NNDGLLRR) is presented on the basis of latent low-rank representation (Lat-LRR). By introducing dual graph manifold regularized constraint, the NNDGLLRR can keep the internal spatial structure of the original data effectively and improve the final clustering accuracy while segmenting the subspace. The introduction of nonnegative constraints makes the computation with some sparsity, which enhances the robustness of the algorithm. Different from Lat-LRR, a new solution model is adopted to simplify the computational complexity. The experimental results show that the proposed algorithm has good feature extraction performance for the heavy redundancy and noise gene expression profile, which, compared with LRR and Lat-LRR, can achieve better clustering accuracy.


---

## Body

## 1. Introduction

With the accelerated pace of modern life, the high incidence of cancer has brought great challenges to human health. How to detect, prevent, and treat cancer effectively has become an international hotspot of medical research. Gene expression profile is a specific cDNA sequence data of cells, which can describe cells’ current physiological function and state. Researches show that tumor cells and normal cells could be identified effectively by analyzing and processing the original gene expression data. However, the scale of the gene expression profile is huge and complex due to the diversity and specificity of the cells; therefore the traditional methods of data analysis and processing have been unable to adapt to these extremely large-scale data.Gene expression profile extracting includes two kinds of methods: linear and nonlinear. Early linear transformation methods include principal component analysis [1–3] (PCA), linear discriminant analysis [4–6] (LDA), and independent component analysis [7, 8] (ICA). The main methods of nonlinear transformation include kernel method [9], neural network [10, 11], manifold learning [12, 13], and sparse representation [14, 15]. In recent years, LRR [16–18] and neural networks have been widely used in feature extraction and classification of gene expression profile. Reference [19] used NMF for gene feature extraction and achieved more satisfactory results. Ref. [20] proposed a gene expression profile classification means based on ontology perception. Ref. [21] proposed a subcellular cooccurrence matrix feature extraction method. Ref. [22] proposed a gene expression profile classification method by neural network hybrid back-propagation. Ref. [23] proposed a supervised way of tumor prediction with multiview.The size of the gene expression profile is large, and there are interrelationships between the samples. The internal spatial structure of the data may be destroyed in the process of linear transformation. In this paper, a model of feature extraction based on NNDGLLRR is proposed on the basis of Lat-LRR, which with low-rank sparse constraint can remove the redundant components of gene expression and suppress the noise. Nonnegative constraints make the calculation with a certain degree of sparsity, in line with the practical significance of the data, and enhance the robustness of the algorithm. And the manifold regularized constraint is introduced, so that the result of feature extraction can describe the spatial structure of the original data more completely.


## 2. Related Work

### 2.1. LRR

LRR is a combination of matrix low-rank decomposition and sparse decomposition. In recent years, it has been widely used in subspace clustering. LRR assumes that the original data comes from different subspaces and performs feature extraction by trying to find the lowest rank representation of the original data. And this low-rank representation coefficient is the reflection of the original data in the spatial distribution of structural information. If the original dataX=x1,x2,x3,…,xn∈Rm×n, each column xi represents a sample, and generally the LRR uses the data itself as a dictionary. Then the model can be as shown in(1)O1=minZ∗+λE2,1s.t.X=XZ+E.The LRR matrixZ=z1,z2,z3,…,zn∈Rn×n, and zi is the linear representation coefficient of the sample xi under the data dictionary X. The original data usually contains a lot of noise, while the sparse constraint can maintain the robustness of the algorithm effectively. Ref. [24] shows the specific solution process of LRR.LetZ=J; we construct the following Augmented Lagrangian function:(2)L=J∗+λE2,1+Λ,Z-J+Π,X-XZ-E+μ2Z-JF2+X-XZ-EF2.The specific update algorithm is as follows.KeepZ=Zk, Λ=Λk; update J:(3)Jk+1=argminJJk∗+μk2Λkμk+Zk-JkF2.Keep J=Jk,  Λ=Λk, and  Π=Πk; update Z:(4)Zk+1=argminZΛkμk+Zk-JkF2+Πkμk+X-XZk-EkF2.Keep Z=Zk, Π=Πk; update E:(5)Ek+1=argminEλEk2,1+μk2Πkμk+X-XZk-EkF2.


### 2.2. Lat-LRR

LRR has two conditions; one is that the original dataX contains enough samples, and the other is that X contains enough nonpolluting data. However, these two conditions are almost impossible to achieve for gene data. On the one hand, the available number of gene samples for research is small because of the high prices of gene sequencing. On the other hand, due to process, instrument electromagnetic interference, and other factors, noise pollution will be produced inevitably in the process of genetic sequencing. To overcome the limitation of LRR, [25] proposed a method of Lat-LRR which expressed the original observation data X as a linear combination of principal feature XZ and latent feature LX for feature extraction. Considering the characteristics of heavy noise in gene expression profile, we added sparsity constraints to the model to construct the following Lat-LRR function:(6)O2=minZ∗+L∗+λE2,1s.t.X=XZ+LX+E.The solution of Lat-LRR is given in [26]. Alternating direction method (ADM) is adopted to solve the model (6). Let Z=J1,  L=J2; we constructed the following Augmented Lagrangian function:(7)L=J1∗+J2∗+λE2,1+Λ,Z-J1+Π,L-J2+Δ,X-XZ-LX-E+μ2Z-J1F2+L-J2F2+X-XZ-LX-EF2.KeepZ=Zk and  Λ=Λk; update J1:(8)J1k+1=argminJ1J1k∗+μk2Λkμk+Zk-J1kF2.KeepL=Lk,  Π=Πk; update J2:(9)J2k+1=argminJJ2k∗+μk2Πkμk+Lk-J2kF2.KeepJ1=J1k,  L=Lk,  E=Ek,  Λ=Λk, and  Δ=Δk; update Z:(10)Zk+1=I+XTX-1XTX-LkX-Ek+J1k+XTΔk-Λkμk.KeepZ=Zk,  E=Ek,  Π=Πk, and  Δ=Δk; update L:(11)Lk+1=X-XZk-EkXT+J2k+ΔkXT-ΠkμkI+XXT-1.KeepZ=Zk,  L=Lk; update E:(12)Ek+1=argminEλEk2,1+μk2X-XZk-LkXk-EkF2.


## 2.1. LRR

LRR is a combination of matrix low-rank decomposition and sparse decomposition. In recent years, it has been widely used in subspace clustering. LRR assumes that the original data comes from different subspaces and performs feature extraction by trying to find the lowest rank representation of the original data. And this low-rank representation coefficient is the reflection of the original data in the spatial distribution of structural information. If the original dataX=x1,x2,x3,…,xn∈Rm×n, each column xi represents a sample, and generally the LRR uses the data itself as a dictionary. Then the model can be as shown in(1)O1=minZ∗+λE2,1s.t.X=XZ+E.The LRR matrixZ=z1,z2,z3,…,zn∈Rn×n, and zi is the linear representation coefficient of the sample xi under the data dictionary X. The original data usually contains a lot of noise, while the sparse constraint can maintain the robustness of the algorithm effectively. Ref. [24] shows the specific solution process of LRR.LetZ=J; we construct the following Augmented Lagrangian function:(2)L=J∗+λE2,1+Λ,Z-J+Π,X-XZ-E+μ2Z-JF2+X-XZ-EF2.The specific update algorithm is as follows.KeepZ=Zk, Λ=Λk; update J:(3)Jk+1=argminJJk∗+μk2Λkμk+Zk-JkF2.Keep J=Jk,  Λ=Λk, and  Π=Πk; update Z:(4)Zk+1=argminZΛkμk+Zk-JkF2+Πkμk+X-XZk-EkF2.Keep Z=Zk, Π=Πk; update E:(5)Ek+1=argminEλEk2,1+μk2Πkμk+X-XZk-EkF2.


## 2.2. Lat-LRR

LRR has two conditions; one is that the original dataX contains enough samples, and the other is that X contains enough nonpolluting data. However, these two conditions are almost impossible to achieve for gene data. On the one hand, the available number of gene samples for research is small because of the high prices of gene sequencing. On the other hand, due to process, instrument electromagnetic interference, and other factors, noise pollution will be produced inevitably in the process of genetic sequencing. To overcome the limitation of LRR, [25] proposed a method of Lat-LRR which expressed the original observation data X as a linear combination of principal feature XZ and latent feature LX for feature extraction. Considering the characteristics of heavy noise in gene expression profile, we added sparsity constraints to the model to construct the following Lat-LRR function:(6)O2=minZ∗+L∗+λE2,1s.t.X=XZ+LX+E.The solution of Lat-LRR is given in [26]. Alternating direction method (ADM) is adopted to solve the model (6). Let Z=J1,  L=J2; we constructed the following Augmented Lagrangian function:(7)L=J1∗+J2∗+λE2,1+Λ,Z-J1+Π,L-J2+Δ,X-XZ-LX-E+μ2Z-J1F2+L-J2F2+X-XZ-LX-EF2.KeepZ=Zk and  Λ=Λk; update J1:(8)J1k+1=argminJ1J1k∗+μk2Λkμk+Zk-J1kF2.KeepL=Lk,  Π=Πk; update J2:(9)J2k+1=argminJJ2k∗+μk2Πkμk+Lk-J2kF2.KeepJ1=J1k,  L=Lk,  E=Ek,  Λ=Λk, and  Δ=Δk; update Z:(10)Zk+1=I+XTX-1XTX-LkX-Ek+J1k+XTΔk-Λkμk.KeepZ=Zk,  E=Ek,  Π=Πk, and  Δ=Δk; update L:(11)Lk+1=X-XZk-EkXT+J2k+ΔkXT-ΠkμkI+XXT-1.KeepZ=Zk,  L=Lk; update E:(12)Ek+1=argminEλEk2,1+μk2X-XZk-LkXk-EkF2.


## 3. Method

### 3.1. NNDGLLRR

Lat-LRR overcomes the problem of too many constraints of LRR dictionary; however, Lat-LRR has limited ability to recover the subspace, and too many auxiliary variables are involved in the process of algorithm solving that involves a lot of matrix singularity value decomposition (SVD) and matrix inversion, which will affect the performance of the algorithm. Ref. [27] proposed a feature extraction method combining manifold constraint and nonnegative matrix factorization (NMF). In the case of NMF reducing dimensionality, the internal spatial structure of the data is maintained by manifold regularized constraint, and good experimental results are obtained. Ref. [28, 29] proposed an image clustering method combining manifold regularized constraint with Lat-LRR. Similar to the image data, the gene expression profile is also constituted by numerical matrix with high redundancy and heavy noise. Considering this characteristic, we constructed a new NNDGLLRR model on the basis of the original model.(13)O3=minZ⁡Z∗+L∗+α2TrZS1ZT+β2TrLS2LT+λE2,1s.t.X=LXZ+E,Z≥0,L≥0,where α, β, and λ are nonnegative constants; the model is a nonnegative latent low-rank representation (NNLLRR) when α and β are equal to zero. Model (13) takes a more general form. The dual regularized constraint is used to preserve the internal spatial structure of the original data, and sparse constraints and nonnegative constraints are used to maintain and enhance the robustness of the algorithm. S1 and S2 are Laplacian matrices, S1=D1-W1,  S2=D2-W2. W1, and W2 are weight matrix, and there are many ways to solve W, and here we use Gaussian thermal weight. The specific solution is as follows:(14)W1ij=e-xi-xjF2/σ;i,j=1,2,3,…,nW2ij=e-xiT-xjTF2/σ;i,j=1,2,3,…,m,where σ is a constant; xi and xj represent the ith column and jth column of  X (ith and jth sample); xi and xj represent the ith row and the jth row of  X,  Dij=∑jWij.ADM is used to solve model (12), and the following augmented Lagrange function is constructed:(15)L=Z∗+L∗+α2TrZS1ZT+β2TrLS2LT+λE2,1+μ2Λμ+X-LXZ-EF2,where Λ is a Lagrangian multiplier; μ is a constant and μ>0.Data in real life is generally nonnegative, and nonnegative constraints will make the calculation with a certain degree of sparseness and enhance the robustness of the algorithm. To maintain the nonnegative of feature extraction, we define the following operators:(16)Paij=aij;ifaij>00;otherwise.The solution of model (15) is divided into three subproblems: first, the solution of variable Z, second, the solution of variables L, and, third, the solution variable of  E.(1) Solving the First Subproblem. Update Z:(17)Zk+1=argminZZ∗+α2TrZS1ZT+μk2Λkμk+X-LkXZ-EkF2.Regarding Taylor second-order expansion to (17), the approximate solution of Z is as follows:(18)Zk+1=argminZZ∗+α2TrZS1ZT+ηZμk2Z-Zk-1ηZμkXTLkTΠkF2=argminZZ∗+ηZμk+αS12Z-Zk-1ηZμk+αS1XTLkTΠk-αZkS1F2=D1/ωZZk+1ωZXTLkTΠk-αZkS1.Nonnegative constraints toZ are as follows:(19)Zk+1ij=PZk+1ij.DefineηZ=∂2h/∂Z2;  h=μk/2Λk/μk+X-LkXZk-EkF2;  ωZ=ηZμk+αS1;  Πk=Λk+μkX-LkXZk-Ek. Ref. [30] gives the solution of Dε·; the solution process is as follows:(20)Dεφ=USεΩVT=argminTεΤ∗+12Τ-φF2.In (20), UΩVT is the singular value decomposition (SVD) of φ, Sε· is the vector form of the singular value contraction operator (SVT), and SεΩ is defined as follows:(21)SεΩ=diagsgnΩiiΩii-ε.(2) Solving the Second Subproblem. Similarly, update  L:(22)Lk+1=argminLL∗+β2TrLS2LT+ηLμk2L-Lk-ΠkZkTXTF2=argminLL∗+ηLμk+βS22L-Lk-1ηLμk+βS2ΠkZkTXT-βLkS2F2=D1/ωLLk+1ωLΠkZkTXT-βLkS2.Nonnegative constraints toL are as follows:(23)Lk+1ij=PLk+1ij.DefineηL=∂2h/∂L2;  ωL=ηLμk+βS2.(3) Solving the Third Subproblem. Update E:(24)Ek+1=argminEλE2,1+Λk,X-LkXZk-E+μk2X-LkXZk-EF2=λE2,1+μk2Λkμk+X-LkXZk-EF2=Θλ/μkΛkμk+X-LkXZk,where Θτ· is a soft threshold operator (ST); Θτ· is defined as follows:(25)Θτψ=sgnψmax⁡ψ-τ,0.The iterative process of each variable of NNDGLLRR is given above. The concrete updating process is shown in Algorithm1.Algorithm 1:Solving NNDGLLRR model with ALM.
Input:X∈Rm×n,  α>0,  β>0,  γ>0
Initialization:Z0∈Rn×n,  L0∈Rm×m,  E0=Λ0=Π0=0,   μ0,μmax,ρ,ε1,ε2
While:max⁡Zk+1-Zk∞,Lk+1-Lk∞>ε1 and X-Lk+1XZk+1-Ek+1∞>ε2 do
(1) update Πk: Πk=Λk+μkX-LkXZk-Ek
(2) update Zk+1: Zk+1ij=PZk+1ij
(3) update Lk+1: Lk+1ij=PLk+1ij
(4) update Ek+1: Ek+1=Θλ/μkΛk/μk+X-LkXZk
(5) update Λk+1: Λk+1=Λk+μkX-LkXZk-Ek
(6) update μk+1: μk+1=min⁡(μmax,ρμk)
End while
Output:L∗


### 3.2. Sparse Representation Classifier (SRC)

Sparse representation is a hotspot in the field of pattern recognition in recent years. SRC has been successfully applied in the field of image classification and has achieved relatively ideal experimental results [31]. Similar to the image data, the gene expression profile is also composed by a series of high redundancy and heavy noise of gene samples. In this paper, the latent features extracted by NNDGLLRR are regarded as data dictionary to construct the following SRC model:(26)O4=argminζDζ-L∗y2+γζ1.According to the result of SRC, we can get the classification result of unknown gene sampley:(27)i∗=argminiDδiζ-L∗y2.The detailed flow of the SRC is shown in Algorithm2.Algorithm 2:The flow of SRC.
Input:L∗∈Rm×m
(1) Compute D=L∗X
(2) Compute the sparse representation coefficient
by Eq. (26);
(3) Compute ei(y)=Dδiζ-L∗y2;
(4) i∗=argminiei(y);
Output:i∗


### 3.3. Algorithm Flow

To sum up, the algorithm can be divided into two parts; one is to use NNDGLLRR to extract latent features of the original gene expression profile, and the other is to use SRC to classify the latent features. The overall flow is as shown in Algorithm3.Algorithm 3:Algorithm flow.
Input:X∈Rm×n
(1) Compute L∗ by Eq. (13);
(2) classify the gene expression profile Eq. (27)
Output:i∗


## 3.1. NNDGLLRR

Lat-LRR overcomes the problem of too many constraints of LRR dictionary; however, Lat-LRR has limited ability to recover the subspace, and too many auxiliary variables are involved in the process of algorithm solving that involves a lot of matrix singularity value decomposition (SVD) and matrix inversion, which will affect the performance of the algorithm. Ref. [27] proposed a feature extraction method combining manifold constraint and nonnegative matrix factorization (NMF). In the case of NMF reducing dimensionality, the internal spatial structure of the data is maintained by manifold regularized constraint, and good experimental results are obtained. Ref. [28, 29] proposed an image clustering method combining manifold regularized constraint with Lat-LRR. Similar to the image data, the gene expression profile is also constituted by numerical matrix with high redundancy and heavy noise. Considering this characteristic, we constructed a new NNDGLLRR model on the basis of the original model.(13)O3=minZ⁡Z∗+L∗+α2TrZS1ZT+β2TrLS2LT+λE2,1s.t.X=LXZ+E,Z≥0,L≥0,where α, β, and λ are nonnegative constants; the model is a nonnegative latent low-rank representation (NNLLRR) when α and β are equal to zero. Model (13) takes a more general form. The dual regularized constraint is used to preserve the internal spatial structure of the original data, and sparse constraints and nonnegative constraints are used to maintain and enhance the robustness of the algorithm. S1 and S2 are Laplacian matrices, S1=D1-W1,  S2=D2-W2. W1, and W2 are weight matrix, and there are many ways to solve W, and here we use Gaussian thermal weight. The specific solution is as follows:(14)W1ij=e-xi-xjF2/σ;i,j=1,2,3,…,nW2ij=e-xiT-xjTF2/σ;i,j=1,2,3,…,m,where σ is a constant; xi and xj represent the ith column and jth column of  X (ith and jth sample); xi and xj represent the ith row and the jth row of  X,  Dij=∑jWij.ADM is used to solve model (12), and the following augmented Lagrange function is constructed:(15)L=Z∗+L∗+α2TrZS1ZT+β2TrLS2LT+λE2,1+μ2Λμ+X-LXZ-EF2,where Λ is a Lagrangian multiplier; μ is a constant and μ>0.Data in real life is generally nonnegative, and nonnegative constraints will make the calculation with a certain degree of sparseness and enhance the robustness of the algorithm. To maintain the nonnegative of feature extraction, we define the following operators:(16)Paij=aij;ifaij>00;otherwise.The solution of model (15) is divided into three subproblems: first, the solution of variable Z, second, the solution of variables L, and, third, the solution variable of  E.(1) Solving the First Subproblem. Update Z:(17)Zk+1=argminZZ∗+α2TrZS1ZT+μk2Λkμk+X-LkXZ-EkF2.Regarding Taylor second-order expansion to (17), the approximate solution of Z is as follows:(18)Zk+1=argminZZ∗+α2TrZS1ZT+ηZμk2Z-Zk-1ηZμkXTLkTΠkF2=argminZZ∗+ηZμk+αS12Z-Zk-1ηZμk+αS1XTLkTΠk-αZkS1F2=D1/ωZZk+1ωZXTLkTΠk-αZkS1.Nonnegative constraints toZ are as follows:(19)Zk+1ij=PZk+1ij.DefineηZ=∂2h/∂Z2;  h=μk/2Λk/μk+X-LkXZk-EkF2;  ωZ=ηZμk+αS1;  Πk=Λk+μkX-LkXZk-Ek. Ref. [30] gives the solution of Dε·; the solution process is as follows:(20)Dεφ=USεΩVT=argminTεΤ∗+12Τ-φF2.In (20), UΩVT is the singular value decomposition (SVD) of φ, Sε· is the vector form of the singular value contraction operator (SVT), and SεΩ is defined as follows:(21)SεΩ=diagsgnΩiiΩii-ε.(2) Solving the Second Subproblem. Similarly, update  L:(22)Lk+1=argminLL∗+β2TrLS2LT+ηLμk2L-Lk-ΠkZkTXTF2=argminLL∗+ηLμk+βS22L-Lk-1ηLμk+βS2ΠkZkTXT-βLkS2F2=D1/ωLLk+1ωLΠkZkTXT-βLkS2.Nonnegative constraints toL are as follows:(23)Lk+1ij=PLk+1ij.DefineηL=∂2h/∂L2;  ωL=ηLμk+βS2.(3) Solving the Third Subproblem. Update E:(24)Ek+1=argminEλE2,1+Λk,X-LkXZk-E+μk2X-LkXZk-EF2=λE2,1+μk2Λkμk+X-LkXZk-EF2=Θλ/μkΛkμk+X-LkXZk,where Θτ· is a soft threshold operator (ST); Θτ· is defined as follows:(25)Θτψ=sgnψmax⁡ψ-τ,0.The iterative process of each variable of NNDGLLRR is given above. The concrete updating process is shown in Algorithm1.Algorithm 1:Solving NNDGLLRR model with ALM.
Input:X∈Rm×n,  α>0,  β>0,  γ>0
Initialization:Z0∈Rn×n,  L0∈Rm×m,  E0=Λ0=Π0=0,   μ0,μmax,ρ,ε1,ε2
While:max⁡Zk+1-Zk∞,Lk+1-Lk∞>ε1 and X-Lk+1XZk+1-Ek+1∞>ε2 do
(1) update Πk: Πk=Λk+μkX-LkXZk-Ek
(2) update Zk+1: Zk+1ij=PZk+1ij
(3) update Lk+1: Lk+1ij=PLk+1ij
(4) update Ek+1: Ek+1=Θλ/μkΛk/μk+X-LkXZk
(5) update Λk+1: Λk+1=Λk+μkX-LkXZk-Ek
(6) update μk+1: μk+1=min⁡(μmax,ρμk)
End while
Output:L∗


## 3.2. Sparse Representation Classifier (SRC)

Sparse representation is a hotspot in the field of pattern recognition in recent years. SRC has been successfully applied in the field of image classification and has achieved relatively ideal experimental results [31]. Similar to the image data, the gene expression profile is also composed by a series of high redundancy and heavy noise of gene samples. In this paper, the latent features extracted by NNDGLLRR are regarded as data dictionary to construct the following SRC model:(26)O4=argminζDζ-L∗y2+γζ1.According to the result of SRC, we can get the classification result of unknown gene sampley:(27)i∗=argminiDδiζ-L∗y2.The detailed flow of the SRC is shown in Algorithm2.Algorithm 2:The flow of SRC.
Input:L∗∈Rm×m
(1) Compute D=L∗X
(2) Compute the sparse representation coefficient
by Eq. (26);
(3) Compute ei(y)=Dδiζ-L∗y2;
(4) i∗=argminiei(y);
Output:i∗


## 3.3. Algorithm Flow

To sum up, the algorithm can be divided into two parts; one is to use NNDGLLRR to extract latent features of the original gene expression profile, and the other is to use SRC to classify the latent features. The overall flow is as shown in Algorithm3.Algorithm 3:Algorithm flow.
Input:X∈Rm×n
(1) Compute L∗ by Eq. (13);
(2) classify the gene expression profile Eq. (27)
Output:i∗


## 4. Results and Discussion

### 4.1. Selecting the Test Data

To test the feature extraction performance of the algorithm, we used diffuse large B-cell lymphoma [32] (DLBCL), mixed lineage leukemia [33] (MLL), lung cancer [34] (LC), acute lymphoblastic leukemia [35] (ALL) gene sequences to make test, and the sample information of each group of genes as is shown in Table 1.Table 1
Test data information.
Data name
Size
Classes
Number
DLBCL
5469 × 77
2
77
MLL
12582 × 72
3
72
LC
12600 × 203
5
203
ALL
12626 × 248
6
248


### 4.2. Accuracy Test

K-means and sparse representation classifier (SRC) are simple and common classifiers. To compare the clustering results of K-means and SRC, the two kinds of classifiers are used to classify the original gene expression profile. Clustering results are shown in Table 2. It is not difficult to find that the classification effect of SRC is significantly higher than that of K-means, which is due to the small number of gene expression profiles. To verify the effectiveness of the algorithm for feature extraction, the extracted features from LRR, Lat-LRR, and NNDGLLRR are classified by SRC. Classification results as shown in Table 2.Table 2
Algorithm identification accuracy under different data sets.
Dataset
Accuracy (%)
K-means
SRC
LRR + SRC
Lat-LRR + SRC
NNDGLLRR + SRC
DLBCL
46.87
69.79
90.62
89.58
94.79
MLL
50.83
71.43
88.33
90.83
97.50
LC
50.79
73.54
87.83
91.26
98.14
AL
43.33
69.44
83.33
87.22
93.32
Average
47.96
71.05
87.53
89.72
95.94Table2 shows that any one of LRR, Lat-LRR, and NNDGLLRR can achieve feature extraction effectively. However, the feature extraction effect of NNDGLLRR is better than that of Lat-LRR. The category and number of samples, as well as dimension of the gene expression profile, will have an impact on the final recognition effect.


### 4.3. The Influence of Graph Regularized Coefficients

Generally, we setα=β. To verify the influence of graph regularized coefficients on feature extraction, we have compared the recognition results of LRR, Lat-LRR, and NNDGLLRR under the condition of different αβ values. The results are shown in Figure 1.Figure 1
Clustering performance of algorithms with different graph regularized coefficients.
(a)
Dataset of MLL
(b)
Dataset of LCThrough the test results of MLL and LC, we can find that manifold regularized constraint has obvious optimization effect on the gene expression profile feature extraction when the values ofα and β are appropriate, and it can significantly improve the recognition effect of feature extraction. However, α and β should not be too large or too small. The optimal graph regularized coefficients may be different for different test data sets.


### 4.4. The Influence of Sparse Representation Coefficients

During the process of gene sequencing, the resulting gene expression profile will usually contain heavy noise due to the sequencing process. To verify the effect of the sparse constraint on the feature extraction, we tested the classification accuracy of LRR, Lat-LRR, and NNDGLLRR for feature extraction under different sparse constraint coefficientsλ. The test results are shown in Figure 2.Figure 2
Clustering performance of algorithms with different sparse representation coefficients.
(a)
Dataset of NHL
(b)
Dataset of ALFigure2 shows that different sparse constraint coefficients have a considerable effect on the final feature extraction results. When the value of λ is appropriate, the performance of Lat-LRR and NNDGLLRR on feature extraction is better than that of LRR. In general, the performance of NNDGLLRR is better than that of Lat-LRR, which proves the validity of manifold constraint again.


### 4.5. Complexity Analysis

Z
∈
R
n
×
n,  L∈Rm×n, and  E∈Rm×n, and we set the lowest ranks of Z and L obtained by the algorithm as r1 and r2. Then the complexity of SVT operation for Z and L is about O(r1n2) and O(r2m2), and the complexity of ST operation for E is about O(kmn). The complexity of construction the Laplacian matrix of Z and L is about O(mn2) and O(m2n); and the complexity of one positive operation for Z and L is about n2 and m2. If the iteration of the algorithm is k, then the overall complexity of LRR, Lat-LRR, and NNDGLLRR algorithms is shown in Table 3.Table 3
Algorithm complexity calculation.
Complexity
SVT
ST
Others
Total
LRR
2
k
r
1
n
2
k
m
n
0
O
(
2
k
r
1
n
2
+
k
m
n
)
Lat-LRR
2
k
r
1
n
2
+
2
k
r
2
m
2
k
m
n
0
O
(
2
k
r
2
m
2
)
NNDGLLRR
k
r
1
n
2
+
k
r
2
m
2
k
m
n
m
n
2
+
m
2
n
+
k
n
2
+
k
m
2
O
(
k
r
2
m
2
)Generally, it is considered thatm≫n for gene expression profile. It can be seen from Table 3 that LRR is the simplest in terms of computational complexity, but the performance of LRR on feature extraction is less effective than that of Lat-LRR and NNDGLLRR, and it is difficult to meet the actual demand. The result of Lat-LRR on feature extraction can be not bad, but the partitioning ability of the subspace is limited, and the operation speed is slow because of too many introduced variables. The variable update algorithm of NNDGLLRR not only reduces the calculated amount, but also achieves satisfactory results on feature extraction.


## 4.1. Selecting the Test Data

To test the feature extraction performance of the algorithm, we used diffuse large B-cell lymphoma [32] (DLBCL), mixed lineage leukemia [33] (MLL), lung cancer [34] (LC), acute lymphoblastic leukemia [35] (ALL) gene sequences to make test, and the sample information of each group of genes as is shown in Table 1.Table 1
Test data information.
Data name
Size
Classes
Number
DLBCL
5469 × 77
2
77
MLL
12582 × 72
3
72
LC
12600 × 203
5
203
ALL
12626 × 248
6
248


## 4.2. Accuracy Test

K-means and sparse representation classifier (SRC) are simple and common classifiers. To compare the clustering results of K-means and SRC, the two kinds of classifiers are used to classify the original gene expression profile. Clustering results are shown in Table 2. It is not difficult to find that the classification effect of SRC is significantly higher than that of K-means, which is due to the small number of gene expression profiles. To verify the effectiveness of the algorithm for feature extraction, the extracted features from LRR, Lat-LRR, and NNDGLLRR are classified by SRC. Classification results as shown in Table 2.Table 2
Algorithm identification accuracy under different data sets.
Dataset
Accuracy (%)
K-means
SRC
LRR + SRC
Lat-LRR + SRC
NNDGLLRR + SRC
DLBCL
46.87
69.79
90.62
89.58
94.79
MLL
50.83
71.43
88.33
90.83
97.50
LC
50.79
73.54
87.83
91.26
98.14
AL
43.33
69.44
83.33
87.22
93.32
Average
47.96
71.05
87.53
89.72
95.94Table2 shows that any one of LRR, Lat-LRR, and NNDGLLRR can achieve feature extraction effectively. However, the feature extraction effect of NNDGLLRR is better than that of Lat-LRR. The category and number of samples, as well as dimension of the gene expression profile, will have an impact on the final recognition effect.


## 4.3. The Influence of Graph Regularized Coefficients

Generally, we setα=β. To verify the influence of graph regularized coefficients on feature extraction, we have compared the recognition results of LRR, Lat-LRR, and NNDGLLRR under the condition of different αβ values. The results are shown in Figure 1.Figure 1
Clustering performance of algorithms with different graph regularized coefficients.
(a)
Dataset of MLL
(b)
Dataset of LCThrough the test results of MLL and LC, we can find that manifold regularized constraint has obvious optimization effect on the gene expression profile feature extraction when the values ofα and β are appropriate, and it can significantly improve the recognition effect of feature extraction. However, α and β should not be too large or too small. The optimal graph regularized coefficients may be different for different test data sets.


## 4.4. The Influence of Sparse Representation Coefficients

During the process of gene sequencing, the resulting gene expression profile will usually contain heavy noise due to the sequencing process. To verify the effect of the sparse constraint on the feature extraction, we tested the classification accuracy of LRR, Lat-LRR, and NNDGLLRR for feature extraction under different sparse constraint coefficientsλ. The test results are shown in Figure 2.Figure 2
Clustering performance of algorithms with different sparse representation coefficients.
(a)
Dataset of NHL
(b)
Dataset of ALFigure2 shows that different sparse constraint coefficients have a considerable effect on the final feature extraction results. When the value of λ is appropriate, the performance of Lat-LRR and NNDGLLRR on feature extraction is better than that of LRR. In general, the performance of NNDGLLRR is better than that of Lat-LRR, which proves the validity of manifold constraint again.


## 4.5. Complexity Analysis

Z
∈
R
n
×
n,  L∈Rm×n, and  E∈Rm×n, and we set the lowest ranks of Z and L obtained by the algorithm as r1 and r2. Then the complexity of SVT operation for Z and L is about O(r1n2) and O(r2m2), and the complexity of ST operation for E is about O(kmn). The complexity of construction the Laplacian matrix of Z and L is about O(mn2) and O(m2n); and the complexity of one positive operation for Z and L is about n2 and m2. If the iteration of the algorithm is k, then the overall complexity of LRR, Lat-LRR, and NNDGLLRR algorithms is shown in Table 3.Table 3
Algorithm complexity calculation.
Complexity
SVT
ST
Others
Total
LRR
2
k
r
1
n
2
k
m
n
0
O
(
2
k
r
1
n
2
+
k
m
n
)
Lat-LRR
2
k
r
1
n
2
+
2
k
r
2
m
2
k
m
n
0
O
(
2
k
r
2
m
2
)
NNDGLLRR
k
r
1
n
2
+
k
r
2
m
2
k
m
n
m
n
2
+
m
2
n
+
k
n
2
+
k
m
2
O
(
k
r
2
m
2
)Generally, it is considered thatm≫n for gene expression profile. It can be seen from Table 3 that LRR is the simplest in terms of computational complexity, but the performance of LRR on feature extraction is less effective than that of Lat-LRR and NNDGLLRR, and it is difficult to meet the actual demand. The result of Lat-LRR on feature extraction can be not bad, but the partitioning ability of the subspace is limited, and the operation speed is slow because of too many introduced variables. The variable update algorithm of NNDGLLRR not only reduces the calculated amount, but also achieves satisfactory results on feature extraction.


## 5. Conclusion

Aiming at the characteristics of high redundancy and heavy noise of gene expression profile, a feature extraction model of NNDGLLRR is proposed in this paper. In the process of experiment, we extracted the features of different gene expression profile by LRR, Lat-LRR, and NNDGLLRR and classified the extracted features by SRC. The experimental results show that the performance of NNDGLLRR on feature extraction is better than that of LRR and better than Lat-LRR slightly, which verified the comparative advantages of NNDGLRR. At the same time, compared with Lat-LRR, the overall complexity of NNDGLLRR is reduced through the improvement of the variable update algorithm. The experiments using different gene expression data sets for testing have made comparatively ideal experimental results, which proves the validity of the dual graph regularized constraint. In summary, the proposed nonnegative low-rank sparse constraint and dual graph regularized constraint are reasonable, and NNDGLLRR has good adaptability to different gene expression profile with high redundancy and heavy noise.



---

*Source: 1096028-2017-03-30.xml*