# Hypergraph Regularized Discriminative Nonnegative Matrix Factorization on Sample Classification and Co-Differentially Expressed Gene Selection

Authors: Yong-Jing Hao; Ying-Lian Gao; Mi-Xiao Hou; Ling-Yun Dai; Jin-Xing Liu
Journal: Complexity
 (2019)

---

## Abstract

Nonnegative Matrix Factorization (NMF) is a significant big data analysis technique. However, standard NMF regularized by simple graph does not have discriminative function, and traditional graph models cannot accurately reflect the problem of multigeometry information between data. To solve the above problem, this paper proposed a new method called Hypergraph Regularized Discriminative Nonnegative Matrix Factorization (HDNMF), which captures intrinsic geometry by constructing hypergraphs rather than simple graphs. The introduction of the hypergraph method allows high-order relationships between samples to be considered, and the introduction of label information enables the method to have discriminative effect. Both the hypergraph Laplace and the discriminative label information are utilized together to learn the projection matrix in the standard method. In addition, we offered a corresponding multiplication update solution for the optimization. Experiments indicate that the method proposed is more effective by comparing with the earlier methods.


---

## Body

## 1. Introduction

With the development of sequencing technology [1] and gene detection technology [2], a lot of genomic data have been collected. Genomic data usually have the characteristics of high-dimensional small samples, and how to extract useful information from massive genomic data has become the most challenging task. To increase the processing efficiency of such high-dimensional data, a series of dimensionality reduction techniques [3] have been proposed. Among the various dimensionality reduction methods, NMF and its improved NMF-based methods are widely used in the field of gene data processing.There is some physiology and psychological evidence that humans rely on part-based representations for some object recognition [4]. The Nonnegative Matrix Factorization (NMF) [5] method is capable of learning the various parts of the face and the semantic features of the text. NMF is a powerful technology for component-based data analysis. It is intended to find two nonnegative matrices to learn the part-based representation of the standard data itself. NMF has been popular for decades and successfully implemented in a wide range of fields, including robotics control [6], image analysis [7], and biomedical engineering [8]. To this end, we will provide a brief introduction to the relevant methods.A variant of NMF called the Graph Regularized Nonnegative Matrix Factorization (GNMF) method [9] was proposed by Cai et al. which takes into consideration the geometric structure between data and uses K-nearest neighbor graph coding to determine the geometry of the data. The method works well in cluster applications but achieves mediocre results in classification problems. To improve the effect of GNMF in the classification, a method named Graph Regularized Discriminative Nonnegative Matrix Factorization (GDNMF) [10] was proposed by Long et al. which considers geometry between data and label information. The discriminating power of different classes is increased by considering the label information. In the GDNMF method, the introduction of the dictionary matrix [11] has achieved excellent performance in the feature selection and classification of genomic data. The GNMF and GDNMF methods construct a simple graph based on the geometric relationships between the sample data, and the high-order relationships between the sample data may be ignored. So Zeng et al. perfectly integrated the hypergraph regularization into the standard NMF, called Hypergraph Regularized Nonnegative Matrix Factorization (HNMF) [12]. The hypergraph regularized takes into account the intrinsic manifold structure of the sample data. This method encodes the geometric information of the data space by constructing a hypergraph rather than a simple graph. The data representations discovered by HNMF are not only partial but also sparse. So HNMF can show better performance. In addition, Peng et al. proposed an NMF deformation method called Parallel Vector Field Regularized Nonnegative Matrix Factorization for Image Representation [13]. This method can effectively improve the calculation speed. To increase the adaptability of the method and avoid the ambiguity of artificial selection, the Flexible Nonnegative Matrix Factorization with Adaptively Learned Graph Regularization was proposed by Peng et al. [14], which can effectively solve the above problems. In summary, various variants of NMF have their own unique advantages in feature selection or cluster classification.On the one hand, discriminative Nonnegative Matrix Factorization has also been well applied in other areas, such as image representation [15], image classification [16], and diesel engine fault diagnosis [17]. On the other hand, hypergraph regularization has become more and more popular in recent years, for example, in image click prediction [18], image ranking [19], and image restoration [20].Inspired by the above work, we propose a novel method called Hypergraph Regularized Discriminative Nonnegative Matrix Factorization (HDNMF), which takes into consideration the sample data with manifold inherent structure. The geometry of the data space is encoded by constructing a hypergraph rather than a simple graph. To consider the part of the data itself and the space characteristics, the label information is seen as a significant factor. We construct the K-nearest neighbor graph [21] to encode the geometry of the data space and increase the discriminative power of different classes. In this paper, an optimization scheme is shown in detail, and the objective function is solved by multiplication iterative update. Experiments demonstrate that our proposed method achieves better results than some NMF variants.


## 2. Materials and Methods

Let the input matrixX be data of m rows and n columns, where rows represent genes and columns represent samples. Usually the value of m is large, which may make data processing of poor accuracy. Therefore, the dimensionality reduction of the data matrix has grown up to be a crucial step. The standard NMF and its improved methods are popular techniques for dimensionality reduction. We will introduce the standard NMF method and several standard NMF improvement methods in this section.


### 2.1. Related Work

#### 2.1.1. Standard NMF (NMF)

Nonnegative factorization was provided by Melvyn W. Jeter and Wallace C. Pye [22] in 1984 and the positive matrix factorization by Paatero and Tapper [23] in 1995. Lee and Seung [24, 25] continued to research on it. It was a matrix factorization method that mainly analyzes high-dimensional data matrices with nonnegative factors. Assuming there are three nonnegative matrices X∈Rm×n, W∈Rm×k, and H∈Rk×n, standard NMF can make the following formula established: X≈WH.The objective function of the standard NMF was used to minimize the Euclidean distance [26] between X and WH by using the multiplicative update rules:(1)O1=X-WHF2,where ·F is the F-norm of the matrix. X is named the input data matrix, W is named the basic matrix, and H is called the coefficient matrix. The elements involved are nonnegative. The update rules are shown as follows in Lee and Seung’s paper:(2)wmk←wmkWHTmkWHHTmk,hkn←hknWTXknWTWHkn.


#### 2.1.2. Graph Regularized NMF (GNMF)

The GNMF [9] method was used to construct the geometry structure by using the nearest neighbor graph. It minimizes the objective function as follows:(3)O=X-WHF2+αTrHLHT,where Tr· is the trace of the matrix, α is the regularization parameter which controls the smoothness of the new representation, L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix.


#### 2.1.3. Graph Regularized Discriminative NMF (GDNMF)

The method GDNMF is proposed, which applies the intrinsic simple geometry between data and discriminative label information to design the objective function. The formula is as follows:(4)O=X-WHF2+αTrHLHT+βS-AHF2,where L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


#### 2.1.4. Hypergraph Regularized NMF (HNMF)

The Laplacian eigenmaps (LE) method is a classical manifold method based on simple graphs. The method mainly considers the relationship between two vertices, while the hypergraph considers the relationship between three and more vertices. In the HNMF method, the hypergraph and NMF combine very well; its objective function is below:(5)O=X-WHF2+αTrHLhyperHT,where Tr(.) is the trace of the matrix and α is the regularization parameter. Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix.


### 2.2. Methodology

Inspired by the GDNMF method and HNMF method, we added the discriminative label information to the HNMF method. The definition and multiplication update rules for HDNMF are shown below.


#### 2.2.1. Hypergraph Regularized Discriminative NMF (HDNMF)

In a simple graph, two vertices are connected by an edge, and the weight of the edge is utilized to represent the affinity relationship between the two vertices. In fact, the interrelationship between multiple vertices is also essential. To solve this problem, the hypergraph [27, 28] emerges, and its edges can establish a link between two or more vertices.The hypergraphGhyper consists of Vhyper, Ehyper, and Whyper, where Vhyper represents the set of vertices; Ehyper represents the hyperedge set; Whyper represents the weight set of the hyperedge; the weight of the hyperedge e is represented as whyper(e). The correlation matrix Hhyper of hypergraph Ghyper is given below [28]:(6)Hhyperv,e1,ifv∈e,0,ifv∈e.The degree of one edge e is as follows:(7)dv=∑e∈E∣v∈ewhypere=∑e∈EwhypereHhyperv,e.The degree of one hyperedge e is as follows:(8)dhypere=e=∑v∈VHhyperv,e.Let Dhyper be a diagonal matrix, which corresponds to the element corresponding to the vertex degree; then the nonstandardized superlattices matrix is as follows:(9)Lhyper=Dhyper-Chyper,where Chyper=HhyperWhyperDhypere-1HhyperT.A label matrixS is defined as follows:(10)si,j=1,ifyi=i,j=1,2,3…,n,i=1,2,3…,c,0,otherwise,where yj∈1,2,3…,c represents the jth sample of the class label of xj, and the number of categories in the training set X is c.We can obtain the following minimization problem:(11)O=X-WHF2+αTrHLhyperHT+βS-AHF2,where Tr(.) is the trace of the matrix, Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


#### 2.2.2. The Update Rules of HDNMF

The multiplication update rules are extended according to the standard NMF’s F-norm [29], to find the local optimum. Equation (11) can be written as follows:(12)F=TrX-WHTX-WH+αTrHLHyperHT+βTrS-AHTS-AH=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT.Equation (12) can be written as a Lagrange function as follows:(13)Lf=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT+TrΦTW+TrΨTH+TrΩTA,where Φ, Ψ, and Ω are the Lagrange multipliers.The partial derivatives ofLf with respect to H, W, and A, respectively, are(14)∂Lf∂H=2WTWH-X+2αHLHyper+2βATAH-S+Ψ=0.(15)∂Lf∂W=2WHHT-2XHT+Φ=0.(16)∂Lf∂A=-2βSHT+2βAHHT+Ω=0.The following formulas can be obtained by using the KKT condition:(17)2WTWH-X+2αHDhyper-Chyper+2βATAH-SknHkn+ΨknHkn=0.(18)2WHHT-2XHTikWik+ΦikWik=0.(19)-2βSHT+2βAHHTpkApk+ΩpkApk=0.We can gain the updating rules for H, W, and A:(20)Hkn←HknβATS+WTX+αHChyperknWTWH+βATAH+αHDhyperkn.(21)Wmk←WmkXHTmkWHHTmk.(22)Apk←ApkSHTpkAHHTpk.The algorithm of HDNMF is shown in Algorithm 1.Algorithm 1:The algorithm of HDNMF.
Input:Data matrix X∈Rm×n, hyper-graph Laplace matrix Lhyper∈Rn×n, indicator matrix S∈Rp×k, parameters α, β, k. Initialization: Randomly initialize three non-negative matrices W∈Rm×kH∈Rk×n and A∈Rp×k. Repeat: (1) Update H by rule (20). (2) Update W by rule (21). (3) Update A by rule (22). Until Convergence. Output: W and H.


#### 2.2.3. Complexity Analysis

In this subsection, we compare the computational complexity of NMF and HDNMF based on multiplication of update rules. The calculation operation count of every iteration is considered by (2) and (20)-(22). The results and parameters are listed in Tables 1 and 2.Table 1
Computational operation counts for each iteration in NMF and HDNMF.
Algorithm NMF HDNMF Addition 2 m n k + 2 m + n k 2 2 m n k + 2 m + 4 n + 2 c k 2 + 2 n + K n + c p k Multiplication 2 m n k + 2 m + n k 2 + m + n k 2 m n k + 2 m + 4 n + 2 c k 2 + 3 n + K n + m + c p + c k Division m + n k m + n + c k Overall O m n k O m n kTable 2
Parameters description.
Parameters Description m Number of features n Number of samples k Number of dimension c Number of class p Number of training samples obtained from each class K Number of nearest neighborsFor the HDNMF method, the matrixC is sparse. So, we need Knk multiplication and addition to count HC. As for the label matrix S, we need nk multiplication and addition to count ATS, and we need cpk multiplication and addition to count SHT.In addition to the multiplication update, HDNMF requestsOp2m to set up the weight matrix C and requests Opc to set up the indicator matrix S. Assume that, after the multiplication iteration of t multiplication, the total cost of NMF is Otmnk; the total of HDNMF the cost is Otmnk+p2m+pc.


## 2.1. Related Work

### 2.1.1. Standard NMF (NMF)

Nonnegative factorization was provided by Melvyn W. Jeter and Wallace C. Pye [22] in 1984 and the positive matrix factorization by Paatero and Tapper [23] in 1995. Lee and Seung [24, 25] continued to research on it. It was a matrix factorization method that mainly analyzes high-dimensional data matrices with nonnegative factors. Assuming there are three nonnegative matrices X∈Rm×n, W∈Rm×k, and H∈Rk×n, standard NMF can make the following formula established: X≈WH.The objective function of the standard NMF was used to minimize the Euclidean distance [26] between X and WH by using the multiplicative update rules:(1)O1=X-WHF2,where ·F is the F-norm of the matrix. X is named the input data matrix, W is named the basic matrix, and H is called the coefficient matrix. The elements involved are nonnegative. The update rules are shown as follows in Lee and Seung’s paper:(2)wmk←wmkWHTmkWHHTmk,hkn←hknWTXknWTWHkn.


### 2.1.2. Graph Regularized NMF (GNMF)

The GNMF [9] method was used to construct the geometry structure by using the nearest neighbor graph. It minimizes the objective function as follows:(3)O=X-WHF2+αTrHLHT,where Tr· is the trace of the matrix, α is the regularization parameter which controls the smoothness of the new representation, L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix.


### 2.1.3. Graph Regularized Discriminative NMF (GDNMF)

The method GDNMF is proposed, which applies the intrinsic simple geometry between data and discriminative label information to design the objective function. The formula is as follows:(4)O=X-WHF2+αTrHLHT+βS-AHF2,where L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


### 2.1.4. Hypergraph Regularized NMF (HNMF)

The Laplacian eigenmaps (LE) method is a classical manifold method based on simple graphs. The method mainly considers the relationship between two vertices, while the hypergraph considers the relationship between three and more vertices. In the HNMF method, the hypergraph and NMF combine very well; its objective function is below:(5)O=X-WHF2+αTrHLhyperHT,where Tr(.) is the trace of the matrix and α is the regularization parameter. Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix.


## 2.1.1. Standard NMF (NMF)

Nonnegative factorization was provided by Melvyn W. Jeter and Wallace C. Pye [22] in 1984 and the positive matrix factorization by Paatero and Tapper [23] in 1995. Lee and Seung [24, 25] continued to research on it. It was a matrix factorization method that mainly analyzes high-dimensional data matrices with nonnegative factors. Assuming there are three nonnegative matrices X∈Rm×n, W∈Rm×k, and H∈Rk×n, standard NMF can make the following formula established: X≈WH.The objective function of the standard NMF was used to minimize the Euclidean distance [26] between X and WH by using the multiplicative update rules:(1)O1=X-WHF2,where ·F is the F-norm of the matrix. X is named the input data matrix, W is named the basic matrix, and H is called the coefficient matrix. The elements involved are nonnegative. The update rules are shown as follows in Lee and Seung’s paper:(2)wmk←wmkWHTmkWHHTmk,hkn←hknWTXknWTWHkn.


## 2.1.2. Graph Regularized NMF (GNMF)

The GNMF [9] method was used to construct the geometry structure by using the nearest neighbor graph. It minimizes the objective function as follows:(3)O=X-WHF2+αTrHLHT,where Tr· is the trace of the matrix, α is the regularization parameter which controls the smoothness of the new representation, L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix.


## 2.1.3. Graph Regularized Discriminative NMF (GDNMF)

The method GDNMF is proposed, which applies the intrinsic simple geometry between data and discriminative label information to design the objective function. The formula is as follows:(4)O=X-WHF2+αTrHLHT+βS-AHF2,where L is the graph Laplacian matrix (L=D-C), C is the weight matrix of the nearest neighbor graph, and D is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


## 2.1.4. Hypergraph Regularized NMF (HNMF)

The Laplacian eigenmaps (LE) method is a classical manifold method based on simple graphs. The method mainly considers the relationship between two vertices, while the hypergraph considers the relationship between three and more vertices. In the HNMF method, the hypergraph and NMF combine very well; its objective function is below:(5)O=X-WHF2+αTrHLhyperHT,where Tr(.) is the trace of the matrix and α is the regularization parameter. Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix.


## 2.2. Methodology

Inspired by the GDNMF method and HNMF method, we added the discriminative label information to the HNMF method. The definition and multiplication update rules for HDNMF are shown below.


### 2.2.1. Hypergraph Regularized Discriminative NMF (HDNMF)

In a simple graph, two vertices are connected by an edge, and the weight of the edge is utilized to represent the affinity relationship between the two vertices. In fact, the interrelationship between multiple vertices is also essential. To solve this problem, the hypergraph [27, 28] emerges, and its edges can establish a link between two or more vertices.The hypergraphGhyper consists of Vhyper, Ehyper, and Whyper, where Vhyper represents the set of vertices; Ehyper represents the hyperedge set; Whyper represents the weight set of the hyperedge; the weight of the hyperedge e is represented as whyper(e). The correlation matrix Hhyper of hypergraph Ghyper is given below [28]:(6)Hhyperv,e1,ifv∈e,0,ifv∈e.The degree of one edge e is as follows:(7)dv=∑e∈E∣v∈ewhypere=∑e∈EwhypereHhyperv,e.The degree of one hyperedge e is as follows:(8)dhypere=e=∑v∈VHhyperv,e.Let Dhyper be a diagonal matrix, which corresponds to the element corresponding to the vertex degree; then the nonstandardized superlattices matrix is as follows:(9)Lhyper=Dhyper-Chyper,where Chyper=HhyperWhyperDhypere-1HhyperT.A label matrixS is defined as follows:(10)si,j=1,ifyi=i,j=1,2,3…,n,i=1,2,3…,c,0,otherwise,where yj∈1,2,3…,c represents the jth sample of the class label of xj, and the number of categories in the training set X is c.We can obtain the following minimization problem:(11)O=X-WHF2+αTrHLhyperHT+βS-AHF2,where Tr(.) is the trace of the matrix, Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


### 2.2.2. The Update Rules of HDNMF

The multiplication update rules are extended according to the standard NMF’s F-norm [29], to find the local optimum. Equation (11) can be written as follows:(12)F=TrX-WHTX-WH+αTrHLHyperHT+βTrS-AHTS-AH=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT.Equation (12) can be written as a Lagrange function as follows:(13)Lf=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT+TrΦTW+TrΨTH+TrΩTA,where Φ, Ψ, and Ω are the Lagrange multipliers.The partial derivatives ofLf with respect to H, W, and A, respectively, are(14)∂Lf∂H=2WTWH-X+2αHLHyper+2βATAH-S+Ψ=0.(15)∂Lf∂W=2WHHT-2XHT+Φ=0.(16)∂Lf∂A=-2βSHT+2βAHHT+Ω=0.The following formulas can be obtained by using the KKT condition:(17)2WTWH-X+2αHDhyper-Chyper+2βATAH-SknHkn+ΨknHkn=0.(18)2WHHT-2XHTikWik+ΦikWik=0.(19)-2βSHT+2βAHHTpkApk+ΩpkApk=0.We can gain the updating rules for H, W, and A:(20)Hkn←HknβATS+WTX+αHChyperknWTWH+βATAH+αHDhyperkn.(21)Wmk←WmkXHTmkWHHTmk.(22)Apk←ApkSHTpkAHHTpk.The algorithm of HDNMF is shown in Algorithm 1.Algorithm 1:The algorithm of HDNMF.
Input:Data matrix X∈Rm×n, hyper-graph Laplace matrix Lhyper∈Rn×n, indicator matrix S∈Rp×k, parameters α, β, k. Initialization: Randomly initialize three non-negative matrices W∈Rm×kH∈Rk×n and A∈Rp×k. Repeat: (1) Update H by rule (20). (2) Update W by rule (21). (3) Update A by rule (22). Until Convergence. Output: W and H.


### 2.2.3. Complexity Analysis

In this subsection, we compare the computational complexity of NMF and HDNMF based on multiplication of update rules. The calculation operation count of every iteration is considered by (2) and (20)-(22). The results and parameters are listed in Tables 1 and 2.Table 1
Computational operation counts for each iteration in NMF and HDNMF.
Algorithm NMF HDNMF Addition 2 m n k + 2 m + n k 2 2 m n k + 2 m + 4 n + 2 c k 2 + 2 n + K n + c p k Multiplication 2 m n k + 2 m + n k 2 + m + n k 2 m n k + 2 m + 4 n + 2 c k 2 + 3 n + K n + m + c p + c k Division m + n k m + n + c k Overall O m n k O m n kTable 2
Parameters description.
Parameters Description m Number of features n Number of samples k Number of dimension c Number of class p Number of training samples obtained from each class K Number of nearest neighborsFor the HDNMF method, the matrixC is sparse. So, we need Knk multiplication and addition to count HC. As for the label matrix S, we need nk multiplication and addition to count ATS, and we need cpk multiplication and addition to count SHT.In addition to the multiplication update, HDNMF requestsOp2m to set up the weight matrix C and requests Opc to set up the indicator matrix S. Assume that, after the multiplication iteration of t multiplication, the total cost of NMF is Otmnk; the total of HDNMF the cost is Otmnk+p2m+pc.


## 2.2.1. Hypergraph Regularized Discriminative NMF (HDNMF)

In a simple graph, two vertices are connected by an edge, and the weight of the edge is utilized to represent the affinity relationship between the two vertices. In fact, the interrelationship between multiple vertices is also essential. To solve this problem, the hypergraph [27, 28] emerges, and its edges can establish a link between two or more vertices.The hypergraphGhyper consists of Vhyper, Ehyper, and Whyper, where Vhyper represents the set of vertices; Ehyper represents the hyperedge set; Whyper represents the weight set of the hyperedge; the weight of the hyperedge e is represented as whyper(e). The correlation matrix Hhyper of hypergraph Ghyper is given below [28]:(6)Hhyperv,e1,ifv∈e,0,ifv∈e.The degree of one edge e is as follows:(7)dv=∑e∈E∣v∈ewhypere=∑e∈EwhypereHhyperv,e.The degree of one hyperedge e is as follows:(8)dhypere=e=∑v∈VHhyperv,e.Let Dhyper be a diagonal matrix, which corresponds to the element corresponding to the vertex degree; then the nonstandardized superlattices matrix is as follows:(9)Lhyper=Dhyper-Chyper,where Chyper=HhyperWhyperDhypere-1HhyperT.A label matrixS is defined as follows:(10)si,j=1,ifyi=i,j=1,2,3…,n,i=1,2,3…,c,0,otherwise,where yj∈1,2,3…,c represents the jth sample of the class label of xj, and the number of categories in the training set X is c.We can obtain the following minimization problem:(11)O=X-WHF2+αTrHLhyperHT+βS-AHF2,where Tr(.) is the trace of the matrix, Lhyper is the hypergraph Laplacian matrix (Lhyper=Dhyper-Chyper), Chyper is the weight matrix of the nearest neighbor graph, and Dhyper is the diagonal matrix. A is initialized randomly in this method. S, A, and H are nonnegative matrices; α and β are nonnegative regularization parameters.


## 2.2.2. The Update Rules of HDNMF

The multiplication update rules are extended according to the standard NMF’s F-norm [29], to find the local optimum. Equation (11) can be written as follows:(12)F=TrX-WHTX-WH+αTrHLHyperHT+βTrS-AHTS-AH=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT.Equation (12) can be written as a Lagrange function as follows:(13)Lf=TrXTX-2TrXTWH+TrHTWTWH+βTrSTS-2βTrSTAH+βTrHTATAH+αTrHLHyperHT+TrΦTW+TrΨTH+TrΩTA,where Φ, Ψ, and Ω are the Lagrange multipliers.The partial derivatives ofLf with respect to H, W, and A, respectively, are(14)∂Lf∂H=2WTWH-X+2αHLHyper+2βATAH-S+Ψ=0.(15)∂Lf∂W=2WHHT-2XHT+Φ=0.(16)∂Lf∂A=-2βSHT+2βAHHT+Ω=0.The following formulas can be obtained by using the KKT condition:(17)2WTWH-X+2αHDhyper-Chyper+2βATAH-SknHkn+ΨknHkn=0.(18)2WHHT-2XHTikWik+ΦikWik=0.(19)-2βSHT+2βAHHTpkApk+ΩpkApk=0.We can gain the updating rules for H, W, and A:(20)Hkn←HknβATS+WTX+αHChyperknWTWH+βATAH+αHDhyperkn.(21)Wmk←WmkXHTmkWHHTmk.(22)Apk←ApkSHTpkAHHTpk.The algorithm of HDNMF is shown in Algorithm 1.Algorithm 1:The algorithm of HDNMF.
Input:Data matrix X∈Rm×n, hyper-graph Laplace matrix Lhyper∈Rn×n, indicator matrix S∈Rp×k, parameters α, β, k. Initialization: Randomly initialize three non-negative matrices W∈Rm×kH∈Rk×n and A∈Rp×k. Repeat: (1) Update H by rule (20). (2) Update W by rule (21). (3) Update A by rule (22). Until Convergence. Output: W and H.


## 2.2.3. Complexity Analysis

In this subsection, we compare the computational complexity of NMF and HDNMF based on multiplication of update rules. The calculation operation count of every iteration is considered by (2) and (20)-(22). The results and parameters are listed in Tables 1 and 2.Table 1
Computational operation counts for each iteration in NMF and HDNMF.
Algorithm NMF HDNMF Addition 2 m n k + 2 m + n k 2 2 m n k + 2 m + 4 n + 2 c k 2 + 2 n + K n + c p k Multiplication 2 m n k + 2 m + n k 2 + m + n k 2 m n k + 2 m + 4 n + 2 c k 2 + 3 n + K n + m + c p + c k Division m + n k m + n + c k Overall O m n k O m n kTable 2
Parameters description.
Parameters Description m Number of features n Number of samples k Number of dimension c Number of class p Number of training samples obtained from each class K Number of nearest neighborsFor the HDNMF method, the matrixC is sparse. So, we need Knk multiplication and addition to count HC. As for the label matrix S, we need nk multiplication and addition to count ATS, and we need cpk multiplication and addition to count SHT.In addition to the multiplication update, HDNMF requestsOp2m to set up the weight matrix C and requests Opc to set up the indicator matrix S. Assume that, after the multiplication iteration of t multiplication, the total cost of NMF is Otmnk; the total of HDNMF the cost is Otmnk+p2m+pc.


## 3. Results and Discussion

On the one hand, HDNMF considers the high-order relationship about samples; on the other hand, the HDNMF method uses the label information to make the method discriminative while constructing the internal geometry of the data. To evaluate the effectiveness and discrimination of the method, the HDNMF method was compared with the other methods (NMF, DNMF, LNMF, GNMF, GDNMF, and HNMF).


### 3.1. Datasets Description

The Cancer Genome Atlas (TCGA), as the largest cancer genome database, has immeasurable information value. The data included cholangiocarcinoma data (CHOL), esophageal cancer data (ESCA), pancreatic cancer data (PAAD), colorectal cancer data (COAD), and head and neck squamous cell carcinoma (HNSC) data. All of this data can be obtained from the TCGA database athttps://cancergenome.nih.gov/. Each dataset consists of two classes, normal samples and diseased samples. The dimension of the datasets is 20502. The rows of the data represent gene features, and the columns represent gene samples.Firstly, the CHOL data contained 9 normal samples and 36 diseased samples, the PAAD data contained 4 normal samples of 176 diseased samples, the HNSC data contained 20 normal samples and 198 diseased samples, the ESCA data contained 9 normal samples and 183 diseased samples, and the COAD data contained 19 normal samples and 262 diseased samples. Then, the normal samples are removed. Finally, we integrate the PAAD, ESCA, and CHOL data into datasets with dimensions of 20502∗395 (INTA 1) and integrate the PAAD, COAD, HNSC, ESCA, and CHOL data into one dataset with dimensions of 20502∗1055 (INTA 2).


### 3.2. Implementation Issues

#### 3.2.1. Parameter Selection

As for the experiments, the datasets used were INTA 1 and INTA 2. For each test, 10-fold cross validation uses training data to adjust parameters. The advantage of cross validation is that all test datasets are independent. The regularized parameter matrix directly affects the classification results of the HDNMF method. In order to select the optimal parameters, we adjust the regularized parameters exponentially in a specific domain, such as from10-4 to 104. This section uses the tenfold cross validation method to automatically adjust the selection of the best regularized parameters in the corresponding training datasets in the following 10t:t∈-4,-3,…3,4 range. The results were shown in Figure 1.Figure 1
The influence of parameters on classification ACC (INTA 1 and INTA 2).As shown in the figure, the warmer the color, the higher the classification ACC. As can be seen from Figure1, dataset INTA 1 can get better results when α=1 and β∈10-4,10-2; as for dataset INTA 2, it can get better results when α∈102,104 and β∈10-4,104. In particular, the value of α has small significant effect on the classification result when β=102.


#### 3.2.2. The Influence of Dimensions on Classification ACC

In this part, we do the HDNMF method when dimension is from 1 to 30 to explore the effect of dimension size on the classification effect. Based on the results in Figure2, the following points can be summarized:Figure 2
Influence of dimension on classification ACC.(1) When the dimension is too small, the classification ACC effect of all methods is not ideal, mainly due to the loss of a large amount of useful information due to the large dimensionality reduction(2) As the dimension increases, the effect of the classification ACC gradually increases and tends to be stable after a certain degree, which is mainly due to the large amount of information loss that can be avoided when the dimension is large enough


#### 3.2.3. HDNMF Time Comparison

We compare the average runtime of all the methods on the two datasets in Table4. The experimental results were obtained by performing all methods on an Inter(R) Core(TM) i7-7700 CPU @ 3.6 GHz Windows Server with 64 G RAM. The time costs of the classification process for the all methods are listed. Obviously, the NMF method with constraints usually requires more runtime, and our proposed HDNMF method requires the longest runtime. The main reason is that the HDNMF method takes a lot of time in constructing the hypergraph and updates the hyperedge in each iterate. Therefore, how to speed up the proposed algorithm is an interesting work.


#### 3.2.4. The Convergence Analysis

The updating rules for the HDNMF method in (20)-(22) are easy to understand and it can be demonstrated that the above rules are convergent. Figure 3 shows the convergence about two datasets; the red dashed lines are the error value of the HDNMF method. We can see that our method will converge in 100 iterations.Figure 3
The convergence performance for INTA1 and INTA2.


### 3.3. Classification

Since HDNMF introduces label information into the NMF method, it is necessary to perform the test of the effect of the classification [28]. There are many improved methods for existing NMFs for classification, such as methods which were mentioned in [30, 31].Classification is a part of the most widely used technologies in the field of data mining. It is a technique for constructing a classifier based on the characteristics of experimental data and then using the classifier to classify samples of unknown categories. The classifier is generally divided into two stages of training and testing; in the training stage, the characteristics of the training datasets are first analyzed, and then an accurate description of the corresponding datasets is generated for each category; in the testing phase, the model classifies the test set to test its classification ACC. In the experimental part, we use the KNN method to classify the sample. For a given set of training samples with class labels, the similarity measure function is used to find the nearest neighbors that are most similar to the samples to be classified, and the decision is made according to the corresponding principle.The specific classification process (K-nearest neighbors, KNN) is as follows: (1) for the training sets, calculate the distance from the unknown sample to all known samples; (2) select the parameterK (where K is the most basic parameter in KNN, representing any number of neighbors); (3) for test sets, the unknown samples are classified into the largest number of categories in the sample according to the majority voting rule.


#### 3.3.1. HDNMF for INTA 1 and INTA 2 Classification

For the dimension equal to 3 for INTA 1 and 5 for INTA 2 the classification ACC are listed in Table3. The optimal results are italic, and the experiment proves that our HDNMF method can achieve better classification results than other methods.Table 3
The classification ACC for all methods.
Methods ACC (INTA 1) ACC (INTA 2) NMF 80.00±0.00181 87.62±0.00195 DNMF 85.10±0.00390 90.79±0.00137 LNMF 83.77±0.00690 82.07±0.004504 GNMF 82.50±0.00140 89.52±0.00135 GDNMF 90.00±0.00009 90.48±0.00019 HNMF 87.50±0.00198 88.57±0.00175 HDNMF 92.50±0.00096 94.88±0.00082Table 4
The time comparison for all methods.
Methods Time/S (INTA 1) Time/S (INTA 2) NMF 6.4064 14.7599 DNMF 7.9114 22.3321 LNMF 7.4628 19.1296 GNMF 7.8372 21.0590 GDNMF 5.0958 16.3989 HNMF 56.5247 93.0869 HDNMF 29.3517 68.6402It is possible to conclude that the introduction of label information is the main reason for the significant classification ACC effect. Therefore, the HDNMF method can not only improve the interpretability of the NMF method, but also overcome the ambiguity of the unsupervised learning training samples.


### 3.4. Co-Differentially Expressed Gene Selection

High-dimensional small sample data contain many unrelated or redundant features. In biological processes, most genes serve as the standard function of life support, so researchers focus on a small number of differentially expressed genes that play a critical role in life. Feature selection [32] is the easiest way to resolve these problems and identify differentially expressed genes. It can replace the standard data by choosing the largest amount of information without losing a lot of information. Here we show the experimental results and analysis of different methods for selecting common differentially expressed genes (co-differentially expressed genes) on integrated datasets. The relevant score (RS) refers to the correlation between genes and diseases. The higher the relevant score, the higher the correlation between genes and diseases. The number of genes (NUM) refers to the number of co-differentially expressed genes selected by each method characteristic that match the proven disease genes in GeneCards.How the GeneCards were used is described as follows: we visited the official website of GeneCards (https://www.genecards.org/). On the one hand, all types of diseased genes that have been validated can be obtained by entering the type of cancer at this web address (exporting the table for the determination of NUM). On the other hand, enter the gene name at this web address to get the details of the gene.


#### 3.4.1. INTA 1 Co-Differentially Expressed Gene Selection Results

The HDNMF method and the NMF series of methods are used to select co-differentially expressed genes of the integrated datasets. We selected 500 genes for each method to evaluate the co-differentially expressed genes selected by using different methods.For the 500 co-differentially expressed genes selected by the seven methods, we compare the three diseases that make up the integrated INTA 1. The comparison data are from GeneCards (https://www.genecards.org/). The co-differentially expressed genes excavated by different methods are listed in Table 5: the number of genes (NUM), the total related score (TRS), and the average related score (ARS). Higher NUM, TRS, and ARS values indicate better method performance. From the table, we can see that the HDNMF method can achieve better experimental results.Table 5
NUM, TRS, and ARS of five methods (INTA 1).
Methods HDNMF NMF DNMF LNMF GNMF HNMF GDNMF PAAD Num 331 327 324 328 327 322 326 TRS 2385.79 2228.45 2351.36 2287.7 2339.88 2061.11 2350.54 ARS 7.2078 6.8148 7.2350 6.9535 7.1556 6.4010 7.2102 ESCA Num 252 257 248 253 254 243 248 TRS 2318.07 2305.29 2194.42 2231.10 2224.08 2185.63 2194.08 ARS 9.1987 8.9700 8.8485 8.8186 8.7562 8.9944 8.8471 CHOL Num 134 132 134 130 132 137 131 TRS 546.15 531.06 527.15 525.56 526.42 504.76 526.63 SRS 4.0758 4.0232 3.9936 4.0428 3.9880 3.6844 4.0201We summarize the co-differentially expressed genes in the shared part in Table6, including the co-differentially expressed genes of each method and the number of unique co-differentially expressed genes excavated by each method. Unique co-differentially expressed genes selected by each method have been indicated in italic. In the table, we can clearly see that there are 0 unique co-differentially expressed genes excavated by the DNMF, LNMF, GNMF, and GDNMF methods; 2 unique co-differentially expressed genes excavated by the NMF and HNMF methods; and 7 unique co-differentially expressed genes mined by our HDNMF method. This indicates that our method works well.Table 6
Co-differentially expressed genes of five methods (INTA 1).
Methods Codifferentially expressed genes Unique NMF MUC6 FSCN1 POSTN HIF1A SLC7A5 S100A2TKT TFRC TTR TP63 SPINK1 2 DNMF SPINK1CTGFS100A2ANPEP 0 LNMF CRPHIF1AEGR1STAT3SLC7A5TFRC 0 GNMF MUC6 FSCN1 POSTN SLC7A5 S100A2 TTR SPINK1 TPM1 CTGF 0 HNMF MUC6 FSCN1 POSTN SLC7A5 TTR TPM1 HIF1A TFRC CRP STAT3 EGR1 0 HDNMF ABCC3 BCL2L1 CRP RHOC TPM1 ERBB3 AGT CTSC HP ANPEP 7In Table7, we summarize the detailed information of the co-differentially expressed genes excavated only by the HDNMF method in the table by GeneCards, including the official name, related diseases, related GO annotations, RS, and paralog gene. All genes in the table have a high related score, which means that these genes can be regarded as important pathogenic genes to help disease research. Medical research in this area will also provide us with unexpected new discoveries. Therefore, the unique co-differentially expressed genes excavated by the HDNMF method can promote further research on these three diseases.Table 7
Unique co-differentially expressed genes by HDNMF (INTA 1).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CTSC Cathepsin C Papillon-Lefevre syndrome and Haim-Munk syndrome Identical protein binding and cysteine-type peptidase activity 76.89 TINAGL1 AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 Null RHOC Ras homolog family member C Breast disease and breast cancer GTP binding and obsolete signal transducer activity 64.59 RHOA HP Haptoglobin Anhaptoglobinemia andPlasmodium falciparum malaria Serine-type endopeptidase activity and hemoglobin binding 59.62 HPR BCL2L1 BCL2-like 1 B-cell lymphomas and follicular lymphoma Protein homodimerization activity and protein heterodimerization activity 49.62 BCL2 ABCC3 ATP binding cassette subfamily C member 3 Dubin-Johnson syndrome and extrahepatic cholestasis Transporter activity and ATPase activity 46.90 ABCC1From Table7, we can see the relevant information of the unique co-differentially expressed genes selected by the HDNMF method. ERBB3 has the highest relevance score. ERBB3 is a protein-coding gene. When it mutates, the tumor suppressor protein does not form properly. In pancreatic cancer, ERBB3 is a preferred dimerization partner for EGFR, and ERBB3 protein expression levels are directly related to the antiproliferative effects of erlotinib (EGFR-specific tyrosine kinase inhibitor) and transient knockdown of ERBB3 expression gain resistance to EGFR targeted therapy. This leads to the onset of cancer. ERBB3 has been shown to be associated with PAAD [33]. ERBB3 forms heterodimers with other kinase-active EGF (epidermal growth factor) receptor family members. Heterodimerization leads to activation of pathways leading to cell proliferation or differentiation. Amplification of this gene and/or overexpression of its protein has been reported in many cancers, such as PAAD, ESCA [34], and CHOL [35]. Therefore, mutations in one gene may be involved in the production of multiple cancers. This suggests that biologists can further study the link between different cancers.


#### 3.4.2. INTA 2 Co-Differentially Expressed Gene Selection Results

Similar to Table5, Table 8 was subjected to the same treatment to obtain NUM, TRS, and ARS of the INDA 2 co-differentially expressed genes. From Table 8, we can see that our method is better than other methods.Table 8
NUM, TRS, and ARS of five methods (INTA 2).
Methods HDNMF NMF DNMF LNMF GNMF GDNMF HNMF PAAD Num 326 326 330 321 321 325 319 TRS 2177.31 2039.68 2164.29 2035.33 2037.44 1974.33 1945.6 ARS 6.6789 6.2567 6.5585 6.3406 6.3472 6.0749 6.0991 ESCA Num 263 265 269 261 263 269 253 TRS 2383.93 2222.66 2274.52 2266.58 2170.57 2266.42 2298.05 ARS 9.0644 8.3874 8.4555 8.6842 8.2531 8.4254 9.0832 HNSC Num 245 248 247 240 247 248 242 TRS 5217.77 4997 5065.33 5044.97 4954.63 5092.82 5117.22 SRS 21.2970 20.1492 20.5074 21.0207 20.0592 20.5356 21.1455 COAD Num 341 338 340 334 335 339 324 TRS 2134.17 1995.88 2014.62 2027.45 1852.71 2010.03 1836.64 ARS 6.2586 5.9050 5.9254 6.0702 5.5305 5.9293 5.6686 CHOL Num 131 136 129 128 128 128 125 TRS 505.85 493.98 455.1 466.04 424.14 464.75 402.63 ARS 3.8615 3.6322 3.5280 3.6409 3.3136 3.6309 3.2210After the intersection of the co-differentially expressed genes selected by the five methods is removed, the co-differentially expressed genes excavated for each method are given in Table9. As showed in Table 9, the number of unique public genes mined by the NMF, DNMF, LNMF, GNMF, GDNMF, and HNMF methods is 0, 0, 4, 0, 0, and 0. The HDNMF method has been tapped into 5. Next comparing the obtained co-differentially expressed genes, we can obtain unique co-differentially expressed genes obtained only by our method, and these genes are necessary for the study of diseases in the future. Therefore, our method is more suitable for mining co-differentially expressed genes. Unique co-differentially expressed genes selected by each method have been indicated in italic.Table 9
Co-differentially expressed genes of five methods (INTA 2).
Methods Codifferentially expressed genes Unique NMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A SERPINA3 POSTN HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 DNMF KRT10 SPINK1 SERPINA3 CDH3 SERPINB5 TP63 SLPI HIF1A EGR1 ANXA5 DUSP1 0 LNMF DMBT1LDHB TFF3 HMGB1 ATF4 4 GNMF SERPINA3 COL1A1 MUC1 KRT7 MMP7 MUC6 GRN SPINK1 AGR2 0 GDNMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A POSTN SERPINA3 HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 HNMF ERBB2 CCND1 MUC1 EPCAM LGALS3 AGR2 CLDN4 IFI27 FSCN1 RAC1 DMBT1 HMGA1 NOTCH3 CD151 PRDX1 EIF5A 0 HDNMF AGR2AGT ALB ANPEP ANXA5 CCND1 CD151 CLDN4 COL1A1 CRP DMBT1 EIF5A EPCAM ERBB2 ERBB3 FSCN1 HMGA1 IFI27 KRT7 LGALS3 MMP7 MUC1 NOTCH3 PRDX1 RAC1 SERPINA3 5Some of the results from these and other studies are summarized below:(1) Unique co-differentially expressed genes mined by HDNMF on the INTA1 datasets are greater than the INTA 2 datasets. This huge difference is due to the complexity of the different datasets. For example, the composition of the INTA1 data has three diseases, and the INTA2 datasets constitute five diseases(2) The genes AGT, ANPEP, CRP, and ERBB3 in Tables8 and 10 have a higher correlation score with the experimental datasets, and these three genes are ignored by most mining methods. Therefore this reflects the accuracy of the HDNMF methodTable 10
Unique co-differentially expressed genes by HDNMF (INTA 2).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CRP C-reactive protein Acute pancreatitis and appendicitis Calcium ion binding and cholesterol binding 81.62 APCS AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 null ALB Albumin Analbuminemia and hyperthyroxinemia, familial hypalbuminemia Enzyme binding and chaperone binding 52.83 AFP ANPEP Alanyl aminopeptidase, membrane Tetrasomy 21 and myelophthisic anemia Virus receptor activity 40.34 LVRN


### 3.5. The Pathway Analysis

Joint biological processes of the selected genes selected in the experiment can be attributed to pathways that help us understand the advanced functions of biology and biological systems at the molecular level. We used the Kyoto Encyclopedia of Genes and Genomes (KEGG) online analysis tool to analyze co-differentially expressed genes identified by HDNMF. In this experiment, 500 genes identified were enclosed in KEGG, and we can obtain the corresponding disease pathway. The FDR of the HDNMF method and other methods are showed in Table11 (INTA 1) and Table 12 (INTA 2). The smaller the FDR, the better the results. So, the HDNMF method achieves better results.Table 11
The FDR of different methods (INTA 1).
Pathway HDNMF NMF DNMF LNMF GNMF GDNMF HNMF Ribosome 7.75E-91 3.16E-86 1.41E-102 9.94E-91 4.28E-93 1.41E-102 9.94E-91 Focal adhesion 6.52E-38 4.44E-35 1.8E-36 7.7E-38 1.8E-36 1.8E-36 7.7E-38 ECM-receptor interaction 6.67E-36 1.91E-32 1.91E-32 4.01E-34 1.91E-32 1.91E-32 4.01E-34 PathogenicEscherichia coli infection 8.01E-17 8.33E-17 1.92E-15 1.7E-15 1.92E-15 1.92E-15 4.91E-17 Leukocyte transendothelial migration 8.01E-17 8.33E-17 1.04E-16 1.08E-18 1.04E-16 1.04E-16 5.28E-18Table 12
The FDR of different methods (INTA 2).
Pathway HDNMF NMF DNMF LNMF GNMF GDNMF HNMF Ribosome 3.15E-100 1.74E-95 5.55E-84 1.98E-95 3.16E-86 4.28E-93 8.69E-98 Focal adhesion 4.1E-35 2.26E-32 2.43E-32 2.43E-32 2.43E-32 5.4E-31 2.43E-32 ECM-receptor interaction 1.8E-32 8.64E-31 9.13E-31 4.13E-29 9.13E-31 4.13E-29 4.13E-29 PathogenicEscherichia coli infection 8.07E-17 2.14E-18 2.21E-18 2.21E-18 8.33E-17 2.21E-18 2.21E-18 Antigen processing and presentation 3.48E-15 2.9E-15 3E-15 1.51E-16 3E-15 1.51E-16 3E-15Taking ECM-receptor interaction as an example, the literature has been shown to be closely related to PAAD [36], ESCA [37], and CHOL [38]. Other pathways can also prove their rationality by consulting the literature.


## 3.1. Datasets Description

The Cancer Genome Atlas (TCGA), as the largest cancer genome database, has immeasurable information value. The data included cholangiocarcinoma data (CHOL), esophageal cancer data (ESCA), pancreatic cancer data (PAAD), colorectal cancer data (COAD), and head and neck squamous cell carcinoma (HNSC) data. All of this data can be obtained from the TCGA database athttps://cancergenome.nih.gov/. Each dataset consists of two classes, normal samples and diseased samples. The dimension of the datasets is 20502. The rows of the data represent gene features, and the columns represent gene samples.Firstly, the CHOL data contained 9 normal samples and 36 diseased samples, the PAAD data contained 4 normal samples of 176 diseased samples, the HNSC data contained 20 normal samples and 198 diseased samples, the ESCA data contained 9 normal samples and 183 diseased samples, and the COAD data contained 19 normal samples and 262 diseased samples. Then, the normal samples are removed. Finally, we integrate the PAAD, ESCA, and CHOL data into datasets with dimensions of 20502∗395 (INTA 1) and integrate the PAAD, COAD, HNSC, ESCA, and CHOL data into one dataset with dimensions of 20502∗1055 (INTA 2).


## 3.2. Implementation Issues

### 3.2.1. Parameter Selection

As for the experiments, the datasets used were INTA 1 and INTA 2. For each test, 10-fold cross validation uses training data to adjust parameters. The advantage of cross validation is that all test datasets are independent. The regularized parameter matrix directly affects the classification results of the HDNMF method. In order to select the optimal parameters, we adjust the regularized parameters exponentially in a specific domain, such as from10-4 to 104. This section uses the tenfold cross validation method to automatically adjust the selection of the best regularized parameters in the corresponding training datasets in the following 10t:t∈-4,-3,…3,4 range. The results were shown in Figure 1.Figure 1
The influence of parameters on classification ACC (INTA 1 and INTA 2).As shown in the figure, the warmer the color, the higher the classification ACC. As can be seen from Figure1, dataset INTA 1 can get better results when α=1 and β∈10-4,10-2; as for dataset INTA 2, it can get better results when α∈102,104 and β∈10-4,104. In particular, the value of α has small significant effect on the classification result when β=102.


### 3.2.2. The Influence of Dimensions on Classification ACC

In this part, we do the HDNMF method when dimension is from 1 to 30 to explore the effect of dimension size on the classification effect. Based on the results in Figure2, the following points can be summarized:Figure 2
Influence of dimension on classification ACC.(1) When the dimension is too small, the classification ACC effect of all methods is not ideal, mainly due to the loss of a large amount of useful information due to the large dimensionality reduction(2) As the dimension increases, the effect of the classification ACC gradually increases and tends to be stable after a certain degree, which is mainly due to the large amount of information loss that can be avoided when the dimension is large enough


### 3.2.3. HDNMF Time Comparison

We compare the average runtime of all the methods on the two datasets in Table4. The experimental results were obtained by performing all methods on an Inter(R) Core(TM) i7-7700 CPU @ 3.6 GHz Windows Server with 64 G RAM. The time costs of the classification process for the all methods are listed. Obviously, the NMF method with constraints usually requires more runtime, and our proposed HDNMF method requires the longest runtime. The main reason is that the HDNMF method takes a lot of time in constructing the hypergraph and updates the hyperedge in each iterate. Therefore, how to speed up the proposed algorithm is an interesting work.


### 3.2.4. The Convergence Analysis

The updating rules for the HDNMF method in (20)-(22) are easy to understand and it can be demonstrated that the above rules are convergent. Figure 3 shows the convergence about two datasets; the red dashed lines are the error value of the HDNMF method. We can see that our method will converge in 100 iterations.Figure 3
The convergence performance for INTA1 and INTA2.


## 3.2.1. Parameter Selection

As for the experiments, the datasets used were INTA 1 and INTA 2. For each test, 10-fold cross validation uses training data to adjust parameters. The advantage of cross validation is that all test datasets are independent. The regularized parameter matrix directly affects the classification results of the HDNMF method. In order to select the optimal parameters, we adjust the regularized parameters exponentially in a specific domain, such as from10-4 to 104. This section uses the tenfold cross validation method to automatically adjust the selection of the best regularized parameters in the corresponding training datasets in the following 10t:t∈-4,-3,…3,4 range. The results were shown in Figure 1.Figure 1
The influence of parameters on classification ACC (INTA 1 and INTA 2).As shown in the figure, the warmer the color, the higher the classification ACC. As can be seen from Figure1, dataset INTA 1 can get better results when α=1 and β∈10-4,10-2; as for dataset INTA 2, it can get better results when α∈102,104 and β∈10-4,104. In particular, the value of α has small significant effect on the classification result when β=102.


## 3.2.2. The Influence of Dimensions on Classification ACC

In this part, we do the HDNMF method when dimension is from 1 to 30 to explore the effect of dimension size on the classification effect. Based on the results in Figure2, the following points can be summarized:Figure 2
Influence of dimension on classification ACC.(1) When the dimension is too small, the classification ACC effect of all methods is not ideal, mainly due to the loss of a large amount of useful information due to the large dimensionality reduction(2) As the dimension increases, the effect of the classification ACC gradually increases and tends to be stable after a certain degree, which is mainly due to the large amount of information loss that can be avoided when the dimension is large enough


## 3.2.3. HDNMF Time Comparison

We compare the average runtime of all the methods on the two datasets in Table4. The experimental results were obtained by performing all methods on an Inter(R) Core(TM) i7-7700 CPU @ 3.6 GHz Windows Server with 64 G RAM. The time costs of the classification process for the all methods are listed. Obviously, the NMF method with constraints usually requires more runtime, and our proposed HDNMF method requires the longest runtime. The main reason is that the HDNMF method takes a lot of time in constructing the hypergraph and updates the hyperedge in each iterate. Therefore, how to speed up the proposed algorithm is an interesting work.


## 3.2.4. The Convergence Analysis

The updating rules for the HDNMF method in (20)-(22) are easy to understand and it can be demonstrated that the above rules are convergent. Figure 3 shows the convergence about two datasets; the red dashed lines are the error value of the HDNMF method. We can see that our method will converge in 100 iterations.Figure 3
The convergence performance for INTA1 and INTA2.


## 3.3. Classification

Since HDNMF introduces label information into the NMF method, it is necessary to perform the test of the effect of the classification [28]. There are many improved methods for existing NMFs for classification, such as methods which were mentioned in [30, 31].Classification is a part of the most widely used technologies in the field of data mining. It is a technique for constructing a classifier based on the characteristics of experimental data and then using the classifier to classify samples of unknown categories. The classifier is generally divided into two stages of training and testing; in the training stage, the characteristics of the training datasets are first analyzed, and then an accurate description of the corresponding datasets is generated for each category; in the testing phase, the model classifies the test set to test its classification ACC. In the experimental part, we use the KNN method to classify the sample. For a given set of training samples with class labels, the similarity measure function is used to find the nearest neighbors that are most similar to the samples to be classified, and the decision is made according to the corresponding principle.The specific classification process (K-nearest neighbors, KNN) is as follows: (1) for the training sets, calculate the distance from the unknown sample to all known samples; (2) select the parameterK (where K is the most basic parameter in KNN, representing any number of neighbors); (3) for test sets, the unknown samples are classified into the largest number of categories in the sample according to the majority voting rule.


### 3.3.1. HDNMF for INTA 1 and INTA 2 Classification

For the dimension equal to 3 for INTA 1 and 5 for INTA 2 the classification ACC are listed in Table3. The optimal results are italic, and the experiment proves that our HDNMF method can achieve better classification results than other methods.Table 3
The classification ACC for all methods.
Methods ACC (INTA 1) ACC (INTA 2) NMF 80.00±0.00181 87.62±0.00195 DNMF 85.10±0.00390 90.79±0.00137 LNMF 83.77±0.00690 82.07±0.004504 GNMF 82.50±0.00140 89.52±0.00135 GDNMF 90.00±0.00009 90.48±0.00019 HNMF 87.50±0.00198 88.57±0.00175 HDNMF 92.50±0.00096 94.88±0.00082Table 4
The time comparison for all methods.
Methods Time/S (INTA 1) Time/S (INTA 2) NMF 6.4064 14.7599 DNMF 7.9114 22.3321 LNMF 7.4628 19.1296 GNMF 7.8372 21.0590 GDNMF 5.0958 16.3989 HNMF 56.5247 93.0869 HDNMF 29.3517 68.6402It is possible to conclude that the introduction of label information is the main reason for the significant classification ACC effect. Therefore, the HDNMF method can not only improve the interpretability of the NMF method, but also overcome the ambiguity of the unsupervised learning training samples.


## 3.3.1. HDNMF for INTA 1 and INTA 2 Classification

For the dimension equal to 3 for INTA 1 and 5 for INTA 2 the classification ACC are listed in Table3. The optimal results are italic, and the experiment proves that our HDNMF method can achieve better classification results than other methods.Table 3
The classification ACC for all methods.
Methods ACC (INTA 1) ACC (INTA 2) NMF 80.00±0.00181 87.62±0.00195 DNMF 85.10±0.00390 90.79±0.00137 LNMF 83.77±0.00690 82.07±0.004504 GNMF 82.50±0.00140 89.52±0.00135 GDNMF 90.00±0.00009 90.48±0.00019 HNMF 87.50±0.00198 88.57±0.00175 HDNMF 92.50±0.00096 94.88±0.00082Table 4
The time comparison for all methods.
Methods Time/S (INTA 1) Time/S (INTA 2) NMF 6.4064 14.7599 DNMF 7.9114 22.3321 LNMF 7.4628 19.1296 GNMF 7.8372 21.0590 GDNMF 5.0958 16.3989 HNMF 56.5247 93.0869 HDNMF 29.3517 68.6402It is possible to conclude that the introduction of label information is the main reason for the significant classification ACC effect. Therefore, the HDNMF method can not only improve the interpretability of the NMF method, but also overcome the ambiguity of the unsupervised learning training samples.


## 3.4. Co-Differentially Expressed Gene Selection

High-dimensional small sample data contain many unrelated or redundant features. In biological processes, most genes serve as the standard function of life support, so researchers focus on a small number of differentially expressed genes that play a critical role in life. Feature selection [32] is the easiest way to resolve these problems and identify differentially expressed genes. It can replace the standard data by choosing the largest amount of information without losing a lot of information. Here we show the experimental results and analysis of different methods for selecting common differentially expressed genes (co-differentially expressed genes) on integrated datasets. The relevant score (RS) refers to the correlation between genes and diseases. The higher the relevant score, the higher the correlation between genes and diseases. The number of genes (NUM) refers to the number of co-differentially expressed genes selected by each method characteristic that match the proven disease genes in GeneCards.How the GeneCards were used is described as follows: we visited the official website of GeneCards (https://www.genecards.org/). On the one hand, all types of diseased genes that have been validated can be obtained by entering the type of cancer at this web address (exporting the table for the determination of NUM). On the other hand, enter the gene name at this web address to get the details of the gene.


### 3.4.1. INTA 1 Co-Differentially Expressed Gene Selection Results

The HDNMF method and the NMF series of methods are used to select co-differentially expressed genes of the integrated datasets. We selected 500 genes for each method to evaluate the co-differentially expressed genes selected by using different methods.For the 500 co-differentially expressed genes selected by the seven methods, we compare the three diseases that make up the integrated INTA 1. The comparison data are from GeneCards (https://www.genecards.org/). The co-differentially expressed genes excavated by different methods are listed in Table 5: the number of genes (NUM), the total related score (TRS), and the average related score (ARS). Higher NUM, TRS, and ARS values indicate better method performance. From the table, we can see that the HDNMF method can achieve better experimental results.Table 5
NUM, TRS, and ARS of five methods (INTA 1).
Methods HDNMF NMF DNMF LNMF GNMF HNMF GDNMF PAAD Num 331 327 324 328 327 322 326 TRS 2385.79 2228.45 2351.36 2287.7 2339.88 2061.11 2350.54 ARS 7.2078 6.8148 7.2350 6.9535 7.1556 6.4010 7.2102 ESCA Num 252 257 248 253 254 243 248 TRS 2318.07 2305.29 2194.42 2231.10 2224.08 2185.63 2194.08 ARS 9.1987 8.9700 8.8485 8.8186 8.7562 8.9944 8.8471 CHOL Num 134 132 134 130 132 137 131 TRS 546.15 531.06 527.15 525.56 526.42 504.76 526.63 SRS 4.0758 4.0232 3.9936 4.0428 3.9880 3.6844 4.0201We summarize the co-differentially expressed genes in the shared part in Table6, including the co-differentially expressed genes of each method and the number of unique co-differentially expressed genes excavated by each method. Unique co-differentially expressed genes selected by each method have been indicated in italic. In the table, we can clearly see that there are 0 unique co-differentially expressed genes excavated by the DNMF, LNMF, GNMF, and GDNMF methods; 2 unique co-differentially expressed genes excavated by the NMF and HNMF methods; and 7 unique co-differentially expressed genes mined by our HDNMF method. This indicates that our method works well.Table 6
Co-differentially expressed genes of five methods (INTA 1).
Methods Codifferentially expressed genes Unique NMF MUC6 FSCN1 POSTN HIF1A SLC7A5 S100A2TKT TFRC TTR TP63 SPINK1 2 DNMF SPINK1CTGFS100A2ANPEP 0 LNMF CRPHIF1AEGR1STAT3SLC7A5TFRC 0 GNMF MUC6 FSCN1 POSTN SLC7A5 S100A2 TTR SPINK1 TPM1 CTGF 0 HNMF MUC6 FSCN1 POSTN SLC7A5 TTR TPM1 HIF1A TFRC CRP STAT3 EGR1 0 HDNMF ABCC3 BCL2L1 CRP RHOC TPM1 ERBB3 AGT CTSC HP ANPEP 7In Table7, we summarize the detailed information of the co-differentially expressed genes excavated only by the HDNMF method in the table by GeneCards, including the official name, related diseases, related GO annotations, RS, and paralog gene. All genes in the table have a high related score, which means that these genes can be regarded as important pathogenic genes to help disease research. Medical research in this area will also provide us with unexpected new discoveries. Therefore, the unique co-differentially expressed genes excavated by the HDNMF method can promote further research on these three diseases.Table 7
Unique co-differentially expressed genes by HDNMF (INTA 1).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CTSC Cathepsin C Papillon-Lefevre syndrome and Haim-Munk syndrome Identical protein binding and cysteine-type peptidase activity 76.89 TINAGL1 AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 Null RHOC Ras homolog family member C Breast disease and breast cancer GTP binding and obsolete signal transducer activity 64.59 RHOA HP Haptoglobin Anhaptoglobinemia andPlasmodium falciparum malaria Serine-type endopeptidase activity and hemoglobin binding 59.62 HPR BCL2L1 BCL2-like 1 B-cell lymphomas and follicular lymphoma Protein homodimerization activity and protein heterodimerization activity 49.62 BCL2 ABCC3 ATP binding cassette subfamily C member 3 Dubin-Johnson syndrome and extrahepatic cholestasis Transporter activity and ATPase activity 46.90 ABCC1From Table7, we can see the relevant information of the unique co-differentially expressed genes selected by the HDNMF method. ERBB3 has the highest relevance score. ERBB3 is a protein-coding gene. When it mutates, the tumor suppressor protein does not form properly. In pancreatic cancer, ERBB3 is a preferred dimerization partner for EGFR, and ERBB3 protein expression levels are directly related to the antiproliferative effects of erlotinib (EGFR-specific tyrosine kinase inhibitor) and transient knockdown of ERBB3 expression gain resistance to EGFR targeted therapy. This leads to the onset of cancer. ERBB3 has been shown to be associated with PAAD [33]. ERBB3 forms heterodimers with other kinase-active EGF (epidermal growth factor) receptor family members. Heterodimerization leads to activation of pathways leading to cell proliferation or differentiation. Amplification of this gene and/or overexpression of its protein has been reported in many cancers, such as PAAD, ESCA [34], and CHOL [35]. Therefore, mutations in one gene may be involved in the production of multiple cancers. This suggests that biologists can further study the link between different cancers.


### 3.4.2. INTA 2 Co-Differentially Expressed Gene Selection Results

Similar to Table5, Table 8 was subjected to the same treatment to obtain NUM, TRS, and ARS of the INDA 2 co-differentially expressed genes. From Table 8, we can see that our method is better than other methods.Table 8
NUM, TRS, and ARS of five methods (INTA 2).
Methods HDNMF NMF DNMF LNMF GNMF GDNMF HNMF PAAD Num 326 326 330 321 321 325 319 TRS 2177.31 2039.68 2164.29 2035.33 2037.44 1974.33 1945.6 ARS 6.6789 6.2567 6.5585 6.3406 6.3472 6.0749 6.0991 ESCA Num 263 265 269 261 263 269 253 TRS 2383.93 2222.66 2274.52 2266.58 2170.57 2266.42 2298.05 ARS 9.0644 8.3874 8.4555 8.6842 8.2531 8.4254 9.0832 HNSC Num 245 248 247 240 247 248 242 TRS 5217.77 4997 5065.33 5044.97 4954.63 5092.82 5117.22 SRS 21.2970 20.1492 20.5074 21.0207 20.0592 20.5356 21.1455 COAD Num 341 338 340 334 335 339 324 TRS 2134.17 1995.88 2014.62 2027.45 1852.71 2010.03 1836.64 ARS 6.2586 5.9050 5.9254 6.0702 5.5305 5.9293 5.6686 CHOL Num 131 136 129 128 128 128 125 TRS 505.85 493.98 455.1 466.04 424.14 464.75 402.63 ARS 3.8615 3.6322 3.5280 3.6409 3.3136 3.6309 3.2210After the intersection of the co-differentially expressed genes selected by the five methods is removed, the co-differentially expressed genes excavated for each method are given in Table9. As showed in Table 9, the number of unique public genes mined by the NMF, DNMF, LNMF, GNMF, GDNMF, and HNMF methods is 0, 0, 4, 0, 0, and 0. The HDNMF method has been tapped into 5. Next comparing the obtained co-differentially expressed genes, we can obtain unique co-differentially expressed genes obtained only by our method, and these genes are necessary for the study of diseases in the future. Therefore, our method is more suitable for mining co-differentially expressed genes. Unique co-differentially expressed genes selected by each method have been indicated in italic.Table 9
Co-differentially expressed genes of five methods (INTA 2).
Methods Codifferentially expressed genes Unique NMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A SERPINA3 POSTN HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 DNMF KRT10 SPINK1 SERPINA3 CDH3 SERPINB5 TP63 SLPI HIF1A EGR1 ANXA5 DUSP1 0 LNMF DMBT1LDHB TFF3 HMGB1 ATF4 4 GNMF SERPINA3 COL1A1 MUC1 KRT7 MMP7 MUC6 GRN SPINK1 AGR2 0 GDNMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A POSTN SERPINA3 HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 HNMF ERBB2 CCND1 MUC1 EPCAM LGALS3 AGR2 CLDN4 IFI27 FSCN1 RAC1 DMBT1 HMGA1 NOTCH3 CD151 PRDX1 EIF5A 0 HDNMF AGR2AGT ALB ANPEP ANXA5 CCND1 CD151 CLDN4 COL1A1 CRP DMBT1 EIF5A EPCAM ERBB2 ERBB3 FSCN1 HMGA1 IFI27 KRT7 LGALS3 MMP7 MUC1 NOTCH3 PRDX1 RAC1 SERPINA3 5Some of the results from these and other studies are summarized below:(1) Unique co-differentially expressed genes mined by HDNMF on the INTA1 datasets are greater than the INTA 2 datasets. This huge difference is due to the complexity of the different datasets. For example, the composition of the INTA1 data has three diseases, and the INTA2 datasets constitute five diseases(2) The genes AGT, ANPEP, CRP, and ERBB3 in Tables8 and 10 have a higher correlation score with the experimental datasets, and these three genes are ignored by most mining methods. Therefore this reflects the accuracy of the HDNMF methodTable 10
Unique co-differentially expressed genes by HDNMF (INTA 2).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CRP C-reactive protein Acute pancreatitis and appendicitis Calcium ion binding and cholesterol binding 81.62 APCS AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 null ALB Albumin Analbuminemia and hyperthyroxinemia, familial hypalbuminemia Enzyme binding and chaperone binding 52.83 AFP ANPEP Alanyl aminopeptidase, membrane Tetrasomy 21 and myelophthisic anemia Virus receptor activity 40.34 LVRN


## 3.4.1. INTA 1 Co-Differentially Expressed Gene Selection Results

The HDNMF method and the NMF series of methods are used to select co-differentially expressed genes of the integrated datasets. We selected 500 genes for each method to evaluate the co-differentially expressed genes selected by using different methods.For the 500 co-differentially expressed genes selected by the seven methods, we compare the three diseases that make up the integrated INTA 1. The comparison data are from GeneCards (https://www.genecards.org/). The co-differentially expressed genes excavated by different methods are listed in Table 5: the number of genes (NUM), the total related score (TRS), and the average related score (ARS). Higher NUM, TRS, and ARS values indicate better method performance. From the table, we can see that the HDNMF method can achieve better experimental results.Table 5
NUM, TRS, and ARS of five methods (INTA 1).
Methods HDNMF NMF DNMF LNMF GNMF HNMF GDNMF PAAD Num 331 327 324 328 327 322 326 TRS 2385.79 2228.45 2351.36 2287.7 2339.88 2061.11 2350.54 ARS 7.2078 6.8148 7.2350 6.9535 7.1556 6.4010 7.2102 ESCA Num 252 257 248 253 254 243 248 TRS 2318.07 2305.29 2194.42 2231.10 2224.08 2185.63 2194.08 ARS 9.1987 8.9700 8.8485 8.8186 8.7562 8.9944 8.8471 CHOL Num 134 132 134 130 132 137 131 TRS 546.15 531.06 527.15 525.56 526.42 504.76 526.63 SRS 4.0758 4.0232 3.9936 4.0428 3.9880 3.6844 4.0201We summarize the co-differentially expressed genes in the shared part in Table6, including the co-differentially expressed genes of each method and the number of unique co-differentially expressed genes excavated by each method. Unique co-differentially expressed genes selected by each method have been indicated in italic. In the table, we can clearly see that there are 0 unique co-differentially expressed genes excavated by the DNMF, LNMF, GNMF, and GDNMF methods; 2 unique co-differentially expressed genes excavated by the NMF and HNMF methods; and 7 unique co-differentially expressed genes mined by our HDNMF method. This indicates that our method works well.Table 6
Co-differentially expressed genes of five methods (INTA 1).
Methods Codifferentially expressed genes Unique NMF MUC6 FSCN1 POSTN HIF1A SLC7A5 S100A2TKT TFRC TTR TP63 SPINK1 2 DNMF SPINK1CTGFS100A2ANPEP 0 LNMF CRPHIF1AEGR1STAT3SLC7A5TFRC 0 GNMF MUC6 FSCN1 POSTN SLC7A5 S100A2 TTR SPINK1 TPM1 CTGF 0 HNMF MUC6 FSCN1 POSTN SLC7A5 TTR TPM1 HIF1A TFRC CRP STAT3 EGR1 0 HDNMF ABCC3 BCL2L1 CRP RHOC TPM1 ERBB3 AGT CTSC HP ANPEP 7In Table7, we summarize the detailed information of the co-differentially expressed genes excavated only by the HDNMF method in the table by GeneCards, including the official name, related diseases, related GO annotations, RS, and paralog gene. All genes in the table have a high related score, which means that these genes can be regarded as important pathogenic genes to help disease research. Medical research in this area will also provide us with unexpected new discoveries. Therefore, the unique co-differentially expressed genes excavated by the HDNMF method can promote further research on these three diseases.Table 7
Unique co-differentially expressed genes by HDNMF (INTA 1).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CTSC Cathepsin C Papillon-Lefevre syndrome and Haim-Munk syndrome Identical protein binding and cysteine-type peptidase activity 76.89 TINAGL1 AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 Null RHOC Ras homolog family member C Breast disease and breast cancer GTP binding and obsolete signal transducer activity 64.59 RHOA HP Haptoglobin Anhaptoglobinemia andPlasmodium falciparum malaria Serine-type endopeptidase activity and hemoglobin binding 59.62 HPR BCL2L1 BCL2-like 1 B-cell lymphomas and follicular lymphoma Protein homodimerization activity and protein heterodimerization activity 49.62 BCL2 ABCC3 ATP binding cassette subfamily C member 3 Dubin-Johnson syndrome and extrahepatic cholestasis Transporter activity and ATPase activity 46.90 ABCC1From Table7, we can see the relevant information of the unique co-differentially expressed genes selected by the HDNMF method. ERBB3 has the highest relevance score. ERBB3 is a protein-coding gene. When it mutates, the tumor suppressor protein does not form properly. In pancreatic cancer, ERBB3 is a preferred dimerization partner for EGFR, and ERBB3 protein expression levels are directly related to the antiproliferative effects of erlotinib (EGFR-specific tyrosine kinase inhibitor) and transient knockdown of ERBB3 expression gain resistance to EGFR targeted therapy. This leads to the onset of cancer. ERBB3 has been shown to be associated with PAAD [33]. ERBB3 forms heterodimers with other kinase-active EGF (epidermal growth factor) receptor family members. Heterodimerization leads to activation of pathways leading to cell proliferation or differentiation. Amplification of this gene and/or overexpression of its protein has been reported in many cancers, such as PAAD, ESCA [34], and CHOL [35]. Therefore, mutations in one gene may be involved in the production of multiple cancers. This suggests that biologists can further study the link between different cancers.


## 3.4.2. INTA 2 Co-Differentially Expressed Gene Selection Results

Similar to Table5, Table 8 was subjected to the same treatment to obtain NUM, TRS, and ARS of the INDA 2 co-differentially expressed genes. From Table 8, we can see that our method is better than other methods.Table 8
NUM, TRS, and ARS of five methods (INTA 2).
Methods HDNMF NMF DNMF LNMF GNMF GDNMF HNMF PAAD Num 326 326 330 321 321 325 319 TRS 2177.31 2039.68 2164.29 2035.33 2037.44 1974.33 1945.6 ARS 6.6789 6.2567 6.5585 6.3406 6.3472 6.0749 6.0991 ESCA Num 263 265 269 261 263 269 253 TRS 2383.93 2222.66 2274.52 2266.58 2170.57 2266.42 2298.05 ARS 9.0644 8.3874 8.4555 8.6842 8.2531 8.4254 9.0832 HNSC Num 245 248 247 240 247 248 242 TRS 5217.77 4997 5065.33 5044.97 4954.63 5092.82 5117.22 SRS 21.2970 20.1492 20.5074 21.0207 20.0592 20.5356 21.1455 COAD Num 341 338 340 334 335 339 324 TRS 2134.17 1995.88 2014.62 2027.45 1852.71 2010.03 1836.64 ARS 6.2586 5.9050 5.9254 6.0702 5.5305 5.9293 5.6686 CHOL Num 131 136 129 128 128 128 125 TRS 505.85 493.98 455.1 466.04 424.14 464.75 402.63 ARS 3.8615 3.6322 3.5280 3.6409 3.3136 3.6309 3.2210After the intersection of the co-differentially expressed genes selected by the five methods is removed, the co-differentially expressed genes excavated for each method are given in Table9. As showed in Table 9, the number of unique public genes mined by the NMF, DNMF, LNMF, GNMF, GDNMF, and HNMF methods is 0, 0, 4, 0, 0, and 0. The HDNMF method has been tapped into 5. Next comparing the obtained co-differentially expressed genes, we can obtain unique co-differentially expressed genes obtained only by our method, and these genes are necessary for the study of diseases in the future. Therefore, our method is more suitable for mining co-differentially expressed genes. Unique co-differentially expressed genes selected by each method have been indicated in italic.Table 9
Co-differentially expressed genes of five methods (INTA 2).
Methods Codifferentially expressed genes Unique NMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A SERPINA3 POSTN HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 DNMF KRT10 SPINK1 SERPINA3 CDH3 SERPINB5 TP63 SLPI HIF1A EGR1 ANXA5 DUSP1 0 LNMF DMBT1LDHB TFF3 HMGB1 ATF4 4 GNMF SERPINA3 COL1A1 MUC1 KRT7 MMP7 MUC6 GRN SPINK1 AGR2 0 GDNMF CCND1 IFI27 FSCN1 LGALS3 RAC1 CLDN4 PRDX1 NOTCH3 COL1A1 HMGA1 EIF5A POSTN SERPINA3 HIF1A TP63 EGR1 SERPINB5 CDH3 KRT10 SPINK1 SLPI DUSP1 0 HNMF ERBB2 CCND1 MUC1 EPCAM LGALS3 AGR2 CLDN4 IFI27 FSCN1 RAC1 DMBT1 HMGA1 NOTCH3 CD151 PRDX1 EIF5A 0 HDNMF AGR2AGT ALB ANPEP ANXA5 CCND1 CD151 CLDN4 COL1A1 CRP DMBT1 EIF5A EPCAM ERBB2 ERBB3 FSCN1 HMGA1 IFI27 KRT7 LGALS3 MMP7 MUC1 NOTCH3 PRDX1 RAC1 SERPINA3 5Some of the results from these and other studies are summarized below:(1) Unique co-differentially expressed genes mined by HDNMF on the INTA1 datasets are greater than the INTA 2 datasets. This huge difference is due to the complexity of the different datasets. For example, the composition of the INTA1 data has three diseases, and the INTA2 datasets constitute five diseases(2) The genes AGT, ANPEP, CRP, and ERBB3 in Tables8 and 10 have a higher correlation score with the experimental datasets, and these three genes are ignored by most mining methods. Therefore this reflects the accuracy of the HDNMF methodTable 10
Unique co-differentially expressed genes by HDNMF (INTA 2).
Official gene symbol Official name Related diseases Related GO annotations RS Paralog gene ERBB3 Erb-B2 receptor tyrosine kinase 3 Retroperitoneal leiomyosarcoma and fatal congenital contracture syndrome 2 Protein homodimerization activity and transferase activity, transferring phosphorus-containing groups 94.06 ERBB4 CRP C-reactive protein Acute pancreatitis and appendicitis Calcium ion binding and cholesterol binding 81.62 APCS AGT Angiotensinogen Renal tubular dysgenesis and hypertension, essential Growth factor activity and serine-type endopeptidase inhibitor activity 65.57 null ALB Albumin Analbuminemia and hyperthyroxinemia, familial hypalbuminemia Enzyme binding and chaperone binding 52.83 AFP ANPEP Alanyl aminopeptidase, membrane Tetrasomy 21 and myelophthisic anemia Virus receptor activity 40.34 LVRN


## 3.5. The Pathway Analysis

Joint biological processes of the selected genes selected in the experiment can be attributed to pathways that help us understand the advanced functions of biology and biological systems at the molecular level. We used the Kyoto Encyclopedia of Genes and Genomes (KEGG) online analysis tool to analyze co-differentially expressed genes identified by HDNMF. In this experiment, 500 genes identified were enclosed in KEGG, and we can obtain the corresponding disease pathway. The FDR of the HDNMF method and other methods are showed in Table11 (INTA 1) and Table 12 (INTA 2). The smaller the FDR, the better the results. So, the HDNMF method achieves better results.Table 11
The FDR of different methods (INTA 1).
Pathway HDNMF NMF DNMF LNMF GNMF GDNMF HNMF Ribosome 7.75E-91 3.16E-86 1.41E-102 9.94E-91 4.28E-93 1.41E-102 9.94E-91 Focal adhesion 6.52E-38 4.44E-35 1.8E-36 7.7E-38 1.8E-36 1.8E-36 7.7E-38 ECM-receptor interaction 6.67E-36 1.91E-32 1.91E-32 4.01E-34 1.91E-32 1.91E-32 4.01E-34 PathogenicEscherichia coli infection 8.01E-17 8.33E-17 1.92E-15 1.7E-15 1.92E-15 1.92E-15 4.91E-17 Leukocyte transendothelial migration 8.01E-17 8.33E-17 1.04E-16 1.08E-18 1.04E-16 1.04E-16 5.28E-18Table 12
The FDR of different methods (INTA 2).
Pathway HDNMF NMF DNMF LNMF GNMF GDNMF HNMF Ribosome 3.15E-100 1.74E-95 5.55E-84 1.98E-95 3.16E-86 4.28E-93 8.69E-98 Focal adhesion 4.1E-35 2.26E-32 2.43E-32 2.43E-32 2.43E-32 5.4E-31 2.43E-32 ECM-receptor interaction 1.8E-32 8.64E-31 9.13E-31 4.13E-29 9.13E-31 4.13E-29 4.13E-29 PathogenicEscherichia coli infection 8.07E-17 2.14E-18 2.21E-18 2.21E-18 8.33E-17 2.21E-18 2.21E-18 Antigen processing and presentation 3.48E-15 2.9E-15 3E-15 1.51E-16 3E-15 1.51E-16 3E-15Taking ECM-receptor interaction as an example, the literature has been shown to be closely related to PAAD [36], ESCA [37], and CHOL [38]. Other pathways can also prove their rationality by consulting the literature.


## 4. Conclusions

We presented a novel matrix factorization method called Hypergraph Regularized Discriminative Nonnegative Matrix Factorization (HDNMF). The method introduced hypergraph and discriminative label information into the standard NMF method. On the one hand, the hypergraph can find high-order geometric relations that are neglected by simple graphs; on the other hand, discriminative label information can make the method have supervisory functions. Experiments have shown that HDNMF can achieve better results than standard NMF and its improved methods. Since the hypergraph was in the process of construction, it got a lot of time to build the hyperedge; the program running time is longer than other methods. How to accelerate the proposed algorithm is an interesting work in the forthcoming work.



---

*Source: 7081674-2019-08-19.xml*