Globality constrained adaptive graph regularized non‐negative matrix factorization for data representation

IET Image ProcessingVolume 16, Issue 10 p. 2577-2592 ORIGINAL RESEARCHOpen Access Globality constrained adaptive graph regularized non-negative matrix factorization for data representation Yanfeng Sun, Corresponding Author Yanfeng Sun [email protected] Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, China Correspondence Yanfeng Sun, Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, China. Email: [email protected]Search for more papers by this authorJie Wang, Jie Wang orcid.org/0000-0002-8116-4386 Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorJipeng Guo, Jipeng Guo Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorYongli Hu, Yongli Hu Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorBaocai Yin, Baocai Yin Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this author Yanfeng Sun, Corresponding Author Yanfeng Sun [email protected] Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, China Correspondence Yanfeng Sun, Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, China. Email: [email protected]Search for more papers by this authorJie Wang, Jie Wang orcid.org/0000-0002-8116-4386 Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorJipeng Guo, Jipeng Guo Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorYongli Hu, Yongli Hu Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this authorBaocai Yin, Baocai Yin Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing University of Technology, No. 100, Pingleyuan, Chaoyang District, Beijing, ChinaSearch for more papers by this author First published: 06 May 2022 https://doi.org/10.1049/ipr2.12510 This research AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Benefiting from the good physical interpretations and low computational complexity, non-negative matrix factorization (NMF) has attracted wide attentions in data representation learning tasks. Some graph-based NMF approaches make the learned representation encode the topological structure by the local graph Laplacian regularizer, which improves the discriminant ability of data representation. However, the performance of graph-based NMF methods depend heavily on the quality of the predefined graph and the complexity of models is high. Here, a globality constrained adaptive graph regularized non-negative matrix factorization for data representation (GCAG-NMF) model is proposed, which not only uses the self-representation characteristics of data to learn an adaptive graph to describe the sample relationship more accurately, but also proposes a graph factorization technique to reduce the complexity of the model and improve the discriminative ability of data representation. Then, an iterative optimizing strategy with low complexity and strict convergence guarantee is developed to optimize the objective function. Experimental results on some databases demonstrate the effectiveness of the proposed model. 1 INTRODUCTION With the increasing development of collection technology, the scale of data collected is increasing, which results in the curse of dimensionality. And, it brings some huge challenges, such as the huge calculation, storage costs and feature redundancy [1]. Thus, learning a low-dimensional and discriminative representation of original data is necessary. Naturally, some dimension reduction methods are proposed and have achieved great successes, such as principal component analysis (PCA) [2], linear discriminant analysis (LDA) [3], and locality preserving projection (LPP) [4]. However, they are weak to provide a straightforward physical interpretation. Non-negative matrix factorization (NMF) [5] gradually became the most popular dimension reduction methods due to its good psychological and physiological interpretations. The main idea of NMF algorithm can be simply described as follows: for any non-negative data feature matrix X $\mathbf {X}$ , NMF algorithm tries to find two non-negative matrices U $\mathbf {U}$ and V $\mathbf {V}$ , so that the constraint X ≈ U V T $\mathbf {X} \approx \mathbf {U}\mathbf {V}^T$ is satisfied. Thus, the NMF allows only non-subtractive combinations of non-negative components. This non-negativity constraints result in the parts-based representation of NMF. In practice, NMF has been applied to many applications, such as document clustering [6], image segmentation [7], text mining [8], image clustering [9, 10] and biological data mining [11]. Due to the good properties of NMF, various approaches [12-15] are proposed to extend the standard NMF by adding additional constraints. Orthogonal NMF [16] reduces the redundant information of data representation by imposing the orthogonality constraints on the factor matrices. NMF by joint locality-constrained and ℓ2, 1-norm regularization (NMF2L) [17] performs both NMF and local constraints to obtain a coefficient representation matrix with row sparsity, which efficiently removes the redundant features. Although NMF and the above extensions have achieved the satisfactory performance, the inherent topological structure of the data itself is neglected. In fact, the high-dimensional data usually are embedded in a non-linear manifold space with low dimension. To make full use of the geometric information, graph regularized NMF (GNMF) [18] was proposed, which ensures that the learned representation preserves the original geometric relationships by graph Laplacian regularizer. We all know that there is often noises in the data, Graph regularized sparse NMF factorisation for imagery de-noising [14] considers the group sparsity and the geometric structure of data by introducing ℓ2, 1-norm and local structure preservation constraints in the objective function. Graph regularized ℓp smooth NMF for data representation [19] model can discover the intrinsic geometry information of the data and get a smoother and more accurate solution by adding graph regularization and ℓp smoothing constraints. In addition, data and feature manifold geometry are important for dimensionality reduction. Then, to preserve the manifold structure of both the feature space and sample space, graph dual regularization NMF (DNMF) [20] was proposed, which focuses on two-sided information. Dual-GNMF with sparse and orthogonal constraints [21] makes use of bi-orthogonal constraints, which can avoid the non-correspondence between data representation and basic vectors. Orthogonal dual GNMF for co-clustering [22] preserves the geometrical structures of data manifold and feature manifold by constructing two different graphs, and ensures the orthogonality of factor matrices, making it have better performance. Although the above graph-based NMF representation learning approaches have achieved improved performance in the data clustering tasks. There are some potential problems, which easily results in sub-optimal representation. First, the above methods adopts the predefined and fixed graph Laplacian regularizer to explore topological structure, which limits the flexibility of model. Second, the K-nearest-neighbor method is usually adopted to construct the graph affinity matrix. Thus, the performance of graph-based NMF models significantly rely on the quality of graph, that is, the parameter K. In other words, the graph are usually not optimal during the representation learning process. That is, once the graphs are constructed by specific algorithms, they will be unchanged during the process of matrix factorization. In order to overcome this limitation, several methods have been proposed. For example, Flexible NMF with adaptively learned graph rbegularization [23] model in which we simultaneously perform the graph learning and matrix factorization tasks. Because this method learns the dynamic graph from the original graph, it will also be seriously affected by the original noise graph. In addition, the similarity matrix is automatically learned from the data. Adaptive local learning regularized non-negative matrix factorization (ALLRNMF) [24] approach not only achieves the adaptive learning, but also reveals a more flexible ability by learning the data similarity matrix. Regularized NMF with adaptive local structure learning [25] can simultaneously perform matrix decomposition and similarity learning, which mainly focuses on local manifold structure. Double manifolds regularized NMF for data representation [26] model which jointly learns an adaptive affinity matrix combined with NMF, and the learned affinity matrix guides the NMF to fit clustering task. Although the graph structures in them can be adaptively learned during NMF process, and these methods have also achieved advanced performance. However, they still have some disadvantages. These methods only focus on the intrinsic local geometric structure of the data and ignore the global structure of the original high-dimensional data. And it is sensitive to noise and outliers. In addition, the calculation of Laplacian matrix may bring additional computational costs. To address the aforementioned issues, we propose a novel adaptive GNMF model named as globality constrained adaptive graph regularized non-negative matrix factorization (GCAG-NMF). The main differences and contributions of the proposed GCAG-NMF are as follows: Different from GNMF and the other graph-based NMF methods which adopt the predefined and fixed graph, GCAG-NMF jointly learns an adaptive graph and adopts it to guide the NMF. According to self-representation property of data, the suitable or even optimal graph with global structure could be adaptively obtained during the process of representation learning, which improves the flexibility of model. Instead of graph Laplacian regularizer, we propose a novel geometric structure preservation technique, that is, graph factorization. Graph factorization is a cosine distance metric, which not only helps the learned representation encode the structural information but also improves the discriminant ability and robustness. And, the graph factorization technique could avoid the additional complexity of calculating the Laplacian matrix. It has great advantages in processing large-scale databases and improves the applicability of the model. An iterative optimization strategy with low complexity and convergence guarantee is proposed to solve the designed model. Experiments on several benchmark databases have demonstrated the effectiveness of GCAG-NMF not only in terms of clustering performance, but also the computational costs. The rest of the paper is organized as follows: In Section 2, we give a brief review of NMF and GNMF models. In Section 3, the main model in this paper is presented, and the corresponding optimization algorithm is provided. Extensive experimental results in clustering tasks are shown in Section 4. Finally, we provide some concluding remarks in Section 5. 2 RELATED WORKS Here, we briefly introduce the closely related works, that is, NMF [5] and GNMF [18] model. The main notations in this article are summarized in Table 1. TABLE 1. Main notations and their descriptions Notation Meaning A $\mathbf {A}$ , a $\mathbf {a}$ , a Matrix, vector, scalar n The number of samples D The dimension of data feature r The number of reduced dimension ∥ · ∥ F 2 $\Vert \cdot \Vert _F^2$ The Frobenius norm 1 ∈ R n $\mathbf {1} \in \mathbb {R}^{n }$ The vector with all entries as 1 X ∈ R D × n $\mathbf {X} \in \mathbb {R}^{D \times n}$ Data feature matrix W ∈ R n × n $\mathbf {W} \in \mathbb {R}^{n \times n}$ Graph affinity matrix U ∈ R D × r $\mathbf {U} \in \mathbb {R}^{D \times r}$ Basis matrix V ∈ R n × r $\mathbf {V} \in \mathbb {R}^{n \times r}$ Coefficient representation matrix Z ∈ R n × n $\mathbf {Z} \in \mathbb {R}^{n \times n}$ Self-expressive coefficient matrix G ∈ R n × r $\mathbf {G} \in \mathbb {R}^{n \times r}$ Graph factorization matrix α, β The trade-off parameters Non-negative matrix factorization model The NMF aims to find two non-negative matrices by minimizing the approximation error between the product of two non-negative factor matrices and the original data matrix. Given a non-negative data matrix X = [ x 1 , x 2 , … , x n ] ∈ R D × n $\mathbf {X} =[\mathbf {x}_1, \mathbf {x}_2, \ldots , \mathbf {x}_n ] \in \mathbb {R}^{D\times n}$ , in which n and D are the number of samples and the feature dimension of each sample. U ∈ R D × r $\mathbf {U} \in \mathbb {R}^{D \times r}$ can be regarded as the basis matrix, V ∈ R n × r $\mathbf {V} \in \mathbb {R}^{n \times r}$ can be regarded as the coefficient representation matrix. NMF model can be mathematically expressed by the following objective function: min U ≥ 0 , V ≥ 0 ∥ X − U V T ∥ F 2 , \begin{equation} \min \limits _{\mathbf {U} \ge 0, \mathbf {V} \ge 0} \Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2, \end{equation} (1)where ∥ · ∥ F 2 $\Vert \cdot \Vert _F^2$ is the Frobenius norm of a matrix. Owing to the non-negativity of factor matrices, NMF learns a parts-based data representation, which is easy to explain in many real applications. A large number of researches have proved that the objective function of model (1) is convex only on U $\mathbf {U}$ or V $\mathbf {V}$ . Therefore, it is not practical to find the global minimum of model (1). In order to optimize the objective function of NMF model, Lee et al. [27] proposed the following iterative updating algorithm. The detailed iterative rules are as follows: u i k ← u i k ( X V ) i k ( U V T V ) i k , \begin{equation} u_{ik}\leftarrow u_{ik}\frac{(\mathbf {X}\mathbf {V})_{ik}}{(\mathbf {U}\mathbf {V}^T\mathbf {V})_{ik}}, \end{equation} (2) v j k ← v j k ( X T U ) j k ( V U T U ) j k . \begin{equation} v_{jk}\leftarrow v_{jk}\frac{(\mathbf {X}^T\mathbf {U})_{jk}}{(\mathbf {V}\mathbf {U}^T\mathbf {U})_{jk}}. \end{equation} (3) Graph regularized NMF model In fact, the intrinsic geometric structure of data is great importance for significantly improving the discriminant ability of data representation. To incorporate the intrinsic geometric structure into the standard NMF, GNMF [18] was proposed, in which intrinsic geometric structure information was encoded by a pre-fixed graph affinity matrix. Then, learned representation preserves the intrinsic geometric structure by the graph Laplacian regularizer. As shown in reference [18], the objective function of GNMF model is as follows: min U ≥ 0 , V ≥ 0 ∥ X − U V T ∥ 2 + λ tr ( V T L V ) , \begin{equation} \min \limits _{\mathbf {U} \ge 0, \mathbf {V} \ge 0}\Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert ^2 +\lambda \text{tr}(\mathbf {V}^T\mathbf {L}\mathbf {V}), \end{equation} (4)where L = D − W $\mathbf {L} = \mathbf {D}-\mathbf {W}$ is the graph Laplacian matrix corresponding to the predefined graph affinity matrix W ∈ R n × n $\mathbf {W} \in \mathbb {R}^{n \times n}$ , in which D $\mathbf {D}$ is a diagonal matrix and D j j = ∑ i = 1 n W i j $\mathbf {D}_{jj}=\sum \nolimits _{i=1}^n \mathbf {W}_{ij}$ . The intrinsic geometric structure of data itself are integrated by minimizing the second term of (4). Although the GNMF and some other graph-based NMF approaches [18, 20] have achieved the satisfied experimental results, there still exist some drawbacks and challenges. The performance of graph-based NMF methods significantly depends on the quality of the predefined affinity graph. The neighbor graph constructed by K-nearest neighbor or ε-nearest neighbor is easy to be inaccurate due to outliers and noises. And, the existing parameter K or ε heavily affects the quality of the graph. Usually, the proper/optimal parameter K or ε is not easy to determine, especially for unsupervised clustering tasks. Constructing graph matrix also brings some additional computational costs. Concretely, the complexity of constructing graph affinity matrix is O(n2), which is not scalable for large-scale databases. In addition, the graphs are predefined and fixed during the representation learning process, which results in sub-optimal representation and limits the flexibility of model. 3 THE PROPOSED MODEL Here, we first propose a novel extension of NMF named globality constrained adaptive graph regularized NMF (GCAG-NMF), then develop a simple and effective optimization algorithm to solve the proposed GCAG-NMF model. Finally, the convergence analysis and complexity analysis of our algorithm are analyzed. Motivation and objective function The main motivation of GCAG-NMF is to adaptively learn the graph by the data itself. Fortunately, the self-representation property of data can help us achieve this goal [28-30]. Specially, we assume that each sample can be linearly reconstructed from other samples, and minimize the error between the original data and the linear reconstruction result. The graph learning model in our algorithm can be defined as follows: min Z ≥ 0 , Z = Z T , Z 1 = 1 ∥ X − X Z ∥ F 2 , \begin{equation} \min \limits _{\mathbf {Z} \ge \mathbf {0}, \mathbf {Z} = \mathbf {Z}^T, \mathbf {Z}\mathbf {1}=\mathbf {1}} \Vert \mathbf {X} - \mathbf {X} \mathbf {Z}\Vert _F^2, \end{equation} (5)where Z ∈ R n × n $\mathbf {Z} \in \mathbb {R}^{n \times n}$ is the self-representation coefficient matrix, 1 $\mathbf {1}$ is the vector of all ones. Due to the meaningful affinity constraints Z ≥ 0 $\mathbf {Z} \ge \mathbf {0}$ , Z = Z T $\mathbf {Z} = \mathbf {Z}^T$ and Z 1 = 1 $\mathbf {Z} \mathbf {1} = \mathbf {1}$ , self-representation coefficient Z $\mathbf {Z}$ could be regarded as a graph matrix, in which element zij denotes the probability that samples x i $\mathbf {x}_i$ and x j $\mathbf {x}_j$ belong to the same subspace, that is, the similarity of x i $\mathbf {x}_i$ and x j $\mathbf {x}_j$ . Different from the K-nearest-neighbor graph, the self-representation graph learned by (5) encodes the global geometric structure of data. Thus, the GNMF model and adaptive graph learning model can be integrated into following joint learning model. min U , V , Z ∥ X − U V T ∥ F 2 + α ∥ X − X Z ∥ F 2 + β tr ( V T L V ) s.t. U ≥ 0 , V ≥ 0 , Z ≥ 0 , Z = Z T , Z 1 = 1 , \begin{equation} \begin{aligned} \min \limits _{\mathbf {U}, \mathbf {V}, \mathbf {Z}} \ &\Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2+\alpha \Vert \mathbf {X} -\mathbf {X} \mathbf {Z}\Vert _F^2+\beta \text{tr}(\mathbf {V}^T\mathbf {L}\mathbf {V}) \\ \text{s.t.} \ & \mathbf {U} \ge 0, \mathbf {V} \ge 0, \mathbf {Z} \ge 0, \mathbf {Z} = \mathbf {Z}^T, \mathbf {Z}\mathbf {1}=\mathbf {1}, \end{aligned} \end{equation} (6)where α and β are the trade-off parameters to balance the importance of the above three terms. And, the L = D − Z $\mathbf {L} = \mathbf {D} - \mathbf {Z}$ is the adaptive graph Laplacian matrix, which preserve the intrinsic geometric structure of data itself. For convenience, we express the model of (6) as AGNMF. In (6), the optimal graph and discriminative representation could be learned simultaneously, which improve the flexibility and robustness of model. However, the optimizing strategy of (6) still faces the challenges of high complexity. Specifically, once the matrix Z $\mathbf {Z}$ is learned, calculating the Laplacian matrix requires O(n2) computational complexity during optimizing V $\mathbf {V}$ . Compared with standard NMF with O(n) complexity, this will limit the scalability of model (6). Importantly, the graph Laplacian regularizer is a Euclidean-based distance metric, which can be reformulated as Tr ( V T L V ) = ∑ i , j = 1 n ∥ V ( i , : ) − V ( j , : ) ∥ 2 2 z i j , \begin{equation} \text{Tr}(\mathbf {V}^T \mathbf {L} \mathbf {V}) = \sum \limits _{i,j=1}^n \Vert \mathbf {V}(i,:) - \mathbf {V}(j,:)\Vert _2^2 z_{ij}, \end{equation} (7)From (7), with the help of self-representation coefficient matrix Z $\mathbf {Z}$ , graph Laplacian regularizer minimizes the Euclidean distance of similar samples to preserve the intrinsic local geometric structure of data. As the Euclidean distance metric is not robust to noises, improving the distance measure is essential to enhance the discriminant ability and robustness. Thus, a graph factorization technique is introduced to help the representation encode the graph structure information rather than Laplacian regularizer. Concretely, we decompose the self-representation coefficient matrix Z $\mathbf {Z}$ by the following form: Z = G G T , G ≥ 0 , G ∈ R n × r , \begin{equation} \mathbf {Z} = \mathbf {G} \mathbf {G}^T, \mathbf {G} \ge \mathbf {0}, \mathbf {G} \in \mathbb {R}^{n \times r}, \end{equation} (8)where G $\mathbf {G}$ encodes the clustering structure information of the original data. Specifically, we have the following equation about the element zij of Z $\mathbf {Z}$ : z i j = G ( i , : ) G ( j , : ) T = | G ( i , : ) | | G ( j , : ) | c o s θ , \begin{equation} z_{ij} = \mathbf {G}(i,:)\mathbf {G}(j,:)^T = |\mathbf {G}(i,:)||\mathbf {G}(j,:)|cos\theta , \end{equation} (9)where θ is the angle of G ( i , : ) $\mathbf {G}(i,:)$ and G ( j , : ) $\mathbf {G}(j,:)$ . Let | G ( i , : ) | = | G ( j , : ) | = 1 $|\mathbf {G}(i,:)|=|\mathbf {G}(j,:)|=1$ , we have zij = cosθ. Naturally, if zij = 0, θ = 90°; if zij = 1, θ = 0°. Thus, the similarity zij reflects the angle distance, that is, cosine distance metric. Then (9) can be considered as the cosine distance metric in a sense, which can enlarge the margin between different classes. Hence, the graph factorization not only helps G $\mathbf {G}$ encode the graph structure but also improves the discriminant ability. Then, maximizing the consistency between G $\mathbf {G}$ and low-dimensional representation V $\mathbf {V}$ helps V $\mathbf {V}$ to encode the geometric structure. Specifically, if G = V $\mathbf {G} = \mathbf {V}$ , we have z i j = V ( i , : ) V ( j , : ) T = | V ( i , : ) | | V ( j , : ) | c o s θ $z_{ij} = \mathbf {V}(i,:)\mathbf {V}(j,:)^T = |\mathbf {V}(i,:)||\mathbf {V}(j,:)|cos\theta$ , that is, V $\mathbf {V}$ encodes the geometric structure Z $\mathbf {Z}$ and the similarity metric of representation V $\mathbf {V}$ is cosine distance metric, rather than Euclidean distance of graph Laplacian regularizer. Compared with the graph Laplacian regularization, graph factorization technique have some advantages. First of all, it does not need to construct predefined graph structure. And, graph factorization technique can reduce the complexity of the model, which the complexity of the graph Laplacian regularization is O(n2), and the complexity of the factorization technique is only O(n). Thus, this technology not only improves the execution efficiency of the model, but also enhances the flexibility and applicability of the model. Second, and more importantly, graph factorization technique is measured using the cosine distance, which expands the boundaries between different classes and helps G $\mathbf {G}$ to encode the structure of graph, so that the learned representation G $\mathbf {G}$ has clear clustering structure. Then, the G $\mathbf {G}$ is used to guide the learning of the coefficient representation V $\mathbf {V}$ of NMF, which effectively improves the discriminant ability of coefficient representation V $\mathbf {V}$ . Finally, by combining (6) with (8), the proposed GCAG-NMF model can be formulated as follows: min U , V , G O ( U , V , G ) = ∥ X − U V T ∥ F 2 + α ∥ X − X G G T ∥ F 2 + β ∥ G − V ∥ F 2 s.t. U ≥ 0 , V ≥ 0 , G ≥ 0 , G G T 1 = 1 . \begin{equation} \begin{aligned} \min \limits _{\mathbf {U}, \mathbf {V}, \mathbf {G}} & \mathcal {O}(\mathbf {U}, \mathbf {V}, \mathbf {G}) = \Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2+\alpha \Vert \mathbf {X} -\mathbf {X} \mathbf {G}\mathbf {G}^T\Vert _F^2\\ &+\beta \Vert \mathbf {G} -\mathbf {V}\Vert _F^2\\ \text{s.t.} \quad & \mathbf {U} \ge \mathbf {0} , \mathbf {V} \ge \mathbf {0}, \mathbf {G} \ge \mathbf {0}, \mathbf {G}\mathbf {G}^T\mathbf {1}=\mathbf {1}. \end{aligned} \end{equation} (10)Obviously, the model (10) requires only data features as input, which is simple and highly operable. Once the model is optimized, we can conduct the standard K-means on the learned representation V $\mathbf {V}$ to obtain the final clustering tasks. Model optimization Here, we propose an efficient iterative updating algorithm to optimize the model (10). It employs an alternative iteration strategy by optimizing one variable while fixing the other variables. This procedure repeats until the convergence condition is satisfied. The detailed updating process for each variable in the tth iteration was described as follows. 3.2.1 Update U $\mathbf {U}$ with fixing others If the other variables are fixed, the objective function associated with variable U $\mathbf {U}$ is min U ≥ 0 ∥ X − U V T ∥ F 2 , \begin{equation} \min \limits _{\mathbf {U} \ge \mathbf {0}}\Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2, \end{equation} (11)which is the same as NMF. Thus, we can directly obtain the solution of U $\mathbf {U}$ by the following updating rule: u i k ← u i k ( X V ) i k ( U V T V ) i k . \begin{equation} u_{ik}\leftarrow u_{ik}\frac{(\mathbf {X}\mathbf {V})_{ik}}{(\mathbf {U}\mathbf {V}^T\mathbf {V})_{ik}}. \end{equation} (12) 3.2.2 Update V $\mathbf {V}$ with fixing others If the other variables are fixed, we can obtain the following objective function associated with variable V $\mathbf {V}$ : min V ≥ 0 ∥ X − U V T ∥ F 2 + + β ∥ G − V ∥ F 2 , \begin{equation} \min \limits _{\mathbf {V} \ge \mathbf {0}} \Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2++\beta \Vert \mathbf {G} -\mathbf {V}\Vert _F^2, \end{equation} (13)The corresponding Lagrangian function is L 1 = ∥ X − U V T ∥ F 2 + + β ∥ G − V ∥ F 2 + Tr ( Φ V T ) , \begin{equation} \mathcal {L}_1 = \Vert \mathbf {X} -\mathbf {U} \mathbf {V}^T\Vert _F^2++\beta \Vert \mathbf {G} -\mathbf {V}\Vert _F^2+\text{Tr}(\Phi \mathbf {V}^T), \end{equation} (14)where Φ is the Lagrange multipliers for non-negative constraints V ≥ 0 $\mathbf {V} \ge \mathbf {0}$ . Taking the partial derivative of L 1 $\mathcal {L}_1$ with respect to V $\mathbf {V}$ and setting it to zero: ∂ L 1 ∂ V = − 2 X T U + 2 V U T U − 2 G β + 2 β V + Φ = 0 . \begin{equation} \frac{\partial \mathcal {L}_1}{\partial \mathbf {V}}=-2\mathbf {X}^T\mathbf {U}+2\mathbf {V}\mathbf {U}^T\mathbf {U}-2\mathbf {G}\beta +2\beta \mathbf {V}+\Phi = \mathbf {0}. \end{equation} (15)Using the KKT condition ϕjkvjk = 0, we get the following equation: − ( X T U ) j k v j k + ( V U T U ) j k v j k − ( G β ) j k v j k + ( β V ) j k v j k = 0 . \begin{equation} \begin{aligned} &-(\mathbf {X}^T\mathbf {U})_{jk}v_{jk}+(\mathbf {V}\mathbf {U}^T\mathbf {U})_{jk}v_{jk}-(\mathbf {G}\beta )_{jk}v_{jk}\\ &+(\beta \mathbf {V})_{jk}v_{jk}=0. \end{aligned} \end{equation} (16)Formula (16) is further simplified to obtain the following updating rule of V $\mathbf {V}$ : v j k ← v j k ( X T U + β G ) j k ( V U T U + β V ) j k . \begin{equation} v_{jk}\leftarrow v_{jk}\frac{(\mathbf {X}^T\mathbf {U}+\beta \mathbf {G})_{jk}}{(\mathbf {V}\mathbf {U}^T\mathbf {U}+\beta \mathbf {V}