Multi‐view intrinsic low‐rank representation for robust face recognition and clustering

IET Image ProcessingEarly View ORIGINAL RESEARCH PAPEROpen Access Multi-view intrinsic low-rank representation for robust face recognition and clustering Zhi-yang Wang, orcid.org/0000-0001-6471-7797 School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, ChinaSearch for more papers by this authorStanley Ebhohimhen Abhadiomhen, School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Department of Computer Science, University of Nigeria, Nsukka, NigeriaSearch for more papers by this authorZhi-feng Liu, School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Jingkou New-Generation Information Technology Industry Institute, Jiangsu University, Zhenjiang, Jiangsu, ChinaSearch for more papers by this authorXiang-jun Shen, Corresponding Author xjshen@ujs.edu.cn School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Correspondence Xiang-jun Shen, School of Computer Science and Communication Engineering, JiangSu University, JiangSu 212013, Zhenjiang, P.R. China; Shu-ying Li, School of Automation, Xi'an University of Posts, and Telecommunications, Xi'an 710121, Shaanxi, P.R. China. Email: xjshen@ujs.edu.cn; angle_lisy@163.comSearch for more papers by this authorWen-yun Gao, Nanjing LES Information Technology Co., LTD, Nanjing, JiangSu, China College of Computer and Information, Hohai University, Nanjing, JiangSu, ChinaSearch for more papers by this authorShu-ying Li, Corresponding Author angle_lisy@163.com School of Automation, Xi'an University of Posts & Telecommunications, Xi'an, Shaanxi, China Correspondence Xiang-jun Shen, School of Computer Science and Communication Engineering, JiangSu University, JiangSu 212013, Zhenjiang, P.R. China; Shu-ying Li, School of Automation, Xi'an University of Posts, and Telecommunications, Xi'an 710121, Shaanxi, P.R. China. Email: xjshen@ujs.edu.cn; angle_lisy@163.comSearch for more papers by this author Zhi-yang Wang, orcid.org/0000-0001-6471-7797 School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, ChinaSearch for more papers by this authorStanley Ebhohimhen Abhadiomhen, School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Department of Computer Science, University of Nigeria, Nsukka, NigeriaSearch for more papers by this authorZhi-feng Liu, School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Jingkou New-Generation Information Technology Industry Institute, Jiangsu University, Zhenjiang, Jiangsu, ChinaSearch for more papers by this authorXiang-jun Shen, Corresponding Author xjshen@ujs.edu.cn School of Computer Science and Communication Engineering, JiangSu University, Zhenjiang, JiangSu, China Correspondence Xiang-jun Shen, School of Computer Science and Communication Engineering, JiangSu University, JiangSu 212013, Zhenjiang, P.R. China; Shu-ying Li, School of Automation, Xi'an University of Posts, and Telecommunications, Xi'an 710121, Shaanxi, P.R. China. Email: xjshen@ujs.edu.cn; angle_lisy@163.comSearch for more papers by this authorWen-yun Gao, Nanjing LES Information Technology Co., LTD, Nanjing, JiangSu, China College of Computer and Information, Hohai University, Nanjing, JiangSu, ChinaSearch for more papers by this authorShu-ying Li, Corresponding Author angle_lisy@163.com School of Automation, Xi'an University of Posts & Telecommunications, Xi'an, Shaanxi, China Correspondence Xiang-jun Shen, School of Computer Science and Communication Engineering, JiangSu University, JiangSu 212013, Zhenjiang, P.R. China; Shu-ying Li, School of Automation, Xi'an University of Posts, and Telecommunications, Xi'an 710121, Shaanxi, P.R. China. Email: xjshen@ujs.edu.cn; angle_lisy@163.comSearch for more papers by this author First published: 02 May 2021 https://doi.org/10.1049/ipr2.12232AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract In the last years, subspace-based multi-view face recognition has attracted increasing attention and many related methods have been proposed. However, the most existing methods ignore the specific local structure of different views. This drawback can cause these methods' discriminating ability to degrade when many noisy samples exist in data. To tackle this problem, a multi-view low-rank representation method is proposed, which exploits both intrinsic relationships and specific local structures of different views simultaneously. It is achieved by hierarchical Bayesian methods that constrain the low-rank representation of each view so that it matches a linear combination of an intrinsic representation matrix and a specific representation matrix to obtain common and specific characteristics of different views. The intrinsic representation matrix holds the consensus information between views, and the specific representation matrices indicate the diversity among views. Furthermore, the model injects a clustering structure into the low-rank representation. This approach allows for adaptive adjustment of the clustering structure while pursuing the optimization of the low-rank representation. Hence, the model can well capture both the relationship between data and the clustering structure explicitly. Extensive experiments on several datasets demonstrated the effectiveness of the proposed method compared to similar state-of-the-art methods in classification and clustering. 1 INTRODUCTION Multi-view face data has become pervasive in the real-world. For example, several images taken from different angles can represent different views of a person, which can be exploited by methods to improve face recognition accuracy. Therefore, a naive method such as in [1-3] concatenates the features in different views to obtain a common feature representation. However, this strategy ignores the correlation between different views, which then incites the ”curse of dimensionality” and result in high computational time. Besides, the methods based on feature concatenation do not comprehensively explore underlying data distribution among different views. So, the concatenated feature representation may not be meaningful. This drawback expedited some more researches, which focused on integrating complementary information through graph construction. In [4], Kumar et al. proposed a method, which found a unified graph by reducing the disagreement between different views. In [5], Xia et al. attempted to separate noise from graphs through a low-rank transition probability matrix. In [6], Huang et al. performed face recognition tasks through an embedding graph learned simultaneously from different views. In [7], Cao et al. proposed a constrained multi-view method, which applied pairwise constraint to facilitate finding a consistent graph. In [8], Li et al. attained high accuracy and low computational complexity by using a bipartite graph to approximate the similarity graphs of different views. Nonetheless, these methods designed based on traditional spectral learning [9-11] did not utilize the self-expressiveness property of data. Therefore, they can be susceptible to noise in the different views. Recently, multi-view subspace methods based on Low-Rank Representation (LRR) [12] and Sparse Subspace Clustering (SSC) [13] have become increasingly popular. Because both baseline methods utilize the self-expressiveness property of data to represent each data point as a linear combination of others in order to learn a computationally efficient data representation. Even so, some studies such as [14] suggested more that LRR may indeed provide a more robust approach to tackle the above drawback considering that it captures a global structure of data. Thus, many methods, which LRR inspired have been proposed over the years. For instance, Structure-Constrained Low-Rank Representation [15] analyzed the structure of multiple disjoint sub-spaces, which is more general for computer vision data. Latent Low-Rank Representation (LatLRR) [16] was proposed for latent detection in images using subspace segmentation. Robust multi-view subspace learning [17] used dual low-rank decompositions to uncover local manifold structures. Multi-Spectral Low-Rank Structured Dictionary Learning (MLSDL) [18] applied low-rank structural incoherence term to reduce the discrepancy between different views. Low-Rank Graph Optimization for Multi-View Dimensionality Reduction (LRGO-MVDR) [19] used an adaptive non-negative weight vector to capture the complementary structure in different face images. Common Subspace based Low-Rank and Joint Sparse Representation (CSLRJSR) [20] found common face representation fusing discriminative features in different views through a joint low-rank and sparse methods. Despite these efforts, two main drawbacks limit the overall performance of the subspace methods. First, because subspace methods assume that similar data points stay near each other, they often do not perform well in practical cases when the distribution of data points within a subspace is arbitrary due to excessive noise. This weakness is because two unrelated data points may stay connected, and it is even more feasible knowing that these methods utilize a fixed shared representation. Secondly, most of these methods consider only the consistency of multi-view data or enhance only the diversity in these data similar to [21], so both common structure and intrinsic structure can mix up easily to degrade recognition accuracy. Regarding the above limitations, we propose a novel method, which integrates clustering structure into low-rank representation to pursue optimization of multi-view intrinsic representation, which is used subsequently for face recognition. To achieve this, first, we follow the intuition of Hierarchical Bayes [22-24] by decomposing the low-rank representation learned for each view into two matrices, namely the intrinsic representation and the specific representation. The intrinsic representation matrix captures the common features of different views. And the specific representation matrix captures specific features of each view through diversity regularization. Then the core of our idea is introduced by imposing the spectral clustering structure into our model in order to guarantee more optimum representation that is devoid of specific redundant information. In other words, an affinity matrix in introduced into our model that captures the clustering structure explicitly, such that the clustering structure is integrated into the low-rank representation to allow our method to find a better intrinsic representation adaptively so that it can be used for face recognition reducing the impact of noisy data. Therefore, our model can be used for face recognition and clustering simultaneously. Figure 1 shows the framework of our proposed method. FIGURE 1Open in figure viewerPowerPoint Framework of our proposed method. For multi-view data matrix X i , we obtain intrinsic low-rank matrix Z 0 by diversity constraint and affinity matrix W by k-nearest neighbor graph. Z 0 holds the intrinsic information across each view and Z i preservers difference between different views. Then with a dynamic approach on the affinity matrix and intrinsic low-rank matrix, the local manifold and clustering structure are simultaneously learned The main contributions of our work are: 1. We propose a novel method where a low-rank representation learned for each view is composed of two low-rank components: an intrinsic representation across views and a specific view representation in each view. The intrinsic representation captures the common structure across all views, while the specific representation captures the diversity among views. With these structures, we can discover the correlation and discrepancy in multi-view face data comprehensively. 2. Furthermore, different from the existing methods which utilize a fixed representation of multi-view face data, our method obtains an intrinsic representation of the data flexibly. We achieve this by integrating a clustering structure into the low-rank representation to allow the clustering structure to be adjusted adaptively while pursuing an ideal low-rank face representation that will suppress the effect of noisy data. As a result, our proposed model can be used for face recognition and clustering simultaneously. 3. We conduct experiments on face benchmark datasets, and the experimental results demonstrate the effectiveness of our proposed method. 2 RELATED WORK Many studies have been conducted in search of an efficient technique that can explore the complementary information in multi-view face data to improve recognition accuracy. The preliminary methods such as feature concatenation [1-3], spectral graph [4-8] are not computationally efficient, and they do not provide robustness against noise. Recently, there has been much more attention on the subspace approach because it provides a mechanism to tackle the above problems. So far, LRR [12] and SSC [13] are the two most effective approaches in subspace segmentation, and they both utilize the self-expressiveness property of data [25]. SSC, however, does not capture a global structure of the data. The reason is that it uses the L1 norm to find a sparse representation of data points individually, which means that SSC may be susceptible to noise. LRR, on the other hand, provides an efficient tool to recover an actual manifold structure from corrupted data by using the data itself as the dictionary (self-expressiveness matrix) to seek a representation matrix with the lowest rank [26]. To illustrate further, let's assume that we have a set of data points X = { x 1 , … , x n } ∈ R d × n where n is the number of data points and d is the dimensionality of each data point, then LRR will represent each data point by a linear combination of others to capture a global structure of data, that is, x i = X p i , i = 1 , … , n , where P = { p 1 , … , p n } ∈ R n × n is the self-expressiveness matrix, which can be found in the following way: min P Rank ( P ) + ∥ E ∥ 2 , 1 s.t. , X = X P + E . (1) However, Equation (1) is hard to achieve due to the discrete nature of the rank function. Instead, the nuclear norm provides a good convex surrogate as follows: min P ∥ P ∥ ∗ + ∥ E ∥ 2 , 1 s . t . , X = X P + E . (2) For the data x i , it is one of the k classes and can be connected by all the data points with weighted value p i j , and such a weighted value can be seen easily as the similarity between them. In other words, the degree of correlation between x i and x j can be characterized by p i j , where closer data points will have a larger weighted value. Therefore, p i j can correctly depict the local manifold structure of data when two similar data points reconstruct one another. Considering the robustness of LRR, many methods have been proposed recently for multi-view face representation based on its principles. In [18], Jing et al. learn common and separate dictionaries. The main idea is to utilize these dictionaries to explore the correlated information and the complementary information simultaneously to find a robust face representation that will enhance face recognition. In [28], Xu et al. proposed a structured low-rank matrix recovery method to solve the problem of huge divergence in multi-view face data. This approach eliminates graph embedding by introducing a class label matrix to learn a discriminative unified matrix. Yet, it fails to consider the specific local structure of different views so, the problem may persist. In [17], Ding et al. had previously used a dual low-rank decomposition approach to separate local structures from the global structure. However, the dual minimization of singular values may cause the face representation matrix to be of high rank since the rank may not approximate correctly in practice [29]. Alternatively, Zheng et al. [30], Brbic and Kopriva [27], and Wang et al. [20] combine the idea of SSC and LRR to find a joint subspace representation that is robust against large-scale noise. In [31], Meng et al. proposed a Multi-View Low-Rank Preserving Embedding (MvLPE) method, which learns the complimentary features in different views by maximizing the agreement between an individual view and a centroid view. Nonetheless, these methods equally ignored the specific local structure of different views as [28]. Therefore, inspired by [17], we utilize the L1 norm to capture the specific local structure of each view such that a common face representation learned is through a cooperative effort to guarantee a more optimum solution that suppresses the effect of noisy data adaptably. 3 THE PROPOSED APPROACH In this section, we present a novel multi-view learning algorithm, which can capture the relationship among data and unfold the clustering structure at the same time. The notations are summarized in Table 1. TABLE 1. The notations used in this paper Notations Explanation X i Data matrix of ith view P i Self-representation matrix of ith view E i Error matrix of ith view Z i Intrinsic structure matrix of ith view Z 0 Common structure matrix of ith view W Affinity matrix L Laplacian matrix 3.1 Structure consistency LRR is a promising method, which intends to build a good affinity matrix by using self-expression. However, it ignores the important data locality. To better exploit local manifold structure, we follow the intuition of Hierarchical Bayes [22-24] and assumed that P can be decomposed into two parts; one is the intrinsic structure shared across each view and the other is the diversity structure of each view. We utilize the intrinsic matrix Z 0 to capture the general structure across different views. Simultaneously, we use Z i to capture specific features of each view. We utilize the constraint P i = Z 0 + Z i to model the assumption above. We envision that the parts unique to each view are relatively sparse structures, so we utilize l 1 norm on Z i and leading to the problem below: min Z , E ∑ i = 1 v P i ∗ + Z i 1 + ∑ i = 1 v E i 2 , 1 s . t . , X i = X i P i + E i , P i = Z 0 + Z i , i = 1 , … , v . (3) 3.2 Diversity regularization Multi-view face data contain intrinsic and complementary features and general shared information at the same time. In this way, the intrinsic feature matrices should have difference between each other. Meanwhile, Z 0 and Z i should be different and complementary. In another word, each view should have less similarity and have weak connection with common feature as well. To achieve this, we introduce a diversity regularization formulated as follow: Θ Z u , Z v = Tr Z u T Z v = ∑ i = 1 n ∑ j = 1 n Z u i j Z v i j , (4)where Z v and Z u are different views. According to the equation above, when Θ ( Z u , Z v ) is minimized, one of Z v and Z u value will be large and the other will be small. Therefore, we can utilize this to compute the differences between each view. We formulate the diversity regularization in our model as: ∑ i = 1 v Θ Z i , Z 0 + ∑ i = 1 v ∑ i ≠ j Θ Z i , Z j = ∑ i = 1 v Tr Z i T Z 0 + ∑ i = 1 v ∑ i ≠ j Tr Z i T Z j . (5) 3.3 Integrating spectral clustering structure Self-representation is a good method to capture the local manifold structure. The representation coefficient is larger when two different samples are similar while it is smaller when two samples are different. However, in many real-world applications, there are possibilities that similar samples belong to different classes. Thus, only a self-representation of data will mislead the clustering task, for the reason that many samples sharing different labels may be clustered together. Therefore, directly using the self-representation of data to clustering will not get ideal results at all time. Because local manifold captured by self-representation is not an ideal clustering structure. Basically, an ideal graph should ensure that samples with the same label should have strong connection while samples from different classes should have weak connection or even not be connected. Therefore, to exactly separate sample data into k clusters, we need a clustering structure with k connected components. In this way, we can deal with the problem that samples with different labels have strong connection. To this end, we impose a rank constraint on the Laplacian matrix derived from the affinity matrix so that it can be an ideal clustering structure. This rank constraint can guarantee that the affinity matrix has exact the k connected components which is the same as clusters number . Meanwhile, the estimated affinity matrix will dynamically approximate local manifold captured from self-representation to exploit structure information among data. To solve the problem, we define an ideal affinity matrix W = { w i T , . . , w n T } ∈ R n × n that can not only utilize self-representation structure, but also is an ideal clustering structure. Theorem 1.If affinity matrix W is nonnegative, the multiplicity K of the eigenvalue zero of the Laplacian matrix L is equal to the number of connected components in the graph associated with W. Given an affinity matrix W, Theory 1 indicates that if r a n k ( L ) = n − k , then W is an ideal affinity matrix that can separate the data points into exact k clusters. Moreover we hope W dynamically approximates Z used to capture the local manifold structure. Thus, we impose a F-norm to appropriately minimize the loss between Z 0 and W. Thus, we have the following formulation: min Z 0 − W F 2 + λ 1 ∑ i = 1 v P i ∗ + Z i 1 + λ 2 ∑ i = 1 v Tr Z i T Z 0 + λ 2 ∑ i = 1 v ∑ i ≠ j Tr Z i T Z j + ∑ i = i v E i 2 , 1 s.t. X i = X i P i + E i , P i = Z i + Z 0 , rank ( L ) = n − k , W i j ≥ 0 , ∑ j W i j = 1 . (6) 3.4 Optimization We propose an optimization method to approximatively solve the objective function in Equation (6) and apply the Augmented Lagrange Multiplier (ALM) [32, 33] to iteratively update all the variables. We denote Θ i as the ith smallest eigenvalue of L. Because L is a positive semidefinite matrix, so Θ i ≥ 0 . In this way, the rank constraint in Equation (6) can be removed after given a large enough λ. Therefore, the problem (6) can be rewritten as follows. min Z 0 − W F 2 + λ 1 ∑ i = 1 v P i ∗ + Z i 1 + λ 2 ∑ i = 1 v Tr Z i T Z 0 + λ 2 ∑ i = 1 v ∑ i ≠ j Tr Z i T Z j + λ 3 ∑ i = 1 k θ i + ∑ i = i v E i 2 , 1 s.t. X i = X i P i + E i , P i = Z i + Z 0 , W i j ≥ 0 , ∑ j W i j = 1 . (7) When λ 3 is large enough, the term ∑ i = 1 k θ i will equal to 0 and the constraint rank ( L ) = n − k will be satisfied. According to Ky Fan's Theorem [19], ∑ i = 1 k θ i is equal to minimizing Tr ( F T L F ) subject to F T F = I . Then Equation (7) can be rewritten as: min Z 0 − W F 2 + λ 1 ∑ i = 1 v P i ∗ + Z i 1 + λ 2 ∑ i = 1 v tr Z i T Z 0 + λ 2 ∑ i = 1 v ∑ i ≠ j tr Z i T Z j + λ 3 Tr F T L F + ∑ i = i v E i 2 , 1 s.t. X i = X i P i + E i , P i = Z i + Z 0 , F T F = I , W i j ≥ 0 , ∑ j W i j = 1 . (8) We introduce auxiliary variables U i = P i to make the objective function become separable and can be easily solved. Then, we use the Augmented Lagrange Multiplier (ALM) [34] to obtain the following augmented Lagrangian function. min E i , D i , Z 0 , Z i , U i ∥ Z 0 − W ∥ F 2 + λ 1 ∑ i = 1 v U i ∗ + Z i 1 + λ 2 ∑ i = 1 v tr Z i T Z 0 + λ 2 ∑ i = 1 v ∑ i ≠ j tr Z i T Z j + λ 3 Tr F T L F + ∑ i = i v E i 2 , 1 + ∑ i = i v H i , X i − X i P i − E i + ∑ i = i v i , P i − U i + ∑ i = i v K i , P i − Z i − Z 0 + μ 2 ∑ i = i v X i − X i P i − E i F 2 + μ 2 ∑ i = i v P i − U i F 2 + P i − Z i − Z 0 F 2 . (9) Step 1: Solving U—We update the variable U by fixing the other variables and dropping the unconcerned terms. Then, we have the following subproblem with respect to the variable U: min U λ 1 ∥ U i ∥ ∗ + J i , P i − U i + μ 2 ∥ P i − U i ∥ F 2 = λ 1 ∥ U i ∥ ∗ + μ 2 ∥ U i − P i + J i μ ∥ F 2 . (10) We solve the subproblem in Equation (10) by using Singular Value Thresholding [35] to get the closed-form solution as U i = S λ 1 / μ D i + J i μ , (11)where S τ [ · ] is a shrinkage thresholding operator which is defined by S μ [ M ] = U H μ [ Σ ] V T , H μ [ Σ ] = min ( 0 , Σ + μ ) + max ( 0 , Σ − μ ) . (12) Step 2: Solving E—Similar to the U subproblem, the optimization problem with respect to E can be formulated as: min E i λ 4 ∥ E i ∥ 2 , 1 + μ 2 ∥ E i − X i − X i P i + H i μ ∥ F 2 . (13) Based on lemma 3.3 in [36], we have the optimal solution as follows: [ E i ] : , k = ∥ [ A i ] : , k ∥ 2 − 1 / μ ∥ [ A i ] : , k ∥ 2 [ A i ] : , k , if [ A i ] : , k > 1 / μ 0 , else . A i = X i − X i P i + H i μ and [ A i ] : , k is the kth column of A i . Step 3: Solving P i —Fixing the other variables, the optimization problem concerned with P i can be rewritten as: min P H i , X i − X i P i − E i + J i , P i − U i + K i , P i − Z i − Z 0 + μ 2 ( ∥ X i − X i P i − E i ∥ F 2 + ∥ P i − U i ∥ F 2 + ∥ P i − Z i − Z 0 ∥ F 2 ) . (14) With respect to P i and setting the derivative of above, we obtain the following solution: P i = ( X i T X i + 2 I ) − 1 X i T X i + Y i μ − E i + Z i + Z 0 + K i μ + U i − J i μ . (15) Step 4: Solving Z i —According to the above subproblems, we can rewrite the optimization problem about Z i as: min Z i λ 1 ∥ Z i ∥ 1 + μ 2 ∥ Z i ∥ F 2 + t r ( Z i T A ) , (16)where A = λ 3 ( Z 0 + ∑ i ≠ j Z j ) + μ ( P i − Z 0 + K i μ ) . This optimization problem can be solved via Singular Value Thresholding. Defining z as an element of Z i , the following is obtained: min z λ 1 ∥ Z i ∥ 1 + μ z 2 2 + a i j z . (17)The solution in Equation (17) is obtained as z = H λ 1 μ ( − a / μ ) . Step 5: Solving Z 0 —By fixing the other variables and removing the irrelevant terms, we rewrite the subproblem as: min Z 0 ∥ Z 0 − W ∥ F 2 + λ 2 ∑ i = 1 v t r ( Z i T Z 0 ) + ∑ i = i v K i , P i − Z i − Z 0 + μ 2 ∑ i = i v ∥ P i − Z i − Z 0 ∥ F 2 . (18) Setting the derivative of the above with respect to Z 0 to zero we obtain the following solution: Z 0 = ( 2 I + 2 λ 2 L + μ v ) − 1 × 2 W + μ ∑ i = 1 v ( P i − K i μ − Z i ) − λ 3 ∑ i = 1 v Z i . (19) Step 6: Solving W—We update W by solving the following problem: Γ ( W ) = ∥ Z 0 − W ∥ F 2 + λ 3 T r ( F T L F ) s . t . , W i j ≥ 0 , ∑ j W i j = 1 . (20) The problem Equation (20) is independent for different i, therefore we can solve the following problem for each i. Γ ( w i j ) = ∑ j ( Z 0 i j − w i j ) 2 + λ 3 ∑ j ∥ f i − f j ∥ 2 2 w i j s . t . , w i j ≥ 0 , ∑ j w i j = 1 . (21) Denoting t i j = ∥ f i − f j ∥ 2 2 , and t i as vector with the ith element equal to t i j , Equation (21) above can be written as [37, 38]: min w i T 1 = 1 , w i j ≥ 0 ∥ w i − z 0 − λ 3 2 t i ∥ 2 2 . (22) The Lagrangian function of Equation (22) is: Γ = 1 2 ∥ w i − z 0 − λ 3 2 t i ∥ 2 2 − η ( w i T 1 − 1 ) − β i T w i , (23)where η and β i are the Lagrangian multipliers. By taking the derivative of Equation (23) with respect to w i and setting to zero, we can obtain: w i − z 0 − λ 3 2 t i − η 1 − β i = 0 , (24) Ac