Multiple Kernel Clustering with Shifted Laplacian on Grassmann Manifold

Authors: grassmann  manifold
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 11:37:09

Abstract: Multiple kernel clustering (MKC) has garnered considerable attention, as their efficacy in handling nonlinear data in high-dimensional space. However, current MKC methods have three primary issues: (1) Solely focuse on clustering information while neglecting energy information and potential noise interference within the kernel; (2) The inherent manifold structure in the high-dimensional space is complex, and they lack the insufficient exploration of topological structure; (3) Most encounter cubic computational complexity, posing a formidable resource consumption challenge. To tackle the above issues, we propose a novel MKC method with shifted Laplacian on Grassmann manifold (sLGm). Firstly, sLGm constructs $r$-rank shifted Laplacian and subsequently reconstructs it, retaining the clustering-related and energy-related information while reducing the influence of noise. Additionally, sLGm introduces a Grassmann manifold for partition fusion, which can preserve topological information in the high-dimensional space. Notably, an optimal consensus partition can be concurrently learnt from above two procedures, thereby yielding the clustering assignments, and the computational complexity of the whole procedure drops to the quadratic. Conclusively, a comprehensive suite of experiments is executed to roundly prove the effectiveness of sLGm.

Main paper: 
 introduction: 
 Clustering stands as a cornerstone in the realm of unsupervised learning, playing a vital role across various applications [25] . Its primary objective lies in partitioning similar data into the same cluster, thereby minimizing intra-cluster sample differences while maximizing inter-cluster differences [8] . In practical scenarios, the data landscape involves diverse sources, resulting in various clustering methods including graph learning based [23] , kernel learning based [15] and subspace learning based [29] etc. Among them, the kernel learning based methods have obtained considerable attention owing to their efficacy in handing nonlinear data, which improves its separability by mapping nonlinear data into the reproducing Hilbert kernel space [13, 26] . Therein, multiple kernel-based clustering (MKC) can circumvent the choice of kernel function and the adjustment of kernel parameters, seamlessly integrating the information from each kernel. Typically, it involves constructing the kernel pool comprising base kernels, capturing the underlying structure from these kernels.Broadly speaking, MKC methods can be roughly classified into two groups: spectral graph-based (SG) and kernel 𝑘-means-based (KKM). For the SG methods, the primary focus lies in learning a superb affinity graph within the kernel space, and thus use spectral clustering (SC) for obtaining the clustering assignments [4, 17] . For the KKM methods, they are customary to integrate base kernel into an optimal one, subsequently employing kernel 𝑘-means on this optimal kernel for acquiring the clustering assignments [10, 22] . Nevertheless, when confronted with a kernel matrix of 𝑛 × 𝑛, these two groups typically encounter cubic computational complexity, presenting a formidable challenge, especially in the context of largescale datasets.To deal with medium and large tasks, the MKC methods based on late fusion paradigm have been proposed. Such methods fuse the information of individual partitions to obtain the underlying shared kernel partition, significantly reducing the computational burden [19, 28] . For example, [21] employs orthogonal transformations to maximize the weighted base partition of individual kernels with a consensus partition. Building upon this foundation, [27] and [20] introduce local and global kernel maximization alignment respectively to delve into the structure embedded within the kernel. [7] proposes to combine the min-max scheme with the late fusion paradigm to simplify the objective function. Nevertheless, when obtaining the base kernel partition, these methods fail to simultaneously preserve energy and clustering information. In addition, in high-dimensional space, where the intrinsic manifold structure has the characteristics of bending and folding, they overlook the distance and topological information, resulting in suboptimal clustering performance.To tackle the aforementioned problems, we propose a novel MKC method with shifted Laplacian on Grassmann manifold (sLGm). The illustration of sLGm is plotted in Fig. 1 , we firstly construct the shifted Laplacian of individual kernel to simultaneously preserve energy-related and clustering-related information. Critically, to mitigate noise interference, the 𝑟 -rank base kernel partition matrix of 𝑟 -rank shifted Laplacian is constructed to derive the consensus underlying structure, and subsequently perform 𝑟 -rank shifted Laplacian reconstruction to obtain the consensus partition. Considering the topological information preservation between individual 𝑟 -rank partition and consensus partition, the Grassmann manifold is introduced to facilitate the acquisition of the optimal consensus partition. Finally, the consensus partition matrix is put into 𝑘-means for the assignments of clustering labels. Overall, the contributions of sLGm are as follows: Consensus underlying clustering structure: =The optimal consensus partition Clustering assignmentsFigure 1 : Illustration of sLGm. Firstly, each K 𝑝 can be treated as an affinity graph, and construct the 𝑟 -rank shifted Laplacian L 𝑝 𝑟 and correspondingly yields the 𝑟 -rank base kernel partition U 𝑝 , and then feed it into the two sub-frameworks. These two sub-frameworks are integrated to jointly learn an optimal F, thereby acquiring the clustering assignments.• sLGm fuses the individual 𝑟 -rank base kernel partitions of the 𝑟 -rank shifted Laplacian to perform reconstruction, which can minimize the noise interference while maximally preserving the clustering information and the reconstructed energy information. • sLGm proposes to utilize the squared projection distance on the Grassmann manifold, to fuse the individual 𝑟 -rank base kernel partitions and the consensus kernel partition, such that topological and heterogeneous information can be acquired simultaneously. • sLGm integrates 𝑟 -rank shifted Laplacian reconstruction and partition fusion on Grassmann manifold into a unified framework, where these two sub-frameworks jointly learn the optimal consensus partition for clustering, reducing the memory and computational complexity to O (𝑛) and O (𝑛 2 ) with the 𝑛 × 𝑛 kernel matrix. 
related work 2.1 spectral clustering with laplacian reconstruction: 
 The concept of spectral clustering (SC) originates from graph theory, which can effectively capture the intrinsic structure of data via spectral decomposition or eigenvectors of Laplacian matrix. Specifically, for a data matrix X ∈ R 𝑛×𝑑 with 𝑛 samples and 𝑑 features, its 𝑘-nearest affinity matrix is expressed by Z ∈ R 𝑛×𝑛 , whose edge between sample-pair can be defined by Gassuian kernel:EQUATION)where 𝜎 is the width parameter, which controls how fast the similarity decays. And the diagonal elements of the degree matrix D of Z ∈ R 𝑛×𝑛 are defined as D 𝑖𝑖 = 𝑛 𝑗=1 Z 𝑖 𝑗 , thereby the Laplacian matrix L = D -Z can be obtained. Furthermore, the corresponding normalized Laplacian matrix L M [16] can be defined asEQUATIONDenoting H ∈ R 𝑛×𝑐 as the indicator matrix with 𝑐 clusters, the objective function of the normalized SC [14] is expressed asEQUATIONThe optimal H is constructed by taking the 𝑐 smallest eigenvectors of L M , and the clustering assignments are obtained by discretizing H with 𝑘-means. To capture complementary information from multiple views, [9, 24] perform Laplacian reconstruction to linearly integrate the base Laplacians, such that obtaining the optimal Laplacian. Such conceptual framework is formalized as follows:minH ⊤ H=I 𝑐 ,𝜷 Tr(H ⊤ L 𝜷 M H) s.t. L 𝜷 M = 𝑣 ∑︁ 𝑖=1 𝛽 𝑖 𝑑 L 𝑖 M , ∥𝜷 ∥ 1 = 1, 𝜷 ≥ 0 (4)where L 𝑖 M is the 𝑖-th view Laplacian matrix, 𝛽 𝑖 is the weight of the 𝑖-th Laplacian.In MKC tasks, direct execution of complex operations on the 𝑛 × 𝑛 kernel matrix will lead to substantial computational and storage overhead, making it impractical for real-world applications when 𝑛 is sufficiently large. Therefore, maintaining a light computational burden is an urgent demand for MKC tasks. Accordingly, Wang et al. [21] propose the late fusion clustering paradigm, which can reduce the computational and memory burden straightway. Specifically, they initially employ kernel 𝑘-means to obtain clustering partitions H 𝑖 for base kernels, and then use the rotation matrix W 𝑖 to linearly combine H 𝑖 to maximize alignment with the consensus cluster partition H ★ . Concurrently, a regularization term is incorporated to ensure that consensus partition and the average partition E are constrained within a comparable range. This late fusion clustering paradigm can be expressed as follows: maxH ★⊤ H ★ =I,{W 𝑖 } 𝑚 𝑖=1 ,𝜼Tr(H ★⊤ C) + 𝜁 Tr(H ★⊤ E),EQUATIONwhere 𝜁 is a trade-off parameter.As specified in Eq. ( 3), the clustering information is encoded within the smallest eigenvectors of the Laplacian matrix L M . However, as illustrated in [3] , when conducting Laplacian reconstruction such as Eq. ( 4), their best low-rank approximation is reconstructed by partially largest eigenpairs. In other words, the eigenvectors associated with the largest Laplacian eigenvalues are sufficient for achieving Laplacian reconstruction, indicating that the energy-related information is predominantly retained in these largest eigenvectors. This uncovers a crucial contradiction between reconstructing the Laplacian matrix and preserving clustering information. To solve this contradiction, [6] shift the normalized Laplacian, which is specifically expressed as follows:EQUATIONwhere L is denoted as shifted Laplacian (sL for short), which has several vital properties [6] : its smallest eigenvalues correspond to the largest eigenvalues of L M , and both of which fall in the interval [0, 2]; it is a symmetric positive semidefinite matrix like L M .Leveraging these properties, it can be inferred that the optimal solution to Eq. ( 3) can be derived from 𝑐 largest eigenvectors of L. Therefore, L can be effectively employed to facilitate Laplacian reconstruction, effectually encode clustering information and energy-related information. 
proposed method 3.1 𝑟 -rank shifted laplacian reconstruction: 
 In MKC, considering the kernel {K 1 , K 2 , . . . , K 𝑚 } with 𝑐 distinct clusters from 𝑚 kernels. Notably, each K 𝑝 ∈ R 𝑛×𝑛 is treated as an affinity matrix, allowing for the exploration of abundant graphbased information embedded in the kernel matrix [18] . Peculiarly, we introduce 𝑘 as a regulator for the number of neighbors associated with each vertex, aiming at eliminating redundant edges. Therefore, the sL L 𝑝 of kernel K 𝑝 can be computed using Eq. ( 6).As illustrated in Sec. 2.3, it is clear that not all eigenvalues make significant contributions to the process of Laplacian reconstruction. Simultaneously, the spectral graph theorem reveals that the key clustering information is predominantly contained within the 𝑟 rank largest eigenvalues of L 𝑝 . In contrast, the remaining smallest eigenvalues carry a disproportionately higher amount of noiserelated information rather than relevant clustering data, resulting in a low signal-to-noise ratio (SNR). To tackle this issue, L 𝑝 is eigendecomposed into 𝑟 -rank part and cluster-irrelevant part:EQUATION)where 𝚲 is the diagonal matrix formed by the eigenvalues of L 𝑝 , i.e., Λ = diag(𝜆 EQUATIONThe optimal solution U 𝑝 can be acquired by extracting the 𝑐 largest eigenvectors of L 𝑝 𝑟 . Inspired by [21] , we introduce the rotation matrix W 𝑝 ∈ R 𝑐 ×𝑐 , which can align the base kernel partition U 𝑝 with the consensus underlying clustering structure U. Usually, it can be described as follows:U = 𝑚 ∑︁ 𝑝=1 𝛼 𝑝 U 𝑝 W 𝑝 s.t. 𝛼 𝑝 ≥ 0, 𝜶 ⊤ 𝜶 = 1, W 𝑝 ⊤ W 𝑝 = I 𝑐 (9)Since Eq. ( 8) can be approximately transformed into min∥L 𝑝 𝑟 - U 𝑝 U 𝑝 ⊤ ∥ 2 F , it is further deduced that L 𝑝 𝑟 = U 𝑝 U 𝑝 ⊤ .Therefore, the combined L 𝑟 can be reconstructed using UU ⊤ , and the subframework of 𝑟 -rank shifted Laplacian reconstruction can be obtained via the SC framework regarding L 𝑟 : maxF ⊤ F=I 𝑐 ,(W 𝑝 ) ⊤ W 𝑝 =I 𝑐 Tr(F ⊤ UU ⊤ F) s.t. U = 𝑚 ∑︁ 𝑝=1 𝛼 𝑝 U 𝑝 W 𝑝 ,𝛼 𝑝 ≥ 0, 𝜶 ⊤ 𝜶 = 1 (10)where F represents the consensus partition and its optimal solution can be derived through the eigen-decomposition of L 𝑟 (i.e., UU ⊤ ). Noteworthily, the above process can be viewed as a late fusion paradigm, but the difference is that we use 𝑟 -sL to form U 𝑝 and U. As 𝑟 -sL is capable of preserving both clustering and energy information while maintaining a high SNR, reconstructing it yields a notably strong performance.The relationship between the base kernel partition U 𝑝 and the consensus kernel partition F can be established using Eq. (10) . However, relying solely on Eq. ( 10) is insufficient, as it fails to account for the information of distance and heterogeneity in the high-dimensional space. Specifically, the intrinsic manifold structure in such space exhibits complexity with featuring folding and distortion characteristics, while Eq. ( 10) cannot adequately preserve the topological structure and the spatial information across multiple kernels. To solve these issues, we introduce the squared projection distance on the Grassmann manifold, as specified in Definition 1 [2] . Definition 1. A Grassmann manifold S(𝑥, 𝑛) is defined as a 𝑥-dimensional subspace in an 𝑛-dimensional space ℜ 𝑛 , where each specific subspace can be mapped to a specific point on the manifold S. Mathematically, each point on a manifold S(𝑥, 𝑛) is broadened to be represented by an orthogonal matrix G, whose columns can be extended to a 𝑥-dimensional subspace of the corresponding 𝑛-dimensional space, denoted 𝑠𝑝𝑎𝑛(G). Suppose two subspaces as 𝑠𝑝𝑎𝑛(G 1 ), 𝑠𝑝𝑎𝑛(G 2 ), and their angle is defined as 𝜃 𝑖 , which can represent the geometric proximity of these subspaces, then the squared projection distance between G 1 and G 2 can be defined asEQUATIONAs stated in Definition 1, it is evident that the projection distance on Grassmann manifold can effectively reduce the subspace discrepancy. Accordingly, we bring F and U 𝑝 into Eq. ( 11) to minimize the kernel dissimilarity and facilitate information fusion:EQUATIONTo integrate kernel information from U 𝑝 and F, while assigning an appropriate weight 𝛾 𝑝 to different U 𝑝 for acquiring the optimal F, Eq. ( 12) can be rewritten as maxEQUATIONIn this way, Eq. ( 13) not only considers the heterogeneous information originating from multiple kernels, but also retains the spatial information and topological information, thereby accomplishing partition fusion. Consequently, by combining two sub-frameworks of 𝑟 -rank shifted Laplacian reconstruction and partition fusion on Grassmann manifold, we derive the following objective function:max 𝜸,𝜶 ,F,W 𝑝 Tr(F ⊤ UU ⊤ F) + 𝜆 𝑚 ∑︁ 𝑝=1 𝛾 𝑝 Tr(FF ⊤ U 𝑝 U 𝑝⊤ ) s.t. U = 𝑚 ∑︁ 𝑝=1 𝛼 𝑝 U 𝑝 W 𝑝 , 𝛾 𝑝 , 𝛼 𝑝 ≥ 0, 𝜸 ⊤ 𝜸 = 1, 𝜶 ⊤ 𝜶 = 1, F ⊤ F = I 𝑐 , (W 𝑝 ) ⊤ W 𝑝 = I 𝑐 (14)where 𝜆 is a balance parameter. As depicted in Eq. ( 14), it primarily encompasses two sub-frameworks, these two complementary subframeworks mutually reinforce one another and collaboratively learn an optimal consensus partition F, consequently yielding improved clustering performance. 
optimization 4.1 the optimal solution: 
 ⊲ For 𝜸 and 𝜶 : When fixing other variables, Eq. ( 14) w.r.t. 𝜸 is expressed asEQUATION)where X 𝑝 = 𝜆Tr(FF ⊤ U 𝑝 U 𝑝⊤ ), and the solution of 𝜸 is𝛾 𝑝 = X 𝑝 √︃ 𝑚 𝑝=1 X 2 𝑝 (16)Likewise, when fixing other variables, the solution of 𝜶 isEQUATION)whereJ 𝑝 = Tr(F ⊤ U 𝑝 W 𝑝 ).⊲ For F: Fix 𝜸 , 𝜶 and W 𝑝 , Eq. ( 14) w.r.t. F can be expressed as maxEQUATIONwhich can be further rewritten as maxEQUATIONwhere H = 𝑚 𝑝=1 𝛼 𝑝 U 𝑝 W 𝑝 ( 𝑚 𝑝=1 𝛼 𝑝 U 𝑝 W 𝑝 ) ⊤ + 𝜆 𝑚 𝑝=1 𝛾 𝑝 U 𝑝 U 𝑝⊤ , and its optimal solution can be acquired via the largest 𝑐 eigenvectors of H.⊲ For W 𝑝 : When fixing other variables, the optimization w.r.t. W 𝑝 can be expressed via maxEQUATIONwhich can be simplified as maxEQUATIONwhere R 𝑝 = 𝛼 2 𝑝 (U 𝑝 ) ⊤ FF ⊤ U 𝑝 , and the optimization w.r.t. W 𝑝 can be obtained via the largest 𝑐 eigenvectors of R 𝑝 .As laid out in Algorithm 1, the complexity of sLGm consists of four subproblems, i.e., 𝜸 , 𝜶 , F and W 𝑝 . For updating 𝜸 and 𝜶 , their computational complexity are O (4𝑐 2 𝑚𝑛); For updating F and W 𝑝 , their computational complexity are O (𝑐𝑛 2 ) and O (𝑐 3 𝑚), respectively. Since 𝑐, 𝑚 ≪ 𝑛, its computational complexity is approximated as O (𝑛 2 ). Accordingly, its memory complexity is O (𝑛).In Algorithm 1, four subproblems need to be solved, all of which have optimal solutions, and it is evident that the objective of Algorithm 1 increases monotonically while optimizing one variable, and the remaining variables remain unchanged. In addition, according The final clustering assignments; for 𝑝 = 1 to 𝑚 do Construct 𝑘-nearest graph of K 𝑝 ; Calculate its degree matrix D 𝑝 and the normalized sL L 𝑝 using Eq. ( 6); Eigen-decompose L 𝑝 using Eq. ( 7) to acquire the 𝑟 -sL L 𝑝 𝑟 ; Calculate the base kernel partition U 𝑝 using Eq. ( 8); end for while not converge do Update 𝜸 and 𝜶 using Eq. ( 16) and Eq. ( 17), respectively; Update consensus partition F using Eq. ( 19); Update rotation matrix W 𝑝 using Eq. ( 21); end while Discretize F by 𝑘-means to acquire 𝑐 clusters.to Lemma 1 [27] , the upper bound of the proposed algorithm can be obtained.Lemma 1. For ∀𝑖, 𝑗, Tr[(𝛽 𝑖 A 𝑖 B 𝑖 ) ⊤ (𝛽 𝑗 A 𝑗 B 𝑗 )] ≤ Tr[(A 𝑖 B 𝑖 ) ⊤ (A 𝑗 B 𝑗 )] ≤ 1 2 (Tr[(A 𝑖 B 𝑖 ) ⊤ (A 𝑖 B 𝑖 )]+Tr[(A 𝑗 B 𝑗 ) ⊤ (A 𝑗 B 𝑗 )]), if A ⊤ 𝑖 A 𝑖 = I 𝑐 and B ⊤ 𝑖 B 𝑖 = I 𝑐 , we have Tr[(A 𝑖 B 𝑖 ) ⊤ (A 𝑖 B 𝑖 )] = Tr(B ⊤ 𝑖 A ⊤ 𝑖 A 𝑖 B 𝑖 ) = Tr(I 𝑐 ) = 𝑐 = Tr[ A 𝑗 B 𝑗 ⊤ (A 𝑗 B 𝑗 )].According to Lemma 1, the first term of Eq. ( 14) can be transformed into Tr(F ⊤ UU ⊤ F) ≤ 1 2 (Tr(FF ⊤ FF ⊤ ) + Tr(UU ⊤ UU ⊤ )) = (𝛼 𝑗 U 𝑗 W 𝑗 )]) ≤ 𝑐 2 (𝑚 4 + 1). In the same way, the second term is less than or equal to 𝜆𝑐 2 (𝑚 2 + 1). As a whole, the proposed algorithm is less than or equal to 𝑐 2 ((𝑚 4 + 1) + 𝜆(𝑚 4 + 1)). Therefore, Algorithm 1 has an upper bound and monotonically increasing, which can obtain a local maximum solution with convergence. 
experiment 5.1 datasets: 
 The experiments use eight datasets, encompassing various types. Specifically, they are BBCSports, Flower17, Handwritten, Caltech101, UCIdigits, Mfeat, YALE and SensVehicle. Preemptively, all kernel matrices associated with these datasets have been pre-computed and are available for download from a publicly accessible website. A brief description of all datasets is listed in Table . 1.sLGm is compared with ten state-of-the-art methods, to summarize, they can be roughly divided into the following categories: SKC method, including B-SKKM; MVC methods, including LMVSC [5] and OMSC [1] ; and MKC methods, including AMKKM, MR-MKKM [12] , LFA [21] , ONMSC-LF [9] , SMKKM [11] , LF-PGR [19] and LF-LKA [27] . For above all methods, their codes are downloaded from the provided website, and all parameters are adjusted according to their description. In particular, for all MVC methods, the kernel B-SKKM O (𝑚𝑛 2 ) AMKKM O (𝑚𝑛 3 ) MR-MKKM O (𝑚𝑛 2 ) LFA O (𝑛𝑐 3 + 𝑚𝑐 3 ) LMVSC O (𝑛𝑚 3 𝑣 + 𝑚 3 𝑣 3 + 𝑛𝑐 2 + 2𝑚𝑣𝑛 + 𝑚𝑛𝑙 ) OMSC O (𝑛 (𝑙 + 𝑐 ) ) ONMSC-LF O (𝑣𝑛 3 + 𝑛𝑐 2 ) SMKKM O (𝑛 3 + 𝑚𝑛 2 ) LF-PGR O (𝑚𝑐𝑛 2 + 𝑐𝑛 2 ) LF-LKA O (𝑛𝑐 2 + 𝑚𝑐 3 ) sLGm O (4𝑚𝑛𝑐 2 + 𝑐 3 𝑚 + 𝑐𝑛 2 )matrix is treated as ordinary data for the algorithm input. In addition, to mitigate the impact of the randomness in 𝑘-means, we record the average results from 30 independent trials. To ensure a fair comparison, all methods are tested on the same device, and ACC, NMI and Purity are utilized as evaluation metrics.In addition, for our method, there are three parameters to be adjusted, i.e., 𝜆, 𝑟 and 𝑘. For 𝜆, set its value to [1e-5, 1e-4, ..., 100] to adjust dynamically. For 𝑟 and 𝑘, for convenience, two additional parameters 𝑙𝑟𝑎𝑛𝑘 and 𝑘𝑏𝑢𝑟 are introduced to adjust them, where 𝑙𝑟𝑎𝑛𝑘 = 𝑟 ÷ 𝑐 and 𝑘𝑏𝑢𝑟 = 𝑘 × 𝑐 ÷ 𝑛, with ranges of [1, 2, ..., 5] and [0.05, 0.1, ..., 1.5], respectively.Table 3 presents the results of all methods across the eight datasets, from which the following observations can be acquired:• Generally, sLGm consistently outperforms other methods on majority of datasets, notably surpassing the latest HF-MKKM, LF-PGR and LF-LKA. These findings suggest that sLGm is a viable method for dealing with nonlinear data. • Compared with SKC, MKC methods are expected to yield superior clustering results. However, AMKKM, MR-MKKM and HF-MKKM methods exhibit suboptimal performance compared to B-SKKM on certain datasets. This indicates that the MKC methods need to choose a judicious and comprehensive learning strategy to exploit the information from base kernels. Notably, sLGm choose a preferable strategy to adeptly integrate information from base kernels, thereby attaining commendable clustering results. • Compared with the MVC methods LMVSC and OMSC, sLGm achieves the best clustering performance, which is because they treat the kernel matrix as plaint data, can not fully explore the graph information, and have limitations for nonlinear data processing. • Compared with several MKC methods using late fusion, i.e., LFA, ONMSC-LF, LF-PGR and LF-LKA, sLGm demonstrates superior performance. This can be attributed to two aspects, one is 𝑟 -rank base kernel partition, enabling the preservation of both clustering and reconstruction information while mitigating the impact of noise. Another is the Grassmann manifold, which enhances the ability to capture topological information. These two aspects contribute to the optimal consensus partition, which lead to an overall improvement in clustering performance.Running time: The complexity of all comparisons are listed in Table 2 , where 𝑚, 𝑐, 𝑣, 𝑙, 𝑛 represent the number of features, clusters, views, anchors and samples, respectively. Fig. 2 plots the comparison of time consumption on all algorithms. For convenience, the logarithm of time is taken as the ordinate, where the larger the value is, the more time will be consumed. Based on this, it is evident that our algorithm exhibits relatively reduced time consumption compared to others, particularly in contrast to LF-PGR, which shares a comparable complexity. This discrepancy arises due to the fact that our algorithm encompasses less subproblems and their solution is simple. Furthermore, sLGm strategically performs sL with 𝑟 -rank rather than full rank, resulting in a concomitant reduction in time consumption. This finding can provide preliminary evidence that sLGm is promising to handle large-scale data.Parameters sensitivity: As previously stated, the algorithm involves three parameters: i.e., 𝜆, 𝑙𝑟𝑎𝑛𝑘 and 𝑘𝑏𝑢𝑟 . To evaluate the sensitivity of these parameters, one variable is held constant over an extensive range while the others are systematically adjusted, as depicted in Fig. 3 . These visualizations reveal that the clustering performance of all parameters is stable within a broad range, reflecting the insensitivity of sLGm to parameter variations. As the aforementioned experimental results presented, sLGm performs superior because the constructed sL can preserve clusterrelated information, while the 𝑟 -sL can remove noise-related information simultaneously. To verify the validity of sL and 𝑟 -sL, we plot comparisons in terms of ACC on BBCSports and YALE, which is shown in Fig. 4 . Note here that, to verify 𝑟 -sL validity, the value of 𝑟 is adjusted, where 𝑟 = 𝑛 means that 𝑟 -sL is disabled. In addition, to visually show the clustering distribution results, the t-SNE visualization on the BBCSports dataset is plotted on Fig. 5 . From Fig. 4 and Fig. 5 , we can observe that: (1) ACC of using 𝑟 -sL is significantly better than that of not using it; (2) ACC is relatively stable in a large range; (3) sLGm with 𝑟 -sL has distinct clusters. The reason is that 𝑟 -sL consists of largest eigenvalues that contain more cluster-related information, while small eigenvalues may contain more noise information hidden in the kernel matrix.Ablation evaluation: To further verify the effectiveness of sLGm, ablation experiments are carried out on BBCSports, Flower17, YALE and SensVehicle by setting 𝜆 = 0 of Eq. ( 14). The corresponding results are listed in Table 4 . The results distinctly indicate that when 𝜆 = 0, signifying the exclusion of partition fusion on Grassmann manifold, its clustering performance experiences an obvious reduction. This observation underscores the effectiveness of the utilization of the squared projection distance on the Grassmann manifold, as it effectively explores both topological and heterogeneous information in the high-dimensional complex space. Convergence evaluation: The convergence of the proposed algorithm has been analyzed in Sec. 4.2, and in order to verify the convergence experimentally, we draw a curve graph of the objective function value and clustering performance across iterations on UCIdigits and SensVehicle, as depicted in Fig. 6 . It is observed that the objective function can converge to the stable value within five iterations. In addition, as the objective function converges, the corresponding clustering performance also stabilizes, indicating the effectiveness of the learnt consensus partition F. 
conclusion: 
 In this paper, we propose a novel multiple kernel clustering method using shifted Laplacian on Grassmann manifold, i.e., sLGm. Specifically, we take Laplacian as the breakthrough, construct 𝑟 -rank shifted Laplacian and the corresponding 𝑟 -rank base kernel partition to achieve reconstruction, which can retain the clustering information and energy information while remove the noise interference. In addition, the squared projection distance is used on the Grassmann manifold to further explore the topology structure. These two sub-frameworks jointly learn an optimal consensus partition to obtain the final clustering assignments. Compared with ten state-of-the-art clustering methods, and carried out a series of experiments on the benchmark datasets to comprehensively prove the effectiveness of sLGm. 
 
 Appendix:  


Decision: The paper received three reviews, by reviewers who declare to be confident, in one case very confident. They all tend towards acceptance, although not enthusiastically  since their final (after rebuttal) recommendations are weak accept, weak accept, borderline accept.
The main limitations seem to be addressable in the camera ready, revised version of the paper, and after the rebuttal phase the reviewers comment that the paper appears to them acceptable.
I recommend accepting the paper for poster presentation.

Reviews: review: paper_summary: paper_summary: The author proposes a novel multi-kernel clustering method based on the shifted Laplacian matrix on the Grassmann manifold. This model constructs 𝑟-rank Shifted Laplacian, preserving both clustering-related and energy-related information while reducing the impact of noise. Additionally, the model introduces a Grassmann manifold for partition fusion, which retains topological information in high-dimensional spaces. Extensive experimentation on real-world datasets validates the feasibility and effectiveness of the model.
strength_weakness: strength_weakness: 1. The concept of the 𝑟-rank Shifted Laplacian is introduced innovatively, which not only preserves clustering-related and energy-related information but also reduces the impact of noise.
2. The experiments are quite extensive and sufficient, which effectively validate the effectiveness of the model.
limitations: limitations: 1. The first and second terms of the model objective are very similar after expansion, could the differences be further explained theoretically?
2. The language quality of this paper still need improvement and enhancement. In Section 5.2, the second paragraph's description is somewhat colloquial and appears a bit disorganized.
3. How is the rotation matrix W initialized, and does its initialization affect the optimal significance of the model?
paper_summary: paper_summary: In this paper, the authors propose a novel multiple kernel clustering framework, which can retain the clustering information, energy information, and topology information while removing the noise interference. In general, it compared with ten comparative methods on eight datasets, the proposed method achieves good results.
strength_weakness: strength_weakness: Overall, the methodology and the framework of this paper is reasonable. Specifically, this paper has the following advantages:
(1) This paper has a corresponding derivation process for the proposed method, and lists the references of the phase management to prove the whole derivation process, which has good readability.
(2) This method proposed a 𝑟-rank shift Laplacian reconstruction technique, which can retain both cluster and energy information, which is not achieved by other methods. Also, Laplacian is used as the breakthrough to construct the shifted Laplacian operator, and manifold learning is introduced, these two can guide the learning of the optimal indicator matrix, thus improving the classification performance. 
(3) In the EXPERIMENT section, the authors have carried out many experiments, and compared it with eight methods in related fields. Specifically, the authors show the effectiveness of sLGm from data analysis, chart analysis, figure analysis, and other aspects.
limitations: limitations: (1) The authors mention late fusion clustering in Sec 2.2, but do not describe much about late fusion when introducing the proposed method. 
(2) Are the best parameters  ,   and   the same across different datasets? If not, some guidance on choosing these three parameters should be provided, which can provide instructions to the reader.
(3) Some sentences of this paper should be refined. For example, the sentence in right column of line 95 in page 1 is too long for the readers to understand.
(4) Some notations are not very clear. For example, in Sec 2.1, the notation $L_M^{\beta}$ should be interpreted concretely.
(5)Can you give a detailed explanation on how to reduce the computational complexity to $o(n^{2})$, whether the late fusion clustering  in Sec 2.2 is used, and how to reduce it? The necessary derivation and reasons need to be given.
paper_summary: paper_summary: The paper proposes a novel Multiple Kernel Clustering (MKC) method called shifted Laplacian on Grassmann manifold (sLGm). This method aims to address three primary issues in existing MKC techniques: ignoring energy information and noise within the kernel, insufficient exploration of the high-dimensional manifold structure, and high computational complexity. The sLGm method constructs a rank-shifted Laplacian to retain clustering and energy-related information while reducing noise. It also introduces a Grassmann manifold to preserve topological information and facilitates partition fusion. The method promises reduced computational complexity and improved clustering performance. The authors validate their approach through extensive experiments on multiple datasets.

While the theoretical formulation and experimental results are solid, further detailed analysis and broader comparisons could provide a more comprehensive validation of the proposed method's advantages.

I'm inclined to accept it but may change my mind due to other reviewers' comments
strength_weakness: strength_weakness: The integration of shifted Laplacian and Grassmann manifolds for clustering is a novel approach. By addressing both clustering and energy information simultaneously while reducing noise, the paper introduces a significant innovation in the field of MKC.
  
The paper presents a clear mathematical derivation for the proposed method. The use of the Grassmann manifold for partition fusion is well-justified theoretically, adding robustness to the clustering process.

The experimental validation is thorough, involving multiple datasets. The results show that the proposed method outperforms several state-of-the-art techniques, demonstrating its effectiveness in various scenarios.
limitations: limitations: While the paper introduces a novel combination of techniques, the core ideas of using shifted Laplacian and Grassmann manifolds are not entirely new. Previous works have explored these concepts separately, eg. (Grassmann) manifold learning [1,2] shifted laplacian [3], and Late fusion[4]. The novelty primarily lies in their integration, which may be seen as incremental by some readers. 

Although the experiments are extensive, the analysis could be more detailed. Specifically, it would be beneficial to include ablation studies to isolate the impact of each component (shifted Laplacian, Grassmann manifold) on the overall performance. This would provide clearer insights into the contributions of each part of the proposed method. The paper claims reduced computational complexity, but empirical evidence on large-scale datasets would better support this claim. Demonstrating scalability through experiments on very large datasets would enhance the argument for practical applicability.

[1] An attention-based framework for multi-view clustering on Grassmann manifold

[2] Multi-view Subspace Clustering on Topological Manifold

[3] Approximate shifted Laplacian reconstruction for multiple kernel clustering

[4] Late fusion multiple kernel clustering with proxy graph refinement