# The Core Cluster-Based Subspace Weighted Clustering Ensemble

Authors: Xuan Huang; Fang Qin; Lin Lin
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

In recent years, the Internet of Things (IoT) technology has developed rapidly and is widely used in various fields. It is of great research significance to uncover underlying patterns and insights from the high-dimensional data of IoT, to excavate valuable information to guide people’s production and life. Clustering can explore the natural cluster structure of the data, which is conducive to further understanding of the data, and is an essential preprocessing step for data analysis. However, clustering is highly dependent on the data. In order to reduce the complexity of the model, reduce the computational cost, and obtain a more robust clustering solution, we combine subspace clustering and ensemble learning to propose a novel subspace weighted clustering ensemble framework for high-dimensional data. The proposed framework first combines random feature selection and unsupervised feature selection to generate a set of base subspaces. Clustering is performed on each base subspace to achieve a set of subspace clustering solutions that generate a set of adaptive core clusters. The size of the core cluster is between the sample and the cluster. In the ensemble process, the core clusters are viewed as the basic unit, and the stability of the cluster is evaluated by measuring the distance between the core cluster pairs, and the similarity between the core clusters and the clusters in the base subspace, and then weighting the subspace clustering solution. Under this framework, we propose four subspace ensemble approaches based on core cluster to improve the accuracy of consensus clustering solutions. Extensive experiments are conducted on multiple real-world high-dimensional datasets, demonstrating that the proposed framework can process high-dimensional data for the IoT, and the proposed subspace clustering ensemble approaches are superior to the state-of-the-art clustering approaches.


---

## Body

## 1. Introduction

The Internet of Things is a network platform that combines various information collection devices and the Internet, which facilitates the sharing of information between various devices. Data acquisition devices are widely used in various application fields, and their collected data has the characteristics of high dimension. The impact of high-dimensional data on data mining is twofold. On the one hand, as the data dimension grows, there are more features that describe the sample from different perspectives, which brings richer and more comprehensive information to data analysis; on the other hand, high-dimensional data increases the complexity of the model and make data analysis difficult.Clustering is an unsupervised learning technique that partitions unlabeled samples into clusters according to the specific criteria. Clustering can reveal the intrinsic properties of the study object and discover its underlying patterns. In the field of data mining and pattern recognition, clustering is often used as data preprocessing, which is the basis for subsequent data analysis. Clustering has always been an active research direction, and in different fields, there are existing multiple clustering algorithms and they achieve satisfactory results. But with the popularity of high-dimensional data, they pose a huge challenge to clustering.Existing clustering approaches for high-dimensional data can initially be roughly divided into two categories: dimensionality reduction, and then clustering and subspace clustering. The former typically uses feature selection or feature extraction techniques to extract features from a high-dimensional space that are relevant to subsequent clustering criteria, thereby reducing the dimensionality of the sample. Then, we performed the clustering in the lower-dimensional space. Subspace clustering assumes that the clusters of high-dimensional data are located in low-dimensional subspaces [1], and the goal is to find the clusters hidden in large dimensions.Clustering is highly dependent on the data. As the dimensions increase, there are a large number of redundant and irrelevant features in the data. This makes the model more complex and the computational complexity grows exponentially. Different clustering algorithms are designed for different types of data, and they can discover the underlying structure of a particular data set, not valid for all types of data. Even with the same clustering algorithm, setting different parameters can group samples into different clusters. As a result, more researchers are focusing on clustering ensemble, trying to achieve better and more robust clustering results by merging multiple base clustering solutions. Recently, spectral clustering [2] has become one of the popular clustering approaches. It first calculates the similarity of the samples to construct an affinity matrix and then decomposes the Laplace vector of the affinity matrix to obtain the eigenvector associated with the feature values. The samples are then mapped to the lower-dimensional space for a final clustering solution.Spectral clustering [3] can find clusters for complex samples, which is simple to implement and can achieve better results compared to traditional clustering approaches. Recently, some researchers have proposed clustering high-dimensional data through subspace clustering ensemble approaches [4–7]. Some of these employ spectral clustering approach [4, 5]. Cai et al. proposed the spectral clustering approach based on random subspace and graph fusion, termed SC-SRGF [5]. It combines the affinity graph of each subspace with the iterative similarity network fusion scheme and performs spectral clustering on the fusion matrix to obtain the final clustering solution. Huang et al. proposed multidiversified ensemble clustering (MDEC) [4], and under the proposed framework, consistent clustering solutions are obtained by performing spectral clustering, bipartite graph, and hierarchical clustering-based consensus function. These approaches [4–7] use random feature selection to generate a set of subspaces, that is, a certain proportion of features are randomly extracted from the original feature set to generate feature subset. The clusters of each subspace are ensemble to achieve the final clustering solution. However, these approaches [4–6] do not take into account the contributions of different feature subspaces during integration. Literature [7] considers the contribution of each subspace clustering solution in the ensemble process, but it selects members with larger contributions to participate in ensemble. This ensemble selection strategy ignores the feature subspace of small contributions, which is easy to bias the clustering results.To the abovementioned problem, this paper proposes a novel adaptive core cluster-based subspace weighted clustering ensemble approach, termed C2SWCE, which considers the similarity of clusters in subspaces in the ensemble process, and the weighted subspace clustering solution helps to improve the accuracy of consensus clustering result. The overall process of C2SWCE is shown in Figure 1. The proposed framework first uses a combination of random feature selection and unsupervised feature selection to generate a set of base subspaces. Then, spectral clustering is performed in the base subspace to achieve the clustering solution for each base subspace, further generating a set of adaptive core clusters. The core cluster is viewed as the basic unit of clustering ensemble, and the clustering solution of the subspace is weighted by calculating the distance between the core clusters in the subspace, the similarity of the core cluster and the cluster. Finally, we propose four consensus functions under this framework, which combine the locally weighted subspace clustering solutions to achieve the consensus clustering solutions.Figure 1
Flow diagram of the C2SWCE.The main contributions of our approaches are summarized as follows:(1)
We combine random feature selection and unsupervised feature selection to generate a set of base subspaces. Random feature selection generates a set of random subspaces, and then, unsupervised feature selection is performed on random subspaces, and the features that retain the local structure of random subspaces are selected, which further reduces the dimensionality of random subspaces. This joint feature selection strategy ensures the diversity of feature subspaces and improves the computational efficiency.(2)
We introduce the concept of the core cluster, which is a collection of samples that are grouped into the same cluster in all subspaces. In this paper, the core cluster is viewed as the basic unit of the clustering ensemble, which improves the computational efficiency of the integration process(3)
We propose locally weighted subspace clustering ensemble framework, which evaluates the stability of clusters by calculating the distance of core clusters in each subspace, and the similarity of core clusters and clusters. We further propose four weighted ensemble approaches based on core clusters, which fuse clusters in base subspace and obtain the consensus clustering solution(4)
The experimental results on several real high-dimensional datasets show that the comprehensive performance of our approaches in clustering accuracy and time complexity is significantly better than that of the state of the art clustering approachesThe rest of this paper is organized as follows: We review the related work in Section2, the locally weighted subspace clustering ensemble approaches are proposed in Section 3, experimental settings and related results and analysis in Section 4, and finally, in Section 5, we conclude this paper.


## 2. Related Work

Clustering ensemble, also known as cluster integration, combines multiple base clustering solutions to obtain better and more robust consensus clustering result. The cluster integration process consists of two stages: the base clustering generation and the ensemble of clusters. In the first stage, different clustering algorithms are performed on the dataset, or different parameters are set by the same clustering algorithm to partition samples, and the goal of this stage is to generate diverse clustering solutions. In the second stage, the input base clustering solutions are fused by the consensus function to obtain the final clustering result.The subspace clustering technique can explore the nature cluster structure of high-dimensional data in different low-dimensional spaces. Subspace clustering ensemble is a method that combines subspace clustering and ensemble learning, which fuses clusters in different feature subspaces to achieve consensus clustering solutions. Among the recently proposed subspace clustering ensemble approaches, Huang et al. proposed a multivariate subspace clustering ensemble framework, termed MDEC [4], in which random sampling was used to generate a set of feature subspaces. The novelty of this approach is that the randomizing a scaled exponential similarity kernel is used to get a large number of multivariate measures for each random subspace, forming a metric-subspace pairs. Perform spectral clustering on the similar matrix in each metric-subspace pair to obtain a clustering solution for each subspace, use the entropy criterion to weight the clusters of the subspace, and then use spectral clustering, bipartite graph and hierarchical clustering approaches to get a consensus clustering solution.Cai et al. proposed a novel spectral clustering approach based on subspace, termed SC-SRGF [5], which first generates a set of random feature subspaces, uses the local structures information of each subspace to form the KNN affinity graph, and then use an iterative similarity network fusion scheme to fuse the affinity graphs of each subspace to obtain a unified affinity graph, and obtain the final clustering solution by performing spectral clustering on a unified affinity graph.Shankar proposed subspace clustering integration framework AP2CE [6], which uses the affinity propagation to produce a representative subset of features and employs multiple distance function metric objects to produce diverse subspace clustering solutions and use the Ncut to partition the consensus matrix to get the final result.Verma et al. proposed double weighting semisupervised ensemble clustering based on selected constraint projection, termed DCECP [7], which treats prior knowledge of experts as pairwise constraints and assigned different subsets of pairwise constraints to different integration members. In addition, an adaptive integration member weighting process is designed to associate different weight values with different integration members. Finally, the final clustering result is obtained using the weighted normalized cut algorithm.Although many successful subspace ensemble clustering approaches have been developed, most existing approaches [4–6] treat each subspace clustering solution equally during the ensemble process. How to weigh the clustering solution according to the contribution of different feature spaces is worth considering in subspace ensemble.


## 3. Proposed Framework

In this section, we describe the overall process of the locally weighted subspace clustering ensemble approach. First, we give a brief overview of the proposed methods and then detail the proposed algorithm from three aspects: subspace generation, subspace clustering, and fusion of clusters in subspace.


### 3.1. Overview

In this paper, we introduce a locally subspace weighted clustering ensemble framework. First, we randomly selectk feature subset from the original feature set to generate k random subspaces, and then selecting features representing the local structure from each random subspace to generate base subspace. Then, spectral clustering is performed on each base subspace to achieve subspace clustering solutions. The adaptive core clusters are generated from the base subspace clustering solution, which is a set of samples that are grouped into the same cluster in all subspaces. Then, by calculating the distance of the core cluster in different subspaces, the stability of the clusters is evaluated and weighted the base subspace clustering solution. Finally, in order to integrate the clustering solutions of the base subspaces and get the consensus results, in the C2SWCE framework, we propose four adaptive core cluster-based consensus functions, to achieve the final clustering solutions.LetX∈Rn×m be the input dataset that contains n samples, each with m features. Let xi (i=1,⋯n) denotes the ith sample, which corresponds to the ith row of X, so that the input data can be represented as X=x1,x2,⋯,xn. Let fi (j=1,⋯m) denotes the ith feature of the sample, which corresponds to the jth column of the X; therefore, matrix X can also be represented as X=f1,f2,⋯,fm.


### 3.2. Generate Subspaces

The natural cluster structure of high-dimensional data is hidden in low-dimensional subspaces [8]. Subspace clustering explores the possibility of grouping samples in different feature sets. Subspace clustering ensemble fuses clusters in different subspaces to obtain the final clustering solution, which is an effective method for clustering high-dimensional data. The recently proposed subspace clustering ensemble approaches [5, 9], which uses random feature selection to generate a set of random subspaces. Random subspace has diversity; it explored the potential cluster of high-dimensional dataset from the perspective of different features to achieve diverse subspace clustering solutions. Literature [10] uses stratified sampling method to generate feature groups and verifies that this method is superior to random sampling. This paper uses the random feature selection to obtain a variety of subspaces, and unsupervised feature selection is performed again on the random subspace to select the features that retain the local structure of random subspace.The specific procedure of feature subspace segmentation is as follows. The original feature setf1,f2⋯,fm is randomly sampled according to the sampling ratio r, the original features are classified into k feature groups G1,G2,⋯,Gk, where G1i=1,⋯k represents the features contained in the ith random subspace. Let Xi~∈Rn×Gi denotes the matrix corresponding to the ith random subspace, n is the number of samples, and Gii=1,⋯,k is the number of features in the ith random subspace. Obviously, it holds that ∀Gi≠ϕi=1,⋯,k and Gi=m⋅r.A set of base subspaces is generated by calculating the Laplace score [11] of features in each random subspace and selecting the important features in random subspace. Then, we construct the KNN graph for each random subspace, which represents the local structure of the random subspace. The KNN graph of the ith subspace is defined as follows:
(1)GiKNN=Vi,Ei,where Vi=v1,v2,⋯,vn is the set of nodes corresponding to the samples x1,x2,⋯,xn in the ith subspace and Ei is the edge set of the ith subspace. We use the Gaussian kernel function to calculate the weights of the edges between the nodes in the subspace and their corresponding KNN nodes. Ei is defined as
(2)Ei=eαβin×n,where
(3)eαβi=exp−xαi−xβi22σ2,ifxαi∈KNNxβiorxβi∈KNNxαi,0,otherwise.In Equation (3), xαi and xβi are the two samples in the ith subspace, respectively. xαi−xβi2 is the Euclidean distance between xαi and xβi, and KNN⋅ is the K-nearest neighbor (KNN) operator, and σ is the mean of Euclidean distance between the sample and its KNN.LetDi=diagEi1 be the degree matrix of the ith subspace, where 1=1,⋯,1T. The Laplacian of the graph is calculated as follows:
(4)Li=Di−Ei.In theith subspace, the Laplace score of fli (l=1,⋯Gi) is
(5)Lfli=f~li⊤Lif~lif~li⊤Dif~li,where(6)f~li=fli−fli⊤Di11⊤Di11.We calculate the Laplace score for each feature in the subspace and arrange them in descending order to select the topd features. We determine the number of second selected features based on the number of features in random subspace. Let r~ represents unsupervised feature selection ratio in the random subspace, that is, from each random feature group Gi (i=1,⋯k), select important features that represent the local structure of the random subspace to generate Gi (i=1,⋯k), it holds that G~i⊂Gi,Gi=Gi⋅r~i=1,⋯k(i=1,⋯k), and ∑i=1kG~i≪m. For convenience, let Si∈Rn×G~i (i=1,⋯k) represent the matrix of the ith base subspace, where n is the number of samples and Gi is the number of features.


### 3.3. Generate Subspace Clustering Solution

LetS1,S2,⋯,Sk represent the base subspaces generated by the joint feature selection strategy, whereSi∈Rn×di (i=1,⋯k) is the ith base subspace, and di=G~i. Let PSi (i=1,⋯,k) be the clustering solution for Si, formally, the clustering solution for the base subspace is PSi=PSix1,PSix2,⋯,PSixn, where PSixj (j=1,⋯,n) is a cluster that contains xj in Si. Let ClSi(l=1,⋯,KSi) denote the lth cluster in the Si, then for ∀xj∈ClSi, it holds that PSixj=ClSi. Thus, the clustering solution of Si also be denoted as
(7)PSi=C1Si,C2Si,⋯,CKSiSi,where KSi is the number of clusters in Si. Then, we can get the clustering solution set of k base subspace, which is represented as P=PS1,PS2,⋯,PSk. Subspace clustering ensemble is the fusion of clustering solutions from multiple subspaces to achieve consistent results P∗.In the clustering process, most approaches view input samples as base units and group them into different clusters. However, when there are more samples, the computational complexity also increases significantly. Huang et al. introduce the concept of superobject [12], which is defined as in the base clustering ensemble where two samples are partitioned into the same cluster, the two samples are in the same original superobject. The size of superobject is between clusters and samples, and it has been proven that viewing superobject as base units when integrated can significantly improve the scalability of data size and simplify the calculation.Inspired by the concept of the superobject [12], this paper groups samples in base subspaces to achieve a set of base subspace clustering solutions P and then generates the adaptive core clusters. The core clusters in high-dimensional space are defined as follows.
(1)
∀xα,xβ∈X, ∀Si(1≤i≤k), PSixα=PSixβ, thenxα,xβ∈o(2)
∀xα∈o,xβ∉o,∃Si(1≤i≤k), PSixα≠PSixβDefinition 1.
LetX be the input dataset and P be set of base subspace clustering solutions. Samples that simultaneously satisfying the following two conditions are core clusters, which are denoted as o.LetO=o1,o2,⋯,oγ be the set of core clusters in a high-dimensional space, where γ is the number of core clusters. It holds that ∀oi≠∅, oi∩oj=∅∀i≠j,i,j=1,2,⋯,γ, and ∪i=1γoi=n.We provide examples of samples clustering in different subspaces. Given a datasetX=x1,⋯,x7, where xi is the ith sample. These samples are grouped into clusters in subspaces, and the relationships between the samples, core clusters, and clusters are described in Table 1. In subspace S1, samples are grouped into two clusters, and in S2 and S3, they are grouped into three clusters, respectively.Table 1
Correspondence between samples, core clusters, and clusters in subspace.
Xx1x2x3x4x5x6x7S1C1S1C1S1C1S1C2S1C2S1C2S1C2S1S2C1S2C1S2C2S2C2S2C2S2C3S2C3S2S3C1S3C1S3C1S3C2S3C2S3C3S3C3S3Oo1o1o2o3o3o4o4According to Definition1, there are four core clusters are generated in the above example. The relationship between the core cluster and the samples is shown in Table 1, namely, x1,x2∈o1, x3∈o2, x4,x5∈o3, and x6,x7∈o4. The core cluster is viewed as the basic units of cluster in subspace, for example in base subspace S1, o1,o2⊂C1S1, o3,o4⊂C2S1.In the conventional subspace clustering approaches, samples are basic units. However, samples correspond to different features in different feature subspaces, so the implicit relationship between samples in different subspaces is also different. A core cluster is a set of samples that are grouped into the same clusters in all subspaces. In the process of subspace ensemble, we view the core cluster as the basic unit, that is, each cluster in subspace is composed of core clusters. We evaluate the stability of clusters in subspace by measuring the distance of core clusters pairs contained in the cluster.Definition 2.
∀Si (1≤i≤k), ∀oα,oβ (α,β=1,⋯γ), the distance between the core clusters pairs is defined as
(8)CCSSioα,oβ=1oαoβ∑xa∈oα∑xb∈oβdSixa,xb,where oα and oβ are the number of samples contained in oα and oβ, respectively. dSi⋅,⋅ is a distance metric function between samples in Si, which can be selected as the Euclidean distance, the Manhattan distance, the Cosine distance, and so on. If the distance between the core clusters pairs is smaller, the more stable the clusters in the corresponding subspace, and vice versa. Clusters are evaluated by the average distance between core clusters.For∀Si (1≤i≤k), there are a total of KSi clusters, and each cluster CjSi(1≤j≤KSi) contains n~jSi core clusters. Obviously, it holds that n~jSi≥1 and ∑j=1KSin~jSi=γ.Definition 3.
For∀CjSi (1≤i≤k,1≤j≤KSi), the average distance of the core clusters is defined as
(9)ACSjSi=2n~jSin~jSi−1∑a=1n~jSi∑b=a+1n~jSiCCSSioa,ob,n~jSi>1,0,n~jSi=1.In Equation (9), CCSsi⋅ is a distance metric function of core clusters pairs in Si and n~jSi is the number of core clusters in CjSi.The smaller the average distance between the core clusters in a cluster, the denser the cluster, that is, the greater the probability that the core clusters will be grouped into clusters in the base subspace, the higher the stability of the clusters. We introduced the cluster stability index (CSI) to describe this relationship.Definition 4.
For∀CjSi (1≤i≤k,1≤j≤KSi), its cluster stability index is defined as
(10)CSIjSi=e−MCSjSi,where MCSjSi is the average distance between the core clusters in CjSi. For MCSjSi∈0,+∞, the range of the cluster stability index is CSIjSi∈0,1.The CSI is an indicator that describes clusters in a subspace. If the cluster has only one core cluster, the distance between the core cluster is 0, and the stability index of the corresponding cluster is 1; the cluster can no longer be divided and is stable. If there are multiple core clusters in a cluster, the greater the average distance between the core clusters, the worse the stability of the clusters, and vice versa.Thus, for∀Si(1≤i≤k), the average of the stability index for all clusters is
(11)CSISi=1KSi∑j=1KSiCSIjSi1≤j≤KSi.When fusing the clustering solutions of base subspace, we weight the subspace by the stability of the clusters in the corresponding base subspace. The weight of each base subspace is calculated by(12)ωSi=CSISi∑i=1kCSISi,where(13)∑i=1kωSi=1.


### 3.4. Subspace Clustering Ensemble

Clustering ensemble is an effective way to improve robustness and stability of clustering solution [13]. We propose four core cluster-based consensus functions that ensemble clustering solutions for each base subspace to achieve the final clustering results.Define the core cluster similarity matrix in base subspace based on whether any two core clusters in the subspace are grouped into the same cluster.Definition 5.
For∀Si, the core cluster similarity matrix is defined as
(14)ASi=aαβSiγ×γα,β∈1,γ,i∈1,k,
where(15)aαβSi=CCSSioα,oβ,ifPSioα=PSioβ,0,otherwise.In Equation (15), PSioα and PSioβ represent clusters containing oα and oβ in Si, respectively. Unlike normal similarity matrices, we define the core cluster similarity matrices rather than similar matrices between samples. At the same time, we achieve the weighted core cluster similarity matrix based on the weights of base subspace, which is represented as
(16)A−1k∑i=1kωSi·ASi.


#### 3.4.1. Hierarchical Clustering Based on the Core Cluster

In this section, we propose the consensus function for hierarchical clustering based on the core clusters, termed C2SWCE_HC. The proposed method views the core cluster as the basic unit and constructs a dendrogram by the core cluster, where the root of the tree corresponds to the dataset and its leaves correspond to all the core clusters in P. Each level of the dendrogram represents the clustering results of different numbers of core clusters, and the final clustering solution can be achieved by specifying a specific level of the dendrogram.The specific steps of integration are as follows: First, theγ core clusters in P as the initial region, which is represented as
(17)Θ0=Θ10,⋯,Θγ0.In Equation (17), Θi0=oi denotes the ith initial region, which corresponds to the ith core cluster. Let M0 be the initial similar matrix. Merge the two most similar core clusters in M0 into one larger cluster and update the similar matrix according to the average link for the next merge of core clusters. Repeat merge the two most similar regions of the similar matrix that were updated in the previous iteration. After each merge, the number of regions is reduced by 1, and after the ith iteration, the merged regions are represented as Θl=Θ1l,⋯,Θγ−ll, and the corresponding similar matrix is updated as follows:
(18)Ml=1ΘilΘjl∑oα∈Θil,oβ∈Θjlaαβlγ−l×γ−l,where Θil is the number of core clusters contained in the ith region after the ith iteration, and the maximum number of iterations of the dendrogram is γ−1.For clarity, C2SWCE_HC is summarized in Algorithm 1.Algorithm 1
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equation (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. InitializeΘ0=Θ10,⋯,Θγ0,M0=A7. Construct the dendrogramforl=1 to γ−1 doAccording toMl−1, merge the two most similar regions to achieve Θl−1UpdateΘl, and achieve Mlend for8. Select the level of the dendrogram according toK, and achieve K clusters9. Map the labels of core cluster to the samples.


#### 3.4.2. Spectral Clustering Based on the Core Cluster

In this section, we introduce the core cluster-based spectral clustering consensus function to ensemble subspace clustering solutions. First, we build the affinity graph that treats the core clusters as graph nodes and the weighted core cluster similarity matrix as the adjacency matrix. The graph is defined as(19)G=V,E,where V=O is the nodes set and E is the edge set. The weights of the edge between the nodes vi and vj is determined by matrix A, that is, Eij=Aij. Let D∈Rγ×γ be the degree matrix of E, the normalized graph Laplacian is computed as
(20)L=D−E=I−D−1/2ED−1/2=I−D−1/2AD−1/2.We perform eigendecomposition onL to achieve the first K eigenvalues, and matrix Ξ∈Rγ×K is constructed by corresponding K eigenvectors and perform K-means on the row vectors of the matrix.For clarity, the C2SWCE_SC algorithm is summarized in Algorithm 2.Algorithm 2
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equations (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. Build the graphG=V,E with V=O,E=A7. Constructed the normalized graph LaplacianL according to Equation (20).8. Perform eigendecomposition onL to achieve the first K eigenvalues and corresponding eigenvectors to build Ξ9. After normalizingΞ, perform K-means to categorized the core clusters10. Map the labels of core cluster to the samples


#### 3.4.3. Bipartite Graph Partition Based on the Core Clusters

Under the C2SWCE framework, we propose the bipartite graph clustering ensemble based on the core clusters, termed C2SWCE_BG. In ensemble process, we use the core clusters and clusters as graph nodes to construct the bipartite graph. The specific steps are described below.In different base subspaces, the core clusters are grouped into a set of clusters. In all base subspaces, the set of clusters is(21)C=C1,C2,⋯,Cnc,where nc=∑i=1kKSi is the number of clusters in P. We view both the core cluster and cluster as graph nodes, and construct the bipartite graph. That is:
(22)GBG=UBG,VBG,EBG,where UBG=OUC is the node set corresponding to the core cluster and the cluster; VBG=C is the node set corresponding to the cluster in P; it holds that UBG=γ+nc, VBG=nc. EBG is the edge set.InC, clusters in the same subspace contain different core clusters, while clusters in different subspaces may contain the same core cluster. Therefore, we use the Jaccard coefficient to measure the similarity of clusters. The core clusters are viewed as base units; the similarity matrix between clusters is defined as
(23)Z=zijnc×nc,where(24)zij=JCi,Cj=Ci∩CjCi∪Cji,j∈1,nc.Clusters in the same base subspace do not intersect, so the Jaccard coefficient between clusters in the same subspace is 0. The similarity between clusters pairs is calculated by Equation (24) and the clusters are weighted based on similarity.The similarity matrix between clusters and the core clusters is constructed as(25)Z~=z~ijγ×nc,where
(26)z~ij=ωSi⋅CSIj,ifoi⊆Cj,0,otherwise.In Equation (26), CSIj is the cluster stability index of Cj in Sl, ωSl is the weight of Cj in Sl (ll∈1,k,i∈i,γ,j∈1,nc).Connect the matrixZ~ and matrix Z to generate matrix EBG∈Rγ+nc×nc, that is,
(27)EBG=eijγ+nc×ncThe entry ofEBG corresponds to the weight of the edge between the two nodes in GBG, denoted as
(28)eij=ωSl⋅CSIj,ifui∈C,vj∈O,vj⊆ui,Jui,vj,ifu~i∈C,v~j∈C,0,otherwise.In Equation (28), CSIj is the cluster stability index of the jth cluster. J⋅,⋅ is the Jaccard coefficient operator, which is calculated according to Equation (24). In GBG, there are no edges between cluster nodes or between the core cluster nodes, only edges between cluster nodes and core cluster nodes.According to the corresponding features in different base subspaces, we use the weight of the subspace corresponding to the cluster to which the core cluster belongs as the weight of the edges of the core cluster and the cluster. The higher the stability of the cluster, the greater its impact during integration and the greater the weights assigned.Finally, we use Tcut [14] to segment GBG. All nodes are partitioned into K disjoint groups. The samples contained in the core clusters and cluster of the same group are partitioned in the same cluster.The specific steps of C2SWCE_BG are summarized in Algorithm 3.Algorithm 3
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculated the similarity of the cluster according to Equations (23) and (24)4. Calculate the similarity of clusters and core clusters according to Equations (25) and (26)5. Construct the bipartite graph according to Equation (22) with U=OUC, V=C6. Combine matrixZ~ and matrix Z to generate EBG7. Segment theGBG by Tcut8. Map the labels of core cluster to the samples


#### 3.4.4. Metacluster-Based Ensemble Clustering

Under the proposed C2SWCE framework, we propose the metacluster-based consensus clustering algorithm, termed C2SWCE_MC. In the proposed approach, clusters are regarded as the basic units that use the similarities between them to divide clusters into different groups, and the samples of the same group are divided into the same cluster.C2SWCE_MC first treats all clusters as nodes and constructs the similar graph, which is defined as:
(29)GM=VM,EM,where VM=C is the node set for all clusters in P and EM is the edge set. The weights of the edges between nodes are defined by the similarity matrix Z of the clusters by Equations (23) and (24).
(30)EM=eijnc×nc,where eij=zij.Finally, we adopt Ncut [15] to partition nodes into K disjoint sets of nodes, which denotes as
(31)MC=MC1,⋯,MCK,where MCi (i∈1,k) represents the ith metacluster, which is the set of clusters. The core clusters are treated as base units in the ensemble. In each subspace, oi(i∈1,γ) is partitioned into a cluster. oi may appear multiple times in MC. Define the discriminant function δ to represent the relationship:
(32)δ=1,ifoi⊆Cβ,0,otherwise.The probability thatoi (i∈1,γ) is grouped into metacluster is
(33)ϕoi,MCj=1MCj∑Cβ⊆MCjδj∈1,nc,where MCj is the number of clusters in the jth metacluster.Finally,oi is assigned to the metacluster with the highest probability, namely,
(34)argmaxMCj∈MCϕoi,MCj.We summarize C2SWCE_MC in Algorithm 4.Algorithm 4
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculate the similarity matrix of clusters inC according to Equation (23) and (24)4. Build the graphGM according to Equations (29)–(31)5. Partition the graph nodes intoK disjoint groups by Tcut, generating MC6. The core clusters are treated as the base unit, the metaclusters corresponding to the core clusters are determined according to Equations (32), (34), and (35)7. Map the labels of core cluster to the samples


## 3.1. Overview

In this paper, we introduce a locally subspace weighted clustering ensemble framework. First, we randomly selectk feature subset from the original feature set to generate k random subspaces, and then selecting features representing the local structure from each random subspace to generate base subspace. Then, spectral clustering is performed on each base subspace to achieve subspace clustering solutions. The adaptive core clusters are generated from the base subspace clustering solution, which is a set of samples that are grouped into the same cluster in all subspaces. Then, by calculating the distance of the core cluster in different subspaces, the stability of the clusters is evaluated and weighted the base subspace clustering solution. Finally, in order to integrate the clustering solutions of the base subspaces and get the consensus results, in the C2SWCE framework, we propose four adaptive core cluster-based consensus functions, to achieve the final clustering solutions.LetX∈Rn×m be the input dataset that contains n samples, each with m features. Let xi (i=1,⋯n) denotes the ith sample, which corresponds to the ith row of X, so that the input data can be represented as X=x1,x2,⋯,xn. Let fi (j=1,⋯m) denotes the ith feature of the sample, which corresponds to the jth column of the X; therefore, matrix X can also be represented as X=f1,f2,⋯,fm.


## 3.2. Generate Subspaces

The natural cluster structure of high-dimensional data is hidden in low-dimensional subspaces [8]. Subspace clustering explores the possibility of grouping samples in different feature sets. Subspace clustering ensemble fuses clusters in different subspaces to obtain the final clustering solution, which is an effective method for clustering high-dimensional data. The recently proposed subspace clustering ensemble approaches [5, 9], which uses random feature selection to generate a set of random subspaces. Random subspace has diversity; it explored the potential cluster of high-dimensional dataset from the perspective of different features to achieve diverse subspace clustering solutions. Literature [10] uses stratified sampling method to generate feature groups and verifies that this method is superior to random sampling. This paper uses the random feature selection to obtain a variety of subspaces, and unsupervised feature selection is performed again on the random subspace to select the features that retain the local structure of random subspace.The specific procedure of feature subspace segmentation is as follows. The original feature setf1,f2⋯,fm is randomly sampled according to the sampling ratio r, the original features are classified into k feature groups G1,G2,⋯,Gk, where G1i=1,⋯k represents the features contained in the ith random subspace. Let Xi~∈Rn×Gi denotes the matrix corresponding to the ith random subspace, n is the number of samples, and Gii=1,⋯,k is the number of features in the ith random subspace. Obviously, it holds that ∀Gi≠ϕi=1,⋯,k and Gi=m⋅r.A set of base subspaces is generated by calculating the Laplace score [11] of features in each random subspace and selecting the important features in random subspace. Then, we construct the KNN graph for each random subspace, which represents the local structure of the random subspace. The KNN graph of the ith subspace is defined as follows:
(1)GiKNN=Vi,Ei,where Vi=v1,v2,⋯,vn is the set of nodes corresponding to the samples x1,x2,⋯,xn in the ith subspace and Ei is the edge set of the ith subspace. We use the Gaussian kernel function to calculate the weights of the edges between the nodes in the subspace and their corresponding KNN nodes. Ei is defined as
(2)Ei=eαβin×n,where
(3)eαβi=exp−xαi−xβi22σ2,ifxαi∈KNNxβiorxβi∈KNNxαi,0,otherwise.In Equation (3), xαi and xβi are the two samples in the ith subspace, respectively. xαi−xβi2 is the Euclidean distance between xαi and xβi, and KNN⋅ is the K-nearest neighbor (KNN) operator, and σ is the mean of Euclidean distance between the sample and its KNN.LetDi=diagEi1 be the degree matrix of the ith subspace, where 1=1,⋯,1T. The Laplacian of the graph is calculated as follows:
(4)Li=Di−Ei.In theith subspace, the Laplace score of fli (l=1,⋯Gi) is
(5)Lfli=f~li⊤Lif~lif~li⊤Dif~li,where(6)f~li=fli−fli⊤Di11⊤Di11.We calculate the Laplace score for each feature in the subspace and arrange them in descending order to select the topd features. We determine the number of second selected features based on the number of features in random subspace. Let r~ represents unsupervised feature selection ratio in the random subspace, that is, from each random feature group Gi (i=1,⋯k), select important features that represent the local structure of the random subspace to generate Gi (i=1,⋯k), it holds that G~i⊂Gi,Gi=Gi⋅r~i=1,⋯k(i=1,⋯k), and ∑i=1kG~i≪m. For convenience, let Si∈Rn×G~i (i=1,⋯k) represent the matrix of the ith base subspace, where n is the number of samples and Gi is the number of features.


## 3.3. Generate Subspace Clustering Solution

LetS1,S2,⋯,Sk represent the base subspaces generated by the joint feature selection strategy, whereSi∈Rn×di (i=1,⋯k) is the ith base subspace, and di=G~i. Let PSi (i=1,⋯,k) be the clustering solution for Si, formally, the clustering solution for the base subspace is PSi=PSix1,PSix2,⋯,PSixn, where PSixj (j=1,⋯,n) is a cluster that contains xj in Si. Let ClSi(l=1,⋯,KSi) denote the lth cluster in the Si, then for ∀xj∈ClSi, it holds that PSixj=ClSi. Thus, the clustering solution of Si also be denoted as
(7)PSi=C1Si,C2Si,⋯,CKSiSi,where KSi is the number of clusters in Si. Then, we can get the clustering solution set of k base subspace, which is represented as P=PS1,PS2,⋯,PSk. Subspace clustering ensemble is the fusion of clustering solutions from multiple subspaces to achieve consistent results P∗.In the clustering process, most approaches view input samples as base units and group them into different clusters. However, when there are more samples, the computational complexity also increases significantly. Huang et al. introduce the concept of superobject [12], which is defined as in the base clustering ensemble where two samples are partitioned into the same cluster, the two samples are in the same original superobject. The size of superobject is between clusters and samples, and it has been proven that viewing superobject as base units when integrated can significantly improve the scalability of data size and simplify the calculation.Inspired by the concept of the superobject [12], this paper groups samples in base subspaces to achieve a set of base subspace clustering solutions P and then generates the adaptive core clusters. The core clusters in high-dimensional space are defined as follows.
(1)
∀xα,xβ∈X, ∀Si(1≤i≤k), PSixα=PSixβ, thenxα,xβ∈o(2)
∀xα∈o,xβ∉o,∃Si(1≤i≤k), PSixα≠PSixβDefinition 1.
LetX be the input dataset and P be set of base subspace clustering solutions. Samples that simultaneously satisfying the following two conditions are core clusters, which are denoted as o.LetO=o1,o2,⋯,oγ be the set of core clusters in a high-dimensional space, where γ is the number of core clusters. It holds that ∀oi≠∅, oi∩oj=∅∀i≠j,i,j=1,2,⋯,γ, and ∪i=1γoi=n.We provide examples of samples clustering in different subspaces. Given a datasetX=x1,⋯,x7, where xi is the ith sample. These samples are grouped into clusters in subspaces, and the relationships between the samples, core clusters, and clusters are described in Table 1. In subspace S1, samples are grouped into two clusters, and in S2 and S3, they are grouped into three clusters, respectively.Table 1
Correspondence between samples, core clusters, and clusters in subspace.
Xx1x2x3x4x5x6x7S1C1S1C1S1C1S1C2S1C2S1C2S1C2S1S2C1S2C1S2C2S2C2S2C2S2C3S2C3S2S3C1S3C1S3C1S3C2S3C2S3C3S3C3S3Oo1o1o2o3o3o4o4According to Definition1, there are four core clusters are generated in the above example. The relationship between the core cluster and the samples is shown in Table 1, namely, x1,x2∈o1, x3∈o2, x4,x5∈o3, and x6,x7∈o4. The core cluster is viewed as the basic units of cluster in subspace, for example in base subspace S1, o1,o2⊂C1S1, o3,o4⊂C2S1.In the conventional subspace clustering approaches, samples are basic units. However, samples correspond to different features in different feature subspaces, so the implicit relationship between samples in different subspaces is also different. A core cluster is a set of samples that are grouped into the same clusters in all subspaces. In the process of subspace ensemble, we view the core cluster as the basic unit, that is, each cluster in subspace is composed of core clusters. We evaluate the stability of clusters in subspace by measuring the distance of core clusters pairs contained in the cluster.Definition 2.
∀Si (1≤i≤k), ∀oα,oβ (α,β=1,⋯γ), the distance between the core clusters pairs is defined as
(8)CCSSioα,oβ=1oαoβ∑xa∈oα∑xb∈oβdSixa,xb,where oα and oβ are the number of samples contained in oα and oβ, respectively. dSi⋅,⋅ is a distance metric function between samples in Si, which can be selected as the Euclidean distance, the Manhattan distance, the Cosine distance, and so on. If the distance between the core clusters pairs is smaller, the more stable the clusters in the corresponding subspace, and vice versa. Clusters are evaluated by the average distance between core clusters.For∀Si (1≤i≤k), there are a total of KSi clusters, and each cluster CjSi(1≤j≤KSi) contains n~jSi core clusters. Obviously, it holds that n~jSi≥1 and ∑j=1KSin~jSi=γ.Definition 3.
For∀CjSi (1≤i≤k,1≤j≤KSi), the average distance of the core clusters is defined as
(9)ACSjSi=2n~jSin~jSi−1∑a=1n~jSi∑b=a+1n~jSiCCSSioa,ob,n~jSi>1,0,n~jSi=1.In Equation (9), CCSsi⋅ is a distance metric function of core clusters pairs in Si and n~jSi is the number of core clusters in CjSi.The smaller the average distance between the core clusters in a cluster, the denser the cluster, that is, the greater the probability that the core clusters will be grouped into clusters in the base subspace, the higher the stability of the clusters. We introduced the cluster stability index (CSI) to describe this relationship.Definition 4.
For∀CjSi (1≤i≤k,1≤j≤KSi), its cluster stability index is defined as
(10)CSIjSi=e−MCSjSi,where MCSjSi is the average distance between the core clusters in CjSi. For MCSjSi∈0,+∞, the range of the cluster stability index is CSIjSi∈0,1.The CSI is an indicator that describes clusters in a subspace. If the cluster has only one core cluster, the distance between the core cluster is 0, and the stability index of the corresponding cluster is 1; the cluster can no longer be divided and is stable. If there are multiple core clusters in a cluster, the greater the average distance between the core clusters, the worse the stability of the clusters, and vice versa.Thus, for∀Si(1≤i≤k), the average of the stability index for all clusters is
(11)CSISi=1KSi∑j=1KSiCSIjSi1≤j≤KSi.When fusing the clustering solutions of base subspace, we weight the subspace by the stability of the clusters in the corresponding base subspace. The weight of each base subspace is calculated by(12)ωSi=CSISi∑i=1kCSISi,where(13)∑i=1kωSi=1.


## 3.4. Subspace Clustering Ensemble

Clustering ensemble is an effective way to improve robustness and stability of clustering solution [13]. We propose four core cluster-based consensus functions that ensemble clustering solutions for each base subspace to achieve the final clustering results.Define the core cluster similarity matrix in base subspace based on whether any two core clusters in the subspace are grouped into the same cluster.Definition 5.
For∀Si, the core cluster similarity matrix is defined as
(14)ASi=aαβSiγ×γα,β∈1,γ,i∈1,k,
where(15)aαβSi=CCSSioα,oβ,ifPSioα=PSioβ,0,otherwise.In Equation (15), PSioα and PSioβ represent clusters containing oα and oβ in Si, respectively. Unlike normal similarity matrices, we define the core cluster similarity matrices rather than similar matrices between samples. At the same time, we achieve the weighted core cluster similarity matrix based on the weights of base subspace, which is represented as
(16)A−1k∑i=1kωSi·ASi.


### 3.4.1. Hierarchical Clustering Based on the Core Cluster

In this section, we propose the consensus function for hierarchical clustering based on the core clusters, termed C2SWCE_HC. The proposed method views the core cluster as the basic unit and constructs a dendrogram by the core cluster, where the root of the tree corresponds to the dataset and its leaves correspond to all the core clusters in P. Each level of the dendrogram represents the clustering results of different numbers of core clusters, and the final clustering solution can be achieved by specifying a specific level of the dendrogram.The specific steps of integration are as follows: First, theγ core clusters in P as the initial region, which is represented as
(17)Θ0=Θ10,⋯,Θγ0.In Equation (17), Θi0=oi denotes the ith initial region, which corresponds to the ith core cluster. Let M0 be the initial similar matrix. Merge the two most similar core clusters in M0 into one larger cluster and update the similar matrix according to the average link for the next merge of core clusters. Repeat merge the two most similar regions of the similar matrix that were updated in the previous iteration. After each merge, the number of regions is reduced by 1, and after the ith iteration, the merged regions are represented as Θl=Θ1l,⋯,Θγ−ll, and the corresponding similar matrix is updated as follows:
(18)Ml=1ΘilΘjl∑oα∈Θil,oβ∈Θjlaαβlγ−l×γ−l,where Θil is the number of core clusters contained in the ith region after the ith iteration, and the maximum number of iterations of the dendrogram is γ−1.For clarity, C2SWCE_HC is summarized in Algorithm 1.Algorithm 1
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equation (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. InitializeΘ0=Θ10,⋯,Θγ0,M0=A7. Construct the dendrogramforl=1 to γ−1 doAccording toMl−1, merge the two most similar regions to achieve Θl−1UpdateΘl, and achieve Mlend for8. Select the level of the dendrogram according toK, and achieve K clusters9. Map the labels of core cluster to the samples.


### 3.4.2. Spectral Clustering Based on the Core Cluster

In this section, we introduce the core cluster-based spectral clustering consensus function to ensemble subspace clustering solutions. First, we build the affinity graph that treats the core clusters as graph nodes and the weighted core cluster similarity matrix as the adjacency matrix. The graph is defined as(19)G=V,E,where V=O is the nodes set and E is the edge set. The weights of the edge between the nodes vi and vj is determined by matrix A, that is, Eij=Aij. Let D∈Rγ×γ be the degree matrix of E, the normalized graph Laplacian is computed as
(20)L=D−E=I−D−1/2ED−1/2=I−D−1/2AD−1/2.We perform eigendecomposition onL to achieve the first K eigenvalues, and matrix Ξ∈Rγ×K is constructed by corresponding K eigenvectors and perform K-means on the row vectors of the matrix.For clarity, the C2SWCE_SC algorithm is summarized in Algorithm 2.Algorithm 2
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equations (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. Build the graphG=V,E with V=O,E=A7. Constructed the normalized graph LaplacianL according to Equation (20).8. Perform eigendecomposition onL to achieve the first K eigenvalues and corresponding eigenvectors to build Ξ9. After normalizingΞ, perform K-means to categorized the core clusters10. Map the labels of core cluster to the samples


### 3.4.3. Bipartite Graph Partition Based on the Core Clusters

Under the C2SWCE framework, we propose the bipartite graph clustering ensemble based on the core clusters, termed C2SWCE_BG. In ensemble process, we use the core clusters and clusters as graph nodes to construct the bipartite graph. The specific steps are described below.In different base subspaces, the core clusters are grouped into a set of clusters. In all base subspaces, the set of clusters is(21)C=C1,C2,⋯,Cnc,where nc=∑i=1kKSi is the number of clusters in P. We view both the core cluster and cluster as graph nodes, and construct the bipartite graph. That is:
(22)GBG=UBG,VBG,EBG,where UBG=OUC is the node set corresponding to the core cluster and the cluster; VBG=C is the node set corresponding to the cluster in P; it holds that UBG=γ+nc, VBG=nc. EBG is the edge set.InC, clusters in the same subspace contain different core clusters, while clusters in different subspaces may contain the same core cluster. Therefore, we use the Jaccard coefficient to measure the similarity of clusters. The core clusters are viewed as base units; the similarity matrix between clusters is defined as
(23)Z=zijnc×nc,where(24)zij=JCi,Cj=Ci∩CjCi∪Cji,j∈1,nc.Clusters in the same base subspace do not intersect, so the Jaccard coefficient between clusters in the same subspace is 0. The similarity between clusters pairs is calculated by Equation (24) and the clusters are weighted based on similarity.The similarity matrix between clusters and the core clusters is constructed as(25)Z~=z~ijγ×nc,where
(26)z~ij=ωSi⋅CSIj,ifoi⊆Cj,0,otherwise.In Equation (26), CSIj is the cluster stability index of Cj in Sl, ωSl is the weight of Cj in Sl (ll∈1,k,i∈i,γ,j∈1,nc).Connect the matrixZ~ and matrix Z to generate matrix EBG∈Rγ+nc×nc, that is,
(27)EBG=eijγ+nc×ncThe entry ofEBG corresponds to the weight of the edge between the two nodes in GBG, denoted as
(28)eij=ωSl⋅CSIj,ifui∈C,vj∈O,vj⊆ui,Jui,vj,ifu~i∈C,v~j∈C,0,otherwise.In Equation (28), CSIj is the cluster stability index of the jth cluster. J⋅,⋅ is the Jaccard coefficient operator, which is calculated according to Equation (24). In GBG, there are no edges between cluster nodes or between the core cluster nodes, only edges between cluster nodes and core cluster nodes.According to the corresponding features in different base subspaces, we use the weight of the subspace corresponding to the cluster to which the core cluster belongs as the weight of the edges of the core cluster and the cluster. The higher the stability of the cluster, the greater its impact during integration and the greater the weights assigned.Finally, we use Tcut [14] to segment GBG. All nodes are partitioned into K disjoint groups. The samples contained in the core clusters and cluster of the same group are partitioned in the same cluster.The specific steps of C2SWCE_BG are summarized in Algorithm 3.Algorithm 3
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculated the similarity of the cluster according to Equations (23) and (24)4. Calculate the similarity of clusters and core clusters according to Equations (25) and (26)5. Construct the bipartite graph according to Equation (22) with U=OUC, V=C6. Combine matrixZ~ and matrix Z to generate EBG7. Segment theGBG by Tcut8. Map the labels of core cluster to the samples


### 3.4.4. Metacluster-Based Ensemble Clustering

Under the proposed C2SWCE framework, we propose the metacluster-based consensus clustering algorithm, termed C2SWCE_MC. In the proposed approach, clusters are regarded as the basic units that use the similarities between them to divide clusters into different groups, and the samples of the same group are divided into the same cluster.C2SWCE_MC first treats all clusters as nodes and constructs the similar graph, which is defined as:
(29)GM=VM,EM,where VM=C is the node set for all clusters in P and EM is the edge set. The weights of the edges between nodes are defined by the similarity matrix Z of the clusters by Equations (23) and (24).
(30)EM=eijnc×nc,where eij=zij.Finally, we adopt Ncut [15] to partition nodes into K disjoint sets of nodes, which denotes as
(31)MC=MC1,⋯,MCK,where MCi (i∈1,k) represents the ith metacluster, which is the set of clusters. The core clusters are treated as base units in the ensemble. In each subspace, oi(i∈1,γ) is partitioned into a cluster. oi may appear multiple times in MC. Define the discriminant function δ to represent the relationship:
(32)δ=1,ifoi⊆Cβ,0,otherwise.The probability thatoi (i∈1,γ) is grouped into metacluster is
(33)ϕoi,MCj=1MCj∑Cβ⊆MCjδj∈1,nc,where MCj is the number of clusters in the jth metacluster.Finally,oi is assigned to the metacluster with the highest probability, namely,
(34)argmaxMCj∈MCϕoi,MCj.We summarize C2SWCE_MC in Algorithm 4.Algorithm 4
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculate the similarity matrix of clusters inC according to Equation (23) and (24)4. Build the graphGM according to Equations (29)–(31)5. Partition the graph nodes intoK disjoint groups by Tcut, generating MC6. The core clusters are treated as the base unit, the metaclusters corresponding to the core clusters are determined according to Equations (32), (34), and (35)7. Map the labels of core cluster to the samples


## 3.4.1. Hierarchical Clustering Based on the Core Cluster

In this section, we propose the consensus function for hierarchical clustering based on the core clusters, termed C2SWCE_HC. The proposed method views the core cluster as the basic unit and constructs a dendrogram by the core cluster, where the root of the tree corresponds to the dataset and its leaves correspond to all the core clusters in P. Each level of the dendrogram represents the clustering results of different numbers of core clusters, and the final clustering solution can be achieved by specifying a specific level of the dendrogram.The specific steps of integration are as follows: First, theγ core clusters in P as the initial region, which is represented as
(17)Θ0=Θ10,⋯,Θγ0.In Equation (17), Θi0=oi denotes the ith initial region, which corresponds to the ith core cluster. Let M0 be the initial similar matrix. Merge the two most similar core clusters in M0 into one larger cluster and update the similar matrix according to the average link for the next merge of core clusters. Repeat merge the two most similar regions of the similar matrix that were updated in the previous iteration. After each merge, the number of regions is reduced by 1, and after the ith iteration, the merged regions are represented as Θl=Θ1l,⋯,Θγ−ll, and the corresponding similar matrix is updated as follows:
(18)Ml=1ΘilΘjl∑oα∈Θil,oβ∈Θjlaαβlγ−l×γ−l,where Θil is the number of core clusters contained in the ith region after the ith iteration, and the maximum number of iterations of the dendrogram is γ−1.For clarity, C2SWCE_HC is summarized in Algorithm 1.Algorithm 1
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equation (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. InitializeΘ0=Θ10,⋯,Θγ0,M0=A7. Construct the dendrogramforl=1 to γ−1 doAccording toMl−1, merge the two most similar regions to achieve Θl−1UpdateΘl, and achieve Mlend for8. Select the level of the dendrogram according toK, and achieve K clusters9. Map the labels of core cluster to the samples.


## 3.4.2. Spectral Clustering Based on the Core Cluster

In this section, we introduce the core cluster-based spectral clustering consensus function to ensemble subspace clustering solutions. First, we build the affinity graph that treats the core clusters as graph nodes and the weighted core cluster similarity matrix as the adjacency matrix. The graph is defined as(19)G=V,E,where V=O is the nodes set and E is the edge set. The weights of the edge between the nodes vi and vj is determined by matrix A, that is, Eij=Aij. Let D∈Rγ×γ be the degree matrix of E, the normalized graph Laplacian is computed as
(20)L=D−E=I−D−1/2ED−1/2=I−D−1/2AD−1/2.We perform eigendecomposition onL to achieve the first K eigenvalues, and matrix Ξ∈Rγ×K is constructed by corresponding K eigenvectors and perform K-means on the row vectors of the matrix.For clarity, the C2SWCE_SC algorithm is summarized in Algorithm 2.Algorithm 2
Input:S1,S2,⋯,Sk, KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculates the average distance between core clusters according to Equations (8) and (9)4. Calculates the CSI of the cluster according to Equation (10), and ωSi is achieved according to Equations (11)–(13)5. Construct the core cluster similarity matrixASi (i=1,⋯,k) according to Equations (14) and (15), and the weighted core cluster similarity matrix A according to Equation (16).6. Build the graphG=V,E with V=O,E=A7. Constructed the normalized graph LaplacianL according to Equation (20).8. Perform eigendecomposition onL to achieve the first K eigenvalues and corresponding eigenvectors to build Ξ9. After normalizingΞ, perform K-means to categorized the core clusters10. Map the labels of core cluster to the samples


## 3.4.3. Bipartite Graph Partition Based on the Core Clusters

Under the C2SWCE framework, we propose the bipartite graph clustering ensemble based on the core clusters, termed C2SWCE_BG. In ensemble process, we use the core clusters and clusters as graph nodes to construct the bipartite graph. The specific steps are described below.In different base subspaces, the core clusters are grouped into a set of clusters. In all base subspaces, the set of clusters is(21)C=C1,C2,⋯,Cnc,where nc=∑i=1kKSi is the number of clusters in P. We view both the core cluster and cluster as graph nodes, and construct the bipartite graph. That is:
(22)GBG=UBG,VBG,EBG,where UBG=OUC is the node set corresponding to the core cluster and the cluster; VBG=C is the node set corresponding to the cluster in P; it holds that UBG=γ+nc, VBG=nc. EBG is the edge set.InC, clusters in the same subspace contain different core clusters, while clusters in different subspaces may contain the same core cluster. Therefore, we use the Jaccard coefficient to measure the similarity of clusters. The core clusters are viewed as base units; the similarity matrix between clusters is defined as
(23)Z=zijnc×nc,where(24)zij=JCi,Cj=Ci∩CjCi∪Cji,j∈1,nc.Clusters in the same base subspace do not intersect, so the Jaccard coefficient between clusters in the same subspace is 0. The similarity between clusters pairs is calculated by Equation (24) and the clusters are weighted based on similarity.The similarity matrix between clusters and the core clusters is constructed as(25)Z~=z~ijγ×nc,where
(26)z~ij=ωSi⋅CSIj,ifoi⊆Cj,0,otherwise.In Equation (26), CSIj is the cluster stability index of Cj in Sl, ωSl is the weight of Cj in Sl (ll∈1,k,i∈i,γ,j∈1,nc).Connect the matrixZ~ and matrix Z to generate matrix EBG∈Rγ+nc×nc, that is,
(27)EBG=eijγ+nc×ncThe entry ofEBG corresponds to the weight of the edge between the two nodes in GBG, denoted as
(28)eij=ωSl⋅CSIj,ifui∈C,vj∈O,vj⊆ui,Jui,vj,ifu~i∈C,v~j∈C,0,otherwise.In Equation (28), CSIj is the cluster stability index of the jth cluster. J⋅,⋅ is the Jaccard coefficient operator, which is calculated according to Equation (24). In GBG, there are no edges between cluster nodes or between the core cluster nodes, only edges between cluster nodes and core cluster nodes.According to the corresponding features in different base subspaces, we use the weight of the subspace corresponding to the cluster to which the core cluster belongs as the weight of the edges of the core cluster and the cluster. The higher the stability of the cluster, the greater its impact during integration and the greater the weights assigned.Finally, we use Tcut [14] to segment GBG. All nodes are partitioned into K disjoint groups. The samples contained in the core clusters and cluster of the same group are partitioned in the same cluster.The specific steps of C2SWCE_BG are summarized in Algorithm 3.Algorithm 3
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculated the similarity of the cluster according to Equations (23) and (24)4. Calculate the similarity of clusters and core clusters according to Equations (25) and (26)5. Construct the bipartite graph according to Equation (22) with U=OUC, V=C6. Combine matrixZ~ and matrix Z to generate EBG7. Segment theGBG by Tcut8. Map the labels of core cluster to the samples


## 3.4.4. Metacluster-Based Ensemble Clustering

Under the proposed C2SWCE framework, we propose the metacluster-based consensus clustering algorithm, termed C2SWCE_MC. In the proposed approach, clusters are regarded as the basic units that use the similarities between them to divide clusters into different groups, and the samples of the same group are divided into the same cluster.C2SWCE_MC first treats all clusters as nodes and constructs the similar graph, which is defined as:
(29)GM=VM,EM,where VM=C is the node set for all clusters in P and EM is the edge set. The weights of the edges between nodes are defined by the similarity matrix Z of the clusters by Equations (23) and (24).
(30)EM=eijnc×nc,where eij=zij.Finally, we adopt Ncut [15] to partition nodes into K disjoint sets of nodes, which denotes as
(31)MC=MC1,⋯,MCK,where MCi (i∈1,k) represents the ith metacluster, which is the set of clusters. The core clusters are treated as base units in the ensemble. In each subspace, oi(i∈1,γ) is partitioned into a cluster. oi may appear multiple times in MC. Define the discriminant function δ to represent the relationship:
(32)δ=1,ifoi⊆Cβ,0,otherwise.The probability thatoi (i∈1,γ) is grouped into metacluster is
(33)ϕoi,MCj=1MCj∑Cβ⊆MCjδj∈1,nc,where MCj is the number of clusters in the jth metacluster.Finally,oi is assigned to the metacluster with the highest probability, namely,
(34)argmaxMCj∈MCϕoi,MCj.We summarize C2SWCE_MC in Algorithm 4.Algorithm 4
Input:S1,S2,⋯,Sk,KOutput:P∗1. Performed clustering on each base subspace to generate a set of subspace clustering ensembleP2. Generate the adaptive core clusters according to Definition13. Calculate the similarity matrix of clusters inC according to Equation (23) and (24)4. Build the graphGM according to Equations (29)–(31)5. Partition the graph nodes intoK disjoint groups by Tcut, generating MC6. The core clusters are treated as the base unit, the metaclusters corresponding to the core clusters are determined according to Equations (32), (34), and (35)7. Map the labels of core cluster to the samples


## 4. Experiments

In this section, we conduct experiments on eight high-dimensional datasets to compare the proposed four subspace clustering ensemble algorithms against several clustering approaches. All the experiments in this paper are conducted in MATLAB R2016a on a PC with 8 Intel 3.40 GHz processors and 8 GB of RAM.


### 4.1. Datasets

In our experiments, we use eight high-dimensional datasets, which including four cancer gene expression datasets and four image datasets. The 4 gene expression datasets, namely, Yeoh02v11, Yeoh02v21, Bhattacharjee20011, and Golub1999v11. The Yeoh02v1 dataset and Yeoh02v2 dataset are the pediatric acute lymphoblastic leukemia dataset, in which Yeoh02v1 dataset contains 2 categories of genes expressed in leukemic blasts, and Yeoh02v2 dataset contains 6 categories of genes expressed in leukemic blasts. Bhattacharjee2001 is the lung tumor samples dataset, which contains 186 lung tumor samples and 17 normal lung tissues. Golub1999v1 is the leukemia dataset, which contains acute myeloid leukemia samples and acute lymphoblastic leukemia samples. The other 4 image datasets, including COIL_203, USPS2, Semeion4 and Multi Featureples4. The COIL_20 dataset is an image dataset containing 20 item objects, and the other 3 image datasets are all handwritten digit datasets. The USPS dataset contains a total of 10 categories and 11,000 samples. For facilitating comparison, we randomly selected 10% of the samples from each category of the USPS dataset to form a dataset containing 1,100 samples, which is represented as USPS_10P.To simplify the description, the 8 datasets will be abbreviated as D1 to D8, respectively. The details of the datasets are given in Table2.
(1)
https://schlieplab.org/Supplements/CompCancer/(2)
http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html(3)
https://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php(4)
http://archive.ics.uci.edu/ml/index.phpTable 2
Description of the datasets.
DatasetAbbreviationSampleDimensionClassYeoh02v1D124825262Yeoh02v2D224825266Bhattacharje2001D320315435Golub99v1D47218682USPS_10PD5110025610COIL_20D61440102420SemeionD7159325610Multiple featuresD8200064910


### 4.2. Evaluation Criterion

We use the normalized mutual information (NMI) and the clustering accuracy (ACC) to evaluate the quality of clustering results. The NMI metrics measure the accuracy of the clustering results according to the shared information of the ground-truth clustering solution and the test clustering solution. LetP denote the clustering solution of the proposed method, and PG denote the groundtruth clustering solution. The NMI score of P with respect to PG is calculated as
(35)NMIP,PG=∑i=1nP∑j=1nGnijlognijn/niPnjG∑i=1nPniPlogniP/n∑j=1nGnjGlognjG/n,where nP and nG are the number of clusters in P and PG, respectively. niP is the number of samples for the ith cluster in P, and njG is the number of samples for the jth cluster in PG; n is the number of input samples, and nij is the number of samples that the ith cluster in P and the jth cluster in PG jointly contain.The ACC measures the ratio of the number of samples that are correctly classified to the number of all samples. The ACC is an indicator that evaluates the accuracy of the clustering result, which is defined as(36)ACCP,PG=∑i=1nδPGxi,mapPxin,where PGxi is the groundtruth label corresponding to xi, and Pxi is the test clustering labels corresponding to xi in proposed approach. map⋅ is the relabel mapping function that aligns the test clustering label with the groundtruth label. δ⋅ is an indicator function, which is defined as
(37)δx,y=1,ifx=y,0,otherwise.The value interval for ACC and NMI score is0,1, and a higher ACC score or NMI score indicates a better clustering effect.


### 4.3. Discussion of the Parameter Selection

There are several parameters in proposed C2SWCE, where k is the number of subspaces, r is the random feature selection ratio, r~ is the unsupervised feature selection ratio, and K is the number of clusters. To increase the diversity of ensemble members, each subspace randomly generates a different number of clusters within range 2,n. We set the random feature selection ratio to r=0.5, and the number of generated random subspaces is in the range of 5,25, with the increment of 5. For each parameter setting, we run the spectral clustering methods 10 times in each random subspace, generating core clusters according to Definition 1 at each time. The average number of generated core clusters is shown in Figure 2.
(1)
The relationship betweenk and γFigure 2
The correspondence betweenk and γ (r=0.5).
(a)(b)In the proposed C2SWCE framework, the core cluster is the basic unit of integration.In Figure2, k is the number of generated random subspaces, k=0 corresponds to the number of input samples, and sij is the number of core clusters. It is observed from Figure 2 that in the corresponding dataset, as the number of subspaces increases, so does the number of generated core clusters. For the image datasets (Figure 2(b)), even if the number of subspaces is set to k=25, the number of core clusters is still much smaller than the number of input samples, and this trend is most pronounced on the D6 and D8 datasets. For the gene express datasets (Figure 2(a)), due to the small number of input samples, when the number of subspaces is set to k=25, the number of generated core clusters is close to the number of input samples.
(2)
The influence of parametersk and r~ on clustering accuracyIn this section, we will compare the effect of the number of a random subspacek and the unsupervised feature selection ratio r~ on the accuracy of clustering ensemble. We set the value interval of parameter k to 5,25 and the increment to 5. For each random subspace, the unsupervised feature selection ratio r~ is set in range of 0.3,0.8, with increments set to 0.1. For each parameter setting of k and r~, we run the proposed methods 10 times, and the average of the NMI scores is shown in Figure 3.Figure 3
The parametersk and r~ correspond to clustering results (NMI).
(a)
D1(b)
D2(c)
D3(d)
D4(e)
D5(f)
D6(g)
D7(h)
D8As shown in Figure3, for the image datasets (D5-D7), parameters k and r~ are insensitive to clustering results. When r~ is fixed, the number of subspaces has little effect on the consensus clustering solutions. When k is fixed, the NMI score is relatively high in the range of 0.6,0.8. For the D1-D3 datasets, as shown in Figures 3(a)–3(c), the higher NMI scores are distributed in parameter k in an interval of 10,20 and parameter r~ in an interval of 0.5,0.8 region. For the D4 dataset, it can be clearly observed that when r~=0.8, the corresponding NMI score is higher.For each dataset, the more base subspaces is divided, the more adaptive core clusters are generated, which adds computational complexity and algorithm runtime. As observed from Figure2 that the number of core clusters generated by parameter k in range 10,20 is moderate. At the same time, when r=0.5 and r~ is set in the range of 0.5,0.8, the ensemble result has a higher NMI score. Therefore, in subsequent experiments, we set k=10, r=0.8, and r~=0.8 in experiments on all datasets.


### 4.4. Compare the Effects of Feature Selection and Weighted Strategies on Ensemble Results

In this section, we first compare the impact of the hybrid feature selection strategies on consensus clustering solutions. For each dataset, the proposed four ensemble methods are run on random subspaces and base subspaces generated by mixed feature selection strategies and compare their clustering results. For fair comparison, each ensemble approach runs 10 times in the unweighted manner, recording the average of its NMI scores. The comparison results are recorded in Table3, with the notation “N” corresponding to the clustering result on random subspace, and the notation “Y” corresponding to the result of clustering result on the base space.Table 3
Comparison of clustering results for random subspaces with clustering results for base subspaces (NMI).
DatasetsC2SWCE_HCC2SWCE_SCC2SWCE_BGC2SWCE_MCNYNYNYNYD10.66250.82700.63810.84490.78230.84360.80410.8808D20.33220.40810.35240.38450.43120.48990.52020.5302D30.41790.52090.23240.35330.35020.45980.41900.4948D40.75930.82570.82030.89550.82030.89550.94770.8203D50.43770.61520.51330.58150.43580.61180.43740.6123D60.81770.91130.76210.78180.80820.90060.80830.9104D70.47420.65960.53310.64120.53180.67040.49130.6402D80.74170.88810.75150.93240.76630.89670.78650.9015As shown in Table3, the clustering effect of the proposed clustering ensemble approaches on the base subspace is differently better than that in the random subspace. Therefore, in subsequent experiments, we use the hybrid feature selection strategy, perform unsupervised feature selection on random subspace, generate a set of base subspaces, and then generate ensemble members by performing the clustering on the base subspace.The proposed four ensemble approaches treat the core cluster as the base unit. The base subspace clustering solutions are weighted by calculating the distance between the core clusters pairs contained in the cluster, or the similarity of core clusters to each cluster. To verify the effect of the weighted clustering integration method, we compared the results of the weighted integration method and the unweighted integration method on the base subspace, respectively. Each ensemble method is run 10 times, and the average NMI of the proposed methods is recorded in Table4. In Table 4, the notation “N” corresponds to the unweighted clustering results of the proposed methods, and the notation “Y” corresponds to the weighted clustering results of the proposed methods.Table 4
Comparison of weighted ensemble manner and unweighted ensemble manner (NMI).
DatasetsC2SWCE_HCC2SWCE_SCC2SWCE_BGC2SWCE_MCYNYNYNYND10.94490.82700.89430.84490.88210.84360.88090.8808D20.53860.40810.56660.38450.56380.48990.52110.5302D30.58550.52090.49190.35330.55140.45980.50780.4948D40.91010.82570.89550.89550.89550.89550.82030.8203D50.66750.61520.58590.58150.56690.61180.54600.6123D60.92240.91130.87800.78180.92770.90060.89560.9104D70.75880.65960.65970.64120.68000.67040.64580.6402D80.90990.88810.94460.93240.93630.89670.90210.9015As observed from Table4, on the other datasets except D4, the fusion results of the proposed four ensemble methods under the weighted ensemble manner are better than the ensemble results of the unweighted ensemble manner. For the D4 dataset, the NMI score of the weighted ensemble manner of the proposed C2SWCE_HC is 0.9101, which is higher than the NMI score of 0.8257 corresponding to the unweighted ensemble manner. However, on the D4 dataset, there are no significant advantages in the clustering results obtained by the other three ensemble methods using a weighted ensemble manner. For other datasets, the consensus clustering results achieved by the weighted ensemble manner are better than those obtained by the unweighted ensemble manner. This illustrates the effectiveness of the proposed weighted ensemble strategies.


### 4.5. Comparison with Base Clustering

Subspace clustering ensemble can fuse multiple subspace clustering solutions to obtain a better accuracy, more robust consensus solution. In this section, we compare the clustering effects of the proposed locally weighted subspace clustering ensemble approaches, namely, C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC against spectral clustering approach. Each method runs 20 times, and their average NMI scores are described in Figure 4.Figure 4
Performance comparison of the proposed approaches with the base clustering.In Figure4, “base” corresponds to base clustering results, and “HC,” “SC,” “BG,” and “MC” correspond to the clustering results of C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC, respectively. As can be seen from Figure 4, on each experimental dataset, the proposed approaches can achieve better and more robust consensus clustering results than the spectral clustering approach. In particular, on the D1, D2, D3, D6, D7, and D8 datasets, the performance of the proposed 4 clustering integration approaches are significantly better than that of the spectral clustering approach.


### 4.6. Comparison with Other Clustering Methods

In this section, we compare the proposed consensus clustering approaches with the state-of-the-art clustering approaches to evaluate their effectiveness. Among comparative clustering approaches,K-means and GNMF [16] are classic clustering methods and SSC [17] and LSC [2] are the clustering methods based on spectral clustering. In the contrasting ensemble methods, MDEC_SC [4], MDEC_HC [4], and MDEC_BG [4] all use spectral clustering to generate the clustering ensemble members, and then, they use spectral clustering, hierarchical clustering, and bipartite graph partition to fuse base clustering solutions, respectively. SC_SRGF [5] adopts spectral clustering to obtain clustering solutions for each subspace; however, ECPCS-HC [18], ECPCS-MC [18], WEAC-AL [19], GP-MGLA [19], LWGP [20], and LWEA [20] all use K-means to generate base clustering solutions. All parameters of the comparison approaches are set as suggested by the corresponding papers.For a fair comparison, we run each clustering approach 20 times on each dataset and the average performance in terms of NMI and ACC are summarized in Tables5 and 6, respectively. In Tables 5 and 6, the highest score is highlighted in bold; the symbol “_” indicating that the algorithm cannot be performed.Table 5
Average performance in terms of NMI over 20 runs by different clustering methods.
D1D2D3D4D5D6D7D8K-means0.24810.22380.34870.52220.45750.74700.52190.7152GNMF0.59620.33690.34850.72230.55720.88030.58850.9216SSC0.29380.34610.59160.12760.42830.86400.60420.8640LSC____0.66460.88520.61890.9266MDEC_SC0.88310.49380.48430.73520.59220.88960.65620.9385SC_SRGF0.23710.19310.36210.58530.55240.88700.59470.8460C2SWCE_SC0.89430.56660.49190.89550.58590.87800.65970.9446ECPCS-HC0.00440.24830.51410.52640.51920.77510.63290.7976MDEC_HC0.88550.46610.52840.78530.65010.91770.70720.8822WEAC_AL0.29940.37640.28860.22400.49620.79210.59590.7009LWEA0.94470.25110.35330.59740.50810.76360.59990.7953C2SWCE_HC0.94490.53860.58550.91010.66750.92240.75880.9099MDEC_BG0.88630.55840.50790.69230.58820.91620.67750.9331GP_MGLA0.27940.35420.28470.21940.48080.79740.53150.6814LWGP0.86200.31520.36300.59740.52460.79240.61930.8736C2SWCE_BG0.88210.56380.55140.89550.56690.92770.68000.9363ECPCS-MC0.08160.31780.42560.56600.54510.80480.66000.8580C2SWCE_MC0.88090.52110.50780.82030.54600.89560.64580.9021Table 6
Average performance in terms of ACC over 20 runs by different clustering methods.
D1D2D3D4D5D6D7D8K-means0.81820.42860.56550.85970.42250.59440.04110.7334GNMF0.93950.55240.77830.94440.59310.81180.63900.9625SSC0.85080.55650.81770.65280.44180.78210.61090.8200LSC____0.65050.79720.63090.9675MDEC_SC0.98990.53950.73200.95830.50310.76980.02100.9713SC_SRGF0.77660.40970.54480.94580.49110.79900.06570.8465C2SWCE_SC0.98990.61090.74330.98610.54230.85830.01350.9793ECPCS-HC0.81610.47620.76950.89440.46920.63070.04850.8244MDEC_HC0.98990.55480.77440.96110.62290.85430.01570.8539WEAC_AL0.67740.50000.36930.26390.45090.70630.02320.6295LWEA0.99600.41940.54190.91670.52180.60760.03080.7820C2SWCE_HC0.98590.61090.61330.98610.54230.85830.01350.9693MDEC_BG0.98990.61290.74880.94170.49510.84600.02390.9682GP_MGLA0.70970.49600.35960.26390.43640.73540.01130.5565LWGP0.98790.48790.49750.91670.52550.67640.07970.9360C2SWCE_BC0.98990.65320.65520.98610.53410.83580.01350.9788ECPCS-MC0.83630.52580.63840.90690.51170.67290.02070.9260C2SWCE_MC0.98990.62100.74140.97220.46960.83330.05460.9115The landmark-based spectral clustering (LSC) [2] approach is not suitable for datasets with number of features greater than the number of samples, so there are no corresponding clustering results on D1-D4 datasets. As shown in Table 5, the SSC achieves the best average performance in terms of NMI on D3 dataset. For the remaining datasets, the clustering ensemble approaches yield better average performance than the traditional clustering approaches. This also confirms the previous conclusion that the clustering ensemble approaches are more suitable than the traditional clustering approaches for high-dimensional data clustering scenarios.As can be seen from Tables5 and 6, the proposed C2SWCE_HC achieves the highest average NMI scores on all datasets among the comparative hierarchical clustering ensemble approaches. Especially on the D1-D4 datasets, the performance of the C2SWCE_HC is significantly better than that of WEAC_AL and ECPCS-HC, and it also achieves the highest average ACC score on the D2, D4, D6, and D8 datasets. Comparing the ensemble approaches based on spectral clustering, the average performance in terms of NMI of C2SWCE_SC is significantly higher than that of other methods on the D1-D4 dataset. For example, on the D1 and D2 datasets, the average NMI scores of SC_SRGF are 0.2371 and 0.1931, respectively, while the average NMI scores of C2SWCE_SC are 0.8943 and 0.5666, respectively, which increased by about 4 times. In addition, compared with the other bipartite graph partitioning ensemble methods, the proposed C2SWCE_BG achieves the highest average NMI score on 6 datasets and the highest average ACC score on 5 datasets. Its performance is significantly better than MDEC_BG, GP_MGLA, and LWGP. In particular, on the D1, D3, and D4 datasets, the average NMI scores of the C2SWCE_BG are 0.8821, 0.5514, and 0.8955, respectively, significantly exceeding the corresponding NMI scores of 0.2794, 0.2847, and 0.2194 for the GP_MGLA. Finally, the metacluster-based ensemble clustering proposed in this paper is significantly better than that of the ECPCS-MC method on all datasets except D7.


### 4.7. Execution Time

In this section, we compare the execution times of different clustering ensemble approaches at the integration phase on the USPS dataset. The USPS dataset contains 10 categories with a total of 11,000 samples. To compare the clustering effects of different ensemble approaches on datasets of different sizes, first, according to the method described in Section4.1, samples of different proportions are randomly selected for each category of the input dataset, to generate the USPS dataset of different sizes. In the experiment, the sample sampling ratio ς is set in interval of 0,1, and the increment set to 0.1. That is, the generated dataset contains nς samples, where n is the number of samples in the USPS dataset. Running various ensemble approaches on the USPS datasets of different sizes, Figure 5 compares their execution time during the integration phase.Figure 5
Execution time of clustering ensemble approaches with different data sizes.As can be seen from Figure5, the execution time of all ensemble approaches increases significantly as the size of the USPS dataset increases, with SC_SRGF and MDEC_SC approaches increasing the fastest. When clustering all samples of the USPS dataset, the proposed C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC approaches are 195.23 s, 209.15 s, 220.72 s, and 231.65 s, respectively. Compared with other ensemble methods, the proposed methods have obvious advantages, especially the computational efficiency of C2SWCE_SC is higher than that of the contrasting spectral clustering ensemble methods MDEC_SC and SC_SRGF. The results show that the spectral clustering method based on the core clusters can reduce the complexity of the spectral clustering method to a certain extent. Further, compared with other hierarchical clustering ensemble methods, the execution time of C2SWCE_HC is comparable to that of the WEAC_AL, but much faster than that of ECPCS-HC and MDEC_HC. At the same time, we also observed that the execution time of the C2SWCE_BG is slightly higher than that of the GP_MGLA, but it has a significant advantage over MDEC_BG.In summary, the proposed approaches have a modest computational cost for ensemble tasks on USPS datasets of different sizes. Compared with the same type of ensemble approaches, the execution time of the proposed 4 approaches have advantages.


## 4.1. Datasets

In our experiments, we use eight high-dimensional datasets, which including four cancer gene expression datasets and four image datasets. The 4 gene expression datasets, namely, Yeoh02v11, Yeoh02v21, Bhattacharjee20011, and Golub1999v11. The Yeoh02v1 dataset and Yeoh02v2 dataset are the pediatric acute lymphoblastic leukemia dataset, in which Yeoh02v1 dataset contains 2 categories of genes expressed in leukemic blasts, and Yeoh02v2 dataset contains 6 categories of genes expressed in leukemic blasts. Bhattacharjee2001 is the lung tumor samples dataset, which contains 186 lung tumor samples and 17 normal lung tissues. Golub1999v1 is the leukemia dataset, which contains acute myeloid leukemia samples and acute lymphoblastic leukemia samples. The other 4 image datasets, including COIL_203, USPS2, Semeion4 and Multi Featureples4. The COIL_20 dataset is an image dataset containing 20 item objects, and the other 3 image datasets are all handwritten digit datasets. The USPS dataset contains a total of 10 categories and 11,000 samples. For facilitating comparison, we randomly selected 10% of the samples from each category of the USPS dataset to form a dataset containing 1,100 samples, which is represented as USPS_10P.To simplify the description, the 8 datasets will be abbreviated as D1 to D8, respectively. The details of the datasets are given in Table2.
(1)
https://schlieplab.org/Supplements/CompCancer/(2)
http://www.cad.zju.edu.cn/home/dengcai/Data/MLData.html(3)
https://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php(4)
http://archive.ics.uci.edu/ml/index.phpTable 2
Description of the datasets.
DatasetAbbreviationSampleDimensionClassYeoh02v1D124825262Yeoh02v2D224825266Bhattacharje2001D320315435Golub99v1D47218682USPS_10PD5110025610COIL_20D61440102420SemeionD7159325610Multiple featuresD8200064910


## 4.2. Evaluation Criterion

We use the normalized mutual information (NMI) and the clustering accuracy (ACC) to evaluate the quality of clustering results. The NMI metrics measure the accuracy of the clustering results according to the shared information of the ground-truth clustering solution and the test clustering solution. LetP denote the clustering solution of the proposed method, and PG denote the groundtruth clustering solution. The NMI score of P with respect to PG is calculated as
(35)NMIP,PG=∑i=1nP∑j=1nGnijlognijn/niPnjG∑i=1nPniPlogniP/n∑j=1nGnjGlognjG/n,where nP and nG are the number of clusters in P and PG, respectively. niP is the number of samples for the ith cluster in P, and njG is the number of samples for the jth cluster in PG; n is the number of input samples, and nij is the number of samples that the ith cluster in P and the jth cluster in PG jointly contain.The ACC measures the ratio of the number of samples that are correctly classified to the number of all samples. The ACC is an indicator that evaluates the accuracy of the clustering result, which is defined as(36)ACCP,PG=∑i=1nδPGxi,mapPxin,where PGxi is the groundtruth label corresponding to xi, and Pxi is the test clustering labels corresponding to xi in proposed approach. map⋅ is the relabel mapping function that aligns the test clustering label with the groundtruth label. δ⋅ is an indicator function, which is defined as
(37)δx,y=1,ifx=y,0,otherwise.The value interval for ACC and NMI score is0,1, and a higher ACC score or NMI score indicates a better clustering effect.


## 4.3. Discussion of the Parameter Selection

There are several parameters in proposed C2SWCE, where k is the number of subspaces, r is the random feature selection ratio, r~ is the unsupervised feature selection ratio, and K is the number of clusters. To increase the diversity of ensemble members, each subspace randomly generates a different number of clusters within range 2,n. We set the random feature selection ratio to r=0.5, and the number of generated random subspaces is in the range of 5,25, with the increment of 5. For each parameter setting, we run the spectral clustering methods 10 times in each random subspace, generating core clusters according to Definition 1 at each time. The average number of generated core clusters is shown in Figure 2.
(1)
The relationship betweenk and γFigure 2
The correspondence betweenk and γ (r=0.5).
(a)(b)In the proposed C2SWCE framework, the core cluster is the basic unit of integration.In Figure2, k is the number of generated random subspaces, k=0 corresponds to the number of input samples, and sij is the number of core clusters. It is observed from Figure 2 that in the corresponding dataset, as the number of subspaces increases, so does the number of generated core clusters. For the image datasets (Figure 2(b)), even if the number of subspaces is set to k=25, the number of core clusters is still much smaller than the number of input samples, and this trend is most pronounced on the D6 and D8 datasets. For the gene express datasets (Figure 2(a)), due to the small number of input samples, when the number of subspaces is set to k=25, the number of generated core clusters is close to the number of input samples.
(2)
The influence of parametersk and r~ on clustering accuracyIn this section, we will compare the effect of the number of a random subspacek and the unsupervised feature selection ratio r~ on the accuracy of clustering ensemble. We set the value interval of parameter k to 5,25 and the increment to 5. For each random subspace, the unsupervised feature selection ratio r~ is set in range of 0.3,0.8, with increments set to 0.1. For each parameter setting of k and r~, we run the proposed methods 10 times, and the average of the NMI scores is shown in Figure 3.Figure 3
The parametersk and r~ correspond to clustering results (NMI).
(a)
D1(b)
D2(c)
D3(d)
D4(e)
D5(f)
D6(g)
D7(h)
D8As shown in Figure3, for the image datasets (D5-D7), parameters k and r~ are insensitive to clustering results. When r~ is fixed, the number of subspaces has little effect on the consensus clustering solutions. When k is fixed, the NMI score is relatively high in the range of 0.6,0.8. For the D1-D3 datasets, as shown in Figures 3(a)–3(c), the higher NMI scores are distributed in parameter k in an interval of 10,20 and parameter r~ in an interval of 0.5,0.8 region. For the D4 dataset, it can be clearly observed that when r~=0.8, the corresponding NMI score is higher.For each dataset, the more base subspaces is divided, the more adaptive core clusters are generated, which adds computational complexity and algorithm runtime. As observed from Figure2 that the number of core clusters generated by parameter k in range 10,20 is moderate. At the same time, when r=0.5 and r~ is set in the range of 0.5,0.8, the ensemble result has a higher NMI score. Therefore, in subsequent experiments, we set k=10, r=0.8, and r~=0.8 in experiments on all datasets.


## 4.4. Compare the Effects of Feature Selection and Weighted Strategies on Ensemble Results

In this section, we first compare the impact of the hybrid feature selection strategies on consensus clustering solutions. For each dataset, the proposed four ensemble methods are run on random subspaces and base subspaces generated by mixed feature selection strategies and compare their clustering results. For fair comparison, each ensemble approach runs 10 times in the unweighted manner, recording the average of its NMI scores. The comparison results are recorded in Table3, with the notation “N” corresponding to the clustering result on random subspace, and the notation “Y” corresponding to the result of clustering result on the base space.Table 3
Comparison of clustering results for random subspaces with clustering results for base subspaces (NMI).
DatasetsC2SWCE_HCC2SWCE_SCC2SWCE_BGC2SWCE_MCNYNYNYNYD10.66250.82700.63810.84490.78230.84360.80410.8808D20.33220.40810.35240.38450.43120.48990.52020.5302D30.41790.52090.23240.35330.35020.45980.41900.4948D40.75930.82570.82030.89550.82030.89550.94770.8203D50.43770.61520.51330.58150.43580.61180.43740.6123D60.81770.91130.76210.78180.80820.90060.80830.9104D70.47420.65960.53310.64120.53180.67040.49130.6402D80.74170.88810.75150.93240.76630.89670.78650.9015As shown in Table3, the clustering effect of the proposed clustering ensemble approaches on the base subspace is differently better than that in the random subspace. Therefore, in subsequent experiments, we use the hybrid feature selection strategy, perform unsupervised feature selection on random subspace, generate a set of base subspaces, and then generate ensemble members by performing the clustering on the base subspace.The proposed four ensemble approaches treat the core cluster as the base unit. The base subspace clustering solutions are weighted by calculating the distance between the core clusters pairs contained in the cluster, or the similarity of core clusters to each cluster. To verify the effect of the weighted clustering integration method, we compared the results of the weighted integration method and the unweighted integration method on the base subspace, respectively. Each ensemble method is run 10 times, and the average NMI of the proposed methods is recorded in Table4. In Table 4, the notation “N” corresponds to the unweighted clustering results of the proposed methods, and the notation “Y” corresponds to the weighted clustering results of the proposed methods.Table 4
Comparison of weighted ensemble manner and unweighted ensemble manner (NMI).
DatasetsC2SWCE_HCC2SWCE_SCC2SWCE_BGC2SWCE_MCYNYNYNYND10.94490.82700.89430.84490.88210.84360.88090.8808D20.53860.40810.56660.38450.56380.48990.52110.5302D30.58550.52090.49190.35330.55140.45980.50780.4948D40.91010.82570.89550.89550.89550.89550.82030.8203D50.66750.61520.58590.58150.56690.61180.54600.6123D60.92240.91130.87800.78180.92770.90060.89560.9104D70.75880.65960.65970.64120.68000.67040.64580.6402D80.90990.88810.94460.93240.93630.89670.90210.9015As observed from Table4, on the other datasets except D4, the fusion results of the proposed four ensemble methods under the weighted ensemble manner are better than the ensemble results of the unweighted ensemble manner. For the D4 dataset, the NMI score of the weighted ensemble manner of the proposed C2SWCE_HC is 0.9101, which is higher than the NMI score of 0.8257 corresponding to the unweighted ensemble manner. However, on the D4 dataset, there are no significant advantages in the clustering results obtained by the other three ensemble methods using a weighted ensemble manner. For other datasets, the consensus clustering results achieved by the weighted ensemble manner are better than those obtained by the unweighted ensemble manner. This illustrates the effectiveness of the proposed weighted ensemble strategies.


## 4.5. Comparison with Base Clustering

Subspace clustering ensemble can fuse multiple subspace clustering solutions to obtain a better accuracy, more robust consensus solution. In this section, we compare the clustering effects of the proposed locally weighted subspace clustering ensemble approaches, namely, C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC against spectral clustering approach. Each method runs 20 times, and their average NMI scores are described in Figure 4.Figure 4
Performance comparison of the proposed approaches with the base clustering.In Figure4, “base” corresponds to base clustering results, and “HC,” “SC,” “BG,” and “MC” correspond to the clustering results of C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC, respectively. As can be seen from Figure 4, on each experimental dataset, the proposed approaches can achieve better and more robust consensus clustering results than the spectral clustering approach. In particular, on the D1, D2, D3, D6, D7, and D8 datasets, the performance of the proposed 4 clustering integration approaches are significantly better than that of the spectral clustering approach.


## 4.6. Comparison with Other Clustering Methods

In this section, we compare the proposed consensus clustering approaches with the state-of-the-art clustering approaches to evaluate their effectiveness. Among comparative clustering approaches,K-means and GNMF [16] are classic clustering methods and SSC [17] and LSC [2] are the clustering methods based on spectral clustering. In the contrasting ensemble methods, MDEC_SC [4], MDEC_HC [4], and MDEC_BG [4] all use spectral clustering to generate the clustering ensemble members, and then, they use spectral clustering, hierarchical clustering, and bipartite graph partition to fuse base clustering solutions, respectively. SC_SRGF [5] adopts spectral clustering to obtain clustering solutions for each subspace; however, ECPCS-HC [18], ECPCS-MC [18], WEAC-AL [19], GP-MGLA [19], LWGP [20], and LWEA [20] all use K-means to generate base clustering solutions. All parameters of the comparison approaches are set as suggested by the corresponding papers.For a fair comparison, we run each clustering approach 20 times on each dataset and the average performance in terms of NMI and ACC are summarized in Tables5 and 6, respectively. In Tables 5 and 6, the highest score is highlighted in bold; the symbol “_” indicating that the algorithm cannot be performed.Table 5
Average performance in terms of NMI over 20 runs by different clustering methods.
D1D2D3D4D5D6D7D8K-means0.24810.22380.34870.52220.45750.74700.52190.7152GNMF0.59620.33690.34850.72230.55720.88030.58850.9216SSC0.29380.34610.59160.12760.42830.86400.60420.8640LSC____0.66460.88520.61890.9266MDEC_SC0.88310.49380.48430.73520.59220.88960.65620.9385SC_SRGF0.23710.19310.36210.58530.55240.88700.59470.8460C2SWCE_SC0.89430.56660.49190.89550.58590.87800.65970.9446ECPCS-HC0.00440.24830.51410.52640.51920.77510.63290.7976MDEC_HC0.88550.46610.52840.78530.65010.91770.70720.8822WEAC_AL0.29940.37640.28860.22400.49620.79210.59590.7009LWEA0.94470.25110.35330.59740.50810.76360.59990.7953C2SWCE_HC0.94490.53860.58550.91010.66750.92240.75880.9099MDEC_BG0.88630.55840.50790.69230.58820.91620.67750.9331GP_MGLA0.27940.35420.28470.21940.48080.79740.53150.6814LWGP0.86200.31520.36300.59740.52460.79240.61930.8736C2SWCE_BG0.88210.56380.55140.89550.56690.92770.68000.9363ECPCS-MC0.08160.31780.42560.56600.54510.80480.66000.8580C2SWCE_MC0.88090.52110.50780.82030.54600.89560.64580.9021Table 6
Average performance in terms of ACC over 20 runs by different clustering methods.
D1D2D3D4D5D6D7D8K-means0.81820.42860.56550.85970.42250.59440.04110.7334GNMF0.93950.55240.77830.94440.59310.81180.63900.9625SSC0.85080.55650.81770.65280.44180.78210.61090.8200LSC____0.65050.79720.63090.9675MDEC_SC0.98990.53950.73200.95830.50310.76980.02100.9713SC_SRGF0.77660.40970.54480.94580.49110.79900.06570.8465C2SWCE_SC0.98990.61090.74330.98610.54230.85830.01350.9793ECPCS-HC0.81610.47620.76950.89440.46920.63070.04850.8244MDEC_HC0.98990.55480.77440.96110.62290.85430.01570.8539WEAC_AL0.67740.50000.36930.26390.45090.70630.02320.6295LWEA0.99600.41940.54190.91670.52180.60760.03080.7820C2SWCE_HC0.98590.61090.61330.98610.54230.85830.01350.9693MDEC_BG0.98990.61290.74880.94170.49510.84600.02390.9682GP_MGLA0.70970.49600.35960.26390.43640.73540.01130.5565LWGP0.98790.48790.49750.91670.52550.67640.07970.9360C2SWCE_BC0.98990.65320.65520.98610.53410.83580.01350.9788ECPCS-MC0.83630.52580.63840.90690.51170.67290.02070.9260C2SWCE_MC0.98990.62100.74140.97220.46960.83330.05460.9115The landmark-based spectral clustering (LSC) [2] approach is not suitable for datasets with number of features greater than the number of samples, so there are no corresponding clustering results on D1-D4 datasets. As shown in Table 5, the SSC achieves the best average performance in terms of NMI on D3 dataset. For the remaining datasets, the clustering ensemble approaches yield better average performance than the traditional clustering approaches. This also confirms the previous conclusion that the clustering ensemble approaches are more suitable than the traditional clustering approaches for high-dimensional data clustering scenarios.As can be seen from Tables5 and 6, the proposed C2SWCE_HC achieves the highest average NMI scores on all datasets among the comparative hierarchical clustering ensemble approaches. Especially on the D1-D4 datasets, the performance of the C2SWCE_HC is significantly better than that of WEAC_AL and ECPCS-HC, and it also achieves the highest average ACC score on the D2, D4, D6, and D8 datasets. Comparing the ensemble approaches based on spectral clustering, the average performance in terms of NMI of C2SWCE_SC is significantly higher than that of other methods on the D1-D4 dataset. For example, on the D1 and D2 datasets, the average NMI scores of SC_SRGF are 0.2371 and 0.1931, respectively, while the average NMI scores of C2SWCE_SC are 0.8943 and 0.5666, respectively, which increased by about 4 times. In addition, compared with the other bipartite graph partitioning ensemble methods, the proposed C2SWCE_BG achieves the highest average NMI score on 6 datasets and the highest average ACC score on 5 datasets. Its performance is significantly better than MDEC_BG, GP_MGLA, and LWGP. In particular, on the D1, D3, and D4 datasets, the average NMI scores of the C2SWCE_BG are 0.8821, 0.5514, and 0.8955, respectively, significantly exceeding the corresponding NMI scores of 0.2794, 0.2847, and 0.2194 for the GP_MGLA. Finally, the metacluster-based ensemble clustering proposed in this paper is significantly better than that of the ECPCS-MC method on all datasets except D7.


## 4.7. Execution Time

In this section, we compare the execution times of different clustering ensemble approaches at the integration phase on the USPS dataset. The USPS dataset contains 10 categories with a total of 11,000 samples. To compare the clustering effects of different ensemble approaches on datasets of different sizes, first, according to the method described in Section4.1, samples of different proportions are randomly selected for each category of the input dataset, to generate the USPS dataset of different sizes. In the experiment, the sample sampling ratio ς is set in interval of 0,1, and the increment set to 0.1. That is, the generated dataset contains nς samples, where n is the number of samples in the USPS dataset. Running various ensemble approaches on the USPS datasets of different sizes, Figure 5 compares their execution time during the integration phase.Figure 5
Execution time of clustering ensemble approaches with different data sizes.As can be seen from Figure5, the execution time of all ensemble approaches increases significantly as the size of the USPS dataset increases, with SC_SRGF and MDEC_SC approaches increasing the fastest. When clustering all samples of the USPS dataset, the proposed C2SWCE_HC, C2SWCE_SC, C2SWCE_BG, and C2SWCE_MC approaches are 195.23 s, 209.15 s, 220.72 s, and 231.65 s, respectively. Compared with other ensemble methods, the proposed methods have obvious advantages, especially the computational efficiency of C2SWCE_SC is higher than that of the contrasting spectral clustering ensemble methods MDEC_SC and SC_SRGF. The results show that the spectral clustering method based on the core clusters can reduce the complexity of the spectral clustering method to a certain extent. Further, compared with other hierarchical clustering ensemble methods, the execution time of C2SWCE_HC is comparable to that of the WEAC_AL, but much faster than that of ECPCS-HC and MDEC_HC. At the same time, we also observed that the execution time of the C2SWCE_BG is slightly higher than that of the GP_MGLA, but it has a significant advantage over MDEC_BG.In summary, the proposed approaches have a modest computational cost for ensemble tasks on USPS datasets of different sizes. Compared with the same type of ensemble approaches, the execution time of the proposed 4 approaches have advantages.


## 5. Conclusion

In this paper, we propose a novel locally weighted subspace clustering ensemble framework, termed C2SWCE. It first uses the hybrid feature selection strategy that combines random feature selection and unsupervised feature selection to generate a set of base subspaces. The strategy combines the diversity of random feature selection to select representative features from each random subspace in an unsupervised manner, continuously reducing the dimensionality of the random subspace. To increase the diversity of subspace ensemble members, in addition to using random feature selection to generate feature subspaces, we also randomly group the samples into different numbers of clusters. Furthermore, we introduce concept of the core cluster. In the ensemble process, the core cluster is viewed as the base unit, which improves the ensemble efficiency to a certain extent. The subspace clustering solution is weighted by evaluating the stability of the cluster. Last but not least, under the proposed framework, four weighted ensemble approaches are proposed to integrate the clustering solutions of the base subspace to achieve the final clustering result. Extensive experiments are conducted on 8 real-world datasets to verify the effectiveness of the proposed ensemble approaches. Experimental results show that compared with the state-of-the-art ensemble methods, our methods have stronger robustness, and the comprehensive performance of clustering accuracy and efficiency has advantages.



---

*Source: 7990969-2022-05-24.xml*