Internal Purity: A Differential Entropy based Internal Validation Index for Clustering Validation
Authors: Bin Cao
Chen Yang
Kaibo He
JING FAN
Venue: Submitted to ICLR 2023 (2023)

Abstract: In an effective process of cluster analysis, it is indispensable to validate the goodness of different partitions after clustering. Existing internal validation indices are implemented based on distance, variance and model-selection. The indices based on distance or variance cannnot catpure the real ``density" of the cluster and the time complexity for distance based indices is usually too high to be applied for large datasets. Moreover, the indices based on model-selection tend to overestimate the number of cluster in clustering validation. Therefore, we propose a novel internal validation index based on the differential entropy, named \textit{internal purity} (IP). The proposed IP index can effectively measure the purity of a cluster without using the external cluster information, and successfully overcome the drawbacks of existing internal indices. Based on six powerful deep pre-trained models and without further fine-tuning using the experimental datasets, we use four different clustering algorithms to compare our index with thirteen other well-known internal indices on five text and five image datasets. The results show that, for 60 test cases in total, our IP index can return the optimal clustering results in 43 cases while the second best index can merely report the optimal partition in 17 cases, which demonstrates the significant superiority of our IP index when validating the goodness of the clustering results. Moreover, theoretical analysis for the effectiveness and efficiency of the proposed index are also provided.

Under review as a conference paper at ICLR 2023
INTERNAL PURITY: A DIFFERENTIAL ENTROPY BASED INTERNAL VALIDATION INDEX FOR CLUSTER- ING VALIDATION
Anonymous authors Paper under double-bling blind review
ABSTRACT
an effective process of cluster analysis, is indispensable to validate the good- ness of different partitions after clustering. Existing internal validation indices are implemented based on distance and variance, which cannnot catpure the real "density" of the cluster. Moreover the time complexity for distance based indices is usually too high to be applied for large datasets. Therefore, we propose a novel internal validation index based the differential entropy, named internal purity (IP). The proposed IP index can effectively measure the purity a cluster without using the external cluster information, and successfully overcome the drawbacks of existing internal indices. Based on six powerful deep pre-trained representation models, we use four basic clustering algorithms teen other well well-known internal indices on five text and five image datasets. The results show that, for 90 test cases in total, our IP index can return the optimal clustering results in 51 cases while the second best index can merely report the optimal partition in cases, which demonstrates the significant superiority of our IP index when validating the goodness of the clustering results. Moreover, theoretical analysis for the effectiveness and efficiency of the proposed index are also provided.
compare our index with thir-
INTRODUCTION
The goal of clustering is to divide of samples into different clusters such that similar samples are grouped in the same clusters. As one of the most fundamental tasks in machine learning, clustering has been extensively studied in many fields, such as text mining (Guan et al. 2012a). image analysis (Zhou et al., 2011) and pattern recognition (Guan et al. (011b)). With the advance of deep learning, has been proved that running any classical clustering algorithm (e.g., K-means) over the learned
representation can yield better results (Xie et al.) [0166] Huang et al. 2020 Dang et al.) (021). The main reason behind this is that the deep neural networks can effectively extract highly non-linear features that are helpful for clustering.
However, besides the data representation, the outcome of clustering is still affected by other factors (Xu & Wunsch, 2005 Yang al., (017). For example, different clustering algorithms usually lead to different clustering results in a specific dataset. Even for the same algorithm, the selection of different parameters may affect the final clustering results (Halkidi al., 2000). Thus, within an effective process of cluster analysis, it is inevitable to validate the goodness of different partitions after clustering and select the best one for application. Here the best one refers to not only the proper parameters but also the best partition that fits the underlying data structure. In fact, many clustering validation measures have been proposed over past years and they can categorized to external validation and internal validation (Wu et al. 2009 Liu et al., 2013). Specifically, external validation indices assume the "true" cluster information known in advance, and they use the supervised information quantify how good is the obtained partition with respect to prior ground truth clusters. However, such prior knowledge is rarely available many real scenarios. Then, internal validation indices become the only option for evaluating the clustering result.
Internal clustering validation usually measures the clustering result based on following two criteria (Liu et al. 2013 Fraley & Raftery 1998): (1) compactness, which measures how closely related

---

Under review as a conference paper at ICLR 2023
A dustar, center dtster center
Cluster
Cluster
(a) Based on distance
(b) Based on variance
Figure 1: Drawback illustration for computing compactness based on distance and variance
the samples within the cluster, and (2) separation, which measures how clusters separated from each other. In general, distance and variance are two main strategies to implement compact- ness and separation. However, validation indices based on these implementations suffers following drawbacks that limit their performance.
First, for distance based index, given two clusters, the same distance computation result cannot guarantee the same compactness. We use the example shown in Fig T(a) for explanation. Partic- ularly, two clusters are grouped into two cubes respectively and each cube represents the volume of the corresponding vector space. Suppose the volume of two cluster vector spaces are the same and the average pairwise distance of each cluster also the same, measures like Silhouette index (Rousseeuw) 1987) will consider they have the same compactness. However, from the density per- spective, the left cluster should be tighter than the right one. Though indices like S S_Dbw (Halkidi) & Vazirgiannis, 2001), DBCV (Moulavi et al. 2014) and DCVI (Xie et al. 2020) also propose the density related concept, the density is still calculated based on the distance instead the volume of vector space. Then, many existing indices require computing pairwise distances for compactness or separation, which would be prohibitively tone-conuuning for a large nigh-dimensional data set (Ceng et al. 2018).
Second, variance based compactness computation usually view lower variance as the indicator of higher tightness. However, this is misleading in some cases. As shown in Fig. [ we easily observe that cluster B is more compact than cluster A. But the variances for both clusters are the same, which makes measures like standard deviation index (SD) (Halkidi al. 2000) fail provide reliable validation. In fact, covariance here is more suitable to measure the compactness, i.e., the covariances among pairs of variables in cluster smaller than that of cluster B.
Therefore, we dedicate this paper to novel internal validation index, named internal purity (IP). Here the purity refers to how "pure" the semantic of set of samples are. For example, a cluster of similar texts describe a specific event. Hence, from the perspective of compactness, we want cluster to be as pure as possible, while for the separation perspective, a clustering partition of lower purity is favored. To evaluate the purity, we apply the idea of information entropy (Shannon, 1948) and more specifically, the differential entropy (Cover & Thomas 199T) is used for evaluation due to that the embedding variables are usually continuous in deep clustering. Furthermore, using differential entropy can help us to overcome the aforementioned drawbacks of existing measures. First of all, unlike the average pairwise distance, the nature of the entropy lends itself to measuring the "information density" (Zu et al.) 2020) of a vector space (Cheng et al. 1999). Then, theoretical studies have shown that computation of differential entropy actually considers the variance of each variable and the covariance between variables (Johnson et al. 2014), which makes it more effectively measure the compactness of cluster. Last but not least, since the differential entropy computation requires merely one iteration over the whole cluster, it needs less computation time compared with computing average pairwise distance.
In fact, from the perspective of information theory, there is another information criterion (Bshhop & Nasrabadi 2006) that can be used for evaluating clustering partitions (Akogul & Erisoglu 2016). i.e., estimating the quantity of information loss based on model performance and complexity. where model performance can be evaluated by using a probalilistic framework, such as log-likelihood and model complexity by the number of parameters in the model (Akogul & Erisoglu 2016). Akaike information criterion (AIC) (Aaikee 1974) and the Bayesian information criterion (BIC) (Schwarz, 1978) are two indices of this criterion. As will be shown later in the paper, IP index can actually be regarded as a form of such criterion through mathematical derivation. But different from AIC and BIC, our IP index also takes into account the traditional internal clustering criteria, i.e. compactness
2

---

Under review as a conference paper at ICLR 2023
and separation, whereas AIC and BIC not. Moreover, some existing works have shown that AIC and BIC indices have poor performance and are prone to overestimate the number of clusters in dataset (Windham & Cutler 1992, Hu & Xu 2004). Hence, our IP index is more competitive for evaluating the clustering results based on its capability to capture different criteria.
Note that, although the term "purity" has been used in external validation (Wu et al. 2009) and the entropy has also been used as the measurement, the definition as well as the computation of our internal purity is quite different since no supervised information is known in advance in our validation setting. To summarize, our contributions are threefold:
1. To tackle the effectiveness and efficiency problems of existing measures, we propose to use differential entropy to measure the purity of a cluster as well as a partition.
2. Following the traditional perspective of compactness and separation, we design a new in- ternal validation index based on the nature of the proposed purity measure. Moreover, theoretical analysis for the effectiveness and efficiency of the proposed internal purity is also provided.
3. Based on six powerful deep pre-trained models, we use four basic clustering algorithms to compare our index with thirteen other well-known internal indices on five text and five image datasets. The results show that for 90 test cases, our IP index can return the optimal clustering results 51 cases while the second best index can merely report the optimal partition in 30 cases.
The remainder of this paper is organized as follows. Section 2 briefly reviews the existing internal indices. Section 3 introduces preliminaries. Section presents our internal purity index. Section reports the results of experimental evaluation. Finally, Section 6 concludes the paper.
RELATED WORK
Table 1: The description of well known internal clustering validation indices.
Measure
Notation
Description
Root-mean-square
standard
RMSSTD RMSSTD cluster square root of variance attributes.
deviation
Calinski-Harabasz index
CH
CH is the ratio of the sum of betweco-clusters seatter and of within-cluster scatter for all clusters.
T index
I
compactness by calculating the distance from the samples in cluster their cluster center and the separation based on the maximum distance between cluster centers.
Dunn index
Dunn
Dunn uses the farthest distance hetween samples in cluster as the compactness and uses the distance between the nearest samples in different clusters the sepuration.
Silhouette index
8
measures compactness hased the pairwise distance in cluster and separation based the average distance belween a sample and all other samples in the next nearest cluster.
Davies Bouldin index
DB
DB measures compactness by calculating the average distance between samples in a cluster their cluster center and the separation based the distance between cluster centers.
Xie- Beni index
XB
XB delines compactness average center based distance and separation the minimal squared distances between cluster centers.
Standard deviation index
SD measures compactness hased variances samples cluster, and separation hased on distances between cluster centers.
S_Dbw validation index
S_Dbw
S. Dhw measures compactness based on variances of samples in a cluster, and separation based average density among clusters.
Clustering validation index
CVNN
CVNN measures compuctness based on pairwise distance in on the idea of i- k-nearest neighbor (kNN).
and separation
bused on nearest neighbors
Density-Bann Bused Validation
Clustering
DBCV
DBCV measures compuctness based on maxinum edge weight of the minimumn spanning tree seachability distance between nodes diffcrent clusters. measures compactness based maximum weight value minimum spanning tree and separation based on the minimum value of minimum distanecs berween samples in different clusters.
soparation based
Density core based clustering
DCVI
validation index
The internal clustering validation indices measure the quality of clustering by the internal informa- tion of the dataset and no other external information is used. We group the internal indices to two categories based on how the criteria of compactness and separation is calculated, i.e., distance-basee and sariance-based We briefly summarize several well known indices in Table
Distance-based. Indices in this category either use pairwise distance or cluster center based dis- tance measure the compactness and the separation. DB (Davies & Bouldin 1979), I (Maulik &
3

---

Under review as a conference paper at ICLR 2023
Bandyopadhyay 2002) and XB (Xie & Beni 1991) measure compactness based average center- based distance. Dunn [unnn 1973) measures compactness based on maximum pairwise distance. DBCV and DCVI select the most representative pairwise distance for measuring the compactness. However, using center based or a single pairwise distance in the cluster to represent the compactness for the entire cluster cannot provide stable performance (Liu al. 2013). Our IP index uses all the samples in the cluster which is more stable, Although S and CVNN can use all samples in the cluster by computing average pairwise distance, they have high time complexity. Moreover, mentioned in Section | these average pairwise distance-basc indices cannot correctly reflect the density of vector space.
Variance-based. Indices in this category assume lower variance indicates better compactness. RMSSTD (Halkidi et al. 2001). CH (Calinski Harabasz 1974), SD and Dbw measure com- pactness based on variances of samples in a cluster. As a representative, CH further measures the separation by computing the between-cluster variance based on cluster centroids. However, these variance-based indices not good measures since two clusters with same variance may have dis- tinct density.
PRELIMINARIES
In this section, we first present the concept of entropy and differential entropy. Then we introduce the computation of differential entropy for multivariate normal distribution. Note that, though the normal distribution assumption made here seems restrictive, it still works for datasets that deviate from the assumption long as we use their deep representations. This is because the distribution of deep representation has been proved to be close to the normal distribution (Lee et al. 2018 Daneshmand et al. 202T)
3.1 ENTROPY AND DIFFERENTIAL ENTROPY
In information theory, the entropy of a random variable is the average level of "surprise" or "un- certainty" inherent in the variable's possible outcomes (Shannon 1948). other words, it can be used to measure the uncertainty of data. Entropy includes two classes: entropy for discrete random variables and entropy for continuous random variables. i.e., differential entropy.
Given a discrete random variable V, with possible outcomes Ui,..., Un, we assume the probability of V being U is P.. The entropy V is formally defined as (Shannon 1948):
Entropy(V) = Σ
P. log P
(1)
Given a continuous random variable V, with a probability density function p(u). The diffcrential entropy is defined (Cover & Thomas 1991):
+00
Dif fiftEnnroyy(V)
p(u) log
3.2 DIFFERENTIAL ENTROPY OF THE MULTIVARIATE NORMAL DISTRIBUTION
The multivariate normal distribution of a d-dimensional random vector H (H1... ..... Hd) can be written the following form (Goodfellow et al. 2016):
H Na(u Σ)
(3)
with d-dimensional mean vector
μ B[H] (E
[H E [12]]
E [1
and d x covariance matrix
Σ E E HH' ) H 1²) Cov H H
(5)
such that 1 < i, normal distribution is given by (Ahmed & Gokhale 1989):
أ < d, and Σ a positive definite matrix. The differential entropy of multivariate
rank Σ
rank Σ)
DiffEnnroyy(H)
In ln(2 + 2
Σ
(6)
2
2
4

---

Under review as a conference paper at ICLR 2023
where, rank(E) is the rank of Σ and |Σ] is the determinant of covariance matrix. If the covari- ance matrix Σ is not full rank, the multivariate normal distribution is degenerate (Rao et al. 1973). Then, the rank E) < d and the determinant of covariance matrix is degenerated as the pseudo- determinant.
INTERNAL PURITY
In this section, first present the implementation details for our proposed internal purity (IP) index, and then provide theoretical analysis for its effectiveness and efficiency.
4.1 IP IMPLEMENTATION
Following the traditional criteria of internal clustering validation, IP index consists of following two main components, i.e., compactness purity (CP) and seperation purity (SP). 'he former measures the average differential entropy of clusters judge the compactness of the clustering result, while the latter evaluates the separation between clusters based on the differential entropy of the space formed by the center of each cluster.
Compactness, Let H be the feature space, where H = [hi, ] clustered into & clusters, i.e. H1, ...,... Hg, the compactness CP for l clusters is defined average differential entropy of & clusters:
} Supposing that N samples
CP E DiJ/Enlroyy(HH) 1=1
Given a specific dataset, we can get feature representation of each sample z; through the pre- trained deep model f(-), i.e. h; = fm;z) Then, for k-th cluster we can get h....,,,, E Hi. where m is the number of data samples in the k-th cluster. So HE m x matrix composed of feature representations corresponding to samples in the Ha. Hę can be viewed
set of points in d-dimensional space. Moreover, we usually assume feature matrix H₃ follows multivariate normal distribution when have prior knowledge of the distribution of these points (Goodfellow al,, 2016) and existing works have demonstrated that the distribution of the hidden representation is close to the normal distribution (Lee et al.) 2018 Daneshmand al. 202T) Hence we obtain the differential entropy of a cluster H4 by Eq. (6)
Separation. Let μa be the centroid of the k-th cluster Hg, i.e., P4 EE hi, h Hg. where m is the number of data samples in the k-th cluster.. The separation S for k clusters is defined as differential entropy the feature matrix H2 formed by cluster centers
SP Dif D--fiffEnrrrppn Entropy(H..)
where H, = [MI, .... Mk }T Here we also assume that the centers of clusters follow multivariate normal distribution. Hence, we can obtain differential entropy of this distribution by Eq. (6)
IP Index. Based on CP and SP, the IP index for a clustering result of & clusters is defined as
IP
As shown above, our IP index takes the form of subtracting the intercluster separation from the intracluster compactness. A lower value of TP indicates a better clustering result.
4.2 THEORETICAL ANALYSIS
Effectiveness. According to Eq. (6), we can see that the differential entropy of multivariate normal distribution is proportional to the determinant of the covariance matrix. The determinant of the covariance matrix is usually called generalized variance (Johnson et al., (2014). For a fixed set of data, generalized variance is proportional to the square of the volume generated by the d deviation vectors (Johnson et al.) 2014). i.e.,
Generalized variance
Σ (N 11) volume )2
(10)
5

---

Under review as a conference paper at ICLR 2023
Based on TQ), know the reason why internal purity can avoid the drawbacks of distance or variance based indices. Specifically, the form volume embedded in the Eq. (10) indicates the real density for given vector space. Then, according to Eq. ( formula (detailed derivation is listed in Appendix Ap:
N
we can further obtain following
1 IP
+ In (2T)
1
In E In Epl
+
rank rank ppp
(11)
2
2
part.1
part.2
This formula is similar to the information criterion (Bishop & Nasrabadi 2006 mentioned in the introduction. According to this criterion, the part_l of Eq. (T1) can be considered as the evaluation of the model performance and the part 2 as the measure for the model complexity. This formula also indicates that our proposed IP index can also capture what traditional information criterion based indices want measure.
Efficiency. The time complexity of IP computation is decided by the complexity of both compact- ness and separation. For compactness, the time complexity of calcualting the covariance matrix is
( (Cxl) and the determinant of covariance matrix complexity is (rf2.76) (Aho & Hoperoft 1974.. Hence, the complexity of calcualting the compactness of cluster is O O(F + d=.sr), ), then the complexity for & clusters is O(Nd + kdd.376). For seperation, we have to calculate k cluster centroids, and the covariance matrix formed by cluster centroids and the corresponding determinant. So the complexity of seperation is (Nd + kd + d2.370). Usually, d < N and k < N, then, the complexity of IP index O(Nd)), which makes it affordable for large-scale and
high-dimensional datasets.
5 EXPERIMENTS
In this section, compare the performance IP with other 13 well-known internal indices, namely, SD, Dunn, I, XB, S, CH, DB, AIC, BIC, S.Dbw, CVNN, DBCV and DCVI. Note that, we didn't consider RMSSTD since it need subjective determination for the shift point of its curve (Halkıdi eet al., 2001 Vendramin et al.] 2010). Similar to existing works (Halkidi et al.] 2000) Liu et al. 2013 Moulavi et al.) 2014), we the task of determining optimal cluster number for evaluation purpose. The general evaluation procedure is as follows: (1) we first use the existing pre-trained deep model to transfer the dataset to representation matrix; (2) set of clustering algorithms are then applied to the representation matrix and different clustering partitions can be obtained with different parameters; (3) finally, we compute the internal index for each partition and get the best partition as well as its corresponding optimal cluster number.
5.1 SETTINGS
Datasets. We use five text datasets and five image datasets. The statistics of these datasets are shown in Table 2 and 3 respectively. For text datasets usages, we cluster the train sets of SearchSnippets (Phan et al. 2008). Biomedical (Xu al. 2017), StackOverflow (Xu et al. (017)) and WebofScience- 11967 (Kowsari et al. 2017). and 10,000 randomly selected texts from each class YahoolAnswers
/Zhang et al.) 2015). For image datasets usages, we cluster the test sets of CIFAR-10 (Krizhevsky al. 2009). MINST (LeCun
al. 1998) and FashionMNIST (Xiao al. 2017). and 10,000 randomly selected imagess in the test set of CINIC-10 (Darlow et al. 2018). For ImageNet- 10
(Chang et al. 2017). the train set is directly used. Moreover, we also use five real-world datasets
from UCI Machine Learning Repository (Frank) (010)) for evaluating how our IP index performs over dataset without deep representation, and details can seen Appendix F
Evaluation Metrics. Since the information of true clusters are known for above datasets, i.e., classes are given, we can use external indices to evaluate the best clustering result each internal index selected and then further judge which internal index is better. Three external indices are used in our experiments, i..., Accuracy (ACC) (Wu Schölkopi] 2006). Adjusted Rand Index (ARI) (Hubert & Arabie 1985) and Normalized Mutual Information (NMI) (Chen et al. 2010). Larger ACC, ARI and NMI indicate better clustering result. Details of the implementation of the three external indices can be seen in Appendix B

---

Under review as a conference paper at ICLR 2023
following six pre- trained models. For text datasets, Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al. 2201)) Sentence-BERT (SBERT) (Reimers & Gurevych 2019) and Simple contrastive sentence embedding framework (SimCSE) Gao et al. 2021) are used. We use the average of all output embeddings as the feature representation of each text sentence the three models. For image datasets, we use Vision Transformer (ViT) (Dosovitskiy et al. 2021), Swin Transformer (Swin) (Liu cct al. 2021) (021) and Bidirectional Encoder Representations from Image Transformers (BEiT) (Bao al., 2021]. The generation of feature representations in different image models follows the suggestion by their original work. Note that since the unsupervised nature of clustering, we don't 't further fine-tune above pre-trained models with the experimental datasets.
Pre-trained Representation Models. To extract feature representations,
Experimental Setup. Above models are implemented based on Huggingface's transformers (Wolf et al. 2020) and Sentence Transformers (Reimers & Gurevych 2019). Detailed model configura- tions are listed Appendix|C] Our internal index is implemented based on scikit-learn (Pedregosa et al. 2011) and SciPy (Virtanen et al., 2020). Then based on above setup, we use four different clus- tering algorithms for generating partitions, including K K-Means, GMM, agglomerative hierarchical clustering (AHC) (Ward Jr 1963) and density- based spatial clustering of applications with noise (DBSCAN) (Ester et al. 1996). In the experiments, the numbers of clusters or components are in the search range from 2 to [VN] (Pal & Bezdek 1995) (Bezdek & Pal 1998) for K-Means, GMM and AHC. Two parameters
needed for DBSCAN, min_samples and eps. For min_samples, we choose min mrin_samples E [5, 100] and step size is 5, For eps, we obtain the minimum and max- imum values of pairwise distances for each dataset and employ 50 different values of eps equally distributed within this range. Morever, since the AIC and BIC are implemented based on the current clustering model, within-eluster-ee wluster-suu----ooooeeeee squares is applied in K K-Means, AHC and DBSCAN (Man- ning 2008), maximum likelihood function applied in GMM. The experimental results are averaged over five random runs for each validation index.
Table 2: Statistics of text datasets.
Table Statistics of image datasets.
Dataset
Split Samples
Classes
Dataset CIFAR-I0
Split
Samples
Classes
SearchSnippets
Train 12340
Train+Teest 17:000+10,000
10
Biomedical
Train
20000
MNIST Train+Test 60,000+10,000
StackOverllow
Train
20000
20
FustionMINIST
Teain+Test 60,000+ 10,000
10
WebofScience- 1 11967
Train
11967
ImageNet-10
Train 13000
10
Answers
Train
1,400,000
10
CINIC-10
Teain+Test 90,000+90,000
10
5.2 EFFECTIVENESS STUDY BASED ON EXTERNAL EVALUATION
The evaluation results based on external indices of ACC, ARI and NMI are shown in Table 5 6
9 and 10 where the best results are highlighted in bold and the optimal each index select, L.. opts, and the true cluster number in each dataset, i.c. dataset-k, are provided. Moreover, we also count the number of best results that each internal index can achieve and report them in Table Obviously, compared with other 13 indices, our IP index has significant advance in achieving the best results. Specifically, for all 90 cases, our IP index can produce 51 best results, while the second top index AIC only has 30 best results. Finally, IP index outperforms other indices greatly with respect to different domains, i.e., text and image. Due to space limitations, every clustering algorithm evaluation results can be seen in the Appendix Dreepeeciiie....eeeell....
Table 4: Number of best clustering results based counting over Table
8
9
and
10
SD Dunn
1
XB S
CH DB S Dbw CVNN DCVI
DBCV
AIC
BIC
IP
Text datasets
0
0
0
0
3
0
0
0
0
0
0
17
0
26
Image datasets
0
0
0
0
0
7
0
0
0
0
0
13
3
25
All datasets
0
0
0
3
7
0
0
0
0
30
3
51
Text datasets. In Table 5 we can also observe that our IP index outperforms other indices except StackOverflow and Yahoo!Answers. In Table 6 the index has the best results in three cases. However, it is distance based method which has higher time complexity than our IP index. In Table 7. although the index AIC has the results in nine cases, the optimal & value selected by it is far from the true number of classes Biomedical and Yahoo!Answers.
Image datasets. In Table 8. CH obtains three best cases in terms of ACC, ARI and NMI on ImageNet- 10, respectively. However shown in Table 9 where Swin is the representation model,
7

---

Under review as a conference paper at ICLR 2023
Table 5: BERT based clustering results on five text datasets. (15 cases: 5 datasets x 3 evaluation metrics)
SearchSSnippess 8
Biomedical- 20
StackOverflow
20
WebofScience 7
YahoolAnswers 10
ACC
ARI
NM1
opla
ACC
ARI
NMI
opta
ACC
ARI
NMI
opte
ACC
ARI
NMI
opi,
ACC
ARI
NMI
opla.
SD
21.56
0.01
2
5
0
0.01
2
5.01
0.01
2
17.6
0
0.02
10.01
0.02
Dunn
21.56
0.01
2
5
0
0.01
2
5.01
0.01
2
17.61
0
0.02
2
10.04
0.05
21.56
0.01
2
5
0
0.01
2
5.01
0.01
2
17.6
0
0.02
10.01
0.02
XB
21.56
0.01
2
5
0
0.01
2
5.01
0.01
2
17.6
0
0.02
10.01
0.02
s 21.56
0.01
5
0
0.01
2
5.01
0.01
2
17.6
0.02
10.01
0.02
CH
33.63
12.22
16.68
9.66
2.52
8.21
2
6.29
0.17
0.53
2
29.39
17.99
28.63
12.9
0.7
1.85
DB 21.56
5
5.01
0.01
2
17.6
0.02
0.02
5.Dbw
21.66
0.01
5
0.01
5.01
0.01
2
17.6
0.02
10.05
0.07
CVNN
21.67
0.01
1.01
5.34
0
0.66
2
5.62
0.02
0.43
2
17.85
0.02
0.46
10.04
0.05
DCVI 21.56
0.01
5
0.01
5.01
0.01
0.02
10.01
0.02
DBCV 23.41 0.5
8.28
0.74
3.87
5.95
0.58
19.43
0.14
0.8
10.01
0.02
AIC 27.84
18.61
37.90
25
24.50
13.88
28,20
40
22.34
22.61
24.69
36
29.52
19.31
BIC 34.65
9.82
2.94
2
6.32
0.46
29.42
18.43
29.34
12.23
1.01
54.18
33.58
31.96
15.86
27.29
16
5.38
0.39
47.93
31.90
40.83
6.90
Table 6: SBERT based clustering results on five text datascts. (15 cases: 5 datasets x 3 evaluation metrics)
SearchSnippels 8
Biomedicaal - 200
StackOverllow 20
WebufScience 7
YahootAmswers 10
ACC
ART
NMI
opt,
ACC
ARI
NMI
opl,
ACC
ARI
NMI
opty
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
SD
21.89
0.06
1.26
13
6.24
0.03
3.8
38
5,11
0
0.38
7
18.92
-0.02
6.77
12
10.4
1.22
14
Dunn
21.58
0.01
0.07
2
5
0
0.02
2
5.01
0
0.01
2
17.62
0
0.02
2
10.01
0.02
1 21.55
0
0.01
2
5.1
0
0.19
2
5.01
0
0.01
2
17.62
0
0.02
10.01
0.02
XB
21.55
0
0.01
2
5.04
0
0.09
3
5.01
0
0.01
2
17.62
0
0.02
10.01
0.02
S
21.56
0
0.02
2
5
0
0.02
2
69.32
58.87
68.85
24
24.83
22.11
50.31
35
11,46
7.73
33.2
96
CH
34.87
13.81
20.16
2
9.9
4.49
14.4
2
9.87
5:42
24.15
2
32.96
24.4
40.34
16.73
4.27
7.83
DB 21.55
0 0.01
2
5.04
0
0.09
3
5.01
0
0.01
2
17.62
0
0.02
10.01
0.02
S.Dbw 21.6
0.09
2
5,46
0
1,46
20
5,28
1.13
17
17.62
0.02
10.04
0.07
CVNN
21.87
0.01
0.63
4
6.1
0.03
2,27
7
5.13
0
0.36
22.97
0.51
14.77
10.04
0.04
DCVI 21.55
0.01
6.24
0.03
3.8
38
5.01
0.01
17.62
0.02
0.02
DRCV
21.49 -0.03
0.74
3
12.8
0.76
6
21.17
2.26
29.04
18
18.42
0.25
0.86
0.34
12.52
AIC 23.92
18:75
47.28
39
31.33
21.43
40.99
58
35.87
35.11
61.24
62
23.08
20.95
49.89
25.25
16.81
34,92
BIC 35.67
15.15
22.76
2
9.90
4.88
15.77
2
9.88
5.96
24.34
22.36
38.61
17.64
10.69
IP 57.30 43.66 53.67
12
46.85
27.87
41.49
17
54.99
19.35
61.4
14
55.19
42.03
53.75
30.29
Table 7: SimCSE based clustering results on five text datasets. (15 cases: 5 datasets x 3 evaluation metrics)
SearchSnippets- 8
Biomedical 20
StackOverflow
20
WehofSeience 7
Yahoo!Answers. 10
ACC
ARI
NMI
opt.
ACC
ARI
NMI
opte
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opte
ACC
ARI
NMI
opts
SD
21.65
0.05
1.03
11
8.36
0,25
7.34
12
6.64
0.02
3.18
10
17.62
0
0.02
10.01
0.02
Dunn
21.56
0
0.01
2
5
0
0,02
2
5.02
0
0.03
2
17.62
0
0.02
10.01
0.02
1 21.56
0
0.01
2
5
0
0,02
2
5.04
0
0.08
2
17.62
0
0.02
2
10.01
0
0.02
XB
21.56
0
0.01
2
5
0
0,02
2
5.04
0
0.08
2
17.62
0
0.02
2
10.01
0
0.02
S
21.56
0
0.01
2
5:03
0
0.08
2
5.02
0
0.06
2
17.62
0
0.02
10.01
0
0.02
CH
30.44
6.29
10.4
2
9.75
3.21
10:21
2
9.21
3.1
9.67
2
31.69
21.95
35.76
2
17.7
4.22
7.92
2
DB
21.56
0
0.01
2
5.1
0
0.43
9
5.04
0
0.08
2
17.62
0
0.02
2
10.01
0
0.02
S_Dbw
21.6
0
0.09
2
6.1
0.03
2.85
22
5.45
0
1.12
14
17.62
D
0.02
2
10.04
0
0.07
3
CVNN
21.65
.0.01
0.52
4
9.15
0.34
7.9
5
8.86
0.21
7.41
4
18.13
0.04
1.01
2
10.03
0
0.04
2
DCVI
21.56
0
0.01
2
5
0
0.02
2
6.33
0.01
3.89
42
17.62
0
0.02
2
10.01
0.02
2
DBCV
21.04
0.27
13:25
114
8.91
1.93
6.03
3
9.29
1.65
5.86
3
18.9
0.27
0.83
10.09
0.2
AIC
24.42
16.24
37.81
3
26.95
16.24
32.03
46
41.91
34.70
55.67
52
55.65
38.40
44.50
8
22.73
11.32
23.95
30
BIC
31.38
9.01
14.31
2
9.79
2.99
9.59
2
7.96
1.15
3.42
2
32.04
22.47
36.68
2
18.14
5.02
9.49
2
IP 50.33 30.74
39.79
12
11.31
0.99
12.65
3
68.74
46.44
59.22
20
48.98
33.74
44.50
11
14.72
0.92 6.19
the best partitions in terms of ACC, ARI and NMI are found by our IP index again, which indicates that the representation model can influence the clustering result. Moreover, when comparing with the results of ImageNet- 10 in annleganddToo models may have great impacts on the clustering result, e.g., the best ARI score of Swin based clus- tering can be as high as 99.66 6.64 while it drops to 0.65 dramatically when the representation model is changed to BEIT: (2) Even for the same representation model, the optimal clustering results found by different internal indices could vary lot, e.g., the ARI score of I index is 0 while our IP index and several indices can provide optimal partition ARI score of 99.64.
is interesting to find that: (1) Different representation
8

---

Under review as a conference paper at ICLR 2023
Table 8: VIT based clustering results on five image datasets. (15 cases: 5 datasets x 3 evaluation metrics)
CHEAR-10-
10
MNNSTT 10
FashionMNIIST 10
ImageNet-10- 10
CINIC-10- 10
ACC
ARI
NM1
opla
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opte
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opla.
SD
10.49
1.39
7
11.34
0.02
2
10.01
0
0.02
2
34.34
5.11
36.03
9
10.01
0.02
Dunn
10.01
0.02
2
11.34
0.02
2
10.01
0
0.02
2
10.03
0
0.06
10.01
0
0.02
10.01
0.02
2
11.34
0.02
2
10.01
0
0.02
2
10.04
0
0.08
10.01
0.02
XB
10.01
0.02
2
11.34
0.02
2
10.01
0
0.02
2
87.52
82.8
91.19
10.01
0.02
S
10.01
0.02
2
11.34
0.02
2
10.02
0
0.04
2
89.13
88.55
90.82
13
10.01
0.02
CH
19.55
8.52
29.94
15.87
1.9
3.53
2
19.81
11.31
25.14
2
92.21
89.12
93.75
19.02
7.6
26.43
DB
10.01
11.34
0.02
10.01
0.02
10.04
0.08
10.01
0.02
5.Dbw
11.52
0.02
11.34
0.02
2
10.01
0
0.02
2
1.38
18.56
10.03
0.08
CVNN
11.09
0.02 2.75
12.95
0.12
2.46
2
12.82
0.34
4.56
3
15.5
0.43
10.18
10.86
0.03
1.71
DCVI
10.01
11.34
0.02
10.01
0.02
10.03
0.06
10.01
0.02
DBCV
11.46 0.13
14.66
1.24
3.17
3
15.69
1.79
4.44
45.14
71.31
10
28.71
30.43
AIC 28.21
26.67
46
13.55
22.90
40
16.94
11.68
33.89
36.74
72.87
27.31
21.82
48.74
BIC 19.55
16.02
1.99
3.69
2
19.87
11.66
26,08
6.44
27.37
19.02
7.60
26.43
75.87
58,89
14.52
3.28
2
12.48
0.39
4.11
17.88
56.16
54.43
Table 9: Swin based clustering results on five image datasets. (15 cases: datasets x evaluation metrics)
CIFAR-10-
10
MNIST 10
FashionMNIST: 10
ImageNet-10.
10
CINIC-10 - 10
ACC
ARI
NMI
opte
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI
opty
ACC
ARI
NMI
opt,
SD
10.01
0
0.02
2
15.31
0.78
8.24
3
10.01
0
0.02
2
$6.1
21.67
59.71
10
10.01
D
0.02
2
Dunn
10.01
0
0.02
2
11.37
0
0.04
2
10.01
0
0.02
2
10.02
0
0.03
2
10.01
D
0.02
2
1
10.01
0
0.02
2
11.37
0
0.04
2
10.01
0
0.02
2
10.02
0
0.03
2
10,01
U
0.02
2
XB
10.01
0
0.02
2
11.37
0
0.04
2
10.01
0
0.02
2
10.02
0
0.03
2
10.01
0
0.02
2
S
10.01
0
0.02
2
11.37
0
0.04
2
10.04
0
0.08
2
10.02
0
0.03
2
10.04
0
0.08
2
CH
20.0
17.35
40.25
2
19.47
4.94
9:45
2
19.99
13:57
33.58
2
99.84
99.64
99.47
10
19.89
14.32
30.65
2
DB
10.01
0
0.02
2
11.15
-0.01
0.38
2
10.01
0
0.02
2
10.02
0
0.03
2
10.01
0
0.02
2
S.Dbw
13.62
0.13
7:21
11
13.77
0.31
6.63
10,52
0
1.16
7
13.92
0.2
8.88
13
10.03
0
0.08
2
CVNN
12.9
0.11
5.5
5
13.63
0.29
6.37
3
11.31
0.03
2.25
4
12.82
0.09
5.55
6
14.87
0.27
9.23
8
DCVI
10.01
0
0.02
2
11.38
0
0.07
2
10.01
0
0.02
2
10.12
0
0.27
2
10.01
0
0.02
2
DBCV
23.44
1.51
23.05
15
16.18
1.23
9.44
3
13.68
0.36
7.42
3
57.78
22.57
60.58
10
22.38
1.95
24.05
12
AIC
43.81
47.01
73.79
33
16.28
8.50
18.35
32
21.25
13.79
34.29
30
50.88
55.04
79.93
42
35.38
29.84
54.39
33
BIC
20.00
17.35
40.25
2
19.49
4.99
9.56
2
17.47
6.73
15.88
2
19.81
4.56
23.70
2
19.89
14.32
30.65
2
IP
95.38
90.12
90.14
10
16.74
1.48
8.57
2
35.18
20.10
34.48
7
99.84
99.64 99.47
10
68.57
52.89
62.36
10
Table
10:
BETT based clustering results on five image datasets. (15 cases: 5 datasets X 3 evaluation metrics)
CLEAR-10-
MNIST 10
FashionMNISTT 10
ImageNet-10-
CINIC-10
ACC
ARI
NMI
opt,
ARI
NMI
opfe
ACC
ARI
apt,
ACC ARI
opt, ACC ARI
opts
10.01
0.02
0.02
2
10.01
0.02
10.01
10.01
Dunn
10.01
0.02
11.38
0.06
2
10.01
10.01
10.03
10.01
0.02
11.36
0.02
2
10.01
0
0.02
10.01
10.01
10.01
0,02
11.36
0.02
2
10.01
0.02
10.01
10.01
10.01
0.02
11.36
0.02
2
10.01
0.02
10.01
10.02
CH
17.21
4.97
9.21
20.38
5.66
13.48
2
19.67
11.79
24.09
2
23.77
7.23
20.93
17.09
3,42
0.02
11.36
0.02
2
10.01
0
0.02
10.01
10.01
5_Dbw
10.01
0.02
10.58
-0.04
1.72
10
12.08
0.23
6.15
20
11.46
0.05
3.18
10.03
0.08
CVNN
10.18
0.36
20.73
2.42
14.48
3
21.9
2.85
22.8
14.45
0.77
5.73
11.32
0.07
DCVI
10.01
0
0.02
2
11.36
0
0.02
2
10.01
0
0.02
2
10.01
0
0.02
10.01
0.02
DBCV
12.28
0.23
1.62
12.05
0.05
1.42
3
10.18
0
0.19
3
13.15
0.39
3.34
10.01
0.03
AIC 13.15
5.37
16.88
43
17.76
10.41
27.92
37
22.93
17.30
41.39
36
16.72
9.69
28.51
57
11.92
4.10
13.84
42
BIC
17.44
5.17
9.61
20.41
5.71
13.57
2
19.78
12.05
24.75
2
18.29
5.62
13.19
2
17.24
3,53
7.44
2
24.71
8.01
13.62
23.85
6.48
18.36
3
37.62
20.75
38.08
7
14.01
0.65
5.23
2
21.17
4.69
9.88
7
6 CONCLUSIONS
In this paper, we propose internal purity (IP), a novel internal validation index. IP index uses the differential entropy to measure the purity of a cluster and the cluster centers of partition. Based on the theoretical analysis, the nature of our IP index can help overcome the effectiveness and efficiency drawbacks of existing internal indices. Extensive experiments over different datasets of text and image domains also show that, our IP index can significantly outperform other thirteen well known internal indices when selecting the optimal partition and cluster number with different deep representation models. Although normal distribution assumption is indeed restrictive in our IP index, it won't it affect the usage of our method over the data after deep representation. Considering that deep representation is already widely used, we believe that our method is still very practical.
9

---

Under review as a conference paper at ICLR 2023
REFERENCES
Nabil Ali Ahmed and DV Gokhale. Entropy expressions and their estimators for multivariate distri- butions. IEEE Transactions on Information Theory, 1989.
Alfred V Aho and John E Hopcroft. The design and analysis of computer algorithms. Pearson Education India, 1974.
Hirotugu Akaike. A new look the statistical model identification. IEEE transuctions on automatic control, 1974.
Serkan Akogul and Murat Erisoglu. A comparison of information criteria in clustering based mixture of multivariate normal distributions. Mathematical and Computational Applications, 2016.
Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv, 2021.
James C Bezdek and Nikhil R Pal. Some new indexes of cluster validity. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998.
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning- Springer, 2006.
Tadeusz Caliński and Jerzy Harabasz. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods, 1974.
Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adap- tive image clustering. In ICCV, 2017.
Wen- Yen Chen, Yangqiu Song, Hongjie Bai, Chih-Jen Lin, and Edward Y Chang. Parallel spectral clustering in distributed systems. IEEE transactions on pattern analysis and machine intelligence, 2010.
Chun-Hung Cheng, Ada Waichee Fu, and Yi Zhang. Entropy- based subspace clustering for mining numerical data. In SIGKDD, 1999.
Dongdong Cheng, Qingsheng Zhu, Jinlong Huang, Quanwang Wu, and Lijun Yang. A novel cluster validity index based on local cores. IEEE transactions on neural networks and learning systems, 2018.
Thomas M Cover and Joy A Thomas. Information theory and statistics. Elements of information theory, 1991.
Hadi Daneshmand, Amir Joudaki, and Francis Bach. Batch normalization orthogonalizes represen- tations in deep random networks. 2021.
Zhiyuan Dang, Cheng Deng, Xu Yang, Kun Wei, and Heng Huang. Nearest neighbor matching for deep clustering. CVPR, 2021.
Luke N Darlow, Elliot Crowley, Antreas Antoniou, and Amos Storkey. Cinic- 10 is not imagenet or c0f arxiv, 2018
David I. Davies and Donald W Bouldin. cluster separation measure. IEEE transactions on pattern analysis and machine intelligence, 1979.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x 16 words: Transformers for image recognition at scale. ICLR, 2021.
Joseph Dunn. A fuzzy relative of the isodata process and its use in detecting compact well- separated clusters. 1973.
10

---

Under review as a conference paper at ICLR 2023
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. A density-basee -aased algorithm for discovering clusters in large spatial databases with noise. In Evangelos Simoudis, Jiawei Han, and Usama M. Fayyad (eds.), KDD, 1996.
Chris Fraley and Adrian E Raftery. How many clusters? which clustering method? answers via model-based cluster analysis. The computer journal, 1998.
Andrew Frank. Uci machine learning repository. http://archiv. ics. uci. edu/ml, 2010.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence
embeddings. In EMNLP, 2021.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
Naiyang Guan, Dacheng Tao, Zhigang Luo, and Bo Yuan. Nenmf: An optimal gradient method for nonnegative matrix factorization. IEEE Transactions on Signal Processing, 2012a.
Naiyang Guan, Dacheng Tao, Zhigang Luo, and Yuan. Online nonnegative matrix factorization with robust stochastic approximation. IEEE Transactions on Neural Networks and Learning Systems, 2012b.
Maria Halkidi and Michalis Vazirgiannis. Clustering validity assessment: Finding the optimal par- titioning of data set. In ICDM, 2001.
Maria Halkidi, Michalis Vazirgiannis, and Yannis Batistakis. Quality scheme assessment in the clustering process. In PKDD, 2000.
Maria Halkidi, Yannis Batistakis, and Michalis Vazirgiannis. On clustering validation techniques. Journal of intelligent information systems, 2001.
Xuelei Hu and Lei Xu. Investigation on several model selection criteria for determining the number of cluster. Neural Information Processing-Lete Letters and Reviews, 2004.
Jiabo Huang, Shaogang Gong, and Xiatian Zhu. Deep semantic elustering by partition confidence maximisation. In CVPR, 2020.
Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classification, 2(1):193-218; 1985.
Richard Arnold Johnson, Dean W Wichern, al. Applied multivariate statistical analysis. Pearson London, UK: 2014.
Kamran Kowsari, Donald E Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S Ger- ber, and Laura Barnes. Hdltex: Hierarchical deep learning for text classification. ICMLA, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
Kimin Lee, Kihok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NIPS, 2018.
Yanchi Liu, Zhongmou Li, Hui Xiong, Xuedong Gao, Junjie Wu, and Sen Wu. Understanding and enhancement of internal clustering validation measures. IEEE transactions on cybernetics, 2013.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
Christopher D Manning. Introduction to information retrieval. Syngress Publishing, 2008
Ujjwal Maulik and Sanghamitra Bandyopadhyay. Performance evaluation of some clustering al- gorithms and validity indices. IEEE Transactions on pattern analysis and machine intelligence, 2002.
11

---

Under review as a conference paper at ICLR 2023
Davoud Moulavi, Pablo A. Jaskowiak, Ricardo J. G. B. Campello, Arthur Zimek, and Jörg Sander. Density-bar -based clustering validation. SDM, 2014.
Nikhil R Pal and James C Bezdek. On cluster validity for the fuzzy c-means model. IEEE Transac- fions on Fuzzy systems, 1995.
Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and com- plexity. Courier Corporation, 1998.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011.
Xuan-Hieu Phan, Le-Minh Nguyen, and Susumu Horiguchi. Learning to classify short and sparse text & web with hidden topies from large-scale data collections. In WWW, 2008.
Calyampudi Radhakrishna Rao, Calyampudi Radhakrishna Rao, Mathematischer Statistiker, Calyampudi Radhakrishna Rao, and Calyampudi Radhakrishna Rao. Linear statistical inference and its applications. Wiley New York, 1973.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In EMNLP, 2019.
Peter Rousseeuw. Silhouettes: graphical aid to the interpretation and validation of cluster analy- sis. Journal of computational and applied mathematics, 1987.
Gideon Schwarz. Estimating the dimension of model. The annals of statistics, 1978.
Claude Elwood Shannon. mathematical theory of communication. The Bell system technical journal, 1948.
Lucas Vendramin, Ricardo JGB Campello, and Eduardo R Hruschka. Relative clustering validity criteria: A comparative overview. Statistical analysis and data mining: the ASA data science journal, 2010.
Pauli Virtanen, Ralf Gommers, Travis Oliphant, al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 2020.
Joe H Ward Jr. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 1963.
Michael P Windham and Adele Cutler. Information ratios for validating mixture analyses. Journal of the American Statistical Association, 1992.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-ar natural language processing. In EMNLP, 2020.
Junjie Wu, Hui Xiong, and Jian Chen. Adapting the right measures for k-means clustering. In KDD, 2009.
Mingrui Wu and Bernhard Schölkopf. A local learning approach for clustering. In NIPS, 2006.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnin a novel image dataset for benchmark- ing machine learning algorithms. arXiv, 2017.
Jiang Xie, Zhong- Yang Xiong, Qi-Zhu Dai, Xiao-Xia Wang, and Yu-Fang Zhang. A new internal index based on density core for clustering validation. Information Sciences, 2020.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In ICML, 2016.
Xuanli Lisa Xie and Gerardo Beni. A validity measure for fuzzy clustering. IEEE Transactions on pattern analysis and machine intelligence, 1991.
12

---

Under review as a conference paper at ICLR 2023
Jiaming Xu, Bo Xu, Peng Wang, Suncong Zheng. Guanhua Tian, and Jun Zhao. Self-taught convo- lutional neural networks for short text clustering. Neural Networks, 2017.
Rui Xu and Donald Wunsch. Survey of clustering algorithms. IEEE Transactions on neural net- works, 2005.
Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong. Towards f-means-friendly spaces: simultaneous deep learning and clustering. In ICML, 2017.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. In NIPS, 2015.
Tianyi Zhou, Dacheng Tao, and Xindong Wu. Manifold elastic net: unified framework for sparse dimension reduction. Data Mining and Knowledge Discovery, 2011.
Mengjie Zu, Arunkumar Bupathy, Daan Frenkel, and Srikanth Sastry. Information density, struc- ture and entropy in equilibrium and non-equilibrium systems. Journal of Statistical Mechanics: Theory and Experiment, 2020.
A
DERIVATION OF EQ.
IP = CP SP
Diff Entroppy(H))
Diff Fntroppyt (p))
According to Eq. and
ran
rank(E.)
+ 1 In (2sT)
rank(Ep)) En)
rank(2,, In (2T)) + 2 In En1
[
2
2
According to Eq. 6
rank(2) E1
rank Ei)
ranck(E))
rank En In In(2 + 2
1 In E..
In (2T))
E.
[1 + (2() rank E,) +
In Eg
{[1 + In(2 rank(E, ) + In |Enl}
2k
1+ In(x))
k
In E |1
+
rank(E,) rank(E.))
2
B EVALUATION METRICS
In our evaluations we used three common external validation indices: Accuracy (ACC), Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI). The ACC and NMI range between 0 and 1, and the ART ranges between and Larger ACC, ARI and NMI indicates better clustering result.
ACC is computed as follows:
ACC Ein 5 (yi, map (c:)
(12)
N
where Yi is the true cluster label. c, is the cluster label obtained by clustering, and 8(x, y) is an indicator function returning 0 (x * y) or (x y)- map(.) transforms the cluster label e; to its true cluster label by the Hungarian algorithm (Papadimitriou & Steiglitz 1998). The larger the ACC value is, the better the clustering result.
ARI is computed follows:
RI E
RI a +
ART
(13)
1- E
C
13

---

Under review as a conference paper at ICLR 2023
where C22y is the total number of possible pairs in the dataset, is the number of pairs of samples that are in the same ground truth class and in the same cluster, is the number of pairs of samples that are in different ground truth class and in different cluster. E|R is the expected RI. Larger ARI indicates better clustering result.
NMI is computed follows:
T(A, B)
NMI B)
(14)
VH(A)H((B
where A is the predicted labels and B the ground truth labels. I is the mutual information and H is the entropy.
C MODEL CONFIGURATION
BERT is based on bert-bass-umcased, SBERT is based on all-distitroberta-rl. SimCSE is based on umssip-iimcss---beertaaaass ViT is based on htt-buxs--ptcchl1-224---- on swin-base--ptcche wwinhh-wwwddoo and BEiT is based on peit-base-patchh66 pt22:, where BERT, SBERT, SimCSE, ViT and BEiT return 768 embedding size, and Swin returns 1024 embed- ding size.
Swin is based
D CLUSTERING EVALUATION RESULTS UNDER SINGLE CLUSTERING ALGORITHM
As before, we first present the statistical results of each clustering algorithm and report them in Table I1 Obviously, compared with other 13 indices, our IP index has significant advance in achieving the best results under each clustering algorithm. Specifically, for all 90 cases, our IP index can produce 52 best results, while the second top index SD only has best results in K-Means. Our IP index can produce 44 best results, while the second top index S only has 15 best results in GMM. Our index can produce 37 best results, while the second top index DBCV only has 17 best results in AHC. Although CH can produce 63 best results while index only has 26 best results in DBSCAN, our index is better than CH in the other three clustering algorithms. Finally, IP index outperforms other indices greatly with respect different domains, i.e., text and image. Then the specific results of each clustering algorithm can been in D D.2 D.3 and D.4 respectively.
Table 11: Number of best clustering results based counting over K-Means, GMM, AHC and DBSCAN
SD Dunn
I
XB
CII
DB
S.Dbw
CVNN
DCVI
DBCV
AIC
BIC
IP
K- MeManns
Text datasets
30
Image datasets
3
7
5
2
7
1
22
All datasets
13
5
0
3
7
7
4
8
1
52
GMM
Text datasets
0
10
25
Image datasets
4
1
19
All datasets
0
5
4
11
44
AHC
Text datasets
0
6
6
0
15
Image datasets
11
22
All datasets
14
17
37
DBSCAN
Text datasets
0
0
0
33
0
0
4
0
0
7
Image datasets
3
0
0
19
All datasets
5
26
D.1 THE CLUSTERING RESULTS ON K-MEANS
In this section, we only use K-Means to evaluate the clustering results. Since the random initial- ization nature of K KMMeans, the experimental results are averaged over five random runs for each validation index. The evaluation results based on external indices of ACC and ARI are shown in
Table 12
13
14
15
16 and 17 the evaluation results in terms of NMI are shown in Table T8 19
14

---

Under review as a conference paper at ICLR 2023
20 21 22 and 23
where the best results are highlighted in bold. Moreover, the optimal & results each index select can be seen in Fig. outperforms other indices and is close the real l value represented in red dash line.
6 and Obviously, for almost all cases, our IP index
Table 12: BERT based K-Means clustering results on five text datasets.
SearchSnippets
Biomedical
StackOverflow
WebofSeience
YahoolAnswers
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
27.98=10.23
17.91=6.21
29.31=0.01
14.64=0.01
8.42+0.01
0,9020.00 40.6620.01
24.36=0.02
19.97=0.03
2.59:0.02
Dunn
27.39=12.76
17:3527.89 89 12.7920.76
7.440.70
7.6543.05
1.4742.91
8.4820.44
6.2420.27
25.9828.58
10.18:3.11
33.6340.03
12.22:0.06
9.6620.00
2.5220.01
6.2920.00
0.1740.00
29.3920.01
17.9910.01
12.9020.03
0.70a0.01
XB
17.2243.37
11.19:2.12
29.31
14.64:0.01
8.4240.01
0.90+0.00
29.3910.01
17.9910.01
12.9010.03
0.7040.01
S
10.05+0.59
6.66+0.24
9.66#0.00
2.52+0.01
8.42+0.01
0,90=0.00
29.39=0.01
17.99=0.01
12.90=0.03
0,70#0.01
CH
33.6340.03
12:22=0.06
9.66=0.00
2.52+0.01
6.29=0.00
0.17=0.00 29, 29.39:0.01
17.99=0.01
12.90=0.03
0,70#0.01
DB
10.2240.39
6,67+0.18
29.31=0.01
14.64=0.01
10.9620.01
2.4120.01 40.6620.01
24.36=0.02
16.52=0.02
1.8920.01
S Dhw
9.85+0.51
6.60+0.15
11.38:0.33
6.58+0.23
12.92=0.66
7,05+0.23 8.20+0.39
6.09+0.12
12.59+ .53 5.1960.56
CVNN
33.6340.03
12:22=0.06
9.66+0.00
2.52+0.01
6.29:0.00
0.17+0.00
29.3990.01
17.99=0.01
12.90=0.03
0.70+0.01
DCVI
10.18+0.93
6, 6.70+0.41
11.5990.29
6.74+0.30
13.14#0.65
6.98+0.35
8.6020.21
6.22#0.19
11.91+0.94
5.03+0.27
DBCV
9.73a0.59
6.4520.13
6.90s0.46
8.49a0.20
6.26a0.26
4.9920.25
12:22:0.06
2.52+0.01
0.1740.00
29.39:0.01
17.99x0.01
12.90=0.03
0.70+0.01
BIC
12:22:0.06
9.66+0.00
2.52+0.01
29.39+0.01
17.99+0.01
12.90=0.03
0.70+0.01
54.18+5.71
33.58=4.04
31,4820,55
15.63=0.12
15.9820.14
4.0920.04
47.93+2.40
31.90:1.75
19.9720.03
2.5910.02
BIC
CWNN
DBCV
DCM
Dumn
P(Ouns]
125
125
100
80
100
100
75
75
50
50
40
20
25
25
SeorchSnopets
StackOvertiow
WebofSceence
Y5hoo Answers
Figure 2: The optimal & value found by each index on BERT representations for text datasets.
Table 13: SBERT based K-Means clustering results five text datasets.
ScarchSnippets
Biomedical
StackOverflow
WebofSeience
YahoolAnswers
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
24.24=10.15
19.2747.29
41.16212.78
26.5027.84
59.66120.26
36.43s15.51
22.8548.38
20.20x6.99 95
17.9344.80
11.9623.05
Dumn
16.9424.66
13.0923.74
21.3746.59
14.5824.54
28.78413.07
17.78a11.57
6..6299.60
13.2248.99 97
22.08=11.56
13.2344.27
1
34.87=0.01 13.81:0.01
9.9040.01
4.49:0.01
13.47+2.01
6.3240.96
32.96:0.00
24.40=0.00
16.7340.02
4.27:0002
XB
23.68=10.79
17.73+7.92
41.23+4.3
24.88+3.90
31.16420.08
16.51=13.09
15.72+16.36
29.44s11.88 .8
12.49+1.07
8.44=0.75
S
10.006
8.31+0.28
1.726+1.79
29.9=0.98
69.32+1 488
$8.87
24.83+3.1 17
22.11.2.29
11.4640.48
7.73=0.33
CH
14.87=0.01
13.81+0.01
9.90+0.01
4.49+0.01
9,87+0.00
5.42+0.00
32.96+0.00
24.40+0.00
16.73+0.02
4.27=0.02
DB
10.94:0.19
8.21
16.88+0.39
11. 11.30=0.39
33.35+17.02
30.44+12.35
29.03=3.31
25.34+2.50
11.41+0.66
7.54=0.19
S D.b
10.92=0.40
8.23+0.03
16.42+0.75
10.84:0.29
20.32:0.86
19.40=1.21
9.94w0.27
8.28+0.17
11.27=0.50
7.54=0.26
CVNN
44.8720.01
13.81:0.01
9,9020.01
4.4920.01
20.76:5.3 32
9.43w2.33
32.9620.00
24.40z0.00
23.4220.01
10.14:0.04
DCVI
10.7920.39
8.18+0.20
17.04x1.47
11.5220.99
63.17412.44
40.4029.5
10.7920.66
8.52a0.16
12.0910.99
7.8720.50
DBCV
10.88a0.13
8.2610.18
16.9610.93
11.00.778
$5.4045.40 400
43.6518.77
10.6740.80
8.66a0.42
11.3610.25
7.5220.13
AIC
34.8740.01
13.8140.01
9,90a0.01
4.4910.01
9,8740.00
5.4240.00
32.9610.00
24.40a0.00
16.7340.02
4.27:0.02
BIC
34.8740.01
13.81+0.01
9.90s0.01
4.49+0.01
9.8740.00
5.4240.00
32.9640.00
24.40+0.00
16.7340.02
4.27:0.02
IP
55.68:4.06
42.5762.72
47.34+1.16
28.59#0.93
72.61=1.62
50.25+33.0
58.60+
44.50x1.46
50.14+2.26
27.07+1.11
15

---

Under review as a conference paper at ICLR 2023
100
125
125
100-
100
80-
60
* 75
20
SearchSnispes
Blomodical
StackOvartiow
WebofSdeence
YahoolAnswers
Figure 3: The optimal ls value found by each index SBERT representations for text datasets.
Table 14: SimCSE based K K-Means clustering results five text datasets.
SearchSnnipets
Biomedical
StackOverflow
WebofScience
Answers
ACC
ARI
ART
ACC
10,77=0.80
7.35+0.69
21.0648.20
9,47=6.35
61,42+2.06
46.07=0.99
49.01=1.77
34.49%1.44
17.29s1.72
8.74+0.83
Dunn
24.24=13.22
15.72:8.27
23. 23.10:1.61
11.34=141
38.60=14.71
28.2526.46
22.47+17.29
16.78&11.43
30.93+11.3
12.3+3.89
30.4420.01
6.2910.01
9.75e0.00
3,2140.00
9.21:0.02
3.10:0.02
31.6920.00
21.95:0.00
17.70:0.04
4.22s0.02
XB
11.25=1.54
7.4910.94
17.3920.01
6.6340.01
60.4023.08
45.6921.47
43.5910.00
28.1920.01
11.2241.66
5.4910.97
10.5240.62
6.8210.44
9.7520.00
3.21a0.00
63.3641661
46.9920.71
31.6910.00
21.9510.00
17.7010.04
4.2240.02
CH
30.44:0.01
6.2910.01
9.7540.00
3.2140.00
9.21a0.02
3.10a0.02
31.6910.00
21.95+0.00
17.7040.04
4.2740.02
DB
10.82:0667
6.9840.19
14.76&1.06
8.9340.54
63.25&3.11
46.71s1.33
43.59:0.00
28.19:0.01
10.2640.25
4.8940.10
S.Dhw
10.32:0.42
6.75+0.09
13.6740.28
8.1440.20
19.5740.55
17.19=0.29
8.6440.62
6.98+0.24
10.2740.5
5.0740.26
CVNN
30.44:0.01
6.2940.01
17.39+0.01
6.6340.01
9.21+0.02
3.000.022
31.69+0.00
21.95+0.00
17.70H0.04
4.2240.02
DCVT
10.2640.31
6.65+0.05
14.47+0.28
8.7940.21
21.49+1.03
18.53e1.13
9.04+0. 19
7.13+0.17
10.38+0.5
5.07+0.21
DBCV
10.04:0.22
6.67+0.20
13.99#0.74
8.44#0.52
21.71=1.54
18.57=1.72
9,32+0.48
7.1740.23
10.41#0.46
5.06+0.24
AIC
30.44:0.01
6.29#0.01
9.75e0.00
3.21+0.00
9.21=0.02
3. 3.10=0.02
31.69=0.00
21.9550.00
17.70=0.04
4.22+0.02
BIC
30.44:0.01
6.2940.01
9.75e0.00
3.21+0.00
9.21=0.02
30+00.02
31.69+0.00
21.95+0.00
17.70H0.04
4.2240.02
IP
50.33:0.82
30.7420.99
37.9620.19
20.0620.10
68.7421.38
46.4421.36
48.9843.73
33.74=2.07
37.03+0.15
14.7520.16
ADC
BIC
CVNNN
DBCV
D5
DCV
Dunn
XB
50
D0w
PIOursi
(00)
100
125
125
100
80
80
100
100
60
75
75
50
50
40
20
25
25
20-
SearchSnopets
StackOverlow
Figure 4: The optimal value found by each index on SimCSE representations for text datasets.
Table 15: ViT based K-Means clustering results on five image datasets.
CIFAR-10
MNIST
MasSionMNIST
ImageNet-10
CINIC-10
ACC
ARI
ACC
ART
ACC
ARI
ACC
ARI
ACC
ARI
SD
76.0720.96
38.84
31.1720.23
11.7130.99
33. 33.4020.01
21.7220.02
12.96
71.44220377
17.6621.71
13.2921.35
Dunn
13.1024.86
10.71:3.00
29. 1920.89
10.6420.31
35.25+2.63 2.63 20.3320.51
57.15:38.37
51.34242.69
29.2823.54
20.03:2.69
19.55=0.00
8.52+0.00
15.8740.00
1.90+0.00
19.81+0.00
11.31=0.00
23.6845.32
9.90s4.10
19.0240.00
7.60+0.00
XB
13.4.88.86 10.71=3.00
28. 78.7880.03
10.5+0.05
19.81=0.00
11.3120.00
92.2145.75
89. 12#6.89
19.0240.00
7.60=0.00
S
68.07=2.29
57.42+0.96
15.8740.00
1.90+0.00
14.1.40000
18.93=0.00
89.13+2.84
88.55&1.85
53.86&1.07
41.1420.69
CH
19.55=0.00
8.52+0.00
15.8740.00
1.90+0.00
19.81
11.31=0.00
92.2145.75
89. 12#6.89
19.0240.00
7.60+0.00
DB
23.1024.86
10.71s3.00
30.8240.07
11.4420.40
19.8120.00
11.3120.00
68.951440.22
60.95248.98
19.0240.00
7.60g0.00
Dbw
18.60s4.88
16.85s4.55
7.50a0.21
4.41a0.07
8.9220.24
6.1640.07
66.31a16.12
50.61424.24
15.33&1.65
11.99a1.1
CVNN
23.10s4.86
10.71
15.87400.00
1.90s0.00
19.8140.00
11.3120.00
52.6448.71
31.2749.35
19.0240.00
7.60=0.00
DCVI
19.7024.89
15:08e0.63
7.6140.33
4.38:0.13
8.89=0.11
6.21+0.19
84.74:30.79
82.49:31.77
15.05+0.76
11.73:0.43
DBCV
17.36:1.05
15.35e0.69
7.7340.26
4.3920.12
9.1020.31
6.2120.13
92.3628.83
91.6427.17
14.6240.76
11.3820.39
19.55=0.00
8.52+0.00
15.8740.00
1.90+0.00
19.81+0.00
8.0022117
19.02+0.00
7.60=0.00
BIC 19.55e0.00
8.52+0.00
15.8740.00
1.90=0.00
19.81=0.00
11.31+0.00
00,0000.01
8.002.11
19.02+0.00
7.60=0.00
75.87=0.05
58.8950.09
31.37:0.36
11.7340.33
39,75+1.08
21.87=0.23
72. 12.14:13.20
58.87420.65
63,3940.6
40.94+2.68
16

---

Under review as a conference paper at ICLR 2023
BIC
CH
CMN
OBCV
06
DCV
Dunn
5,0bw
(P(Our))
maLk
75
75
75
12
75
10-
45
45
45
45
MNNSS
FashionMNISST
ImageNNt-10
Figure
The optimal & value found by each index on VIT representations for image datasets.
Table 16:
Swin based K- K-Means clustering results on five image datasets.
CIEAR-10
MNIST
FashionMNIST
ImageNet-10
CINIC-10
ACC
ART
ACC
ART
ACC
ART
ACC
ARI
ACC
ARI
SD
82.9845.17
79.2223.95 27.6920.03
9.13#0.02
30.85=0.00
20.6520.00
99.8426.00
99.64:0.00
63 63.1243.42
51,65=2.45
Donn
1941946.46
78 78.8446.64 22.2740.60
10.56:0.10
19:99=0.00
13.57=0.00
83.78415.
82.85e16.46
35.64+13.51
19.44=11.03
1
20.0010.00 17.3540.00 19.4740.01
4.9420.00
19.99-0.00
13.57.0.00
19.81a0.06
4.56g0.07
19.8910.00
14.3240,01
XB
86.7640.02 82.11+0.06 21.1243.60
5.78&1.89
34.27=0000 01 20.7620.03
99.84:0.00
99.64=0.00
56.69#15.68
44.02s12.84
19.47=0.01
15.22a0.00
90.75=1.70
65.2942.47
52.94=2.19
20.00+0.00
17.3540.00
19.47+0.01
4.94+0.00
19.99=0.00
99.54.0.00
19.89+0.00
14.32+0.01
30.8520.00
6887740.11
$2.62a0.17
S_Dbw
20.07=0.51 19.90#0.41
8.32+0.37
4.38+0.32
9.05+0.44
6.20=0.13
99.84:0.00
99.64=0.00
20.7243.52
16.93+3.28
CVNN 86. 76.0000.02 82.1140.06
19.47s0.01
4.94x0.00
19.99=0.00
13.57a0.00
99,8420.00
99.64:0.00
27.01+3.98
19.72=3.02
DCVI
20.6620.86
20.28
8.1910.34
4.25x0.12
9.7020.68
6.6220.28
89.81=14.11
89.18215.15
18.5242.05
14.81=1.11
DBCV
21.14=1.04
20.8841.02
8.3720.34
4.35x0.19
9.3420.29
6.4420.08
99.8420.00
99.6420.00
20.43+1.39
16.30s1.25
AIC
20.00+0.00
17.35+0.00
19.47+0.01
4.94+0.00
19.99=0.00
13.57=0.00
19.81=0.06
4,56+0.07
19.8 19.8000.00
14.32+0.01
BIC
20.00>0.00
17.35x0.00
19.47=0.01
4.94x0.00
19, 99.9900.00
13.57.0.00
19.81=0.06
4.56s0.07
19.89a0.00
14.32=0.01
IP
95.38a0.02
90.12>0.05
27.30=0.01
8 8.60&0.01
35. 18:0.30 20, 10:1070.67
99.8420.00
99.64=0.00
68 68.05+1.77
11.80
(P(Ourg)
14:
75
75
75
75
10
30
30
30
30
01
01
MNIST
ImageNet-30
Figure The optimal value found by each index on Swin representations for image datasets.
Table 17: BETT based K-Means clustering results on five image datasets.
CIFAR-10
MNIST
Fashion MNIST
ImageNet-10
CINIC-10
ACC
ARI
ACC
ACC
ACC
ACC
ARI
SD
21.49=2.10 6.2620.84
20.38:0.00
5.6620.00
30.89=0.00
20.0640.00
32.4425.43
16.65=2.08
18.1040.91
5.83:0.07
Dunn
8.54+1.36
3.67+0.55
9.9240.73
6.5740.32
13.45+1.27
10.33+0.71
12.80+2.60
6.25+1.25
7.31#0.42
2.64a0.19
I
17.2110.00 4.97a0.01
19.7740.01
5.7240.00
19.6710.00
11.7910.00
23.77a0.00
7.2310.00
18.0340.07
2.9710.01
XB
22.20=0.02 6.24=0.01
19.77+0.01
5.72+0.00
19.67=0.00
11.79*0.00
23.77+0.00
7.23+0.00
18.76+0.02
3.39=0.01
S
17.2140.00 4.9740.01
19.7740.01
5.7240.00
19.6740.00
11.7940.00
17.3740.03
4.0840.03
17.0910.00 3.42a0.00
CH
17.21 4.97=0.01
20.38:0.00
5.66e0.00
19.67=0.00
11.79x0.00
23.77=0.00
7.23+0.00
17 17.09x0.00 3.42+0.00
12.9241.37
5.44#0.61
19.7740.01
5.7240.00
19.67a0.00
23.7740.00
7.23+0.00
18.7640.02
3.39a0.01
S.Dbw
7.6340.33
3.29=0.15
9.73+0.48
6.46#0.29
12.51=0.58
9.8620.53
11.64+1.24
6.7740.63
7,0420.47
2.63:0.20
CVNN
17.2140.00
4,97#0011
20.38:0.00
5.6640.00
41.49:2.10
23.9140.88
31.62+0.03
12.2540.03
17,0940.00
3.4240.00
DCVI
7.6620.16
3,30a0.10 10
9.48+0.27
6.42+0.12
12. 13,0.00
9.51
11.4420.52
6.68+0.31
7.1420.43
2.5620.16 16
DBCV
7.76e0.24
3,33+0.08
9.22+0.29
6.22+0.11
12. 15=0.88
9.37+0.35
11.06e0.25
6.58+0.15
7.25+0.63
2.59+0.15
AIC
17.2110.00 4,9740.01
20.3810.00
5.6640.00
19.6710.00
11.7940.00
17.3740.03
4.0840.03
17.0940.00
3.42a0.00
BIC 17.2140.00
4.97+0.01
20.38:0.00
6.660.000
19.67=0.00
11.79*0.00
17.37+0.03
4.08+0.03
17.09+0.00
3.42+0.00
IP
23.79:0.04
7.16=0.07
20.38=0.00
5.6660000
37.99=0.53
23.1640.19
37.35+1.50
18.47+1.13
23.6450.06
8.37+0.03
17

---

Under review as a conference paper at ICLR 2023
105
75
75
75
90
75
75
60
60
45
30
30
30
30
15
CIFAR
MNIST
FashionMNIST
ImageNet-10
CINC
Figure The optimal & value found by each index on BEiT representations for image datasets.
Table 18: BERT based K M-Maans clustering results in terms of NMI on five text datasets.
SearchSnippets
Biomedical
StackOverflow
WebofScience
Yahoo!Answers
SD
37.51=0.89
26.6 26.620.02
2.31=0.01
36.2620.07
6.9+0.02
Dunn
37.43=0.69
28.84=0.16
5.93=12.07
35.78=0.17
19.03+0.88 0.2
16.68+0.1
8.21+0.02
0.53#0.0
28.63=0.01
1.85=0.01
XB
36.87=0.24
26.620.02
2.51=0.01
28.63=0.01
1.85+0.01
S
37.5
8.21=0.02
2.51=0.01
28.63=0.01
1.85+0.01
CII
16.68+0.1
8.21+0.02
0.53+0.0
28.63=0.01
1,85=0.01 0101
DB
37.05=0.05
26.620.02
6.99=0.01
36.26=0.07
4.79=0.02
S Dbw
37.47=0.47
28.95#0.1
28.4240.46
35.9620.25
19.87 26
CVNN
16.68+0.1
8.21+0.02
0.53+0.0
28.63+0.01
1.85+0.01
DCVI
37.41+0.56
29.01=0.17
38.349
35.67=0.28
20.08=0.37
DBCV
37.3020.37
28.95=0.14
28.4420.77
35.72=0.20
19.98+0.20
AIC
16.68+0.10
8.21+0.02
0.53#0.00
28.63+0.01
1.85+0.01
BIC
16.68+0.10
8,21+0.02
0.53+0.00
28.63=0.01
1.85+0.01
IP
42.31+3.01
27.0920.23
10.3680.12
40.3320021
6.90a0.02
Table 19: SBERT based K- K-Means clustering results in terms of NMI five text datasets.
SearchSnippets
Biomedical
StackOverflow
WebofScience
Yahoo!Answers
SD Dunn
46.56+2.27
56.15.1.56
67. 67.69
49.52+2.64
34.1940.64
45.25#1 54
39.61=0.95
49.53=12.21
46.2+3.58
32.72+3.2
I
20.16+0.06 04
14.4+0.02
30.8443.87
40.34+0.0
7.83+0.03
XB
46.34+2.44
38. 38.9822.85
49.97=14.62
51.89=3.15
33.06:0.2 21
S
43.57=0.42
42.5621103
68.8540.79
50.31=0.63
33.2+0.39
CII
20.16+0.04
04.42
24.15+0.01
40.34:0.0
7.83+0.03
DB
43.520.44
38.7920.18 18
65.69=12.34
51.5+0095
33.06:0.55 34
S Dbw
43.42=0.22
38.73=0.14
56.35+0.45
44.22=0.32
33.15#0.14
CVNN
20.16+0.04
14.4+0.02
41.61+6.51
40.34+0.0
17.17+0.05
DCVI
43.2620.16
38.91#0.34
71.3645.24
44.27=0.22
33.0420.2
DBCV
43.59+0.55
14.0444
67.50+2.55
44.32+0.39
32.99.0.26
AIC
20.1640.04
14,40+0.02
24.15+0.01
40.34+0.00
7.83+0.03
BIC
20.1640.04
14,40=0.02
24.15+0.01
40.34=0.00
7,8320.03
IP
53.4521.26
42.16E1.04
72.9410.22
54,3410.22
34.801.888
Table 20: SimCSE based K- -caans clustering results in terms NMI on five datascts.
SearchSnippets
Biomedical
StackOverflow
WebofScience
Yahoo!Answers
SD
37.06+0.53
20.81+6.61
58.6840.86
44.6=0.26
23.82=0.3
Dunn
37.78+0.9
23.05=1148
52.1
4189=22224
22.03+0.79
I
10.4+0.01
10.21=0.01
9.67=0.06
35.76+0.0
7.92+0.04
XB
36.8420.36
17.85=0.03
58.69=0.9
43.53=0.02
23.13+0.3
S
36.69=0.44
10.21=0.01
59.2840.35
35.7640.0
7.92=0.04 04
CII
10.4+0.01
10.21+0.01
9.67+0.06
35.76+0.0
7.9240.04 04
DB
37.1340.5
32.08+0.18
59.12-0.9
43.53=0.02
23.3+0.14
S Dhw
37.06+0.19
32.02+0.28
11.4
39.39=0.12
23.3340.24
CVNN
10.4+0.01
17.85+0.03
9.67+0.06
35.76+0.0
7.9240.04
DCVI
37.02+0.2 29
32.1+0.21
51.55+0.17
39,55=0.18
23.27+0.39
DRCY
37.12+0.67
31.92+0.25
51.25+0.50
39.55=0.19
23.1240.24
AIC
10.40+0.01
10.2140.01
9.67+0.06
30.76+0.00
7.9240.04
BIC
10.40+0.01
10.2140.01
9.67+0.06
35.76+0.00
7.9240.04
IP
39.79-0.57
31.5520.08
59.2740.79
44.50=1.09
20.93+0.11
18

---

Under review as a conference paper at ICLR 2023
Table 21: VIT based K-Means clustering results in terms of NMI five image datasets.
CIFAR-10
MNIST
FasHionMNIST
ImageNet-10
CINIC-10
71.26:0.66
20.38=0.42
33.31=0.02
87.49 95
45,420.57
Dunn
34.92+6.81
18 88.640.58
33.15=1.
67.19430.54
42. 84.3.11 18
29.94=0.0
3,53=0.0
25.14+0.0
35.96=9.23
26.43+0.01
XB
34.9246.81
18.34+0.04
25.14+0.0
93.75+2.19
26.43+0.01
S
67.9411.29
3,53+0.0
29.67=0.0
90.82=0.92
55 05.5520
29.94=0.0
3.53+0.0
25.14:0.0
93.75+2.19
26.43=0.01
DB
34.9246.81
19.86+0.37
25.14+0.0
70.62=35.5
26.43+0.01
S.Dbw
53.7+2.06
23.77=0.33
33.31=0.13
78.58+10.88
44, 03:0.44
CVNN
34.92+6.81
3,53=0.0
25.1440.0
68,42=8.85
26.43=0001
DCVI
50.72+4.67
24.09+0.28
33.29+0.11
87,89&1844
44.79
DBCV
52.80#0.56
24.18=0.24
33,20=0.19
93.22=4.06
44.76#0.12
AIC
29.94+0.00
3,53=0.00
25.14=0.00
31,51=4422
26 26.4300.01
BIC
29.94+0.000
3,53+0.00
25.14=0.00
31,51+4.42
26.43+0.01
IP
70.2420.07
20.7420442
34.8820.71
82.9328.36
56.8920.73
Table 22: SWin based -MManss clustering results in terms of NMI on five image datasets.
CIFAR-10
MNIST
MNSSI
CINIC-10 ImageNet-10
SD
87.75+1.38
14.1920.03
34.8920.0
99,47=0.0
62.9=1.44
Dunn
86.77+2.03
17.9=0.11
33.58+0.0
93.33=6.56
53.86+3.46
I
40.25=0.0
9.45=0.01
33.5820.0
23.720.31
30.65
XB
88.7720.05
10.41+2.14
35.7620.08
99.47=0.0
58.0628.9
89.9=0.59
9.45=0.01
29.48=0.01
94.61=0.47
63.88=0.51
40.25=0.0
9.45=0.01
33.58+0.0
99.47=0.0
30.65+0.03
DB
88.77+0.05
14.19=0.03
34.8%20.0
99.47=0.0
62.41
S Dbw
63.45+0.26
20.07=0.18
33.42+0.29
99.47=0.0
50.18+1.46
CVNN
88.77+0.05
9.45+0.01
33.58+0.0
99.47=0.0
39.91
DCVI
63.4420.55
19.76+0.11
33.6320.2
95.77=5.36
49.21=0.4
DBCV
63.64#0.52
19.85=0.26
33.63=0.24
99,47=0.00
49,78=0.60
AIC
40.25+0.00
9.45+0.01
33.5840.00
23.70=0.31
30.65+0.03
BIC
40.25+0.00
9,45+0.11
33.58+0.00
23.70=0.31
30.65+0.03
IP
90.1420.02
13.69H0.02
34,10.0.33
99,4720.00
61.978197
Table 23: BEiT based K -Means clustering results in terms of NMI on five image datasets.
CTFAR-10
MNIST
MNSSi
ImageNet-10
CINIC-10
SD
11.58:0.76
13.4820.0
36.2720.0
28.71=0.33
12.8620.44
Dunn
17.7620.33
30.53=0.15
40.6820.32
24.77=8.03
14.55+0.26
I
9.21=0.02
15.3620.0
24.0920.0
20.93+0.0
7.9920.06
11.03+0.02
15.36+0.0
24.09=0.0
20.93=0.0
7.97=0.02
9.21=0.02
15.3640.0
24.0920.0
10.420.01
6,9340.0
CH
9.21=0.02
13.4820.0
24.0920.0
20.93+0.0
6.9340.0
DB
16.92+0.11
15.36*0.0
24.09#0.0
20.93=0.0
7.97=0.02
S Dbb
17,8+0.15
30.74=0.41
20.66=0.25
1
14.82+0.12
CVNN
9.21#0.02
13.4820.0
39.98=1.29
23.5820.03
6,93+0.0
DCVI
17.99:0.12
30.84=0.37
10.53=0.18
28.55
14.73+0.2
DBCV
17.77+0.17
30.84+0.45
40.57=0.24
28.47=0.18
14.75+0.17
AIC 9.21#0.02
13.48=0.00
24.09=0.00
10.40=0.01
6.93+0.00
BIC
9.21+0.02
13.48=0.00
24.09-0.00
10.40=0.01
6,93+0.00
IP
12.3920.06
13.4820.00
39.24=0.21
28.50=0.62
13.6920.10
D.2 THE CUSTERING RESULTS ON GMM
In this section, we only use GMM to evaluate the clustering results. Since the random initialization nature of GMM, the experimental results are averaged over five random runs for each validation
index. The evaluation results based on external indices of ACC and ARI II are shown in 24
25 26 27
28 and 29 the evaluation results in terms of NMI are shown Table 30 31
35 34 and 35 where
the best results are highlighted in bold and the optimal & value each index selecet is provided in Fig.
g I0
12 and [3] Obviously, for almost all cases, our index outperforms other indices and close to the real value represented in red dash line.
19

---

Under review as a conference paper at ICLR 2023
Table 24: BERT based GMM clustering results on five text datasets.
SearchSnippets
Biomedical
SerckOverllow
WebufScience
Answer/nswers
ACC
ART
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
27.19+7.62 15
29.4540.05
14.7340.15
13.98#1.90 .96
3.4840.66 40.88:0.01
24.69=0.01
19.76=1.95 .5
3.06+1.00
Dunn
30.72+15.00
15.60=6.96
14.31+8.35
5.5645.12
7.18+1994
0.5940.98
12.5445.19
8.9943.76
26.13+3.34
9.7643.00
34.6520.11 14.0620.27
9.8240.00
2.9420.00
6.32+0.00
0.1510.00
29.42a0.01
18:4390.06
12.23g0.26
0.24a0.05
XB
24.46:10.60
15/42:6.13
26.0443.78
12.3842.60
8.0120.00
0.6540.00
29.42a0.01
18.4340.06
12.23=0.26
0.24a0.05
9.87+0.52
6.58+0.22
9.8240.00
2.94:0.00
8.01=0.00
0.65+0.00
29.42=0.01
18:43:0.06
12.23=0.26
0.24#0.05
CH
34.65=0.11
14.06=0.27
9.8240.00
2.94:0.00
6.32+0.00
0.1540.00
29.42=0.01
18.4340.06
12.23=0.26
0.24:0.05
9.84+0.76
6.62+0.34
27.38+3.58
13.55+2.17
10.73+0.01
2.34+0.01
40.88+0.01
24.69+0.01
14.78=2.58
1.25+0.97
S.Dbw
9.44+0.51
6.35+0.29
11.66+0.38 38
6.78=0.26
12.54=0.46
6.71+0.26 8.23+0.20
5.93e0.
13.5243.
5.66+1.31
CVNN
34.65+0.11
14.06=0.27
14.8843.35
6.65#2.36
6.32+0.00
0.15+0.00
29.42=0.01
18.4540.00
0606
20 20,80=2.99
3.82+1.98
9.63+0.52
6.4480.24
11.7840.42
12.73+0.40
6.8540.28
6.30+0.33
12.04=0.35
DRCV
9.73+0.58
6.57+0.35
11.9840.65
6.99=0.52
12.59+0.39
6.7340.23
6.25+0.21
12.51=0.60
5.2240.19
27.84a1.16
18.6120.62
24.50a1.69
13.8811.33
22.34:0.48
9.99s0.30
24.69a0.01
29.52=1.72 11.6600.2
34.65e0.11
14.06s00.27
9.82+0.00
2.94.0.00
6.32s0.00
0.15a0.00
29.42a0.01
18.43:0.06
12.23=0.26 0.24a0.05
50.68+2.20
30.71=2.89
31.96200.11
5.05&1.00
45.54=2.47
29.92=1.96
20.54:2.65
3.70=1.80
DBCV
Dunn
100
100
100
80-
60
60
40
25
25
20-
7
ScarchSnispets
Biomediical
StackOverflow
WebotScience
Yahoo/Answers
Figure 8: The optimal & value found by each index on BERT representations for text datasets.
Table 25: SBERT based GMM clustering results five text datasets.
SearchSnnipets
Biomedical
StackOverflow
WehofScience
YahoolAnswers
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
26.25:8.66
20.43+6.14
35.15+1342
22.6027.93
68.51:4922
44.19=1.52
29,9247.39 39
25.6514.88
16.9G23.0
11.3622.20
47.67=17.86
31.03=15.55
22.96+6.51
14.95s4.38
16.34=8.99
12.31=9.13
48.45#9.4
35.3646.95
28.41=15.81
14.68
35.6740.01
15400.02
9.9040.00
4.8840.00
6.48=1.47
32.9640.06
22.3645.31
17.64+0.01
5.83+0.02
24.44=13.63
18.61+9.88
38.32+5.06
23.23+3.83
56.
16.17719.87
35.29=13:00
15.96=15.02
32.11+11.81
16.10:4.46
10.63+2.87
10.73a0.61
8.1820.27
47.98:1.09
29.86s0.65
70.54:3.54
55.39:2.68
50.09&18.15
11.71:0.55
7.7150.36
35.6740.01
15.15.0.02
9.90+0.00
4.8840.00
14.48s6.51
9.72+5.59
35.50+5.75
26.14:3.15
17.64=0.01
5.83+0.02
10.8720.60
8.2810.18
17.30=1.17
11.5240.54
66.2229.54
42.3325.46
30.3745.56
26.4424.10
11.4420.66
7.5320.22
10.5420.50
16.3720068
10.88s0.37
18.0020.48
11.38%0.57
7.50s0.19
CVNN
35.67=0.01
15.0020
9,90+0.00
4.8840.00
14.46*6.47
8.43+3.52
37.25#9.66
27.80+6.87
17.64=0.01
8.8300002
DCVI
10.83+0.92
8.3420.00
17.07=1.12
11.30z0.36 36
60.63+7.88
39.93=5.34
11.59#1.12
9,33#0.98
12.10=0.72
8.00=0.63
DBCV
10.6120.51
8.1120.08
17.4120.76
11.5620.74
59.76117.44
43.85212.50
11.22:0.96
8.9020.4
11.3920.56
7.5420.20
AIC
23.92+1.16
18.7540.70
31.33+1.09
16
35.87+2.84
35.11=2.28
23,08+1.53
20.95+1.02
25.25+0.99
16.81=0.57
BIC
35.67+0.01
15. 1540.02
9,90+0.00
4 4.8800000
9.88+0.02
5.96+0.42
32.9640.06
22.36+5.31
17.64+0.01
53830002
57.3014.07
43.6642.36
46.85+1.94
27.87=1.68
72.37=1.53
48.45:4.56
55.1944.24
42.0333.05
45.03+3.29
23.31=2.54
meak
100
125
125
100
80-
100
100
60
48
25
20
SearchSnigpas
Blomeiiical
StackOvarfio
WrbafScience
Figure 9: The optimal i value found by each index SBERT representations for text datasets.
20

---

Under review as a conference paper at ICLR 2023
Table 26: SimCSE based GMM clustering results on five text datasets.
SearchSnippets
Biouediccal
StackOverllow
WebufScience
YahoolAnswers
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
15.8364.84
10.15=3.17
32.91=4.88
18.4133511
61.5926.52
43.41=5.43
55.10:9.34
36.16s4.02
16.3542.34
8.28=1.13
Dunn
33.02+17.80
14.36=11.62
18.74=8.19
8.8845.44
26.20+21.59 1,9
15.46=16.05
32.04=0.01
22.47=0.00
21 70#9.6
6.84+4.06
1
31.3820.07
9.0120.08
9.7920.01
2.9910.05
7,9610.39
1.1520.30
32.0420.01
22.47.0.00
18.1420.02
5.0240.03
XB
16.25g8.42
10.77=5.87
30.73a4.27
16.05s2.12
63.6923.62
45.6420.57
39.03:6.38
26.00s3.23
16.02s2.16
8.14
S
10.49=1.13
6.76=0.43
9.79=0.01
2.99:0.05
62.7623.02
46.48=1.35
32.04:0.01
22.47=0.00
18.14+0.02
5.02:0.03
CH
31.38=0.07
9.0120.08
9.7920.01
2.99:0.05
12.76:0.07
6.8720.16
32.04:0.01
22.47=0.00
18.1420.02
5.02:0.03
DB
10.03+0.58
6.53+0.18
=4.14
8.7740.60
62.63+3.41
45.37+1.95
47.91=6.26
32.42+5.57
10.03+0.40
S.Dbw
10.06=0.60
6.50=0.14
13.53:0.61
8.39+0.31
19.82+0.58
17.14=0.33
8.70=0.33
6.85+0.66
9.98+0.43
4.85+0.19
CVNN
31.38=0.07
9.01=0.08
13.63:2.10 10
5.86+0.69
12.76=0.07
6.8700116
32,04:0.01
22.47:0.00
18. 18.14=0.02
5.02=0.03
DCVI
10.82:0.51
6.94+0.21
13.51=0.90
8.39+0.47
19.81=0.95
17.15=0.51
9.06#0.59
7.16+0.33
9.89x0.47
DBCV
10.19=0.41
6.61+0.18
13,94=0.79
8.64+0.45
19.65+0.67
9.50=0.93
7.4++055
11.92=1.91
5.91=0.98
AIC
24.42s0.688
16.24.0.67
26.95a0.12
16.24a0.11
41.91a1.33
34.70s1.16
55.65a3.13
38.40=0.60
22.73a1.23 11.32a0.63
31.38s0.07
9.01
9.79:0.01
2.99:0005
7.96s0.39
1.15
32,04.0.01
22.47.0.00
48.29:3.11
28.32+2.62
37.99=0.19
20.1330.27
67.49:2.51
46.21=1.64
46.41=3.1
35,25=2.82
100
100
100
60
75
20
ScarchSnispets
Biomeolical
StackOverflow
WeborScience
Yahwo/Answerr
Figure 10: The optimal k value found by each index SimCSE representations for text datasets.
Table 27: ViT based GMM clustering results on five image datasets.
CIFAR-10
MNIST
FashionMMIST
Image Net-10
CINIC-10
ACC
ARE
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
69.1127.08
53.0547.04
29.68 7.
10.711132
32.48+1442
20.68 42 92.27211.64
74.07=17.78
29.36+12.92
20.61=7.21
Dunn
47.27422.57
37.26420.70
27.63+0.78
9.1240.86
36.4042.84
20.57+0.59
29.66=14.00
20.69:18.36
31.0444.50
22.81+4.38
21.5644.01
13.3143.64
16.02+0.00
1.9940.00
19.8740.00
11.6640.00
23.54+5.34
7.83+5.13
19.50+0.26
11.93+241
47.29a18.92
34.67418.36
28.5020.94
10.1310.95
19.8740.00
11.6640.00
88.33a10.28
83.08214.68
33.7428.85
21.7348.83
67.8343.34
55.1262.36
16.02:0.00
1.99s0.00
30.1140.05
18.94a0.02
93.07a4.07
91.18:4.83
56.74a1.70
41.36:0.87
CII
19.7840.12
12.16:203
16.02=0.00
1.99s0.00
19.8740.00
11.6640.00
91.79a8.84
87.97=12.49
19.50:0.26
11.93+241
56.61=21.38
44.40=21.24
30.00:1.93
11.01=1.52
19.8740.00
11.6620.00
55.09240.42
43.88:48.49
20.4428.11
12.6923.87
S.Dbw
17.4842.12
16.01s2.14
7.3749.25
4.2340.15
8.8020.28
6.1920.08
68.18:25.51
61.39:24.20
16.34s2.99
12.91+2.11
CVNN
31.12216.21
19.98+12.77
16:02=0.00
1.99+0.00
19.8740.00
11.6640.00
54.54=17.79
37.92424.98
21.15+3.77
13.71+5.17
DCVI
20.8345.34 34
19.66E5.79
7.38+0.29
4.23+0.15
8.87+0.32
6.29=0.18
51.29=25.36
45.91+22.08
08
15.33e1.13
12.02+0.88
DBCV
16.3340.37
14.75=0.39
4.34+0.12
6.35#0.15
81.05a19.39
79.76s19.90
12.07+0.80
28.21*1.97
26.6742.46
16.94+0.72
11.68+0.51
33.01+3.63
36.74+4.01
21.82+0.22
19.78+0.12
12.16.2
1.99+0.00
19.8740.00
11.6640.00
19.7440.18
6.44+2.80
19.50+0.26
11.93+241
73.60+3.53
56.39:3.39
30.78=1.05
12.05+0.25
38.90=1.50
21.3240.61
80.96:9.00
73.42+16.84
59,47+4.88
38.70=5.14
meak
100
100
100
100
80
80
80
80
60
60
40-
60
40
40
20
20
20
20
2
4
CIBAN-10
MNISST
RishoonMRSS
ImageNet-10
CINIC-10
Figure 11: The optimal & value found by each index ViT representations for image datasets.
21

---

Under review as a conference paper at ICLR 2023
Table 28:
Swin based GMM clustering results on five image datasets.
CIFAR-10
MNIST
FushionMNIST
ImageNet-10
CINIC-10
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
SD
74.07=21.13 68.91425.07 28.46:00555
10.1240.60
30.82:0.04
20.60=0.09
71.76426.80
62.11434.91
64.8246.10
50.99:4.16
Dunn
19.90+20.25
36.86422.86
20.17+1.05
7.16+2.97
14.7445.20
10.06=3.56 16.6666666 43 43.87425.01 39.57+29.21
19.8940.00
14.65+0.01
1
23.84a5.40 14.0346.86
19:4920.00
4.9920.00
17.4742.30
6.73x6.26
29.87a17.30
18.27427.22
19.8920.00
14.6540.01
XB
72.37=16.35
68.17419.63
19:4940.00
4.9920.00
34.26:0.02
20.77=0.01
83.72&11.50
79.47=16.85
55.96416.19
19
43.67413.05
S
83.33=8.91
81.3647.57
19:49=0.00
4.99:0.00
24.65=0.02
15.29=0.02
89.65=9.85
91.1748.50
50.51=15.02
43.66z9.85
CH
23.8728.67
19.3248.78
19:49x0.00
4.9920.00
22.79=2.56
14.62:0.94
88.53:13.27
84.43=19.60 69
19.8910.00
14:65=0.01
DB
82.7648. 17
78.84+5.87
28.32+0.59
9,92+0.79
20.60=0.09
69.77422.93
64.12436.75
13+1.343
53.41=0744
5.Dbw
22.27+2.93
21.81+4.03
8.48+0.77
4.45+0.37
9.10=0.15
6.25+0.05
60.25+34.43
54.26+43.70
20.37+4.74
16.6+44.3
CVNN
58.54=18.89
51.50:22.25
19:49=0.00
4.99=0.00
22.79=2.56
14.62=0.94
61.81435.55
51,92+43.60 60
19.89+0.00
14.65=0.01
DCVI
21.19=1.21
20.73&1.67
8.33+0.53
4.46a0.30
9.35=0.53
59.82424.42
57.67=31.36
17.2740.42
13.75=0.43
DRCV
22. 19+2.3
21.36+2.61
8.3740.61
4.43+0.34
9.49=0.17
88.11+10,26
87.30=1
17.52+1.38
13.92+1.18
AIC
43.81a2.50
47.01a3.22
16.28x0.72
8.5020.46
21.2551.51
13.7920.80
50.88a6.23
35.38&1.05
29.84
BIC
19.9420.13
13.4646.11
19.49:0.00
4,99a0.00
6.7346.26
14.65=0.01
80.96a8.35
76.1247.75
28.42+0.74
9.84a0.73
35.63=0.63
71.76:26.80
62.11=34.91
68.57+2.84
52.89=1.34
CVNN
DBCV
DCN
Dutn
P[Curi]
100
100
80
80-
80
80-
60
60-
40
40
20
10
20-
01
CIFAR-10
RastiGMMNIST
ImageNet
CINIC
Figure 12: The optimal It value found by each index on Swin representations for image datasets.
Table 29: BEiT based GMM clustering results on five image datasets.
CIFAR-10
MNIST
ImageNNt-10
CINIC-10
ACC
ACC
ACC
ACC
ARI
SD
21.4842.12
6.2420.86 20.4120.00
5.7120.00
30.9340.00
20.1820.00
32.6721.49
16.3820.58
17.33+1.18
5.6520.30
Dunn
14.5743.94
4.8340.51
13.6246.34
8.0622.58
37.01=13.94
22.12+7.14
22.53a5.43
7.72+4.03
8.1713.26
2.92+1.07
17.44+0.01 5.17#0.01 23.3993.32
8.00=2.10
19.78+0.00
12.05=0.01
22.81
6.98+2.11
18.37:0.07
3.09 10
XB
21.75+1.02 6.1220.28 8 20. 1540.63
5.72+0.01
19.7840.00
12.05=0.01
22.81+2.87
6.98+2.11
18.77:0.02
3.40+0.01
S
17.4420.01
5.1720.01 15.00.0
5.7220.01
19.7840.00
12.05=0.01
21.5423.49
6.20=2.59
17.24=0.01
3.53+0.01
CH
17.44+0.01
5.1740.01 20.41:0.00
5.71+0.00
19.7840.00
12.05=0.01
24.10=0.07
8.07+0.07
17.24:0.01
3.53+0.01
DB
15.85+2.58
6.69=1.09
22.2443.11 11
7.12+1.92
1.22
19.7840.00
12.05=0.01
24.93=6.12
9.04+5.71
18.77-0.02
3.40+0.01
S.bb
7.26+0.38
3.15#0.10 10 9.25+0.25
6:20=0.16
12.37+0.42
9.49+0.42
11.13+0.60
6.51+0.33
6.82+0.28
2.45+0.05
CVNN
17.4410.01
5.1740.01
23.94a3.22
7.91&2.00
38.9343.04
23.02a1.09
22.49a5.22
8.22&1.54
17.2440.01
3.5340.01
DCVI
7.4910.24
3.24:0.04
9.2310.27
6.23+0.12
12.30a0.66
9.40w0.27
11.30a0.79
6.5510.39
6.81
2.4510.06
DBCV
7.73+0.71
3.30a0.17
9.32+0.42
6.31&0.19
12.3140.43
9.46+0.32
11.39a0.77
6.5740.30
8.21+1.97
2.9140.61
AIC
13.15+0.49
5.3740.20
17.7640.61
10.41=0.43
22.93+1.37
17.30=0.93
16.72a0.77
9.69+0.43
11.9240.72
4,1040.11
17.4420.01
5.1720.01
20.4120.00
5.7120.00
19.7840.00
12.0520.01
18.2920.53
5,6242.08 08
17.2420.01
3.53#0.01
24.71=0.84
8.01=0.77 23.99:3.27
8.06+2.17
37.42+1.10
34.54=1.97
14.80=1.99
23.42+0.56
7.56+1993
40C
BIC
CVNN
DBCV
08
DCV
Dunn
XB
50
5.0b
P(Ourt
roak
100
100
300
100
80
80
80
60
60
60
60
40
40
40
40
20
20
20-
20
0
2
CHIA--10
MNNIST
PastionMNUST
ImageN 1mageNet-10
CINIC-30
Figure 13: The optimal & value found by each index on BEiT representations for image datasets.
22

---

Under review as a conference paper at ICLR 2023
Table 30: BERT based GMM clustering results in terms of NMI five text datasets.
SearchSnippets
Biomedical
StackOverflow
WebofScience
VahoolAnswers
SD Dunn
37.58+0.78
26.71=0.09
9.22+1.41
36.58=0.02
7,46&1.88
29.92=9.00
13.7347.16
1.6622.67
36.36=0.80
17.0244.09
I
19.73=0.38
9,93=0.03
0,46=0.00
29.34=0.09
1.01=0.08
XB
37.21=0.62
23.74 16
1.95=0.00
29.34=0.09
1.01
S
37.28+0.53
9,93+0.03
1,95#0.00
29.34=0.09
1,01=0.08
CH
19.73+0.38
9.93+0.03
0,46:0.00
29.34=0.09
1.01=0.08
DB
37.18+0.56
25.25+2.76
6.83+0.02
36.58+0.02
3.31
S.Dbw
37.14+0.59
29.06=0.16
28.23+0.34
35.57=0.20
20.10+0.24
CVNN
19.73=0.38
16.23+3.96
0.46=0.00
29.34=0.09
8.02+2.56
DCVI
37.19+0.54
19.
28. 19+0.50
35.71
20.00
DBCV
37.25#0.49
28, 28.9940.22
28.21#0043
35.65=0.27
19 984116
AIC
37.90+1.35
28.20=0:42
22.61+0.90
36.58=0.02
19.31
BIC
19.7340.38
9,93+0.03
0,46#0.00
29.34=0.09
1.01
IP
39.7243.34
27.29H0.22
10:2420.75
39,9920.86
7.90E2.39
Table 31: SBERT based GMM clustering results in terms of NMI five text datasets.
SearchSnippets
Biomedical
StackOverflow
WebofScience
YahoolAnswers
SD Dunn
47.39+1.1
40.99
13.45=1.32
11.19=1.30
33.8 $3.8940.8 88
38.06+12.30
35.70+7.21
35.38=14.75
19.1374.77
24.87
I
22.76+0.08
15.77+0.01
26,0343.54
38.61
10.69
XB
46.07=2.45
37.98+2.70
07.6448.79
52.77=3.41
33.68+ .000
S
43.33+0.50
42.35+1.02
70.06
55.55=3.59
32.87 24
CH
22.76+0.08
15.7740.01
33.03+11.52
42.54+3.66
10.69
DB
43.3920.44
39.06+0.23
72.45+3.47
51.63=1.18
32.94
S_Dbw
43.24=0.41
38.94+0.27
56, 16+0.141
14.10=0.16
32.99 32.99+0.28
CVNN
22.76+0.08
15.7740.01
31.81=10.45
43.09
10.69
DCVI
43.3620.51
39.02=0.30
70.7842.69
44.6620.47
32,99
DBCV
43.32+0.35
39.07=0.44
67.9416.02
44.43=0.24
32.87+0.20
AIC
47.28+0.69
40.99+0.49
61.2440.70
49.89=0.57
34.9240.26
BIC
22.7620.08
15.77=0.01
24.3420.41
38.61=5.12
10.6940.04
IF
53.67#0.74
41,49E1.37
73.3611.38
53.75E1.52
31.0542.63
Table 32: SimCSE based GMM clustering results in terms of NMI on five text datasets.
SearchSnippets 36.3620074
Biomedical
SverlOverflow
WebolScience
YnseerA
SD
30.0523.48
56.7413.70
45.44=1.52
23.67
Dunn
24.91=11.65
18.6428.55 52
25.18222.26
36.68=0.03
11.93+5.44 5.44
I
14.31=0.07
9.59=0.23
3.42+0.87
36.68:0.03
9.4920.05
XB
37.34
28.08
58.4410.42
40.85=3.82
23.62+0.27
S
36.53+0.43
9.5920.23
58.7940.62
36.68=0.03
9.4920.05
CH
14.31=0.07
9.5920.23
17.38+0.35
36.68=0.03
9.4920.05
DB
36.64=0.49
31.99+0.16
58.22+1.31
14.47=1.17
23.28
S_Dbw
36.52=0.41
31.98=0.15
51.26+0025
39.13=0.16
23.17+0.26
CVNN
14.31=0.07
15.47=1.66
17.38+0.35
36.68=0.03
9.4920.05
DCVI
36.58+0.27
31.95+0.18
51.31+0.17
39.33=0.26
23 23.23
DBCV
36.4920.45
32.01=0.17
51.2240.20
39.43=0.33
23. 23.3440.30
AIC
37.8120.82
32.03=0.22
55.6740.49
45.72=1.02
23.95+0.36
BIC
14.3140.07
9,5926.23
3,42+0.87
36.68:0.03
9,49=0.05
IP
37.53+2.35
31.62+0.16
$9.04:1.18
44.09=0.67
19.74+2.35
D.3 THE CUSTERING RESULTS ON AHC
In this section, we only use AHC evaluate the clustering results. The evaluation results based on
external indices of ACC, ARI and NMI are shown in Table 36 results highlighted in bold. Moreover, the optimal & each index select, i.e. optæ, and the true
37
38
39
40 and 41 where the best
cluster number each dataset, i.c. dataset-k, are provided in table. Obviously, for almost all cases, IP index outperforms other indices and close to the real l value.
23

---

Under review as a conference paper at ICLR 2023
Table 33: VIT based GMM clustering results in terms of NMI five image datasets.
CIFAR-10
MNIST
MNSSi
ImageNet-10
CINIC-10
SD
10+2773.
19.2742.05
31.80+2.09
88.9745.79
48.07=2.57
Dunn
54.67=16.57
16.84+1.01
33.67=1.32
45.84=22.36
44.33=3.45
I
84.3945.86
3.69=0.00
26.08:0.02
31.42=11.87
29.52=1.71
XB
14.94=14.94
17.74 14
26.08+0.02
91.94+4.53
43,49+9.78
S
68 08.52#1.01
3.69=0.00
29 09.6520.06
93.39+2.09
55.65=0.73
CH
31.93+1.09
3.69=0.00
26.08=0.02
93.29+3.84
29.52=1.71
DB
60.54+17.25
19.38+2.
26.08+0.02
58.61=36.52
41.16+8.23
S.Dbw
53.23+1.10
23.85+0.17
33.3620.22
83.8449.27
15.17=1.16
CYNN
42.37=15.17
3.69#0.00
26.0820.02
68.65=15.96
31,99=6.20
DCVI
54.7042.32
23.77+0. II
33 13.4
69.08+22.06
44.67=0.44
DBCV
52.47=0.299
23.8240.2 24
33.41+0.31
87.9248.31
44.53=0.24
AIC
57.69+0.95
22.90+0.32
33.89+0.59
72.87+1033
48.74=0.35
BIC
31,93+1009
3.69+0.00
26.08+0.02
27.37+6.07
29.52+1.71
IP
69.5821.44
70.39H0.70
34.0920.54
88.1744.96
54,4322.33
Table 34: SWin based GMM clustering results terms of NMI on five image datasets.
CIFAR-10
MNIST
FushionMNIST
ImageNet-10
CINIC-10
SD
81.29=11.95
15.4620.88 89
34.8420.11
81.69=22.68
61,3622.92
Dunn
60.62=15.95
13.06+4.8
33.67=0.37
63.54=26.27
31.41=0.02
37.72+11.88
9.56#0.00
15.88=16.19
40.07=26.62
31.41=0.02
XB
81.8247.77
9.5620.00
35.7020.16
92.8645.52
57.20=8.49
S
86,39:2.44
9,56=0.00
29.63+0.02
95.33+3.89
57.42=6.34
CH
43.57=11.96
9.56+0.00
31.23+2.17
94.35+6.23
31.41=0.02
DB
86.36:2.29
15.16#1.00 04
34.8420.11
78.83=30.79
63.05=1.19
S Dbw
64.05+1.55
19.84+0.34
33.34+0.22
69,50=33.65
50.15=1.84
CVNN
74.76
9.56#0.00
31.23+2.17
69,56=33.83
31.41=0.02
DCVI
63.6920.82
19.8620.27
33.4720.19
75.52=28.87
48.98=0.52
DBCV
63.83+1.10
19.87
33.54+0.22
95,25+3.81
48.98=0.70
AIC
73.79+1.01
18.35+0.23
34,68+0.31
79.9 79.93+2.47
54,39=0.22
BIC
35.0829.18
9.56+0.00
15.88=16.19
32.40=11.47
31.41=0.22
85.57$2.73
15.0540.82
34. 10
81.69=22.68
62.3620.7T
Table 35: BEiT based GMM clustering results in terms of NMI on five image datascts.
CIFAR-10
MNIST
FashionMNIST
ImageNet-10 28.39=0.65
CINIC-10
SD
116000.72
13.57=0.00
36.40=0.01
13.03=00.29
Dunn
12.74s4.29
29.10+2.33
41.33 1.25
20.15
14.5&+0.57
9.61#0.02
18.8743.23
24.75=0.02
20.27+3.51
8.3240.10
XB
11.07:0.01
14.29+0.98
24.75=0.02
20,2743.51
7.97+0.03
9.61
14.29+0.98
24.75=0.02
17.97+5.46
7.44+0.02
CII
9.61#0.02
13.57+0.00
24.75=0.02
21.83+0.02
7,44=0.02
DB
17.11=0.27
17.48=3.79
24.75=0.02
21.61=5.15
7.97=0.03
S_Dbw
17.72+0.12
30.8220228
10.57=0.13
18.51
14.81
CVNN
9.61#0.02
17.5423.63
39,58=1.34
17.8425.44 4.44
7.44+0.02
DCVI
17.6840.08
30.77=0.24
40.49=0.19
28.53=0.23
14.82+0.13
DBCV
17.73+0.05
30.72+0.23
40.55a0.14
28.5320.15
14.60A0.39
AIC
16.8820.28
27.92=0.64
41.3920.36
28.5120.29
13.8420.12
BIC
9.61=0.02
13.57=0.00
24.75=0.02
13.19=1.59
7.44=0.02
IP
13.62+1.10
17.37+3.99
39.52-0.45
25.39=1.40
13.03+0.86
Table 36: BERT based AHC clustering results on five datasets.
SearchSnippels
Biomedical
StackOverflow
WebofSccience 7
YahoolAnswers-
ACC
ARI
NMI
opts
ACC
ARI
NMI
opfe
ACC
ARI
NMI
opti
ACC
ARI
NMI
opts
ACC
ARI
NMI
opta
SD
13.08
8.07
38.17
98
25.61
11.87
22.53
10.86
1.56
7.69
42.58
25.51
40.41
18.5
2.77
6.42
3
Dunn
19.21
12.5
39.08
12.26
6.93
27.95
141
7.16
28.78
20.48
13.75
36.2
50
29.73
9.68
16.78
15
37.27
12.44
24.83
9.49
2.77
8.89
0.18
0.64
31.52
21.39
34.22
2
18.3
2.75
6.38
4
XB
13.08
8.07
38.17
98
28.52
13.24
25.08
10
10.66
1.56
7.69
31.52
21.39
34.22
2
18.5
2.77
6.42
3
30.79
8.77
18.2
2.77
8.89
0.7
3.12
31.52
21.39
34.22
15.76
2.06
3.84
2
CH
30.79
8.77
18.2
9.49
2.77
8.89
0.18
31.52
21.39
34.22
15.76
2.06
3.84
2
7.06
37.96
25.08
42.58
10.75
7.02
37.92
7.22
27.84
7.16
9.69
20.66
37.89
13.94
31.52
21.39
37.92
27.95
7.02
37.92
27.95
9.69
31.52
21.39
15.76
31.52
21.39
15.76
3.95
24

---

Under review as a conference paper at ICLR 2023
Table 37: SBERT based AHC clustering results on five text datasets.
SearchSnippets
8
Biomediccal
- 20
StackOverllow
WebufScience
7
YahoolAmswers
10
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
SD
35.57
25.65
49.98
32
32.46
19.31
33.88
42
33.66
10.43
50.1
8
30.12
24.23 47.3
29
29.65
16.86
32.17
28
Dunn
36.09
15.29
28.18
2
9.79
3.03
11.07
2
9.55
3.05
19:76
2
32.66
23.76
38.21
18.08
5.49
11.96
1 36.09
15.29
28.18
2
13.73
5.06
16.87
3
13.4
3.29
25.39
3
32.66
23.76 38.21
18.08
5.49
11.96
XB
14.95
9.61 44.41
111
17
6.06
19.86
4
33.66
10.43
50.1
8
56.46
44.25 52.39
13
17.66
10.13
31.23
S 71.99
53.75
58.41
10
9.79
3.03
11.07
2
62.84
46.21
60.04
26
42.84
50.05
48.68
24.18
33,95
CH
36.09
15.29
28.18
2
9.79
3.03
11.07
2
9.55
3.05
19.76
32.66
23.76
38.21
18.08
5.49
11.96
DB
14,95
9.61
44.41
111
17.76
10.74
34.44
141
26.01
7.29
42.83
44.25
52.39
13.53
7.44
30.59
S.Dbw 14.95
9.61
44.41
111
17.76
10.74
34.44
138
51.75
17.02
59.97
13
11.96
8.44
41.59
109
13.53
7.44
30.57
100
CVNN 36.09
15.29
28.18
2
9.79
3.03
11.07
2
21.72
5.74
37.4
5
32.66
23.76
38.21
18.08
5.49
11.96
DCVI 14.95
9.67
44.42
17.76
10.74
34.44
141
26.93
24.17
53.42
11.96
8.44
41.59
109
13.53
7.44
30.57
DBCV
14.95
9.61
111
17.76
10.74
34.44
141
26.93
24.17
53.42
11.96
8.44
41.59
109
13.53
7.44
30.57
AIC 36.09
15.29
28.18
2
9.79
3.03
11.07
9.55
3.05
19.76
32.66
38.21
11.96
BIC 36.09
15.29
28.18
9.79
3.03
11.07
2
9.55
3.05
19.76
23.76
38.21
11.96
IP
61.01
33.17
24.4
8.8
25.89
54,99
19.35
61.4
42.84
50.05
30.29
Table 38: SimCSE based AHC clustering results on five text datasets.
SearchSnippees 8
Biomedical - 20
StackOverflow
20
WeboofSeience 7
VasoolAnsswrss- 10
ACC
ARI
NM1
opla
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI opt..
SD
12.1
6.77
111
16.26
4.07
16.28
35.94
26.34
50.39
76
50.25
33.08
44.24
10
25.96
4.37
17.99
Dunn
48.7
26.19
40.03
12
17.69
10.48
29.97
112
8.97
2.31
15.85
2
28.92
19.04
30.28
2
16.36
1.74
8.25
43.23
14.67
28.97
4
12.68
3.59
12.91
3
12.86
4.38
25.03
3
28.92
19.04
30.28
2
21.54
3,21
13.79
XB
12.1
6.77
111
16.26
4.07
16.28
4
8.97
2.31
15.85
2
40.13
24.49
38.35
3
16.36
1.74
8.25
S
12.1
6.77
37
111
9.7
2.97
9.91
2
54.28
36,8
53.09
30
28.92
19.04
30.28
2
16.36
1.74
8.25
CH
32.21
8.82
15:97
2
9.7
2.97
9.91
2
8.97
2.31
15.85
2
28.92
19.04
30.28
2
16.36
1.74
8.25
DB
12.1
6.77
111
16.26
4.07
16.28
57.99
23.79
54.46
19
50.25
33.08
44.24
10
9.55
4.41
21.78
100
5_Dbw
12.1
6.77
14.44
8.69
30.09
24.81
19.33
48.93
141
10.79
7.28
38.48
109
4.65
21.84
CVNN
32.21
8.82
15.97
9.7
2.97
9.91
2
8.97
2.31
15.85
2
28.92
19.04
30.28
2
21.54
3.21
13.79
DCVI
12.1
6.81
37.01
110
14.44
8.69
30.09
141
24.81
19.33
48.93
141
10.79
7.28
38.48
109
9.55
4.41
21.78
100
DBCV
12.1
6.81
37.01
14.44
8.69
30.09
141
24.81
19.33
48.93
141
10.79
7.28
38.48
109
4.41
21.78
AIC
32.21
15.97
9.7
2.97
9.91
2
8.97
2.31
15.85
2
28,92
19.04
30.28
16.36
8.25
BIC
32.21
9.7
2.97
8.97
2,31
15.85
2
28.92
19.04
30.28
16.36
1.74
8.25
38.82
13.99
25.74
28.86
13.73
25.62
II
36.31
15.93
43.76
10
$2.45
34.44
44.56
21.54
3.21
13.79
Table 39: ViT based AHC clustering results on five image datasets.
CIFAR-10-
10
MNIST -
10
FashionMNIST 10
ImageNet-10- 10
CINIC-10- 10
ACC
ART
NMT
opty
ACC
ART
NMI
optz
ACC
ART
NMT
opty
ACC
ART
NMI
opt,
ACC
ART
NMT
opty.
SD
26.44
10.03
36.34
3
31,07
11.74
19.67
9
30,38
16.69
35.1
4
87.52
82.8
91.19
61.4
35.04
52.82
10
Dunn
50.37
38.95
56.87
6
29.57
10.74
17.82
6
30,38
16.69
35.1
4
19.85
9.56
33.86
2
18:43
6:32
23.59
1 18.47
6.02
24.6
2
23.52
5.43
9.22
4
19.79
9.91
28.48
2
29.65
16.54
49.2
18.43
6:32
23.59
XB
26.44
10.03
36.34
3
21.82
3.78
8.22
3
19.79
9.91
28.48
2
87.52
82.8
91.19
18.43
6.32
23.59
S
60.92
49.9
63.19
16
16.88
2.69
5.33
2
19.79
9.91
28.48
2
91.85
92.36
92.99
12
57.36
35.96
52.69
CH
18.47
6.02
24.6
2
16.88
2.69
5.33
2
19.79
9.91
28.48
2
87.52
82.8
91.19
18.43
6:32
23.59
DB
26.44
10.03
36.34
3
30.01
11.45
19.61
8
19.79
9.91
28.48
2
87.52
82.8
91.19
18.43
6.32
23.59
S.Dbw 22.16
18.31
52.76
81
8.81
4.76
24.04
99
9.96
6.26
32.55
99
87.52
82.8
91.19
17.01
12.04
43.28
100
CVNN
26.44
10.03
36.34
3
16,88
2.69
5.33
2
22.97
11.36
27.83
3
59.23
49.23
78.3
6
18.43
6.32
23.59
DCVI
50.37
38.95
56.87
6
8.81
4.71
24.02
100
9.96
6.16
32.52
100
39.58
25.55
61.52
17.06
12.18
43.31
99
DBCV
20.
17
16.16
51.81
100
8.81
4.71
24.02
100
9.96
6.16
32.52
100
91.85
92.36
92.99
12
17.01
12.04
43.28
100
AIC
18.47
6.02
24.6
2
16.88
2.69
5.33
2
19.79
9.91
28.48
2
19.85
9.56
33.86
2
18.43
6.32
23.59
BIC
18.47
6.02
24.6
2
16.88
2.69
5.33
2
19.79
9.91
28.48
2
19.85
9.56
33.86
18.43
6.32
23.59
IP
70.92
51.77
64.26
10
29.37
10.74
17.82
6
38.57
22.17
34.24
8
87.52
82.8
91.19
9
46.74
28.84
48.71
7
D.4 THE CUSTERING RESULTS ON DBSCAN
In this section, we only use DBSCAN to evaluate the clustering results. The evaluation results based on external indices of ACC, ARI and NMI are shown in Table 42 45
45
46 and 47 where the
best results are highlighted in bold and Top-3 best results are underlined. Moreover, the optimal k each index select, i.c. opt and the true cluster number in each dataset, i.e dataset-k, are provided in table. Our index is either on par or slightly better than competing indices.
25

---

Under review as a conference paper at ICLR 2023
Table 40: Swin based AHC clustering results on five image datasets.
CHEAR-10-
10
MNIST 10
FashionMNIST 10
ImugeNet-10-
CINIC-10
10
ACC
ARI
NMI
opla
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opla.
SD
85.92
80.08
86.43
9
19.36
4.09
7.85
3
31.8
21.96 37.75
99.66
99.25 98.94
10
64.77
47.09
58,26
12
Dunn
19.94
18.12
42.4
2
23.54
6.38
9.96
4
19.97
13.79 34,16
2
39.93
18.49 57.59
19.35
14.62
33.03
19.94
18.12
42.4
2
16.93
2.01
4.46
2
19.97
13.79
34.16
2
29.93
10.73 43.02
19.35
14.62
33,03
XB
85.92
80.08
86.43
9
16.93
2.01
4.46
2
34.43
20.54
36.51
5
99.66
99.25 98.94
10
27.79
18.64
43.35
89
83.9
85.83
11
16.93
2.01
4.46
2
19.97
13.79 34,16
2
91.03
91.92 94.97
14
43.62
38.8
52.97
CH
19.94
18.12
42.4
16.93
2.01 4.46
2
19.97
13.79 34.16
2
99.66
99.25 98.94
10
19.35
14.62
33.03
DB
85.92
80.08
86.43
16.93
20.54
36.51
98.94
27.79
18.64
43.35
5.Dbw
46.3
46.79
72.51
8.66
4.35
19.86
99
9.81
6.36
33.21
100
99.25
98.94
10
20.7
15:57
48.79
CVNN
76.7
72.92 83,99
16.93
2.01
4.46
2
19.97
13.79
34,16
2
99.25
98.94
10
27.79
18.64
43.35
DCVI
21.2
8.66
6.36
33,21
100
18.49
57.59
14.08
DBCV
19.94
8.66
100
6.36
33,21
100
98.94
14.08
AIC
19.94
16.93
4.46
19.97
13.79
34.16
35.28
14.62
33,03
BIC
19.94
16.93
4.46
19.97
13.79
34.16
35.28
33.03
93.62 86.56
24.03
35.03
21.09
35.54
98.94
57.78
Table 41: BETT based AHC clustering results on five image datasets.
CIFAR-10-
10
MNIST 10
MNShion
- 10
ImageNet-10-
- 10
CINIC-10- 10
ACC
ARI
NMI
opta
ACC
ARI
NMI
opla
ACC
ARI
NMI
opta
ACC
ARI
NMI
opt.
ACC
ARI
NMI
opte
SD
20.67
6.21
9.42
5
20.87
6.04
12.07
34.74
21.87
38.85
16.54
3.92
11.21
2
17.19
3.82
6.93
2
Dunn
10.19
4.14
16.66
79
11.17
6.93
32.16
100
15.48
11.7
41,01
84
20.45
4.67
16.25
3
7.55
2.34
13.6
100
1
16.96
3.96
7.56
2
22.61
5.9
18.2
19.9
12.47
25.97
20.45
4.67
16.25
3
17.88
3.5
8.46
3
XB
17.59
3.85
7.53
3
20.87
6.04
12.07
19.9
12.47
25.97
2
20.45
4.67
16.25
3
17.19
3,82
6.93
2
5 16.96
3.96
7.56
2
20.87
6.04
12.07
19.9
12:47
25.97
2
20.45
4.67
16.25
3
17.19
3,82
6.93
2
CH
16.96
3.96
7.56
2
20.87
6.04
12.07
19.9
12.47
25.97
2
16.54
3.92
11.21
2
17.19
3.82
6.93
2
DB
8.44
3.37
17.15
98
22.61
5.9
18.2
19.9
12.47
25.97
20.45
4.67
16.25
3
9.22
2.79
12.96
73
S.Dbw
8.16
3.31
17.2
100
11.17
6.93
32.16
100
13.48
10.15
40.73
97
12.78
6.87
28.54
113
7.55
2.34
13.58
99
CVNN
16.96
3.96
7.56
2
20.87
6.04
12.07
42.72
23.03
42.06
10
16.54
3.92
11.21
2
17.19
3.82
6.93
2
DCVI
8.16
3.31
17.2
100
11.17
6.93
32.16
100
12.6
9.71
40.64
100
12.78
6.87
28.54
114
7.55
2.34
13.6
100
DBCV
8.16
3.31
17.2
100
11.17
6.93
32.16
100
12.6
9.71
40.64
100
12.78
6.87
28.54
114
15.06
4.36
12.15
29
AIC
16.96
3.96
7.56
2
20.87
6.04
12.07
19.9
12.47
25.97
16.54
3.92 11.21
2
17.19
3.82
6.93
2
BIC
16.96
3,96
7.56
2
20.87
6.04
12.07
19.9
12.47
25.97
16.54
3.92
11.21
2
17.19
3,82
6.93
2
IP
25.23
7.05
12.6
8
23.85
6.48
18.36
37.62
20.75
38.08
32,19
15.9
26.49
14
21.17
4.69
9.88
7
Table 42: BERT based DBSCAN clustering results on five text datasets.
SearchSnippets -
Biomedical 20
StackOverflow 20
WebofSeience -7
Yahoo! 10
ACC
ARI
NMI opt, ACC ARI NMI opt,
ACC
ARI NMI
opt,
ACC ARI NMI opl,
ACC ARI NMI
opiz.
SD
21.56
0
0.01
2
5
0
0.01
5.01
0
0.01
2
17.6
0
0.02
2
10.01
0
0.02
2
Dunn
21.56
0
0.01
2
5
0
0.01
5.01
0
0.01
2
17.61
0
0.02
2
10.04
0
0.05
2
1 21.56
0
0.01
2
0
0.01
5.01
0
0.01
2
17.6
0
0.02
2
10.01
0
0.02
2
XB
21.56
0
0.01
2
5
0
0.01
5.01
0
0.01
2
17.6
0
0.02
2
10.01
0
0.02
2
S 21.56
0
0.01
2
0
0.01
5.01
0.01
2
17.6
0.02
10.01
0
0.02
2
CH
24.63
0.29
5.39
2
9.22
1.45
5.27
6.45
0.28
0.91
2
25.86
3.49
14.52
11.86
0.18
0.94
2
DB
21.56
0
0.01
2
0
0.01
5.01
0.01
2
17.6
0.02
10.01
0
0.02
2
Dbw
21.64
0.01
0.18
3
0
0.01
5,01
0.01
2
17.6
0.02
10.05
0.01
0.07
3
CVNN
21.67
0.01
1.01
14
5.34
0
0.66
5.62
0.02
0.43
2
17.85
0.02
0.46
10.04
0
0.05
2
DCVI
21.56
0
0.01
2
0
0.01
5.01
0.01
2
17.6
0.02
10.01
0
0.02
DBCV 23.41
0.5
1.13
3
8.28
0.74
3.87
5.95
0.06
0.58
3
19.43
0.14
0.8
10.01
0
0.02
2
AIC 21.56
2
5.01
0.03
5.01
0.01
2
17.6
0.02
10.02
BIC 21.56
0
2
5.01
0.03
5.01
0.01
2
17.6
0.02
10.02
0
0.04
IP 21.74
-0.38
4.34
2
9.23
1.36
7.2
5.38
0.01
0.39
2
18.38
0.05
0.34
13.65
0.95
2.86
2
26

---

Under review as a conference paper at ICLR 2023
Table 43: SBERT based DBSCAN clustering results on five datasets.
SearchSSnppels-
Biomedical
20
StackOverllow
WebofScience
7
YahoolAnwwers
ACC
ARI
NMI
opty
ACC
ARI
NMI
opta
ACC
ARI
NMI
opts
ACC
ARI
NMI
opta
ACC
ARI
NMI
opt,
SD
21.89
0.06
1.26
13
6.24
0.03
3.8
5.11
0
0.38
18.92
0.00
6.77
12
10.4
1.22
14
Dunn
21.58
0.01
0.07
2
0
0.02
5.01
0
0.01
17.62
0.02
2
10.01
0.02
2
1
21.55
0.01
2
0
0.19
5.01
0
0.01
17.62
0.02
2
10.01
0.02
2
XB
21.55
0.01
2
5.04
0
0.09
5.01
0
0.01
17.62
D
0.02
2
10.01
0.02
2
21.56
2
0.02
5.01
0
17.62
10.01
0.02
2
CH
24.46
2.18
4.37
13.08
4.39
17.73
26.05
4.78
37.69
42.56
11.19
35,83
21.33
3.23
14.61
0.09
5.01
17.62
10.01
S.Dbw 21.6
1.46
0
17.62
10.04
CVNN
21.87
0.01
0.03
2.27
5.13
0
22.97
0.51
14.77
10.04
0.04
2
DCVI
17.62
10.01
21.49
0.76
13.8
21.17
2.26
16.78
12.52
0.06
BIC
17.62
21.9
0.08
11.96
Table 44: SimCSE based DBSCAN clustering results on five text datasets.
SearchSnippets
Biomedical
20
StackOverflow
20
WebofScience 7
Yahoo!Answers: 10
ACC
ARI
NMI
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
ACC
ARI NMI
opt,
SD
21.65
0.05
1.03
8.86
0.25
6.64
0.02
10
17.62
0.02
0.02
Dunn
21.56
0
0.01
0
0,02
5.02
0.03
2
17.62
0.02
10.01
0.02
21.56
0
0.01
0
0.02
5.04
0.08
2
17.62
0.02
10.01
0
0.02
XB
21.56
0
0.01
0
0.02
5.04
0.08
2
17.62
0.02
10.01
0.02
S
21.56
0
0.01
5.03
0
0.08
5.02
0.06
2
17.62
0.02
10.01
0
0.02
CH
26.29
3,09
5.79
9.18
2.27
7.02
7.54
0.37
1.77
2
26.46
6.05
15.68
14.78
0.95
6.26
DB
21.56
0
0.01
5.1
0.43
5.04
0.08
2
17.62
0.02
10.01
U 0.02
S.Dbw
21.6
0
0.09
6.1
0.03
2.85
5.45
1.12
14
17.62
0,02
10.04
0.01
0.07
CVNN
21.65
-0.01
0,52
9.15
0.34
7.9
8.86
0.21
7.41
4
18.13
0.04
1.01
2
10.03
0
0.04
2
DCVI
21.56
0
0.01
2
0
0.02
6.33
0.01
3.89
42
17.62
0.02
10.01
0
0.02
2
DRCV
21.04
0.27
13.25
114
8.91
1.93
6.03
9.29
1,65
5.86
3
18.9
0.27
0.83
3
10.09
0
0.2
4
AIC
21.56
0
0.01
2
0
0.02
5.02
0
0.03
2
17.62
0.02
2
10.01
0
0.02
2
BIC
21.56
0
0.01
2
0
0.02
2
5.02
0
0.03
2
17.62
0
0.02
2
10.01
0
0.02
2
IP
21.6
-0.29
1.88
11.31
0.99
12.65
5.4
0.01
0.31
2
25.9
5.66
15.16
14.72
0.92
6.19
2
Table 45: ViT based DBSCAN clustering results on five image datasets.
CIFAR-10-1
10
MNIST 10
FashionMNISTT 10
ImageNet-10- 10
CINIC-I0- 10
ACC
ARI
NMI
opte
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt,
ACC
ARI
NMI
optz
ACC
ARI
NMI
opto
SD
10.49
0
1.39
7
11.34
0.02
2
10.01
0.02
2
34.34
5.11
36.03
9
10.01
0
0.02
2
Dunn
10.01
0
0.02
2
11.34
0.02
2
10.01
0.02
2
10.03
0
0.06
2
10.01
0
0.02
2
10.01
0
0.02
2
11.34
0.02
2
10.01
0.02
2
10.04
0
0.08
2
10.01
0
0.02
XB
10.01
0
0.02
2
11.34
0.02
2
10.01
0.02
2
10.04
0
0.08
2
10.01
0
0.02
2
S
10.01
0.02
2
11.34
0.02
2
10.02
0.04
2
75.64
62.69
79.51
10
10.01
0.02
CH
32.68
16.62
40.09
5
15.55
0.82
3.99
2
19.68
3.19
12.92
3
75.63
59.05
78.06
9
38.18
13.33
42.86
DB
10.01
0.02
2
11.34
0.02
10.01
0.02
2
10.04
0
0.08
2
10.01
0.02
D.bb
11.52
0.02
4.32
16
11.34
0.02
10.01
0.02
2
20.41
1.38
18.56
14
10.03
0.08
CVNN
11.09
0.02
2.75
7
12.95
0.12
2.46
12.82
0.34
4.56
3
15.5
0.43
10.18
10.86
0.03
1.71
DCVI
10.01
0.02
2
11.34
0.02
10.01
0.02
2
10.03
0
0.06
2
10.01
0.02
DBCV
11.46
0.13
1.51
3
14.66
1.24
3.17
15.69
1.79
4.44
3
67.96
45.14
71.31
10
28.71
4.25
30.43
AIC
10.01
11.35
0.03
10.01
2
0.06
10.02
0.04
BIC
10.01
11.35
0.03
10.01
0.03
2
10.03
0
0.06
10.02
0.04
37.38
11.91
44.58
14.52
3.28
12.48
0.39
4.11
2
53.08
17.88
56.16
25.01
3.63
26.38
27

---

Under review as a conference paper at ICLR 2023
Table 46:
Swin based DBSCAN clustering results on five image datasets,
CIFAR-10
MNIST
10
FishionMNIST
ImageNet-100
10
CINIC-10
10
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opt;
ACC
ARI
NMI
opty,
ACC
ARI
NMI
opts
ACC
ARI
NMI
opta
SD
10.01
0.02
15.31
0.78
8.24
10.01
0.02
56.1
21.67
59.71
10.01
0.02
2
Donn
10.01
0.02
11.37
0.04
10.01
0
0.02
10.02
0.03
10.01
0
0.02
2
10.01
0.02
11.37
0.04
10.01
0
0.02
10.02
0.03
10.01
0
0,02
2
10.01
0.02
11.37
0.04
10.01
0
0.02
10.02
0.03
10.01
0,02
2
10.01
0.02
11.37
0.04
10.04
0.08
10.02
10.04
2
33.74
18.58
42.17
18.98
5.89
19.23
3,09
16.7
73.78
43.41
73.46
14.56
1.27
2.48
2
10.01
13.62
7:21
13.77
6.63
10.52
13.92
8.88
10.03
2
CVNN
12.9
0.11
5.5
13.63
6.37
11.31
0.03
2.25
12.82
14.87
0.27
9.23
0.07
10.01
23.44
1.51
23.05
13.68
0.36
7.42
57.78
22.38
1.95
10.04
42.17
8.57
Table 47: BEiT based DBSCAN clustering results on five image datasets.
CIFAR-10 10
MNIST 10
FashionMNIST 10
ImageNet-100
CINIC-10 10
ACC
ARI
NMI
opde
ACC
ARI
NMI
opl;
ACC
ARI
NMI
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI
opt
SD
10.01
0
0.02
2
11.36
0.02
2
10.01
0
0.02
2
10.01
0.02
2
10.01
0.02
2
Dunn
10.01
0
0.02
2
11.38
0.06
2
10.01
0
0.02
2
10.01
0.02
2
10.03
0.08
2
10.01
0
0,02
2
11.36
0.02
2
10.01
0
0,02
2
10,01
0.02
2
10.01
0.02
2
XB
10.01
0
0.02
2
11.36
0.02
2
10.01
0
0,02
2
10.01
0.02
2
10.01
0.02
2
10.01
0
0.02
2
11.36
0.02
2
10.01
0
0.02
2
10.01
0.02
2
10.02
0.04
2
CH
14.26
1.24
2.71
2
11.77
2.11
2
25.11
15.58
33.44
3
17.05
2.29
8.19
2
13.08
0.54
2.69
2
DB
10.01
0
0.02
2
11.36
0.02
2
10.01
0
0,02
2
10.01
0.02
2
10.01
0.02
2
S_Dbw
10.01
0
0.02
2
10:58
-0.04
1.72
12.08
0.23
6.15
20
11.46
0.05
3.18
12
10.03
0.08
2
CVNN
10.18
0
0.36
2
20.73
2.42
14.48
21.9
2,85
22.8
6
14.45
0.72
5.73
3
11.32
0.07
0.99
2
DCVI
10.01
0
0.02
2
11.36
0.02
10.01
0
0.02
10.01
0.02
10.01
0.02
2
DBCV
12.28
0:23
1.62
12.05
0.05
1.42
10.18
0
0.19
3
13.15
0.39
3.34
10.01
0.03
2
AIC
10.01
0
0.03
2
11.36
0.02
10.01
0
0,03
2
10.02
0.03
2
10.01
0.03
2
BIC
10.01
0
0.03
2
11.36
0.02
10.01
0,03
2
10.02
0.03
2
10.01
0.03
13:03
0.55
2.73
2
20.73
2.42
14.48
26.65
15.04
31,26
3
14.01
0.65
5.23
2
13.04
0.57
1.95
2
E EVALUATION RESULTS ON UCI DATASETS
In this section, we compare our index with other 13 indices five real-world datasets from UCI Repository (Frank 2010). Table 48 lists the basic statistics of these datasets. The evaluation results based on external indices ACC, ARI and NMI are shown Table 49] where the best results are highlighted in bold and the optimal & each index select, i.e. opłR, and the true cluster number in each dataset, i.e. dataset-kk, are provided. Our index is either on par or slightly better than competing indices.
Table 48: Statistics of UCI datasets.
Dataset Wine
Samples
Attributes
Classes
178
13
3
Satimage
6,435
36
6
Ecoli
336
7
8
ControlChart
600
60
Letter
20,000
16
26
28

---

Under review as a conference paper at ICLR 2023
Table 49; Clustering results in terms of ACC and ARI five UCI datasets.
Wine 3
Satimage- 6
Ecoli 88
ChntrolChart
6
Letter 26
ACC
ARI
NM1
opt,
ACC
ARI
NMI
opts
ACC
ARI
NMI
oply
ACC
ARI
NMI
opt,
ACC
ARI
NMI opla.
SD
100
100
100
2
99.72
99.56
99.18
79.63
75.43
78.02
2
100
100
100
83.7
77.01
81.24
Dunn
69.23
59.72
74.63
2
99.51
99.22
98.87
4
83.91
80.1
78.57
71.94
59.43
70.04
69.29
65.8
88.24
100
100
100
2
100
100
100
3
60.87
44.64
63.09
100
100
100
49.32
16.53
41.27
XB
100
100
100
2
100
100
100
3
81.63
86.46
80.94
2
100
100
100
33.92
14.59
69.3
S
100
100
100
2
100
100
100
84.81
80.41
100
33,92
14:59
69.3
CH
97.85
95.98
94.03
2
99.56
99.28
98. 77
79.63
75.43
78.02
80.99
69.04
74.05
33.92
14.59
69.3
DB
100
86.46
100
S_Dbw
100
100
51.02
69.19
78.02
41.27
CVNN
100
100
100
100
100
86.46
80.94
100
74.58
100
100
86.46
80.94
DBCV
100
80.41
0,02
3.39
10.11
39.49
0.02
0.15
3.39
39.49
100
80.41
100
F TF-IDF BASED CLUSTERING RESULTS
Table 50: TF-IDF based clustering results in terms of ACC and ARI on five text datasets.
SearchSnippets 8
Biomedical
20
StackOverflow
20
WebofSeience- 7
ACC
ART
NMI
opta
ACC
ARI
NMI
opts
ACC
ARI
NMI
opt,
ACC
ARI NMI
opts
ACC
ARI
NMI
opts.
SD
70.09
55.
81.01
30.77
1.32
30.4
10
64.53
30.59
65.53
15
17.8
0.05
21.14
0.24
9.04
Dunn
86.76
80.73
81.3
87.17
74.01
68.15
2
64.53
30.59
65.53
15
91,52
87.06
86.94
21.36
0.41
1.21
62.5
31.93
56.09
31.31
-211
14.99
2
58.08
25.7
57.4
8
18.01
0.09
21.71
0.41
2.11
XB
62.5
31.93
56.09
31.31
-2.1
14.99
2
51.77
21.39
50.17
6
17.8
0.05
21.71
0.41
2.81
14.37
9.43
53.9
167
17.14
8.16
31.47
135
30.01
16.06
39.78
132
11.46
7.05
35.63
103
16.88
0.99
16.02
CHI
62.5
31.93
56.09
31.31
2.1
14.99
2
58.57
29.24
66
31
32.55
23.71
38.76
21.14
0.24
9.04
DB
62.5
31.93
$6.09
30.77
1.32
30.4
10
58.08
25.7
57.4
8
17.8
0.05 0
2
21.71
0.41
2.81
Dbw
62.5
31.3
56.09
31.31
22.1
14.99
2
34.06
5.7
25.48
2
17.8
0
0.05
21.71
0.41
2.81
CVNN
19.76
-0.3
10.42
51
31.31
-2.1
14.99
2
51.77
21.39
50.17
6
17.8
0
0.05
2
21.71
0.41
2.81
DCVI
62.5
31.93
56.09
2
31.31
-2.1
14.99
2
58.08
25.7
57.4
8
17.8
0
0.05
2
21.71
0,41
2.81
DBCV
14.37
9.43
53.9
167
86.54
72.81
68.46
2
20.22
12.88
56.06
168
18.01
0
0.09
3
10.05
0
0.03
AIC
33.64
10.13
14.25
2
8.64
1.3
3.85
2
6.97
0.6
2.49
2
30.97
15.34
24.8
2
16.63
2.79
5.07
BIC
33.64
10.13
14.25
2
8.64
1,3
3.85
2
6.97
0.6
2.49
2
17.8
0
0.05
2
16.63
2.79
5.07
IP
33.64
10.13
14.25
8.64
1.3
3.85
2
6.97
0.6
2.49
2
39.17
18.1
31.19
3
16.21
1.79
3.41
In this section, we evaluate ther clustering results on five text datasets. Text representations are obtained by computing TF-IDF features on the 1500 most frequently occurring word stems. As shown in Table 50 our method degrades when the data is represented without the assumption of normal distribution.
29


Reviews:
[{"review_id": "GK8jP6nF2V", "reviewer": "Reviewer_7KWv", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a novel internal validation index for clustering, based on the differential entropy, named internal purity. The method is evaluated on different datasets, encoded with deep net representations, and compared with standard alternatives.", "summary_of_the_review": "Somehow interesting paper dealing with a crucial problem. The approach can be better contextualized, some assumptions (like Gaussianity of Clusters) are rather strong, and better alternatives exists.\n\nUPDATE AFTER REBUTTAL\nI carefully read the responses, the clarifications, and the additional material, and I thank the authors for the significant efforts made in clarifying my doubts. Even if I consider that the paper has potential, I’ll maintain my score unchanged, since I still have some doubts, especially on the assumptions: even if I acknowledge the efforts made by the authors in improving this aspect in the rebuttal, I still consider that the Gaussianity assumption is rather strong, and that better alternatives exist.\n------", "strength_and_weaknesses": "Positive points:\n- The topic is definitely interesting\n- The paper is well written and easy to read\n- The structure of the manuscript is clear\n- The experimental evaluation is rather extensive\n\n\nNegative points:\n\n1) the computation of the index can be problematic.\nThe method is based on the estimation of the differential entropy of the clusters. The estimation of the entropy is based on the assumption that data follows multivariate normal distribution, which I consider a rather strong and problematic assumption:\n\n- In many clustering problems data are not normally distributed\n- There can be problems in estimating covariance matrices in such high dimensional spaces\n- Better alternatives exist: for example, you can consider alternative non parametric bypass entropy estimators. Actually different studies have been conducted on estimating Entropies without making any assumption on the shape (which may be not realistic) and without a direct estimation of the density (which may be very problematic in high dimensional spaces). A starting example can be the non parametric bypass entropy estimator of the Renyi entropy proposed in:\n\nD. Pal, B. Poczos, and C. Szepesvari, “Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphs,” in Advances in Neural Information Processing Systems 23, 2010, pp. 1849-1857.\n\n\n2) Some comments on the experimental part.\n\n- All the experiments are based on text-images which are pre-encoded with deep net representations. Is this crucial for the applied method? In other words, how well this method could generalize to non-deep representations? What about the application on original representations of non-image and non-text datasets?\nMoreover, in the paper I did not find mention to the size of the embedding, which I guess is high (please report it in the paper). How does the method work in moderately dimensional datasets? I’m sure that other methods would increase their performances. \nFinally: while for representation recent state-of-the-art methods have been used, for clustering only classic approaches have been considered, such as K-means, Dbscan, GMM and HC. What if we use more recent and better performing methods? Moreover, most of these methods are returning convex clusters (i.e. somehow following Gaussianity). \n\nWithout considering these three aspects it is difficult to assess the general applicability of the proposed method\n\n\n-  While the proposed criterion is meant to be used in internal validation, at the end the evaluation is based on external measures, i.e. accuracy and ARI. Of course it is a somehow standard and well established way, still it would be nice at least to discuss alternatives and (possibly) provide examples – like, for example, evaluation based on context knowledge, in the spirit of what is discussed in papers like: \nUlrike von Luxburg, Robert C. Williamson, Isabelle Guyon: Clustering: Science or Art? ICML Unsupervised and Transfer Learning 2012: 65-80\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bsn4cj4vVfH", "reviewer": "Reviewer_HicV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a internal cluster validation index that is based on the notion of 'differential entropy' to assess the compactness and separability of a clustering solution. It relies on the assumptions i) that each cluster can be modeled using a Gaussian distribution and ii) that clusters centers also follow a Gaussian distribution.  The proposed index is compared to several alternatives using embeddings obtained from deep neural networks pretrained on image and text datasets. \n", "summary_of_the_review": "The paper presents an internal cluster validation index that is based on the differential entropy of multidimensional normal distribution. \nGiven a cluster i with data mean \\mu_i and data covariance Σ_i its compactness is computed as the entropy of the normal distribution N(\\mu_i, Σ_i). The dispersion among clusters is computed as the entropy of the set of cluster centers, assuming that they are Gaussian distributed.\n\nMajor concerns about the paper.\n1) The Gaussian assumption is too restrictive.\n2) Differential entropy is essentially the determinant of Σ, which is a well-known measure of cluster variance.  \n3) Experiments focus on embeddings from pretrained deep networks. The method should be evaluated on synthetic datasets with clusters of various shapes and known ground truth. It should also be tested on typical low dimensional benchmark datasets (e.g. from the UCI repository).\n4) There is no sufficient intuitive justification on why the proposed index would perform better than several existing ones also relying on separation and compactness.", "strength_and_weaknesses": "Strength\nThe paper is well-written and clearly explained.\n\nWeaknesses\nThere is no significant technical contribution. Combining internal and external cluster separation is typical in most clustering quality indices.\nDifferential entropy depends on the determinant of the covariance matrix, a well-known measure of cluster variance. \nGaussian assumption is made both for the data distribution inside a cluster and the distribution of cluster centers, which is actually too restrictive.\nThere is no sufficient intuitive justification on why the proposed index would perform better than existing ones.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Be3hW8oYE7a", "reviewer": "Reviewer_MXF3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an internal validation metric for model selection in clustering based on the average difference in differential entropy between each cluster and the cluster means when each cluster is treated as having a Gaussian distribution, as is the collection of cluster means.\n\nSome experiments are included to show the practical performance in comparison with other internal metrics when data are clustered after being embedded with pre-trained neural networks.", "summary_of_the_review": "Overall, while there is arguably some evidence for practical relevance, the experiments are a bit too thin and potentially contrived to suit the proposed method, and there are questions regarding the correctness of the implementation of other metrics, to warrant acceptance in a top tier conference. Furthermore, even if there were no issues with the experiments, the lack of a sound justification (either theoretical, or at least a well argued heuristic) for the proposed method means that its performance in general is questionable.\n\nUPDATE:\n\nAfter considering the authors' responses to my and other reviewers' comments, as well as the reviewers' comments themselves, I do not see a reason to adjust my initial score/assessment of the paper.", "strength_and_weaknesses": "Unfortunately I do not see any substantial strengths in the paper. Although there is some evidence of practical relevance, the experiments are a bit contrived and furthermore the implementation of some other metrics, such as AIC and BIC, is incorrect. First, AIC and BIC in their basic formulations should not be applied to the solution based on a hard mixture, such as k-means. Furthermore, if the authors would like to use these then it is sensible to use the same parameter estimates for them as in the proposed method since AIC and BIC are applicable for GMMs, which is the \"assumption\" underlying the proposed method.\n\nWeaknesses:\n- The authors argue that compactness metrics based on variance are flawed, and yet they propose one which is based solely on the differences between the volumes of the covariance matrices of the clusters and the between class covariance. It is not clear why their proposal is not similarly flawed.\n- There does not seem to be any theoretical underpinning for the proposed approach, whereas if the \"assumption\" is that the data are from a Gaussian mixture then there are well motivated alternatives with sound theoretical underpinnings, such as AIC and BIC.\n- The presentation and notation is unclear.\n- The authors report the performance in terms of ACC and ARI, however other external validation metrics may tell a very different story.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "81cs1fau8J", "reviewer": "Reviewer_5DqQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new measure for internal clustering evaluation. To compute this measure, one first computes the differential entropy for each cluster (assuming that the elements are sampled from a multivariate normal distribution) and then subtracts the differential entropy for the cluster centers from the average entropy of individual clusters. Experimental results on several datasets with known cluster labels show the advantages of the new measure over existing ones.", "summary_of_the_review": "On the one hand, the experiments show significant superiority of the proposed measure over existing ones on the considered datasets. On the other hand, the properties of the proposed measure are not investigated theoretically or empirically. Thus, it is not clear how it will behave in other tasks. Additional analysis is needed to understand to which datasets the measure is applicable. Therefore, my current rating is borderline reject.", "strength_and_weaknesses": "Strength:\n- Experimental results show significant improvements on the considered datasets\n\nWeaknesses:\n- The assumption about the normal distribution is quite strong, and it is not clear how deviations from this assumption would affect the quality of the measure\n- There is no theoretical or empirical analysis of the properties of the proposed measure\n\nDetailed comments are below.\n\nI am not very familiar with the literature on internal clustering evaluation measures, but there are works on theoretical analysis of such measures, for instance, papers discussing axioms desirable for these measures [1] or providing guarantees for convex clusters (Halkidi & Vazirgiannis, 2001). Then, an empirical evaluation was conducted by Liu et al. (2013), where the effect of noise, density, and shapes was analyzed. The current paper would benefit from more detailed related work where different approaches to comparing clustering measures are discussed.\n\nAdditionally, a more detailed analysis of the proposed measure would help to demonstrate its advantages. For instance, does this measure satisfy some desirable theoretical properties? Is it biased towards big or small clusters? (Such biases are formalized for external cluster evaluation measures, e.g., in [2].) How the measure performs for clusters with different levels of noise, different shapes, etc.? Such theoretical or empirical analysis can show the benefits and limitations of the proposed measure.\n\nMore details throughout the text can be helpful:\n- The beginning of page 2: formalization of distance-based indices would help a reader to understand the motivating example.\n- Formal definitions of the measures from Table 1 are needed to understand the existing measures and the differences between them and the proposed measure. (In Table 1, the intuition behind the measures is given.)\n- More details on the information criterion are needed to understand the analogy at the beginning of page 6.\n- Also, I could not get the main message of the paragraph “Effectiveness” in Section 4.2.\nThe detailed experimental setup can be moved to Appendix, which may give more space to explanations throughout the text.\n\nMinor comments:\n- Note that the differential entropy in equation (2) is for a one-dimensional case, while it is later used for d-dimensional vectors.\n- To compute CP in equation (7), parameters of the multivariate normal have to be estimated, which needs to be mentioned in the text.\n- The variable m (the number of samples in the k-th cluster) can be changed to reflect the dependency on k, e.g., m_k\n\nQuestions for the authors:\n- Q1: The measure assumes that the elements within each cluster and cluster centers have multivariate normal distributions. In practice, this can be violated. What would happen in such cases? How sensitive is the measure to deviations from this assumption? Also, can we approximate the density and then approximate the integral (2) without this assumption?\n- Q2: Could you please specify how accuracy is computed for clustering evaluation?\n\nSome typos:\n- “these implementation” (page 2, line 3)\n- “the volume of two cluster vector space are the same” (page 2, line 8)\n- “has also be used” (page 3, line 6)\n- “the distance form the samples” (page 2, definition of I index)\n- “are not good measure” (page 4)\n- “ACC and ARI indicates” (page 6, line -7)\n- “besults” (page 7, line -7)\n- “CH obtain” (page 7, line -5)\n- “Tabel” (page 7, line -5)\n- Missing articles throughout the text\n\n[1] Ben-David S., Ackerman M. Measures of clustering quality: A working set of axioms for clustering. NIPS 2008.\n\n[2] Gösgens M., Tikhonov A., Prokhorenkova L. Systematic analysis of cluster similarity indices: How to validate validation measures. ICML 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject