Mini-cluster Guided Long-tailed Deep Clustering
Authors: Zhixin Li
Yuheng Jia
Guanliang Chen
Hui LIU
Junhui Hou
Venue: ICLR 2026 Poster (2026)
Keywords: Deep Clustering; Clustering; Long-tailed Clustering
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose a novel deep clustering method that can effectively handle the long-tailed data.

Abstract: As an important branch of unsupervised learning, deep clustering has seen substantial progress in recent years. However, the majority of current deep clustering methods operate under the assumption of balanced or near-balanced cluster distributions. This assumption contradicts the common long-tailed class distributions in real-world data, leading to severe performance degradation in deep clustering. Although many long-tailed learning methods have been proposed, these approaches typically rely on label information to differentiate treatment across different classes, which renders them inapplicable to deep clustering scenarios. How to re-weight the training of deep clustering models in an unsupervised setting remains an open challenge. To address this, we propose a mini-cluster guided long-tailed deep clustering method, termed MiniClustering. We introduce a specialized clustering head that divide data into much more clusters than the target number of clusters. These predicted clusters are referred to as mini-clusters. The mini-cluster-level predictions serve as the guide for estimating the appropriate weights for classes with varying degrees of long-tailedness. The weights are then incorporated to re-weight the self-training loss in model training. In this way, we can mitigate model bias by re-weighting gradients from different classes. We evaluate our method on multiple benchmark datasets with different imbalance ratios to demonstrate its effectiveness. Further, our method can be readily applied to the downstream of existing unsupervised representation learning frameworks for long-tailed deep clustering. It can also adapt label-dependent long-tailed learning methods to unsupervised clustering tasks by leveraging the estimated weights. The code is available at https://github.com/LZX-001/MiniClustering.

Published as a conference paper at ICLR 2026
MINI-CLUSTER GUIDED LONG-TAILED DEEP CLUSTERING
Zhixin Li', Yuheng Jia : Chen', "ien",Houe Hui Liu Junhui Hou
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China
2 Key Laboratory of New Generation Artificial Intelligence Technology and Its
Interdisciplinary Applications (Southeast University), Ministry of Education, China
3 Faculty of Information Technology, Monash University, Melbourne, 3800, Australia
4 School of Computing Information Sciences, Saint Francis University, Hong Kong, China
5 Department of Computer Science, City University of Hong Kong, Hong Kong. China
yhjiaje (riaaessee seu. edu. cn, quanliang chenemonash.edu. h2 h2liuusfu. ferdsfuueedu... edu. hk, jh. hohuuuiity.... edu.hk
{1
ABSTRACT
As an important branch of unsupervised learning, deep clustering has seen sub- stantial progress recent years. However, the majority of current deep cluster- ing methods operate under the assumption of balanced or cear-balanced cluster distributions. This assumption contradicts the common long-tailed class distribu- tions in real-world data, leading to severe performance degradation in deep clus- tering. Although many long-tailed learning methods have been proposed, these approaches typically rely on label information to differentiate treatment across different classes, which renders them inapplicable to deep clustering scenarios. How re-weight the training of deep clustering models in unsupervised set- ting remains an open challenge. To address this, we propose a mini-cluster guided long-tailed deep clustering method, termed MiniClustering. We introduce spe- cialized clustering head that divide data into much more clusters than the target number of clusters. These predicted clusters are referred as mini-clusters. The mini-cluster-leve predictions serve as the guide for estimating the appropriate weights for classes with varying degrees of long-tailenn toig--aiiss The weights are then incorporated to re-weight the self-training loss in model training. In this way, we can mitigate model bias by re-weighting gradients from different classes. We eval- uate method on multiple benchmark datasets with different imbalance ratios to demonstrate its effectiveness. Further, our method can be readily applied to the downstream of existing unsupervised representation learning frameworks for long-tailed deep clustering. It can also adapt tabel-dependen long-tailed learning methods to unsupervised clustering tasks by leveraging the estimated weights. The code is available at ht wttpp:: // / thub. com/ com/LZX LZX- 001 wwwiiiiiililiii Maaniclusterii Clutering..
INTRODUCTION
Deep clustering is an unsupervised learning task, aiming to train deep neural networks to partition data points into clusters in the absence of labels. Existing deep clustering algorithms often assume a balanced class distribution in the dataset, which is inconsistent with the long-tailed distribution commonly seen in natural data (Li and Jia, 2025; Zhang et al., 2024a; Jiang et al., 2021). In real- world scenarios, data are often distributed in a taiged manner, where some classes (head classes) contain large number of samples while others (tail classes) are represented by only a few. These long-tailed distributions pose a significant challenge in deep learning, models tend to be biased toward head classes while underperforming on tail ones, leading to significant performance drop (Zhang et al., 2023). In supervised long-tailed learning, common solution is to employ class- balancing strategies to mitigate such model bias: re-sampling the number of samples from different classes (Estabrooks al., 2004; Liu al., 2008; Zhang and Pfister, 2021), re- re-weighting each class
*Corresoonding author: Yuheng Jia, Junhui Hou.

---

Published as a conference paper at ICLR 2026
in the loss function (Lin et al., 2017: Cui et al., 2019; Ren et al., 2020), and adjusting the output logits of each class (Menon et al., 2021; 1; Zhao et al., 2022; Tao et al., 2023). While these methods differ in form, their essence lies in assigning class-specific weights to head and tail classes, thereby differentially treat head and tail classes in training. To introduce such a re- re-balancing mechanism that mitigates model bias induced by data imbalance, knowing label frequencies as a necessary prior is typically required.
Unfortunately, deep clustering is totally unsupervised. Despite also facing severe performance degradation under long-tailed distributions (Li and Jia, 2025), it lacks labels and therefore can- not directly upply the aforementioned methods. To extend the principles of supervised long-taii tailed learning to long-tailed deep clustering, it is crucial to assess the appropriate weights for training each class an unsupervised manner.
To address the aforementioned issues, we investigate the characteristics of clustering under long- tailed distributions and have found some important phenomena. We observe that head class samples often occupy larger embedding space compared to tail class ones, which often leads to head class samples being assigned to multiple clusters while tail class samples end up sharing clusters with others. Therefore, we cluster the embeddings into a larger number of groups, referred to as mini- clusters, than the final target number of clusters, enabling finer-grained partitioning. In this way, average purity across clusters is improved. However, the number of mini- mini-clusters is much larger than the desired number of clusters (referred to as target-clusters), which does not align with our objective for clustering. To establish the connection between target-clusters and mini-clusters, the sample assignments between them. We find that clusters representing head classes tend to be associated with larger number of mini-clusters, while those corresponding tail classes are linked to fewer. These phenomena are detailed in Section 3.1. Inspired by these phenomena, we introduce a network architecture with two clustering heads, where the target-cluster head outputs predictions over the desired number of clusters, and the mini-cluster head has a higher prediction dimensionality than target-clusters. Then, we decompose each target-cluster into its constituent mini-clusters. We align the predictions from the mini-cluster head and the target-cluster head to determine how many mini-clusters each target-cluster is divided into, thereby estimating the training weight for each class. In this manner, we enable the application of re- weighttgg strategies to long-tailed deep clustering in way analogous to common supervised long-tailed learning method, effectively re-balancing the training process and alleviating model bias. We name this method MiniClustering, which is short for mini-cluster guided clustering.
analyze
In summary, the main contributions of this work follows.
We reveal key phenomena in long-tailed clustering, based on which we propose an unsupervised method to estimate class weights, effectively re-weighting model training.
We introduce a novel paradigm based on two clustering heads with different prediction granulari- ties. Based on estimated class-specific weights, the proposed MiniClustering effectively address long-tailed deep clustering. Extensive experiments showcase its superiority.
We pioneer class-level re-weighting in unsupervised long-tailed learning, extending this strategy from supervised learning to deep clustering. MiniClustering enables supervised re- re-weighting losses to be adapted for
2 RELATED WORK
Deep Clustering. Deep clustering trains deep neural networks to assign unlabeled data points to dis- tinct clusters. where the process is conducted without any manual annotations (Zhou et al., 2024). Compared to supervised learning, it can effectively save time and labor in label annotation. Deep clustering methods can be broadly categorized into two paradigms: those based on representation learning (Tao et al., 2020; Huang et al., 2023; 2014) and those based on clustering heads (Van Gans- beke et al., 2020; et al., 2021; Niu et al., 2022; Li et al., 2022: Qian, 2023). In contrast to traditional clustering (Hartigan and Wong, 1979; Von Luxburg, 2007; Xie et al., 2025; Zhang et al., 2025a;b), deep clustering based on representation learning employs neural networks to model and optimize data representations. They leam suitable embeddings through various approaches, such as autoencoders (Huang et al., 2014; Xie et al., 2016; Guo et al., 2017) and contrastive learning (Huang et al., 2023; Li et al., 2023; Tsai al., 2021). Then clustering techniques such K-means
2

---

Published as a conference paper at ICLR 2026
(Hartigan and Wong, 1979) spectral clustering (Von Luxburg, 2007) are applied to form cluster partitions. Deep clustering methods based on attached clustering heads typically introduce a cluster- ing head following feature extraction network and train it using pseudo-labels. Such approaches can directly produce clustering predictions without relying on additional techniques, making their architecture similar with the supervised learning paradigm (Van Gansbeke et al., 2020; Li et al,, 2021; Qian, 2023; Tian et al., 2017; Jia et al., 2025; Li et al., 2025a), In general, most existing deep clustering methods are trained and evaluated primarily on balanced datasets, overlooking the long-tailed distributions commonly observed in natural data. As result, their designs are often not robust to long-tail tailed data, leading to significant performance degradation (Li and Jia, 2025).
Long-tailed Learning. In reality, data distributions are often taigeda with head classes con- taining abundant samples and tail classes having scarce samples. Training models on such data can lead to biased models that are more prone to predicting common head classes while overlooking rare tail classes. When labels are involved during training, we can re-balance the model according to the frequency of each class's s occurrence, thereby mitigating this bias. Common re-balancing techniques include: re-sampling balances the number of samples by over-sampling tail classes or under- sampling head classes (Chawla al., 2002; Wang et al., 2019; Zhang and Pfister, 2021); re- weighting balances the class-wise contribution to gradient updates by assigning class-specifie weights in the loss function according label frequencies (Cui et al., 2019; Cao al., 2019; Tan et al., 2020); logit adjustment balances the prediction difficulty across different classes by modify- ing the logits in biased models (Tian et al., 2020; Menon et al.. 2021: Zhang et al., 2021b). These methods all require prior knowledge of labels, thereby applying varying degrees of influence to dif- ferent classes. Consequently, they cannot be directly applied to unsupervised long-tailed learning. Methods for unsupervised long-tailed learning are scarce. Some approaches focus on unsupervised representation learning under long-tailes distributions (Jiang et al., 2021; Zhou et al., 2022; Li and Jia, 2025). They typically balance the loss between rare and frequent samples to mitigate model bias. However, they do not necessarily ensure the discriminability of the learned embeddings in long-tailed deep clustering. Other methods address long-tailed clustering fine-tuning large pre-trained models (Zhang et al., 2024a;b). Due to the strong prior knowledge encoded in these models, their susceptibility long-tailed bias is often less severe.
downstream task by
To the best of our knowledge, effectively training a clustering head for long-tailed deep clustering, following the principles of supervised long-tailed learning based re- weighting, remains unex- plored. Furthermore, how to accurately obtain class-specific weights to re- balance model training in an unsupervised manner is critical yet under-innesttiaaed problem.
PROPOSED METHOD
3.1 MOTIVATION
Inspired by the aforementioned issues, we identify three important phenomena in long-tailed deep clustering.
Phenomenon 1: Head classes occupy more embedding space than tail classes. Following (Tang et al., 2020; Cao et al., 2019), we construct the long-tailed version of CIFAR-10 dataset (Krizhevsky et al., 2009) with an imbalance ratio of 10, where we conduct unsupervised represen- tation learning using BYOL (Grill et al., 2020), SimCLR (Chen et al., 2020) and MoCo (He et al., 2020). These methods serve as the foundation for state-of-the-art deep clustering algorithms, as both representation-baat based approaches and those employing clustering heads typically rely on high- quality representation learning (I (Huang et al., 2023; Van Gansbeke et al., 2020; Li et al., 2021; Li and Jia, 2025). We record the confusion matrices of the K-means (Hartigan and Wong, 1979) clustering results based on the representations learned by these three methods, to illustrate the membership relationship between predicted clusters and ground-truth classes. Figure 1, the class indices range from O to 9, with the number of samples decreasing progressively. In the confusion matrix, the intensity of each cell indicates the number of samples from a given ground-truth class (indexed by row) that are assigned to a specific predicted cluster (indexed by column). It can be observed that samples from head classes (i.e., classes with smaller indices) are predominantly distributed across multiple clusters (e.g.. in SimCLR, samples from class 0 is primarily assigned to cluster O, 7 and 8.). In contrast, the clusters to which tail classes are predominantly assigned may contain domi-
large number of samples from other classes, and in some cases, these clusters
3

---

Published as a conference paper at ICLR 2026
nated by head classes (e.g., in SimCLR, class 9 and class share the same cluster). K- K-means is a distance-based clustering method defined in Euclidean space, where each sample assigned to the cluster whose centroid is nearest. This tends to produce clusters of comparable size. Under such a mechanism, head classes, due to their larger number of samples, naturally occupy a more extensive region in the embedding space and thus partitioned into multiple clusters. Tail classes, which span smaller region of the embedding space, are forced to share clusters with other classes.
Prodicted Clucter Distribution
Predicted Clucter literbuttin
203
Predict Cluster Distribure
316
11
14
27
1551
800
630
514
12
1752
19
18
6:28
193
256
576
72
147
302
600
13
261
051
40
70
547
500
470
96
037
28
11
20
530
233
536
80
618
23
3
320
54
125
50
151
11
203
220
53
26
18
21
14
37
372
03
42
21
45
16
115
421
321
1
8
True Class Distribution
True Class Distribution
True Class Distribution
(a) BYOL
(b) SimCLR
(c) Moco
Figure 1: Confusion matrices of predicted cluster distributions and true class distributions for dif-
ferent methods. It can be observed that samples from head classes are often separated into multiple clusters, while samples from tail classes are forced to share cluster with other categories. This dicates that, under long-tailed distributions, unsupervised training learns representations where head classes occupy a larger embedding space than tail classes.
Phenomenon Mini-clusters can enhance purity. Di- rectly utilizing current cluster information to guide train- ing leads to inaccuracies, as some predicted clusters con- tain samples from multiple ground-truth classes while certain classes are distributed across multiple predicted clusters. A natural approach is to cluster the data into larger number of clusters than the true number of classes, using finer-grained division to increase the purity of each cluster, thereby facilitating subsequent processing. We perform clustering the embeddings of CIFAR-10, which has 10 classes, using a higher number of clusters. We use purity, defined the proportion of the largest class in certain cluster, to measure these clusterings. As shown in Figure 2, this approach can improve the average purity across all clusters.
Purity
BYOL
0.6
SimCLR
Moco
20
30
40
Cluster Number
Figure 2: As the number of clusters in- creases, the average purity of all clusters is improved.
IPOE
Samge Ountt Dloos
Ao ctucto clacttrraaaeee
Semole Coum
10-
Ground truth Cosses
Ground tuth Casses
Ground buth Cosoes
(a) Varying Methods
(b) Varying M
(c) Varying d
Figure 3: Across different methods, numbers of mini-clusters M, and the assignment threshold 6, head classes are usually assigned greater number of mini-clusters compared to tail ones.
Phenomenon 3: Head classes assigned more mini-clusters than tail classes. We refer to the aforementioned fine-grained clusters as mini-clusters, while those with the number aligned to the desired true number of classes are termed target-clusters. Inspired by the aforementioned
4

---

Published as a conference paper at ICLR 2026
discussion, we aim to establish the relationship between the target-clusters and the mini-cluster We conduct experiments on the long-tailed version of CIFAR-10 (Krizhevsky et al., 2009), where the number of samples in each class decreases exponentially from index 0 to 9. Assuming there are M mini-clusters in total, we assign a mini-cluster to a target-cluster the proportion its samples belonging to that target-cluster exceeds predefined threshold 6. Specifically, cluster assignment for head and tail classes under varying methods, varying M and varying 6. We the method BYOL, M 40 and 8 0.3 as the default configuration, and varied one variable at a time for analysis. As shown in Figure 3, across different methods, numbers of mini-clusters M, and the assignment threshold 6, head classes are consistently assigned more mini-clusters than tail classes. This suggests that mini-cluster assignment can serve as an indicator of whether class is head or tail class, thus enabling re-balanced model training.
evaluated the mini-
Theoretical Analysis. Previous observations have shown that head classes are often assigned more mini-clusters than tail classes. We now provide a rigorous theoretical derivation, proving that when mini-cluster purity and representation quality satisfy certain conditions, classes with more samples dominate a larger number of mini-clusters. We assume that samples from K classes follow a long- tailed distribution and are clustered into M mini-clusters. The purity of in each mini-cluster is at least p. Let Smi and Smax be the minimum and maximum numbers of samples in clusters respectively. A mini-cluster is said to be dominated by class l: if its purity with respect to class is at least p. Let m& denote the number of mini-clusters dominated by class k. Let e denote the number of samples from class k that lie outside the mini-clusters dominated by class k. Let Nu denote the number of samples in class k. Then we can obtain below theorem:
Theorem 1. For classes i and j. if the number of samples N; > Nj, then the number of occupied mini-clusters m, > my holds when
N1Smax
P > (N; G))miu
If is too small or is too large, it may lead to the failure of m; >> my when N; > Nj- Both the minimum purity p mini-clusters and the number e, of samples from class that lies outside the quality of the embeddings. If the embedding
the mini-clusters dominated by class depend quality is low and the model struggles to consistently group samples of the same class into the same mini-clusters, the above property may fail to hold. Therefore, in proposed MiniClustering, a model pre-trained with unsupervised representation learning to ensure a reasonable embedding distribution.
A detailed proof and analysis can be found in the Appendix L.
3.2 MINICLUSTERING
Inspired by previous observations, we propose mini-cluster guided long-tailed deep clustering method, named MiniClustering, We introduce clustering head training strategy based the self- labeling architecture (Li et al., 2021; Van Gansbeke et al., 2020; Qian, 2023), which leverages mini- cluster-level predictions balance the training losses of classes with varying long-tailed degrees.
The network architecture of the proposed method consists of three components: an encoder fa that extracts embeddings from the raw data; target-cluster head fe that produces predictions with the same dimensionality as the predefined target number of clusters; a mini-cluster clustering head fm that outputs predictions at the finer-grained mini-cluster level. Both fe and f' are linear classifiers that take embeddings as input, but their output logit dimensions are the number of target-cluster K and the number of mini-clusters M, respectively. This model is trained jointly with three losses, including mini-cluster self- tef-trrainng loss L.., re-weighted target-cluster self-training loss C, and similarity alignment loss L. The network architecture of MiniClustering is shown in Figure 4. After data augmentation, the transformed input data is fed into the encoder fa to obtain embeddings, which are then passed to the target-cluster head ft and the mini-cluster head f' respectively for prediction and self-labeling strategy is applied for training unsupervisedly. The predictions from the mini-cluster head are matched with those from the target-cluster head according to a specific criterion described below. The number of mini-clusters belonging to each target-cluster determines the re- balancing weight for that class. Additionally, we align the similarities of outputs from the two clustering heads to prevent training desynchronization.
5

---

Published as a conference paper at ICLR 2026
Mini-Cluster Head fm
Mini-Cluster Self-training
Encoder fe
Guide Re-weighting
Similarity Alignment
X(
Target-cluster Head ft
Re-weighted Target-cluster Self-training
Data Augmentation
Figure 4: The network architecture of MiniClustering.
Notations. Given a batch B in the dataset, and 2; E B is sample in this batch, then xr. and x?? are two augmented views to serve as the input for encoder fe- fe(2)) and fe fe(x)) are two embedding accordingly. They are both fed to target-cluster and mini-cluster head later. Specifically, a! is used for prediction, while xt is used to compute pseudo-labels. P (f(((a(z)))
R and p = offm (x)))) E RRM are the the predictions from target-cluster head fa and mini-cluster head fm. where a(-) the softmax function. of max (pt) and ctth max(p,)) (p:"t) are confidences of the predictions respectively. of the pseudo labels generated by the target-cluster head and mini-cluster head, respectively, for 24, yf and y/"h denote one-hot vectors, where the positions corresponding to the pseudo labels are set to respectively, and all other entries are 0. denotes the size of the set. (-)7 is the transpose of the matrix.
and D = argmax (f are
Mini-cluster self-training loss. The mini-cluster head maps the input embeddings to dimen- sionality higher than the number of target-clusters, enabling finer- emer-grained predictions. To train the mini-cluster head, we adopt threshold based self-labeling strategy (Van Gansbeke et al., 2020) based on cross-entropy (CE) loss,
M
Pm
Mig log(p?)),
S.h = (i cm
(1)
Sm
LES j=1
where Sth is the set containing indices of samples whose confidences output from f' exceed the threshold т, y and p.,, denote the jth element in the corresponding vectors. Through a threshold- based self-labeling method, we can obtain reliable pseudo-labels, thereby improving the quality of training.
Re-weighted target-cluster self-training loss. The predictions output by the target-cluster head fi correspond to the actual number of classes, but faces an imbalanced class distribution. As illustrated in Section 3.1, head classes occupy larger embedding space compared to tail classes. In the mini-cluster setup, samples from head classes are expected to be assigned to a greater number of high-purity mini-clusters, while those from tail classes assigned to fewer. Consequently, the number of mini-clusters associated with each target-cluster can reflect the spatial distribution of the corresponding class. We posit that target-clusters occupying more mini-clusters are spatially more dispersed and are thus more likely to be head classes, whereas those occupying fewer mini-cluster are more likely be tail classes. Inspired by common approaches supervised long-tailed learning (Cui et al., 2019; Ren et al., 2020; Tan et al., 2020), we accordingly re- weightt the loss for head and tail classes during model training to achieve re-balance and mitigate the model bias. The class-wise weights are estimated based on the number of mini-clusters occupied by each target-cluster. The weight for target-cluster is
M
W%
M max Σ
1 ビ
1 Tx
8), 0.5)
(2)
77y1

---

Published as a conference paper at ICLR 2026
where Tk. {i|æ; E B. off target-cluster & and mini-cluster at the same time, T2 = {i|z; E B, 9," = j} is the set containing indices of samples predicted to mini-cluster j. 1(-) (-) is indictor function, which returns when the condition is satisfied else O. The Σ 1( Tx. a > 5) in the Eq. (2) counts the number of mini- clusters occupied by samples from target-cluster k. We assign mini-cluster to target-cluster when over threshold proportion of its samples are from that target. The weight of each target-cluser then determined based on the number of mini-clusters affiliated with it. To prevent division by zero, we set lower bound of 0.5 on the number of assigned mini-clusters, ensuring that target-clusters with assigned mini-clusters have larger weights in training. Note that this is soft assignment scheme, as theoretically, when the threshold is less than 0.5, the mini-cluster could be counted toward multiple target-clusters. In Eq. (2), larger numerators lead to smaller weights for head classes and, conversely, larger weights for tail classes. To re-balance gradient updates, we re-weight threhhod-bbsse self-labeling in below manner,
k. 0tth = is the set containing indices of samples predicted to
c, =
Wo log(el,), 8 = EB d
(3)
SH
iES;
where S is the set containing indices of samples whose confidences output from fm exceed the threshold
Similarity alignment loss. The target-cluster head and the mini-cluster head are updated by dif- ferent losses, which can easily lead to misalignment during training. resulting in the failure of the re-weighting strategy and potential model training conflicts. Due to the different output dimensions of the target-cluster head and the mini-cluster head, we adopt a similarity-baset alignment method to ensure consistency between the outputs of the two clustering heads. We consider Pt and pm are matrices formed by row-wise stacking of the prediction results from ft and fm on the batch, respectively. They are of size N X K and N x M, respectively, where N is the batch size. As samples with close predictions in the target-clusters should also be close in the mini-clusters, the similarity matrices pt pt" and pm pml should be close, too. We bring the similarity matrices of the two closer together using the mean squared error (MSE) loss
L, = 1P' - Pm pm
(4)
where ||.
|| is the Frobenius norm. This loss is similar to the within/cross-gran graph consistency pro- posed in (Lin al., 2023), which encourages the alignment of the edges across samples in similarity graphs.
Overall procedure and objective. Following (Cai al., 2023), we initialize the network weights in ft and fm using the cluster centers obtained from K-means (Hartigan and Wong, 1979) to accelerate training. Additionally, we initialize the network bias zero at the beginning. Please note that encoder fe, target-cluster head ft and mini-cluster head fm are all in backpropagation. Before performing MiniClustering, the encoder fe can be pre-trained on long-tailed datasets in advance using any unsupervised representation learning and specific settings can be found in Section 4.1. The overall objective of MiniClustering is
L= L. + aLm + BL.
where a and 33 are trade-off parameters to balance three terms. After training, we use the predictions from the target-cluster head for clustering, The algorithmic pseudocode is summarized in Appendix B.
4 EXPERIMENTS
4.1 EXPERIMENT SETTINGS
Datasets. We conduct experiments on benchmark datasets, including CIFAR-10 (Krizhevsky et al., 2009), CIFAR-20 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011), Tiny ImageNet (Le and Yang, 2015) and ImageNet-LT (Liu et al,, 2019). CIFAR-20 --EAR-20 is CIFAR- 1F0 using 20 super-classes as the cluster number. On STL-10, we use all data from both the training and test sets for training and clustering. Unless otherwise specified, for all other datasets, use only the training set. Following
7

---

Published as a conference paper at ICLR 2026
(Cao et al., 2019; Zhou et al., 2020), the imbalance ratio to determine the number of samples for each class long-tai tail dataset construction. The number of samples per class follows an expo- nential decay, where the ratio between the largest and smallest classes is defined as the imbalance ratio.
Implementation Details. Following (Li and Jia, 2025), adopt ResNet-18 as backbone for coder fe- Following (Van Gansbeke et al., 2020), linear classifiers are adopted as the clustering heads for Jz and fm. The number of target-clusters K, in most deep clustering methods (Huang et al., 2023; Li et al., 2021; Van Gansbeke et al., 2020; Li and Jia, 2025; Qian, 2023; Hartigan and Wong, 1979), is a known prior. corresponding to the number of classes. Mini-cluster number M is hyperparameter. For extracting meaningful embeddings, we adopt BYOL framework (Grill al., 2020) and train fe advance on the same long-tailed datasets as MiniClustering. We take the target network in BYOL as the encoder fe- To align with the training epochs in (Li and Jía, 2025), we pre-train Se for 800 epochs and then execute MiniClustering for 200 epochs, unless otherwise specified. In MiniClustering, we adopt batch size of 512, following (Li and Jia, 2025). We set constant learning rate of le-5 and a self-labeling threshold 0.99, unless otherwise specified. For other training details and hyperparameter settings of each experiment, please refer to Appendix C.
Table I: Clustering performance percentage (%) on three benchmark datasets (imbalance ratio 5) across various methods.
Datasets
CIFAR-TO
CIFAR-20
STL-IO
Metries
ACC
CAA
NMI
ARI
ACC
CAA
NM1
ARI
CAA
NMI
SCAN
55.6
60.3
54.0
39.3
38.8
39.7
42.6
25.1
48.8
49.8
46,2
34.9
SDCLR
44.1
50.5
43.4
38.0
40.2
38.7
40.3
23.8
35.8
37.4
34.4
19.3
CC
25.2
20.9
18.3
1.47
16.0
12.0
18.0
00.0
41.5
30.0
46.2
22.4
IDFD
56.7
63.0
51.8
37.1
31.2
30.7
30.5
16.2
41.6
41.2
39.6
24.3
CoNR
41.4
43.5
34.9
23.3
23.4
21.6
12.1
31.3
29.0
ProPos
51.4
59.2
52.2
34.1
40.7
39.3
42.6
26.4
37.2
39.3
34.5
21.8
DMICC
40.6
42.5
36.9
25.4
25.2
23.1
20.6
7.5
45.9
45.6
45.3
35.5
SeCu
56.1
59.0
54.5
40.3
34.8
33.7
35.8
19.9
51.6
50.8
48.2
39.1
LFSS
58.1
63.1
59.7
43.3
43.5
42.2
46.8
28.6
46.6
43.7
46.0
34.0
ConMix
61.6
65.4
59.8
45.6
41.0
43.9
48.8
BYOL (Baseline)
60.4
66.4
62.0
46.5
38.5
39.2
41.3
23.4
48.5
46.4
49.2
35.4
MiniClustering (Ours)
74.3
72.6
69.9
62.1
45.6
42.3
47.5
30.7
54.7
52.7
54.4
42.8
Table 2: Clustering performance percentage (%) on three benchmark datasets (imbalance ratio
10) across various methods.
Datasets
CIFAR-I0
CIFAR-20
STL-10
Metries
ACC
CAA
NMI
ARI
ACC
CAA
NMI
ARI
ACC
CAA
NMI
SCAN
52.0
50.1
59.4
43.8
37.0
32.4
37.8
23.1
47.1
45.4
47.5
SDCLR
38.9
44.3
42.5
26.5
37.8
35.9
39.6
22.9
34.6
37.9
32.2
17.3
CC
40.6
27.5
43.9
18.8
19.9
14.3
21.9
1.1
43.0
35.3
44.7
25.4
IDFD
47.5
54.9
48.4
33.1
28.7
27.2
28.6
15.1
38.6
34.7
36.8
22.8
CoNR
31.4
44.3
29.2
17.8
20.3
17.9
17.3
8.0
34.8
32.5
30.7
21.0
ProPos
46.1
49.3
52.5
34.2
36.8
33.6
40.1
22.5
38.2
37.2
DMICC
36.6
39.5
36.8
25.9
24.7
21.9
20.7
10.1
41.3
41.9
38.7
30.3
SeCu
51.5
57.0
54.8
39.6
32.0
30.7
33.3
17.6
48.2
49.6
50.0
36.8
LFSS
56.3
59.7
57.9
43.0
41.1
40.0
46.8
28.4
43.3
42.3
45.4
30.8
ConMix
53.3
58.2
$7.1
39.3
43.6
48.2
BYOL (Baseline)
51.9
55.2
56.3
41.7
39.8
37.1
41.9
25.0
41.8
46.8
29.6
MiniClustering (Ours)
64.6
61.4
63.9
56.7
44.3
41.5
47.2
30.4
52.0
54.4
50.8
39.1
4.2
MAIN RESULTS
Following (Li and Jia, 2025), we conduct experiments on CIFAR-10, CIFAR-20 and STL-10 with imbalance ratios of and 10 to evaluate clustering performance MiniClustering. We compare state-of-the--rt methods, including SCAN (Van Gansheke et al., 2020), SDCLR (Jiang et al., 2021),
CC (Li et al., 202 1), IDFD (Tao et al., 2020), CoNR (Yu et al., 2023), ProPos (Huang et al., 2023), DMICC (Li al., 2023), SeCu (Qian, 2023), LFSS (Li et al., 2025b) and ConMix (Li and Jia, 2025). We adopt B YOL (Grill et al., 2020), which used for pre-training the encoder fe, as the baseline, and apply K-means (Hartigan and Wong, 1979) to obtain its cluster assignments. We reproduced SCAN, SeCu, and LFSS on the long-tailed datasets following their original experimental configu- rations. Results of the other methods are cited from ConMix (Li and Jia, 2025). Following (Li and
8

---

Published as a conference paper at ICLR 2026
Jia, 2025), we assess the effectiveness of long-tailed deep clustering across four metrics: accuracy (ACC), class-averaged accuracy (CAA), normalized mutual information (NMI, and adjusted rand index (ARI). The comparison results of method against various approaches are presented in Table and 2, with the best performance highlighted bold.
It can be observed that MiniClustering achieves significant advantage over other kate-of-the-an approaches. It achieves the best performance across all metrics under all experimental settings. Compared to the baseline BYOL, the proposed approach shows particularly significant improve- ment. For example, on CIFAR-10 with imbalance ratios of and 10, MiniClustering outperforms BYOL by 13.9% and 12.7% ACC, and achieves improvements of 15.6% and 15.0% in ARI, re- spectively. The experimental results demonstrate the effectiveness of MiniClustering in addressing the long-tailed deep clustering problem.
To comprehensively evaluate the effectiveness of our method, we conducted extensive experiments, including ablation studies (detailed in Section 4.3), hyperparameter analysis (detailed Appendix D). evaluation of incorporating MiniClustering into downstream of unsupervised representation learning framework (detailed Section 4.4), applying supervised long-tailed learning losses into MiniClustering (detailed in Section 4.5), facing extreme class imbalance (detailed in Appendix E ). validation on large-scale datasets (detailed in Appendix F), statistical significance analysis (de- tailed in Appendix H), visualization (detailed in Appendix I), evaluation text datasets (detailed in Appendix J). These results collectively demonstrate the robustness and practical utility of our approach.
4.3 ABLATION STUDY
The ablation studies are shown in Table 3. Compared to the baseline BYOL, we achieve significant perfor- mance improvements, demonstrating the effectiveness the MiniCluster- ing strategy whole. If only use threshold-basee self-labeling to train single target-cluster-eveell clus- tering head, performance improves over the baseline but remains far in- ferior to MiniClustering. This not only validates the rationality our use of self- self-labeling scheme, but also demonstrates the effectiveness the proposed mini-cluster guided re-weighting strategy. is worth noting that self-labeling training achieves a higher NMI score than MiniClustering. but significantly lower ACC and CAA. This suggests that standard self-labeling tends to merge samples from tail classes into other clusters. The same phenomenon occurs when Lm is absent, indicating that L.. helps separate tail classes from other classes during prediction.
Table 3: Ablation studies on CIFAR-10 with an imbalance ratio of 10.
ACC
CAA
NMI
ARI
BYOL Only threshold-busee self-labeling MiniClustering wlo threshold constraint
51.9
53.2
56.3
41.7
55.6
57.6
66.3
52.5
54.1
55.1
58.5
45.6
Only L, MiniClustering w/o L. MiniClustering w/o C.. MiniClustering w/o L. MiniClustering
59.9
58.1
63.5
$2.0
46.7
39.0
51.1
26.7
64.5
43.6
65.4
50.3
56.0
57.6
63.5
40.0
64.6
61.4
63.9
56.7
We can also see that without threshold to limit the samples involved in loss calculation in Mini- Clustering, the performance would drop. We tested training with only the C, involved. In this scenario, the loss of the target-cluster head is effectively reighteh but it is not synchronized with the mini-cluster head, leading to a performance drop. When L, absent in training, the weight of target-cluster head is not updated, leading poor performance. When L. is absent, the two clustering heads may desynchronize, reducing performance.
4.4 MINICLUSTERING AS DOWNSTREAM COMPONENT IN UNSUPERVISED REPRESENTATION LEARNING PIPELINES
MiniClustering operates on an encoder fe pre-trained via representation learning to generate suitable embeddings, and can be readily applied downstream of various unsupervised representation learn- ing frameworks. While we report performance using BYOL (Grill et al., 2020) as the backbone, the method demonstrates consistent effectiveness across other frameworks as well. We conduct exper- iments on CIFAR-10 10 and STL-10 with an imbalance ratio 10, employing encoders pre- pre-trannd with SimCLR R and MoCo. The results in Table 4 demonstrate that MiniClustering keeps effective

---

Published as a conference paper at ICLR 2026
when applied with different pre- pre-training frameworks, confirming its compatibility and robustness across representation learning methods.
Table 4: Clustering performance percentage (%) on CIFAR-10 and STL-10 (imbalance ratio 10) based various frameworks.
Datasets
CIFAR-I0
STL-10
Frameworks
Methods
ACC
CAA
NMI
ARI
ACC
CAA
NMI
Baseline
51.9
55.2
56.3
41.7
41.2
41.8
46.8
29.6
BYOL
MiniClustering
64.6
61.4
63.9
56.7
47.7
54.3
48.1
32.5
Bascline
43.4
47.9
45.8
30.4
38.1
38.8
37.5
21.6
SimCLR
MiniClustering
51.5
58.5
55.5
37.5
41.4
41.1
40.3
Baseline
37.9
41.1
38.1
26.3
30.5
30.6
21.1
12.8
Moco
MiniClustering
41.7
47.0
44.3
29.6
35.4
39.8
30.4
19.1
4.5 APPLYING EXISTING LONG-TAILED LEARNING LOSSES IN MINICLUSTERING
In MiniClustering, we estimate the weight for each target-cluster based the number of mini- clusters assigned to and directly apply re- weighting to the softmax loss. In supervised long-tailed learning, studies have proposed various re-weighting schemes based on class weights (Zhang al., 2023), Although these methods originally require label frequencies to compute the weights, our proposed approach enables their adaptation deep clustering by providing an unsupervised esti- mation of class-specific weights, thereby extending their applicability to the unsupervised setting. We select several well-established and empirically effective reightttt methods, including CB Softmax (Cui et al., 2019), BALMS (Ren et al., 2020) and EQL (Tan et al., 2020), to validate the generality and robustness of the weights estimated by MiniClustering. We replace L, with the respective loss functions from these works and employ the weights estimated by our method, con- ducting experiments on CIFAR-10 and STL-10 with an imbalance ratio of 10. As shown in Table 5, our approach successfully extends existing supervised long-tailed learning losses to the unsuper- vised clustering setting, achieving competitive performance. In some cases their performance even surpass the original MiniClustering, because MiniClustering uses only the basic weighted softmax loss for demonstration. The details these losses are Appendix
Table Clustering performance of employing various long-tailes learning losses in our scheme on CIFAR-10 and STL-10 (imbalance ratio 10).
Datasets
CIFAR-IP 10
STL-10
Losses
ACC
CAA
NMI
ARI
ACC
CAA
NMI
ARI
Ours
64.6
61.4
63.9
56.7
52.0
54.4
50.8
39.1
CB Softmax
64.1
61.3
63.0
55.3
49.1
53.3
47.8
33,6
BALMS
67.3
61.9
64.5
60.0
47.8
52.9
47.3
33.1
EQL
67.4
61.9
64.7
60.2
46.8
53.0
47.7
31.9
CONCLUSION
In this paper, we proposed mini-cluste guided long tailed deep clustering method MiniClustering, aiming address the critical problem of how re-weight model training in unsupervised man- ner. We introduced an additional mini-cluster head to enable fine-grained predictions and enhance the average cluster purity. By aligning the predictions from the target-cluster head with those from the mini-cluster head, we determine the appropriate weight for each target-cluster. We demonstrated the effectiveness of our approach across multiple datasets and further show that it can be seamlessly integrated into various unsupervised representation learning frameworks. Moreover, the estimated weights can be applied to different long-taii tailed learning losses, effectively extending previously pro- posed supervised long-tailed learning methods the domain of long-tailed deep clustering.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grants U24A20322, 62576094 and 62422118, in part by the Hong Kong UGC under grants UGC/FDSI I I/E03/24, UGC//DS1 under Grants 11219324 and N. N.CityU111417 This research work was also supported by the Big Data Computing Center of Southeast University.
and in part by the Hong Kong Research Grants Council
REFERENCES
Shaotian Cai, Liping Qiu, Xiaojun Chen, Qin Zhang, and Longteng Chen. Semantic-ehhanced image clustering. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 6879-6878, 2023.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with bibellddstrbbutio-aweeeee margin loss. Advances in Neural Information Processing Systems, 32, 2019.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-1607. PMLR, 2020.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15, pages 215-223, PMLR, 11-13 Apr 2011.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class- balaseea loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pages 245, 2009.
Andrew Estabrooks, Taeho Jo, and Nathalie Japkowicz. A multiple resampling method for learning from imbalanced data sets. Computational intelligence, 20(1):18-38) 2004.
Cean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent a new approach self-supervised learning. In Advances in Neural Information Processing Systems, volume 33, pages 21271-2111 212244. Curran Associates, Inc., 2020.
Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 1753-1759. International Joint Conferences on Artificial Intelligence Organization, 2017.
A. Hartigan and M. A. Wong. Algorithm as 136: k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1): 100-1088 1979.
Kaiming He. Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020.
Peihao Huang, Yan Huang, Wei Wang, and Liang Wang. Deep embedding network for clustering. In 2014 22nd International Conference Patlern Recognition, pages 1532-1537, 2014.

---

Published as a conference paper at ICLR 2026
Zhizhong Huang. Jie Chen, Junping Zhang, and Hongming Shan. Learning representation for clus- tering via prototype scattering and positive sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7509--522) 2023.
Yuheng Jia, Jianhong Cheng, Hui LIU, and Junhui Hou. Towards calibrated deep clustering network. In The Thirteenth International Conference on Learning Representations, 2025.
Ziyu Jiang, Tianlong Chen, Bobak J Mortazavi, and Zhangyang Wang. Self-damaging contrastive learning. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 4927-4939, PMLR, 18 24 Jul 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Hanyang Li, Yuheng Jia, Hui LIU, and Junhui Hou. You can trust your clustering model: A parameter-free self-boosting plug-in for deep clustering. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a.
Hongyu Li, Lefei Zhang, and Kehua Su. Dual mutual information constraints for discriminative clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages
Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive cluster- ing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8547- 8555, 2021.
Yunfan Li, Mouxing Yang, Dezhong Peng, Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. International Journal of Computer Vision, 1300)):225--2211, 2022.
Zhixin Li and Yuheng Jia. Conmix: Contrastive mixup at representation level for long-tailed deep clustering. In The Thirteenth International Conference on Learning Representations, 2025.
Zhixin Li, Yuheng Jia, Hui LIU, and Junhui Hou. Learning from sample stability for deep clustering. In Forty-second International Conference on Machine Learning. 2025b.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Oct 2017.
Yijie Lin, Mouxing Yang, Jun Yu, Peng Hu, Changqing Zhang, and Xi Peng. Graph matching with bi-level noisy correspondence. In Proceedings of the IEEE/CVF international conference on computer vision, pages 23362 23371, 2023.
Xu-Ying Ying Liu, Jianxin Wu, and Zui-Hua Zhou. Exploratory undersampling for class-imbalance learn- ing. IEEE Transactions Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2):539-550, 2008.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X. Yu. Large- scale long-tailed recognition in open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019.
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. Umap: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29), 2018.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In International Conference on Learning
Representations, 2021.
John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: A frame- work for adversarial attacks, data augmentation, and adversarial training in nlp. In Proceedings of the 2020 conference on empirical methods in natural language processing: System demonstra- tions, pages 119-126, 2020.
12

---

Published as a conference paper at ICLR 2026
Chuang Niu, Hongming Shan, and Ge Wang. Spice: Semantic pseudo-labeling for image clustering. IEEE Transactions on Image Processing, 31:7264-722 7278, 2022.
Qi Qian. Stable cluster discrimination for deep clustering. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 16645-16654 October 2023.
Md Rashadul Hasan Rakib, Norbert Zeh, Magdalena Jankowska, and Evangelos Milios. Enhance- ment of short text clustering by iterative classification. In International conference on applications of natural language to information systems, pages 105-117. Springer, 2020.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), pages 3992, 2019.
Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al. Balanced meta-softmax for long- tailed visual recognition. Advances neural information processing systems, 33:4175- 41866, 2020.
Jingru Tan, Changbao Wang. Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjie Yan. Equalization loss for long-tailes object recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2020.
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in Neural Information Processing Systems, 33:1513-1524, 2020.
Yaling Tao, Kentaro Takagi, and Kouta Nakata. Clustering- representation learning via instance discrimination and feature decorrelation, In International Conference on Learning Rep- resentations, 2020.
Yingfan Tao, Jingna Sun, Hao Yang. Li Chen, Xu Wang, Wenming Yang, Daniel Du, and Min Zheng. Local and global logit adjustments for long-tailed learning. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 11783-11792. October 2023.
Junjiao Tian, Yen-Cheng Liu, Nathaniel Glaser, Yen-Chang Hsu, and Zsolt Kira. Posterior re- calibration for imbalanced datasets. In Advances in Neural Information Processing Systems, vol- ume 33, pages 8101-8113, Curran Associates, Inc., 2020.
Kai Tian, Shuigeng Zhou, and Jihong Guan. Deepeluster: A general clustering framework based on deep learning. In Machine Learning and Knowledge Discovery in Databases, pages 809-825, Cham, 2017. Springer International Publishing.
Tsung Wei Tsai, Chongxuan Li. and Jun Zhu. Mice: Mixture of contrastive experts for unsupervised image clustering. In International Conference on Learning Representations, 2021.
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Mare Proesmans, and Lue Van Gool. Scan: Learning to classify images without labels. European Conference on Com- puter Vision, pages 268 188-285.22 Springer, 2020.
Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395-416 2007.
Yiru Wang, Weihao Gan, Jie Yang, Wei Wu, and Junjie Yan. Dynamic curriculum learning for imbalanced data classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, October 2019.
Junyuan Xie. Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In Proceedings of The 33rd International Conference on Machine Learning, volume 48, pages 478 487, New York, New York, USA, 20-22 Jun 2016. PMLR.
Yutong Xie, Fuchao Yang, and Yuheng Jia. Partial label clustering. In Proceedings of the Thirty- Fourth International Joint Conference Artificial Intelligence, pages 6678-6686, 2025.
13

---

Published as a conference paper at ICLR 2026
Jiaming Xu, Bo Xu, Peng Wang, Suncong Zheng. Guanhua Tian, and Jun Zhao. Self-taught convo- lutional neural networks for short text clustering. Neural Networks, 88:22-31, 2017.
Chunlin Yu, Ye Shi, and Jingya Wang. Contextually affinitive neighborhood refinery for deep clus- tering. Advances Neural Information Processing Systems, 36, 2023.
Chuyu Zhang, Hui Ren, and Xuming He. p2 OT: Progressive partial optimal transport for deep imbalanced clustering. In The Twelfth International Conference Learning Representations, 2024a.
Chuyu Zhang, Hui Ren, and Xuming He. SP² OT: Semantic-regularizeed progressive partial optimal transport for imbalanced clustering. arXiv preprint arXiv:2404.03444 2024b.
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang- Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O Amold, and Bing Xiang. Supporting clustering with contrastive learning. In Proceedings of the 2021 conference of the North American chapter of the association for compu- tational linguistics: Human language technologies, pages 5419-5430, 2021a.
Songyang Zhang, Zeming Li, Shipeng Yan, Xuming He, and Jian Sun. Distribution alignment: A unified framework for long-tail visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2361-2370, June 2021b.
Xu Zhang, Yuheng Jia, Mofei Song, and Ran Wang. Similarity and dissimilarity guided co- association matrix construction for ensemble clustering. IEEE Transactions on Knowledge and Data Engineering, 2025a.
Xu Zhang, Haoye Qiu, Weixuan Liang, Hui Liu, Junhui Hou, and Yuheng Jia. Generalization performance of ensemble clustering: From theory to algorithm. In International Conference on Machine Learning, pages 75901-75927. PMLR, 2025b.
Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. Deep long-tailed learning: A survey. IEEE transactions pattern analysis and machine intelligence, 4509::1075--18816 2023.
Zizhao Zhang and Tomas Pfister. Learning fast sample re-weighting without reward data. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 725-734, Octo- ber 2021.
Yan Zhao, Weicong Chen, Xu Tan, Kai Huang, and Jihong Zhu. Adaptive logit adjustment loss for long-tailed visual recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 3472-3480, 2022.
Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. Bbu: Bilateral branchh network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, June 2020.
Sheng Zhou, Hongjia Xu, Zhuonan Zheng, Jiawei Chen, Zhao Li. Jiajun Bu, Jia Wu, Xin Wang, Wenwu Zhu, and Martin Ester. A comprehensive survey on deep clustering: Taxonomy, chal- lenges, and future directions. ACM Computing Surveys, 57(3):1 2024.
Zhihan Zhou, Jiangchao Yao, Yan-Feng Wang, Bo Han, and Ya Zhang. Contrastive learning with boosted memorization. In Proceedings of the 39th International Conference on Machine Learn- ing, volume 162, pages 27367-27377. PMLR, 17-23 Jul 2022.
A STATEMENT ON THE USE OF LARGE LANGUAGE MODELS (LLMS)
We used the LLM solely to assist with writing, including grammar and spell checking, and improv- ing sentence clarity to enhance readability. All content generated by the LLM was manually checked by the authors, who take full responsibility for it. The LLM was not involved in any other aspects of the paper. The writing logic every sentence, as well as all experimental designs and conceptual contributions, were developed and completed by the authors.
14

---

Published as a conference paper at ICLR 2026
B ALGORITHMIC PSEUDOCODE
To improve readability, the complete workflow of the MiniClustering is illustrated in the following algorithm:
Algorithm MiniClustering
Require: Long-tailed dataset D, pre-trained encoder fer Number of target-clusters K and mini- cluster M, self-training threshold т, assignment threshold ö, trade-off parameters a and B epoch number T'.
Ensure: K target-clusters.
Initialize target-cluster head ft and mini-cluster head fm using K-means.
2: for = to I do
3:
Sample a batch {
4: Compute the prediction {p. (pr 1
Ji=1 confidence score {er Ji=1, and pseudo-label {ii") Ji=1
from the mini-cluster head.
5:
Compute mini-cluster self- self- training loss Lm by Eq. (1).
6:
== and pseudo-label (0g from Compute the prediction {pf / confidence score [c]],- the target-cluster head.
7:
Compute class-specific weights [w by Eq. (2).
8:
Compute re- weighted target-cluster self-training loss L, by Eq. (3). Compute similarity alignment loss Lg by Eq. (4).
9:
10:
Compute overall objective by Eq. (5).
11:
Update fer ft and fm-
12:
end for
Predict the data in D using fc-
C TRAINING DETAILS AND HYPERPARAMETER SETTINGS
We provide additional experimental details here that are not mentioned in the main text due to space constraints. Following (Chen al., 2020; Li and Jia, 2025), for all experiments on CIFAR- 10 and CIFAR-20, we set the first convolution layer have kernel size of 3x3 and a stride of 1, and remove the first max-pooling layer in ResNet-18 due to relatively small image sizes. We adopt the data MiniClustering, including
augmentation in (Chen et al., 2020). There four hyperparameters mini-cluster assignment threshold 6, mini-cluster number M, and two trade-off parameters and 3. To facilitate better reproduction of method, we provide the default hyperparameter settings used to obtain the reported results on different datasets. On CIFAR- 10 (imbalance ratio = 5), we set mini-cluster number M to 30, the assignment threshold 5 to 0.7, trade-off parameters a and and 1. On CIFAR-10 (imbalance ratio = 10), we set M to 40, 8 to 0.3, a to 0.2, and 3 to 0.5, On CIFAR-20 (imbalance ratio = 5), we set M 30, (imbalance ratio 10), we set M to 30, to 0.3, to 0.2, and 3 to 1. On STL-10 (imbalance ratio 5), we set M to 20, to 0.7, a to and 3 to 0.2. On STL-10 (imbalance ratio = 10), we set M to 40, å to 0.3, to 1, and 8 to 0.2. On CIFAR-10 (imbalance ratio 100), we set M to 20, 8 to 0.3, to 1, and 1.
0.5
0.3, a to 0.5, and 3 0.8. On CIFAR-20
D HYPERPARAMETER ANALYSIS
We present a detailed hyperparameter analysis using the control variates method. The effect of single hyperparameter is examined while keeping all others fixed their default values (please refer to Appendix C). The results are shown in Figure 5. We conduct experiments on CIFAR-10 and STL-10 with an imbalance ratio 10, using various hyperparameter settings. Our method con- sistently outperforms the baseline across the vast majority of hyperparameter combinations, except in one particular hyperparameter configuration. This demonstrates the effectiveness of our method under different hyperparameter settings, i.e.. CAA is slightly lower than that of the bascline when M = 50 on CIFAR-10. This demonstrates the effectiveness our method under different hy- perparameter settings. The clustering performance varies within reasonable range when different
15

---

Published as a conference paper at ICLR 2026
65
(a) CIFAR-10, M
(b) CIFAR-10,
(c) CIFAR- 10,
(d) CIFAR-10,
0
CAR
OSA
23
35
STL-10, M
(I) STL-10.
(g) STL-10. a
STL-10,
Figure 5: Hyperparameter analysis on CIFAR-10 10 and STL--10 with an imbalance ratio of 10.
hyperparameters are chosen. achieve better performance, the following recommendations are worth considering:
The number mini-clusters M. suggest selecting a value greater than the actual number of classes but more than four times that number. When M is set equal to the actual number of classes, the resulting partitioning closely resembles that of the target clusters, thereby failing to leverage the benefits offered by mini-clusters for performance improvement. For instance, STL- 10, setting M=10 leads to a noticeable performance drop. Conversely, when M is too large, the overly fine-grained partitioning hinders the encoder from learning representations well-suited for predicting target clusters. For example, on CIFAR-10, performance degrades when M is set to 50 or 60.
Mini-cluster assignment threshold 8. We find relatively stable across various settings and rec- ommend using 0.3 or 0.5, which assigns more mini-clusters to each target cluster. And a more inclusive choice of is also beneficial when embedding quality is poor.
Trade-off' parameters a and B. has more pronounced impact on performance. For instance, CIFAR- 10 tends to favor smaller values of a. while STL-10 prefers larger values. This behavior may be related to the intrinsic characteristics the datasets. Therefore, we suggest using 0.5 as a default setting for if no prior knowledge about the dataset is available. In contrast, the choice of 3 exhibits relatively stable performance across different values, with most settings yielding comparable results.
E MINICLUSTERING UNDER EXTREME IMBALANCE
To validate MiniClustering under extreme imbalance, we follow ConMix (Li and Jia, 2025) and evaluate CIFAR-10 with imbalance ratio of 100. As shown in Table 6, method maintains superior performance, demonstrating its effectiveness and generalizability in long-tailed scenarios.
Table 6: Clustering performance in percentage (%) on CIFAR-10 (imbalance ratio 100) across various methods.
Methods
SimCLR
SDCLR
BYOL
SCAN
CoNR
ProPos
DMICC
SeCu
LFSS
ConMix
Ours
ACC
32.2
33.5
28.1
38.2
31.3
25.1
37.7
40.4
CAA
31.6
32.3
35.5
40.4
20.9
38.3
26.1
22.2
28.5
38.5
44.6
NMI
36.4
39.4
43,1
48.1
25.6
42,4
41.1
15.0
46.1
53.4
54.0
ARI
20.2
22.2
25.3
28.2
14.6
24.9
22.4
10.6
28.4
33.5
34.6
16

---

Published as a conference paper at ICLR 2026
F MAXICLUSTERING ON LARGE-SCALE DATASETS
Since the mini-cluster head of our method needs to generate predictions for a number of clusters several times greater than target K, it may impose substantial training overhead on datasets with very large number of target clusters. To validate the generalizability of our method on large- scale datasets with high number of classes, we propose conceptually analogous variant termed MaxiClustering In contrast to the mini-cluster head, which produces number of clusters exceeding ground-truth class count, the maxi-cluster head generates reduced number clusters. In the network architecture, replace the mini-cluster head with maxi-cluster head, while keeping all other components unchanged. We propose the below weight estimation strategy for MaxiClustering:
M Wk = max Σ
ThJ 73
8), 0.5),
(6)
where Tk. (ila; E = Tthh = o is the set containing indices of samples predicted to target-cluster & and maxi-cluster; j at the same time, Tx {ila; E B, if k} is the set containing indices of samples predicted to tlrget-cluster k. MaxiClustering and MiniClustering share the same loss functions, differing only in their weight estimation strategies.
In the MaxiClustering setting, most target-clusters are typically assigned to only one maxi-cluster due the high similarity among samples from the same class. However, some target-clusters assigned to multiple maxi-clusters, which arises from inconsistency between the partitions of maxi- clusters and target-clusters. Under such circumstances, because target-clusters corresponding tail classes contain fewer samples, they are more likely to be assigned multiple maxi-clusters under the same assignment threshold. And they should be assigned higher weights during training to compensate for their scarcity.
To evaluate the effectiveness of MaxiClustering, we conduct experiments on Tiny ImageNet (Le and Yang, 2015) with an imbalance ratio of 10 and ImageNet-LT (Liu et al., 2019), following Li and Jia (2025). Tiny ImageNet contains 200 classes. experimental setting, we only use the training set for constructing the long- tailed version. Also, we adopt the same experimental setup for MaxiClustering as used the main results. reproduce SCAN (Van Gansbeke et al., 2020), SeCu (Qian, 2023), and LFSS (Li et al., 2025b), while results for the other methods were quoted from ConMix (Li and Jia, 2025). The clustering results in Table 7 demonstrate the effectiveness of this approach in large-scale, long-tailed clustering scenarios.
consistent with ConMix (Li and Jia, 2025) in
Table Clustering performance in percentage (%) on Tiny ImageNet (imbalance ratio 10) across various methods.
Methods
SimCLR R
SDCLR
BYOL.
SCAN
CoNR
ProPos
DMICC
ScCu
LFSS
ConMix
ACC
16.0
14.6
16.1
15.9
8.8
15.0
9.27
11.5
22.9
16.5
25.5
CAA
14.2
12.9
14,0
15.7
8.2
13.2
8.8
8.7
18.4
14,5
19.4
NMI
33.9
35.2
34.2
38.8
28.2
34.4
28.0
39.7
28.7
34.9
40.6
7.9
3.9
8.5
14.0
ImageNet-LT is long-tailed subset of ImageNet-1K -K (Deng et al., 2009), comprising 1 115.8K images
across 1,000 classes, with the number of samples per class ranging from to 1280. We following the setting in (Li and Jia, 2025), training ResNet-50 using BYOL (Grill et al., 2020) for 160 epochs and then perform MaxiClustering for 40 epochs. Performance of other compared methods cited from (Li and Jia, 2025). As shown in Table 8, MaxiClustering outperforms other methods. To facilitate the reproduction of our results, we provide a detailed description the experimental
Table 8: Clustering performance in percentage on (%) ImageNet-LT
Methods
SimCLR
SDCLR
BYOL
IDFD
CoNR
DMICC
ConMix
Ours
ACC
14.7
13.7
14.8
4.42
6.38
5.24
15.4
15.7
CAA
11.3
10.5
10.9
5.58
6.85
5.94
12.2
12.8
NMI
51.4
50.3
50.6
35.6
38.9
37.9
51.6
52.3
ARI
9.14
9.30
10.2
1.17
2.19
1.83
11.4
11.7
17

---

Published as a conference paper at ICLR 2026
setup. For MaxiClustering Tiny ImageNet (imbalance ratio 10), set constant learning rate to le-6, r to 0.9, M to 100, 8 to 0.3, a to and / to 0.5. The training consists of 800 epochs of BYOL, followed by 200 epochs of MaxiClustering, consistent with the experimental settings in the main results. For MaxiClustering on ImageNet-I.I, we set constant learning rate to le-5, T to 0.5, M to 200, 5 to 0.3, to I and [3 1. The training consists of 160 epochs of B BYOI,, followed by 40 epochs of MaxiClustering, where we use ResNet-50 as the backbone for encoder fe. This setting differs from our common experimental configuration.
G LONG-TAILED LOSSES IN DETAIL
In Section 4.5, we demonstrate that existing loss functions for supervised long-tailed learning (Cui et al., 2019; Ren et al., 2020; Tan et al., 2020) can be integrated with MiniClustering. Our specific approach to replace f, with the corresponding long-tailed learning losses, and substitute the weights calculated based label frequencies in the original method with our estimated weights, leaving everything else unchanged in MiniClustering. Here we present the formulas for these long- tailed losses.
CB Softmax (Cui et al., 2019) uses the number of samples per class as weights to calculate the effective number for each class for re-weighting softmax loss. Similarly, we leverage the number of mini-clusters assigned to each target-cluster, i.e., M/Wg; in Eq. (2) accordingly.
K
1
1-7 M/w wh.s log(pig) St = = ( E (i|
B (7)
Son-Sof
S
EES; j=l
where y is hyperparameter. The weight Eq. (1) is replaced by the new weight derived from the effective number based method, serving the same purpose to reduce the loss contribution from head classes and increase it for tail classes.
BALMS (Ren et al., 2020) re-weights softmax function according to label frequencies. We adopt this re-weight softmax loss in Eq. (3) as follow,
T3 exp(2;)
OBALMS (2:) = Σ3 TTy exp(2;))
(8)
where is the in element of 2, indicating the logit for the in target-cluster. represents the proportion mini-clusters assigned to the in target-cluster relative to the total. We replace the softmax function in f, with this one and cancel the weight outside the softmax loss.
EQL (Tan et al., 2020) re- re-weights softmax loss as well,
exp(2;)
(z)) E, Wi exp(2;)
where wy represents the weights estimated in Eq (2) for juh target-cluster. In this way, allow tail classes to have larger gradients while head classes have smaller gradients.
H STATISTICAL SIGNIFICANCE ANALYSIS
To demonstrate that the performance improvement is not due to random chance, we ran our method multiple times using different random seeds and computed the mean and standard deviations of its performance. For a fair comparison. we reproduced the most competitive methods, including SeCu, LFSS, and ConMix, for evaluation. Experiments were conducted on imbalanced versions of 10 and CIFAR-20 with an imbalance ratio of 10. We repeated each experiment five times using the same random seeds for all methods. The results are presented Table 9. As shown, our method consistently achieves significantly better performance in terms of mean values compared to other approaches, especially exhibiting a clear advantage on the CIFAR- 10 dataset. We use the t-test for significance testing and mark with an asterisk (*) the metrics where method is significantly better than other methods at the 95% confidence level. Our method is significantly better than the most competitive baseline in the majority of cases.
18

---

Published as a conference paper at ICLR 2026
Table 9: Statistical significance analysis of clustering performance CIFAR-IO and CIFAR-20 (imbalance ratio 10), with each method run five times using different random seeds.
CIFAR-I0
CIFAR-20
Method SeCu
ACC
CAA
NMI
ARI
ACC
CAA
NMI
ARI
33.120.9
58.121.0*
57.120.2*
41.320.3*
39.020.4
37.740.5
40.920.4*
25.020.2
LESS
54.5&1.8*
59.1+1.6*
57.920.14
42.0+0.3*
41.9H0.74
40.640.4*
46.040.3
28.240.1
ConMix
52.2+1.0
$6.7#1.2*
56.740.5*
40.420.6*
40.840.4*
38.640.2*
43.620.1
26.710.2*
MiniClustering
65.1+0.4
61.2+0.1
63.6+0.3
57.2+0,4
43.8+0,5
41.5+0.44
46.3+0.8
30.1=0.4
(a) Before Training
(b) After Training
Figure 6: Visualization of learned embeddings before and after training on CIFAR-10 (imbalance ratio 10). The class colors ordered from highest lowest sample count follows: blue, orange, green, red, purple, brown, pink, gray, dark yellow, light blue.
I VISUALIZATIONS OF THE LEARNED EMBEDDINGS
We present the encoder output embeddings before and after training with MiniClustering. We use UMAP (McInnes et al., 2018) to reduce the high-dimensional features of CIFAR-10 (imbalance ratio = 10) to two dimensions and visualize them as scatter plots in Figure 6. can be observed that after training, the embeddings of tail classes have become more distinguishable from those of head classes. For example, the dark yellow tail class now farther away from the blue head class (red box). Additionally, the pink and gray tail-class samples, which were relatively close to the purple head class before training, have become more separated from it after training (blue box).
J MINICLUSTERING ON TEXT DATASETS
Our method can also be applied to non-image data, its effectiveness does not rely image- specific components. To demonstrate this, we conduct experiments on text datasets. Specifically, construct imbalanced versions of the StackOverflow (Xu al., 2017) and AgNews (Rakib al., 2020) datasets with an imbalance ratio of 10, and train MiniClustering on them. Following (Zhang al., 2021a), we backbone for extracting features. We use WordNet (?) for synonym replacement as the data augmen- tation technique (Morris et al.. 2020). As in the image experiments, we train both the target-cluster head and the mini-cluster head using the estimated class-specife weights. Since distilbert-bant seart--bae--ile aase-nli- stsb-mean-tor meen-tokens itself is pre-trained model, we can directly use for MiniClustering. To validate the effectiveness, we will use K-means the embeddings output by distilbert baseert-baaeeniee tokens before training as baseline, and employ the self-labeling strategy with a threshold as com- parative method (i.e., only target-cluster head). We also reproduced ConMix (Li and Jia, 2025) on text datasets for comparison. The experimental results are shown in Table 10. Our method demon- strates significant improvement compared to the baseline and the self-labeling approach, proving its effectiveness. The experimental results confirm the effectiveness our approach in non- -m-iaaa scenarios.
(Reimers and Gurevych, 2019) the
19

---

Published as a conference paper at ICLR 2026
Table 10: Performance comparison on text datasets (imbalance ratio 10).
StackOverflow
NgWews
Method
ACC
CAA
NMI
ARI
ACC
CAA
NMI
ARI
Baseline
57.7
49.9
51.5
43.4
53.1
50.1
31.2
25.0
Self-labeling
59.0
51.9
53.3
45.4
56.6
52.5
38.6
30.7
ConMix
58.0
49.1
53.4
45.5
55.6
54.4
38.4
31.1
Ours
65.3
57.7
59.0
55.8
58.7
55.3
39.7
32.7
Table 11: Clustering results (in percent %) of various methods on balanced datasets and imbalance datasets with an imbalance ratio 10.
CIFAR-10
ClFAR-20
Averago
Data Type
Balanced
Imbulanced
Balanced
Imbalanceed
ART
ART
NMI
IDDD
81.5
71.1
66.3
47.3
48.4
33.1
42.5
42.6
26.4
28.7
28.6
15.1
50.1
47.7
35.3
Propos
ConMix
80.9
70.7
65.6
53.3
57.1
40.8
46.0
45.5
29.8
41.7
43.6
27.0
55.4
54.2
40.8
MiniClustering
56.7
K CLUSTERING PERFORMANCH ON BALANCED DATASETS
To evaluate overall performance of the proposed MiniClustering, we trained and evaluated our model on the balanced datasets CIFAR-10 and CIFAR-20. We used the combined set of the training and test splits of these datasets, while keeping all other experimental settings consistent with those described in the main text. The experimental results are shown in Table 11.
We compare our method with recent state-of-the-ar balanced deep clustering approaches, including IDFD (Tao et al., 2020) and ProPos (Huang et al., 2023), as well as with ConMix (Li and Jia, 2025), a novel long-tailed deep clustering method.
Experimental results demonstrate that method still achieves strong performance balanced datasets, outperforming both IDFD and ConMix. Although it falls slightly short of ProPos, the performance gap acceptable, where our method excels. Also, we compute the average of all metrics across the four datasets (long-tailed and balanced CIFAR-10 and CIFAR-20). It can be seen that our method outperforms other approaches, demonstrating superior comprehensive performance.
primary focus addressing long-tailed deep clustering,
L PROOF AND ANALYSIS OF THEOREM
We assume that samples from K classes follow a long-tailed distribution and are clustered into mini-clusters. The purity of in each mini-cluster is at least Let Smin and Smax be the minimum and maximum numbers samples in mini-clusters, respectively. by class & if its purity with respect to class &o is at least p. Let my denote the number of mini-clusters dominated by class Let é, denote the number of samples from class that lie outside the mini- clusters dominated by class Let N₂ denote the number of samples in class k. Then, that
mini-cluster is said to dominated
derive
N P.S. E S
IED.
DEDs
where Dy denotes the set of indices of mini-clusted dominated by class k. Pt denotes the purity of mini-cluster i, which is not less than and S; denotes the size mini-cluster i. Therefore, we can compute upper bound for my follow:
N
m&R
p.Smin
and larger N₂ implies larger upper bound for my-
Also, we can assume that all mini-clusters D3 contain only samples from class and derive the upper bound of Ng:
N₂ Σ P.S. + Ek mmax + Eks
20

---

Published as a conference paper at ICLR 2026
and obtain the lower bound of my:
N₂ Ek Smax
mIk
For two classes and j. D. > D means that the number of samples in class i is greater than that in class j. When the lower bound of my is greater than the upper bound of myj it necessarily follows
that m, > Mj. Thus, when N,-o N,
we can conclude sufficient condition for m, > m,-
Smax
phmin
N₁ SySmax
p
(N; 4))mmn
To facilitate analysis, we assume Smax R Smin. meaning that the sizes of all mini-clusters are ap- proximately equal. Then obtain that p N; Therefore, if is too small is too large, may lead to the failure of mj when N₁ Nj.
Moreover, both the minimum purity mini-clusters and the number e; of class samples located outside the mini-clusters dominated by class depend the quality of the embeddings. If the embedding quality low and the model struggles to consistently group samples of the same class into the same mini-clusters, the above property may fail to hold. Therefore, in MiniClustering, use a model pre-trained with unsupervised representation learning to ensure a reasonable embedding distribution.
21


Reviews:
[{"review_id": "AIrOHyJxkk", "reviewer": "Reviewer_KT3s", "summary": "The paper introduces a novel method for unsupervised long-tailed deep clustering to address the challenges posed by imbalanced data distributions. The proposed method attempts to mitigate the bias towards majority classes by introducing an auxiliary \"mini-clustering\" head which over-clusters the data (into many more clusters than the target number) to estimate class-specific weights for a re-weighted self-training loss.", "questions": "- Can you provide any formal argument or empirical correlation analysis linking the number of mini-clusters assigned to a target-cluster and the true class frequency?\n- If some minority classes are not well separated in the mini-clustering, how does this affect the final performance?\n- Could you provide standard deviations or confidence intervals for the results in Tables 1 and 2 to assess statistical significance? \n- Could you provide some visualizations of the learned embeddings (e.g., t-SNE or UMAP plots) to illustrate how well the method separates classes, especially minority ones?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The idea of estimating imbalance through over-clustering (mini-clusters) is creative and conceptually plausible.\n- The loss formulation is clearly described and easy to follow.\n- Comprehensive comparisons across multiple datasets and imbalance ratios, with also ablation studies to validate design choices. The addition of the maxi-clustering variant is also appreciated.\n- The paper is generally well-written and easy to read.", "weaknesses": "- While the method is motivated by empirical observations (Figures 1–3), there is no theoretical justification connecting the number of mini-clusters a target cluster occupies to its degree of long-tailedness. The proposed weight computation (Eq. 2) is purely heuristic, and the paper offers no probabilistic or geometric reasoning explaining why these weights should correlate with true class frequencies.\n- The method’s weight estimation depends on pseudo-labels from two heads — both trained simultaneously. Early in training, these pseudo-labels are noisy, which could cause unstable or biased weight updates. No mitigation (e.g., confidence annealing, temporal ensembling, or delayed weighting) is described, and the paper does not analyze the effect of pseudo-label noise on convergence.\n- The method relies on a good quality of the mini-clustering head to estimate class frequencies, but there is no analysis of how sensitive the method is to errors in this estimation.\n- The experimental results, while comprehensive, lack statistical significance analysis. It is unclear whether the observed improvements over baselines are statistically significant or could be due to random variation.\n- Figure 2 and 3 are a bit too small to read comfortably.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sTiGLffzSD", "reviewer": "Reviewer_amJw", "summary": "To address the problems of current deep clustering, where multiple assumptions about data distribution are unbalanced and existing long-tail learning methods rely on labels and do not apply to unsupervised scenarios, this paper proposes MiniClustering (a long-tail deep clustering method guided by mini-clustering). By introducing a dual clustering head (a target clustering head + a mini-clustering head, where the former outputs a prediction of the number of target clusters and the latter outputs a more fine-grained mini-cluster prediction), unsupervised category weight estimation is achieved based on the core phenomenon that \"head class samples are associated with more mini-clusters and tail classes with fewer mini-clusters\". Finally, its effectiveness is verified across multiple datasets, including CIFAR-10/20 and STL-10.", "questions": "1. Are there any rules for designing the number of mini-clustering heads, and what impact does it have on the stability of the model?\n2. How does batch size affect model performance? In particular, how does the model perform when the batch size is smaller than the total number of classes?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes an architecture of \"target clustering head + mini clustering head\" that estimates weights through the collaboration of clustering heads with different prediction granularities. This differs from the traditional single clustering head design and provides a new technical path for unsupervised reweighting.\n2. The algorithm still demonstrated reliable performance even in scenarios with extreme imbalances and large-scale data.", "weaknesses": "1. The proposed MiniClustering relies on several key hyperparameters that need to be manually tuned across different datasets and imbalance ratios. It lacks automated adaptation capabilities, which increases the threshold for practical application.\n2. All experiments were based on image data and did not involve non-image modalities such as text, audio, and video. Due to significant differences in data distribution and embedding characteristics across modalities, it is impossible to determine whether the \"mini-clustering mechanism\" of MiniClustering applies to non-image scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KBVQlBkEQd", "reviewer": "Reviewer_QMja", "summary": "This paper addresses the long-tailed distributions in deep clustering. Since existing deep clustering methods assume class-balanced data, they struggle with real-world imbalanced scenarios. The authors propose MiniClustering that introduces an auxiliary mini-cluster head to generate finer-grained clusters than the final target number. These mini-clusters are used to estimate class-specific weights in an unsupervised manner, enabling re-weighted training analogous to supervised long-tailed learning. The framework includes three loss components: mini-cluster self-training loss, re-weighted target-cluster loss, and a similarity alignment loss to prevent desynchronization between clustering heads. Extensive experiments on CIFAR-10, CIFAR-20, STL-10, Tiny ImageNet, and ImageNet-LT, across varying imbalance ratios, show that MiniClustering consistently outperforms state-of-the-art baselines on multiple metrics.", "questions": "Please refer to weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The dual-head design is intuitive. It allows the model to approximate class-wise importance without needing labels, which is both elegant and practical in clustering scenarios.\n2. The experimental results are strong and well-supported. Visualizations and benchmarks clearly show the method works better than existing ones across a variety of datasets and imbalance levels.", "weaknesses": "1. The main novelty of the paper lies in identifying the relationship between mini-clusters and target clusters, but these observations do not appear entirely original. The notion of mini-clusters has existed for years, with similar ideas explored in works like IIC and PICA. More specifically:\n\n   a. Phenomena 1 and 3 would benefit from more conclusive, clearly stated takeaways rather than descriptive observations.\n\n   b. Phenomenon 2 (“Mini-clusters can enhance purity”) is not very convincing, since purity naturally increases when clusters are subdivided. In the extreme case of infinite subdivisions, purity can trivially reach 1, which does not truly reflect clustering quality.\n\n   c. Phenomenon 3 does not directly correspond to estimating class probabilities. The analysis is limited to comparing the relative proportions of head versus tail classes, without demonstrating whether this distinction can reliably separate them.\n\n2. The results on balanced datasets are missing. It would be informative to see comparisons under standard settings—for example, relative to the results in Table 2 of ProPos. While this does not affect the core focus on long-tailed distributions, it would help situate the method’s overall competitiveness.\n3. The similarity alignment loss resembles a second-order alignment loss introduced in *Graph Matching with Bi-level Noisy Correspondence*. A proper citation and discussion of this connection would strengthen the methodological grounding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ReSFr4xeWl", "reviewer": "Reviewer_dtmG", "summary": "The paper introduces a novel method to cluster longt-tailed datasets where the class distributions are imbalanced. The proposed method, called MiniClustering, enables class re-weighting in unsupervised learning(clustering). The network uses two heads \n* A target-cluster head ($f_t$) that predicts the desired number of clusters ($K$).\n* A mini-cluster head ($f_m$) that predicts a much larger number of clusters ($M$).\nBy analyzing how many mini-clusters are associated with each target-cluster, the method estimates which target-clusters represent \"head\" vs. \"tail\" classes. It then calculates a weight for each target-cluster: clusters associated with many mini-clusters (head classes) get a smaller weight, and those with few (tail classes) get a larger weight. This weight is used in a re-weighted self-training loss ($L_r$) for the target-cluster head. The full model is trained with this loss, a self-training loss for the mini-cluster head ($L_m$), and a similarity alignment loss ($L_s$) to keep the two heads synchronized.", "questions": "1. How can the hyperparameters be tuned in a real-world, fully unsupervised scenario?\n2. How robust is the core assumption (Section 3, Phenomenon 3) across different types of data and encoders? \n3. Also related to 2., since there is a MiniClustering and MaxiClustering. What is the theoretical basis for this inversion, and does it imply the Phenomenon 3 is just a heuristic dependent on the $M/K$ ratio?\n4. How much of the performance gain comes from the pre-training versus the re-weighting? How much would the baseline improve if it would continue the 200 epochs of clustering?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The papers primary contribution is a functional mechanism to perform class balancing in a unsupervised setting. Which is mostly limited to supervised learning until now.\n2. The method is rather flexible in terms of representation learning framwork. The authors demonstrate this with BYOL,SimCLR and MoCO\n3. The authors provide a comprehensive ablation study to justify each component of the method and providing some insides how these components influence the clustering quality", "weaknesses": "1. Hyperparameter Sensitivity, Minicluster introduces 4 new hyperparametes that lack a unsupervised tuning strategy:\n* Number of mini-clusters\n* threshold for assigning a mini-cluster to a target-cluster\n* $\\alpha$ and $\\beta$ as trade-off weights for the losses. This parameters are quite significant as shown in figure 5.\n2. The MaxiClustering variant proposed in the Appendix feels a post patch that contradicts some of the core logic of the method.\n* MiniClustering ($M > K$) is based on the premise that head classes spread across many mini-clusters.\n* MaxiClustering ($M < K$) is based on the inverted premise that tail classes spread across many maxi-clusters.\nMaxiClustering inverts the re-weighting logic, hypothesizing that tail classes are associated with more maxi-clusters. This seems to be a direct contradiction of the \"phenomenon\" that motivated the entire paper.\n3. Implicit Assumption of K. This is a common limitation/weakness of most SoTA e.g. Secu and SCAN (ref. in the paper) but a weakness nonetheless", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)