Samples Are Not Equal: A Sample Selection Approach for Deep Clustering
Authors: Zhengxing Jiao
Yaxin Hou
Jun Ma
Yuhang Li
Ding Ding
Yuheng Jia
Hui LIU
Junhui Hou
Venue: ICLR 2026 Poster (2026)
Keywords: Deep Clustering
Clustering
Sample Selection
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We recognize that not all samples contribute equally to training a deep clustering model, so we select the most important ones for efficient training.

Abstract: Deep clustering has recently achieved remarkable progress across various domains. However, existing clustering methods typically treat all samples equally, neglecting the inherent differences in their feature patterns and learning states. Such redundant learning often drives models to overemphasize simple feature patterns in high-density regions, weakening their ability to capture complex yet diverse ones in low-density regions. To address this issue, we propose a novel plug-in designed to mitigate overfitting to simple and redundant feature patterns while encouraging the learning of more complex yet diverse ones. Specifically, we introduce a density-aware clustering head initialization strategy that adaptively adjusts each sample's contribution to cluster prototypes according to its local density in the feature space. This strategy mitigates the bias towards high-density regions and encourages a more comprehensive attention on medium- and low-density ones. Furthermore, we design a dynamic sample selection strategy that evaluates the learning state of samples based on the feature consistency and pseudo-label stability. By removing sufficiently learned samples and prioritizing unstable ones, this strategy adaptively reallocates training resources, enabling the model to consistently focus on samples that remain under-learned throughout training. Our method can be integrated as a plug-in into a wide range of deep clustering architectures. Extensive experiments on multiple benchmark datasets demonstrate that our method improves clustering accuracy by up to $\textbf{6.1}$\% and enhances training efficiency by up to $\textbf{1.3$\times$}$. Code is available at [https://github.com/notoaudrey/Samples-Are-Not-Equal](https://github.com/notoaudrey/Samples-Are-Not-Equal).

Published as a conference paper at ICLR 2026
SAMPLES ARE NOT EQUAL: A SAMPLE SELECTION APPROACH FOR DEEP CLUSTERING
Zhengxing Jiao', Yaxin Hou', Jun Ma', Yuhang Li', Ding Ding', Yuheng Jia Hui Liu", Junhui Hou*
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China KKey Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China "Scoool of Computing Information Sciences, Saint Francis University, Hong Kong, China 'Department of Computer Science, City University of Hong Kong, Hong Kong, China {jiaozx, yaxin, 22 2202422322 0242351, yuhangli, di dingding @sjiaa edu. en, h2 hiliuus www.fu.eeeeee edu. hk, jh. yiu edu. hk
ABSTRACT
Deep clustering has recently achieved remarkable progress across various do- mains. However, existing clustering methods typically treat all samples equally, neglecting the inherent differences in their feature patterns and learning states. Such redundant learning often drives models to overemphasize simple feature patterns high-density regions, weakening their ability to capture complex yet diverse ones in low-density regions. To address this issue, propose novel plug-in designed to mitigate overfitting to simple and redundant feature patterns while encouraging the learning of more complex yer diverse ones. Specifically, we introduce a density-aware clustering head initialization strategy that adap- tively adjusts each sample's contribution to cluster prototypes according to its local density in the feature space. This strategy mitigates the bias towards high- density regions and encourages a more comprchensive attention on medium- and low-density ones. Furthermore, we design dynamic sample selection strat- egy that evaluates the learning state of samples based on the feature consis- tency and pseudo-label stability. By removing sufficiently learned samples and prioritizing unstable ones, this strategy adaptively reallocates training resources, enabling the model to consistently focus samples that remain under-learned throughout training. Our method can be integrated as a plug-in into a wide range of deep clustering architectures. Extensive experiments multiple bench- mark datasets demonstrate that our method improves clustering accuracy by up 6.1% % and enhances training efficiency by up 1.3x. Code is available https: wtw.://ww..coooooocuueeeeeeeeeee Saaettaaadddee/SSpleeeseeee Are Not pot-Eqqe eqqu-Equl
INTRODUCTION
Deep clustering integrates the representation learning ability of deep neural networks with the com- mon objective used in traditional clustering algorithms, achieving significant advances in recent years (Zhou et al., 2025; Ren et al., 2025). Existing deep clustering methods utilize self-supervised learning techniques (He et al., 2020; Chen et al., 2020a; Grill et al., 2020) to handle complex high- dimensional data effectively, thus mitigate the limits of traditional clustering algorithms (Xie et al., 2016; Zhang et al., 2025).
Recent deep clustering methods (Gansbeke al., 2020; Li et al., 2021; Jia et al., 2025) use pre- trained encoders to obtain discriminative representations. A key property of these representations is that adjacent representations often share semantic similarity. Consequently, these methods exploit this underlying structure to generate supervisory signals for training. To further investigate the rep- resentation distribution in the feature space, analyze the local density using the k-nearest neigh- bor distance and observe a consistent phenomenon across datasets, most samples concentrated in
"Corresponding author.

---

Published as a conference paper at ICLR 2026
high density
Ra Rancom
medium dersity
CDC
low disniity
Oues
Sanple 621
602
2024
k-nearest reighhor distance
(a)
(c)
Figure 1: (a) Sample size distribution across varying densities in the STL-10 dataset, with density estimated by k-nearest neighbor distance. (b) Random selected images from different density re- gions of the STL-10 dataset. High-density samples are more simple and redundant, while medium- and low-density samples are more complex and diverse. (c) Performance across different density regions under three initialization strategies (i.e., Random, CDC, and Ours), showing our proposed method's consistent improvements, particularly in low-density samples.
high-density regions, while medium- and low-density regions contain far fewer samples (Fig. 1(a)). Moreover, images selected from different density regions reveal distinct feature patterns. High- density samples tend to be simple and redundant, while medium- and low-density samples exhibit greater complexity and diversity (Fig. 1(b)). For example, the STL--10 truck class, high-density samples usually represent typical truck appearances, capturing views from the side oblique front with holistic features. In contrast, medium- and low-density samples present richer variations, cov- ering various truck models, multiple viewpoints, and fine-grained local details. As illustrated in Fig. I(c), such differences in feature patterns lead to notable performance disparities, simple and redundant high-density samples are learned quickly due their abundance and ease, while more complex and diverse medium- and low-density samples remain difficult to master. We attribute this phenomenon to the overfitting of simple and redundant feature patterns.
In this paper, we introduce novel plug-in designed to mitigate overfitting to simple and redundant feature patterns while encouraging the learning of more complex and diverse ones. Our method tackles this challenge from two perspectives. First, we propose a density-aware clustering head initialization strategy, which adaptively weights each sample's contribution its cluster prototype based on local density, thus reducing the bias caused by simple and redundant ples dominating the clustering head initialization. Second, we develop a dynamic sample selection strategy guided by feature consistency and pseudo-label stability, which identifies and temporarily suspends training on already well-learned samples. This strategy adaptively reallocates the model's learning capacity toward more unstable samples, thereby fostering more efficient and comprehen- sive learning trajectory.
sam-
The proposed method is plug-and-play, enabling seamless integration with existing deep cluster- ing methods while significantly improving their clustering performance and training efficiency. In summary, the main contributions of this work are as follows.
We identify general phenomenon in deep clustering where sample distributions are dominated by simple and redundant samples in high-density regions. This dominance causes models overfit- ting to simple and redundant samples and overlook complex and diverse samples in low-density regions, limiting their discriminative power.
We propose two strategies: (i) awwse clustering head initialization employs local density based adaptive weighting prevent cluster prototype bias towards high-density regions; and (ii) feature consistency and pseudo-label stability based dynamic sample selection identifies and tem- porarily discards sufficiently learned samples during training, enabling models allocate more capacity to unstable samples.
Extensive experiments on multiple benchmark datasets show that our method consistently boosts the performance of state-of-the-arr deep clustering methods by up to 6.1 %. Moreover, it improves training efficiency by up to 1.3x, achieving superior results with fewer training samples.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Deep Clustering. Deep clustering leverages the representation learning capability of deep neural networks to perform clustering. Existing methods can be broadly divided into two categories. The first paradigm consists of two-stage methods, where a deep neural network is first trained to ex- tract -in-dimensional representations, which are then clustered using traditional algorithms such as K-Means or spectral clustering (Xie et al., 2016; Huang et al., 2014). While this paradigm simplifies optimization, its performance is inherently constrained by the quality of extracted rep- resentations, which may not be optimally aligned with the final clustering objective. The second paradigm encompasses end-to-end iterative methods that jointly optimize representations and clus- ter assignments. These methods utilize the model's own outputs as supervisory signals and can be further distinguished by their learning strategy: self-training iteratively generates pseudo-labets for all samples to guide optimization (Xie et al., 2016; Guo et al., 2017); self-labeling further selects high-confidence predictions to provide more stable and reliable supervisory signals (Gansbeke al., 2020; Jia et al., 2025; Wu et al., 2025; Li et al., 2025a); and contrastive learning enhances represen- tations by managing similarity relationships across different data views (Shen al., 2021; Li al., 2021: Li & Jia, 2025; Li et al., 2025b). Recently, methods that leverage large-scale pre-trained mod- els like CLIP have emerged as a powerful approach. By utilizing pseudo-labels derived from textual representations or enforcing cross-modal neighborhood consistency, these techniques introduce rich external knowledge, leading to significant performance gains (Cai et al., 2023; Li et al., 2024; Qiu et al., 2024). Overall, due to their effective exploitation of high-confidence samples, end-to-end self-labeling methods represent a particularly promising direction in deep clustering.
Sample Selection. Sample selection has been widely studied in deep learning for enhancing train- ing efficiency, model robustness, and generalization. Existing methods can be broadly classified as either static or dynamic. Static methods perform one-time sample selection, either prior to or during the initial stage of training, to construct a compact yet representative subset. For example, data pruning techniques eliminate redundant or low-value samples based metrics such forget- ting events (Toneva et al., 2019; Killamsetty et al., 2021c) or influence functions (Paul et al., 2021), while core-set methods (Xia et al., 2023; Braverman al., 2022) seek identify a minimal subset that best approximates the full data distribution. Although effective at reducing computational cost, a key limitation of static methods is their inability to adapt to the evolving importance of samples as training progresses. In contrast, dynamic methods continuously adjust sample weights or sam- pling probabilities throughout the training process to prioritize the most informative or challenging examples (Nguyen et al., 2023: Yuan et al., 2025). A prominent dynamic technique is importance sampling, where a sample's ss selection probability is proportional to its estimated utility. Common utility metrics include loss values (i.e., emphasizing hard examples), gradient norms (i.e., quan- tifying a sample's 's contribution to parameter updates), and prediction uncertainty (i.e., identifying ambiguous samples near decision boundaries) (Chang et al., 2017a; Mindermann et al., 2022; Kil- lamsetty et al., 2021b). By adaptively focusing learning resources on such informative data, these dynamic mechanisms promote more efficient training and improved model generalization.
Active Learning. Sampling strategies are also a central topic in Active Learning (AL), where the goal is improve model performance with limited annotation budgets. AL methods select infor- mative samples based on criteria such as uncertainty (Gal et al., 2017) and diversity or representa- tiveness (Sener & Savarese, 2018) Recent work further shows that sample importance should be updated dynamically as the model changes (Ash et al., 2020; Killamsetty et al., 2021a).
However, most existing sample selection methods are designed for supervised learning. These strategies become inapplicable in the unsupervised deep clustering paradigm due to the absence of ground-truth labels. To bridge this gap, we introduce novel dynamic sample selection strategy specifically tailored for deep clustering. Our method enables the model to adaptively discard suf- ficiently learned samples throughout the training process, enhancing both clustering performance and training efficiency in fully unsupervised manner. Although AL focuses on selecting samples for labeling, the idea of estimating how "valuable" each sample is is related to our setting. In deep clustering, no labels are available, and our goal is different: instead of querying informative sam- ples, we aim to remove samples that have already been well learned so the model can focus on more challenging ones.
3

---

Published as a conference paper at ICLR 2026
3 PROPOSED METHOD
Overview. Our proposed deep clustering enhancement component explicitly incorporates sample- level diversity modeling during both the clustering head initialization and training phases. This approach is designed counteract learning bias caused by imbalanced sample feature patterns. The method consists of two core components: a density- clustering head initialization strategy and dynamic sample filtering strategy based on feature consistency. In the initialization phase, we use a pre-trained feature encoder to obtain preliminary feature representations (Sec. 3.1). These representations are then used to execute our density-aware -aware clustering head initialization strategy, which aims to generate a set of initial cluster prototypes not dominated by high-density, redundant samples. During the iterative training phase (Sec. 3.2), we adopt a dynamic training strategy. We filtering strategy to identify and temporarily remove samples that
use the feature consistency-bu have become stable. This allows the model to focus more learning resources on complex or unstable samples that have not been fully learned. Notably, our method is designed plug-in that can be readily incorporated into a wide range of deep clustering architectures, consistently improving their clustering performance. We present the pseudo-code of our method in Appendix D.
Notation. Denote by Du = {ax E (1. N}} a training set of N unlabeled samples belonging K semantic clusters. A deep clustering model is composed of feature encoder fo(-) and a clustering head g(-), where 0 denotes the network parameters of the feature encoder. The feature encoder maps input sample to high-dimensional feature representation
fo(x)) while the clustering head predicts the pseudo-label distribution p 90(z), where e denotes the parameters of the clustering head. For each sample Is, we apply weak augmentation i and strong augmentation Ω² to get two different views x,"" and x. and their predictive distributions are denoted as P." and P...
3.1 DENSITY-AWARE CLUSTERING HEAD INITIALIZATION
Most clustering-head-be based methods leverage self-supervised learning to obtain a pre-trained fea- ture encoder and then attach a clustering head for further training. However, the parameters of the clustering head are typically initialized randomly (Gansbeke et al., 2020; Li et al., 2021), which unstable and often disrupts the pre-trained representations, thereby slowing convergence and re- ducing robustness. To address this, CDC (Jia et al., 2025) introduces prototype-hased initialization, which mitigates the degradation caused by random initialization. Nevertheless, despite its stabil- ity, prototype-based initialization struggles with imbalanced feature distributions. Since it computes cluster prototypes by averaging all sample features, the resulting prototypes are inevitably domi- nated by the abundant, redundant samples in high-density regions, failing to capture the true data structure. Consequently, the initial prototypes biased toward high-density regions hinder the model from effectively learning diverse feature patterns.
To mitigate this issue, we propose a density- aware clustering head initialization strategy to adaptively adjust each sample's contribution to the initial pro- totypes according to its local density in the feature space (Fig, 2(b)). The procedure consists of two key steps: (1) perform an initial clustering assignment using features extracted by pre-trained backbone; (2) recompute the cluster prototypes via density- weighted aggregation within each cluster.
(a)
(b) Desity-weighted Prototype
Original Prototype
we begin by extracting feature repre- sentations for all samples using the pre-trained en- coder fa(.), resulting in Z = {aI...., Z. We then perform initial clustering by applying the standard K-Means algorithm on Z to obtain pseudo- labels L f4, f4.....INN Based these assign- ments, we introduce aense re-estimation of cluster prototypes. For each sample, we first estimate its local density using the average k-nearest
Specifically,
Figure 2: (a) Cluster prototypes domi- nated by high-density samples. (b) Density- aware initialization, where sample weights are adaptively adjusted according to density, with color intensity indicating weight magni- tude.
4

---

Published as a conference paper at ICLR 2026
neighbor distance within the same cluster. The corresponding local density weight w; is defined as:
1 1 2.
04 exp
(1)
k
where =10) denotes the j-th nearest neighbor of 2, within its assigned cluster, and a is a tunable hyperparameter controlling the sensitivity of the weights to distance. Larger values of a make the weights more sensitive to changes in local density. Once the density weights are computed, the prototype Cj for cluster j is obtained by the weighted average:
Exec, W/ 2:
Gj Σxec, wi
where Cj denotes the set of samples assigned to cluster j. This density- re-estimation enhances the contribution of low-density samples while attenuating the dominance of high-density ones. Fi- nally, the refined prototypes C = {cl,...,cK } employed to initialize the clustering head weights of the deep clustering model. By incorporating density information, this strategy better preserves cluster structures across regions of varying density and mitigates bias toward redundant samples, thereby providing a stronger starting point for subsequent deep clustering training.
3.2 DYNAMIC SAMPLE SELECTION
In unsupervised clustering tasks, models tend to quickly learn to group samples from high-density regions, which are often characterized by simple and redundant patterns. Continuing to allocate computational resources to these well-learned samples inefficient and may hinder effective learn- ing of samples from low-density regions, which are typically more complex and information-rich. We initially considered density-based sample removal strategy, where local density is estimated via k-nearest neighbor distance to discard redundant samples in high-density regions. However, this approach is computationally expensive due to repeated neighbor searches and global distance computations, and density estimates in high-dimensional feature spaces may be imprecise, poten- tially leading to the unintended removal informative samples and limiting model generalization. propose dynamic sample selection strategy that adaptively allocates
To address this challenge, learning resources based on each sample's 's learning state during training. Unlike supervised learning, clustering lacks explicit ground-truth labels, making it difficult to assess sample difficulty. Tradi- tional sample difficulty measures based on loss or gradient dynamics often fail to accurately reflect the actual learning state of samples. Although pseudo-labels offer an intuitive alternative, they are prone to noise, particularly in the early stages of training, which risks introducing bias into sample selection.
Our method builds upon the principle of prediction consistency. The central idea is that a well- learned sample should yield stable and robust feature representations, such that the model produces consistent predictions even when the input undergoes different transformations perturbations. In other words, if the model has developed reliable understanding of a sample, its predicted distribu- tion should remain largely invariant under reasonable augmentations. This property not only reflects the stability of the learned representation but also serves as an implicit indicator of the sample's learning state. By leveraging prediction consistency, we can more effectively distinguish between samples that are already well captured by the model and those that require further attention, thereby guiding dynamic resource allocation during training.
Prediction Consistency Between Weak and Strong Views. For each input sample s,, we generate two augmented versions: weakly augmented view x. and a strongly augmented view x Passing these through the model yields their predictive distributions pM and P.s. To quantify their alignment, we define the prediction consistency score S, as the cosine similarity between the two distributions:
S, cos(P." P!))
(3)
A higher value of S; indicates that the model produces consistent predictions under different levels of augmentation, suggesting that the corresponding sample is being reliably captured by the learned representation.
Stability Evaluation via Second-Order Differences. High consistency at a single time step does not necessarily indicate that sample is stably learned, as even uncertain samples may occasionally
5

---

Published as a conference paper at ICLR 2026
exhibit strong agreement by chance. To better capture temporal stability, track each sample's prediction consistency over three consecutive training epochs. Let denote the current epoch index. For sample 24, the second-order difference is computed across the tth, (t (t-1)th, and (t - 2) epochs as:
A -: 25((-1) S(--2)
(4)
This metric reflects the "acceleration" of consistency changes, effectively capturing fluctuations in the learning dynamics. A sample is considered stable when
A <,
(5)
where is a predefined stability threshold. Under this condition, the sample exhibits both high prediction consistency and temporal stability, signifying that it has reached well-learned state.
Pseudo-Label Consistency Check. As an additional requirement, we further impose that sample's s pseudo-labels remain consistent across the most recent three epochs. If sample's 's pseudo-label changes during this period, it is regarded as unstable, even if its prediction consistency shows only minor fluctuations. This criterion helps filter out noisy or ambiguous samples that may otherwise be misclassified as stable.
Based on the above criteria, we design a dynamic sample selection strategy that consists of two steps:
Exclusion. A sample is regarded as stable if it simultaneously satisfies the following two condi- tions: (i) its prediction consistency remains stable, and (ii) its pscudo-label does not change over the recent epochs. Such samples are temporarily excluded from the training batch in order to avoid redundant computation.
Reinclusion. In contrast, if a sample exhibits significant fluctuations in prediction consistency (i.e., | 2 e), or if its pseudo-label changes during the observation window, it is considered unstable. These unstable samples are retained in the training queue and may even be assigned higher training priority to encourage further refinement.
This strategy allows the model to adaptively allocate computational resources toward unstable sam- ples that require further learning, while deprioritizing stable ones. As a result, it mitigates the bias introduced by uneven feature distributions and promotes more balanced representation learning across diverse data regions. Unlike most existing sample selection strategies in supervised learning, which primarily rely on loss values gradient magnitudes, our method is tailored for unsupervised clustering by leveraging prediction consistency and pseudo-label stability as more reliable indicators of the learning state.
EXPERIMENT
4.1 EXPERIMENT SETTINGS
Datasets. We evaluate our method on six standard benchmarks, including CIFAR-10 (Krizhevsky, 2009), CIFAR-20 (Krizhevsky, 2009), STL-10 (Coates al., 2011), ImageNet- (Chang et al., 2017b), ImageNet-Dogs (Chang et al., 2017b), and Tiny-ImageNet (Le & Yang, 2015). Following previous works (Chang al., 2017b; Jia et al., 2025), we construct ImageNet-10, ImageNet-Dogs, and Tiny-ImageNe by selecting 10, 15, and 200 subsets from ImageNet- 1k (Deng et al., 2009), respectively. We strictly follow the experimental protocols of the corresponding baselines to ensure a fair and consistent comparison.
Baselines. We integrate our proposed plug-in into four advanced deep clustering algorithms, includ- ing CC (Li al., 2021), TCL (Li al., 2022), SCAN (Gansbeke et al., 2020), and CDC (Jia et al., 2025). In addition, we compare the enhanced versions against several representative deep clustering methods, including BYL (Grill al., 2020), NNM (Dang et al., 2021). GCC (Zhong al., 2021), IDFD (Tao et al., 2021), TCC (Shen et al., 2021), SPICE (Niu et al., 2022), ProPos (Huang et al., 2023), SeCu (Qian, 2023), and CoNR (Yu al., 2023).
Experiment Settings. We evaluate clustering performance using three standard metrics, i.c., Ac- curacy (ACC) (Li & Ding, 2006), Normalized Mutual Information (NMI) (Strehl & Ghosh, 2002),

---

Published as a conference paper at ICLR 2026
and Adjusted Rand Index (ARI) (Hubert & Arabie, 1985). Higher scores indicate better alignment
with the ground-truth clustering. For CC (Li et al., 2021), TCL (Li et al., 2022), SCAN (Gansbeke et al., 2020), and CDC (Jia et al., 2025), we adopt MoCo-v2 (Chen et al., 2020b) as the pre-trained backbone and strictly follow the data augmentation protocols of CDC (Jia et al., 2025).
Additional experiment settings details are provided in Appendix B.
4.2 MAIN RESULTS
Comparison of clustering performance (%) on six standard benchmarks. The best result for each method is highlighted in bold, while the overall best result is marked with an underline.
Table 1:
CIFAR-10
CIFAR-20
STL-10
ImageNet-10
ImageNot-Dogs
Tuy--IaageNet
Method
Avg.
ART
NMT
ACC
ACC.
NMT ART
NMI
BYOL [NcuriPS 20]
87.5
75.2
53.3
75.4
60.9
NNM (CVPR221]
84.3
74.8
70,9
484
31.6
80.8
69.4
65.0
GCC [ICVV21]
85.6
76.4
72.8
47.2
47.2
30.5
78.8
68.4
63.1
90.1
84.2
82.2
52.6
49.0
362
13.8
34.7 75
56.7
IDFD [CC
81.5
71.1
66.3
42.5
42.6
26.4
75.6
64.3
$7.5
95.4
89.8
90.1
59.1
54.6
41.3
TCC [NourIPS'2]]
90.6
79.0
73.3
49.1
47.9
31.2
81.4
73.2
68.9
89.7
84.0
82.5
59.5
55.4
41.7
SPICE (TIP"22]
91.7
85.8
83.6
58.4
58.4
42.2
92.9
86.0
95.9
90.2
91.2
67.5
62.7
52.6
30.5
44,9
16.1
68.7
ProPos [TPAMP22]
94.3
88.6
88.4
60.6
86.7
75.8
96.2
91.8
77.5
73.7
67.5
46.0
SeCu [ICCV233
95.0
86.1
85.7
55.1
83.6
73.3
CoNR (NcurPPS22]
93.2
86.7
86.1
60.4
60.4
44.3
92.6
85.2 84.6
96.4
91.1
92.2
79.4
74.4
66.7
30.8
46.1
18.4
71.6
CC [AAA[21]
86.3
77.6
74.3
52.5
50.9
35.2
80.0
72.7 67.7
90.5
87.6
84.6
63.3
60.0
49.3
13.9
43,0
5.5
60.8
CC+Ours
18.5
81.3
79.7
57.4
57.5
42.3
91.4
83.2
82.5
97.2
93.1
93.9
66.1
63.3
52.8
19.1
46.5
7.6
66.9 (+6.1)
TCL [UFCV222]
88,2
80.4
76.8
53.1
52.9
35.7
86.8
79.9
75.7
88.4
83.3
80.0
64.4
62.3
51.6
17.2
45.5
7.4
62.7
TCL+
90.0
84.1
80.6
57.9
57.6
41.1
90.4
81.8
80.6
95.3
91.1
90.1
73.7
70.1
60.5
22.1
46.6
10.1
68.0
(+5.3)
SCAN (POCC"20]
90.2
83.7
81.0
52.1
54.4
91.4
83.4
97.2
93.9
71.8
60.6
51.9
SCAN=Ours
92.2
85.6
84.5
57.3
40.2
85.1
97.8
95.3
76.7
73.9
67.2
52.3
(42.3)
CDC [JCLR'25]
94.2
88.1
61.9
60.9
46.1
93.0
85.8
85.6
97.1
92.7
93.6
79.5
77.0
70.5
31.3
45.0
18.0
72.7
CDC+Ours
94.7
88.7
89.0
62.7
61.5
46.6
93.6
86.9
86.8
97.2
92.9
93.8
84.3
78.2
7388
32.2
46.1
18.6
73.8 (+1.1)
Significant Clustering Improvement. As shown in Table 1, our method delivers consistent and substantial improvements when integrated with four representative deep clustering methods. On six benchmark datasets, it achieves performance gains of up to 6.1 % when integrated with CC (Li et al., 2021) and a significant 1.1% % even when combined with the state-of-the-ar method CDC (Jia al., 2025). Furthermore, our plug-in enables CDC to achieve the highest performance on five out of the six datasets (i.e., CIFAR- 10, CIFAR-20, STL-I0, 10, ImageNet-Dogs, and Tiny-ImageNet), while its integration with SCAN (Gansbeke et al., 2020) yields the best result on ImageNet-10. These findings demonstrate the effectiveness of our method across a diverse range of baselines.
Improved Training Efficiency. Beyond clustering performance, evaluate the training efficiency of our method. By dynamically pruning well-lcarned samples, the number of samples involved in gradient updates decreases progressively. This reduction lowers computational cost without degrad- ing clustering performance. Table 2 quantifies the average number of samples retained per epoch when integrating our strategy with SCAN (Gansbeke et al., 2020) and CDC (Jia et al., 2025) on CIFAR-10, 10, CIFAR-20, and 10. The results demonstrate that our method consistently reduces the number of training samples across all settings, leading notable training time savings. Notably, this leads to an average efficiency improvement of 1.3x, confirming that our method enhances both clustering performance and practical training efficiency.
Table 2: Comparison of training efficiency.
Method CIFAR-10
CIFAR-20 STL-10
Avg.
CDC+Ours
17.4%
12.1% %
14.3%
14.6%
SCAN+Ours
37.7%
52.9%
29.9%
40.2%
CC+Ours 44.5%
30.9%
28.1%
34.5%
Speed Up
1,3x
4.3 FURTHER ANALYSTS
Ablation Study. We conduct ablation studies on CIFAR-10, CIFAR-20, and STL-10 to validate the contribution of each proposed module. As summarized in Table 3, both modules consistently im-
7

---

Published as a conference paper at ICLR 2026
prove performance. Specifically, the Density-Aware Clustering Head Initialization (DACHI) yields average gains of 1.2%. 1.6%, and 1.1% on CIFAR-10, CIFAR-20, 20, and STL-10, respectively, for both CDC (Jia et al., 2025) and SCAN (Gansbeke et al., 2020). This indicates that the proposed clustering head initialization strategy better captures diverse feature patterns. The Dynamic Sample Selection (DSS) module further enhances robustness to diverse sample feature patterns, providing additional performance gains. The synergistic effect of combining DACHI and DSS achieves the best results on all datasets. For instance, on STL-10, our proposed method outperforms the original CDC and SCAN by 1.0% and 1.8%, respectively. These consistent improvements validate the effectiveness of our proposed modules in boosting clustering performance.
Table 3: Ablation study of our proposed modules on three benchmarks (i.e., CIFAR-10, CIFAR-20, and STL-10). We enhance both CDC and SCAN with Density- Awaree Clustering Head Initialization (DACHI) and Dynamic Sample Selection (DSS).
CIFAR- 10
CIFAR-20 20
STL-I0
Method
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
Avg.
CDC
94.2
88.1
88.1
61.9
60.9
46.1
93.0
85.8
85.6
78.2
+DACHI
94.5
88.4
88.6
62.5
61.5
46.6
93.5
86.6
86.4
78.7+0.5
+DSS
94.5
88.4
88.7
62.3
61.4
46.4
93.3
86.3
86.0
78.6,0.4
Ours
94.7
88.7
89.0
62.7
61.5
46.6
93.6
86.9
86.8
78.94o.7
SCAN
90.2
83.7
81.0
52.1
54.4
38.0
91.4
83.4
82.6
73.0
+DACHI
91.7
85.2
83.8
55.1
57.2
39.8
92.5
84.8
84.5
74.9 1.9
+DSS
90.5
83.9
81.4
53,9
56.4
39.6
91.9
84.0
83.4
73.9
Ours
92.2
85.6
84,5
55.4
57.3
40.2
92.7
85.1
84,9
75.3+
Mitigating Overfitting Redundant Patterns. As shown in Table 4, all methods perform well on high-density regions with relatively simple patterns, while differences become more pronounced in medium- and low-density regions that contain more complex and diverse patterns. Compared with CDC, our method maintains comparable or slightly better performance in high-density regions, while consistently achieving gains in medium- and low-density regions. For instance, on STL-10,, our method improves accuracy by 1.0% in the medium-density region and 2.4% in the low-density region. These results indicate that our approach performs better on complex, low-density patterns, effectively mitigating the tendency of clustering models to overfit redundant high-density patterns and enabling better learning of diverse and informative samples. Moreover, Fig. 3(a) illustrates the training dynamics ImageNet- Doggg.. CDC rapidly fits simple high-density patterns in the early stage but soon stagnates, suggesting overfitting to redundant patterns. In contrast, our method, though slower converge initially, continues to improve steadily in later stages and ultimately surpasses CDC by a clear margin. This training behavior demonstrates that our approach not only avoids early overreliance on simple patterns but also sustains the discovery of complex patterns in medium- and low-density regions, leading to more generalizable representations.
Table 4: Comparison of clustering accuracy (%) across densities.
CIFAR-I0 10
STL-10
Method
High
Medium
Low
High
Medium
Low
Random
96.3
88.4
85.6
90.1
84.1
70.1
CDC
97.5
90.3
87.1
95.4
89.2
79.1
Ours
97.7
90.9
88.4
95.6
90.2
81.5
Effect of Sample Pruning Threshold e. Table 5 further investigates the effect of the sample pruning threshold on clustering performance (i.e., ACC). The pruning ratio is directly proportional to the threshold c. For instance, on CIFAR-10, the ratio rises from 17.4% at e 0.1 nearly 49.9% at = 0.5 and a similar trend can be observed on 100.110. We find that moderate pruning (e 0.1 0.3) improves ACC. This suggests that removing well-learned samples enables the model to focus more complex and informative examples, thereby enhancing performance. In contrast, aggressive pruning (E = 0.4 ~ 0.5) causes performance degradation, indicating that excessive sample removal discards valuable training samples. These results demonstrate the importance of selecting an appropriate threshold c to balance performance gains with training efficiency (more details can be seen in Appendix C.
Parameter Sensitivity Analysis. Fig. 3(b) shows the impact of the density weighting coefficient a on clustering performance. When a 0 (i.e., density weighting). performance on CIFAR-10
8

---

Published as a conference paper at ICLR 2026
Comparioo of Durs v CBC Imageneidogs
ACC Performance vS.
k Sensibivity Analysis
Our CBC
Accuracy (ACC) 16
STL
46.4
98.
56.1
95-
565
96.5
54.5
61
72-
03.3
22.4
ACC
94
13.5
93.5
Aa
61.0
93-
43
92
20
40
60
80
Epach
(b)
(c)
Figure 3: (a) Training curves the ImageNet-Dogs dataset. (b) Sensitivity of clustering perfor- mance to the density weighting coefficient a on CIFAR- 10, CIFAR-20, and STL-10. (c) Sensitivity
of clustering performance to the number of nearest neighbors & on CIFAR-10, CIFAR-20 20, and STL- 10.
Table 5: Effecc of sample pruning threshold e on clustering performance.
Datasel
Pruned
ACC
Pruned
ACC
Prused
ACC
Pruned
ACC.
Pruned
ACC
Pruned
ACC
CIFAR-10
0%
94.2
17.4%
94.5
26.5%
94.5
35.1%
94.3
43.7%
94,0
49.9%
93.9
STL-10
0%
93.0
14.2%
93.
27.0%
93.3
35.8%
93.0
58.1%
92.8
68.7%
92.4
and STL-10 is relatively low. Increasing a to a moderate range (a = 2) steadily improves clustering, achieving near-optimal results in most cases. Specifically, on CIFAR-10, ACCriser from 94.2% to 94.5%, while on STL-10, the peak ACC of 93.7% occurs at a = 4, though further in- creases in a cause minor fluctuations. These observations suggest that moderate density weighting effectively captures diverse feature structures, whereas excessively large weights provide little ad- ditional benefit. Overall, setting a between 1 and 3 offers robust and effective choice. Next, we analyze the sensitivity of our method to the choice of in density estimation, where sample density measured by the average distance to each sample's 's k nearest neighbors. Varying from 10 to 50, Fig. 3(c) shows that clustering performance remains highly stable, with maximum differences below 0.1% across all i values. This confirms that our density estimation is robust to and does not require fine-tuning of this hyperparameter (more details can be seen in Appendix C.2).
Our Pruning Strategy is Better. Previous experiments demonstrated that our pruning strategy achieves better performance with higher training efficiency. To further validate its effectiveness, we compare DSS with several alternatives, including no pruning. loss-based pruning (where the second- order difference- random pruning at different ratios 10% and 30%). As shown in Fig. 4(a), DSS consistently achieves the best results on CI, STL-10, and the averaged scores. In contrast, loss-based pruning leads to a significant drop in performance (92.5% and 92.9%). while random pruning maintains performance at low ratio (10%) but degrades notably as the pruning ratio increases. These results confirm that DSS is a superior pruning strategy.
feature consistency loss is replaced with the standard training loss), and
Stability Analysis of Feature Consistency. As mentioned earlier, our method evaluates the stability of sample learning states using the second-order difference of feature consistency (A² A lower S indicates smoother feature evolution and more stable learning process. Fig. 4(b) illustrates the overall trend of S during training. At the early stage, the model's learning state is unstable, with A values remaining high. As training progresses, the model gradually learns and adapts more samples, leading to steady decrease in S. Notably, CDC maintains consistently higher A values and continues to fluctuate in later training stages, suggesting that it overfits simple feature patterns while failing to capture more complex ones. In contrast, our method converges to a more stable state, effectively alleviating this issue. Further results in Fig. 4(c) confirm the superiority of approach, showing consistent improvements across CIFAR-10, CIFAR-22 -20, and STL-10.
Discussion on Initialization Strategies. K-Means is one of the most common strategies in cluster- ing, and it works well on most natural image datasets. For many natural datasets, such as CIFAR-10 and STL-10, using K-Means directly can give reasonable and reliable initial centers for later cluster- ing steps. However, the success of K-Means depends on its basic assumptions: the clusters should be roughly spherical, convex, and balanced. When a dataset does not satisfy these assumptions, K-Means may not provide good starting point.
9

---

Published as a conference paper at ICLR 2026
Comparison Different Pruning Stratagles
Compariion Durs vo CDC
Stalilty Seature conestency
Ous CDC
CDC
5.405
0.315
0.28
30
100
Epoch
CIAM CIRAA-20
STL-10
Agg.
(a)
(b)
(c)
Figure 4: (a) Comparison of different pruning strategies. (b) Comparison of mean 5 on CIFAR- 10, CIFAR-20, and STL-10. (c) Comparison of mean S values on CIFAR datasets between our method and CDC.
COIL-20 is non-natural image dataset with complex manifold structure. Its feature distribution not spherical or convex. In this case, K-Means initialization cannot capture the true structure of the data and may lead to suboptimal clustering results. Therefore, explored more suitable initialization method for this type of data. Based on the distribution of COIL--20, we use spectral clustering as the initialization strategy. Spectral clustering builds similarity graph and performs clustering in a low-dimensional space, which makes it more suitable for datasets with complex geometric shapes. After obtaining the spectral embedding, we apply our density center correction to the final K-Means step, so the initial centers better match the true data structure.
The results in Table 6 show that spectral clustering performs much better than K-Means on COIL- 20, and adding our module on top of it gives further improvement. Moreover, the results further show that choosing an appropriate initialization strategy for different data distributions important. They also support the generality of our approach: as long as the initialization fits the data well, our density-aware correction module can further improve the clustering quality.
Table 6: Performance of different initialization strategies on COIL-20.
Method
ACC
NMI
ARI
SCAN
92.6
95.2
90.3
SCAN + M-Means
82.7
92.1
77.3
SCAN + Spectral Clustering
99.2
98.9
98.4
SCAN + Spectral Clustering Ours
99.3
99.0
98.6
CONCLUSION
In this paper, we presented a novel plug-in designed mitigate overfitting to simple and redun- dant feature patterns, which adaptively adjusts training based on sample density and learning states. Analysis of pre-trained features shows that high-density regions contain many redundant samples, while low-density regions have fewer, more diverse samples. To address this issue, we incorpo- rated neighborhood density information into clustering head initialization, effectively reducing bias from redundant high-density samples and enhancing representations of complex low-density sam- ples. Additionally, we introduced a dynamic sample selection strategy, defining a stability measure based prediction consistency between weakly and strongly augmented views to prioritize unsta- ble or under-learned samples. Extensive experiments across multiple datasets and baselines demon- strate that our method consistently outperforms existing approaches, achieving smaller fluctuations in prediction consistency and significantly improved clustering accuracy, especially on samples with complex patterns where conventional methods often struggle.
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grants U24A20322, 62576094 and 62422118, in part by the Hong Kong UGC under grants UGC/FDSI I/E03/24. UGC/FDS1//E03/7 under Grant I 12219324. This research work was also supported by the Big Data Computing Center of Southeast University.
and in part by the Hong Kong Research Grants Council
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We confirm that our work on deep clustering adheres to the ICLR Code of Ethics. The research utilizes publicly available datasets containing no personally identifiable information. As strictly technical contribution focused on algorithmic improvement, we foresee no ethical issues or potential for misuse.
REPRODUCIBILITY STATEMENT
The complete code for implementing data preprocessing. model training, and evaluation is provided in the Supplementary Material to facilitate the reproduction of our results. Also, our experimental setup is described detail in the paper. The basic experimental setup and dataset-specific hyperpa- rameters for reproducing the results in Sec. 4.2 detailed in Appendix B.
REFERENCES
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds, 2020.
Vladimir Braverman, Vincent Cohen-Addad, Shaofeng H.-C. Jiang, Robert Krauthgamer, Chris Schwiegelshohn, Mads Bech Toftrup, and Xuan Wu. The power of uniform sampling for coresets. In Annual Symposium Foundations of Computer Science, pp. 462-473, 2022.
Shaotian Cai, Liping Qiu, Xiaojun Chen, Qin Zhang, and Longteng Chen. Semantic-enhanced image clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6869- 6878, 2023.
Haw-Shiuan Chang, Erik G. Learned-Milles and Andrew McCallum. Active bias: Training more accurate neural networks by emphasizing high variance samples. In Advances in Neural Informa- tion Processing Systems, pp. 1002-1012, 2017a.
Jianlong Chang, Lingfeng Wang. Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adap- tive image clustering. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 5880-5888, 2017b.
Ting Chen, Simon Korbllith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations, In Proceedings of the International Conference on Machine Learning, pp. 1597-1607, 2020a.
Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. CoRR, abs/2003.04297, 2020b.
Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 215-223, 2011.
Zhiyuan Dang, Cheng Deng, Xu Yang, Kun Wei, and Heng Huang. Nearest neighbor matching for deep clustering. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 13693-13702, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In Proceedings of the International Conference on Machine Learning, pp. 1183-1192, 2017.
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Mare Proesmans, and Luc Van Gool. SCAN: learning to classify images without labels. In Proceedings of the European Confer- ence on Computer Vision, pp. 268-285, 2020.

---

Published as a conference paper at ICLR 2026
Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your latent A new approach to self-supervised learning. In Advances in Neural Information Processing Systems, 2020.
Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1753-1759, 2017.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9726-9735, 2020.
Peihao Huang, Yan Huang, Wei Wang. and Liang Wang. Deep embedding network for clustering. In Proceedings of the International Conference Pattern Recognition, pp. 1532-1537, 2014.
Zhizhong Huang, Jie Chen, Jumping Zhang, and Hongming Shan. Learning representation for clus- tering via prototype scattering and positive sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7509-755 7524, 2023.
Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classification, 2:193-218, 1985.
Yuheng Jia, Jianhong Cheng, Hui Liu, and Junhui Hou. Towards calibrated deep clustering network. In The International Conference on Learning Representations, 2025.
Krishnateja Killamsetty, Durga S, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. In Proceedings of the International Conference on Machine Learning, pp. 5474, 2021a.
KrishnaTeja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, Abir De, and Rishabh K. Iyer. GRAD-MATCH: gradient matching based data subset selection for efficient deep model training. In Proceedings of the International Conference on Machine Learning, pp. 5464 5474, 2021b.
KrishnaTeja Killamsetty, Durga Sivasubramanian. Ganesh Ramakrishnan, and Rishabh K. Iyer. GLISTER: generalization based data subset selection for efficient and robust learning. In Pro- ceedings of the AAAI conference artificial intelligence, pp. 81 8110- 81 18, 2021c.
Alex Krizhevsky.
Learning multiple layers of features from 2009.
tiny images.
https://wwwecsstoronttocc edul
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Hanyang Li, Yuheng Jia, Hui LIU, and Junhui Hou. You can trust your clustering model: A parameter-free self-boosting plug-in for deep clustering. In Advances in Neural Information Pro- cessing Systems, 2025a.
Tao Li and Chris H. Q. Ding. The relationships among various nonnegative matrix factorization methods for clustering. In Proceedings of the International Conference on Data Mining, pp. 362-371, 2006.
Yunfan Li, Peng Hu, Jerry Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8547-8555, 2021.
Yunfan Li, Mouxing Yang, Dezhong Peng. Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. International Journal of Computer Vision, (3009):2205-222)) 2022.
Yunfan Li, Peng Hu, Dezhong Peng, Jiancheng Lv, Jianping Fan, and Xi Peng. Image clustering with external guidance. Proceedings of the International Conference on Machine Learning, 2024.
12

---

Published as a conference paper at ICLR 2026
Zhixin Li and Yuheng Jia. Conmix: Contrastive mixup representation level for long-tailed deep clustering. In International Conference on Learning Representations, 2025.
Zhixin Li, Yuheng Jia, Hui Li, and Junhui Hou. Learning from sample stability for deep clustering. In Proceedings of the International Conference on Machine Learning, 2025b.
Sören Mindermann, Jan Markus Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal. Prioritized training on points that are learnable, worth learning, and not yet learnt. In Proceedings of the International Conference on Machine Learning, pp. 15630-15649, 2022.
Truong Thao Nguyen, Balazs Gerofi, Edgar Josafat Martinez-Noricga, François Trahay, and Mo- hamed Wahib. KAKURENBO; adaptively hiding samples in deep neural network training. In Advances in Neural Information Processing Systems, 2023.
Chuang Niu, Hongming Shan, and Ge Wang. SPICE; semantic pseudo-labeling for image clustering. JEEE Transactions on Image Processing, 31:7264-7278, 2022.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. In Advances in Neural Information Processing Systems, pp. 20596-20607, 2021.
Qi Qian. Stable cluster discrimination for deep clustering. In Proceedings of the IEEE/CVE Inter- national Conference on Computer Vision, pp. 16599-16608, 2023.
Liping Qiu, Qin Zhang, Xiaojun Chen, and Shaotian Cai. Multi-level cross-modal alignment for image clustering. In Proceedings of the AAAI conference on artificial intelligence, pp. 14695- 14703, 2024.
Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, Xiaorong Pu, Philip S. Yu, and Lifang He. Deep clustering: A comprehensive survey. IEEE Transactions on Neural Networks and Learning Systems, 36(4))5858-587) 2025.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018.
Yuming Shen, Ziyi Shen, Menghan Wang, Jie Qin, Philip H. S. Torr, and Ling Shao, You never cluster alone. In Advances in Neural Information Processing Systems, pp. 27734-27746, 2021.
Alexander Strehl and Joydeep Ghosh. Cluster ensembles A knowledge reuse framework for combining multiple partitions. Journal of machine learning reseurch, 3:583-617, 2002.
Yaling Tao, Kentaro Takagi, and Kouta Nakata. Clustering-friendly representation learning via instance discrimination and feature decorrelation. In International Conference on Learning Rep- resentations, 2021.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey Gordon. An empirical study example forgetting during deep neural network learning. In International Conference Learning Representations, 2019.
Jianlong Wu, Zihan Li, Wei Sun, Jianhua Yin, Liqiang Nie, and Zhouchen Lin. Clusmatch: Improv- ing deep clustering by unified positive and negative pseudo-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-cfficient deep learning. In International Conference on Learning Representations, 2023.
Junyuan Xie, Ross B. Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering anal- ysis. In Maria-Florina Balcan and Kilian Q Weinberger (eds.), Proceedings of the International Conference on Machine Learning, pp. 478-487, 2016.
Chunlin Yu. Ye Shi, and Jingya Wang. Contextually affinitive neighborhood refinery for deep clus- tering. In Advances in Neural Information Processing Systems, 2023.
13

---

Published as a conference paper at ICLR 2026
Suqin Yuan, Runqi Lin, Lei Feng, Bo Han, and Tongliang Liu. Instance-dependent early stopping. In International Conference on Learning Representations, 2025.
Xu Zhang, Yuheng Jia, Mofei Song, and Ran Wang. Similarity and dissimilarity guided co- association matrix construction for ensemble clustering. IEEE Transactions on Knowledge and Data Engineering, 37(11):694-6777, 2025.
Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang. Minghua Deng, Liqiang Nie, Zhouchen Lin, and Xian-Sheng Hua. Graph contrastive clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9204-9213, 2021.
Sheng Zhou, Hongjia Xu, Zhuonan Zheng, Jiawei Chen, Zhao Li, Jiajun Bu, Jia Wu, Xin Wang, Wenwu Zhu, and Martin Ester. A comprehensive survey on deep clustering: Taxonomy, chal- -919:8,, 2025.
lenges, and future directions. ACM Computing Surveys,
14

---

Published as a conference paper at ICLR 2026
Appendix
A STATEMENT ON THE USE OF LARGE LANGUAGE MODELS (LLMs)
The authors used the LLM solety to assist with grammar, spelling, and sentence clarity. The authors reviewed and bear full responsibility for all content generated by the LLM. The LLM contributed in no other way to the paper. The research idea, experiment design, and all other content were developed and completed by the authors.
B MORE EXPERIMENT SETTINGS DETAILS
Datasets. Following (Gansbeke et al., 2020), we construct the CIFAR-20 dataset using 20
Table 7: Summary of datasets.
superclasses of the CIFAR-100 dataset, and
Dataset
Samples
# Classes
Image Size
build the ImageNet-10. ImageNet-Dogs, and
CIFAR-I0
60,000
10
32x32x3
Tiny-ImageNet datasets from 10, 15, and 200 classes ImageNet- respectively. For Tiny- ImageNet, perform the complete training and testing process on the training set, and adopt the merged datasets for both training and Tiny-ImageNet 100,000
CIFAR-20
60,000
20
32x32x3
STL-10
13,000
10
96x96x3
ImageNet-10
13,000
10
224x224x3
ImageNet-Dogs
19,500
15
224x224x3
200
64x64x3
testing for other datasets. Specifically, we extend the STL-10 dataset with 100,000 relevant unla-
beled samples during pre-training with MoCo-v2, which are removed afterwards.
Backbones. We adopt ResNet-34 as the backbone in both our method and all baselines to ensure a fair comparison. To better accommodate feature extraction on small datasets such as CIFAR-10 and CIFAR-20, we replace the first convolutional filter (7x7, padding 3, stride 2) with 3x3 filter (padding 2, stride 1), and remove the first max-poolle max--oooling layer following (Huang et al., 2023; Jia al., 2025). Besides backbone network, we attach clustering head following the original design of each baseline method to encode the learned representation to cluster assignments.
Experiment Settings. For representation learning, strictly follow the experimental settings from CDC (Jia et al., 2025) to pre-train the backbone with MoCo-v2. For data augmentation protocols, we strong and standard augmentation from SCAN (Gansbeke et al., 2020). We adopt the adaptive moment estimation (adam) optimizer, and use learning rate from settings from CDC (Jia et al., 2025). We set the number of training epochs to 100 epochs for all datasets. Meanwhile, all experiments that integrated our method, we set = 2.0 and A: = 10 across all datasets. We set = 0.01 on STL-10 and ImageNet-10 when applying our method to CC (Li al., 2021), while 0.1 for all other cases. All experiments are conducted on an NVIDIA RTX 3090 GPU.
Baselines. To ensure fair comparison, we re-implement SCAN (Gansbeke et al., 2020), CC (Li et al., 2021), TCL (Li et al., 2022) and CDC (Jia et al., 2025) using the same backbone pre- praitrained with MoCo-v2 following the experimental settings of CDC, and we follow the original design of the clustering heads from each baseline method. We directly copy the results reported by other deep clustering methods, including BYOL (Grill et al., 2020), NNM (Dang et al., 2021), I), GCC (Zhong et al., 2021), IDFD (Tao et al., 2021), TCC (Shen et al., 2021), SPICE (Niu et al., 2022), Pro- Pos (Huang et al., 2023), SeCu (Qian, 2023) and CoNR (Yu et al., 2023).
C MORE EXPERIMENT
C.1 EFFECT OF PRUNING THRESHOLD
Table 8: Effect of pruning threshold e on pruned samples and clustering performance (ACC, NMI, ARI).
C- 0.L
C- 0.2
C-03
<-
C- 0,5
Phuned
NMI
ARI
Pruned
CIFAR-10
17.4%
88.4
26.59
94.5
88.5
85.1
87.5
14.2%
86.3
93.3
35.8%
15

---

Published as a conference paper at ICLR 2026
C.2
SENSITIVITY ANALYSIS
Table 9: Sensitivity analysis of density weighting on CIFAR- 10, CIFAR-20, and STL-10.
Datasct
a 0
a
a
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
CIFAR-10
94.2
88.1
88.1
94.4
88.2
88.4
94.5
88.4
88.6
CIFAR-22
61.9
60.9
46.1
46.6
62.5
46.6
STL-10
93.0
85.8
85.6
93.3
86.3
86.0
93.5
86.6
86.4
3
4
Dataset
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
CIFAR-10
94.5
88.4
88.6
88.3
88.5
94.3
88.1
88.2
CIFAR-20
62.4
61.4
46.5
62.4
61.4
46.5
62.5
61.3
46.5
STL-10
86.4
86.2
93.7
86.8
93.4
86.4
86.2
Sensitivity Evaluation of Clustering Performance to Density Weighting. As shown Table 9, further investigate the sensitivity of clustering performance to the density weighting coefficient a. When a = 0 (i.e., without density weighting), the clustering performance across all three datasets relatively lower. As increases to moderate range (i.e.,
2), the clustering performance improves steadily, reaching an optimum in most cases. For instance, on CIFAR-I0, ACC increases from 94.2% 94.5% and NMI from 88.1% % to 88.4%. On STL-10, the peak ACC of 93.7% is achieved at a = 4. but the clustering performance suffers from minor fluctuations with a further increase in a. This indicates that moderate density weighting effectively captures diverse feature structures, while excessively large weights yield no additional gains. In summary, value of between and represents a robust and effective setting at most cases.
Sensitivity of Density Estimation to the Choice of k. Our method estimates sample density using average distance to nearest neighbors. To assess the sensitivity to k; we vary from 10 to 50 and evaluate the clustering performance on CIFAR-10, CIFAR-20, and STL-10. As shown in Fig. 5, the clustering performance across all metrics (i.e., ACC, NMI, and ARI) remains highly stable. For example, CIFAR- 10, ACC fluctuates only between 94.4% and 94.5%, while variations in NMI and ARI are within 0.2%. This stability is consistent across CIFAR-22 and STL-10, with maximum performance differences below 0.2% across all values. These results demonstrate that density estimation robust to the choice of k indicating that the method's performance does not depend on fine-tuning this hyperparameter.
CIFAR-10
CIFAR-20
STL-10
96
MNI
64
Score (96)
ARI
Score (96)
Score (%))
02
92
ACC
ACC
5:
88
86
84
60
Figure 5: Sensitivity of clustering performance to the number of nearest neighbors on CIFAR-10. CIFAR-20, and STL-10.
16

---

Published as a conference paper at ICLR 2026
C.3 ACCURACY OF OVERALL SAMPLES AND REMOVED SAMPLES
We evaluated CuC+Ours on CIFAR-10 and STL-10, comparing the per-epoch accuracy of removed samples with the accuracy over all samples, shown in Fig. 6. The results show that removed samples consistently have higher accuracy at each stage, indicating that the model usually makes correct predictions when marking samples as "stable".
Taining Accuracy Remove Remove d-mmpll Accuracy Over Epochs
Tainling Accuracy vs. Removed-sample Accuracy Over Epochs
Accuracy
Accuraccy
Overall Accuracy
Overall Accuracy
Removed Samples Accuracy
Remeved Samples Accuracy
Epoch
Epoch
(a)
(b)
Figure 6: (a) Training accuracy and removed-sampl accuracy curves CIFAR- 10 over epochs.
(b) Training accuracy and removed-sample accuracy curves on STL-10 over epochs.
C.4 VISUALIZATION OF SEVERAL STABLE (REMOVED) AND UNSTABLE (KEPT) SAMPLES
Figure 7: Visualization of some unstable samples from STL-10.
Figure 8: Visualization of some stable samples from STL-10.
C.5 ADDITIONAL RESULTS UNDER MULTIPLE RANDOM SEEDS
To further evaluate the stability of our method, we conducted experiments with five different random seeds on CIFAR-10, 10, 20, and STL-10. The results reported in Table show that our method consistently improves clustering performance across all metries (ACC, NMI. ARI) and across all datasets. Compared with the baseline methods (SCAN and CDC), the performance gains of our approach remain stable under different random initializations, with low standard deviations.
17

---

Published as a conference paper at ICLR 2026
Table 10: Clustering performance ACC, NMI, ARI (mean %) on CIFAR-IO, CIFAR-20, and STL-10 over five runs with different random seeds.
CIFAR-I0
CIFAR
STL.10
Method
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
SCAN
88.6+11199
83.2+0.33
79.4+1.33
51.0=1.0
53.9-099
37.1+0.77
83.5+00.6
82.7++0.8
SCAN+Ours
91.8:0.5
85,0+0.6
83.7:110
55.3=044
56.7=0.5
40.5=007
92.5++011
85.0+-.33
84.6++0.3
CDC
94.1100.3
87.910.4
87.8:10.6
61.7=033
61.1=02
46.0:0.2
93.010.1
85.940.11
85.610.11
CDC+Ours
94.6110.2
88.5..0.2
88.7.104
62.4-0.3
61.4-0.1
4..31
93.5.10.1
86.7.10.2
86.6.10.3
C.6 EARLY STOPPING STRATEGY
We further conducted experiments showing that sample stability assessment indeed serve as a practical hcuristic for carly stopping. Empirically, as training progresses, the number of samples that the model has "mastered" and therefore temporarily removes keeps increasing. This growth gradually slows down and eventually becomes stable. Based on this observation, we designed a simple but effective stopping rule: track the number of removed samples each epoch and record the maximum removed count. When this maximum remains unchanged for the most recent K epochs, we stop training.
We evaluated our proposed stopping strategy in combination with CDC on CIFAR- 10 and STL-10, testing several values of K(10, 20, 30, 40). We report the stopping time, the performance at the stopping point, and the best achievable performance. As shown in Table 11, for K = 10, 20, the performance at stopping is within % of the best performance. For K 30, 40, the stopped model achieves performance almost identical to the optimal result. This shows that our stopping strategy can find a point that is very close to the best result without training for all epochs.
Table 11: Performance of the sample stability based early stopping strategy on CIFAR-10 and STL-
CIFAR-10
STL-10
Stop Epoch
ACC
NMI
ART
Stop Epoch
ACC
NMI
ARI
Epoch=100
94.4
88.3
88.5
93.4
86.5
86.2
Best
94.6
88.5
88.7
93.5
86.6
86.7
K=10
49
94.0
87.2
87.4
59
93.0
85.8
85.5
K=20
69
94.3
87.9
88.2
77
93.1
85.9
85.7
K=30
69
94.3
87.9
88.2
98
93.4
86.6
86.4
K=40
98
93.4
86.6
86.4
C.7 EXPERIMENTS ON NON-IMAGE DATA
To further evaluate the generality of method, conduct additional experiments three image datasets: CNAE-9, Semeion, and News20. These datasets cover diverse modalities, including high-dimensional sparse TF-IDF vectors and binarized digit patterns. We apply our module on top of SCAN and report the results in Table 12. Across all datasets, our method consistently improves over the SCAN baseline, indicating the effectiveness of our approach. These results show that our module is not tied to any specific data type and can generalize well to non non-image modalities.
Table 12: Generality of our method across different data modalities.
CNAE-9
Semeion ACC
News20
ACC
NMI
ARI
NMI
ARI
AСC
NMI
ARI
SCAN
55,9
43.6
34.8
54.0
46.8
33.4
60.6
59.6
46.5
SCAN+Ours
63.9
51.7
43.3
55.8
52.3
40.8
62.5
60.2
47.5
18

---

Published as a conference paper at ICLR 2026
D ALGORITHM
We present the pseudo-code of our proposed method in Algorithm 1.
Algorithm 1: The proposed algorithm
1: Input: Unlabeled training data D. = {z; : E (1,
2: Output: Encoder fe (.), clustering head g(), and clusters.
3: Load the pre-trained parameters of fo(.).
4: Extract features using fo (-) and perform K-Means.
5: Compute density weight by Eq. (1) and re-estimate density-weightee prototypes using Eq. (2).
6: Initialize clustering head parameters with the proposed density-weightt prototypes.
7: Initialize empty of stable samples D, and an updated set of retained samples D; Du..
8: for epoch= 2, ... do
9: for each sample 24 Du, do Forward pass get P. and P. and compute S; using Eq. (3); Check second-order difference using Eq. (5) and check pseudo-label consistency; if prediction consistency is stable and pseudo-labels are consistent then
10:
12:
13:
Add x; to Dai 14: end if
15: end for
16: Update model parameters using samples in D, Du Dzi
17:
end for
19


Reviews:
[{"review_id": "5p3GbIbsbi", "reviewer": "Reviewer_WmJa", "summary": "The paper proposes improvements to the self-labeling stage of deep clustering. First, it introduces a density-aware clustering head initialization that downweights redundant high-density samples and upweights rare/low-density samples when forming prototypes. Second, it proposes a dynamic sample selection strategy that prunes stable samples based on pseudo-label stability and consistency between weakly and strongly augmented views, allowing training to focus on unstable or underlearned samples. Overall, this leads to improved clustering results across several standard image clustering benchmarks and results in better training efficiency, since fewer samples are actively optimized in later epochs.", "questions": "- How are models selected and tuned? What are the standard deviations across runs, e.g. for 5 to 10 seeds. How stable is the method overall?\n- You run for 100 epochs, but given that you explicitly track stability and prune stable samples, could those same signals (e.g., the fraction of samples no longer changing) be used as a stopping heuristic?\n- What happens if you continue training far beyond 100 epochs (e.g., 500+)? Do you eventually prune almost all samples, or does the model start overfitting the small subset of rare or ambiguous samples that never get pruned?\n- At the moment a sample is pruned as stable, how often is it actually assigned the correct ground-truth class? Showing that would address the concern that you might be confidently pruning wrong assignments and locking in errors. That could, however, have the benefit of reducing error propagation by no longer learning on those errors.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper targets an important stage used in state of the art deep clustering algorithms: self-labeling. The motivation is clearly laid out and depicts a practical issue: overfitting to dense/easy samples while underfitting rare samples.\n- The method improves both clustering accuracy and efficiency across several standard datasets and multiple baselines, suggesting it is broadly useful. The authors also provide a hyperparameter study on (\nα,\nk, \nϵ) and show that the approach is generally robust to \nα and k.", "weaknesses": "- Training is always stopped after a fixed 100 epochs. Unsupervised stopping criteria remain an open problem in the self-labeling stage. The proposed framework (especially with dynamic pruning) could, in principle, offer a stopping signal, but this is not analyzed.\n- The evaluation does not clearly report model selection details or variance across seeds. Because self-labeling inherently involves noise and instability (due to training on pseudo-labels), averaged results and standard deviations are important to properly interpret the reported gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E2R5mAZqQz", "reviewer": "Reviewer_37QQ", "summary": "The paper introduces a sample selection approach for deep clustering that addresses the problem that most clustering models treat all samples equally, even though samples in high-density regions are often redundant and simple, while those in low-density regions are complex and informative.\nBy relying more on samples in sparse regions (based on kNN distances), the tested models learn more effectively. The authors propose two main components:\n\n1. Density-Aware Clustering Head Initialization (DACHI) – adjusts each sample’s weight when computing cluster prototypes, so that clusters are not dominated by redundant high-density samples.\n\n2. Dynamic Sample Selection (DSS) – uses prediction consistency and pseudo-label stability to identify well-learned samples and temporarily remove them from training, letting the model focus on under-learned samples. \n\nThe method is plug-and-play and can be integrated into existing deep clustering algorithms.", "questions": "Q1) How does your method perform on data with consistent density, e.g. COIL20?\n\nQ2) How does your method perform on tabular data?\n\nQ3)  How is your sampling strategy related to Active Learning strategies? What could we learn from there, what are similarities?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "S1) Sampling strategies have not been researched enough on in the deep clustering area. The author's method can be combined with a lot of different deep clustering methods, potentially advancing the field broadly. \n\nS2) The methods are quite simple and intuitive. \n\nS3) By discarding some samples in each epoch the methods are even accelerated.", "weaknesses": "W1) Experimental evaluation could be in more depth: how does the improvement of results depend on \n\na) the model complexity\n\nb) the number / sizes of datasets\n\nc) the number of concepts per class/cluster. \n\nI'm missing some synthetic experiments here. Also, it would be interesting what happens for datasets with consistent densities throughout the datasets, e.g. COIL 20. \nFurthermore, I would be interested in how well the methods works on, e.g., tabular data, as the paper focuses on image data.\n\nW2) Sampling strategies are a major object of research in Active Learning. Setting the method into this context would improve the paper. \n\nW3) How much runtime do the computations for the sampling need? Is the runtime acceleration dependent on properties of the data that can be predicted? \n\nW4) Using k-Means as initlal clustering should be discussed more. It is not clear to me how the value of k for the initial k-Means clustering was chosen. What happens if the data does not follow typical assumptions fitting k-Means, e.g., for video data?\n\nEspecially the first three pages have quite some redundant text that could be shortened in order to tackle the above weaknesses or answer the questions below. \n\nMinor stuff: \n- Table 3 appears way before it is referred to in the text which hinders the reading flow.\n- Type in line 90", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t1yyFxdUKH", "reviewer": "Reviewer_8obk", "summary": "To address the problem that existing deep clustering methods treat all samples equally, resulting in excessive focus on simple redundant features in high-density areas and neglect of complex and diverse features in low-density areas, the paper proposes a plug-in module, which includes a density-aware clustering head initialization strategy (adaptively adjusting the contribution to the clustering prototype according to the local density of the sample to reduce bias in high-density areas) and a dynamic sample selection strategy (evaluating the sample learning status based on feature consistency and pseudo-label stability, and prioritizing resources to samples that have not been fully learned); this module can be seamlessly integrated into a variety of deep clustering architectures. Experiments on benchmark datasets show that clustering accuracy improves, and the performance gain in medium- and low-density areas (complex features) is more significant.", "questions": "1. One key technique is initializing the clustering head, but this is highly dependent on the model's pre-training. How to deal with non-pre-trained models?\n2. While this method improves overall training efficiency, the dynamic sample selection process requires additional computation and tracking: generating weak/strong augmentation views for each sample, calculating prediction consistency (cosine similarity), and tracking changes in second-order differences and pseudo-labels over nearly three epochs. How can this additional overhead be quantified?\n3. There are fluctuations in consistency in the early stages of training. How to judge the credibility of the consistency constraints?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method proposed in the article is implemented as a plug-in module. It can be seamlessly integrated into a variety of mainstream deep clustering architectures without requiring significant modifications to the original model's core structure. The integration process only requires replacing the initialization step (density-aware clustering head initialization) and the embedded training loop (dynamic sample selection), thereby significantly reducing the application cost in existing systems and providing strong adaptability.\n2. Based on the local density of samples (k-nearest neighbor distance), the contribution weight of samples to the cluster prototype is adaptively adjusted to reduce the dominance of high-density samples on the prototype, retain the clustering structure of medium and low-density areas (complex and diverse features), and avoid bias in the initialization stage.", "weaknesses": "1. Density-aware clustering head initialization relies on features extracted by a pre-trained encoder. The document uses MoCo-v2 to pre-train ResNet-34 but does not discuss the impact of the pre-trained encoder's quality on the method. If the feature representations extracted by the pre-trained encoder are of poor quality (e.g., low discrimination for complex samples), subsequent density calculations and K-Means initial clustering based on these features will be biased, thereby affecting the accuracy of cluster prototype initialization.\n2. In the early stages of training, the model has not yet fully learned, and the accuracy of pseudo-labels is low. At this time, screening samples based on pseudo-label stability may mistakenly judge \"complex samples that really need to be learned\" as \"unstable and need to be retained\", resulting in unreasonable allocation of early training resources.\n3. The experiment only verifies the effectiveness of the method on the image dataset, and does not involve other modal data such as text, speech, and time series. The adaptability of the method to non-image modalities has not been verified, and the applicable scenarios are limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lteDlNXdj5", "reviewer": "Reviewer_K4YF", "summary": "The paper identifies a key problem in deep clustering: existing methods treat all samples equally, causing them to overfit to simple, redundant feature patterns found in high-density regions of the feature space. To solve this, the authors propose a two-part plug-in designed to be integrated into existing deep clustering models:\n* Density-Aware Clustering Head Initialization (DACHI): This strategy addresses the initialization bias. Instead of a standard prototype (which is just the average feature of all samples in a cluster and is thus dominated by high-density samples ), it computes a density-weighted prototype.\n* Dynamic Sample Selection (DSS): This strategy adaptively manages training resources. It identifies and temporarily removes \"sufficiently learned\" samples from training batches. This allows the model to reallocate its capacity toward more \"unstable\" or under-learned samples.", "questions": "1. Related to weakness 1, could you provide a detailed analysis of the total wall-clock time per epoch, including the significant computational and memory overhead introduced by DSS? Specifically, what is the cost of calculating and storing prediction consistency histories for all $N$ samples at every epoch? The DACHI requires a k-nearest neighbor search within each initial cluster. How does this initialization step scale with very large datasets and high-dimensional features?\n2. Related to weakness 2. Is there a more adaptive or heuristic-based method to set this threshold?\n3. Have you investigated the interplay between your two modules? e.g. does the improved DACHI initialization lead to a larger or smaller set of samples being pruned by DSS later in training?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1. The dynamic sample selection (DSS) strategy provides a novel and sensible method for curriculum learning in a fully unsupervised setting.\n2. The proposed method successfully improves both clustering accuracy and training efficiency.\n3. The method is evaluated as a plug-in for four different deep clustering baselines, demonstrating its general applicability. The ablation studies validate the contribution of both DACHI and DSS.", "weaknesses": "1. The paper claims a training speedup of 1.3, but this only accounts for the reduced batch size during the model update step. It fails to discuss the significant overhead of the selection mechanism itself.\n2. The method introduces new and sensitive hyperparameters. The DSS strategy's pruning threshold, $\\epsilon$, is particularly problematic. As shown in Table 5, performance is highly dependent on this value: moderate pruning ($\\epsilon=0.1-0.3$) helps, but aggressive pruning ($\\epsilon=0.5$) degrades performance. The paper provides no clear heuristic for setting this critical value, which was manually set to 1e-1 or 1e-2 depending on the dataset.\n3. The 6.1% gain on CC is large, the improvement on the most recent and powerful baseline, CDC, is a more modest 1.1% average.\n4. The paper repeatedly uses the term \"overfitting\"  to describe the model's bias toward high-density samples. This is a slightly imprecise use of the term.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)