Towards Calibrated Deep Clustering Network
Authors: Yuheng Jia
Jianhong Cheng
Hui LIU
Junhui Hou
Venue: ICLR 2025 Poster (2025)
Keywords: Deep clustering
unsupervised calibration
clustering
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose the first deep clustering model with well calibration.

Abstract: Deep clustering has exhibited remarkable performance; however, the over confidence problem, i.e., the estimated confidence for a sample belonging to a particular cluster greatly exceeds its actual prediction accuracy, has been over looked in prior research. To tackle this critical issue, we pioneer the development  of a calibrated deep clustering framework. Specifically, we propose a novel dual
head (calibration head and clustering head) deep clustering model that can effectively calibrate the estimated confidence and the actual accuracy. The calibration head adjusts the overconfident predictions of the clustering head, generating prediction confidence that matches the model learning status. Then, the clustering head dynamically selects reliable high-confidence samples estimated by the calibration head for pseudo-label self-training. Additionally, we introduce an effective network initialization strategy that enhances both training speed and network robustness. The effectiveness of the proposed calibration approach and initialization strategy are both endorsed with solid theoretical guarantees. Extensive experiments demonstrate the proposed calibrated deep clustering model not only  surpasses the state-of-the-art deep clustering methods by 5× on average in terms of expected calibration error,  but also significantly outperforms them in terms of clustering accuracy. The code is available at https://github.com/ChengJianH/CDC.

Published as a conference paper at ICLR 2025
TOWARDS CALIBRATED DEEP CLUSTERING NETWORK
Yuheng Ji JJianhong Cheng', Hui Liu", Junhui Hou"
'Scool of Computer Science and Engineering, Southeast University Yam Pak Charitable Foundation School of Computing and Information Sciences, Saint Francis University
DDpartment of Computer Science, City University of Hong Kong Ly, cheng seu. edu. cn, h2liuus .ett.edd..iii hk, jh. www.hoitttytooe edu. .kk
ABSTRACT
Deep clustering has exhibited remarkable performance; however, the over- confidence problem, i.c., the estimated confidence for sample belonging to particular cluster greatly exceeds its actual prediction accuracy, has been over- looked in prior research. To tackle this critical issue, we pioneer the development of a calibrated deep clustering framework. Specifically, we propose a novel dual- head (calibration head and clustering head) deep clustering model that can effec- tively calibrate the estimated confidence and the actual accuracy. The calibration head adjusts the overconfident predictions of the clustering head, generating pre- diction confidence that matches the model learning status. Then, the clustering head dynamically selects reliable high-confidence samples estimated by the cali- bration head for pseudo-label self-training. Additionally, we introduce an effec- tive network initialization strategy that enhances both training speed and network robustness. The effectiveness of the proposed calibration approach and initial- ization strategy are both endorsed with solid theoretical guarantees. Extensive experiments demonstrate the proposed calibrated deep clustering model not only surpasses the sfate-of-the-an deep clustering methods by 5x on average terms of expected calibration error, but also significantly outperforms them in terms of clustering accuracy. Code is available at https://github.com www.o//hng..annilll.i.
INTRODUCTION
Clustering aims to categorize input samples into distinct groups, where samples within the same group exhibit greater similarity than those in other groups, without utilizing any ground-trute super- visory information. Recently, deep searning-based clustering methods, a.k.a. deep clustering, have showcased remarkable clustering performance owing to the powerful feature representation capa- bility of deep neural networks (Li et al., 2021; Qian, 2023). However, training a deep clustering network is challenging as no supervisory information is available. Existing methods solve the train- ing dilemma by introducing some side tasks in the training process. For example, DEC (Xie et al., 2016) introduces the auto-encoder during the training, and (Chhabra et al., 2022) adopts a generative adversarial network.
The pseudo- hatioo-llablin--ueeeee methods first extract features by self-supervised learning (like MoCo (Chen al., 2020b)), then add clustering head produce the clustering prediction, and further use the model's own outputs as pseudo labels to guide the unsupervised clustering network train- ing, have become mainstream in deep clustering ( (Van Gansbeke et al., 2020; Niu et al., 2022; Qian, 2023). Although those methods have achieved remarkable clustering performance, they have two drawbacks. First, they generally use a fixed threshold select the pseudo labels, which ignores the learning status of the model, i.e., in the early training stages, a higher threshold will select fewer and mass-imbalanced training samples, which depress the convergence speed, while in the later training
"Corresponding author.

---

Published as a conference paper at ICLR 2025
Oukyuds
Gulg
Depask
Daraa
(a) Supervised
(b) SCAN
(c) SPICE
(d) LS
CDC
(f) Training Process
Figure I: Reliability diagrams of different methods on CIFAR-20. In the ideal case, the confidence of model's output should be roughly equal to its accuracy, which means the confidence the out- put is well-calibrated. However, the previous deep clustering models faced severe overconfidence problem, i.e. the estimated confidence largely exceeds its real accuracy. We propose a calibrated deep clustering (CDC) model that enhances both confidence calibration and clustering performance.
stages, the fixed threshold is more likely to introduce noisy pseudo-labels due to the increasing pre- dicted confidence. More critically, those methods all implicitly assume that the output confidence of the model can reflect the true probability of a sample belonging to a certain cluster. However, those methods all face a serious overconfidence problem, i.e., the predicted confidence of the model's output will largely exceed the actual prediction accuracy. As shown in Fig. 1, the overconfidence problem of the state-of-the-art (SOTA) deep clustering methods like SCAN (Van Gansbeke et al., 2020) and SPICE (Niu et al., 2022) is even worse than the supervised learning model, as they rely on the pseudo-label labels to train the network, and the inevitable noise in the pseudo-labed aggravates the overeonfidence problem. Accurate confidence estimation is important for machine learning model, especially in building trustworthy decision decision-making systems like medical diagnosis (Mimori et al., 2021), autonomous vehicles (Feng al., 2019), and financial trading (Zhao et al., 2020), suggesting when we can trust the prediction of a learning model. However, confidence calibration has not been studied in deep clustering networks, and common calibration methods like Temperature Scaling (Guo et al., 201 2017a) rely on a labeled validation set, which is not accessible in deep clustering. More- over, as shown in Fig. 1-(d), the regularization-bant based calibration methods like Label Smoothing (LS) (Müller et al., 2019) will over-penalize the high-reliable samples, making the prediction of reliable and unreliable samples indistinguishable. Second, the current methods initialize the clustering head randomly, which is unstable and will destroy the feature learned by self-supervised learning, i.e., as shown Fig. 1-(f), the clustering accuracy (ACC) of MoCo equipped with K-means is 50.7%, while its performance will degrade to 10.4% after random initialization.
Our Contributions. calibrated deep clustering network (CDC). Our method centers a dual-head network structure featuring a clustering head and a calibration head. The calibration head introduces a novel calibra- tion technique that refines the prediction confidences from the clustering head across all samples, thereby providing precise confidence estimations. Simultaneously, the clustering head leverages the calibrated confidences from the calibration head monitor class-specific learning progress and strategically select high-confidence samples for effective pseudo-labeling, This symbiotic relation- ship between the two heads enhances the overall performance by exchanging critical information. Additionally, we introduce a potent initialization technique for both heads. As illustrated in Fig. 1()), our method exhibits exceptional clustering performance post the proposed initialization strategy. More importantly, the efficacy of our calibration approach and initialization strategy is underpinned by theoretical demonstrations.
Pioneering a novel approach, we delve into the development of a meticulously
Extensive evaluations on six benchmark datasets showcase the superiority of our method over state- of-the-art deep clustering methods by nearly 5x on average in terms of expected calibration error (ECE) and substantial advancements in clustering accuracy.
2 RELATED WORK
Deep Clustering. Deep clustering methods can be broadly classified into two categories: (1) Clus- tering based on representation learning which first learns a representation of the data and then applies a clustering technique like K-means (Huang et al., 2014; Qi et al., 2024) and Spectral Clus- tering (Jia et al., 1021; Peng et al., 2021) to obtain the final clustering, (2) Iterative deep clustering with self- supervison which aims to learn the data representation and performs clustering simulta- neously under the supervision of self self-training (Xie et al., 2016; Guo et al., 2017b), self-labeling (Van Gansbeke et al., 2020; Niu et al., 2022; Qian, 2023) and contrastive information (Li et al.,
2

---

Published as a conference paper at ICLR 2025
2021: Shen et al., 2021). Unlike previous methods that primarily rely data- features, recent methods such as SIC (Cai et al., 2023) and TAC (Li et al., 2024) use the vision-language model CLIP to mine clusters based on neighborhood consistency pseudo labels from the tex- tual space, which facilitates image clustering. Overall, self-labeling methods have demonstrated superior performance by selecting high-confidence instances through thresholding soft assignment probabilities and updating the entire network by minimizing the cross-entropy loss of the selected instances. However, they typically employ a fixed threshold for pseudo-label assignment, neglecting the model's learning status. How to determine suitable threshold is still a challenging task.
Confidence Calibration. Accurate confidence estimation is crucial for real- applications like self-driving cars (Feng et al., 2019) and disease diagnosis (Mimori et al., 2021), where prediction systems must not only be accurate but also indicate their likelihood of error, enhancing system in- terpretability. However, modern neural networks usually face the overconfidence problem, i.e., the estimated confidence by a neural network usually exceeds its real prediction accuracy (Minderer et al., 2021). To address this, calibration methods in supervised learning are broadly divided into two categories: (1) Post-calibration methods calibrate the model's output after model training. An example is Temperature Scaling (Guo et al., 20)7 which adjusts the softmax using labeled val- idation set. (2) Regularization based methods penalize the confidence of the model's predictions during the training process via some regularization techniques like Label Smoothing (Müller al., 2019), which uses soft labels to reduce overfitting, and Focal Loss (Mukhoti et al., 2020), which emphasizes difficult samples and penalizes overconfident predictions. The L1 Norm (Joo & Chung, 2020) is also used as an additional regularization to manage confidence levels. Recent studies have also investigated calibration under unsupervised domain adaptation. For instance, (Wang et al., 2020) developed a transferable calibration framework that reduces errors by accurately assessing discrepancies between source and target domains, while (Wang et al., 2023a) employs differen- tiable density ratio estimator to manage uncertainties under domain shifts. These studies tackle calibration in the domain adaptation setting, where labeled source data is available.
As demonstrated in Fig. 1(b) and (c), deep clustering methods exhibit a more pronounced overcon- fidence issue compared to supervised learning models. This heightened problem stems from state- of-the-art deep clustering techniques relying on pseudo-labeling, where incorrect pseudo-labels ex- acerbate the overconfidence challenge. Regrettably, past research has largely overlooked this issue. Furthermore, conventional calibration techniques unsuitable for directly addressing overconfi- dence in unsupervised clustering because post-calibration methods necessitate a labeled validation set for tuning hyperparameters, while regularization techniques can compromise the quality of lected pseudo-labels and, consequently, impair clustering performance. Consequently, this paper explores the development of calibrated deep clustering neural network simultaneously enhance clustering performance and achieve precise confidence estimation.
3 PROPOSED METHOD
Overview. As shown in Fig. 2, our model consists of dual-head network comprising a clustering head and a calibration head. Initially, we pre-train the backbone network using MoCo-v2 (Chen al.. 2020b) and implement feature prototype-based initialization strategy to bolster the initial discriminative capabilities of both heads (Sec. 3.3). Subsequently, we proceed to train the clustering head by leveraging high-confidence pseudo-labels through a cross-entropy loss, where the calibra- tion head provides the confidence levels for the samples (Sec. 3.2). Concurrently, the calibration head undergoes training with a novel calibration approach to gauge and estimate sample confidence from the clustering head (Sec. 3.1). Both heads are optimized simultaneously to facilitate mutual enhancement. (Sec. 3.4)
Notation. Denote by D. {z;; i E 21, training set of N unlabeled samples. Weak and strong augmentation strategies from (Van Gansbeke et al., 2020) are used to obtain W(x;) and A(x;) for each sample z;, Let f(O; æx;) be the feature extractor, g(Oatui -) be the clustering head, g( (( :) be the calibration head, a (.) be the softmax function, and C' be the predetermined classes.
3.1 CALIBRATION HEAD
In each training batch, we select B samples from D,,. In the remainder of this paper, all the operators are batch-wise without special mention.
3

---

Published as a conference paper at ICLR 2025
propagation
Data flow
Outpat view
progagation
Sample fitter
Losa fumetion
Weighta trainable module
strongly- augmented
Filter
FeaModel
ClaHead
g00...
A
Unlobaled Samples
q
peeuuo label
pi
صا
augmentedd
ehare
C.st
pi
libe
Mep
Ae RM
R tvc
R..m
FeaModel 1
CalHead (R )
iL
argmaz
wonkly- augmented
Wa,
p
Copy
Copy
Initialized MLP
FeaModel
W R
W
Figure 2: Illustration of the proposed CDC framework. The calibration head (CalHead) penalizes the overconfident predictions from the clustering head (CluHead). The clustering head, in turn, uses the calibrated confidence provided by the calibration head to select tigh-confidence samples for training. Note that the calibration head has the same structure as the clustering head.
The calibration head serves the purpose of aligning the output confidence the model with its true accuracy. However, the well-known calibration methods not directly applicable here: (1) Post-calibration methods like Temperature Scaling requires a labeled validation set to tune its hyper- parameter, which not accessible in unsupervised clustering tasks. (2) Regularization-base ods like Label Smoothing (LS), Focal Loss (FL), and L1 Norm (L1) penalize the prediction confi- dence of all the samples, which are harmful for failure detection, i.e., misclassification errors cannot be detected by filtering out low-confidence predictions. As indicated in Fig. 1-(e), although LS promotes the ECE performance, it will also reduce the high-confidence predictions.
meth-
Therefore, we selectively penalize confidence for samples in unreliable regions while preserving confidence in reliable regions. To this end, first use the K-means algorithm to divide the em- beddings matrix (before the clustering head and calibration head) into K mini-clusters and record
the average prediction of each mini-cluster in the clustering head qN Ex.eq, P."
1Q where Q denotes the samples belonging to the k-th mini-cluster, IQu] is the total number samples in that mini-cluster, and the prediction of the clustering head is 1o (g f(@::((@@)) (x)))) Then,
take the average prediction of each mini-cluster by the clustering head as the target distribution of the samples in that mini-cluster of the calibration head, i.e..
Leal Σ
Σ m;E
qn log (p:")),
where the prediction of the calibration head is pfaal (g f(O; xx))). The above calibration method aligns the feature representation of the samples with their output in the clustering head.
Furthermore, we add negative entropy loss make the predicted class distribution more uniform, avoiding all samples being divided into the same cluster:
1 Cen fin=coa
E log p!g.
where poal denotes the j-th column of the calibration head for batch of samples. Combining Eqs. (1) and (2), the overall loss of the calibration head is:
C = Lost + Wen Len,
(3)
where hyperparameter Wen for simplicity.
Theoretical Analysis. Assuming K-means divides features into (a + b) regions, where a regions reliable regions that do not cross clustering decision boundaries, and the set of samples in the
4

---

Published as a conference paper at ICLR 2025
reliable regions are denoted as T, and regions unreliable regions that cross the clustering de- cision boundaries with samples in the unreliable regions denoted as F. We use expected calibration error (ECE) (Nacini et al., 2015) to measure the difference between the confidence of predictions and the actual accuracy, serving as an indicator of how well-calibrated the confidence estimates are. To calculate ECH, all N samples' confidence scores grouped into / bins and computed by N Jace (B:) - avg.com (B:) I. where aco(.) and avg.comf..0 means accuracy and average confidence in each bin.
ECE = Ei=1 |Bdl|
Theorem (Region-awart Penalty). Let Con and Con foat be the average confidence of the predictions before and after calibration, respectively. Then, confidence penalty occurs only in unre- folu while not in reliable regions with E fran T
liable regions with E [Con S E [Confolu
F
E [Confrlu. Proof. See Appendix A.3.
Theorem 2 (Improve Calibration). The calibration errors given by the clustering head and the calibration head are denoted as ECEdu and EC Under some mild conditions, we have ECEct I ECEds Proof. See Appendix A.4.
Theorem indicates that the proposed calibration method only penalizes the confidence of the unre- liable samples, which avoids the over-punishment problem on the high confidence prediction sam- ples by the previous regularization based calibration methods. Theorem 2 indicates the proposed calibration method can promote calibration theoretically.
Remark. Considering the characteristics of the calibration head outlined earlier, we implement two specific designs:
Stop gradient. Given the incorporation of uncertain samples within the calibration head, we fo- cus solely on optimizing the calibration head, steering clear of network-wide loss optimization.
Dual-head decoupling. To address conflicts between calibrated and overconfident predictions within the same head, we adopt a dual-head framework to effectively separate and manage these divergent predictions.
3.2 CLUSTERING HEAD
Due to the lack of ground-truth information, the most crucial issue in training the clustering head is generating highly reliable pseudo labels. Previous methods (Van Gansbeke et al., 2020; Niu al., 2022: Qian, 2023) usually adopt fixed and predefined threshold to select samples as the pseudo labels. However, such fashion overlooks the fact that the model's output confidence continuously increases during the training process. Specifically, in the early training strategy, the threshold is typically set too high, resulting the selection of only a limited number of samples and slowing down the training speed. As training progresses, the confidence of all samples increases, leading to the selection of more samples as pseudo labels. Consequently, more incorrect pscudo labels are included, reducing the clustering performance. Furthermore, as previously analyzed. deep clustering networks face a significant overconfidence problem, which hinders the model's ability to perceive its learning status, further complicating the determination of an appropriate threshold. Additionally, previous methods utilize a single global threshold for all classes, which biases the model towards selecting more samples from easy classes (those with higher confidence) while disregarding hard classes (those with lower confidence).
To address these challenges, we use the confidence of the calibration head estimates to determine the threshold dynamically for different classes. This approach allows a more accurate estimation of the model's learning status and helps mitigate the discrepancies in data utilization across categories.
Specifically, we first sort all samples in class by their probability values in descending order, and then select the top |B/C| samples with the highest probabilities to get TOP(c), where |-] denotes the lower approximation operator. Then, we determine the number of samples selected as the pseudo-labels for the e-th cluster via
((c)
Σ a.ETOP(e)
P
xrcoal J, Ve 1,2, C,
where p. pmr denotes the confidence of the weak augmentation of x; estimated by the calibration head. As the calibration head is well-calibrated, the average confidence of each cluster estimated
5

---

Published as a conference paper at ICLR 2025
by the calibration head can well capture the learning status of each cluster, i..., if M(c) is large (resp. small), the model is more (resp. less) confident about the prediction for the samples in the e-th cluster; accordingly, we should select more (resp. less) samples from this cluster the pseudo- labels. Finally, we select the top Mc samples from each cluster to constitute the final pseudo-laheled samples: S = (( y:)), where yr argmax p. -. returns the cluster index producing the largest prediction as the pseudo label for x,.
After selecting the pseudo labels, we the cross-entropy loss to optimize the parameters of the feature extractor and the clustering head:
Cola
|S|
Σ 25
y:log p clu
(5)
where |S| denotes the total number of selected pseudo-labeled samples, p.-cku denotes the confi- dence of the strong augmentation of æ; estimated by the clustering head.
3.3 INITIALIZATION OF CLUSTERING AND CALIBRATION HEADS
After pre-training the backbone using MoCo-v2, a clustering head, typically a multilayer percep- tron (MLP), is added to generate clustering predictions. Figure 1-(f) illustrates that the MoCo-v2 pre-trained backbone yields a quite discriminative representations, for example, applying K-means to MoCo-v2 pre-trained features 50.7%. In contrast, a randomly initialized clustering head on this backbone only reaches an accuracy of 10.4%. This indicates that the discriminative power of the pre-trained backbone does automat- ically transfer to the clustering head. Consequently, the common practice of random initialization can result in low-quality pscudo-labels and poor clustering performance.
the CIFAR-20 dataset achieves clustering accuracy of about
Therefore, we propose a feature prototye--bsee initialization strategy for the clustering head (also the same approach for the calibration head) to extend the discriminative capabilities of the pre- trained backbone. As detailed in the lower right corner of Fig. 2, we employ a three-layer MLP for the clustering head. This MLP consists of D input units, H hidden units, and C output units. The variables include E ND (input), h E R-NHH (hidden), E R-NC'' (output), and two linear layer weights (11) E "xD and W22) E RC To initialize the first linear layer (1), we perform K-means clustering on the input variable z, and split into H mini-clusters. Then we use each clustering center (prototype) to initialize W11), i.e.
(1) = Kmeans (z),
(6)
where Kmeansh (z) returns H prototypes of by the K-means clustering. Note that we set the bias of the linear layer to O. By the above setting, one neuron of the output will denote a prototype, and this neuron will yield large values (resp. small values) for samples belonging to (resp. not belonging) the mini-cluster related to that prototype. marking it has the discriminative ability transferred from the pre-trained backbone. Besides, the MLP also embeds a BN layer and a ReLU layer. For the second layer weight (2),, we apply the same strategy by initializing with K-means on h, resulting in (2) Kmeanse (h). After the initialization with feature prototypes, we further orthogonalize the weights to improve the discriminative ability of the network.
Proposition 1. Assuming z" as the prototype obtained by applying K-means to the features z, when the linear layer weights W =z" the cluster assignment of the output W z aligns well with z".
Proof. See Appendix A.I
Proposition 1 demonstrates that the proposed strategy can effectively propagate the discriminability from the feature space to the output space, thereby outperforming random initialization.
3.4 JOINT TRAINING AND FINAL PREDICTION
Joint Optimization. After the pre-training by MoCo-v2, and the initialization of the clustering head and the calibration head, we optimize the clustering head and feature extractor by minimizing Eq.

---

Published as a conference paper at ICLR 2025
and optimize the calibration head by minimizing Eq. 3. The overall training process of model is summarized in Algorithm 1.
Final Prediction. The clustering head achieves clustering results similar to that of the calibration head but with a worse ECE. Therefore, we use the calibration head to make the final prediction of i-th sample 0; Le., p (g cat f (O:
Algorithm 1: The training process of Calibrated Deep Clustering
Data: Training dataset D. = {æv E 2, N,,,
Input: C' predefined clusters, K mini-clusters; feature extractor f(O:)), the clustering head g g(0etui :), the calibration head g g(ecat))) +), and softmax function ((.).
Initialization:
Learning feature extractor @ via MoCo-v2;
Initializing MLPs O. and Ocol the proposed initialization approach;
for epoch < to EPOCHS do for b - to [N/B] do Establish batch dataset D. C Dw:
p.uccal = grad.
(g (0eal; (@; (x)))));
Establish pscudo-labeled samples S by Eqs. 4;
24 <- grad.
(0;
pllu no grad. [g (O (Q:z;)));
Employ K-means on = to get partition Qx: Calculate mini-clusters target distribution qu: for sub_iter + 1 to B/B, do Establish sub-batch dataset Deub C Ds: pivd <- a (g (0au: f A(zrn)))) Calculate Letu by Eq. 5, update e and Octu: Prub < a (g (Oual; (@; a)):))));
Calculate Land and Cen. by Eqs. 1-2, update 0
end
end end
EXPERIMENTS
4.1 EXPERIMENT SETTINGS
Datasets and Backbones. We conducted experiments on six widely used benchmark datasets, CIFAR-10, 10, CIFAR-20 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011). ImageNet- 10, ImageNettDogs (Chang et al., 2017), and Tiny-ImageNet (Le & Yang, 2015). Similar to (Huang al., 2022) and (Niu et al., 2022), used ResNet-34 and MLP (512d-BN (loffe & Szegedy, 20L5 (Nair & Hinton, 2010)-C'd) for all experiments.
Implementation Details. The comparison methods can be divided into two types. (I) Clustering based on representation learning which can achieve clustering by using K-means (Lloyd, 1982): MoCo-v2 (Chen al., 2020b), SimSiam (Chen al., 2020a), BYL (Grill et et al., 2023), ProPos (Huang et al., 2022) and CoNR (Yu et al., 2024). (2) Iterative deep clustering with self-superriisio:: DivClust (Metaxas et al., 2023), CC (Li al., 2021), TCC (Shen al., 2021), TCL (Li et al., 2022), SeCu (Qian, 2023), SCAN (Van Gansbeke al., 2020), and SPICE (Niu et al., 2022). More details are shown in B.I.
DMICC (Li
For CDC, we trained the model for 100 epochs using the Adam optimizer with learning rate of 5e-5 for the encoder and Ic-4 for the MLP. The learning rate the encoder on CIFAR-20 and Tiny-ImageNet was adjusted to le-5 for better learning of noisy pseudo labels. We set B=1,000 for CIFAR-10, CIFAR-20, and STL-10, B=500 for ImageNet- 10 and ImageNet-Dogg B=5,000 for Tiny-ImageNet. We set K=500 for CIFAR- 10. K=40 for CIFAR-22 20 and ImageNet- K=150 for STL-10 and ImageNet- 10, and K=1,000 for Tiny-ImageNe For simplicity. the predictions of the clustering and calibration heads are denoted as CDC-Clu and CDC-Cal, respectively.

---

Published as a conference paper at ICLR 2025
Table 1: The clustering performance ACC, ARI (%) and calibration error ECE (%) of various deep clustering methods trained on six image benchmarks. The best and second-best results are high- lighted in bold and underlined. respectively. 1 (J) means the higher (resp. lower), the better.
CIFAR-ID 10
CIFAR-20
STI-10
ImageNet 10
ImageNet Dogs
Tiny ImageNet
Method
ACC1
ARIf
ECE
ACC1
ARI1
ECT
ACC1
ARI
ECE,
ACC1
ARI
ECT.
ACC1
ARI
ECE
ACC1
ARI
ECE
K-means
22,9
4.9
NEA
13.0
288
NBA
19.2
6.1
NIA
24.1
5.7
N/A
10.5
2.0
NIA
2,5
0.5
N/A
McCo-v2
82.9
64.9
NIA
50.7
26.2
A/A
68.8
45.5
N/A
56.7
30.9
N/A
62.8
48.1
NIA
25.2
11.0
N/A
Simsiam
70.7
53.1
N/A
33.0
16.2
NAA
49.4
34,9
NA
78.4
68.8
NA
44.2
27.3
N/A
19.0
8,4
N/A
BYOL
57.0
47.6
NIA
34.7
21.2
N/A
56.3
38.6
N/A
71.5
54.1
NA
58.2
44.2
N/A
11.2
4.6
N/A
DMICC
8288
69.0
N/A
46.8
29.1
N/A
80.0
62.5
N/A
96.2
91.6
N/A
58,7
43.8
N/A
ProPos
94,3
88.4
N/A
61.4
45.1
N/A
86.7
73.7
N/A
96.2
91.8
NA
77.5
67.5
N/A
29.4
17.9 N/A
CoNR
93.2
86.1
NIA
60.4
44.3
NAA
92.6
84.6
N/A
96.4
92.2
NAA
79,4
66.7
N/A
30.8
184
N/A
DivClust
81,9
CC
85.2 72.8
28.4
29.7
67.7
90.6
TOC
90.6
73.3
49.1
31.2
81.4
68.9
89.7
82.5
59.5
41.7
TCL
88.7
78.0
53.1
35.7
86.8
75.7
89.5
83.7
51.6 64.4
SeCu-Size
90.0
81.5
8.1
52.9
38.4
13.1
80,2
63.1
9.9
SeCu
92.6
85.4
4.9
52.7
39.7
41.8
83.6
69.3
6.5
SCAN.2
84.1 74.1
10.9
50.0
34.7
37.1
87.0
75.6
7.4
95.1
89.4
2.7
63.3
49.6
26.4
27.6
15.3
27.4
SCAN-3
90.3 80.8
6.7
51.2
35.6
39.0
91,4
82.5
6.6
97.0
93.6
1.5
72.2
58.7
19.5
25.8
13.4
48.8
SPICE-2
84.4
70.9
15.4
47.6
30.3
52.3
89.6
79.2
10.1
92.1
83.6
7.8
64.6
47.7
35.3
30.5
16.3
48.5
SPICE-3
91.5
83.4
7.8
58.4
42.2
40.6
93.0
85.5
6.3
95.9
91.2
67.5
52.6
32.5
29.1
14.7
N/A
CDC (Ours)
94,9
93.1
79.3
70.3
20.0
CDC-Cal (Ours)
94,9
89.5
1.1
61.7
46.6
93.0
85.6
0.9
94.1
79.2
70.0
7.7
33.9
19.9
11.0
Supervised
89.7
78.9
4.0
71.7
50.2
11.0
80.4
62.2
10.0
99.2
98.3
0.9
93.1
85.7
0.9
47.7
24.3
5.1
-MoCo-v2
94,1
87.5
2.4
83.2
68.4
6.7
90.5
80.7
3.5
99.9
99.8
0.4
99.5
99.0
0.9
53.8
30.9
8.4
Evaluation Metrics. We used three common clustering metries: clustering Accuracy (ACC) (Li & Ding, 2006), Normalized Mutual Information (NMI) (Strehl & Ghosh, 2002), and Adjusted Rand Index (ARI) (Hubert & Arabie, 1985) to evaluate the clustering performance, and used the expected calibration error (ECE) (Naeini et al., 2015) to evaluate the calibration performance. Moreover, applied the failure rejection ability metrics AUROC, AURC, and FPR95 (Zhu et al., 2022) to evaluate the separation quality of low-confidence and high-confidence predictions.
4.2 MAIN RESULTS
Superior Clustering Ability. shown in Tab. 1, applying K-means to the pre-trained features sig- nificantly outperforms directly applying K-means to the raw features, demonstrating the importance of deep representation learning in clustering. Along this line, ProPos and CoNR have continuously improved the clustering performance but cannot provide confidence prediction. For self-labeling- based methods, CDC-Cal outperforms DivClust, CC, SeCu, SCAN, and SPICE on almost all the datasets, validating the importance of dynamic threshold setting. On CIFAR-10 10 and STL-10, CDC- Cal with the same pre-trained model (MoCo-v2) increases accuracy by 0.8% and 2.5% than super- vised learning. On large-scale datasets, CDC-Cal has achieved improvements of 1.4%, 11.7%, and 4.8% on ImageNet-10, ImageNet- MengeNee-DDog, and Tiny-ImageNe Mcanwhile, the results of the clus- tering head CDC-Clu are close those of the calibration head CDC-Cal. In summary, our method ranks first in 11 out of 12 cases in all the clustering metrics, suggesting its superior clustering ability.
Excellent Calibration Performance. The calibration error of CDC-Cal is far superior that of CDC-Clu and all the compared methods. CC does not apply the pseudo-labelingg strategy, so its ECE relatively low. This explains that overfitting to the wrong pseudo-labels is a reason for the high calibration error. SPICE-2 uses dual-softmax losses to encourage sharper predictions, which leads to more severe overconfidence than SCAN, while our calibration head significantly reduces the cali- bration error CIFAR- -154%%+1.11)) CIFAR-21 compared to SPICE-2. Fig. 1 provides a visual comparison of the ECE on CIFAR-20, where our method achieves well-aligned predictions and better clustering performance simultaneously.
and STL-10 (7.8%~0.88)
Competitive Failure Rejection Ability. Fig. 3 compares different regularization bataalaiiiziio---beedeede calibration methods. CDC-Cal improves AUROC by 8.6%, decreases AURC by 10.6%, and decreases FPR95 by 6.5% compared to the second-ber best method, demonstrating its effective error rejection capability. The second line of Fig. further demonstrates method can better separate correct and mis-
8

---

Published as a conference paper at ICLR 2025
classified samples compared with the regularization-bare based calibration methods, which is beneficial for selecting reliable pseudo-labels for feedback into the network. Although Focal Loss (FL) may produce lower ECE than method, its clustering ACC is much worse. Besides, method outperforms FL. on all the failure rejection metrics (AUROC, AURC, and FPR95), suggesting our method can better separate the correct prediction and the wrong prediction.
69: 15.1
012
45
40.6
74
3..
72
20
20
20.4
SCAM
SCAN
CDC
SCAN 15
TL
SCA
CDC
SCAN
(a) ACC (%) 1
(b) AUROC (%) 1
(c) AURC (%)
(d) FPR95 (%)
(e) ECE (%)
wnrg
A137
WYTE
35
$7
Confience
Canbbrre
Cardanda
Cantarncs
Gerdaace
(1) SCAN
(g) LS
(h) FL
LI
i CDC-Cal(Ours)
Figure 3: The failure rejection ability comparison CIFAR-20. The second row shows the confi- dence distribution correct and misclassified samples, demonstrating that our method has stronger ability to separate failure predictions.
100
1//11 M.r
80
ACC (M)
80
Randon Inc
SCA22
Ransom
SCAM2
SCAN
SCAM
CDC
CDC
Epoch
Epoch
CIFAR-10
(b) ImageNct-Dogs
Figure 4: The training process on CIFAR-10 and IaageNet-Dogs, CDC-Cal has (i) fewer training stages, (ii) better initialization strategy, and (iii) more stable performance improvement.
4.3 ABLATION STUDY
I. Initialization. Fig, 4 shows the benefits of our initialization strategy in improving training speed and stability. Our method can transform the discriminative ability of the pre-trained features to the clustering head directly. Tab. 2-I quantitatively demonstrates that prototype-base basee initialization leads to a notable enhancement in clustering performance. Specifically, when we exclude the initial- ization stage from our method (w/o Init.+CDC), the clustering performance experiences significant decline, showing the efficacy of the initialization strategy.
IL Confidence-Aware Selection. Tab. 2-11 shows that if we fix the threshold at certain values, the clustering performance will drop significantly, suggesting the effectiveness of the proposed confidence-aware thresholding strategy.
III. Single-head Setting. Tab. 2-III shows single-head setting, the network employs its own overconfident predictions for confidence-awant sample selection, which may lead to the selection of numerous incorrect aseudo-labels, resulting in performance degradation including both the cluster- ing performance and the calibration performance.

---

Published as a conference paper at ICLR 2025
Table 2:
The ablation results of the proposed model.
CIFAR-10
CIFAR-20
STL-10
Type
Settings
ACC1
NMI
ARIT
ECE,
ACC1
NMI
ARIT
ECE,
ACCT
NMIT
ARI
ECE|
After Randomly Init.
19.1
7.6
3.1
8.5
10.4
5.7
1.0
4.9
19.2
6.0
2.4
8.5
1
After Proposed Init.
87.2
79.8
76.1
1.0
56.4
56.9
41.2
5.2
89.8
80.9
79.3
2.7
w/o Ini,,+DDC
89.4
86.5
83.5
3.3
44.4
52.3
31.0
11.9
73.3
70.0
60.6
17.9
Fixed Thre. (0.99)
80.6
69.8
65.8
6.5
54,9
55.9
37.1
15.1
89.6
81.0
79.3
1.0
Fixed Thre. (0.95)
91.9
85.2
83,9
3.5
50.8
49.2
30.5
12.6
91.8
84.0
83.3
1.1
Fixed Thre. (0.90)
92.7
86.5
85.3
43.3
43.2
27.3
4.1
93.0
86.1
85.7
1.0
Fixed Thre, (0.80)
93.6
87.5
86.9
49.9
50.5
33.6
3.9
93.0
86.1
85.7
CDC-Cal (Ours)
94.9
89.3
1.1
61.7
46.6
93.0
85.8
85.6
0.9
III
Single-head (Clu)
93.9
88.0
87.5
2.3
59.7
61.3
45.3
31.6
92.6
85.3
84.9
5.1
IV
Single-head (Clu+Cal)
94.8
89.0
89.1
1.8
57.8
58.7
43.1
21.2
93.0
85.7
85.5
3.0
V
Cal (w/o Stop Gradient)
93.0
86.0
85.7
2.0
49.6
52.1
34.2
12.4
86.7
76.3
63.9
2.5
IV. Dual-head Decoupling. Tab. 2-IV shows that combining the conflicting losses of encouraging overconfidence and penalizing confidence in a single head (Clu+Cal) will decrease both the calibra- tion performance and the clustering performance especially CIFAR-20.
V. Stop Gradient for the Calibrating Head. If we not stop the gradient for the calibration head, the clustering performance will degrade significantly, especially on CIFAR-20 and STL-10. This because the calibration loss relies on unreliable regions to provide penalties, which will include unreliable information in the feature extractor if do not stop the gradient.
VI. Mini-cluster Number K. The value of K represents the number of regions and directly influences the confidence penalty. The choice K depends on the complexity of the dataset. For challenging datasets with over-confident predic- tions, lower K value is needed to increase the confidence penalty, with the extreme case being K = 1, where all samples approach uniform distribution, resulting in the maximum penalty. For simpler datasets, higher K value is necessary to reduce the confidence penalty. If K equals the number of samples, each sample independent, and the penalty zero. Fig. 5 shows how this trend affects confidence. On CIFAR-20, variations of K by +20% lead to ACC changes of +1.3% and ECE changes of +0.5%, which is acceptable. More ablation studies are shown in the Appendix B.5.
(a) ACC varying K
(b) ECE varying
Figure 5: The ACC and ECE are bust to varying K.
CONCLUSION
In this paper, have introduced novel calibrated deep clustering model that outperforms SOTA deep clustering models in terms of both clustering performance and calibration performance. No- tably, our model is the first deep clustering method capable of calibrating the output confidence. In particular, the expected calibration error of our is 5 times better than certain compared meth- ods. achieve these results, we proposed dual-head clustering network consisting of clustering head and calibration head. The clustering head utilizes the calibrated confidence estimated by the calibration head to select highly reliable pseudo-labels. The calibration head, on the other hand, calibrates the output confidence of all samples using novel regularization loss. Additionally, introduced an effective initialization strategy for both the clustering head and the calibration head, significantly promoting training stability. Through our calibrated deep clustering neural network, the output confidence of the model is well aligned with its true accuracy. Moreover, as we do not need any label to train the network, the above benefit be gained quite easily.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grants U24A20322 and 624221 18, and in part by Hong Kong UGC under Grants UGC/FDSI 1/E02/22 and UGC/FDSI 1/E03/24. This work also supported by the Big Data Computing Center of Southeast University.
REFERENCES
Shaotian Cai, Liping Qiu, Xiaojun Chen, Qin Zhang, and Longteng Chen. Semantic-enhanne image clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 6869-6875 2023.
Jianlong Chang, Lingfeng Wang. Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adap- tive image clustering. In Proceedings the IEEE/CVF International Conference Computer Vision, pp. 5880-5888, 2017.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International Conference on Machine Learning. pp. 1597-1607, 2020a.
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020b.
Anshuman Chhabra, Ashwin Sekhari, and Prasant Mohapatra. On the robustness of deep clustering models: Adversarial attacks and defenses. Advances in Neural Information Processing Systems, 35:20566-20579, 2022.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In International Conference Artificial Intelligence and Statistics, pp. 215- 223, 2011.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Di Feng, Lars Rosenbaum, Claudius Glaeser, Fabian Timm, and Klaus Dietmayer. Can we trust you? on calibration of a probabilistic object detector for autonomous driving. arXiv preprint arXiv: /909. 12358, 2019.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint arXiv: 02677, 2017.
nann-Batteen Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a approach to self-supervised learning. Advances in Neural Information Processing Systems, 12:21271-21284 2020.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pp. 1321-1330, 2017a
Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In International Joint Conference on Artificial Intelligence, volume 17, pp. 1753-1759, 2017b.
Peihao Huang, Yan Huang, Wei Wang, and Liang Wang. Deep embedding network for clustering. In International Conference on Pattern Recognition, pp. 1532-1537, 2014.
Zhizhong Huang. Jie Chen, Jumping Zhang, and Hongming Shan. Learning representation for clus- tering via prototype scattering and positive sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7509-7555 2022.

---

Published as a conference paper at ICLR 2025
Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classification, 2:193-218, 1985.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pp. 448 456, 2015.
Yuheng Jia, Hui Liu, Junhui Hou, Sam Kwong, and Qingfu Zhang. Multi-view spectral clustering tailored tensor low-rank representation. IEEE Transactions Circuits and Systems for Video Technology, 31(12):47444797 2021.
Taejong Joo and Uijung Chung. Revisiting explicit regularization in neural networks for well- calibrated predictive uncertainty. arXiv preprint (rXiv:2006.06399, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Hongyu Li, Lefei Zhang, and Kehua Su. Dual mutual information constraints for discriminative clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp- 8571-8579. 2023.
Tao and Chris Ding. The relationships among various nonnegative matrix factorization methods for clustering. In International Conference on Data Mining, pp. 362-371, 2006.
Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive cluster- ing. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 8547-8555, 2021.
Yunfan Li, Mouxing Yang, Dezhong Peng, Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. International Journal of Computer Vision, 130(9):2205-222) 2022.
Yunfan Li, Peng Hu, Dezhong Peng, Jiancheng Lv, Jianping Fan, and Xi Peng. Image clustering with external guidance. In International Conference Machine Learning, pp. 27902,, 2024.
Stuart Lloyd. Least squares quantization in pem. IEEE Transactions on Information Theory, 28(2): 129-137, 1982.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In Interna- tional Conference on Learning Representations, 2016.
Ioannis Maniadis Metaxas, Georgios Tzimiropoulos, and Ioannis Patras. Divelust: Controlling di- versity in deep clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3418 3428, 2023.
Takahiro Mimori, Keiko Sasada, Hirotaka Matsui, and Issei Sato. Diagnostic uncertainty calibra- tion: Towards reliable machine predictions in medical domain. In International Conference on Artificial Intelligence and Statistics, pp. 3664-3672, 2021.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34:15682-15699 2021.
Jishnu Mukhoti,, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr and Puneet Doka- nia. Calibrating deep neural networks using focal loss. Advances in Neural Information Process- ing Systems, 13:15288-155999 2020.
Rafael Müller, Simon Korblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances in Neural Information Processing Systems, 37:4694-4703, 2019.
12

---

Published as a conference paper at ICLR 2025
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, pp. 2901-2907, 2015.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning, pp. 807-814, 2010.
Chuang Niu, Hongming Shan, and Ge Wang. Spice: Semantic pseudo- for image clustering. IEEE Transactions on Image Processing, 31:7264-7278, 2022.
Bo Peng, Jianjun Lei, Huazhu Fu, Yalong Jia, Zongqian Zhang, and Yi Li. Deep video action clustering via spatio-temporal feature learning. Neurocomputing, 456:519-527, 2021.
Jianhan Qi. Yuheng Jia, Hui Liu, and Junhui Hou. Superpixel graph contrastive clustering with semantic-invariant augmentations for hyperspectral images. IEEE Transactions on Circuits and 2024.
Systems for Video Technology. 34(11):11360-1133
Qi Qian. Stable cluster discrimination for deep clustering. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 16645-16654, 2023.
Yuming Shen, Ziyi Shen, Menghan Wang, Jie Qin, Philip Torr, and Ling Shao. You never cluster alone. Advances in Neural Information Processing Systems, 34:27734-27746, 2021.
Alexander Strehl and Joydeep Ghosh. Cluster ensembles-a knowledge reuse framework for com- bining multiple partitions. Journal of machine learning research, 3:583-617, 2002.
Hui Tang and Kui Jia. Towards discovering the effectiveness of moderately confident samples for supervssee learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14658-14667, 2022.
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Scan: Learning to classify images without labels. In European Conference on Com- puter Vision, pp. 2682268882 285. 2020.
Haoxu Wang, Anqi Liu, Zhiding Yu, Junchi Yan, Yisong Yue, and Anima Anandkumar. Learning calibrated uncertainties for domain shift: A distributionally robust learning approach. In Interna- tional Joint Conference Artificial Intelligence, pp. 1460-1469, 2023a.
Ximei Wang, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable calibration with lower bias and variance in domain adaptation. Advances in Neural Information Processing Sys- tems, 33: 13:19222-19223, 2020.
Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, et al. Freematch: Self-adaptive thresholding for semi-supervised learning. In International Conference on Learning Representations, 2023b.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning, pp. 478 487, 2016.
Yang You, Igor Gitman, and Boris Ginsburg. Large batch training of convolutional networks. arXiv preprint arv: 1000: 03888, 2017.
Chunlin Yu, Ye Shi, and Jingya Wang. Contextually affinitive neighborhood refinery for deep clus- tering. Advances in Neural Information Processing Systems, 044:5788-7990 2024.
Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang. Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo la- beling. Advances in Neural Information Processing Systems, 34: 18400-18419, 2021.
Linjun Zhang, Zhun Deng. Kenji Kawaguchi, and James Zou. When and how mixup improves calibration. In International Conference on Machine Learning, pp. 26135-26160, 2022.
Shengjia Zhao, Tengyu Ma, and Stefano Ermon. Individual calibration with randomized forecasting. In International Conference on Machine Learning, pp. 11387-11397, 2020.
Fei Zhu, Zhen Cheng, Xu- Yao Zhang, and Cheng-Lin Liu. Rethinking confidence calibration for failure prediction. In European Conference on Computer Vision, pp. 518-536, 2022.
13

---

Published as a conference paper at ICLR 2025
A THEORETICAL PROOF
A. PROOF OF PROPOSITION
Proposition 1 Assuming 2" as the prototype obtained by applying K-means to the features 2, when the linear layer weights W = =z", the cluster assignment of the output Wz aligns well with z".
proof.
Without loss of generality, we assume the features is normalized. When conducting K-means on feature z to derive prototypes z², we need to solve the following optimization problem.
minz- - = =11 ming 2z" || 2 + ||="||
min x m00 W=s'
mazw Wz
The above process demonstrates that the optimal clustering assignment in the feature space is also applicable in the output space, indicating consistency between the feature space and the output space.
A.2 INTUITIVE EXPLANATION
Recalling the setup in Sec. 3.1, we use K-means to divide the features into (a + b) regions, where regions are Reliable Regions that do not cross clustering decision boundaries and regions are Unreliable Regions that cross the clustering decision boundaries.
Reliable Region
Unreliable Region
-1/+1
-1/+1
0. 9998
0.02
0.60
0.40
0.97
0.03
0. 70
0.30
In the reliable region (the left half of Fig. 6). the mean predicted value for four samples within this region is [0.97, 0.03], which is consistent with the average maximum confidence of 0.97 for each sample. In the unreliable re- gion (the right half of Fig. 6), the mean value voted by four samples within this region is (0.45, 0.55], while the aver- age maximum confidence of these samples 0.70. This in- consistency will be the source of confidence penalty by our method.
0.97
0.03
0. 20
0.80
0.96
0.04
0.30
0.70
0.97
0.03
0.44
0.55
Avg. Conf.
0.97
0.03
0.30
0.70
Figure 6: Calibration Example
Therefore, by identifying regions with similar features in the feature space, our method can maintain predictions in reliable regions, and penalize predictions in unreliable regions. Accordingly, high confidence will be kept and better failure rejection ability will can be achieved.
A.3 PROOF OF THEOREM 1
Theorem 1 (Region-aware Penalty) The average confidence of the predictions before and after calibration are denoted Confalu and Con peal feal, Then, confidence penalty occurs only unreli- able regions with E foal
< I fol while not in reliable regions with E foal] T
F
F
E [Con
proof.
Considering a binary classification problem like (Zhang et al., 2022), let's assume that the fea- tures of the data (z,y) E R (-1,1) 1} are generated from mixture of two Gaussian distri- butions, where each class corresponds to one Gaussian distribution. Then, the following condi-
tional distribution holds: zly
Assuming the classifier is C (z) sgn (0"²). where θ = Ei EEM The confidence vector is denoted as [p+1 (z), p-1 (z))² with Pa (z) =
N
exp( =2/0==//0)++1 where e {-1, +1].
In reliable region, we have
E [[onnfcal] E 2ET
[peat (z)]
[pelu (z)]
[ps (z)] 11 fclu]
ZET
*ET
14

---

Published as a conference paper at ICLR 2025
In unreliable region, we have
1
E
[Confcal)=
=1
E #EF
(z)]
2 22)
(D (z))
+I
[ (z)]
2 2 #EF
(p_T (z))
F
2
#EF
#EF
1
E
cat
(2)
2
2
1
E
(p49) (z))
+
(p=9) (z))
REF
=EF,0"Z<0
=EF,0"=20 =20
+1 1
E
(x)
E
(p=1 (z))
+
E
p (z))
FE
=EF,6" z<U
3EP,0"=20
nal (2)
E
[0ST (z)
(v=1) (z) pff (z))
#EF
=EF <<0
+1 1
E
P+1
cal
(z)]
E =EF,0T=20
(p41) (z))
FEF
+1 1
B
1
(z)]
E =EF,0r=<0
(p (2))
FER
+1 I
E
ond
(2)
N 2
E
[P49 (z) - (Dff (z) p-d (=))
ZEF
=EF,0" =20
[z)] Σ 2)
I
E
p+1
(z)]
II
E
[p
E
(p (z))
#EF
#EF
=EF,OT=<0
E
p+1
cal
(z)]
2
+1
(z)] 2 1
1
E
(p_T
FE
ZEF
=EF,0=20
1
I
E
P+1
cal
(z)]
E
[V- (z)
P+1
cal
[z)]
(I)
2EF
2
=EF,0"Z<0
I
E
(z)
2
E
[ (z) p_y (z) (II)
#EF
2
=EP,OT=20
E
(v=4 (z)) +
E
(pfdl (2)) (I) - (II)
#c =FF,0 =<00
*E
+20
= =E F [Com pola] (I) (II)
As when 0T < 0, we have foo (z) p44 (z) = poly (2) - 2 (2) O.
While when 0T x 2 0, we have p4f (z) p-y (z) = p41 pz)
Hence, E fody E foal = (I) (II) 2 0 and we can conclude that our method can
F
penalty over-confidence predictions only in the uncertain regions.
A.4 PROOF OF THEOREM? 2
Theorem 2 (Improve Calibration) The calibration errors given by the clustering head and the calibration head are denoted as EC gelu and ECEaa Under some mild conditions, we have EC < EC
proof.
First, ECE is defined as:
ECE = E |ACC (v) Po (2)
u 0Tz
In reliable region, we have ECE#" EC
E
[ACC (va) - clu (z) LACOcall (vo) DCal (2)
0t T
As the two heads have the same clustering assignments, we have
15

---

Published as a conference paper at ICLR 2025
E T
ACCdd
E
[ACCo))
E Con
= =B = fonnfon
then
ECET ECET O.
In unreliable region, we have
ECEE" - EC v6 06=07 F
E
|ACCdu (vs) pelu (z) |AC (vs) - peal (z)
As the calibration head uses the predictions of the clustering head, resulting in similar clustering assignment outcomes for both, thus we have
ACC (vv) = ACC (vs) = ACC.
(1) If po (z) ACC, patl (z) ACC, the model faces overconfidence, we have
ECE
ECEP
E clu
Pe
(z) cal (=)]
sEF
(I)
(II) 20
(ii If pd (z) ACC, panl (z) ACC, and
E ppdt (z))
E
[pea (2)
BACC
ZEF
holds, we have
ECE,' ECE
[peku (z)] E ZEF
+
cal (z)] 2EACC 2 20,
ZEP
(iit) If pet (z) < ACC, peal (z) < ACC, the model faces underconfidence. We do not consider this situation as deep networks are more prone to overconfidence.
(iv) If te (z) ACC, pedl (z) ACC, due to pelu (z) 2 peal (z) always holds in Theorem 1, there is no solution.
In summary, we can get
ECE < ECEda
B EXPERIMENTAL DETAILS AND MORE RESULTS
B. IMPLEMENTATION DETAILS
Datasets. As shown in Tab. 3, we conducted experiments used benchmark datasets, CIFAR-10 (Krizhevsky et al., 2009), CIFAR-20 (Krizhevsky et al., 2009), STL-11 (Coates al., 2011), ImageNet-10 (Chang al., 2017), ImageNet-Dogs (Chang et al., STL-10
six widely
Table 3: A summary of the datasets
Dataset
#Samples #Classes Image Size
CIFAR-10
60,000
10
32x32
CIFAR-20
60,000
32x32
13,000
10
96x96
2017), and Tiny-ImageNet (Le & Yang. 2015). Similar to (Van Gansbeke et al.. ImageNet-Dogs
ImageNet-10
13,000
10
224x224
19,500
15
224x224
2020), we used 20 superclasses of the CIFAR-100 dataset to construct the CIFAR-20, and 10, 15, and 200 subclasses of the ImageNet-Ik (Deng et al., 2009) dataset to extract the ImageNet- 10, ImageNet-Dogg, and Tiny-ImageNet. Mcanwhile, we extended STL-10 by 100,000 relevant unlabeled data during the pretext training and removed them afterwards. We used the dataset from ImageNet- 10, ImageNet- Dogs, and ImageNe datasets for both training and testing, while the rest datasets were trained and tested the merged datasets.
Tiny-ImageNet
100,000
200
64x64
Backbones. To facilitate training on small datasets such as CIFAR- and CIFAR-20, we followed (Huang et al., 2022) and modified the first convolution layer's kernel of the ResNet-34 with kemel
16

---

Published as a conference paper at ICLR 2025
size of 3 x 3, padding of 2 and stride of 1, and removed the first max- max-pooling layer. Moreover, All experiments are conducted on an NVIDIA RTX 3090 GPU.
Representation Learning. For MoCo-v2, SimSiam and BYOL, according to (Van Gansbeke al., 2020), (Chen et al., 2020a), (Chen et al., 2020b) and (Grill et al., 2020), we apply the same augmen- tation strategy for all datasets including random ResizedCrop with image size reported in Tab. and a random HorizontalFlip, followed by a random ColorJitter and random Grayscale. Moreover, we use GaussianBlur except for CIFAR- 10 and CIFAR-20 due the size of image. We adopt the ResNet-34 as backbone and a 2-layer MLP head (hidden layer 4096-d, with BN and ReLU, output layer 256-d) as projector. Besides, BYOL use another similar 2-layer MLP head as predictor. For opitimizer, MoCo-v2 and SimSiam adopt SGD optimizer. while BYOL adopt LARS (You et al., 2017) optimizer. we train all these methods over 1,000 epochs with base learning rate 0.5 and batch size 256 on the datasets shown in Tab. 3. We update the learning rate with cosine decay learn- ing rate schedule (Loshchilov & Hutter, 2016) and start with a warmup (Goyal et al., 2017) for 50 epochs. For the momentum hyperparameter т, we set 0.99 and 0.996 for MoCo-v2 and BYOL. For other hyperparameters, we set the temperature and weight decay as 0.2 and 0.0001 for all methods, and queue size 32768 for MoCo-v2.
Comparison Method. (1) Clustering based on representation learning. The parameter settings for MoCo-v2, SimSiam, and BYOL have been outlined previously. For ProPos, due to the difference of backbone, we re-implemented the results on Tiny-ImageNet dataset using ResNet-34. For other datasets, since the backbone, data augmentation and dataset partition scheme the same as ours, we cite the results of (Huang et al., 2022). For DMICC (Li et al., 2023), and CoNR (Yu et al., 2024), we directly cited the results of the paper. (2) Iterative deep clustering with self-supervision. For methods requiring a pre-training stage (CC, SCAN, SPICE), we selected the MoCo-v2 model. During the formal training statge, under a consistent data augmentation and dataset partitioning scheme, we used ResNet-34 to re-implement CC, SCAN and SeCu. For CC, we used a smaller initial learning rate of 5e-5 to fine-tune the model, and the rest of the parameters were consistent with (Li et al., 2021). For SCAN and SeCu, we used the same hyperparameters as the (Van Gansbeke et al., 2020) and (Qian, 2023). For SPICE, except for SPICE-3 used WideResNet-28-2 for Cifar 10, WRN-28-8 for CIFAR-20, and WRN-37-2 for STL-10, the rest of the experiments used ResNet-34, and these results are from the official model library'. Since SPICE-3 has not released their model on the DivClust (Metaxas et al., 2023), TCC (Shen et al., 2021) and TCL (Li et al., 2022), we directly cited the results of the paper.
dataset, we directly cited the results and did not test the model's ECE. For
Supervised Baseline. We trained on the training datasets of CIFAR-10, 10, CIFAR-20, STL.10 and ImageNe,,,,,,, and evaluated the results the test or validation datasets. Due to lack of the designated test datasets on ImageNet-10 and ImageNet-Dogg datasets. We used Adam optimizer with weight decay of 0.0001 for all the experiments, and the batch size was set to 256. We adopted cross entropy on strong augmented samples to train baseline model for 500 epochs with an initial learning rate of 0.001, and fine-tuned the MoCo-v2 model 50 epochs with an initial learning rate of 0.0001 to obtain a better baseline.
we reported the results on the training
Sample Selection Strategy. For our method, the clustering head selects only high-confident sam- ples for training, and the calibration network utilizes all samples during the training process. The above strategy aims to obtain a more accurate estimation of the overall confidence and enhance the model's robustness.
Augmentation Strategy. For the clustering head of our method, we employed weakly augmented samples to generate pseudo-labels and used the strongly augmented samples as the distribution to be learned. For the calibration head, no augmentation is included to eliminate any representation differences caused by sample transformations.
B.2 RELIABILITY DIAGRAMS
The reliability diagrams for all models trained on CIFAR-20 are shown in Fig. 7. Our proposed method demonstrates lower calibration error when the overall average confidence and clustering accuracy are comparable.
17

---

Published as a conference paper at ICLR 2025
Outpum
Supervised
Supervised (MoCo.22)
(c)
(d) SCAN-2
SCAN-3
(f) SeCu-Size
ScCa
(h) SPICE-2
SPICE-3
CDC-Clu (Ours)
(k) CDC-Cal (Ours)
Figure 7: Reliability diagrams on CIFAR-20.
B.3 CLUSTERING PERFORMANCE: NMI
In addition to the ACC and ARI results shown in Sec. 4.2, our method also achieved 5 out of best results NMI (Tab. 4).
Table 4: The clustering performance NMI six image benchmarks. The best results are shown in bold, while the second best results are underlined.
Method
10
CIFAR-20
STL-10
ImageNet-10 10
ImageNet-Dogs
Tmag-em
K-means
8.7
8.4
12.5
11.9
5.5
6.5
MoCo-v2
77.7
56.4
69.3
54.1
62.8
43.0
Simsiam
67.7
32.6
61.8
81.7
45,9
36.4
BYOL
65.5
43.9
67.0
70.3
63.4
29.6
DMICC
74.0
45.2
68.9
91.7
58.1
ProPos
88.6
60.6
75.8
90.8
73.7
46.0
CoNR
86.7
60.4
91.1
74.4
46.2
DivClust
72.4
44.0
89.1
51.6
CC
76.9
47.1
72.7
88.5
65.4
32.5
TCC
79.0
47.9
73.2
84.8
55.4
TCL.
81.9
52.9
79.9
87.5
62.3
SeCu-Size
79.3
51.6
SeCu
86.1
55.1
73.3
SCAN-2
79.3
52.2
78.8
88.9
60.1
43.1
SCAN-3
82.5
51.2
83.6
92.5
67.4
40.7
SPICE-2
73.5
44.3
80.4
82.8
$6.9
44.9
SPICE-3
85.4
57.6
86.1
90.2
62.6
42.7
CDC-Clu (Ours)
89.3
60.6
86.0
93.1
76.7
47.2
CDC-Cal (Ours)
89.3
60.9
85.8
93.2
76.5
47.5
Supervised
78.8
59.8
67.3
97.7
86.6
59.3
+MoCo-v2
86,9
72.9
99.7
98.7
64.0
B.4 DIFFERENCE BETWEEN LABEL SMOOTHING (LS) AND CDC
LSS reduces confidence for all samples uniformly. while method adopts region-aware penalty that only reduces the confidence for unreliable regions while keeping the high confidence for reliable regions. As shown in Fig. 8, I.S lacks high-confidence samples (e.g., confidence 0.9) and shows
18

---

Published as a conference paper at ICLR 2025
overlapping peaks for correct and incorrect predictions with different hyper- ppeer--armmetrs.. This indi- cates that LS, regardless of the hyperparameter adjustments, tends to over-penalize high-confidence samples, leading to diminished model performance.
24 da
va
24
(a) LS=0.10
(b) LS=0.20
(c) LS=0.30
(d) LS=0.40
(c) LS=0.50
(1) CDC-Cal (Ours)
Figure 8: (d) Label Smoothing (LS) shows worse separation between correct and incor-
rect predictions, regardless of hyperparameter adjustments from 0.10 0.50. (f) CDC-Cal (Ours).
B.5 ABLATION STUDY
Batch size and Number of mini-clusters (B, K). As demonstrated in Tab. 5, our model exhibits considerable robustness to variations within reasonable range for B and especially challeng- ing datasets such as CIFAR-20. Variations of K by +20% and B by #50% only result in the changes in overall accuracy of +1.4%.
Insight for B. We constrain GPU memory usage to within 9GB. For datasets with smaller image size, larger batch sizes are utilized (c.g., B=1000 for CIFAR- 10 and CIFAR-20, which have image sizes of 32 x x2). Conversely, smaller batch sizes are employed for datasets with larger image sizes (e.g., B=500 for ImageNet-10 10 and ImageNet-Dogs, which have image sizes of 224x
Insight for K. K is tuned empirically according the complexity of the dataset. For challenging datasets with over-confident predictions, a lower K value is needed to increase the confidence penalty, with an empirical ratio of B B/K < Conversely, for simpler datasets, higher value is necessary to reduce the confidence penalty, supported by an empirical ratio of B K 2 10.
Table 5: Influence of (B, K) on clustering accuracy ACC (%) and expected calibration error ECE (%) across STL-10 and CIFAR-20 datasets. "Std." indicates the standard deviation of the data in this row.
#B
500
1000
1500
Dataset
Std.
#K
-20%
0%
+20%
-20%
0%
+20%
-20%
0%
+20%
ACC
93.0
93.1
93.2
93.1
93.0
93.1
92.8
92.8
93.1
0.1
STL-10
ECE
1.1
1.2
1.6
1.2
0.9
1.2
1.2
1.0
0.8
0.2
ACC
58.1
61.0
60.2
58.8
61.7
62.6
59.1
60.0
59.2
1.4
CIFAR-20
ECE
6.9
7.6
3.6
4.9
4.9
4.6
4.4
1.4
3.9
1.7
Influence of Hidden layer size and weight wen on clustering accuracy ACC (%) and expected calibration error ECE (%) across STL-10 and CIFAR-20 datasets.
Table 6:
Dataset
#Hidden
256
384
512
640
768
#wen
0.0
0.1
0.5
1.0
5.0
10.0
ACC1
93.2
93.1
93.0
92.6
92.2
ACCt
92.9
93.0
92.8
93.0
93.1
93.2
STL-10
ECEJ
1.7
1.1
0.9
0.6
1.0
ECE]
1.1
1.0
1.3
0.9
0.9
1.0
ACC1
61.7
60.6
61.7
58.3
57.1
ACC1
57.0
57.7
58.8
61.7
58.2
56.9
CIFAR-20
ECEJ
3.7
4.1
4.9
3.5
2.5
ECE]
2.4
2.7
1.8
4.9
2.7
1.7
Hidden Layer Size (H). The choice of 512-dim is commonly used in MLP, and provides good balance between initialization complexity and performance, as Tab. 6 shows empirically.
Loss Weight (Wen). As shown in Tab. 6, method is robust to wens simplicity the experiments. Moreover, removing the negative entropy loss Len will degrade the clustering performance, especially on CIFAR-20.
fix it at Wen 1 for
19

---

Published as a conference paper at ICLR 2025
Advantage of the Proposed Calibration Loss over the Other Regularization- Calibration Losses. We provide more detailed reliability diagrams on CIFAR-20 20, where the calibration head is applied with different regularization losses. Although these methods reduce calibration errors, their clustering performance does not significantly improve. This is related to the penalty imposed on high-confidence samples by these methods: over-penalizing high-confidence samples, which is evident in the reliability diagram in Fig. 9. Since samples with similar features are more likely have consistent outputs, our method imposes smaller penalty high-confidence samples than on low-confidence samples, which effectively maintains the high confidence predictions of reliable samples.
Expected Ourputs
Expacted Outpets
Expected
Expacted Outpets
Outputs
accured
Co65-54.55 5%
Cord
108-11000
Cand 5%
Confidence
Lample
Confidence
Canfidence
Confidence
(a)
CDC-Label Smoothing
(b) CDC-Foccl Loss
CDC-LI Norm
(d) CDC-Cal (Ours)
Figure 9: Reliability diagrams on CIFAR-20 with the calibration head applied different regulariza- tion losses. In the high-confidence region, all compared calibration losses face decline clustering accuracy.
B.6 TIME AND SPACE COMPLEXITY
Table 7: Running time and memory requirements of SCAN and CDC. All experiments run on one 3090 GPU and ResNet-34.
CIFAR-10
CIFAR-20
STL-10
ImageNet-10 10
NmtgeNettDDog
Tiny- Imag-Nna
Million Purameters of CDC
21.8
21.8
21.8
21.8
21.8
22.0
SCAN2 (100 eps)
Running Time
+SCAN3 (200 eps)
4.7
1.3
(Hour)
CDC (100 eps)
8.7 (+4.1)
2.3 (-2.3)
1.3 (40.1)
6.0 (+2.0)
5.8 (+0.3)
5.6 (-4.7)
SCAN2
5789 (bs=256)
5798 (bs=256)
9237 (bs=256)
13489 (bs= 256)
18907 (bs=256)
6480 (bs=256)
Memory
(MB)
SCAN3
5478 (bs=1000)
5479 (bs=1000)
7598 (bs=1000)
10764 (bs=256)
12608 (bs=256)
7414 (bs=1000)
CDC
5620 (bs=1000)
4849 (bs=1000)
5822 (ba= 1000)
8396 (hs=500)
8396 (bs=500)
7138 (bs=5000)
As shown in Tab. 7, our dual-head network does not significantly increase the number of parameters compared to the original ResNet-34 (21.80 million parameters). Regarding GPU memory, we use a combination of batch and sub-batch strategies (see Algorithm in 3.4), leading to larger batch sizes and lower memory usage compared to competing methods under the same optimizer.
For running time, condensed the two-stage process (totaling 300 epochs) used in SCAN into a single stage (only 100 epochs). We speed up K-Means with K-Means++ initialization and the PyTorch implementation as ProPos (Huang et al., 2022). Therefore, we save 2.3 hours on CIFAR- 20 and 4.7 hours on Tiny-ImageNet. Although training time increases on CIFAR-10 and ImageNet- 10 due to the large number of mini-clusters, the potential benefits in terms of enhanced failure rejection capability and improved clustering performance often outweigh these costs.
20

---

Published as a conference paper at ICLR 2025
C FURTHER DISCUSSION
C.I INSIGHTS FROM
LEARNING (SSL)
C.I.1 ENHANCING SELF-TRAININ WITH MODERATELY CONFIDENT SAMPLES
Based on (Tang & Jia, 2022), we explore more moderately confident samples via Gradient Synchro- nization Filter (GSF) and Prototype Proximity Filter (PPF) to boost learning, which is conceptually aligned with our approach to pscudo-labeling. From the Tab. 8, we find that GSF and PPF can only bring marginal improvements in performance. The reason is the highly overlap in sample selection between the CDC's strategy and those selected by GSF and PPE, the expansion of selected samples less than 5%.
Nevertheless, related techniques hold potential to further enhance model, which we intend to explore thoroughly in the future.
Table 8: Clustering performance (ACC, NMI, ARI %) on STL- 10 and CIFAR-20.
STL-10
CIFAR-20
Method
ACC1
NMI
ARI
ACC1
NMI1
ARI1
CDC
93.03
85.85
85.57
61.66
60.94
46.57
CDC+GSF
92.98
85.77
85.48
61.92
61.20
46.79
CDC+PPF
93.03
85.79
85.57
62.12
61.30
46.57
C.1.2 ENHANCING SELF-TRAINING WITH DYNAMIC THRESHOLDING SSL
We explore the application of the thresholding strategies from FlexMatch (Zhang et al., 2021) and FreeMatch (Wang et al., 2023b) in our CDC framework by two approaches (Apply Directly and Integration).
Apply Directly. Consistent with CDC, after pre-training with MoCo-v2 and performing prototype- based initialization, we utilize pseudo-label learning, evaluating the learning state based on the model's output. As shown in the first two rows of Tab. 9, CDC outperforms FlexMatch and FreeMatch on both datasets. The reasons are as follows. FlexMatch and FreeMatch rely on the prediction confidence estimated by well-calibrated model, which is reasonable in semi-supervised learning (SSL) scenarios where some labels are known. In deep clustering, the model updates thresholds based over-confident predictions, which can lead the selection of more noisy la- bels, causing performance degradation-this effect is more pronounced in challenging datasets like CIFAR-20. Specifically, selecting 80.2% of the overall samples corresponds to accuracy of 54.9% in FlexMatch, selecting 90.8% of the overall samples also corresponds to an accuracy of 48.8% in FreeMatch, whereas selecting 55.2% of the overall samples corresponds an accuracy of 61.7% in CDC.
Integration. Consistent with the previous settings, only the threshold selection strategy CDC is replaced by FlexMatch and FreeMatch. As shown in the last two rows of Tab. 9, CDC outperforms FlexMateh and FreeMatch on both datasets. The reasons are as follows. Under the predictions of well-calibrated model, FlexMatch requires dataset-specific global threshold. When the global threshold for STL-10 and CIFAR-20 is set to 0.95 as recommended by (Zhang et al., 2021), 1), results in significantly different sample selection scales (91.2% in STL-10 and 3.2% in CIFAR- 20), affecting performance improvements. FreeMatch is more robust, similar to CDC, with both methods being free from manually setting thresholds. The difference between FreeMatch and CDC on CIFAR-20 lies in the handling of head classes (classes with a large number of selected samples): 1) Match uses local threshold adjustments modify the overall threshold (the average of the
21

---

Published as a conference paper at ICLR 2025
maximum probability values for all samples), with local thresholds after maxNorm ranging from to
1. Thus, the thresholds for head classes are less than the overall threshold. Accordingly, FreeMatch will introduce more samples for these classes (and more wrong pseudo-labels). 2) CDC directly selects reliable samples hased on calibrated probabilities for each class, ensuring that the thresholds for head classes
reduced by the overall threshold.
Table 9: Clustering performance ACC, NMI, ARI(S) and calibration error ECE (%) of Semi- Supervised Learning methods applied directly to self-labeling (first two rows) and integrated into CDC (last rows) STL-10 and CIFAR-20. Sel. represents the proportion of selected samples.
STL-10
CIFAR-20
Method
ACC 1 NMI
ART
ECF. Sel. (%)
ACC
NMIT
ART
ECR, Sel. (%)
FlexMatch (Zhang et al., 2021)
92.6
85.6
84.8
5.8
90.6
54.9
55.6
39.2
37.4
80.2
FreeMatch (Wang et al., 2023b)
93.0
85,9
85.6
5.1
88.3
48.8
54.9
16.8
44.0
90.8
CDC-Cal
93.0
85.8
85.6
0.9
89.4
61.7
60.9
46.6
4.9
55.2
CDC+FlexMatch CDC+FreeMatch
92.3
84.6
84.0
2.0
91.2
57.2
59.4
42.1
6.5
3.2
92.9
85.3
0.7
81.9
57.8
43.6
1.4
64.3
3000
3000
Wrong Correct
Wrrong Comect
2000
Numbar of Samples
2000
1500
1500
Numbee
1000
1000
Totali 38346
33333
500
Acc-
500
Acc-lD 74
10
20
Clusses
Classes
FreeMatch
(b) CDC-Cal
Figure 10: The distribution of selected high-confidence samples on each class (sorted by the num- ber of correctly predicted samples per class) after 100 epochs of training on CIFAR-20. "Total" and "Correct" represent the number of selected and correctly predicted high-contidence samples, respectively, while "ACC" represents the accuracy of selected samples. Among the top classes with the highest number of correct predictions, FreeMatch selectes 12,671 samples with an accu- racy of 89.8%, while CDC selectes 10,602 samples with an accuracy of 94.9%. In the classes with the lowest number of correct predictions, FreeMatch selectes 5,833 samples with an accuracy of 38.87% whereas CDC selectes 6,011 samples with accuracy of 44.52%.
Fig. 10 shows the category-specific view of selection preferences on CIFAR-20, where FreeMatch's lower thresholds in high-confidence areas introduce wrong pseudo-labels impacting perfor- mance improvements.
22

---

Published as a conference paper at ICLR 2025
C.2 STANDARD DEVIATION FOR DEEP CLUSTERING METHODS
Table 10: The clustering performance ACC, NMI, ARI (meantstd %) and calibration enoor ECE (mean*std %) of various deep clustering methods trained on CIFAR-1O, CIFAR-20 and STL-10. All experiments are conducted over 8 different runs.
CIFAR-IO
CIFAR-20
STL-I0
Method
ACC1
NMI1
ART
ECE,
ACC1
NMI
ARI
ECEL
ACC1
NMIT
ARI
ECEL
MoCo-v2
78.922.
75.352.
61 41.00.0
47.82.0
53.9+18
23.2419
70.3+4.0 09.4=188
45.5+2.4
Simsiam
72.347.7
71.523.2
57.1+5.3
35.321.3 38.143.0
19.4212
55.0w3.2 61.42,11
36.642.0
BYOL
65.124.7 70.121.6
54.843.1
33.3+1.6 41.644.6
19.0214
59.145.5 67.7a3.0
43.444.4
CC
82.5+3.6
75.242.0
70.343.2 8.43,44
46.422.3
48.841.1
31.1+15
25.8
82.9+7.7
76.0=5.2
71.847. 10.347.4
SCAN-2
87,3+1.6
8
77,542.2
74+1.8
49.7=1.7
53.0+1.1
35.2+1.8
16.6+17
88.6#0.9
79.6.0.7
77.6+1.2
5.2+1.1
SCAN.3
91.0+1.2
83.6+1.5
5.941.1
51.1=19
39.3+1.8
91.8+0.3
84.1 83,4+0.6
CDC-Cal
94,3+0.6
88.5+0.7 88.2#1.1 1.3+0.2
59,4=1.3
60.240.6
44.9+1.1 3.5&1.2
93.1
86,0#0.2
85.6+0.3 1.120.2
Table 11: The clustering performance ACC, NMI, ARI (meantstd %) and calibration error ECE
(meantstd %) of various deep clustering methods trained on ImageNet-10, ImageNet-Doo and Net-lemaeeNe.. All experiments are conducted over 8 different runs.
Net 10
ImageNet Dogs
Tiny ImugeNet
Method
ACC1
NMF
ART
ECE
ACC
NMF
ARTI
ECR
ACCF
NM
ARIT
ECE
MuCo.v2
67.428.6
45.7=11.1
65.421.9
65.742.6
51.0a28
25.1a1.2
42.641.3
10.611.2
Simsiam
78.8+3.7
81.825.8
70.5+8.2
49.9:5.0
51.816.9
33.626.7
17.443.1
34,812.6
7.5+1.5
BYOL
71.845.8
77.124.9
60.248.0
59.848.1
63.749.0
42,9279
1288s31
30.742.8
5.341.5
CC
91.9+2.
88.8+2.2
86.34399
6.742.5
63.9.5.2
60.343.7
49.5+5.3 20.644.2
13.6a0.7
41.044.3
5.740.2
4.6+0.7
SCAN-2
93.3+3.0
88.5+2.0
87 8739
5.1+3.0
65.2+4.0
62.242.4
51.8+34 25 8+3.7
27.1
49.943.4
14.260.6
23.3+2.1
SCAN-3
94.8+3.1
91.3+2.2
90.744.2
3.7+29
72.324.0
68.92288
60.624.0 20.2:4.1
24.9a0.9
39.840.6
12.5+0.6
49.2=114
CDC-Cal
97.220.1 92.920.3
93.820.2
0.940.1
75.5+2.0
73.7&1.5 66.1#2.2 10.4216
33.720.5 47.620.2 19.920.2 11.1
From the Tabs. 10-11, we can observe that our method has much smaller variance than most com- pared methods, while at the same time, produces the best clustering performance and calibration performance. This is because the compared methods usually employ the random initialization on the clustering head, making model training unstable. While method proposes novel initial- ization strategy, utilizing feature prototypes from the pretrained model to initialize the clustering head and the calibration head. This approach significantly reduces the variance in method's performance as evidenced in Fig.4.
C.3 TRANSFER OF HYPERPARAMETER ACROSS DATASETS
We apply the hyperparameters from ImageNet10 (detailed in Sec. 4.1) to the 50, 100, and 200 subsets of ImageNet and compared them with the SCAN method. We conduct the fair comparison under the same backbone network (ResNet-50) and the same subset divisions. For SCAN, we test the trained model from the SCAN code repository?
Results. The results in Tab. 12 demonstrate a robust improvement in calibration error metrics with our method. showing an average reduction of 8.4%. Mcanwhile, the ACC of our method consistently surpasses that of the SCAN method.
Table 12: The clustering performance ACC, NMI, ARI (%) and calibration error ECE (%) of SCAN and CDC on 50, 100, and 200 subsets of ImageNet.
ImageNet50
ImageNet100
ImageNet200
Method
ACC1
ECE|
ACCt
ECEJ
ACC1
ECE|
SCAN-2
75.1
15.7
66.2
21.9
56.3
28.1
SCAN-3
76.8
14.2
68.9
18.8
58.1
25.8
CDC-Cal
77.8
7.5
71.2
13,0
61.2
13.2
23

---

Published as a conference paper at ICLR 2025
C.4 GENERAL APPLICABILITY OF CDC AS SELF-LABELING STAGE
We apply CDC as post-clustering self-labeling stage to models like SeCu and SCAN. The results presented in Tab. 13 that incorporating CDC following SCAN and SeCu not only enhances cluster- ing accuracy but also significantly decreases the Expected Calibration Error (ECE). This proves the general applicability of our method.
Table 13: Changes in clustering performance metries (ACC, NMI, ARI %) and Expected Calibration Error (ECE %) after 100 epochs of training across the CIFAR- 10 and CIFAR-20 datasets.
CIFAR-I0
CIFAR 20
Method
ACC1
NMI
ARI
ECE]
ACC1
NMI
ARIt
ECE]
SCAN-2 +CDC
+1.4
+2.3
+2.2
-3.2
+2.6
+4,5
43.9
-6.8
SCAN-3 +CDC
+1.2
+2.6
+2.4
-4.3
+2.0
+4.2
+3.0
-11.9
SeCu-Size +CDC
+1.9
+2.6
+3.1
-0.5
+3.4
+3.1
+3.4
-00.8
ScCu +CDC
+0.3
+0.2
+0.5
-3.7
+1.1
+0.9
+1.2
-6.8
C.5 OOD DETECTION FOR DEEP CLUSTERING METHODS
Datasets: We employ CIFAR-10 and CIFAR-20 in-distribution (ID) datasets when mixing data from unrelated datasets. We use Near-OOD datasets (CIFAR-20 and Tiny ImageNet for CIFAR-10; CIFAR-10 and Tiny ImageNet for CIFAR-20) and Far-OOD datasets (SVHN, Textures, Places365).
Procedure: For the methods of CC, SCAN, SPICE, SeCu, and CDC (Ours), max softmax scores (MSP) of ID and OOD samples to assess the model's performance on AUROC and FPR95 metrics.
Table 14: The AUROC (%) and FPR95 (%) of various deep clustering methods trained on CIFAR-
10 for -fist--distiitutiion (OOD) detection.
OOD-
CIFAR-I00 100
Tiny ImageNet
SVHN
Textures
Places365
Method
AUROCT FPR95,,
AUROC1 FPR95.
AUROC
FPR95.
AUROC1 FPR95.
AUROC1 FPR95,,
CC
89,9
55.3
88.0
55.2
94.4
34.9
91.3
43.2
93.2
39.9
SCAN-2
87.3
66.2
89.1
59.6
93.6
41.5
91.4
50.3
91.8
52.5
SCAN-3
88.6
64.2
92.2
49.8
95.2
33.2
92.8
47.0
93.2
46.7
SPICE-2
57.0
80.8
57.1
80.6
60.5
73.9
58.1
78.7
59.4
76.1
SPICE-3
70.9
93.8
38.7
$2.5
91.4
52.9
ScCu-Size
77.1
80.1
80.6
77.7
81.8
74.5
80.7
84.5
79.5
79.6
SeCu
86.8
65.8
91.5
52.0
92.1
51.3
91.9
53.3
91.4
55.8
CDC-Cal
93.0
37.8
94.8
27.4
97.8
12.4
95.9
23.5
96.1
23.0
Table 15: The AUROC (%) and FPR95 (%) various deep clustering methods trained on CIFAR- 20 for -uitof-disttibution (OOD) detection.
OOD
CIFAR-10
Tiny-ImageNet
SVHN AUROC*
Textures
Places365
Method
AUROCT FPR95
AUROCT FPR95]
FPR95
AUROC1
FPR9S
AUROC1 FPR95
CC
69.9
86.2
77.2
77.7
85.8
63.4
76.7
77.0
80.7
72.6
SCAN-2
69.3
88.5
79.8
73.8
82.3
70.9
83.2
70.0
80.5
75.2
SCAN-3
68.2
89.1
82.4
71.4
86.1
66.5
82.8
70.7
82.4
74.4
SPICE-2
51.4
92.9
52.7
90.2
54.2
87.3
53.2
89.2
SPICE-3
70.2
86.4
85.0
87.8
68.1
85.2
83.7
76.6
SeCu-Size
63.8
92.1
67.8
92.7
68.0
87.9
71.6
90.4
68.1
89.7
SeCu
73.5
83.9
82.3
74.9
86.8
74.2
82.9
77.0
82.8
76.1
CDC-Cal
76.8
82.6
85.7
64.6
92.2
43.8
87.8
60.6
87.4
64.0
24

---

Published as a conference paper at ICLR 2025
Results:As shown in Tab. 14 and Tab. 15, method demonstrates superior OOD performance compared to comparative methods, indicating the significant role of calibration in rejecting unknown samples. For instance, CDC trained CIFAR- 10 achieved 2.9% increase in AUROC and 19.5% decrease FPR95 compared to the most effective baseline method.
40
Out of Distribution Distribution
100
of Distribution
40
35
Out of Distribution
Distribution
Out of Distribution
Distribution
30
80
30-
30-
25
Deersity
60
25-
25-
20
20
20-
15
15-
15-
10
10
10
8.0
8.0 0.6 Confidence
8.0
8.0
Confiderce
Confidence
Contidence
(a) SCAN-3
(b) SPICE-3
(c) SeCu
(d) CDC-Cal
Figure 11: The OOD detection for different deep clustering methods trained on CIFAR-10. The out- of-distribution dataset is CIFAR-100 CDC-Cal demonstrates relatively stronger separation
between in-distribution and OOD samples.
We also plot the density-confidence in Fig. 11 to illustrate that the OOD regions of our method are more concentrated in areas of low confidence, demonstrating enhanced separation ability between in-distribution and OOD samples.
25


Reviews:
[{"review_id": "F9meavGE9X", "reviewer": "Reviewer_bpNw", "summary": "This paper addresses the over-confidence problem in deep clustering, where the predicted confidence for a sample significantly exceeds its actual accuracy. The authors propose a novel calibrated deep clustering framework that consists of a dual-head model, including a calibration head and a clustering head. The calibration head corrects the overconfident predictions from the clustering head, aligning estimated confidence with the model's learning status. The clustering head then selects reliable high-confidence samples for pseudo-label self-training. Additionally, the paper introduces a new network initialization strategy that improves training speed and robustness. The proposed model demonstrates significant improvements over state-of-the-art methods, achieving a 10× reduction in expected calibration error and enhanced clustering accuracy, supported by solid theoretical guarantees.", "questions": "See Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Novel Approach to Calibration: The introduction of a dual-head model that effectively calibrates confidence estimates addresses a critical gap in deep clustering, enhancing the reliability of predictions.\n\n- Improved Self-Training Mechanism: By leveraging high-confidence samples for pseudo-label self-training, the proposed method improves the quality of the training data, which is crucial for achieving better clustering performance.\n\n- Robust Experimental Validation: The extensive experiments conducted demonstrate the effectiveness of the calibrated deep clustering model, showcasing significant improvements in both expected calibration error and clustering accuracy compared to existing methods, which adds credibility to the proposed approach.", "weaknesses": "- This paper would benefit from a more comprehensive literature review. Specifically, it overlooks some closely related works, such as [a,b], which address the over-confidence issue for standard supervised learning or unsupervised domain adaptation (i.e., unlabeled target data clustering with the help of labeled source data). I recommend that the authors include a discussion of these works and its connections to their research. This will help to contextualize their contributions within the existing body of knowledge.\n\n[a] Revisiting the Calibration of Modern Neural Networks. NeurIPS, 2021.\n\n[b] Transferable Calibration with Lower Bias and Variance in Domain Adaptation. NeurIPS, 2020.\n\n- The authors mention that the calibration head adjusts the overconfident predictions of the clustering head to generate prediction confidence that aligns with the model's learning status. However, in the context of completely unsupervised learning and clustering, it is not clear how the model's learning status is determined without any labeled data. I recommend that the authors provide a more detailed explanation of the methodology used to assess the model's learning state in the absence of supervision. This clarification is essential for understanding the effectiveness of the calibration head and its role in improving prediction confidence.\n\n- The authors state that the clustering head dynamically selects reliable high-confidence samples estimated by the calibration head for pseudo-label self-training. However, it would be beneficial to explore whether insights from existing semi-supervised learning (SSL) methods, such as those discussed in [c], could be incorporated to further enhance the self-training process.\n\nAdditionally, a critical analysis of current self-training methods is warranted to determine whether they contribute to poor calibration of predictions. It would be valuable to quantify the extent of this calibration issue and discuss its implications on model performance. By addressing these points, the authors could strengthen their argument and provide a more comprehensive understanding of the calibration dynamics in their approach.\n\n[c] Towards Discovering the Effectiveness of Moderately Confident Samples for Semi-Supervised Learning. CVPR, 2022.\n\n- The authors note that existing approaches rely on a fixed threshold to select pseudo labels, which does not account for the model's learning status. This can lead to issues during different training stages: a higher threshold in the early stages may result in selecting fewer and class-imbalanced samples, hindering convergence speed, while in later stages, a fixed threshold may introduce noisy pseudo-labels due to increased predicted confidence.\n\nIt would be beneficial for the authors to consider incorporating insights from existing SSL works, such as [d] and its subsequent studies, which have proposed dynamic thresholds and class-wise thresholds. These methods could potentially enhance the adaptability of pseudo-label selection throughout the training process, improving both the quality of the training data and the overall performance of the model. A discussion on how these approaches could be integrated or compared with the proposed methodology would strengthen this work.\n\n[d] FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling. NeurIPS, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v8D8VdSZvg", "reviewer": "Reviewer_yP9a", "summary": "The authors identify the problem of overconfident predictions in deep clustering methods. To overcome this problem, they propose a dual head design with a dedicated calibration head aside the usual clustering head to improve deep clustering performance and calibration. Further, they propose an initialization strategy for the dual heads that improves stability when transitioning between the pretraining and deep clustering phases.", "questions": "- What exactly is reported in Table 2? Are these the best or average results over multiple runs? If yes, how many runs were conducted and what are the standard deviations across runs? See also the corresponding weakness described above.\n\n- Hyperparameter sensitivity analysis w.r.t. K in Figure 8 shows that CDC is stable for different K for CIFAR-20 and STL-10, but it is set quite different for each data set.  Is there a heuristic to set K or does K need tuning w.r.t. the ground truth labels for each data set? See also the corresponding weakness described above.\n\n- Is the MoCo-v2 pretraining used for CC, SCAN and SeCu as well or just for your method? Based on your description it is not entirely clear whether you just use the same ResNet-34 backbone for these methods or if you use MoCo-v2 pretraining for them as well. \n\n- Please clarify how you conducted the MoCo-v2 pretraining and which hyperparameters you used for each data set.\n\n- Based on my understanding your method could be used as a dedicated self-labeling stage on top of any deep clustering method, similar to how SCAN's self-labeling is currently used to improve clustering performance after the joint deep clustering stage. I would be interested to see how your method would improve, e.g., SeCu after its deep clustering stage. This would further illustrate the general applicability of your method and increase its significance for the deep clustering community.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality**\n- The authors identify and investigate the issue of overconfidence in deep clustering methods. Based on their findings they propose a novel Calibrated Deep Clustering (CDC) method.  This is a quite an interesting angle to improve deep clustering methods and connects the field to confidence calibration methods from (semi)-supervised learning. \n\n**Quality**\n- The authors provide extensive experiments across different benchmark data sets and deep clustering methods. Further, they made the extra effort of reimplementing SeCu, SCAN and CC to obtain a unified benchmarking environment increasing the fairness of the comparison. I would highly encourage the authors to release their code with the full benchmarking environment, as this will benefit the deep clustering community. Additionally, the authors provide ablation studies and a hyperparameter analysis for their algorithm CDC.\n\n**Clarity**\n- The paper is well written, and all design decisions are clearly motivated\n\n**Significance**\n- Their method CDC combines SCAN’s self-labeling procedure with a calibration head.  As self-labeling is widely used and their method does not make specific assumptions about the underlying deep clustering method, I believe this method can impact many deep clustering methods going forward.", "weaknesses": "- Unclear reporting of experimental results.  The authors fail to mention if they report average performance results in Table 2 and other figures, or if they report the results of a single (best) run. As many deep clustering methods have high variability across different random seeds it would be important to report standard deviations as well, e.g., SCAN reports standard deviations of up to 3% for CIFAR-20.  Currently, it is not possible to judge whether the proposed method CDC is stable across multiple runs, because no standard deviations are reported and it is not mentioned if several runs are conducted at all.\n\n- The hyperparameter for mini-clusters K needs to be specified quite differently for each data set ranging from 40 to 1000. Further, K does not seem to be related to the ground truth number of clusters of each data set and the authors do not provide any unsupervised heuristic that can be used to set K. The same holds true for the batch size B.\n\n- The authors do not specify how they tuned their MoCo-v2 backbone. Further, no hyperparameters for the pretraining are reported. This is an important aspect that needs to be reported for reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3dMlGnoFro", "reviewer": "Reviewer_8jfQ", "summary": "The paper considers calibration in the context of deep clustering, more specifically pseudo-label self-training based deep clustering. The authors propose an architecture consisting of a backbone (trained in a self-supervised manner) and two heads, where one is the common clustering head, while the other is a calibration head. The calibration head obtains calibrated confidences by overclustering the data, computing the average prediction for each of the clusters, and then aligning the predictions of the calibration head for members of the clusters to the clusters average prediction. The better calibrated predictions can then be leveraged for the pseudo-label selection. Beyond the calibration head, the authors propose a new initialization strategy for clustering (and calibration) heads that does ensure initialization that resembles the k-means performance on the backbone features as well as a mechanism to dynamically select the threshold for pseudo-label selection for each class. The proposed approach is evaluated on six datasets, obtaining promising performance and lower expected calibration error compared to prior deep clustering baselines.", "questions": "Could the authors comment on the sensitivity with respect to the hyperparameters as well as on the variability in results over multiple runs?\n\nShould the subscripts be switched in Theorem 1? So $E_F[Conf^{cal}]$ <= $E_F[Conf^{clu}]$ and  $E_T[Conf^{cal}]$ = $E_T[Conf^{clu}]$?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is overall well written and the contributions presented in a clear and well-motivated manner. \n\n- Considering calibration in deep clustering is growing more important as state-of-the-art methods rely increasingly on pseudo-label self-training and has been not been addressed previously to the reviewers knowledge.\n\n- The proposed approaches are intuitive and effective as indicated by the empirical results and the ablation study. \n\n- Empirical results are supported by theoretic insights and analysis.", "weaknesses": "Hyperparameter selection is challenging in a clustering setting when no labeled data is available. It is therefore beneficial to have approaches that are robust with respect to hyperparameters. Currently, the sensitivity to the added hyperparameters is not discussed and relative specific hyperparameters are defined for different datasets (B, K) or not provided (H). How were these hyperparameters selected and how sensitive is the model to these choices. Do the authors have insights on how these should be chosen in the typical clustering setting where labeled data is unavailable? \n\nOne weakness of the work is that results are currently reported over single runs and without information about variability. Deep clustering approaches tend to be less stable than their supervised counterparts and some more information on the robustness of these results would therefore be beneficial.\n\nThe claim in the abstract regarding surpassing state-of-the-art deep clustering methods by 10x in term of expected calibration error should be revised to clarify that this is compared to only some of the baselines and on some of the considered datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I34WdQPeGn", "reviewer": "Reviewer_KQtX", "summary": "This paper proposes a dual head deep clustering model that can effectively calibrate the estimated confidence and the actual accuracy.", "questions": "No", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "This paper proposes a dual head deep clustering model that can effectively calibrate the estimated confidence and the actual accuracy.", "weaknesses": "According to the discussion between the authors and other reviewers, I altered my rating.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7OXoKdJ2E1", "reviewer": "Reviewer_1qud", "summary": "This paper proposes calibrated deep clustering, a method to calibrate the confidence of deep clustering networks. Specifically, it proposes a separate calibration head alongside the traditional clustering head to optimize the learning and prediction confidence of clusters. This approach leads to better-calibrated models as it ensures that the calibration head consistently achieves lower errors compared to the clustering head. The authors provide a theoretical proof to demonstrate that the calibration error from the calibration head is less than or equal to that of the clustering head, supporting the validity of their method.", "questions": "1. The 'classifier $g(\\theta, \\cdot)$' is mentioned only in the Notation and Algorithm 1. However its role in the proposed pipeline remains unclear to me. Can you clarify its specific function within the pipeline? Is this term meant to generally refer to both the clustering head and calibration head?\n2. Can you provide a detailed algorithm outlining how prediction is done?\n3. In Equation 4, is the total loss simply the sum of the two individual losses? Is there a hyperparameter that controls the weighting between them? Are the two losses on similar scales? Additionally, is the use of the negative entropy loss inspired by prior works? It would be helpful to see an ablation study that removes this term and compare the results.\n4. How does the algorithm perform in the presence of out-of-distribution (OOD) samples? Specifically, could you provide a Density-confidence plot as in figure 3, when OOD samples are introduced in the dataset similar? One possible way to generate OOD samples is by adding random Gaussian noise to selected samples or by mixing data from irrelevant datasets.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "1. The authors provide a comprehensive comparison of their method against 14 prior works. They have also tested the method on 6 datasets, which is a sufficient number to validate their approach. The experimental results clearly demonstrate that CDC achieves significantly better performance than previous methods. The experimental design, including both the main experiments and ablation studies, is rigorous and well-justified.\n2. The figures are well-designed and effectively highlight the advantages of the proposed approach. Both the presentation of the results and the visuals are compelling and easy to interpret.\n3. The authors offer sound theoretical proof showing that the proposed method achieves a lower expected calibration error and penalizes only unreliable regions, adding to the method’s credibility.\n4. The motivation behind the work is well-justified. Both the introduction and abstract are clearly written and provide a strong foundation for the paper.", "weaknesses": "1. The authors should reconsider the structure of the paper. In my view, the methods section should first focus on the network and training design, followed by secondary aspects like initialization strategies. While initialization techniques can improve performance and speed up convergence, they are supplementary rather than a primary contribution. Therefore, dedicating excessive space to this, even with theoretical justifications, is unnecessary. Additionally, the experimental results section is too brief. I recommend condensing the methods description and moving a more detailed analysis of results from the appendix into the main body of the paper.\n\n2. The notation is inconsistent and confusing. Several terms are introduced without adequate mathematical explanation. For example, $p_i^{clu}$ is first mentioned in Line 230 as \"the prediction of $x_i$ by the clustering head\" but lacks a formal mathematical definition. I suggest including an equation, such as defining the clustering head as $g(\\theta_{clu},\\cdot)$ and expressing $p_i^{clu} = g(\\theta_{clu}, x_i)$. A similar issue with $p_i^{cal}$ in Line 235. When introducing the proposed pipeline, the authors should clearly guide the reader through the entire process, from input sample $x$ to the final loss function, in a step-by-step manner in the methods section.\n\n3. The title of Section 3 should be 'Proposed Method'.\n\nOverall, the paper presents promising results and good analysis. However, the presentation of the methodology is lacking, which makes the paper difficult to follow. Revising the organization and clarity of the exposition is neccessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)