Calibrated Information Bottleneck for Trusted Multi-modal Clustering
Authors: Shizhe Hu
Zhangwen Gou
Shuaiju Li
Jin Qin
Xiaoheng Jiang
Pei Lv
Mingliang Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-modal Clustering
Information Bottleneck
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: This paper proposes a novel CaLibrated Information Bottleneck (CLIB) framework for trusted multi-modal clustering.

Abstract: Information Bottleneck (IB) Theory is renowned for its ability to learn simple, compact, and effective data representations. In multi-modal clustering, IB theory effectively eliminates interfering redundancy and noise from multi-modal data, while maximally preserving the discriminative information. Existing IB-based multi-modal clustering methods suffer from low-quality pseudo-labels and over-reliance on accurate Mutual Information (MI) estimation, which is known to be challenging. Moreover, unreliable or noisy pseudo-labels may lead to an overconfident clustering outcome. To address these challenges, this paper proposes a novel CaLibrated Information Bottleneck (CLIB) framework designed to learn a clustering that is both accurate and trustworthy. We build a parallel multi-head network architecture—incorporating 
one primary cluster head and several modality-specific calibration heads—which achieves three key goals: namely, calibrating for the distortions introduced by biased MI estimation thus improving the stability of IB, constructing reliable target variables for IB from multiple modalities and producing a trustworthy clustering result. Notably, we design a dynamic pseudo-label selection strategy based on information redundancy theory to extract high-quality pseudo-labels, thereby enhancing training stability. Experimental results demonstrate that our model not only achieves competitive clustering accuracy on multiple benchmark datasets but also exhibits excellent performance on the expected calibration error metric. Code is available at \textcolor{red}{https://shizhehu.github.io/}.

Published as a conference paper at ICLR 2026
CALIBRATED INFORMATION BOTTLENECK FOR TRUSTED
CLUSTERING
Shizhe Hu Zhangwen Gou'uSuuaju Li', Jin Qin', Xiaoheng Jiang!?
Pei Ly & Mingliang 123.2
"School of Computer Science and Artificial Intelligence, Zhengzhou University
2 Artifiiiil Intelligence and Robotics Institute, Zhengzhou University
SShandong Bosuan Zhixin Information Technology Co., Ltd
*State Key Laboratory of Metabolic Dysregulation & Prevention and Treatment of Esophageal Cancer
ieshi z gmail com
ABSTRACT
Information Bottleneck (IB) Theory renowned for its ability learn simple, compact, and effective data representations. In multi-modal clustering, IB theory effectively eliminates interfering redundancy and noise from multi-modal data, while maximally preserving the discriminative information. Existing IB- (B-based multi-modal clustering methods suffer from low-quality pseudo-labels and over- reliance on accurate Mutual Information (MI) estimation, which is known to be challenging. Moreover, unreliable or noisy pseudo-labels may lead to an over- confident clustering outcome. To address these challenges, this paper proposes a novel Cilribeeeed Information Bottleneck (CLIB) framework designed to learn a clustering that is both accurate and trustworthy. We build parallel multi-head net- work architecture- itw.eccuureeiiioooorrrrtntnn one primary cluster head and several modality- specific calibration heads-which achieves three key goals: namely, calibrating for the distortions introduced by biased MI estimation thus improving the stability of IB, constructing reliable target variables for IB from multiple modalities and producing a trustworthy clustering result. Notably, we design a dynamic pseudo- label selection strategy based on information redundancy theory to extract high- quality pseudo-labels, thereby enhancing training stability. Experimental results demonstrate that our model not only achieves competitive clustering accuracy on multiple benchmark datasets but also exhibits excellent performance on the ex- pected calibration error metric. Code available at https://shizhehh. github.io/.
INTRODUCTION
Information Bottleneck Theory (Tishyy et al. 2000 Hu et al. 2024) provides an elegant theo- retical framework for learning concise and effective data representations, centered finding an optimal balance between data compression and information preservation. This is achieved by creat- ing compressed representation T' for an input X that optimized retain features relevant to target Y. The optimization is governed by the objective function:
Emin I(X:T) BI(T;Y)
(1)
Where I(::)) denotes the mutual information. The parameter / balances the trade-off between compression and relevance. By minimizing (X;T) while simultaneously maximizing [[T; Y), IB is able to learn compact yet informative representation of the input data. In the context of deep learning, the advent of methods such as the Deep Variational Information Bottleneck (Alemi et al.) 2017) has enabled the integration of the IB principle with neural networks.
Since IB capable of extracting pure features, it has been utilized multi-modee modal clustering meth- ods. However, applying the IB principle directly to multi-modal clustering faces fundamental
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Spyl
K- k-5,5
Feature
Oudlit
Decamti Top-k
CalHcad,
X
H
Filter
Aree
W
Botteneck
CluHead
Edis
Wm
X m
H
Eral
Filter
Pseudo-label
CalHcad,
Calibration sigmal
Feature Extraction
Calibration
Figure 1: Illustration of the proposed CLIB framework. First, compact features H' are extracted from various modalities using IB. These multi-modal features are then integrated into fused feature representation H' While the features from each modality serve as inputs multiple calibration heads, the fused representation is processed by the cluster head. A screening mechanism is employed select high-quality pseudo-labels from the output of calibration heads, which in turn supervise the training of the cluster head. This not only generates high-quality clustering results but also backpropagates a calibration signal IB via the gradients from the cluster head, thereby guiding the learning of IB.
challenge: the efficacy of IB highly dependent on a reliable target variable. In supervised learn- ing, this target typically the ground-truth label; in self-supervised learning, this role is filled by pseudo-labels, which act as the model's internal proxy for correctness. In current multi-modal clus- tering approaches utilizing IB (Hu al. 2025 Yan et al. 2024), model-generated pseudo-labels are commonly used for guidance. A critical issue arises because these pseudo-labels are often noisy, particularly during the initial training stages. This creates vicious cycle: low-quality pseudo-labelt guide the learning of poor representations, which in turn produce flawed pseudo-label for the next iteration. This feedback loop can amplify initial errors, causing the model to become progressively overconfident in its incorrect predictions. Therefore, a calibration mechanism is necessary to break this degenerative cycle and mitigate model overconfidence.
The Computation of Mutual Information, which measures the degree of association between two random variables, is critical to the performance of IB. However, calculating MI for high- dimensional, complex data (e.g., image text features) using its classical mathematical definition, which involves integration or summation over the true probability distributions, is extremely diffi- cult, if not practically infeasible. [Zzyz et al.] (2023) points out that existing estimators (Belghazi) et al. 2018: Song & Ermon 2020) are often evaluated only on simple data distributions (e.g., mul- tivariate normal distributions). which fails reflect their performance on complex, real-world data, further underscoring the difficulty of accurate MI estimation. This difficulty poses significant barrier to the effecttive application of the IB principle complex, real-world scenarios.
To address these challenges, we propose a novel CLIB framework, which leverages the IB principle learn compact representations from each modality. These representations then integrated via an adaptive weighted mean fusion strategy to establish unified cross cross- semantic space. Archi- tecturally, as shown in FFgure I CLIB consists of multiple modality-speeifee calibration heads and single cluster head that operates on the fused representation. These heads engage in a reciprocal learning process, providing mutual supervision while strategically decoupling the calibration and final clustering objectives.
Notably, we introduce a pseudo-label selection mechanism based on information redundancy, quan- tified by entropy, to perceive the clustering quality of individual modalities and subsequently filter for high-confidence pseudo-labels to supervise the model's training. Furthermore, we adopt a two- stage training methodology: IB is first pre-trained to learn robust features before the calibration module is introduced, which significantly reduces the noise propagated into the calibration stage. This creates symbiotic relationship: IB provides compact features for the downstream cluster-
2

---

Published as a conference paper at ICLR 2026
ing task, while the calibration module offers corrective feedback through backpropagated gradients, thereby refining the feature extraction process itself. Crucially, this mechanism ensures that the feature learning is not misled by noisy pseudo-labels. To our knowledge, this is the first work ad- dressing the trusted multi-modal clustering problem with calibrated IB framework.
Our contributions are threefold:
We pioneer method applying IB with calibrated, high-quality, and low-noise target vari- ables which enhances the robustness of IB extract compact and discriminative features from multi-modal data.
Unlike existing IB methods, CLIB mitigates the over-reliance on precise MI estimation. Even if the MI estimated for a particular modality is biased, the calibration mechanism al- lows for its correction by leveraging information from other modalities, thereby enhancing the overall clustering performance of this framework. To our knowledge, this is the first work to introduce calibration for the performance issues IB that arise from the inaccurate estimation of MI.
Benefiting from the calibration mechanism, our proposed method not only achieves supe- rior accuracy but also demonstrates significantly lower Expected Calibration Error (ECE) compared to state-of-the-ard multi-modal clustering methods. This enhances the trustwor- thiness of the IB framework and effectively alleviates the issue of model overconfidence.
2 RELATED WORK
2.1 MUTUAL INFORMATION ESTIMATION
To overcome the challenge of calculating MI for nigh-dimensional data, previous studies have pre- dominantly adopted several strategies for approximation: Variational Inference- frrrencce-Based Approxima- tion method (Barber & A 2004 Alemi et al.) (017) transforms the complex calculation of MI into optimizable lower-bound estimation problem by introducing parameterized variational distribution to approximate the true posterior distribution. The performance of this method is highly dependent on the expressive power of the chosen variational distribution, as an overly simplistic distribution family can result in an inaccurate estimation of MI. Neural Network-Based Estimation (Belghazi et al., 2018 Song & Ermon 2020) utilizes the strong fitting capabilities of neural net- works to estimate MI by learning its divergence form directly from samples via gradient descent, thus circumventing the need for explicit probability density estimation. In a mini-batch stochas- tic gradient descent training environment, the method's 's gradient estimates can be biased, requiring the introduction of bias-coorection techniques to improve its stability and performance. Contrastive Learning- -Based Estimation den Oord et al., 2018 Qiu et al. 2021) transforms the problem of MI estimation into a contrastive learning task, learning an effective lower bound by maximizing the similarity between "positive sample pairs" and minimizing it for "negative sample pairs". The performance of this approach is sensitive to the quantity and quality of negative samples, as an in- sufficient number of negative samples can impair the learning of discriminative features and thus reduce the accuracy of the MI estimation.
The aforementioned analysis reveals that all estimation methods for MI have certain drawbacks and inherent inaccuracies, In the context of IB basddded multi-modal clustering, it is crucial to mitigate the effects of these potential inaccuracies obtain more precise clustering results.
2.2 MULTT--ODAL CLUSTEERNG
Multi Modal data is characterized by complementary modalities, which provide distinct facets of information about the same object. In recent years, series of multi-modal clustering methods has been proposed, including approaches that utilize Attention Mechanisms (Akbari et al., 2021 Huang ctt al. 2023), Adversarial Learning (Ganin et al. 2016). class of approaches have focused on lever- aging pseudo-labels in self-supervised frameworks. Pseudo-labels generated using fixed thresholds (Sohn et al. 2020; Gansbeke et al. 2020) often suffer from low quality and are riddled with noise. Consequently, they misdirect the model's learning trajectory. This problem is exacerbated over iterative training, as the model becomes increasingly confident in its incorrect predictions- pro- cess of error accumulation that results in highly overconfident model.
3

---

Published as a conference paper at ICLR 2026
Since IB is able to extract compact and discriminative features beneficial for clustering tasks, many IB-based clustering methods have been proposed. PTIB (Lou et al. 2025a) proposes a "peer-review" mechanism to learn modality weights in parameter-free manner through mutual evaluation and trustworthiness scoring between modalities. SWIB (Lou et al.] 2024) introduces self-supervised information from pseudo-labels into the weighted IB, learning weights by combining view quality with clustering consistency. MSDIB (Hu et al. 2025) proposes a "multi-aspect self-guided" strat- egy that comprehensively utilizes the private information, shared information, and pseudo-labels of modalities to learn cluster-frient friendly representations through the information bottleneck. Overall, due to its powerful representation learning capabilities, IB has been applied various clustering algo- rithms in recent years. However, applying IB in an unsupervised setting requires the construction of high-quality, reliable target variables, which necessitates a mechanism for filtering pseudo labels.
2.3 TRUSTWORTHY CLUSTERING
While deep clustering models have achieved significant performance, they often suffer from "over- confidence," where the model's predicted confidence far exceeds its actual accuracy et al.) 2025) Accurate confidence estimation crucial for building trustworthy decision-makinn systems, espe- cially in safety-critical domains such as medical diagnosis (Mimori et al., (021)) and autonomous driving (Feng et al. 2019). PTIB (Lou et al. (025a) achieves trustworthy clustering by emulating a "peer-review" mechanism. wherein different data modalities mutually evaluate one another to learn reliable fusion weights. AGARL (Yang et al.) 2024) employs an "alternating generative adversarial" strategy to enforce the alignment of feature representations from different views, thereby learning a consistent and robust shared representation for clustering. TMC (Han et al. 2021) quantifies the uncertainty of each view by treating network outputs as "evidence" and subsequently fuses multi- view information based on evidence theory to obtain trustworthy results. Most existing confidence calibration techniques are not applicable unsupervised clustering. For instance, post-processing methods like Temperature Scaling (Guo et al. (017)) require a labeled validation set, which is un- available in clustering. Meanwhile, regularization techniques like Label Smoothing (Müller et al,, 2019) excessively penalize nigh-confidence, reliable samples, preventing the model from effectively distinguishing between reliable and unreliable predictions. This gap highlights the need for calibra- tion methods specifically designed for the unsupervised clustering paradigm.
PROPOSED METHOD
Problem Definition and Overall Framework. In the multi-modal clustering task, we are given dataset X = {xI T2.......... ] containing N samples. Each sample z; is composed of data from M different modalities, L.., 22 2 2.M where x²" represents the m-th modality for sample i. Our objective is to partition these N samples into C' predefined clusters. In this section, we will provide a detailed introduction to our proposed CLIB framework. First, we describe how the IB principle is applied to learn both effective mpdelit and fused features. Then we will elaborate on the core of the framework: the calibration and clustering strategy, which consists of calibration heads and cluster head, as well as learning-aware strategy based information redundancy for filtering high-quality pseudo-labela The overall loss function for the framework is given below, where a is parameter that balances the contributions of the feature extraction and the calibration strength.
Liotal = LIB + aLCal
Notably, regarding pseudo-label sereening strategies, Jaa et al. (2025) and Li et al. (022) utilize the maximum probability value of the distribution to dynamically select pseudo-labeis, whereas this paper employs a pseudo-label screening method based on information redundancy theory.
3.1 REPRESENTATION LEARNING BASED ON INFORMATION BOTTLENECK
Feature Extraction with Information Compression. To extract compact and discriminative fea- tures from each modality, we have designed the loss function:
M Lc 1
H²')
Σ 1H'
(3)
=1/=141
4

---

Published as a conference paper at ICLR 2026
Here, X is the input for modality i, and H' is its learned representation. The first term, I(X', H"), serves as a compression objective. By minimizing it, we encourage the network to discard irrelevant information from the imput. The second term, -I(H', HH),, aims to maximize MI between repre- sentations of different modalities. This objective forces the unimodal encoders to learn a shared semantic space, process often referred to feature alignment. The alignment term ensures that in the process of compression, the model preserves semantic information that is common across modalities, which assumed to be discriminative. Minimizing Equation 3 thus produces compact and aligned unimodal features.
Information Preservation in Fusion. To effectively integrate the features learned from different modalities, we employ an adaptive weighted-average fusion mechanism. This mechanism assigns learnable weight Wm to the representation of each modality H and the final fused representation H' w/ H! + Wilt t + War HM with Σm= Wm 1. This adaptive mechanism allows the model to dynamically adjust the importance of different modalities based their contribution to the final clustering task. The Information Preservation loss function defined as:
M
hp
Σ 1==1
T(H H')
(4)
To ensure that the fused global representation retains the specific information from each modality, we maximize MI between each modal representation HT and the fused representation H Combining the two parts above, the total information bottleneck loss function is defined as:
LIB Lc 3
(5)
Here, larger value for B places greater emphasis on the feature fusion process to yield a more comprehensive fused representation, albeit the expense of unimodal feature purity. In contrast, smaller 3 is geared towards information compression to obtain more compact features, potentially the cost of modality-specific fidelity in fused feature. Through the optimization of [qqatioo 5) compact modality -epeciiit-sscciiic features are obtained, along with fused features that sufficiently extract global common semantics.
Notably, because the features extracted in the early stages of training are still noisy and not fully compressed, prematurely introducing the calibration mechanism would allow a large amount of noise into the calibration module and propagate erroneous calibration signals, preventing the model from learning correctly. To address this, we have designed a two-stage training method that warms up IB, introducing the calibration module only after the learned features have become relatively stable.
3.2 CALIBRATION MECHANISM OF THE PARALLEL MULTI-HEAD ARCHITECTURE
After the warm-up stage of IB, the model has learned compact and effective features. The goal of the calibration mechanism is to use multi-head collaboration to resolve potential issues that could affect the final clustering performance, such as MI estimation bias and low-quality data in a spe- cific modality, while also addressing the problem of model overconfidence. The overall loss of the calibration module could be:
= = + Len LetuH + Lcon
Where Cealitt and LoGH optimizes the calibration heads and cluster head respectively, Cye serves entropy regularization to prevent trivial solutions, and Loon is the consistency loss that forces the model to output a flat distribution when opinions conflict.
3.2.1 CALIBRATION HEADS
The primary task the calibration heads is to learn a probability distribution for each single modal- ity and to perceive the learning status of the current modality. A modality with good learning status should theoretically produce a more discriminative clustering probability vector, which allows us to assess whether the modality has been sufficiently learned.
First, we run the K-Means algorithm on the fused feature space to partition the samples into G. pseudo-clusters, denoted as Q.. Then, for each pseudo-cluster Q.. we calculate the mean
5

---

Published as a conference paper at ICLR 2026
of the output probabilities prl from the cluster head for all samples within that cluster: 9. [Q.] Em,eq. p. Here, [Qel is the number of samples in pseudo-cluster Qe. This cuuster-averaging strategy leverages the neighborhood structure of samples in the feature space, effectively smoothing the noise from individual sample predictions and thus providing more stable and reliable su- pervisory signal for the training of the calibration heads. Subsequently, we optimize each calibration head by minimizing
M Σ Le (pg(p)
LaakH -
-
1
N. E.EQq m=m
Where B denotes the batch size, and pim is the output probability from the calibration head for the m-th modality of sample 2;,.
Furthermore, to prevent the calibration heads from producing trivial solutions during optimization an entropy regularization loss Cre is introduced:
Lye
Σ
peal Pm
log(pm))
(8)
M
The pasl denotes the average predicted probability distribution over the entire batch for the m-th modality's calibration head. Equation 8 aims to encourage prediction diversity and avoid model collapse.
Note. As each calibration head acts as a modality-specii -peecfcc "expert" that only observes single- modality features, which may contain noise conflicting information between modalities. For this reason, we stop the gradient backpropagation from the calibration heads to the bottleneck. Not doing so would require the bottleneck to satisfy multiple, potentially contradictory objectives simul- taneously, which could lead to training instability or confused direction in feature learning. This is not merely theoretical; we also observed this conflict the ablation study detailed
Section 4.3)
3.2.2 PSEUDO-LABEL SCREENING MECHANISM
Corresponding to the calibration heads, the optimization of the cluster head, in turn, relies on the pseudo-labels generated by each calibration head. As mentioned in the introduction, low-quality pseudo-labels hinder the performance of unsupervised clustering and the application of information bottleneck theory. This paper designed dynamic sample screening mechanism based informa- tion redundancy, thoroughly assessing the learning state of the current calibration head and dynam- ically selecting high-quality pseudo-labels. This filters out ambiguous samples that the model is "unsure" about, allowing the model to learn from simple and reliable structures first and avoiding premature, overconfident, and incorrect judgments on difficult samples.
Generally, information redundancy is defined as: R(P)
H(P) where H(P)
Hun
-E p, (p:) and Hmax = H (Peven). Information redundancy can perceive the quality of probability vector, the more "peaked" the vector, the higher the score. For instance, one-hot vector receives the highest score, represents an inability to distinguish which cluster it belongs to. For example, 1 x D vector Peven: ( b..... b) receives the lowest score, R(Peven) = 0. Our screening mechanism is variant information redundancy. The quality score for single sample is:
1. The more uniform the vector, the more it
S(P) = 1 H(P)
H
While computing quality scores, samples are ranked according their top probability values, and the top Ksel = 1>:1 S(p).) samples from modality m are selected into the pseudo-label set S. As training progresses, the model becomes more confident, quality scores generally increase, and the number of selected samples dynamically grows, allowing the model to learn from more diverse data. This allows the model to learn from simple and reliable structures first, avoiding premature overconfident judgments on difficult samples without negatively impacting the cluster head's performance. See more discussion in Appendix A.2

---

Published as a conference paper at ICLR 2026
3.2.3 CLUSTER HEAD
After screening for high-quality pseudo-labels, since the gradient backpropagation from the cluster head is not cut off, we use it to optimize our cluster head and the bottleneck. This allows the calibration signal be propagated back to the bottleneck, implicitly constructing high-que nigh-quality target variables for IB in an unsupervised setting. The cluster head is optimized as:
LeluH
1
Σ ES
Yi log(p;"")
(10)
[S|
Here, Vs.m is the pseudo-label generated by the m-th calibration head for sample 24, and pf is the predicted probability distribution from the cluster head for that sample. By leveraging the pseudo- label set in [Equation I0, the calibration signal is delivered to the cluster head. This not only allows the cluster head perceive modality-speciff features but also mitigates the issue of model overcon- fidence. Concurrently, the calibration signal is transmitted back to IB through the cluster head.
Since the model's core task is multi-modal clustering, and the final result is given by the cluster head which operates on the fused features, the cluster head's objective aligns well with the IB opti- mization. Therefore allowing gradient backpropagation from the cluster head means that the update signal for the backbones primarily comes from the effectiveness of the fused features. This incen- tivizes the backbones to learn representations that are "most beneficial for fusion" rather than just optimal for single modality.
Theorem 1 (Calibration of biased MI estimation) The pseudo-label screening mechanism allows Equation 10 to selectively learn more less from the pseudo-labels of given modality. This enables our model to autonomously rectify the adverse effects caused by modalities with biased MI estimation. See Appendixx A.3
In addition to the aforementioned methods, we also utilize the Kullhack-Leibler divergence to en- force consistency among different heads. This regularization term encourages the representations of all views to align within unified semantic space, enhancing the robustness and discriminative power of the fused representation.
Leon
DKL(P.ol Il p
(11)
Here, p.dd represents the clustering result from the m-th view. When different modalities produce conflicting clustering results for the same sample, the consistency loss forces the model to output a flatter probability distribution, thereby honestly expressing its uncertainty, effectively lowering ECE and making our framework trustworthy.
3.3 OPTIMIZATION
To minimize the first term of Lc. we estimate MI using a variational inference-based approach. As established in Theorem 2 (See Appendix A.4), this yields tractable upper bound: T T(X:; H') < Minimizing this bound serves minimize the MI. To maxi-
1 EM Eg, {D (pxxpp(("))) (A")gl=))) mize the second term, the computation of MI is thus transformed into the optimization of its tractable lower bound in Theorem 3 (See Appendix A.5): I(H ((H 2 log(N) LNT-Xont (II'' H³). This enables us to maximize the lower bound of MI by minimizing the NT-Xent loss. For the optimiza- tion of Lp. we use neural network estimator (Belghazi al. 2018) approximate I I( H*) because Hm and HJ are both high-dimensional variables. This avoids the explicit estimation of probability density for high-dimensional data, which is computationally intractable.
Mr
Since Lea is directly computable from the equations above, the details of its optimization procedure are omitted here for brevity.
3.4 DISCUSSION WITH ОTHE METHODS
While CLIB shares methodological similarities with SDCIB et al. 2025b), MSDIB et al. 2025), and DDMC et al., 2025) in their utilization of MI estimation techniques such as vari- ational inference neural estimators diverges fundamentally in its underlying philosophy and

---

Published as a conference paper at ICLR 2026
operational mechanisms. Existing works typically operate under the assumption that these tech- niques can yield precise MI estimates. However, in the context of high-dimensional and complex data, MI estimation is inherently susceptible to unavoidable estimation bias, thereby compromising the quality of the features extracted by the IB framework. The distinctive advantage of CLIB lies not in the pursuit or assumption of "flawless" estimator, but in its innovative calibration mechanism. By leveraging complementary cross-modal information, CLIB rectifies the adverse impacts induced by estimation bias, effectively breaking the vicious cycle of "error accumulation" often triggered by noisy pseudo-labels in existing frameworks. It is worth noting that while CLIB is designed to handle biased estimation, the IB framework remains incapable of extracting meaningful features the estimator fails entirely from the outset. Consequently, we are still motivated to employ the most suitable existing estimation techniques for specific targets to provide the best possible initial feature representation. Furthermore, unlike aforementioned approaches that focus exclusively on clustering accuracy, CLIB is the first to explicitly incorporate ECE into the optimization objective. This in- tegration not only enhances clustering robustness but also effectively mitigates the issue of model overconfidence, providing more reliable decision support for safety-critical applications.
4 EXPERIMENT
4.1 EXPERIMENTAL SETUP
Datasets. We conducted experiments on five widely used benchmark datasets, and a Multi-Layer Perceptron (MLP) was adopted as the backbone for all datasets. Caltech-2V (Fei-Fei et al. 2004):
Contains 1,4 1,440 images in 7 classes. Each sample is represented by two modalities: Wavelet moments (Shen & Ip. 1999) and CENTRIST features (Wu Rehg 201T). Caltech-3V: Identical to Caltech- 2V in images and classes, but incorporates a third modality, LBP features (Ojala et al. 2002). ESP- Game (von Ahn & Dabbish 2005): Comprises 11,032 images across classes, utilizing image features and corresponding textual descriptions two modalities. IAPR (Grubinger et al. 2006): A collection of images and semantic descriptions. Our study uses a filtered subset of 7,855 images from 6 categories, each with at least four labels. The two modalities are image features and text. MIRFlickr (Huiskes & Lew 2008): A denoised dataset of 12, images in 6 categories, which also employs image features and textual descriptions as its two modalities.
Implementation Details. The experiment was conducted with batch size of 64 using the Py PoToch 2.4.1 platform (Python 3.8) on Windows 10 system equipped with a 24GB NVIDIA RTX-4090D GPU. The cluster head is implemented as an MLP(512d (d-foddBB([offee & Szegedy
& Hinton -Cd), and calibration heads are designed with an identical architecture. We select traditional Single/All modal clustering methods and 11 latest multi- modal clustering methods for comparison to demonstrate the superiority of our method. The classical clustering algorithms are K- K-Means(KMM), Normalized Cuts(Neuts) and all-view version of them. We run the single-modal method on each modality and report the best clustering results. The rest of selected methods are EAMC (Zhou & Shen 2020), SiMVC & CoMVC (Trosten et al. (2021), MFLVC (Xu et al. 2022).
DSMVC Tang & Liu 2022), DealMVC (Yang et al. 2023), ), ICMVC (Chao et al. 2024), DIVIDE
(Lu et al. 2024b). MVCAN
(Xu et al. 2024), ROLL (Sun et al. 2025 COPER (Eisenberg al.
2025).
To ensure that the calibration process is not corrupted by excessive noise, we employ two-stage training strategy. The first stage is a 100-epoch warm-up period dedicated to stabilizing the feature extraction of IB. Only after the IB can reliably extract features do proceed to the second stage, 100-epoch calibration period. We ran the model 20 times, selecting the highest accuracy at the lowest loss prevent local maxima. For a more comprehensive analysis, the clustering performance was evaluated using two popular metrics: Clustering Accuracy (ACC) (Li & Ding 2006) and Normalized Mutual Information (NMI) (Strehl & Ghosh] (0022). Higher values of these metrics indicate better clustering performance. The trustworthiness is evaluated using ECE (Zhu et al. (022). Lower ECE values indicate lower over-confidence and therefore suggest better trustworthiness.
4.2 EXPERIMENTAL RESULTS
Results Analysis. As shown in Table I the proposed CLIB model demonstrates comprehensive superiority, achieving top performance three evaluation metrics across all five datasets. Re-
8

---

Published as a conference paper at ICLR 2026
Table 1:
Clustering performance ACC, NMI (%) and calibration ECE (%) five multi-modal
datasets (The bold denotes the best while underline the second best).
Caltech-2V
Caltech-3V
ESP-Came
MIRFlickr
IAPR
ACC
NMI
BCB
ACC
NMI
ECE
ACC
NMI
BCE
ACC
NMI
BCE
ACC
NMI
BCB
41.6
30.5
N/A
46.3
31.3
48.4
33.5
Ncuts
39.9
31.2
N/A
42.6
25.4
N/A
46.5
29.9
NIA
48.4
26.1
NIA
41.9
18.9
N/A
46.4
31.4
31.5
ALLNeuts
42.8
5.2
N/A
43.7
25.5
N/A
33.6
18.9
NIA
48.2
26.2
N/A
42.2
18.9
NIA
EAMC (C)PR'20)
40.3
26.6
35.7
38.9
21.4
SIMVC (C)
51.1
36.9
30.2
56.9
50.4
21.3
35.3
16.2
25.6
45.6
26.3
35.4
42.7
18.5
47.9
CoMVC (CVPR'21)
59.2
49.2
18.7
54.1
50.4
25.6
51.8
38.2
32.5
49.
30.6
43.2
46.7
21.5
36.8
DSMVC(CVPR'22)
57.9
49.8
31.2
63.7
54.2
26.4
32.4
27.5
31.4
48.4
29.5
31.8
38.9
15.3
24.6
MELVC (CVPR'22)
61.5
53.6
30,3
63.1
56.6
34.2
52.1
40.1
21.7
53.8
32.8
19.5
47.3
22.6
23.6
DealVCC (ACM MM 23)
47.6
37.9
19.1
59.2
56,2
27.6
42.7
24.7
49.3
32.1
21.9
25.8
ICMVC (4)AT'24)
49.6
37.9
28.5
64.7
53.7
39.4
45.8
29.5
25.2
43.5
24.4
20.6
37.1
16.8
43.8
(AAAI'24)
N/A
71.6
58.5
MVCAN (CVPR 24)
46.5
37.5
23.5
70.1
62.6
24.1
50.2
35.5
33.5
50.5
31.5
26.3
34.4
17.2
22.4
ROLL (CVPR'25)
45.7
35.8
19.7
61.8
49.6
16.9
41.5
23.6
25.7
46.2
27.3
24.6
17.4
23.1
COPER (ICLR22))
61.5
55.1
40.1
63.9
56.7
32.4
50.7
32.2
29.6
46.4
34.1
31.5
47.2
25.5
28.4
CL.TB (Ours)
68.6
60.5
16.0
77.8
69.3
10.9
56.3
39.9
12.1
55.4
39.5
10.5
51.6
29.1
7.8
garding clustering performance, CLIB substantially surpasses previous approaches. For example, achieves an ACC of 77.8% on the Caltech-3V dataset, outperforming DIVIDE by 6.2%, and an ACC of 56.3% on the ESP-Game dataset, exceeding MFLVC by 4.2%. Similarly, its NMI score of 69.3% on Caltech-3V 3V is significantly higher than the second-best method, MVCAN (62.6%). CLIB demonstrates exceptional performance on the ECE metric, showcasing its strong model calibration capabilities and trustworthiness.
ESP-Game and MIRFlickr, CL.B reduced ECE by more than half compared the previous best results. Furthermore, on IAPR, its ECE of 7.8% represents nearly threefold reduction in error compared to the runner-up, MVCAN (22.4 22.4%) Note. ECE metric is marked "N/A" for hard-clustering methods like KM and DIVIDE, as ECE is not meaningful for their frameworks.
Parameter Sensitivity Analysis. We investigated the parameter sensitivity of our method through grid search, where hyperparameters a and were varied within the (0.1) interval 0.1 increments. The experimental results, presented in Fngiie reveal that method maintains robust performance across all datasets, with no significant degradation under a wide range of parameter configurations. Specifically, the ACC variance remained within 20% across all datasets, with an average peak-to- trough drop of 17.87%. This observation underscores the method's 's insensitivity to parameter choices and highlights its reliable and stable nature.
Convergence Analysis. Figure 3 illustrates the convergence curves of the overall loss function, ACC, and NMI metrics across all 5 selected datasets. The model's performance improved signifi- cantly after the introduction of the calibration module at the 101st epoch. All three metrics stabilized within 40 epochs after this introduction (i.e., at epoch 140), which indicates that our model can con- verge both quickly and stably.
4.3 ABLATION STUDY
To validate the effectiveness of each component and configuration of our framework, we conducted additional ablation experiments. The results in Table 2 indicate that both Lre and Lealit enhance clustering performance. LealiH contributes to clustering by filtering high-quality pseudo-labels, which simultancously increases ACC and reduces ECE. In contrast, while Lcon effectively reduces
Figure 2: Parameter analysis on Caltech-2V, Caltech-3V, ESP-Game, MIRFlickr and IAPR dataset.

---

Published as a conference paper at ICLR 2026
Caltech-yy
Game
MiRFiikk
JAPR
1000
ACC 1MR
LOSS
Epoch
Epoch
Epech
Epoch
Epoch
Figure 3: Training process of CLIB on multiple datasets.
Table 2:
The ablation results of the proposed model.
Caltech. 2V
Caltech. 3V
ESP
Game
MIRFlickr NM1
IAPR
Settings
ACC
NMI
ECE
ACC
NMI
ECE
ACC
NMI
BCE
ACC
BCE
ACC
NM1
ECE
CiB + RelaH
57.7
43.1
36.2
60.6
52.5
25.5
43.8
31.2
21.3
51.0
34.1
25.6
39.1
18.9
35.4
Colat E.o
61.2
50.1
20.0
64.6
57.4
19.5
45.6
35.9
22.6
51.8
35.4
23.2
42.3
18.2
27.4
CiB + RelaH + Rostifl
61.7
53.5
19.2
67.8
59.3
11.6
46.1
35.3
21.5
53.8
36.2
18.5
19.9
20.3
18.9
Colutt Loom
59.7
47.8
19.8
61.1
56.7
12.4
44.2
31.0
16.3
51.5
35.7
16.1
37.8
19.1
15.0
I. No warm-up for IB
59.6
57.1
19.4
61.7
58.4
15.1
48.6
34.6
21.9
52.8
38.8
27.8
40.1
22.4
32.9
Cali heads backprop
59.9
59.3
23.4
65.9
58.8
13.5
48.5
33.4
24.3
50.4
36.2
31.3
45.4
24.4
24.3
CLIB
68.6
60.5
16.0
77.8
69.3
10.9
56.3
39.9
12.1
55.4
39.5
10.5
51.6
29.1
7.8
ECE by resolving conflicting views, it yields limited clustering gains. The slight ACC drop on IAPR (39.1% to 37.8%) stems from the constraint preventing the model from making "bold guesses" on ambiguous samples. In safety-critical scenarios where uncertainty is paramount, we suggest increasing the weight of Leon enhance trustworthiness.. Conversely, for applications prioritizing clustering accuracy over calibration, we advise decreasing the weight of Lcon The results of the ablation study validate the efficacy of the configurations within our framework, confirming that each component positively contributes to the overall performance of our model.
I.Remove the warm-up stage for IB. As shown in Table 2
removing the warm-up stage and ap- plying callibration from the outset introduces substantial noise into the training process. This has two adverse effects: it reduces the quantity selected pseudo-labels and increases the likelihood of these labels being noisy. Consequently, the model experiences performance degradation but avoids a complete collapse. This result affirms the necessity of the warm-up stage while simultaneously illustrating the efficacy of the pseudo-label screening mechanism in filtering noise.
Backkpoopggtting the gradient from calibration heads. Since one calibration head assigned to each modality, this approach may allow modality-specific redundant information to enter the calibration signal. This information then passed back to via gradient backpropagation, which can impair the bottleneck's 's ability to learn high-quality features. This ultimately leads to decline in clustering performance.
5 CONCLUSION
This paper introduces CLIB, a novel IB-based multi-modal clustering method. It utilizes a dy- namic screening mechanism to learn from high-quality pseudo-labels, which in turn calibrates IB to enhance feature extraction. Supported by theoretical proof and experiments, CLIB achieves compet- itive clustering performance with lower model overconfidence. Our approach also circumvents the adverse effects of MI estimation bias in high-dimensional data, creating new application possibili- ties for IB in multi-modal clustering. However, CLIB is currently limited to complete, single-label multi-modal datasets and cannot address incomplete or multi-label data scenarios. Moreover, like many clustering methods shown Lu et al. (2024a), the reliance on a predefined number of clusters restricts its flexibility. Future research will focus on developing adaptive methods to determine the number of cluster automatically and extending the calibration mechanism to handle missing modal- ities and multi-label correlations.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (project no. 62576320, 62572444, 62325602), Henan Province Outstanding Youth Science Fund Program (project no. 252300421223).
ETHICS STATEMENT
This paper does not involve any potential ethics issues.
REPRODUCIBILITY STATEMENT
We have exerted substantial efforts guarantee the reproducibility of research. The main manuscript offers explicit descriptions of the proposed framework, algorithmic modules, and evalu- ation methodologies. Complete proofs of theoretical findings and detailed derivations of key propo- sitions are included in the appendix. For the experimental section, we utilized public datasets and provided thorough accounts the experimental setups. further facilitate reproducibility, the full source code and implementation specifics will be made publicly available upon the paper's accep- tance.
REFERENCES
Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, and Boqing Gong- VATT: transformers for multimodal supe-supe learning from raw video, audio and text. In Advances in Neural Information Processing Systems, pp. 84206-24221 2021.
Alexander A. Alemi, lan Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In 5th International Conference on Learning Representations. OpenReview.net, 2017.
David Barber and Felix Agakov. The im algorithm: variational approach to information maxi- mization. Advances in neural information processing systems, 16:201, 2004.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, R. Devon Hjelm, and Aaron C. Courville. Mutual information neural estimation. In Proceedings of the 35th International Conference on Machine Learning, volume 80, 530-539, PMLR, 2018.
Guoqing Chao, Yi Jiang, and Dianhui Chu. Incomplete contrastive multi-vier clustering with high- confidence guiding. In Thirty-Eighth AAAI Conference on Artificial Intelligence, pp. 11221- 11229. AAAI Press, 2024.
Pawel Czyz, Frederic Grabowski, Julia E. Vogt, Niko Beerenwinkel, and Alexander Marx. Beyond normal: On the evaluation of mutual information estimators. In Advances in Neural Information Processing Systems, 2023.
Ran Eisenberg, Jonathan Svirsky, and Ofir Lindenbaum. COPER: correlation-based permutations for multi-view clustering. In The Thirteenth International Conference on Learning Reprexenta- lions. OpenReview.net, 2025.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 178. IEEE Computer Society, 2004.
Di Feng, Lars Rosenbaum, Claudius Gläser, Fabian Timm, and Klaus Dietmayer. Can we trust you? calibration of probabilistic object detector for autonomous driving. CoRR, 12358,
2019.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor S. Lempitsky. Nomain-adversann training of neural networks. J. Mach. Learn. Res., 17:59:1-55 1-59:55, 2016.

---

Published as a conference paper at ICLR 2026
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Mare Proesmans, and Luc Van Gool. SCAN: learning to classify images without labels. In Proceedings of the 16th European Conference Computer Vision, volume 12355, pp. 268-285 Springer, 2020.
Michael Grubinger, Paul Clough, Henning Müller, and Thomas Deselaers. The iapr tc- 12 bench- mark: A new evaluation resource for visual information systems. In International workshop ontolmage, volume 2, 2006.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference Machine Learming, volume 70, pp. 1321-1330. PMLR, 2017.
Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view classifica- tion. In 9th International Conference on Learning Representations. OpenReview.net, 2021.
W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. 1970.
Shizhe Hu, Zhengzheng Lou, Xiaoqiang Yan, and Yangdong Ye. A survey on information bottle- neck. IEEE Trans. Pattern Anal. Mach. Intell., 46(8))6325-5555 5344, 2024.
Shizhe Hu, Jiahao Fan, Guoliang Zou, and Yangdong Ye. Multi-aspect self seff-guiee deep informa- tion bottleneck for multi-modal clustering. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, pp. 17314-17322. AAAI Press, 2025.
Zongmo Huang, Yazhou Ren, Xiaorong Pu, Shudong Huang, Zenglin Xu, and Lifang He. Self- supervised graph attention networks for deep weighted multi-view clustering. In Thirty-Seventh AAAI Conference Artificial Intelligence, pp. 7936-7943. AAAI Press, 2023.
Mark J. Huiskes and Michael S. Lew. The MIR flickr retrieval evaluation. In Proceedings of the 1st ACM SIGMM International Conference on Multimedia Information Retrieval, pp. 39-43, ACM, 2008.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference Ma- chine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 448-456. JMLR.org, 2015.
Yuheng Jia, Jianhong Cheng, Hui Liu, and Junhui Hou. Towards calibrated deep clustering network. In The Thirteenth International Conference on Learning Representations. OpenReview.net, 2025.
Tao Li and Chris H. Q. Ding. The relationships among various nonnegative matrix factorization methods for clustering. In Proceedings of the 6th IEFE International Conference on Data Mining, pp. 362-371. IEEE Computer Society, 2006.
Yunfan Li. Peng Hu, Jerry Zitao Liu, Dezhong Peng. Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. In Thirty-Fifih AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Sympo- sium on Educational Advances Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 8547-8555. AAAI Press, 2021.
Yunfan Li, Mouxing Yang, Dezhong Peng, Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. Int. J. Comput. Vis., (0009))220-22211 2022.
Zhengzheng Lou, Chaoyang Zhang, Hang Xue, Yangdong Ye, Qinglei Zhou, and Shizhe Hu. Self- supervised weighted information bottleneck for multi-view clustering. In Proceedings of the Thirty-Thire International Joint Conference on Artificial Intelligence, pp. 2024.
ijcai.org,
Zhengzheng Lou, Hang Xue, Chaoyang Zhang, and Shizhe Hu. A peer-review look multi- modal clustering: An information bottleneck realization method. In Forty-second International Conference on Machine Learning, 2025a.
12

---

Published as a conference paper at ICLR 2026
Zhengzheng Lou, Ke Zhang, Yucong Wu, and Shizhe Hu. Super deep contrastive information bottle- neck for multi-modal clustering. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025. OpenReview.net, 2025b.
Yiding Lu, Haobin Li, Yunfan Li, Yijie Lin, and Xi Peng. A survey deep clustering: from the prior perspective. Vicinagearth, 1(1):4, 2024a.
Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, and Xi Peng. Decoupled contrastive multi-view clustering with high-order random walks. In Thirty-Eighth AAAI Conference on Arti- ficial Intelligence, pp. 14193-14201. AAAI Press, 2024b.
Yiqiao Mao, Xiaoqiang Yan, Qiang Guo, and Yangdong Ye. Deep mutual information maximin for cross-modal clustering.
Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 8893-8901. AAAI Press, 2021.
Takahiro Mimori, Keiko Sasada, Hirotaka Matsui, and Issei Sato. Diagnostic uncertainty calibration: Towards reliable machine predictions in medical domain. In The 24th International Conference Artificial Intelligence and Statistics, volume I 130, pp. 3664-3672. PMLR, 2021.
Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, pp. 4696-4705, 2019.
Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, pp. 807-814. Omnipress, 2010.
Timo Ojala, Matti Pietikäinen, and Topi Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. JEEE Trans. Pattern Anal. Mach. Intell., 24: 971-987, 2002.
Zexuan Qiu, Qinliang Su, Zijing Ou, Jianxing Yu, and Changyou Chen. Unsupervised hashing with contrastive information bottleneck. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pp. 959-965. ijcai.org, 2021.
Dinggang Shen and Horace Ho- Shing lp. Discriminative wavelet shape descriptors for recognition of 2-d patterns. Puttern Recognit., 32:151-165, 1999.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Do- gus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems, 2020.
Jiaming Song and Stefano Ermon. Understanding the limitations of variational mutual information estimators. In 8th International Conference Learning Representations. OpenReview.net, 2020.
Alexander Strehl and Joydeep Ghosh. Cluster ensembles A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res., 3:583-617, 2002.
Yuan Sun, Yongxiang Li, Zhenwen Ren, Guiduo Duan, Dezhong Peng, and Peng Hu. ROLL: robust noisy pscudo-label learning for multi-view clustering with noisy correspondence. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 30732-30741. Computer Vision Foundation IEEE, 2025.
Huayi Tang and Yong Liu. Deep safe multi-view clustering: Reducing the risk of clustering perfor- mance degradation caused by view increase. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 202-211. IEEE, 2022.
Naftali Tishby, Fernando C. N. Percira, and William Bialck. The information bottleneck method. CoRR, physics/0004055, 2000.
Daniel J. Trosten, Sigurd Løkse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering rep- resentation alignment for multi-view clustering. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 1255-1265. Computer Vision Foundation IEEE, 2021.
13

---

Published as a conference paper at ICLR 2026
Aäron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. CoRR, 007.03748, 2018.
Luis von Ahn and Laura Dabbish. ESP: labeling images with a computer game. In Knowledge Collection from Volunteer Contributors, pp. 91-98. AAAI, 2005.
Yanzheng Wang, Xin Yang, Yujun Wang. Shizhe Hu, and Mingliang Xu. Diversity-oriented deep multi-modal clustering. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Jianxin Wu and James M. Rehg. CENTRIST: A visual descriptor for seene categorization. JEEE Trans. Pattern Anal. Mach. Intell., 33:1489-11555 2011.
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. Multi-level feature learning for contrastive multi-view clustering, In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16030-16039, IEEE, 2022.
Jie Xu, Yazhou Ren, Xiaolong Wang, Lei Feng, Zheng Zhang, Gang Niu, and Xiaofeng Zhu. Inves- tigating and mitigating the side effects of noisy views for self-supervised clustering algorithms in practical multi-viens scenarios. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22957-22966. IEEE, 2024.
Wenbiao Yan, Yiyang Zhou, Yifei Wang, Qinghai Zheng, and Jihua Zhu. Multi-view semantic consistency based information bottleneck for clustering. Knowl. Based Syst., 288:1 11448, 2024.
Wenqi Yang. Minhui Wang, Chang Tang, Xiao Zheng, Xinwang Liu, and Kunlun He. Trustworthy multi-view clustering via alternating generative adversarial representation learning and fusion. Inf. Fusion, 107:102323, 2024.
Xihong Yang, Jiaqi Jin, Siwei Wang, Ke Liang, Yue Liu, Yi Wen, Suyuan Liu, Sihang Zhou, Xin- wang Liu, and En Zhu. Dealmve: Dual contrastive calibration for multi-view clustering. In Proceedings of the 31st ACM International Conference Multimedia, pp. 337-346. ACM, 2023.
Runwu Zhou and Yi-Dong Shen. End-to-end adversarial-attention network for multi-modal cluster- ing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14607-14616. Computer Vision Foundation IEEE, 2020.
Fei Zhu, Zhen Cheng, Yao Zhang, and Cheng-Lin Liu. Rethinking confidence calibration for failure prediction. In Proceedings of the 17th European Conference on Computer Vision, volume 13685, pp. 518-536. Springer, 2022.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
In the supplemental material:
Appendix A.
Statement on the use of large language models.
Appendix A
Discussion about pseudo-label screening mechanism.
Appendix A.
Proof of theorem 1.
Appendix A 4
Proof of theorem 2.
Appendix A.5
Proof of theorem 3.
Appendix A.6
T-SNE visualization analysis.
Appendix A.
Analysis of modality weights.
Appendix A.8
Study on difficult samples.
Appendix A.9
Complexity analysis.
A.1 STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
In accordance with academic integrity guidelines, we hereby declare that no Large Language Models (LLMs) were used in the conceptualization, experimentation, analysis, writing of this subsection and the related parts of this paper. All content was produced by the authors independently.
A.2 DISCUSSION ABOUT PSEUDO-LABEL SCREENING MECHANISM
As discussed Section 2.2 pscudo-labels generated using fixed thresholds often suffer from low quality and are riddled with noise. To overcome this limitation, researchers began exploring more dynamic pseudo-label refinement strategies. This is evident the evolution from initial works like CC(Li et al. 2021), which used data augmentation for supervision, to its successor, TCL(Li al.) 2022]. TCL introduced a "confidence-basee
boosting" strategy specifically to dynamically filter reliable pseudo-labels from noisy, self-generated signals, demonstrating that handling unreliable supervision is a critical issue in deep clustering.
For Table CLIB-S, stands for the version of CLIB applying
H(P)
S,(P)
(12)
72 X Hmax
for quality score function. CLIB-S, applies the original information redundancy theory and CLIB- S2 is the proposed version. The results indicate that CLIB-S, is the best- best-performing version. As the changes become progressively smaller, the performance of CLIB-S. and subsequent versions (n 2 4) began to converge; therefore, they were excluded from further experimentation and presentation.
Using the original information redundancy theory poses a problem: in extreme case, all output vectors from a calibration head are perfectly uniform, then no pseudo-labels from this modality will be selected guide the cluster head's learning, resulting Cluster Head. When a modality is completely unable to distinguish between clusters, but the final output fails to perceive this confusion, the model is highly likely to make overconfident predictions, thereby affecting the clustering result. In contrast, the proposed screening method ensures that at least half of the samples are selected into the pseudo-label set. is less about selecting high-qualit, samples but more about excluding the lowest-quality ones.
very sparse training signal for the
Jia et 2025) and Li al. (2022) adopted pseudo-label selection method based on maximum probability. However, by focusing only the maximum probability, this approach tends to overlook other information provided by the cluster probability vector. As shown in Figure 4| the vector for cluster 2 provides information not only about the most likely cluster but also about the least likely ones. Therefore, should achieve a higher score from quality score function than the vector for cluster I from S(-). If apply the selection by the maximum probability, they will recieve the same score. We also experimented with pseudo-label screening mechanism based on maximum probability, which did not perform well our proposed method.
15

---

Published as a conference paper at ICLR 2026
Table 3:
The results of different screening strategies
Caltoch-2V
Caltech-3V
ESP-Game
MIRFlickr
IAPR
Screening stratogies
ACC
ACC
ACC
ACC
ACC
Select by max probability
60.7
54.6
23.3
70.6
61.0
11.9
49.6
35.9
27.0
53.9
39
35.7
42.8
26.1
21.7
CLIB--S
26.1
CLIBS-; (Proposed)
68.6
60.5
16.0
77.8
69.3
10.9
56.3
39.9
12.1
55.4
39.5
10.5
51.6
29.1
7.8
68.1
6
10
S-0.
478
0
0
5=0 6303
0.2
0.2
Figure 4: Different quality score functions for cluster and cluster respectively,
A.3 PROOF OF THEOREM 1
This proof demonstrates that the proposed calibration mechanism effectively mitigates the adverse effects of estimation biases within IB.
Let cm represent the bias in the mutual information estimation for a given modality m. The estimated mutual information, [H ), is related to the true mutual information, I(H"; H)), by:
i( H IHH"; H') + Em
A large estimation bias, indicated by large |cml. leads IB to produce a less discriminative feature representation H This reduction in discriminative power results in the calibration head generating more uniform, "flatter," probability distribution pou for the samples of that modality. A flatter distribution is characterized by higher entropy.
Consider two modalities, mi and mg. with biases Em, and Emg. respectively. modality m, has negligible bias (em, + 0) and modality m2 has a significant bias (/Emal > of their respective calibrated probability distributions will satisfy:
0), the resulting entropy
((mm)
(p
The sample quality score, S(P), is designed to be negatively correlated with the entropy H(P), as shown by its derivative:
aS(P)
1
(H(P) 2Hmaz
The number of high- samples selected for modality m, denoted Kese calculated K E (p(pp)) } This establishes direct link between estimation bias and the number of selected samples. The logical chain is as follows:
H(pm)) > H(mm!) = S(pra)) < S((ro))) -
Resel < ma
Kese mi
In summary, modality with larger estimation bias will have fewer samples selected for the sub- sequent training step.
We now analyze how this sample selection mechanism influences the parameter updates of IB, denoted by 0. The objective is to minimize the cluster head loss, ColuH, which is computed using the union of selected samples from all modalities. The total gradient the loss with respect the IB parameters 0 is the sum of contributions from each modality:
M
V VoLetuH
Σ Gm
m==
where Gm is the gradient contribution originating from the set of samples S, selected for modality m, with |Sml Kset, From Equation 10 this contribution is defined as the gradient of the loss computed exclusively on these selected samples:
Gm Vol(m) Vo
Σ S.
-i log (pp(t))
16

---

Published as a conference paper at ICLR 2026
By linearity of the gradient operator, this can be expressed as:
Gm Σ Vo (o l)
The magnitude of this gradient contribution, |mG]l is determined by the vector sum of the indi- vidual sample gradients. We assume that for given modality, the gradient vectors for each sample are directionally coherent, as they all contribute to optimizing the same feature extraction param- eters 0. Under this reasonable assumption, the magnitude of the summed vector approximately proportional to the number of terms in the summation:
||Gmll
Σ
Ve (- log(o) |Smll
Combining the findings from both sections, we see that for our two modalities my (low bias) and m2 (high bias), have Kael Keel This directly implies that the magnitude of their respective gradient contributions will satisfy:
|[ ll [|Gmall
During optimization, the total gradient VoLetut = Gi G2 +G is vector sum. In this sum, the gradient vector G1, originating from the more reliable modality with smaller estimation bias, possesses a significantly greater magnitude. Consequently, the direction of the total gradient update will be predominantly influenced by Gr. This ensures that IB's parameters are primarily updated based on information from more reliable modalities, effectively mitigating the negative impact of modalities corrupted by high estimation bias.
A.4 PROOF OF THEOREM 2
In our model, key optimization objective is to minimize the mutual information T(X:; H') be- tween the imput data X' and its compact representation Ht. However, the direct computation of this mutual information is intractable as it involves the posterior probability p(z"(h'), which is dif- ficult to handle. Similar to Hu et al. (2025), we employ a variational inference approach to derive a trainable approximate upper bound, and by lowering it, we could minimize I(X; H').
First, the mutual information [(x;; H is defined as:
p(a' (h') 1(x;; [ p(a", h²) log p(x²)
Since the posterior probability (t') is unknown, we introduce a parameterized variational distri- bution q(x') approximate the true marginal probability p(z"). To ensure the effectiveness of this approximation, we utilize the Lulbbeck-Leibler (KL) divergence to constrain the distance between q(x") and p(x"). Based on the non-negatyvyty of KL divergence:
p(x')
(x (xx(p(=")llaaa')) = p(x*) log
2 0.
g(x²)
Through a simple transformation, we can obtain:
f p(x²) log p(a")))) - ff p(x²) log g(xt) 2
p(x')= 2 q(x²).
This implies that substituting p(x*) with g(xx) provides an upper bound for the original mutual information expression:
T(X;; HT') = 1 p(at, Iet log p(x*)
pla" ht))
S < /Dlat, plat, (laa,n")ien 1o)log p(a'(h") ((xx)
/
Next, by leveraging the property of the joint probability distribution, p(a", ,nt) = can rewrite the inequality as:
we
1(x;; < / )p( log ((x))
p(x h²)
17

---

Published as a conference paper at ICLR 2026
Table 4: Variable definitions of Theorem 3
Variable
Definition
Hus Ho N
A batch of feature representations from modalities and v.
The batch size.
hg e H
The feature vector of the i-th sample from modality
h E Hy
The feature vector the i-th sample from modality v.
A "positive pair," sampled from the joint distribution p(h", h").
fim(h1,hu h22)
A similarity metric for feature vectors, typically cosine similarity: h.'h2
sim(h1, h2)) [A; |1llb]
T
A temperature hyperparameter.
To eliminate extraneous elements, we use Monte Carlo sampling (Hastings) 1970) to approximate the integral over h', which yields: 1(';; & p(a)))))) log p(a"Ih") g(xr)
We assume that p(=" follows a Gaussian distribution, with its mean and variance learned by the sharing specific encoder network (Alemi et al. 2017: Mao al. 2021). Here we use the reparame- terization trick, expressing h as:
ht p(x") (x+) where ~ I).
The upper bound of the mutual information can thus be expressed as an expectation with respect to 8, and is ultimately simplified to the expectation of KL divergence:
p(x*(h²) ((x²)
I(x;; S Eg, log
- Eg,
To prevent the model from producing a trivial solution by assigning all samples to few clusters, we impose a constraint on ((z;) to enforce a uniform distribution over the classes: Σ; g(x;) = [eP
where N is the total number of samples and |CI is the number of clusters. Finally, by averaging over
all samples in the dataset, we obtain the final optimizable objective function:
N
1(x;;
Es,
A.5 PROOF OF THEOREM 3
The core idea of this proof is that the NT-Xent loss function is, in fact, an estimator for a lower bound on the mutual information between the representations of two modalities. Consequently, by minimizing the NT-Xent loss, we are effectively maximizing this lower bound, which serves as proxy for estimating the mutual information itself.
The standard definition of the mutual information can be expressed as the KL divergence between the joint distribution p(h", h²) and the product of the marginal distributions p(h"))(hh)) expectation form, this is:
TH,,, H.) = Eppee An) log p(h" h2)
p(h" p(h"[ha)
p(h" )p(h))) = Ep(lera)) llog
p(he))
Directly computing this MI is intractable, as it requires knowledge of the high-dimensional probabil- ity density functions p(h" h()) and p(h"). The Noise-Contrestive Estimation (InfoNCE) framework provides a way to estimate a lower bound on this quantity.
The idea of InfoNCE is to introduce scoring function (or critic), f(1", h²), designed to approximate the log-density ratio:
f(h", h²) Ro log p(h" hhm)
p(h()
18

---

Published as a conference paper at ICLR 2026
We can formalize this by stating that the density ratio is proportional to the exponentiated score function:
p(h"lh")
p(h") & exp(f(h",,'))
To convert this proportionality into an equation, we must normalize the expression. The normaliza- tion term is found by ensuring that the probability distribution integrates to one, which is achieved by taking the expectation over all possible h" drawn from the marginal distribution p(h"):
p(h"
exp(f h²))
p(h)) BRew (exp( (1h, [)))
Substituting this back into the definition of mutual information yields an expression based our critic f: I(Hus Hy) = (e [f h²) log (Ba"
The expectation term Bxp===((h)) [-] in the denominator is still intractable. We can, however, approx- imate using Monte Carlo method. We draw N independent and identically distributed samples N
{hy ( Jj=1 from the marginal distribution p(h"). ) For a sufficiently large N, the Law of Large Numbers states:
N
1
cxp(f(h" (x)) N
=
j=1
By substituting this Monte Carlo approximation, we introduce inequality. Because the logarithm log(-) is a concave function, Jensen's 's Inequality (olg X)] & log(E(X])) tells us that replacing the log of the true expectation with the log of the sample mean results lower bound. Therefore:
exp(f(A) 45)) ]
N
1HH,, 2
f(h h,) - log
Σ N
This expression is the InfoNCE lower bound mutual information.
The NT-Xent loss a specific instance of the InfoNCE objective. In this context, the scoring func- tion f(hi, h22 is defined as the temperature-scaled cosine similarity:
f(1,, h2) = sim(h1, h2)/T
The NT-Xent loss for batch is defined as:
N
log exp(sim (1 h/)/T)
LXT.Xem (Hu, Ho)
Σ
j=1 exp ,, h2)/T)
We can now show the direct relationship between the InfoNCE bound and this loss function. In a practical implementation, the expectation over positive pairs, E ay), is approximated by the empirical mean over the N pairs in the batch. Starting with the InfoNCE lower bound:
N
N
I I(H,, H.)
N
f(0 MT) loog
exp([(M),))) (0))))))
N
N
1
1
log(exp(f(1) (())),))))))))))))))))))))))(( log
Σ
exp(f(h),
N
N
N
exp(f(h,, hz))
Σ
log
N
N EE exp(f(h;, hz))
exp(S(h)) ( k())
log
+ log(N)
N
プラ= exp( (1hh)
exp(f(h) kP))
log
N Σ exp( (hi hz))
+
Σ log(N)
-NNTTXoo Ho) + log(N)
19

---

Published as a conference paper at ICLR 2026
Cattech: 2V Eooch 300
Coltech 3V M Esoch 100
ESP Sa--aame Epoch 100
HIS Flickr Epoch 100
IPR Epoch 100
Cattecb-2y Fooch 200
Eatecc-1y 2 Footh
Pspocamme Tpach 200
Milliee 20och
Teoch
Figure 5: T-sne visualization results multiple datasets.
This final expression, [HH,,, H.) 2 fixed batch size N, a smaller may result in a looser lower bound for the MI estimation. However, since log(N) is constant, its derivative with respect the model parameters is zero. Therefore, this term does not influence the direction of the gradient update in Equation 3] From the above, minimizing the NT-Xent loss is equivalent to maximizing variational lower bound on the mutual information HH,, H.)-
-or--xem( H.,), demonstrates the connection. For
A.6 T-SNE VISUALIZATION ANALYSIS
To further illustrate the impact of the calibration mechanism on the final clustering structure, Fig- ure 5 presennss t-SNE visualizations for all datasets at epoch 100 (prior to the introduction of calibra- tion) and epoch 200 (after calibration). can be observed that following 100 epochs of calibration, the clustering structures across all datasets become significantly more compact and well-separated. On the Caltech-3V dataset, where the model achieves its best performance, the clusters are clearly distinct; data samples within most clusters are densely distributed, while samples from different clusters maintain a substantial separation distance.
A.7 ANALYSIS OF MODALITY WEIGHTS
To investigate the dynamic adjustment capability of the CLIB framework during multi-modal fu- sion, we tracked the evolution of modality weights (wm) throughout the training process across five datasets. As illustrated in Figure the weight variations in all datasets exhibit a distinct two-stage characteristic, which aligns consistently with our designed training strategy. In the MIRFlickr, ESP- Game, and IAPR datasets, the weights diverged rapidly, with the model tending to assign excessively high weights to dominant modality. However, upon the introduction of the Calibration Mechanism at epoch 100, significant turning points appeared in the weight curves all datasets, verifying the effectiveness of the calibration signals. Following this shift, the weights stabilized quickly. This demonstrates that CLIB's s adaptive weighted-avernas fusion mechanism is capable of dynamically allocating weights based the actual discriminability of each modality.
Cattrb 2V
CaR-
H9 Spmn
1394
Figure 6: Evolution of modality fusion weights multiple datasets
20

---

Published as a conference paper at ICLR 2026
Table 5: Clustering performance difficult samples.
Caltech-2V NMI
Caltech-3V
ESP-Game NMI
MIRFlickr NMI
IAPR
ACC
ECE
ACC
NMI
ECB
ACC
ECB
ACC
ECE
ACC
NMI
ECE
At epoch 101
28.6
27.2
28.2
18.9
24.4
25.5
27.2
15.2
21.9
27.9
17.2
27.4
26.8
14.6
32,9
epoch
21.4
35.2
21.5
39.7
24.3
A.8 STUDY ON DIFFICULT SAMPLES
During the initial stage of calibration, samples with low quality score were excluded from the pseudo-label set to filter out noise. To investigate the final clustering performance of these sam- ples, we defined the bottom 20% of samples-ranker by their quality score at the 101st epoch-as "difficult samples." Their final performance is presented in Table We recorded the clustering per- formance at epochs 101 and 200. The results indicate that after 100 epochs of calibration, the per- formance of hard samples improved. This is attributed to the fact that, although hard samples were initially excluded from the pseudo-label set, the quality scores for all samples gradually increased as the model learned. Consequently, an increasing number of samples entered the pseudo-labe set, enabling the model eventually learn from these hard samples.
A.9 COMPLEXITY ANALYSIS
Let D denote the input dimension of the samples, LB and P3 represent the number of layers and the hidden dimension of the backbone network, respectively, and HD be the dimension of the features output by the backbone. The 1(X', term has complexity of Backkoone & O(N M (DPB + (Lg 1) + P)HD)). Concurrently, the I(H H)) term's 's total com- plexity is O(M2N2 HD). Finally, the I I(HT H') term, calculated via a neural network estimator, assuming cost of O(N per modality. Therefore, the total complexity for LIB is TIB = Backbone + O(M22HH)) O(NMPMINE).
incurs cost of D(NMPMIN
The computational cost of LCat determined by three primary operations. First, the M + heads, which map H--dimensional inputs to C' outputs via a Pa--dimensional hidden layer, introduce a (MM++))--M +
complexity of Teedd a preceding K-Means step, running KM iterations for C' clusters on N Hn-dimensiooall features, which incurs a cost of O(K Ar NCHD). Third, the pseudo-label screening for N-M samples requires approximately O(N MC). The remaining losses are computed in O(N or less. Thus, the total O(N
Second, the Leatih calculation requires
complexity for the Ccal module is Tcal & THeads
Let K worrmup be the number of warm-up epochs and Keat be the number of calibration epochs. The total time complexity Trotal K Kwarmup TIB B+ Keat (TıB + Teat).
21


Reviews:
[{"review_id": "iuFg2FaMei", "reviewer": "Reviewer_6kV7", "summary": "By seeing the challenging problems faced by existing information bottleneck-based multi-modal clustering methods, in this work a novel calibrated information bottleneck is proposed for trusted multi-modal clustering to learn more accurate and trustworthy clustering outcome. It mainly presents a parallel multi-head network architecture containing clustering and calibration heads for outputting high-quality data assignments. Lots of different kinds of experiments have illustrated the superiority of the method on multiple benchmark datasets with metrics of clustering accuracy and calibration error.", "questions": "1) The method requires the number of clusters, C, to be specified in advance. How sensitive is the model to the choice of C? If C is set incorrectly, how much are the model's performance and calibration affected?\n\n2) Could you provide the specific network architectures for the Backbones and the CalHead/CluHead? For instance, what kind of encoders were used for the image and text modalities, respectively?\n\n3) What are the limitations that the proposed method still exist in calibrating the multi-modal clustering results? Could you provide some future insights in this area for readers?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper is well-written with clear logic. The flow from problem introduction and method description to experimental analysis is fluent. Figure 1 provides a clear and understandable illustration of the whole framework, helping readers quickly grasp the model's workflow.\n\n2) The control over the gradient flow is very fine-grained, it blocks the gradient from the calibration heads but allows it to back-propagate from the main cluster head. This design ensures that the feature learning of IB serves the final clustering task while avoiding potentially contradictory signals from the calibration objectives of different modalities.\n\n3) The parameter sensitivity analysis shows that the model maintains stable performance over a wide range of choices for hyper-parameters. This indicates that the method does not require excessive tuning and possesses good practical value.", "weaknesses": "1)  Eq. 3 of the paper relies on the NT-Xent loss. The effectiveness of NT-Xent is highly dependent on the Batch Size (N), as its MI lower bound is log(N)-L_(NT-Xent). If N is too small, this bound becomes loose, leading to poor alignment. The paper does not mention the Batch Size used in the experiments or its impact.\n\n2) The paper does not detail the specific network architectures used for feature extraction and the various heads, where they are also important in reproducing the results for readers.\n\n3) The paper relies entirely on quantitative metrics. For a clustering task, providing t-SNE visualizations of the feature space would be highly persuasive. A comparison of the feature distributions after the warm-up and after calibration could visually demonstrate how the framework improves inter-class separation and intra-class compactness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wzkMwaodGo", "reviewer": "Reviewer_VCs7", "summary": "This paper introduces an information bottleneck-based multi-modal clustering method, which addresses the issue of overconfident clustering results caused by low-quality or unreliable pseudo-labels in multi-modal clustering. The parallel multi-head architecture is proposed to correct distortions in mutual information estimation. Experiments demonstrate that the proposed method achieves superior performance compared with existing baselines across multiple datasets.", "questions": "I expect the authors to address my concerns in the weakness section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work addresses the reliance on reliable target variables and the overconfidence caused by noisy pseudo-labels, which is a commonly encountered problem in clustering.\n2. The proposed dynamic pseudo-label screening strategy based on information redundancy offers a promising alternative to existing probability-based thresholding.\n3. The proposed framework takes the inherent difficulty of precise MI estimation into consideration, alleviating the negative impacts of such estimation biases.", "weaknesses": "1. The proposed method consists of M+1 heads, a two-stage training process, and multiple loss terms, which may increase computational overhead compared to baseline methods. The authors are encouraged to discuss the model’s computational complexity to better assess its practical applicability.\n2. In the current implementation, gradients are blocked from the calibration heads while allowing backpropagation from the cluster head to the IB to avoid contradictory objectives. It would be helpful if the authors clarify what specific contradictions were observed and whether the fusion-based cluster head’s objective is consistently better aligned with the IB optimization than the single-modality calibration heads.\n3. The pseudo-label screening excludes high-entropy, uncertain samples during training. It would be interesting to investigate the final clustering performance on these “difficult” samples after convergence. Would they yield much better performance than before?\n4. There are some other clustering methods (such as Twin Contrastive Learning for Online Clustering, IJCV 2022) that also use pseudo labels to boost clustering performance. The authors could clarify the differences between this work and previous ones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G5Ue8P5Otn", "reviewer": "Reviewer_ofuD", "summary": "Information bottleneck theory effectively removes redundancy or noise in multi-modal clustering while preserving discriminative information, but existing IB-based methods face challenges such as low-quality pseudo-labels, over-reliance on accurate mutual information estimation and unreliable clustering outcomes. This work proposes a calibrated information bottleneck framework featured a parallel multi-head network to reach three typical goals, calibrating biased MI estimation to enhance IB stability, building reliable IB targets from multi-modal data samples and getting trusted results. A dynamic pseudo-label selection strategy grounded in information redundancy theory improves training stability by filtering high-confidence labels. Experiments show the method achieves promising results across many benchmarks.", "questions": "1. In the adaptive fusion, could you provide the specific weights w_m learned during the experiments? This would help verify whether the model is truly adapting the importance of different modalities.\n\n2. The ablation study for L_con shows a potential trade-off between ACC and ECE.  In a practical application, if one must prioritize one over the other, how would you advise them to adjust the model?\n\n3. Is this the first work addressing the trusted multi-modal clustering problem with calibration? If yes, please explicitly describe it in the paper. If not, please deeply discuss the differences with the related works.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The proposed CLIB framework is innovative. Its parallel multi-head architecture successfully decouples the calibration objective from the final clustering objective.\n\n2.Theorem 1 given in the paper is insightful. It theoretically connects the difficult problem of MI estimation bias with the pseudo-label screening mechanism via a clear logical chain.\n\n3.The experimental section is solid. On five widely-used benchmark datasets, CLIB achieves state-of-the-art performance on all three metrics. Particularly, the significant reduction in ECE robustly demonstrates CLIB's effectiveness in mitigating model overconfidence.", "weaknesses": "1.One of the core motivations of the paper is that MI estimation is difficult and biased. However, in the actual implementation, different parts of the model use three different MI estimation strategies. The authors do not explicitly explain the rationale for choosing this.\n\n2.The implementation environments of the code are not given in the experiments, which may influence the reproductivity. A complete experimental details is usually needed in the experimental setup subsection.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LHjOZ9ikH4", "reviewer": "Reviewer_omRQ", "summary": "This paper proposes CLIB, a calibrated information bottleneck framework for multi-modal clustering. It introduces a multi-head architecture with dedicated calibration heads to mitigate the impact of noisy pseudo-labels and biased mutual information estimation—key issues in existing IB-based methods. A dynamic label selection strategy further improves training stability. Experiments show CLIB achieves state-of-the-art clustering accuracy and superior calibration on multiple benchmarks.", "questions": "The method relies on pseudo-labels for both clustering and calibration, yet these labels are inherently noisy and evolve during training. How does the proposed calibration mechanism avoid reinforcing or amplifying incorrect pseudo-labels in the early or unstable stages of training? A brief analysis or design justification on the robustness of calibration to label noise would significantly strengthen the paper's claim of producing \"trusted\" clustering.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-Well-motivated problem: The focus on improving trustworthiness and calibration in multi-modal clustering is timely and important.\n-Innovative architecture: The multi-head design with dedicated calibration heads is a clever way to decouple representation learning from label refinement.", "weaknesses": "-Limited discussion on calibration in clustering: While ECE is adapted for clustering, the paper does not fully address the challenge of defining \"correctness\" without ground-truth labels during calibration evaluation.\n-Lack of computational analysis: No comparison of training time or model complexity is provided, making it hard to assess practical efficiency.\n-Hyperparameter sensitivity: The balance between IB objectives and calibration is controlled by hyperparameters; their robustness is not thoroughly analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)