Towards Robust Uncertainty Calibration for Composed Image Retrieval
Authors: Yifan Wang
Wuliang Huang
Yufan Wen
Shunning Liu
Chun Yuan
Venue: NeurIPS 2025 poster ()
Keywords: Image Search
Composed Image Retrieval
Uncertainty Estimation
Primary area: applications

Abstract: The interactive task of composed image retrieval aims to retrieve the most relevant images with the bi-modal query, consisting of a reference image and a modification sentence. Despite significant efforts to bridge the heterogeneous gap within the bi-modal query and leverage contrastive learning to reduce the disparity between positive and negative triplets, prior methods often fail to ensure reliable matching due to aleatoric and epistemic uncertainty. Specifically, the aleatoric uncertainty stems from underlying semantic correlations within candidate instances and annotation noise, and the epistemic uncertainty is usually caused by overconfidence in dominant semantic categories. In this paper, we propose Robust UNcertainty Calibration (RUNC) to quantify the uncertainty and calibrate the imbalanced semantic distribution. To mitigate semantic ambiguity in similarity distribution between fusion queries and targets, RUNC maximizes the matching evidence by utilizing a high-order conjugate prior distribution to fit the semantic covariances in candidate samples. With the estimated uncertainty coefficient of each candidate, the target distribution is calibrated to encourage balanced semantic alignment. Additionally, we minimize the ambiguity in the fusion evidence when forming the unified query by incorporating orthogonal constraints on explicit textual embeddings and implicit queries, to reduce the representation redundancy. Extensive experiments and ablation analysis on benchmark datasets FashionIQ and CIRR verify the robustness of RUNC in predicting reliable retrieval results from a large image gallery.

Towards Robust Uncertainty Calibration for Composed Image Retrieval
Yifan Wang', Wuliang Huang', Yufan Wen', Shunning Liu', Chun Yuan
Tsinghua Shenzhen International Graduate School, Tsinghua University
Institute of Computing Technology. Chinese Academy of Sciences
Syifan-wa220mal Sannguulingg19@@iccee.
wenyf248mails, 1sn24@mails, yuanc0sz). laaanguuaedd...
Abstract
The interactive task of composed image retrieval aims to retrieve the most relevant images with the bi-modal query, consisting of a reference image and a modification sentence. Despite significant efforts bridge the heterogeneous gap within the bi-modal query and leverage contrastive learning to reduce the disparity between positive and negative triplets, prior methods often fail to ensure reliable matching due to aleatoric and epistemic uncertainty. Specifically, the aleatoric uncertainty stems from underlying semantic correlations within candidate instances and anno- tation noise, and the epistemic uncertainty is usually caused by overconfidence in dominant semantic categories. In this paper, we propose Robust UNcertainty Cali- bration (RUNC) to quantify the uncertainty and calibrate the imbalanced semantic distribution. To mitigate semantic unbiguity in similarity distribution between fusion queries and targets, RUNC maximizes the matching evidence by utilizing a high-order conjugate prior distribution fit the semantic covariances in candidate samples. With the estimated uncertainty coefficient of each candidate, the target distribution calibrated to encourage balanced semantic alignment. Additionally, we minimize the ambiguity in the fusion evidence when forming the unified query by incorporating orthogonal constraints on explicit textual embeddings and im- plicit queries, reduce the representation redundancy. Extensive experiments and ablation analysis on benchmark datasets FashionIQ and CIRR verify the robustness of RUNC in predicting reliable retrieval results from a large image gallery.
Introduction
The task of composed image retrieval (CIR) [1, 2, 2, 5] is emerging as a multi-modal interaction form to accommodate the flexible search requirements. Distinguished from traditional single-modal image retrieval cross-modal retrieval, the queries of CIR support two modalities, i.e., reference images and modification texts that illustrate alternations on the reference images. To identify the most correlated images among the massive candidate images, the core objective is to establish semantic connections between target images and bi-modal queries through similarity measurement and bridge the heterogeneous modality gap across different modalities within queries. Exploring multi-modal data integration and natural interaction requirements on this task could provide underlying support for tasks such visual question answering (VQA) [6, 7, 8), visual reasoning [9, 10, 11], etc. the basis of multi-modal understanding [12].
One widely recognized challenge for composed image retrieval is the comprehension of the semantic conflict in the bi-modal query. Despite the modality gap in the query composition, the modifications
Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

expressed by the query text have led to disagreements with the reference images, hindering the understanding of the
input and formation of a unified query representation. Existing
opistemic uncertainty from insufficien: leemiin of the secuuntic "Asien-izspire""
aleatoric uncertainty from sementic correcations
Semantic Covariance
Semantic Distribution block
Mumber of Samples
shiny black and more Asian- inspired
Astan-
inspired
4
1
1n
Agt
Reference Image
Top Tanking Images
Random Sampling Images
Semantic Clusters
Figure 1: Illustration of uncertain matching results. (leff) Unreliable top-ranking results disrupted by insufficient learning of Assin-insppred " and partial semantic correlations. (right) Strong correlations within the semantic matrix and imbalanced semantic distributions underlying visual candidates.
works were devoted to mining the effective components within the queries by multi-granular feature matching, e-g., word-level [13, 14], patch-level [15], token-level [16. 17], and hierarchical-level [18]. Inspired by the significant achievement of vision-language models on the massive corpus, recent models adopted transformer-basee
encoders in CLIP [19, 20] and BLIP [21. 22] to enhance visual and textual features [23] and align semantics in the query, owing to multiple inter- and intra-attention mechanisms extract salient information. Besides, capture the correspondences between multi- modal queries and targets, current approaches [24, 5] mainly project the query and target features in the joint space and measure pairwise similarities in the contrastive learning framework. This pipeline regards the CIR task as classification task to distinguish between matched and mismatched triplets.
Contrastive learning essentially follows the assumption that there is no overlap between the distribu- tions of positive and negative triplets. However, images inherently contain rich semantic concepts that cannot be explicitly assigned as independent labels, i.e., negative candidates exhibit certain underly- ing semantic dependencies with ground tround-truth truth positive images. Fig. 1 illustrates that the annotated "negative" images 1 and ground-truie images Lat exhibit strong visual relations, while the semantic correlations remain disregarded within contrastive learning paradigm. Moreover, it is inevitable that not all images that satisfy the query requirements are labeled as positives, and the modification descriptions lack clarity in accurately conveying the intended image. The collaborative impacts of the above factors lead to aleatoric uncertainty, which caused by the implicit semantics dependencies and noise labelling issues in the datasets. Additionally, semantic concept imbalance is also significant in images where specialized designs like "Asian-inspired" are limited in the fashion domain, as seen in Fig. 1 (right). Furthermore, statistical properties of widely-used softmax function in either attention mechanisms [25, 13, 23, 21] or cross-entropy calculation, further increase the biased estimation, leading to overconfident predictions dominated by the explicit semantic concepts and overlook of discriminative details (as shown in IT, 12. and 13 in Fig. 1). The imbalanced distributions of semantic concepts in visual features and coceennncettiionn on salient features result in epistemic uncertainty, which tends to overfit the majority semantics and result in low confidence when facing minority semantic categories.
To address the aforementioned challenges, we propose Robust UNcertainty Calibration (RUNC) to ensure credible semantic bridging between bi-modal queries and visual targets, as shown in Fig. 2. In order to perceive the latent semantic correlations and quantify uncertainty, we incorporate Normal Inverse Gamma distribution evidential priors to fit the semantic covariances in the candidate images. Through maximizing the evidence by the model and imposing a penalty on the incorrect evidence, the inferred probabilistic distributions estimate aleatoric and epistemic uncertainty each candidate image. With more emphasis on uncertain images with rare and ambiguous semantics, we assign uncertainty coefficients to calibrate the target distribution when supervising the query distribution. To further decrease the ambiguity when composing fusion representations for the coupled query images and texts and ensure the semantic consistency, implicit query embedding is introduced to drive the query embeddings to distill more retained visual semantics rather than redundant representations in text modifiers through aligning with the targets and orthogonal to query texts. Experimental results on widely-adopted datasets FashionIQ and CIRR verify that RUNC yields robust and reliable rankings.
In summary, the proposed RUNC makes the following contribution:

---

query
Text
Learnable Queries
Q-Former
P
IF
Encoder
be samo Ireed day with pippy
Uncertainty Distibution
Lat
surving
implicit
Modification Text
Text Encoder
Image Encoder
Y.v. f.B
Reference Image
Implictt Matching
Uncertainty Percaptron
Lunn
Image Encoder
Correfation measurement
Frozen
Concatenation
Text Tokens
Covariance Matrik
Image Gollery
Condidate Visual features
Learnable Prompts
Wrtual Guidance
Figure 2: The Framework of RUNC. Uncertainty perceptron introduces evidential priors to fit the semantic covariance and yield uncertainty distribution to calibrate the supervision on the fusion query. Implicit guidances p* are incorporated to distill effective features for retention and modification.
We propose a novel uncertainty calibration approach to address the misguidance dominant semanties and semantic overlaps when matching bi-modal queries with target images.
We employ high-order evidential priors to quantify the aleatoric and epistemic uncertainty and adaptively adjust the semantic distribution imbalance based on uncertainty estimation.
To minimize the ambiguity when fusing hybrid-modal queries, assign orthogonal inde- pendent constraints on explicit textual embeddings and implicit queries to distill effective features for retention and modification.
2
Related Works
2.1 Composed Image Retrieval
To address the growing need for flexible retrieving with multi-modal data, composed image retrieval focuses on exploring the integration of dual-modal input of reference images and modification text and matching with the desired images. Existing works are dedicated to mitigating the heterogeneous gap of multi-modal data in the query domain by extracting effective semantic components across different modalities [1, 26, 27] via multiple projection layers [1]. cross-modal attention [18. 28], and graph- based propagation [13, 29]. For instance, Fashion VLP [30] introduced multi-layer self-attention on the combined tokens of visual regions from the reference image and words from modifiers. To facilitate correlations with query images and query texts, Bai et al. [31] incorporated sentence-level prompts with visual features and textual tokens in the Q-Former structure [32]. Through contrastive learning, the fusion queries are guided toward the target features and pushed away from other candidate features by comparing positive and negative samples. DCNet [33], ComposeAE [27], and CaLa [22] applied bi-directional constraints to strengthen the semantic consistency across references, modifiers, and targets. Recent advances [34, 16] reconstructed triplet data with higher-quality to ensure that contrastive learning effectively captures semantic alignment, which requires expensive annotation efforts. With a focus on the separation of positive and negative matching the above approaches, the latent semantic correlations among the images, particularly false negatives, may hinder the actual semantic alignment. Distinguished from previous methods, this work deploys prior distributions model the intrinsic covariance of candidate features and addresses the semantic imbalance by uncertainty calibration to enhance the robustness.
Uncertainty Estimation
Though deep learning models are currently excelling across various domains, most of them typically provide predictions without considering the confidence of the outcomes [35, 36). The prediction uncertainty stemming from data noise, model overconfidence, and biased learning significantly

---

impact the robustness of noisy labels, generalization on unseen classes, and model interpretability. Based on Bayesian Neural Networks (BNN) [37] and Monte Carlo Dropout, uncertainty models [38] estimated prediction variance through multiple forward propagation samples in out-of-distribution (OOD) scenarios [39, 40]. MGUR [4] applied weighted Gaussian noises on whitened features to simulate data jitter and one-to-man many correspondences. Subjective Logic (SL. formalized the concept of belief assignment in the Dempster-
theory of evidence as a Dirichlet distribution [41, 42] to quantify the belief mass and uncertainty. Deep Evidential Regression (DER) [43, 44] introduced high-order priors to capture the confidences supporting the model prediction. Evidential deep learning avoided the computational bottlenecks of traditional Bayesian methods by obtaining uncertainty in single forward inference. In this work, the proposed RUNC extends the uncertainty estimation to explore latent correlations in the coupling visual semanties and calibrate the imbalanced semantic distributions when aligning composed queries and targets.
Methodology
3.1 Problem Setting
The multi-modal dataset for composed image retrieval includes N query-to-target pairs. Each matched data (I,, M,, M,) contains one reference image I,, one modification sentence M, and one target image I, To bridge the modality gap between the visual and textual inputs, the retrieval pipeline utilizes pretrained encoders [19, 21] to project all the raw image inputs into the semantic latent space. We denote candidate visual features as (t. J for N images in the gallery, where 6; E Rdd and d denotes dimensions. In the branch of combining the cross-modal query. the lightweight Q-Former [32] is employed to obtain the interactive prompt embedding p along with text features m, with the input of visual features from the original reference images, instructions from the modification text, and learnable queries. Afterward, the prompt embedding p is further projected in the same latent space as t to yield the fusion query representation q.
The essence of the composed image retrieval task lies in accurately measuring the semantic distances between the queries and targets in the embedding space. With the aim of pushing the fusion query representations towards the target features (q. - t.) while separating query representations from irrelevant candidate samples (g, <-> tz), existing frameworks [45, 21] mainly adopt the contrastive loss to classify positive pairs (qi, b.) and negative pairs (q: t,يī + i):
B
exp(s./TT)
L = B
log
(I)
Σラ exp(s1//T)
where the batch size and refers to the temperature parameter. The matching score 8ij between composed query i and candidate feature is computed based on the cosine distance 8úy = 1
g,
where means the 1.2 normalization.
Though cross-entropy loss is efficient in large-scale retrieval applications, it cannot be fully adapted to this complicated interactive retrieval task. In this context, contrastive loss is primarily focused on aligning the matching scores matrix 8 E IR BxB with the diagonal matrix. Specifically, for normalized features, the matching value for the fusion query to the target features should ideally approach 1, while the matching value for all the other samples should be as close to as possible. However, due to the strong semantic correlations among most candidate images in the image gallery and noise labels during the dataset construction process, non-diagonal negative samples may exhibit semantic overlap. Furthermore, the fused features derived from the complex multi-modal interactive inputs inherently introduce semantic uncertainty and instability, which results in loose and unstable semantic construction with traditional supervision loss. To this end, the uncertainty of alignments between the cross- modal query and candidate images is imperative to estimate to construct robust retrieval model. As shown in Fig. 2, we deploy distribution-ban semantic ambiguity and bring aleatoric and epistemic uncertainty into consideration when setting the training objective.
uncertainty estimation to quantify the
3.2 Uncertainty Estimation
Priors of Semantic Covariance Matrix. As aforementioned, the high frequency of coupling semantic concepts (as shown in Figure (left)) in the candidate images poses challenges to the

---

model's prediction of similarities between queries and targets. Using hard-coded supervision with binary values of or 0 for positive and negative sample pairs respectively fails to accurately represent the real retrieval scenario. In the proposed RUNC, we approach the prediction of matching values in the retrieval process as a regression problem, assuming that the distribution of the semantic correlations conforms to Gaussian distributions and is independent and identically distributed (i.i.d). The correlations across various target semantic concepts are acquired by the interactions of candidate features in the target image space, to measure the potential semantic overlaps between different pairs.
Cjj
where Cij stands for the semantic covariance score between the i-th sample and the j-th sample.
The objective of uncertainty estimation is to estimate prior distribution reconstruct the variance and mean of the Gaussian distribution of semantic concepts [46, 47, 43J. Therefore, a high-order Normal Inverse Gamma (NIG) prior is introduced model the Gaussian output:
(Clá C2i. .... cN.) W1,, o;); P4 NN(,
of (a,
(3)
where r()) represents Gamma function.
To estimate the posterior distribution g(H; 02) p(us, hi, V/, as BB) for the i-th target image semantic representations, we factorize the distribution in the form of conjugate prior as q(p,, o7) ((4))(66)) ), which is reformulated as:
8.0h Vii
1 joct1 exp( 28; + v2(%% - pa) 87
p(ui, 87 14, as
as, 3,) I((as) / 2T8
(4)
26
Given the fusion query representations and the evidential distribution from uncertainty perceptron, we then maximize the model evidence to support the observations by maximizing the likelihood of observing the semantic covariance matrix:
NLL log(p(ci)/4a V; 0c. 8:))
log( "" 142
a log Q4 + (as + "))o(o((s)) - p4) + 52) + log( T(as) T(as + 11)
1
(5)
2
where Ω = 23. + v4). Compared to directly imposing the hard-encoded labels on the distances be- tween the fusion query and target, introducing evidential distribution capture semantic interactions between various instances facilitates reliable similarity measurements in a more nuanced way.
Uncertainty-Guudeed Semantic Calibration. From the perspective of Bayesian Inference, NIG distribution is a conjugate prior of Gaussian distribution, and its corresponding parameter could be intuitively interpreted as virtual observations. To reveal the shape characteristics, the mean could be regarded as the sample mean calculated from v virtual observations with the mean of y, and the variance could be considered as an estimation based on virtual observations with the mean of y and the sum of squared deviations 2v. Thus, the total evidence, which is the sum of all the inferred virtual observations comprised of all the virtual observation information of means and variances, is defined as 2n + a. Based on the model parameters of the uncertainty model in Section 3.2, the statistical moments target semantics are computed through first-order moments of the NIG distribution:
3 1-1
B
Elu] =
E18
Variul v(a - 1)'
(6)
where the latter two notions could also be interpreted as aleatoric and epistemic uncertainty of semantic distribution. The total uncertainty is further obtained by:
B: U4: E + Var[u:] a;
3r
Belss + 11)
(7)
14 1) 21 -1)
Note that semantic distribution is imbalanced in the realistic retrieving process. For instance, semantics related to colors and objects tend to appear frequently, while those involving specific details like "whimsical and vintage" are rare. Consequently, the top retrieval results tend to predict semantic categories that are more commonly represented. To avoid imbalanced learning from diverse semantics, introduce uncertainty coefficients cross-entropy computation based on the semantic
5

---

uncertainty fitted by NIG distribution. When a semantic embodies high uncertainty, the corresponding weight ought set larger, so that more penalty would be enforced on this sample during the training phase, avoiding neglecting infrequent semantic categories. The refined balanced contrastive loss is:
exp(8R/TT
Dbc
B E Uy log B
Σラ= exp(81g/T)
Penalty on Misleading Evidence. As the likelihood- based loss maximizes the evidence, it may lead evidence magnification associated with the target, whereas the model discrimination on matched and hard negative samples would be limited. In particular, explicit semantics tend to overpower the outcomes and mislead the model training. whereas those with limited occurrences could be overlooked during this process. Since the misleading evidence of dominant semantics could be effective in most cases, the model is prone to maintain this incorrect evidence. It is contradictory to our target to actively reveal uncertainty when dealing with ambiguous decision boundaries instead of giving wrong predictions. Thus, we introduce the regularization term on misleading evidence to ensure robust ranking from reliable evidence. Note that TE( and Var[u] in Eq. 6 both demonstrate that uncertainty shows positive correlations with the parameter 3 and evidence by virtual observation theory corresponds to and Theoretical analysis is illustrated in the supplementary material. Hence, the regularization term is defined as:
C.EG = (c.j- - 7))2 (2v; + a; +
3.
The overall uncertainty loss combines the likelihood function to maximize the model evidence and penalty for misleading evidence with controllable weight i1:
Lunc
Σ (C, LL + +AIGREE)
(10)
B
3.3 Implicit Concept Matching
Apart from uncertainty matching caused by underlying semantic covariance and distribution imbal- ance, complicated information sources from different modalities in the hybrid-modal query also introduce ambiguity when composing the fusion query. The reference image contains redundant visual information, e.g., objects and attributes that would be replaced the modification sentences, yet these salient features are inclined to be amplified through attention layers in the transformer- Q-Former structure, significantly corrupting the semantic representation the fusion query by sub- stantial noise. Moreover, learnable prompts p involved in the computation of fusion query attempts to capture visual semantics aligned with text, however the learnable prompts p may have unclear concepts due to the misleading visual redundancy. Simply equipping supervision between fusion queries and candidates could result in substantial redundant information in embedding p. especially repeatedly expressing semantics already revealed by text embeddings of modifiers.
In the training phase, we additionally integrate virtual guidance p" to directly lead the learning of prompt and fully unleash the potentials of p. MSE loss is utilized to align the learnable embeddings with virtual guidance p" as L. 1|p p' To encourage the learnable queries acquire highly correlated messages with the targets. we deploy symmetrical representation q" to mirror the fusion query q, as shown in Fig. 2, which combines virtual guidance p' and text tokens of modifiers. After encoding by the text encoder using shared weights, we yield the implicit query q* which is expected to be aligned with the target features:
B Loe Σ
exp exp(s
log
(1I)
1==1 exp(stg/T)
where siy the cosine similarity between the implicit query q" and target images.
In order to force the learnable embeddings to grasp implicit semantics in reference images rather than duplicated semanties mentioned in the modification text, we incorporate orthogonal loss to differentiate the retention and modification characteristics:
Lort Σ (Cov(p",m)is) m)
= -' _ (p') m) m)
(12)
if1

---

The overall training objective is the aggregation of all the loss functions as: -= Lie + X2Lune Lort + aa Lee. where 2 a trade-off parameter. The first two terms are dedicated to quantifying the uncertainty by NIG priors and calibrating the imbalanced correlations, and the remaining terms provide soft supervision from the implicit query to minimize fusion ambiguity.
Table 1: Retrieval results on FashionIQ. The best results are marked in bold.
Dress
Shirt
Toptee
Average
Methods
@@ 10
R@50
R @ 10
R@ 50
R@ 10
R@50
R@ 10
R@50
Mean
VAL [18]
21.12
42.19
21.03
43.44
25.64
49.49
22.60
45.04
33.82
CIRPLANT [48]
14.38
34.66
13.64
33.56
16.44
38.34
14.82
35.52
25.17
CoSMo [49]
21.39
44.45
16.90
37.49
21.32
46.02
19.87
42.65
31.26
CLVC-Net [50]
29.85
56.47
28.75
54.76
33.50
64.00
30.70
58.41
44.56
ARTEMIS [24]
27.16
52.40
21.78
43.64
29.20
54.83
26.05
50.29
38.17
Fashion VLP [30]
26.77
53.20
22.67
46.22
28.51
57.47
25.98
52.30
39.14
NSFSE [28]
31.12
55.73
24.58
45.85
31.93
58.37
29.17
53.24
41.26
CLIP4Cir [45]
31.63
56.67
36.36
58.00
38.19
62.42
35.39
59.03
47.21
CRN [51]
30.20
57.15
29.17
55.03
33.70
63.91
31.02
58,70
44.86
MGUR [4]
32.61
61.34
33.23
62.55
41.40
72.51
35.75
65.47
50.61
SPN [34]
38.82
62.92
45.83
66.44
48.80
71.29
44.48
66.88
55.68
FAME- ViL. [52]
42.19
67.38
47.64
68.79
50.69
73.07
46.84
69.75
58.29
CaLa [22]
42.38
66.08
46.76
68.16
50.93
73,42
46,69
69.22
58.05
SPRC [31]
49.18
72.43
55,64
73.89
59.35
78.58
54.92
74.97
64.85
CCIN [53]
49.38
72.58
55.93
74.14
57.93
77.56
54.41
74.76
64.59
RUNC (Ours)
48.93
73.53
57.26
75.32
60.38
79.86
55.52
76.23
65.88
4 Experiments
4.1 Experimental Settings
The proposed RUNC is employed on two widely-used composed image retrieval datasets covering various modification requirements in real-life retrieval scenarios. FashionIQ [54], con- centrating on fashion item retrieval, addresses the retrieval for modifications in attributes including colors, patterns, textures, and design details across dress, toptee, and shirt categories. The whole dataset contains 77,684 fashion pictures and each matched triplet is constituted of a reference image, a modification sentence, and one target image. Following [1, 18), the dataset split by the proportion of 3:1:1 for training, validating, and testing respectively. CIRR [48] involves more natural scenes and query texts more focus on alterations the relationships among subjects, backgrounds, and multiple subjects within intricate images. It consists of 21,552 images collected from the NLVR² dataset [55] and constructs 36,554 matched pairs. To further evaluate the model when facing different scenarios, CIRR also provides subset setting and each subset includes six visually similar images.
Datasets.
Evaluation Metrics. Following [24, 34, 31], we employ the Recall rate K (R@K) K) as the main metric to evaluate the model performance, which is defined as the ratio of matched ground-truth images ranked the top-K predictions by the model. In Fashion R 10 and R @550 results are shown on dresses, toptees, and shirts. In CIRR, apart from R@ 1, R Φ 5, R @ 10, and R @ 50 metrics, we also provide Raubset K results evaluated in the subsets.
Implementation Details. We exploited the visual and textual encoders as BLIP-2vrr - model and initialized parameters from pre-trained EVA-CLIP [32] weights. The visual encoder remained frozen and the remaining layers were fine-tuned in the training phase. The virtual guidance was disabled during the inference phase. The uncertainty perceptron was implemented as one feed -orw-fd network (two linear layers) with softplus activation function. The dropout rate was set as 0.2. The dimensions of fusion and candidate features were fixed as 256 in the embedding space and the number of learnable queries was set as 32. We set X₁ as 0.01 to in Eq. 10. We used Adam W optimizer and set the learning rate as 10-5 FashionIQ and 10-5 in CIRR with cosine annealing decay. The training and inference time of the proposed model are 2 14.33 and 27.4s 7s respectively. The
7

---

Table 2: Retrieval results on CIRR test set. The best results are marked in bold.
Recall @ K
Reubset @ K
Methods
K=1
K=5
K=10
K=50
K=1
K=2
K=3
Avg(R @,, Ruubset @
TIRG [1]
11.04
35.08
51.27
83.29
23.82
45.65
64.55
29.45
CIRPLANT [48]
19.55
52.55
68.39
92.38
39.20
63.03
79.49
45.88
ARTEMIS [24]
16.96
46.10
61.31
87.73
39.99
62.20
75.67
43.05
NSESE [28]
20.70
52.50
67.96
90.74
44,20
65.53
78.50
48.35
CLIP4Cir [45]
33.59
65.35
77.35
95.21
62.39
81.81
92.02
63.87
CompoDiff [56]
22.35
54.36
73.41
91.77
35.84
56.11
76.60
45.10
BLIPACIR [21]
40.17
71.81
83.18
95.69
72.34
88.70
95.23
72.07
SSN [57]
43.91
77.25
86.48
97.45
71.76
88.63
95.54
74.51
SPN [34]
45.33
78.07
87.61
98.17
73.93
89.28
95.61
76.00
CaLa [22]
49.11
81.21
89.59
98.00
76.27
91.04
96.46
78.74
SPRC [31]
51.96
82.12
89.74
97.69
80.65
92.31
96.60
81.39
ENCODER [20)
46.10
77.98
87.16
94.64
76,92
90,41
95.95
77.45
DIPNEC [3]
47.24
80.20
89.07
97.87
73.97
89.74
95.72
77.09
RUNC (Ours)
53.81
83.47
91.11
98.22
80.87
92.36
96.94
82.17
experiments were implemented in Pytorch single NVIDIA A800 GPU and trained for 30 epochs for FashionIQ and 50 epochs for CIRR².
4.2 Comparison with State of the Arts
Table and Table report quantitative comparisons of our RUNC with the advanced methods FashionIQ and CIRR datasets, respectively. Detailed architecture descriptions of the compared methods are illustrated in the supplemental material. The proposed RUNC achieves competitive performances on benchmarks of interactive image retrieval. Specifically, gain of 1 1.26% on mean @550 in FashionIQ and rise of 1.37% on R @@ 10 in CIRR compared with SPRC [31] verify the effectiveness our proposal. For fashion retrieval, the consistent growth across R@ 10 and @550 metrics suggests that this approach accurately grasps long-tail search demands. Compared with CaLa [22] and SPRC [31] in the same backbone, the overall improvements
requirements and also adeptly caters to
significant, more intrinsic semantic correlations underlying similar candidate fashion images and unclear modification descriptions in fashion queries bring uncertainty when comparing hard negatives and targets. By incorporating evidential distribution to estimate the uncertainty and calibrate the semantic imbalance, RUNC provides more reliable and robust predictions overall. Apart from recall results on all the images the gallery, the increased metrics on subset settings as 3.95% on Recallaubset @ in CIRR compared with the latest ENCODER [20] implicates that our proposed RUNC could identify the authentic intention from the bi-modal query when selecting from a set of closely resembling candidates.
4.3 Ablation Analysis
Analysis of Effective Components. To assess the efficacy of the model design in this work, ablative results on independent components are shown Table 3, where "w/o UE" means disabling uncertainty estimation on semantic correlations, "w/o UGC" refers using original form of contrastive loss without uncertainty coefficients in Eq. 8, "w/o ICM" means removing implicit concept matching, and "w/o SC" means replace the semantic covariance matrix e with ground-truth labels in Eq. 5. The remarkable drop in recall rates of "w/o UE" demonstrates that uncertainty quantification is crucial to boost the performance by perceiving the underlying correlations in candidates, and results of "w/o UGC" further verifies that uncertainty-aware calibration could avoid over-reliance on dominant categories and promote effective learning from negative samples. As the decline of results in "w/o ICM" shows, implicit query guides the distillation of inherent visual elements from reference images by aligning with the targets and independent constraints with textual embeddings. Comparing the proposed model and "w/o SC", the discrepancy highlights the significance of underlying associations in visual candidates and the effectiveness of combining evidential regression with semantic covariance in quantifying semantic ambiguity.
Code is available RUNC-source.
8

---

Table 3: Ablative study on effective components Table 4: Analysis of uncertainty estimation on R 1050 metric.
RUNC.
FashionIQ
CIRR
Models
Dress
Shirt
Toptec
Mean
Models
R@ 10
R 50
R R@S
Raubset @
GMM
71.69
73,26
77.41
74.12
w/o UE
53.07
74.77
81.51
78.23
BMM
71.83
73.99
77.91
74.58
w/o UGC
53.79
75.03
82.28
78.07
EDC
72.83
75.22
78.74
75.60
w/o ICM
54.76
75.22
82.47
79.15
MGUR
72.19
74.09
78.73
75.00
w/o SC
54.19
75.14
82.30
78.88
MPC
72.29
74.53
78.84
75.22
Ours
55.52
76.23
83.47
80.87
Ours
73.53
75.32
79.86
76.23
Analysis of Uncertainty Estimation. As shown in Table 4, we also investigate different models to quantify the uncertainty and lead to the following observations. i) "GMM" and "BMM" are sensitive to noise and fail to model aleatoric uncertainty, resulting in unstable retrieval results. Besides, the estimation dependent on the EM algorithm brings computation cost and retrieval latency. ii) Compared to Evidential Dirichlet Classification (EDC) [41], the improvement of RUNC implies that NIG distribution flexibly handles heteroscedastic noise, which better aligns with the coupling semantic correlations in this retrieval task. iii) Though MGUR [4] and MPC [58] utilize probabilistic distribution to model the data uncertainty, insufficient semantic supervision may result in additional noise by nondirective distributions. In comparison, high-order priors in RUNC is superior in quantifying the underlying uncertainty and enhance the interpretability of ranking results.
4.4 Further Discussion
Impact of Uncertainty Supervision. To evaluate the sensitivity of A2 controlling the uncertainty estimation loss in different datasets, Fig. 3(a) presents recall rates on different settings. The optimal value of 2 for FashionlQ is slightly bigger than CIRR for more severe aleatoric uncertainty issues in FashionIQ. More sensitivity analysis could be referred to in the supplementary material.
Impact of Evidential Learning. To verify the necessity of introducing evidential learning to enhance the robustness, we also conduct an ablative setting as directly using semantic covariance matrix c as weights in contrastive learning, denoted as "w/o edl" in Fig. 3(b). Although correlation weights could promote discriminant learning compared with baseline, it is less superior than proposed evidential learning for accumulating errors from precomputed similarities and neglecting the aleatoric uncertainty by data noise and false negatives.
65
vieiler will Blak
70-
60
Sven
NS-
0.25
2.0
Spocts
(a) Param Ag
(b) Evidential Learning
Figure 4: Visualization of uncertainty on CIRR.
Figure 3: Analysis of Aa and evidential learning.
4.5 Visualization Analysis
Visualization of Uncertainty. To enhance comprehension of the retrieval results based on uncertainty, we also present retrieval examples visualize the uncertainty quantification in Fig. For the right- most image with low resolution and sharing semantics like "black fur" and "puppy" with the user intent, the estimated uncertainty value leads to an increase in the penalty by Eq. 8.

---

Visualization of Implicit Concept Learn- ing. Additionally, we also present activation heatmaps by GradCAM [59] on the last module in the vision encoder with other top-ranking re- sults in Figure 5. The prompt embedding high- lights the area corresponding to the dog which is coherent with the reference images, and underlines the area of the yellow pillow on the couch. The collaboration between p and m ables the model to accurately select the ground- truth image from other candidates, especially enhancing the perception of object contours, categories, or backgrounds provided by the reference image in p.
B-Modi Qaary
Tagin krage
sits yol
Figure 5: Heatmaps on Target Images with Other Top ranking results.
Failure Cases. Failure cases of the proposed RUNC are shown in Figure 6, where the ground-truth images (red boxes) are not ranked the top by our model. For the first example, since the query text in the first instance does not specify the need to change the type of animal in the image, the top-1 image in this model depicts a monkey sleeping on the bend the tree trunk, which can be regarded as false negative sample. It also indicates that the proposed RUNC is capable of perceiving the basic query requirements of users effectively. Regarding the second example, due to the complexity of modifying the semantics in the text, and being influenced by factors such as lighting and angles, the model fails to precisely grasp the requirements of "hind legs" and "industrial setting" during measuring the distance between the query and candidate images.
5 Conclusion
In this paper, we introduced a novel robust uncertainty calibration model dubbed RUNC to quantify the uncertainty and mitigate the imbalanced semantic learning for interactive image retrieval. To encourage learning from infrequent semantic concepts and mitigate interference caused by semantic overlaps, high-order evidential priors are deployed to estimate the aleatoric and epistemic uncertainty, and target distribution is further adjusted based on uncertainty coefficients. Moreover, we employed an orthogonal loss between explicit textual embeddings and implicit queries to minimize the semantic ambiguity of fusion query from reference images and modification texts. The effectiveness and robustness of RUNC have been demonstrated by extensive experimental results and ablation studies. The potential for extending this approach to other nulti-modal learning tasks provides promising prospects for further investigation.
Limitations, Since the proposed RUNC utilizes evidential learning that re- lies on the hypothesis of
Bi-Modal Query
focus aninal which sleeps the bend of the tree trunk
prior distribution, it may
Top Ranking
Images
face challenges in con- tinuous learning and lack adaptability to dynamic environments. When the data distribution rapidly changes, frequent updates of uncertain distribution parameters required, making it difficult for the model quickly adjust confidence estimates. Besides, certain amount of data is required in RUNC to fit the prior distribution. In situations involving small sample sizes, the model may struggle accurately learn the parameters of uncertainty, potentially resulting in either overestimation or underestimation of confidence levels.
Bi-Modal
Place dag standing on hind legs, Add another dog, Place dogs in cowerctal,, industrial setting with orange background
Query
Top Ranking Images
Figure 6: Failure cases in the proposed RUNC.
6 Acknowledgments
This work supported by the National Key R&D Program of China (2022Y1B440114000000040))) SSTIC Grant (KJZD22023922311500000 1 GJHZ202402181 136040088.
10

---

References
[1] N. Vo, L. Jiang, C. Sun, K. Murphy. L. Li, L. Fei-Fei, and J. Hays, "Composing text and image for image retrieval- an empirical odyssey." in Proceedings of the Computer Vision and Puttern Recognition Conference. TEEE, 2019, pp. 6439-6448.
[2] Y. Chen and L. Bazzami, "Learning joint visual semantic matching embeddings for language-guiden retrieval," in European Conference on Computer Vision, vol. 12367. Springer, 2020, pp. 336-152.
[3] Y. Wang, W. Huang, and C. Yuan, "Aligning composed query with image via discriminative perception from negative correspondences"" in AAAI Conference on Artificial Intelligence. AAAI Press, 2025, pp. 8078-8086.
[4] Y. Chen, 7. Zheng, W. Ji, I.. Qu, and T. Chua, "Composed image retrieval with text feedback via multi- grained uncertainty regularization," in International Conference on Learning Representations. OpenRe- view.net, 2024.
[5] Z. Sun, D. Jing, G. Yang, N. Fei, and Z. Lu, "Leveraging large vision-language model as user intent-aware encoder for composed image retrieval," in AAAI Conference on Artificial Intelligence. AAAI Press, 2025, pp. 7149-7157.
[6] Y. Lin, Y. Xie, D. Chen, Xu, C. Zhu, and L. Yuan, "REVIVE: regional visual representation matters in knowledge-hased visual question answering," in Anmual Conference on Neural Information Processing Systems, 2022.
[7] Y. Peng. C. Hao, X. Hu, Peng. X. Geng, and X. Yang. "LIVE: learnable in-context vector for visual question answering," in Annual Conference on Neurul Information Processing Systems, 2024.
[8] W. Lin, J. Chen, J. Mei, A. Coca, and B. Byrne "Fine-grained late-interaction multi-modal retrieval for retrieval augmented visual question answering." in Annual Conference on Neural Information Processing Systems, 2023.
[9] T. Gupta and A. Kembhavi, "Visual programming: Compositional visual reasoning without training." in Proceedings of the Computer Vision and Puttern Recognition Conference. IEEE, 2023. pp. 14 953--14 962.
[10] H. Al-Tahan, Q. Garrido, R. Balestriero, D. Bouchacourt, C. Hazirbas, and M. Ibrahim, "Unibench: Visual reasoning requires rethinking vision-language beyond scaling." in Annual Conference on Neural Information Processing Systems, 2024.
[11] S. Jaiswal, D. Roy. B. Fernando, and C. Tan, "Learning to reason iteratively and parallelly for complex visual reasoning scenarios," in Annual Conference on Neural Information Processing Systems, 2024.
[12] X. Jiang, Z. Huang, X. Xu, J. Song, F. Shen, and H. T. Shen, "Phge: Procedural heterogeneous graph completion for natural language task verification in egocentric videos." in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 8615-8624.
[13] F. Zhang, M. Xu, and C. Xu, "Geometry sensitive cross-modal reasoning for composed query based image retrieval," IEEE Trans. Image Process., vol. 31, pp. 1000-1011 2022.
[14] F. Huang, L. Zhang, X. Fu, and S. Song, "Dynamic weighted combiner for mixed- image retrieval." in AAAI Conference on Artificial Intelligence. AAAI Press, 2024. pp. 2303-2311.
[15] Y. Chen, J. Zhou, and Y. Peng. "SPIRIT: style-guided patch interaction for fashion image retrieval with text feedback," ACM Trans. Multim. Comput. Commun. Appl., vol. 20, no. 6. pp. 167: 167:1-167:17, 2024.
[16] Y. K. Jang, D. Kim, Z. Meng, D. Huynh, and S. Lim, "Visual delta generator with large multi-modal models for semi-supervised composed image retrieval," in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2024, pp. 16 1684..
[17] H. Wen, X. Song. J. Yin. J. Wu, W. Guan, and L. Nie. "Self-training boosted multi-factor matching network for composed image retrieval." IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, 5, pp. 3665-3678. 2024.
[18] Y. Chen, S. Gong, and L. Bazzani, "Image search with text feedback by visiolinguistic attention learning," in Proceedings of the Computer Vision and Pottern Recognition Conference. IEEE, 2020, pp. 2998 3008
[19] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, "Learning transferable visual models from natural language supervision." in International Conference on Machine Learning, vol. 139. PMLR, 2021, pp.
[20] Z. Li, Z. Chen, H. Wen, Z. Fu, Y. Hu, and W. Guan, "ENCODER: entity mining and modification relation binding for composed image retrieval," in AAAI Conference on Artificial Intelligence. AAAI Press, 2025, pp. 5101-5109
[21] Z. Liu, W. Sun, Y. Hong, D. Teney, and S. Gould, "Bi-directional training for composed image retrieval via text prompt learning." in Winter Conference on Applications of Computer Vision. 2024, pp. -5500
11

---

[22] X. Jiang. Y. Wang. M. Li. Y. Wu, B. Hu, and X. Qian. "Cala: Complementary association learning for augmenting comoposed image retrieval," " in Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2024, pp. 2177-2187.
[23] A. Baldrati, M. Bertini, T. Uricchio, and A. Del Bimbo, "Composed image retrieval using contrastive learning and task-oriented clip-based features," " ACM Trans. Multimedia Comput. Commun. Appl., vol. 20, no. 3, 2023.
[24] G. Delmas, R. S. de Rezende, G. Csurka, and D. Larlus, "ARTEMIS: attention-based retrieval with text-cxplicit matching and implicit similarity," International Conference on Learning Representations. OpenReview.net, 2022.
[25] E. Dodds, J. Culpepper, S. Herdade, Y. Zhang, and K. Boakye, Mlodallity-ggnostte attention fusion for visual scarch with text foedback," CoRR, vol. abs/2007.00145, 2020.
[26] B. Schroeder and S. Tripathi, "Structured query-based image retrieval using scene graphs." in Proceedings of the Computer Vision and Pattern Recognition Conference. Computer Vision Foundation IEEE, 2020. pp. 680- 684.
[27] M. U. Anwaar, E. Labintcev. and M. Kleinsteuber. "Compositional learning of image-text query for image retrieval," in Winter Conference on Applications of Computer Vision. IEEE, 2021. pp. 1139-1148.
[28] Y. Wang, L. Liu, C. Yuan, M. Li, and J. Liu, "Negative-sensitive framework with semantic enhancement for composed image retrieval," " IEEE Trans. Multim., vol. 26, pp. 7608-7621, 2024.
[29] M. Shin, Y. Cho, B. Ko, and G. Gu, "Rtic: Residual learning for text and image composition using graph convolutional network," CoRR, vol. abs/2104.03000 04.0111 2021.
[30] S. Goenka, Z. Zheng, A. Jaiswal, R. Chada, Y. Wu, Hedau, and P. Natarajan, "Fashionvlp: Vision language transformer for fashion retrieval with feedback," " in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2022, pp. 14 095
[31] Y. Bai, Xu, Liu, Khan, F. Khan, W. Zuo, R. S. M. Goh, and Feng. "Sentence-level prompts benefit composed image retrieval," in International Conference on Learning Representations. OpenReview.net, 2024.
[32] J. Li, D. Li, S. Savarese, and S. C. H. Hoi, "BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models," in International Conference on Machine Learning, vol. 202. PMLR, 2023, pp. 19 130-19742.
[33] J. Kim, Y. Yu, H. Kim, and G. Kim, "Dual compositional learning in interactive image retrieval," in AAAI Conference on Artificial Intelligence. AAAI Press, 2021. pp. 1771-1779.
[34] Z. Feng, Zhang, and Z. Nie, "Improving composed image retrieval via contrastive learning with scaling positives and negatives," in ACM International Conference on Multimedia. ACM, 2024, pp. 1-10.
[35] Y. Liu, R. Shen, and X. Shen, "Novel uncertainty quantification through perturbation-assitte sample synthesis." IEEE Trans. Puttern Anal. Mach. Intell., vol. 46, no. 12, pp. 7813-7824. 2024.
[36] X. Peng, F. Qiao, and L. Zhao, "Qut-of-domain generalization from a single source: An uncertainty guantification approach," " IEEE Trans. Patiern Anal. Mach. Intell., vol. 46, no. 3, pp. 1775-1787, 2024.
[37] G. Franchi, A. Bursuc, E. Aldea, S. Dubuisson, and I. Bloch, "Encoding the latent posterior of bayesian neural networks for uncertainty quantification," IEEE Trans. Pottern Anal. Mach. Intell., vol. 46, no. 4. pp. 2027-2040, 2024.
[38] Y. Cui, W. Yao. Q. Li, A. B. Chan, and C. J. Xue, "Accelerating monte carlo bayesian prediction via approximating predictive uncertainty over the simplex," IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 4, pp. 1492-1506, 2022.
[39] A. Zhou and S. Levine, "Amortized conditional normalized maximum likelihood: Reliable out of distribu- tion uncertainty estimation," in International Conference Machine Learning, M. Mcila and T. Zhang, Eds., vol. 139. PMLR, 2021, pp. 12 803 12 12812.
[40] F. Lu, K. Zhu, W. Zhai, K. Zheng, and Y. Cao, "Uncertainty-wware optimal transport for semantically coberent distr--iittiittto detection." in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2023, pp. 3282-3291.
[41] M. Sensoy, L. M. Kaplan, and M. Kandemir, "Evidential deep learning quantify classification uncertainty." in Annual Conference on Neural Information Processing Systems. 2018 pp. 3183-3193.
[42] X. Jiang, X. Xu, L. Zhu, Z. Sun, A. Cichocki, and H. T. Shen, "Resisting noise in pscudo labels: Audible video event parsing with evidential learning." IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 6. pp. 80874-10 888, 2025.
[43] A. Amini, W. Schwarting, A. Soleimany, and D. Rus, "Deep evidential regression." in Annual Conference Neural Information Processing Systems, 2020.
12

---

[44] J. Lou. W. Liu. Z. Chen, F. Liu, and J. Cheng, "Elfnet:: Evidential local-global fusion for stereo matching," in International Conference on Computer Vision. IEEE, 2023, pp. 17738-17 747.
[45] A. Baldrati, M. Bertini, T. Uricchio, and A. D. Bimbo, "Conditioned and composed image retrieval combining and partially fine-tuning clip- tip-based features," in CVPR Workshops. IEEE, 2022, pp. 4955- 4964.
[46] A. Malinin and J. Gales, "Predictive uncertainty estimation via prior networks," in Annual Conference on Neural Information Processing Systems, 2018, pp. 7047-7058.
[47] M. Sensoy, L. Kaplan, and Kandemir, "Evidential deep learning quantify classification uncertainty." in Annual Conference on Neural Information Processing Systems, 2018, pp. 3183-3193.
[48] Z. Liu, C. R. Opazo. D. Teney, and S. Gould, "Image retrieval real-life images with pre-trained vision- and-language models," in International Conference Computer Vision. TEEE, 2021, pp. 2105-2114.
[49] S. Lec, D. Kim, and B. Han, "Cosmo: Content-style modulation for image retrieval with text feodback," in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2021, pp. 802-812.
[50] H. Wen, X. Song, X. Yang, Zhan, and Nie, "Comprehensive linguistic-visual composition network for image retrieval," in Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2021, pp. 1369-1378.
[51] Q. Yang, M. Ye.
Cai, Su, and B. Du, "Composed image retrieval via cross relation network with hierarchical aggregation transformer," IEEE Transactions on Image Processing, vol. 32. pp. 4543-4554, 2023.
[52] X. Ham, X. Zhu, L. Yu, L. Zhang. Song, and T. Xiang, "Fame-vil: Multi-tasking vision- laagua- model for hetcrogeneous fashion tasks," in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2023, pp. 2669-2680.
[53] L. Tian, Zhao, Hu, Yang, H. Li, L. Jin. Z. Wang, and Li, "CCIN: compositional conflict identification and neutralization composed image retrieval," Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2025, pp. 3974-3983.
[54] H. Wu, Y. Gao, X. Guo, Z. Al-Halah, Rennie, Grauman, and Feris, "Fashion IQ: A new dataset towards retrieving images by natural language feedback," in Proceedings of the Computer Vision and Pattern Recognition Conference. IEEE, 2021, pp. 11 337-11317.
[55] A. Suhr, S. Zhou, A. Zhang, Zhang, H. Bai, and Y. Artzi, "A corpus for reasoning about natural language grounded photographs." ACL. Association for Computational Linguistics. 2019, pp. 6418 6428.
[56] G. Gu. S. Chun, W. Kim, H. Jun, Y. Kang, and S. Yun, "Compodiff: Versatile composed image retrieval with latent diffusionn" " Trans. Mach. Learn. Res., vol. 2024, 2024.
[57] X. Yang, D. Liu, H. Zhang, Y. Luo, C. Wang, and J. Zhang, "Decompose semantic shifts for composed image retrieval," CoRR, vol. abs/2309.0000 0953399,,93,,,.... 2023.
[58] N. Andrei. Chen, and Akata, "Probabilistic compositional embeddings for multimodal image retrieval." in Proceedings of the Computer Vision and Puttern Recognition Conference. IEEE, 2022, pp.
[59] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks gradient-basse based localization," in International Conference on Computer Vision. IEEE, 2017, pp. 618 626.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of the work performed by the authors in the Section "Limiitaionss".
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: Our paper only involves a small amount of theory, which has been illustrated in the Methodology (as seen in Section 3) and supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We disclosed all the information needed reproduce the main experimental results of the paper. We also provided implementation details in the Experiments (as seen in Section 4.1).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The manuseript provides detailed information to reproduce the results. We also provided the corresponding references and links.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
Mobllic/guides/CdeSuummi.
www.ssubbbbiooooooooooooooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //niss. cccccc/pp.o wwtpccccpcc//ccccccccccccccccltt wwtcc//gggg//eeeeeeeeeeee Sotps//ooooeeessiooonnnooo sotobiiiiiioooococccc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The manuscript provides detailed information about the experimental setting and implementation details in Experiments (as in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Error bar is not commonly used in this Composed Image Retrieval task.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We presented our compute resources in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics attps://neuril: cc cccppblic//eeeeeececeeccceeeeeee htwte//tiiiisieeeiiiiieeieeeee
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discussed both potential positive societal impacts and negative societal impacts of the work performed in this paper and supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The creators or original owners assets (e.g., code, data, models) used in the paper are properly credited, and the license and terms of use are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com comdaatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is used only for polishing the writing of this manuscript and does not impact the core methodology, scientific rigorousness, originality of the research
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https : htt::/rpps.cc/ooffeeeeeeeeeeeeee wwtccc////ooocaaaeeee /cc//oofeereccc//22////... for what should or should not be described.
20


Reviews:
[{"review_id": "nQbEvJpRBG", "reviewer": "Reviewer_QsfS", "summary": "This paper introduces Robust Uncertainty Calibration (RUNC), a novel approach designed to improve composed image retrieval (CIR) by explicitly modeling semantic uncertainty in matching bi-modal queries to target images. RUNC quantifies uncertainty by using a Normal-Inverse Gamma (NIG) prior distribution to model the semantic covariances among candidate images. This estimated uncertainty is then used to calibrate the training objective, promoting a more balanced semantic alignment by focusing on ambiguous and rare concepts. To further enhance the query representation, RUNC incorporates an Implicit Concept Matching (ICM) module that reduces redundancy by enforcing orthogonality between textual and implicit visual components. Experiments conducted on the benchmark FashionIQ and CIRR datasets confirm that RUNC achieves robust and reliable performance, outperforming previous state-of-the-art methods.", "questions": "1. The connection between the Implicit Concept Matching (ICM) module and the paper's central theme of uncertainty should be clarified. \n2. To allow for a complete assessment of the method's practical trade-offs, the paper should include a quantitative analysis of the comprehensive computational overhead. A table comparing training time or GFLOPs against multiple baselines would enable a better evaluation of the model's efficiency.\n3. The qualitative analysis requires expansion beyond the single success case shown in Figure 4.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1.\tThe paper introduces a novel framework for Composed Image Retrieval (CIR) that explicitly quantifies both aleatoric (data noise) and epistemic (model overconfidence) uncertainty using a Normal-Inverse Gamma (NIG) distribution to model semantic correlations.\n2.\tThe method uses the estimated uncertainty to create a balanced contrastive loss, which adaptively increases the penalty on ambiguous or rare samples, mitigating the influence of dominant semantics and improving learning from difficult examples.\n3.\tRUNC achieves robust and competitive results on two widely-used benchmark datasets, FashionIQ and CIRR, outperforming previous advanced methods.\nWeaknesses\n1. Implicit Concept Matching (ICM) appears only loosely connected to the paper’s central theme of uncertainty modeling. Although the authors claim it alleviates fusion ambiguity, it is unclear whether this ambiguity truly maps onto the aleatoric or epistemic uncertainty defined in the paper. ICM seems better characterized as an independent module for refining fusion representations rather than as an integral part of the uncertainty‑calibration pipeline. While beneficial, its role within the Uncertainty Calibration should be articulated more explicitly and positioned more cautiously.\n2. The calculation of the semantic covariance matrix requires computing pairwise cosine similarities between the features of all candidate samples within each batch. This operation could become computationally expensive and may impact training efficiency, particularly when using a large batch size.\n3. The clarity of the framework diagram in Figure 2 could be improved. Several key symbols, such as p, p*, and q* may be confusing for the reader. The inclusion of a formal legend or more descriptive labels to define these variables would significantly enhance the diagram's readability.\n4. Figure 4 only shows one successful retrieval case to intuitively demonstrate the uncertainty quantification of the model. Analyzing the instances where the model fails and their corresponding uncertainty scores will provide a deeper understanding of the limitations of the model and help verify whether the quantified uncertainty is truly effective in challenging scenarios.", "was_revised": true, "final_justification": "After carefully reading the rebuttal of the authors, most of my concerns have been addressed. Therefore, I raise my rating to borderline accept.", "rebuttal": "***Q1/W1: Clarify the connection of between the Implicit Concept Matching (ICM) module and uncertainty.***\n\n**A1:** Thank you for providing this suggestion.  We are sorry for the confusion and would like to clarify the ICM module as follows:\n- In our design, ICM is **not an independent feature fusion module**, but a mechanism specifically aimed at mitigating the uncertainty, particularly the aleatoric uncertainty, in the composition of reference images and modifications.  As the reference images usually have contradictory semantics with modifications (the modified visual regions), partially alignments, and redundant visual information (e.g.. model poses, lighting, irrelevant backgrounds), the fusion queries $\\boldsymbol{q}$ generated  from reference images and modification texts are prone to possessing unstable representations and inconsistent semantics as the users desired. To this end, we introduce an implicit query $\\boldsymbol{p}^\\prime$ to directly guide the learnable prompt $\\boldsymbol{p}$ during fusion query generation, to facilitate the learning of more explicit semantics with direct alignments to the target images.  From the perspective of similarity measurement in the retrieval task, the ICM module **acts as a front-end to eliminate uncertainty** and ensure the semantic consistency of inputs (i.e., query fusion), the remaining parts of the proposed RUNC could be regarded as back-end constraints to avoid over-confidence of output (i.e., matching scores). They collaborate to minimize uncertainties in the composed image retrieval task.\n- To verify the connections between ICM and uncertainty, we have implemented ablative experiments to present the trends of aleatoric uncertainty scores during training as follows. The model without the ICM module generally **has large uncertainty scores** compared to our proposal and **struggles with training convergence**, which demonstrates that ICM contributes to the reduction of aleatoric uncertainty. Without the guidance of implicit queries, the inconsistent and low quality of hybrid-modal inputs consistently causes confusion when matching with candidate samples. \n\nModels|Epoch 0|Epoch 5|Epoch 10|Epoch 15|Epoch 20\n----|----|----|----|----|----  \nw/o ICM|0.902|0.614|0.334|0.199|0.209 \nw ICM|0.692|0.234|0.118|0.086|0.086\n\n- Besides, to provide a vivid presentation of how ICM works, **Figure 8 in supplementary materials** shows heatmaps of the prompt embedding $\\boldsymbol{p}$ and text features $\\boldsymbol{m}$ after the ICM module, where $\\boldsymbol{p}$ highlights the maintained regions in references, and $\\boldsymbol{m}$ specifies the correlated modified areas. Additionally, Table 3 demonstrates that the ICM module brings progressive improvements.\n\nTo sum up, the ICM module concentrates on reducing the aleatoric uncertainty in query compositions and **collaborates with subsequent designs as the early purification of features** to jointly ensure the reliability of the predicted matching scores. Quantitative and visualization results also demonstrate the effectiveness of the ICM module.\n\n***Q2/W2: Computational overhead, especially concerns about the covariance matrix.***\n\n**A2:** Thank you for your insightful comment.  We appreciate the reviewer’s concern regarding the computational cost of the semantic covariance matrix in each batch. \n- In practical training, our method has been evaluated **under moderate batch sizes** (within 256, comparable to the current mainstream methods) in a single NVIDIA A800 GPU. And the additional overhead is manageable and does not cause significant slowdowns in training, as seen in the table below.\n- Moreover, we emphasize that the target features used in this computation are **readily accessible and already indexed in the batch-wise memory**, as they are shared across covariance computations and semantic alignments. This enables the pairwise cosine similarity computation to be performed using efficient matrix operations without additional forward passes or repeated encoding steps.\n- In our experiments, we observed that the proposed RUNC incurs 2.6G FLOPS compared to a baseline model (BLIP2-ViT/G). Besides, we compare the training time and inference time as shown in the table below. Compared with CaLa[1] with complex twin compositors and multiple operations on target features, our model exhibits low computational complexity during training and **demonstrates high conciseness and efficiency in the reasoning phase**, as the matching scores can be directly calculated from the cosine distance between the query fusion feature and the candidate features.\n\nModels | Training Time(s) | Inference Time(s) \n----  | ----  |----  \nCaLa[1] | 263.1 | 45.7\nSPRC[2] | 209.5 | 27.2\nOurs  | 214.3 | 27.4\n\n- Furthermore, given the improvement of incorporating semantic covariance in boosting the retrieval performances (as shown in Table 3 in the main paper), the semantic covariance and uncertainty estimation in our methods consistently improve recall rates across multiple benchmarks. We believe this constitutes a **reasonable trade-off** between computational cost and retrieval performance, especially in settings where robustness to semantic ambiguity is critical. We would include a discussion of computational challenges when scaling to very large batch sizes in the revised manuscript. In future work, we also consider exploring efficient memory bank sampling or low-rank projections to further improve scalability.\n\n***Q3/W4: Analysis of failure cases in addition to a single success case depicted in Figure 4.***\n\n**A3:** Thank you for raising this important concern. Due to the length limit of the main manuscript, we have added **failure cases in FashionIQ and CIRR in Figure 11 and 12 in the supplementary materials**. Specifically, for the second failure case in Figure 11, the uncertainty scores for the top-10 instances are all over 0.34, which could be caused by ambiguous queries (“attractive” in the query text is too vague to identify the targets) and semantic overlap (all the top-10 images embody characteristics of “black”, “bold”, “dress” that are very visually similar to the target one) in these instances. We appreciate your suggestions and would add failure cases in the final version with uncertainty scores and analysis.\n\n***W3: The clarity of the framework diagram in Figure 2 could be improved***\n\n**A4:** Thanks for your suggestions. Specifically, $\\boldsymbol{p}$ means learnable prompts, and $\\boldsymbol{p}^\\ast$ means virtual guidance that maintains the identical shape as $\\boldsymbol{p}$. $\\boldsymbol{q}$ is the fusion query, and $\\boldsymbol{q}^\\ast$ refers to the implicit query that keeps the same shape as $\\boldsymbol{q}$. These two sets of symbols appear in pairs, with the asterisk superscripted features ($\\boldsymbol{p}^\\ast$, $\\boldsymbol{q}^\\ast$) guiding the semantic expression of the original features ($\\boldsymbol{p}$, $\\boldsymbol{q}$) in ICM module. We would add legends for key symbols in the revised version to improve the diagram's readability.\n\nRefs:\n\n[1] Xintong Jiang, Yaxiong Wang, Mengjian Li, Yujiao Wu, Bingwen Hu, Xueming Qian: CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval. SIGIR 2024.\n\n[2] Yang Bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Khan, Wangmeng Zuo, Rick Siow Mong Goh, Chun-Mei Feng: Sentence-level Prompts Benefit Composed Image Retrieval. ICLR 2024.\n\nWe sincerely appreciate your valuable suggestions and recognition of our work, which have greatly contributed to the improvement of our manuscript."}, {"review_id": "u1DLtZwiHQ", "reviewer": "Reviewer_K9V5", "summary": "A novel method named RUNC (Robust Uncertainty Calibration) is proposed in this paper. The core idea of RUNC is to achieve more robust semantic matching by quantifying and calibrating uncertainty. This approach specifically addresses the unreliable matching results found in existing methods for Composed Image Retrieval tasks, a limitation primarily stemming from aleatoric and epistemic uncertainty. Aleatoric uncertainty stems from the data itself, while epistemic uncertainty is typically caused by overconfidence in dominant semantic categories. RUNC introduces Uncertainty Quantification and Semantic Calibration, alongside an Implicit Concept Matching module, to address these challenges.", "questions": "1. The specific network structure of the \"Uncertainty Perceptron\" module, including its number of layers, dimensions, and activation functions, is not sufficiently detailed in the main text. Could the authors provide a more precise architectural description of this component?\n2. Given that the paper's core is \"Robust Uncertainty Calibration,\" how robust is RUNC to input noise (e.g., slight corruption in reference images, typos in modification texts) or even adversarial attacks, beyond handling inherent data uncertainty? Does the model's uncertainty estimation remain accurate under these conditions?\nIf the authors can provide convincing experimental results demonstrating RUNC's robustness to input noise and adversarial attacks and accuracy of uncertainty estimation under these conditions, my Significance scores will increase.\n3. Considering that RUNC introduces additional modules, could the authors provide a more detailed analysis of its computational efficiency, including comparisons of training and inference times against key baselines? This would strengthen the argument for its practical applicability.\n4. The explanation of the implicit query's generation process, \"virtual guidance,\" could be more concrete. Specifically, could the authors clarify whether this virtual guidance is a learnable parameter, how it is initialized, and what mechanisms ensure its effectiveness in distilling visual semantics without redundant textual information? A pseudo-code or more detailed algorithmic steps would be highly beneficial.\nIf the authors can provide a clear and precise architectural description for the \"Uncertainty Perceptron\" module and offer a concrete explanation of the implicit query's generation process and mechanisms, my Clarity score will increase.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths：\n1. The RUNC framework, based on high-order Normal Inverse Gamma distributions for quantifying aleatoric uncertainty and epistemic uncertainty, is built upon a solid theoretical foundation.\n2. The ablation studies are very thorough, clearly demonstrating the positive contributions of RUNC's various components: Uncertainty Estimation, Uncertainty-Guided Semantic Calibration, Implicit Concept Matching, and Semantic Covariance, to the overall performance.\n3. The mathematical principles and workflow of RUNC's various modules – Uncertainty Perceptron, Uncertainty-Guided Semantic Calibration, Penalty on Misleading Evidence, and Implicit Concept Matching – are described in detail, complemented by an intuitive framework diagram (Figure 2).\n4. The paper innovatively introduces the NIG distribution, utilized for uncertainty quantification in evidential deep learning, into the specific task of CIR, skillfully integrating it with CIR task characteristics for handling semantic covariances and imbalanced semantics.\n\nWeaknesses:\n1. The theoretical analysis of \"high-order conjugate prior distributions\" could be more rigorously elaborated in the main text or provided as a more comprehensive overview in the supplementary material.\n2. A more comprehensive comparison with some of the latest SOTA methods is needed to more fully demonstrate the competitiveness of the proposed method.\n3. The specific network structure of the \"Uncertainty Perceptron\" module, including its number of layers, dimensions, and activation functions, is not sufficiently detailed in the main text. The explanation of the implicit query's generation process could be more concrete, including whether it is learnable, how it is initialized, and how its effectiveness in distilling visual semantics without redundant textual information is ensured.", "was_revised": true, "final_justification": "After careful consideration of their responses and alignment with fellow reviewers, I will maintain my original rating.", "rebuttal": "***Q1/W3: Architectural description of uncertainty perception.***\n\n**A1:** Thank you for this valuable suggestion. The uncertainty perceptron module is implemented with two linear layers followed by a Softplus activation function.  The dropout rate is set as 0.2. The input and hidden dimensions, denoted as $N_q$, are set equal to the number of learnable queries, fixed at 32 in our implementation. The output dimension is $4N_q$, which is split to predict the NIG parameters $\\gamma$, $\\nu$, $\\alpha$, and $\\beta$. With the guidance of semantic covariances of candidates, the inferred probabilistic distributions from the uncertainty perceptron estimate aleatoric and epistemic uncertainty in each candidate and emphasize learning on uncertain negative images. We would add a detailed implementation of the uncertainty perceptron in the revised version.\n\n***Q2: How robust is RUNC to input noise.***\n\n**A2:** Thanks for raising this important question. We would address your concerns as follows:\n- Our work focuses on quantifying semantic uncertainty in composed image retrieval using a Normal-Inverse-Gamma (NIG) distribution to model both aleatoric and epistemic uncertainty. **Aleatoric uncertainty** arises from vague modification expressions, low-resolution references, or noisy annotations, while **epistemic uncertainty** reflects overconfidence in dominant semantics. When input noise (e.g., image corruption or typos) disrupts semantic alignment, our model increases predicted uncertainty, avoiding overfitting to corrupted inputs.\n- To further investigate the robustness of our proposed model, we have conducted additional experiments in FashionIQ under two categories of noise: (1) Visual noise (V): introduce Gaussian noise to obtain the jittered visual representations. (2) Textual noise (T): randomly swap the characters of words in modifications. The compared results in two noise settings reveal that the model demonstrates greater sensitivity to textual noise, which significantly disrupts the users' core requirements, particularly in attribute modifications. As for the visual noise, the reference image inherently contains contradictory and redundant semantics. With our designed uncertainty-guided calibration and implicit concept alignment, the proposed model could quantify the uncertain semantic alignments and utilize reliable alignment via implicit matching to compose robust features. Generally, these results support the robustness of our framework even in the presence of externally introduced noise.\n\nModels|Dress|Shirt|Toptee|Avg R@50 \n----|----|----|----|----\nBaseline(V)|72.83|71.77|74.34|72.98\nOurs(V)|73.14|73.36|77.24|74.58\nBaseline(T)|70.11|70.09|75.44|71.88\nOurs(T)|71.44|72.57|77.77|73.93 \n\n***Q3: More analysis of computational efficiency, including comparisons of training and inference times.***\n\n**A3:** Thank you for the suggestion. To address the concern of computation costs, we compare the training time and inference time in the table below. The comparison results are tested in a single A800 GPU on FashionIQ, and the “training time” means the average training time for all the triplets in one epoch. Specifically, CaLa[1] incorporated twin visual compositors and constructed complementary associations, which introduces computation latency both in training and inference time. Compared with SPRC[2], our proposed model brings limited computation complexity during training, since we only introduce a light-weight uncertainty-based perceptron and the implicit matching shares similar structures with the query fusion. As for the inference stage, the candidate features are indexed from the frozen visual encoders in advance to save inference time. Thus, the implicit concept matching is disabled, and the matching scores can be directly calculated from the cosine distance between the query fusion feature and the candidate features. Generally, our proposed model **exhibits computational efficiency comparable to state-of-the-art methods**.\n\nModels|Training Time(s)|Inference Time(s) \n----|----|----  \nCaLa[1]|263.1|45.7\nSPRC[2]|209.5|27.2\nOurs|214.3|27.4\n\n***Q4/W3: The explanation of the implicit query generation process.***\n\n**A4:** Thank you for this insightful comment.  The virtual guidance $\\boldsymbol{p}^\\ast$  is initialized with a zero-mean Gaussian distribution with a standard deviation specified by the Q-Former configuration, and it is updated during end-to-end training. The detailed algorithm steps are:\n\n1.Initialize learnable embedding $\\boldsymbol{p}^\\ast$ with Gaussian distribution.\n\n2.Expand $\\boldsymbol{p}^\\ast$ to batch size.  # keeps the same shape size as $\\boldsymbol{p}$.\n\n3.Generate implicit query $\\boldsymbol{q}^{*}$： use shared text encoders to integrate modification text tokens and learnable virtual guidance $\\boldsymbol{p}^\\ast$ and normalize the output $\\boldsymbol{q}^\\ast$.\n\n4.Compute cosine similarity scores $\\boldsymbol{s}^\\ast$ based on implicit fusion query $\\boldsymbol{q}^\\ast$ and target embeddings $\\boldsymbol{t}$.\n\n5.Compute contrastive loss $\\mathcal{L}\\_{ce}$, alignment loss $\\mathcal{L}\\_{al}$, and orthogonal loss $\\mathcal{L}\\_{ort}$.\n\nThe effectiveness in distilling informative visual semantics is ensured by the joint cooperation of constraints and symmetrical design to provide implicit guidance. Specifically, $\\mathcal{L}\\_{ce}$ align targets with the implicit query $\\boldsymbol{q}^\\ast$. Through the shared encoders and similar inputs, the learnable virtual guidance $\\boldsymbol{p}^{*}$ is expected to maintain the semantics consistent with the targets. Then the orthogonal loss $\\mathcal{L}\\_{ort}$ forces virtual guidance $\\boldsymbol{p}^\\ast$ to be independent with text embeddings $\\boldsymbol{m}$, to **remove overlapping semantics with the text in $\\boldsymbol{p}^\\ast$  and preserve maximum information from the reference images**. $\\mathcal{L}\\_{al}$ further aligns virtual guidance $\\boldsymbol{p}^\\ast$ with the original visual prompts $\\boldsymbol{p}$. The above process guarantees the distillation of visual semantics without redundant textual information.\n\nBesides, ablation experiments in Table 3 and visualization of implicit matching in Figure 6,7, and 8 also demonstrate that this design could reduce redundant information in the prompt, ensuring efficient extraction of information that matches the target.\n\n***W1: Theoretical analysis of high-order conjugate priors needs more rigor.***\n\n**A5:** Thank you for your suggestions. The conjugate prior is defined as: if prior $p(\\theta)$ and likelihood $p(x \\mid \\theta)$ form a conjugate pair, the posterior $p(\\theta \\mid x)$ remains in the same family. High-order conjugate priors extend this consistency to complex likelihoods. The NIG is a second-order conjugate prior modeling unknown mean $\\mu$ and variance $\\delta^2$.\n\nGiven data $\\boldsymbol{x} \\sim \\mathcal{N}(\\mu, \\delta^2)$, the likelihood is: $p(\\boldsymbol{x}|\\mu, \\delta^2) = \\frac{1}{\\sqrt{2\\pi\\delta^2}}\\exp \\left(-\\frac{(x-\\mu)^2}{2\\delta^2}\\right)$.\nThis can be rewritten as an exponential family form: $p(x|\\boldsymbol{\\theta}) \\propto \\exp\\left( \\begin{bmatrix} \\frac{\\mu}{\\sigma^2} \\ -\\frac{1}{2\\sigma^2} \\end{bmatrix}^\\top \\begin{bmatrix} x \\ x^2 \\end{bmatrix} + \\Phi(\\boldsymbol{\\theta}) \\right)$, with $\\boldsymbol{\\theta}=(\\mu, \\delta^2)$ and sufficient statistics $\\mathbf{T}_x=(x, x^2)$.\nThe prior (as in Eq. 4) is: $p(\\mu, \\delta^2) \\propto (\\delta^2)^{-(\\alpha + \\frac{1}{2}) - 1} \\exp\\left(\\begin{bmatrix} \\frac{\\nu\\gamma}{\\sigma^2} \\ -\\frac{\\nu}{2\\delta^2} \\end{bmatrix}^\\top \\begin{bmatrix} \\mu \\ \\mu^2 \\end{bmatrix} - \\frac{1}{\\delta^2} \\left( \\beta + \\frac{\\nu\\gamma^2}{2} \\right)\\right)$.\n\nAs the kernel of the NIG distribution matches the exponential family form of the Gaussian likelihood, the posterior distribution remains NIG. Besides, the updates of its hyperparameters rely on second-order statistics. The **detailed updation of parameters of $\\gamma,\\nu, \\alpha, \\beta $ could be referred to Section A.3 in the supplementary details**. \n\nIn our proposed RUNC, the NIG prior models full posteriors over mean and variance, providing richer uncertainty than point or diagonal-Gaussian models. This hierarchical prior captures both aleatoric and epistemic uncertainty (Eq. 6, 7) and reflects prediction confidence and ambiguity spread. We will provide more theoretical analysis in the final version.\n\n***W2: More comprehensive comparison with some of the latest SOTA methods.***\n\n**A6:** Thank you for your suggestions. We have added overall results comparison with CoPE [3] and CCIN [4] in the table below. Specifically, CoPE used Gaussian distributions to represent queries and targets. CCIN utilized LLM-based analysis to identify conflicting attributes and generate instructions.  Our RUNC still demonstrates competitive performance on two datasets. Though R@5 and R@10 scores of CCIN[5] are slightly higher, CCIN introduced a Large Language Model to facilitate analysis on conflict semantics and performed LoRA fine-tuning on the ViT backbone. In comparison, our proposed model is a concise end-to-end training framework dedicated to distribution calibration based on uncertainty.\n\nMethods|FIQ R@10|FIQ R@50|FIQ Avg.|CIRR R@1|CIRR R@5|CIRR R@10|CIRR R@50|CIRR Avg.\n----|----|----|----|----|----|----|----|----\nCoPE[1]|44.50|68.60|56.55|49.18|80.65|89.86|98.05|79.55\nCCIN[2]|54.41|74.76|64.59|53.41|84.05|91.17|98.00|81.66\n**Ours**|**55.52**|**76.23**|**65.88**|**53.81**|83.47|91.11|**98.22**|81.65\n\nRefs:\n\n[1] Jiang et al. CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval. SIGIR 2024.\n\n[2] Bai et al. Sentence-level Prompts Benefit Composed Image Retrieval. ICLR 2024.\n\n[3] Tang et al. Modeling Uncertainty in Composed Image Retrieval via Probabilistic Embeddings. ACL (1) 2025.\n\n[4] Tian et al. CCIN: Compositional Conflict Identification and Neutralization for Composed Image Retrieval. CVPR 2025.\n\nWe sincerely appreciate the time and effort in reviewing our manuscript."}, {"review_id": "0jr4tDaWb6", "reviewer": "Reviewer_gQCa", "summary": "This paper addresses the challenge of semantic ambiguity and uncertainty in composed image retrieval (CIR), where a retrieval query consists of a reference image and a modification text. The authors identify that prior contrastive learning-based methods often fail to ensure reliable matching due to both aleatoric and epistemic uncertainty arising from implicit semantic overlaps and dataset biases. To tackle this, they propose RUNC (Robust UNcertainty Calibration), a novel framework that estimates and calibrates uncertainty using evidential priors, such as Normal-Inverse Gamma distributions, to balance semantic alignment. RUNC also introduces orthogonal constraints between explicit textual embeddings and implicit query signals to reduce redundancy and ambiguity in the fused query representation. Experiments on benchmark datasets like FashionIQ and CIRR demonstrate that RUNC achieves more robust and reliable retrieval rankings.", "questions": "1.Is the Gaussian and i.i.d. assumption on semantic matching scores reasonable? The method assumes that semantic similarities follow an independent Gaussian distribution, which may not hold in real-world multimodal data with contextual and correlated semantics. Have the authors tested the robustness of the model when this assumption is relaxed?\n2.How essential is Q-Former to the overall performance? The Q-Former is reused from BLIP-2 with limited adaptation. Could the authors provide a baseline using simpler fusion (e.g., concatenation or attention) to clarify its actual impact?\n3.Are the learnable prompts in Q-Former interpretable? Given the role of learnable prompts in query construction, could the authors provide attention maps or qualitative analyses to show what these prompts are attending to?\n4.Could a progressive ablation be added to show module synergy? Instead of only showing removal of modules, a progressive build-up of the model (e.g., Base → +UE → +UGC → Full) would help demonstrate how each component contributes incrementally.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "This paper proposes a novel uncertainty-aware composed image retrieval framework that integrates Gaussian regression modeling with implicit query supervision. The method demonstrates solid technical quality, with well-structured theoretical derivations and comprehensive ablation studies. The writing is generally clear, though the explanation of some component abbreviations (e.g., UE, UGC) could be improved.  The proposed approach addresses practical challenges in multi-modal retrieval, such as semantic ambiguity and label imbalance, and shows competitive performance on two benchmarks. The use of Normal Inverse Gamma distribution for uncertainty modeling and virtual guidance for implicit query alignment are relatively novel, although further clarification of the difference from prior contrastive methods would enhance the contribution. Overall, the paper is technically sound and presents meaningful and original contributions to the field.", "was_revised": true, "final_justification": "After carefully reading the authors' response, I found that for Q1, the authors addressed my confusion regarding the independent and identically distributed Gaussian distribution assumption. Their model does not assume global independence, but instead uses a normal-inverse gamma (NIG) distribution to model the uncertainty of each query, which aligns well with multimodal contextual dependencies. For Q2, the authors provided compelling experiments demonstrating the role and necessity of the Q-Former. For Q3, the authors added qualitative analysis in the supplementary materials, enhancing the interpretability of the learnable cues. For Q4, the authors demonstrated the cumulative benefits of their proposed module through comprehensive ablation experiments, validating the effectiveness of their approach. Overall, the authors provided a clear response, addressing key concerns, resolving my concerns, and improving the persuasiveness of their article.", "rebuttal": "***Q1: Is the Gaussian and i.i.d. assumption on semantic matching scores reasonable? The method assumes that semantic similarities follow an independent Gaussian distribution, which may not hold in real-world multimodal data with contextual and correlated semantics.***\n\n**A1:** Thank you for this insightful question. We agree that semantic similarities in multimodal data may exhibit contextual correlations. This is also why we incorporate semantic covariance $\\boldsymbol{c}$ to measure the correlation of semantic concepts. However, our proposed model **does not assume global i.i.d. Gaussian distributions across all semantic pairs**. Rather, we follow the evidential modeling paradigm [1,2] using the Normal-Inverse-Gamma (NIG) distribution to **capture uncertainty in semantic matching for each query sample**. Specifically, for the $i$-th instance, we consider the semantic correlations with all candidate samples as a random variable  $\\boldsymbol{c}_i$ and is sampled from a Gaussian distribution $\\boldsymbol{c}_i \\thicksim \\mathcal{N}(\\mu_i, \\delta_i^2).$ We further model the uncertainty in both $\\mu_i$ and $\\delta_i^2$ via a Normal-Inverse-Gamma prior. This assumption does not require independence across different samples, but rather treats the matching distribution for each query as conditionally independent.\n\nFitting semantic features in multimodal data via high-dimensional Gaussian models has been **widely adopted in various models** in fields such as vision, language, and image-text alignment [3,4,5]. In this work, we incorporate NIG distribution to quantify the uncertainty in semantic alignment, where the mean could be regarded as the semantic center, and the covariance and statistical moments in Eq.6 characterize uncertainty. Using distribution matching instead of point vectors could help alleviate overfitting issues and improve the robustness of the model. Besides, high-order conjugate prior for the Gaussian distribution could **effectively capture the \"one-to-many\" semantic correlations**, which is consistent with the nature of the multi-modal retrieval process.\n\n***Q2: How essential is Q-Former to the overall performance? Could the authors provide a baseline using simpler fusion (e.g., concatenation or attention) to clarify its actual impact?.***\n\n**A2:** Thank you for your questions. Q-Former in our proposed RUNC serves as a fusion module to extract aligned visual-textual semantics from the reference image and modification sentences. We would like to clarify that the **Q-Former is not the core contribution and novelty of our method**. Rather, we employ Q-Former in this work primarily for its strong capability to **extract semantically rich visual embeddings** from pre-trained frozen vision encoders, which facilitate semantic alignments to mitigate the heterogeneous gap. Besides, the employment of the Q-Former structure is in line with advanced models such as CaLa[6] and SPRC[7]. Our contribution concentrates on uncertainty-guided semantic alignment and calibration on the similarity distributions built on top of the multi-modal query fusion.\n\nTo evaluate the impact of Q-Former, we have conducted additional experiments replacing the Q-Former with a simpler fusion mechanism, as shown in the Table below. “Concat” means concatenation of image and text features followed by a shared MLP. It can be seen that simple fusion strategies underperform our proposed approach in both datasets, as they have difficulty in fully capturing and integrating complex visual and linguistic features. Nonetheless, our uncertainty-aware mechanism does not depend on the Q-Former structure and is compatible with alternative fusion strategies.\n\nModels | FIQ R@10 | FIQ R@50 | FIQ Avg. | CIRR R@5 | CIRR $\\mathbf{R}_{subset}@1$ | CIRR Avg.\n ----  | ----  |----  |---- |---- |---- |----\nConcat  | 47.67 | 69.75 | 58.71 | 80.15 | 76.16 | 78.16\nSingle Attention Layer | 48.97  | 71.63 | 60.30 | 80.94 | 77.03 | 78.99\nQ-Former  | 55.52 | 76.23 | 65.88| 83.47 | 80.87 | 82.17\n\n***Q3: Are the learnable prompts in Q-Former interpretable? Given the role of learnable prompts in query construction, could the authors provide attention maps or qualitative analyses to show what these prompts are attending to?***\n\n**A3:** Thank you for raising this important concern. In our implementation, the Q-Former takes the input of image embeddings of the reference image from a frozen ViT encoder, text tokens of modification sentences, and a fixed set of learnable query tokens. The learnable prompts $\\boldsymbol{p}$, as the outputs of Q-Former, encode attentive visual images by utilizing the initialized learnable queries to attend to the image regions for aligning with texts. For this specific task, the learnable prompts $\\boldsymbol{p}$ target at **enhancing visual representations of the semantics preserved in the reference images** (e.g., basic outlines of the fashion items, main objects and species, the reversed attributes).\n\nAttention maps and qualitative analysis are essential for providing a vivid presentation of learnable prompts. Due to the length limit of the main manuscript, we presented **activation heatmaps by GradCAM in Figure 8 in the supplementary materials**. As shown in the first example of Figure 8, the learnable prompts $\\boldsymbol{p}$ highlight the area corresponding to the dog, which is coherent with the reference images. In contrast, the text features $\\boldsymbol{m}$ underlines the area of the yellow pillow on the couch. The collaboration between $\\boldsymbol{p}$ and $\\boldsymbol{m}$ enables the model to accurately select the ground-truth image from other candidates, especially enhancing the perception of object contours, categories, or backgrounds provided by the reference image in $\\boldsymbol{p}$.\n\n***Q4: Could a progressive ablation be added to show module synergy?***\n\n**A4:** Thank you for your suggestions. To address your concern, we have conducted ablative experiments by incrementally adding modules to the base model in the table below. “UE” refers to adding the constraints of $\\mathcal{L}^{NLL}$ proposed in the uncertainty estimation. “UGC” refers to assigning uncertainty coefficients to calibrate the distributions based on the quantified uncertainty scores. “ICM” refers to adding the Implicit Concept Matching module to guide the reference images. Note that the Uncertainty-Guided Calibration(UGC) is not an independent module, and instead is built on the Uncertainty Estimation(UE) module to penalize uncertainty samples adaptively on the estimated uncertainty scores. \n\nThe qualitative results of adding UGC bring the most remarkable results when comparing other individual modules, suggesting the **significance of considering uncertainty in match scores** and emphasizing contrastive learning from uncertain instances. The UE module introduces the NIG distribution to fit the semantic covariances, which is the basis of distinguishing uncertain samples. Though it does not directly refine the similarity distributions, it still brings improvements especially in fashion dataset, as the semantic correlations serve as soft supervisions to promote latent alignment learning. The growth of ICM also implies that implicit concept matching could mitigate the ambiguity of fusion queries. The integration of all the modules aims to alleviate uncertainty issues in query compositions and query-to-target matching.\n\nModels | FIQ R@10 | FIQ R@50 | FIQ Avg. | CIRR R@5 | CIRR $\\mathbf{R}_{subset}@1$ | CIRR Avg.\n ----  | ----  |----  |---- |---- |---- |----\nBaseline  | 51.45  |  73.09  | 62.27 |  79.08  |  75.48  | 77.28\n+UE  |  53.28  | 75.11 |  64.20  |  79.79  |  77.52  |  78.66\n+UGC  |  54.76  | 75.22 |  64.99  |  82.47  |  79.15  |  80.81\n+ICM  |  53.07  | 74.77 |  63.92  |  81.51  |  78.23  | 79.87\nOurs(all)  | 55.52 | 76.23 | 65.88| 83.47 | 80.87 | 82.17\n\nRefs:\n\n[1] Sensoy et al. Evidential Deep Learning to Quantify Classification Uncertainty. NeurIPS 2018: 3183-3193\n\n[2] Amini et al. Deep Evidential Regression. NeurIPS 2020\n\n[3] Kendall et al. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? NIPS 2017: 5574-5584\n\n[4] Chun et al. Probabilistic Embeddings for Cross-Modal Retrieval. CVPR 2021: 8415-8424\n\n[5] Huang et al. Cross-Modal Recipe Retrieval With Fine-Grained Prompting Alignment and Evidential Semantic Consistency. IEEE Trans. Multim. 27: 2783-2794\n\n[6] Jiang et al. CaLa: Complementary Association Learning for Augmenting Comoposed Image Retrieval. SIGIR 2024: 2177-2187\n\n[7] Bai et al. Sentence-level Prompts Benefit Composed Image Retrieval. ICLR 2024\n\nWe sincerely appreciate your review of our manuscript"}, {"review_id": "2XWD2salqm", "reviewer": "Reviewer_gEs2", "summary": "This paper proposes to adopt uncertainty estimation into Composed Image Retrieval problems to address the problem of ambiguous and imbalanced semantics of the bi-modal queries. High-order evidential priors are employed for semantic distribution adjustment. Orthogonal independent constraints are applied to two branches of queries for disambiguation. Experiments on common datasets show its effectiveness under multiple metrics.", "questions": "1. Why is it necessary to adopt an uncertainty estimation method for the problem?\n2. Why does the learnable prompts have unclear semantic concepts (Sec. 3.3)? Is there any evidence?\n3. What are the differences between the two branches of frozen and trainable text encoders? What are the differences between Eq.(1) and Eq.(11)? The variables are matched and using one single branch may also work.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n - The method was rigorously designed and theoretically analyzed to solve the significant uncertainty problem in CIR with great novelty. \n - The Experiments section have compared the proposed method with numerous methods on various metrics, while presenting ablations and discussions on both the proposed components and uncertainty estimation.\n\nWeaknesses:\n1. Clarity and motivation: \n - Some parts lack detailed mathematical definitions (assumption at Line 161-162), motivation statements, and citations of the core method used (Section 3.2), making the method section hard to read. \n - Temperature $\\tau$ is often writen as divided rather than multiplied due to it's meaning. In the paper it is written as multiplied.\n2. There is no subscript for variables in Eq.(12). Meanwhile, it is recommended to explain the meaning and effect of Eq.(12) as it's good for the clarity of the proposed Implicit Concept Matching.", "was_revised": true, "final_justification": "The rebuttal clarified the motivation of key designs in their work, provided experiments to justify the paper's declarations, and improved on many technical issues. Therefore, I decide to raise my score to accept.", "rebuttal": "***Q1: Why is it necessary to adopt an uncertainty estimation method for the problem?***\n\n**A1:** Thank you for raising this important problem. We clarify the significance of uncertainty estimation and our design as follows:\n- In the setting of composed image retrieval, candidate images **exhibit underlying semantic overlaps** with the targets as the images inherently contain rich semantic concepts. This phenomenon is more pronounced in fashion datasets, potentially leading to false negatives. The latent correlation between the matched and mismatched instances causes difficulty for the model to discriminate the positives from the negatives. Thus, we estimate the uncertainty scores of instances to enhance the distinction of hard negatives. \n-  The imbalanced distributions of semantic concepts in visual features tend to cause **overfitting and overconfidence in dominant semantics**. Thus, the ranking results are inclined to be misguided by partially related semantics while ignoring discriminant details as the user requires (as seen in Figure 1(left)). The overconfidence in partial characteristics could be quantified by **epistemic uncertainty** based on the evidential learning theory.\n- Vague expression of modification requirements (query texts), low-resolution and totally irrelevant reference images (query image), and noisy labels (some matched images are not annotated as positives) introduce **aleatoric uncertainty (uncertainty in data)** in this task and motivate us to capture the intrinsic semantic alignments while reducing the uncertainty in similarity distributions.\n\n***Q2: Why do the learnable prompts have unclear semantic concepts (Sec. 3.3)? Is there any evidence?***\n\n**A2:** Thank you for raising this important concern. Please let us further clarify the generation of learnable prompts $\\boldsymbol{p}$. Q-Former structure employs the initialized learnable queries to attend to the image regions for aligning with texts. In the setting of composed image retrieval, the modifiers specify changes to reference images. In other words, the input image (reference image) and the input text (modification text) **are not semantically aligned**, and they usually have contradictory semantics. However, 1) the Q-Former structure based on multi-head attention mechanisms attempts to align the query token with the image patches and **forces the alignment between the inconsistent input images and texts**. 2) Since the query token of Q-Former is a trainable embedding that attempts to capture local semantics of the image and further fuse with texts, the composed feature is inclined to **be misleadingly guided and reserves visual regions irrelevant to the targets** when the textual description contradicts the image content. 3) The traditional contrastive loss simply pushes apart mismatched pairs and brings matched pairs closer, which **lacks strong supervision signals** to penalize embeddings that contain incorrect semantics. The above reasons theoretically lead to the unclear semantic concepts, which motivate us to design the implicit concept matching (ICM) module to minimize the ambiguity when fusing queries. \n\nBesides, the comparison of aleatoric uncertainty scores of the complete RUNC in comparison with the model without the ICM module. The model without ICM generally has large uncertainty scores and struggles with training convergence, implying that the learnable prompts inherently have unstable and ambiguous semantic representation, and the proposed model could alleviate the uncertainty issues by incorporating implicit guidance.\n\nModels|Epoch 0|Epoch 5|Epoch 10|Epoch 15|Epoch 20\n----|----|----|----|----|----  \nw/o ICM|0.902|0.614|0.334|0.199|0.209 \nw ICM|0.692|0.234|0.118|0.086|0.086\n\n***Q3: What are the differences between the two branches of frozen and trainable text encoders? What are the differences between Eq.(1) and Eq.(11)?***\n\n**A3:** Thanks for raising this question. We apologize for the confusion regarding the design of the implicit concept matching (ICM) module. In our proposed RUNC, one branch yields visual prompts $\\boldsymbol{p}$ and fusion query $\\boldsymbol{q}$ for uncertainty-based matching, while the other branch (i.e., ICM) introduces symmetrical embeddings to guide prompts $\\boldsymbol{p}$, aiming to reduce uncertainty in $\\boldsymbol{q}$. Both branches share the same text encoder architecture, and ICM uses shared weights to ensure feature consistency and avoid noisy gradients. We will revise Figure 2 and notations to improve clarity.\n\nRegarding Eq.1 and Eq.11: Eq.1 defines a standard contrastive loss on matching scores $\\boldsymbol{s}$ between fusion query $\\boldsymbol{q}$ and target $\\boldsymbol{t}$, but it **is not used in our framework**, as it treats all negative samples equally and overlooks latent semantic overlap. Instead, we use Eq.8 to adjust the candidate distribution based on estimated uncertainty, penalizing uncertain instances. In contrast, Eq.11 **imposes constraints on $\\boldsymbol{s}^\\ast$** to distill visual semantics from the target $\\boldsymbol{t}$ to the implicit query $\\boldsymbol{q}^\\ast$ in the ICM module. This enhances the representation of $\\boldsymbol{p}^\\ast$, which shares the same shape as prompt $\\boldsymbol{p}$, and guides its learning via $\\mathcal{L}_{al} = \\lVert \\boldsymbol{p} - \\boldsymbol{p}^\\ast \\rVert$. Removing the ICM branch results in prompts retaining more noise and irrelevant features. Ablation results in Table 3 confirm that the implicit query branch effectively distills target-aware visual elements and strengthens alignment through independent constraints with text embeddings.\n\n***W1: Detailed mathematical definitions (assumption at Line 161-162), motivation statements, and citations of the core method used (Section 3.2)***\n\n**A4:** Thank you for your valuable feedback. We are sorry for the confusion in the uncertainty estimation. \n\nGiven the initial matching scores between the composed queries and candidate instances, the Uncertainty Estimation module in Section 3.2 aims to maximize the likelihood of our observations. In the context of evidential learning [1,2,3], the latent semantic correlations are drawn i.i.d from distributions with unknown mean and variance parameters. The detailed mathematical definitions could be referred to Eq. 3 as: $\\boldsymbol{c}_i \\thicksim \\mathcal{N}(\\mu_i, \\delta_i^2)$, where $\\boldsymbol{c}$ means the semantic covariance scores. \nThe core motivation for this part lies in **semantic overlaps within candidate images**. As Figure 1(left) shows, negative candidate image $\\boldsymbol{I}\\_4$ shares strong semantic correlations with the targets $\\boldsymbol{I}\\_ {gt}$ and user requirements, while $\\boldsymbol{I}_4$ is annotated as negatives. The matching score between $\\boldsymbol{I}_4$ and the query is expected to be 0 in traditional contrastive loss, hindering the semantic distance measurement in the latent space. Instead, we employ the semantic covariances $\\boldsymbol{c}$ as a soft supervision to guide the distributions of match scores, and quantify the uncertainty scores for candidates from evidential learning. We would reorganize the beginning of Section 3.2 to improve the readability of the method and add corresponding works in the illustrations. \n\n***W2: Temperature is often written as divided rather than multiplied.***\n\n**A5:** Thanks for your advice. We are sorry for the lack of standardization in the formulas and have revised the corresponding formulas as:\n$\\mathcal{L}\\_{cl}=-\\frac{1}{B} \\sum\\_{i=1}^{B} \\log \\frac{\\exp(\\boldsymbol{s}\\_{ii} /\\ \\tau)}{\\sum\\_{j=1}^{B} \\exp(\\boldsymbol{s}\\_{ij}/\\\\tau)}$ (1), $\\mathcal{L}\\_{bc}=-\\frac{1}{B} \\sum\\_{i=1}^{B} u_i \\cdot \\log \\frac{\\exp(\\boldsymbol{s}\\_{ii} /\\ \\tau)}{\\sum\\_{j=1}^{B} \\exp(\\boldsymbol{s}\\_{ij}) /\\ \\tau }$ (8), and $\\mathcal{L}\\_{ce}=-\\frac{1}{B} \\sum\\_{i=1}^{B} \\log \\frac{\\exp(\\boldsymbol{s}\\_{ii}^* /\\ \\tau)}{\\sum\\_{j=1}^{B} \\exp(\\boldsymbol{s}^*\\_{ij}/\\\\tau)}$ (11).\n\nThese formulas would be updated in the revised version.\n\n***W3: Explanation of Eq.12***\n\n**A6:** Thanks for your questions, and we are sorry for the confusion in Eq.12. The revised formula for Eq.12 is：\n$\\mathcal{L}\\_{ort}=\\sum\\_{i\\neq j} \\left(\\text{Cov}(\\boldsymbol{p}^\\ast, \\boldsymbol{m})\\_{ij}\\right)^2=\\sum\\_{i \\neq j} \\left( \\frac{1}{B} ((\\boldsymbol{p}^\\ast)^\\top \\boldsymbol{m} \\right)\\_{ij})^2$.\n\nThe orthogonal loss in Eq.12 aims to penalize the correlations between features $\\boldsymbol{p}^\\ast$ and $\\boldsymbol{m}$ in different dimensions. In CIR task, $\\boldsymbol{m}$ from text tokens usually have explicit modification requirements, while the reference images are coupled with the visual semantics to be retained, removed, and any redundant information. The constraints in Eq.12 avoid feature redundancy and **compel the implicit query $\\boldsymbol{p}^\\ast $ to learn complementary information and prevent feature overlap with text**. Hence, the implicit query $\\boldsymbol{p}^\\ast$ could further guide the actual visual prompts $\\boldsymbol{p}$ to distill effective retention information from reference images. We would add more illustrations of Eq.12 in the revised version.  \n\n\nRefs:\n\n[1] Malinin et al. Predictive Uncertainty Estimation via Prior Networks. NeurIPS 2018: 7047-7058\n\n[2] Sensoy et al. Evidential Deep Learning to Quantify Classification Uncertainty. NeurIPS 2018: 3183-3193\n\n[3] Amini et al. Deep Evidential Regression. NeurIPS 2020\n\nWe generally thank you for your time and effort in reviewing our manuscript. Thank you again for your valuable feedback."}]

Author rebuttal:

Meta-review: This paper proposes RUNC (Robust UNcertainty Calibration), a novel framework for composed image retrieval (CIR) that explicitly models both aleatoric and epistemic uncertainty in matching bi-modal queries (reference image + modification text) to target images. RUNC employs a high-order Normal-Inverse Gamma (NIG) distribution to estimate semantic covariance among candidate images and uses this uncertainty to calibrate the training objective, promoting balanced semantic alignment. To further refine query representation, an Implicit Concept Matching (ICM) module enforces orthogonality between explicit textual embeddings and implicit visual cues. Experiments on FashionIQ and CIRR datasets demonstrate that RUNC achieves robust and reliable retrieval, outperforming prior state-of-the-art methods. Reviewers note that the framework is technically solid, theoretically motivated, and demonstrates meaningful contributions to multi-modal retrieval. Weaknesses include limited clarity in the connection of ICM to uncertainty calibration, computational overhead of pairwise covariance calculations, and some missing details about module architecture and mathematical assumptions. Despite these, the authors provide thorough ablations, qualitative analyses, and rebuttal clarifications that address key concerns.
Decision: Accept (poster)