SEGA: Shaping Semantic Geometry for Robust Hashing under Noisy Supervision
Authors: Yiyang Gu
Bohan Wu
Qinghua Ran
Rong-Cheng Tu
Xiao Luo
Zhiping Xiao
Wei Ju
Dacheng Tao
Ming Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Robust Hashing
Semantic Geometry
Noisy Supervision
Primary area: applications

Abstract: This paper studies the problem of learning hash codes from noisy supervision, which is a practical yet challenging task. This problem is important in extensive real-world applications such as image retrieval and cross-modal retrieval. However, most of the existing methods focus on label denoising to address this problem, but ignore the geometric structure of the hash space, which is critical for learning stable hash codes. Towards this end, this paper proposes a novel framework named Semantic Geometry Shaping (SEGA) that explicitly refines the semantic geometry of hash space. Specifically, we first learn dynamic class prototypes as semantic anchors and cluster hash embeddings around these prototypes to keep structural stability. We then leverage both the energy of predicted distributions and structure-based divergence to estimate the uncertainty of instances and calibrate the supervision in a soft manner. Moreover, we introduce structure-aware interpolation to improve the class boundaries. To verify the effectiveness of our design, we give the theoretical analysis for the proposed framework. Experiments on a range of widely-used retrieval datasets justify the superiority of our SEGA over extensive strong baselines under noisy supervision.

SEGA: Shaping Semantic Geometry for Robust Hashing under Noisy Supervision
Yiyang Gu Bohan Wu'", Qinghua Ran', Rong-Cheng Tu Xiao Luo
Zhiping Xiao*i, Wei Ju', Dacheng Tao², Ming Zhang
State Key Laboratory for Multimedia Information Processing,
School Computer Science, PKU-Anker LLM Lab, Peking University School of Mathematical Sciences, Peking University
Nanyang Technological University UCLA University of Washington
,yiyanggu,wxtun jurpku,juveiii julpeuujjeeiimaaaae castaannn_s__pe_se edu. cn,
rqh1430stu. pku. edu. cn,
(rongcheng tu, dacheng. tao) edu.sg. xiaoluoêcs ucla edu, patxiaolus, edu
Abstract
This paper studies the problem of learning hash codes from noisy supervision, which is practical yet challenging task. This problem is important extensive real-world applications such as image retrieval and cross-modal retrieval. However, most of the existing methods focus on label denoising to address this problem, but ignore the geometric structure of the hash space, which is critical for learning stable hash codes. Towards this end, this paper proposes a novel framework named Semantic Geometry Shaping (SEGA) that explicitly refines the semantic geometry of hash space. Specifically, we first learn dynamic class prototypes as semantic anchors and cluster hash embeddings around these prototypes keep structural stability. We then leverage both the energy of predicted distributions and structure-based divergence estimate the uncertainty of instances and calibrate the supervision in soft manner. Moreover, we introduce structure-awan interpolation to improve the class boundaries. To verify the effectiveness of our design, we give the theoretical analysis for the proposed framework. Experiments on range of widely-used retrieval datasets justify the superiority our SEGA over extensive strong basclines under noisy supervision.
1 Introduction
Deep hashing methods are significant for large-scale and efficient information retrieval. The goal of these approaches is to generate compact binary codes retaining good semantic similarity [56]. In the past few years, supervised deep hashing has performed well in various retrieval tasks, such However, label noise is still
image retrieval and multi-modal retrieval R4
35
69
32
outstanding challenge when it comes to practical applications (50] 20 [11. In real-world datasets, [8]. These problems may stem from inconsistent human annotations, noisy web collection, loose category definitions, and intrinsic semantic ambiguity [23 (65) The discrete property of hash codes makes this problem worse. Hashing models are sensitive to noisy supervision, where small label errors can cause large output shifts. This leads fragmented representations and unstable similarity preservation 47
labels are often incomplete, ambiguous, or incorrect (68, 60
When supervision
"Equal contribution.
CCorrepoondig authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

is unreliable, models that fit labels directly tend to overfit noise 66 class structure 38). The main challenge in noisy multi-label hashing not only to maintain class separability under weak signals but also to build an embedding space that is semantically consistent, geometrically smooth, and robust to supervision noise.
14
instead of learning real
In order to alleviate this problem, previous studies mainly focused on denoising labels or regularizing predictions. These include approaches that reweight or clear training samples [39 estimate noise transition matrices 43 apply auxiliary consistency constraints [68], or perform confidence- baadd
sample selection 20 31 These strategies have been proven to be effective in reducing the impact of noisy supervision, mainly by improving the quality of labels or learning stability. However, they basically follow a label-centric paradigm, assuming the label, whether observed or corrected, remains the main source of supervision. In contrast, we propose complementary perspective that focuses on modeling and reinforeing the semantic geometry of the representation space. The goal of deep hashing is to encode the inputs into discrete codes that reflect semantic similarity. The learned hash codes may be suboptimal when hash embeddings fail to align with the underlying semantic manifolds. In contrast, the semantical and topological consistency of the hash space can make the hash codes more robust against label noise. It raises two key challenge to build such a geometry-aware framework for robust hash learning: (I) How can we evaluate the uncertainty of supervision accurately in the presence of label noise? It is difficult to determine the uncertainty of supervised signals from traditional confidence measures under heavy label noise. However, can capture additional cues of uncertainty from the geometric structure of representations. (2) How to capture the underlying semantic structure of data effectively under noisy multi-label supervision? It is promising to shape the semantic geometry of hash space to align the semantic relationship among the instances.
Towards this end, this paper proposes a novel framework named Semantic Geometry Shaping (SEGA) that explicitly refines the semantic geometry of hash space. It integrates three mutually promot- ing components into a closed-loop training system. First, Prototype- Guidedd Semantic Anchoring aligns instance embeddings with dynamic class prototypes that evolve during training, which serve Guuiddd as structural anchors. Second, Uncertainty- Guided Supervision Calibration calculates unified uncertainty score by combining energy-based prediction confidence and structure- based divergence, allowing soft weighting of noisy labels. Third, Structure-Aware Mixup interpolates between samples with different uncertainties but similar semantics to refine ambiguous regions, so as to promote continuity and improve decision boundaries. These modules enable representations, supervision, and uncertainty to coevolve in a tightly coupled geometric loop. We have verified SEGA from both theoretical and empirical aspects, and the results show that it can approximately evaluate the reliability of labels to achieve supervised weighting, and maintain semantic consistency in the interpolation process, thus reducing structural fragmentation and regularizing decision boundaries under noise. Experiments on noisy multi-label hashing benchmarks show that SEGA achieves state-of-the-at robustness, particularly under extreme label corruption. In addition to numerical improvement, the visualization results also confirm that SEGA can construct discriminative and semantically coherent representations under noisy supervision. These findings suggest that shaping semantic geometry. rather than merely correcting supervision, is crucial for resilient hash algorithms.
Our main contributions under noisy supervision. The multi-label hashing problem is reformulated as semantic structure alignment rather than simple label recovery. (2) Novel Methodology. We propose SEGA, a closed- loop framework that unifies prototype anchoring, uncertainty-guidde soft labeling, and structure-aware interpolation into a geometry-awan evaluate SEGA on four benchmark datasets. It consistently achieves state-of-the-art performance and shows strong robustness across diverse noisy multi-label settings.
follows. (1) New Perspective. We present a geometric view of learning
semantic learning process. (3) Extensive Experiments. We
2 Related Work
Deep Hashing Methods. Deep hashing has become a core approach for scalable similarity retrieval. learns compact binary codes that preserve semantic relations in Hamming space. Supervised methods usually build pointwise, pairwise, or triplewise objectives based on label-derived simi-
larity 34 69 32 For instance, GreedyHash [51] utilizes the greedy strategy to address the
gradient-vanishing problem in discrete hashing optimization. CSQ 633 proposes a global central similarity metric to improve hash learning efficiency. Unsupervised hashing methods usually leverage the intrinsic relationships in data to avoid label dependence. For example, WCH (62 introduces
2

---

Training
Prototype Priidey Semantic Anchoring
Uncertainty Guinded
Images
Supervision Calibration
Class manifolds
Energy
Geomeetic Isolation
Scalib
Soft Label y;
Binary Hash Code
Bructure-Aware Interpolation
Noisy Labels
Casine Similarity
+1++++1
Deep
+1-1 +1-1
Hashing
Interpolation on
+1-1 +1
Model
Noisy
Clean
Hash
Lmix
Embedding
Econt
Figure 1: An overview of the proposed framework SEGA for robust multi-label hashing under noisy supervision. It integrates prototype anchoring, uncertainty-calinn calbbrtt soft labels, and structure- aware interpolation into closed-loop system that aligns semantic supervision with representation geometry.
weighted contrastive learning and mutual attention mechanism to enhance data similarity mining. These methods perform poorly when the provided supervision contains noise, as they are unable to effectively utilize the useful signals within the noisy supervision reduce the interference caused by the noise. Recent efforts toward noise-robust hashing include DIOR (55) and STAR [4]] DIOR utilizes a dual-partition strategy to relieve the impact of noisy labels. STAR leverages a hybrid sample selection mechanism and selective centroid learning to help capture similarity structure. however, they mainly focus on label denoising and local noise reduction, but overlook the structural fragility of the global hash space. Towards this end, propose novel framework that explicitly refines the semantic geometry of hash space to alleviate the influence label noise.
Learning with Label Noise. Label noise is common in real-world datasets and has motivated
extensive researches on robust learning against noise [50 26 The existing methods can be mainly divided into three categories. The first category of methods refines losses by modeling noise transformation
The second one leverages sample selection to identify clean 31 68) The third one utilizes regularization to alleviate memorization of noisy labels [66] Co-teaching 20 introduced a deal-network design that filters noisy samples through cross- selection of small-loss instances. Mixup [66] provides implicit regularization by linearly interpolating samples and labels, reducing overfitting to corrupted data. Recent advances combine contrastive with noise-robust representations [13 61 and apply curriculum learning to control memorization dynamics M. Early learning regularization 38 limits overfitting by focusing on clean patterns in early training. Despite this progress in image classification, robust learning for structured outputs such as hashing remains less explored, especially under ambiguous multi-labe supervision.
13 45].
data 20
21 [57]
learning 99 127
3 Methodology
3.1 Problem Definition
Let D = {(x,, v:)l be training set of N examples, where x; E RRd is an input instance and y: E {0. 1] is a one-hot or multi-hot label vector over C' semantic categories. In practical settings, these labels are often noisy due to annotation errors, class ambiguity, or missing tags. The goal is to leam hash function H x + b E (-1,++11 that encodes each sample into an L-bit binary code, such that semantically similar inputs mapped nearby codes in the Hamming space.
3.2 Framework Overview
SEGA addresses this goal by shaping a robust semantic geometry in the embedding space, even under noisy multi-labe supervision. We adopt deep hashing model f x r h R to produce continuous hash embeddings, which are binarized as b sign(h). However, directly training on corrupted labels y, often induces semantic misalignment, especially when representations are influenced by unreliable or ambiguous supervision. To address this problem, SEGA shapes the semantic geometry of the representation space with three key parts. First, pwototype-guiddee semantic
3

---

anchoring aligns embeddings with class prototypes to keep the geometry stable. Second, uncertainty- guided supervision calibration reduces the weight of unreliable labels using energy and structural divergence. Third, structure-aware interpolation mixes only semantically similar samples with different uncertainty levels to create smooth transitions near class boundaries. The components form closed loop. Supervision, uncertainty, and geometry co-evolve and reinforce each other through training. The overview of the proposed framework SEGA illustrated in Figure]
3.3 Prototype-Guided Semantic Anchoring
SEGA builds semantic scaffold in the hash space using set of learnable class prototypes (pr) Each prototype Pe E R. is randomly initialized and updated together with the model. These prototypes act anchors that guide instance embeddings toward meaningful semantic directions. For each input 24, we obtain its continuous hash representation h; f(æ:) and normalize as h: = ha/ hi]]- Similarly, we normalize all prototypes to get unit-length semantic anchors: p. = Pe/plppllll. The logit vector for classification then computed as:
(I) which yields cosine similarities between the embedding and each class prototype. To assess the semantic reliability of supervision, we define soft-alignment score that evaluates the agreement between model prediction and the observed (possibly noisy) label vector y::
4. h (p..PP....,pc!.
S(D cos (ar(,,), y:).
where (.)) denotes the softmax function. This score measures how well the model's predicted distribution aligns with the noisy label the prototype-induced space. Based on the gr-th percentile of the alignment scores (s (5()) J we compute dynamic threshold to partition the dataset into
clean set De. containing semantically reliable samples, and a complementary noisy set Dn
Percentile((S (( }:4r); De= = [n ;
( 2 Tr
D₂ = D\
Clean samples in De are supervised using softmax-based prototype alignment loss the noisy label vector as a probabilistic guide:
55 that treats
exp(p% h2)) ブラ=1 exp(p] h.)
Lekean
Σ E.ED
log
(4)
Yik
To regularize noisy samples in D, and prevent semantic drift, we adopt contrastive loss [46 27 54
It pulls embeddings of related samples closer and pushes unrelated ones apart by a margin m. For two samples Fi; æÿ Dn, with noisy labels y; and yj. we define:
Leom
Σ
[Riasl [lh, - h2 1hh--h,1' Meaas) max(0, li,, hn]1)²]
(5)
where [(-5] = 1 y. y > (i.e., positive pair), and |[zy = otherwise (i.e., negative pair).
Prototype-guided alignment provides reliable supervision. Contrastive regularization stabilizes learning in noisy regions. Together, they help build a structured representation space that preserves semantic geometry under label noise.
3.4 Uncertainty-Guide Supervision Calibration
The previous module divides the dataset into clean and noisy parts using prototype alignment. This hard split ignores the reliability differences within each part. Some clean samples lie near decision boundaries or show structural ambiguity. Some noisy samples still contain useful semantic information. To handle this, we introduce novel metric to evaluate uncertainty [44 [1] of samples and then calibrate the supervision in soft manner. Specifically, we combine energy-based prediction confidence and structure-based neighborhood divergence to estimate uncertainty score. We then leverage this score to reduce the weight of supervision for ambiguous samples and filter out unreliable gradients. It also guides later sample selection in structure- interpolation.

---

Energy-based prediction confidence. We measure the confidence of the model using the energy 40 67 of logit distribution:
E, = log Σ exp(
(6)
A high energy value means that the distribution is flat and the prediction is less certain. We normalize
E; min, Ej the energy in each mini-batch as É, max, E,-min, E, so that all values stay comparable scale.
Structure-based divergence. Confidence alone cannot reflect spatial consistency in the embedding space. We therefore define divergence score. measures how far sample spreads from its K nearest neighbors in geometry:
1 Σ cos(h hy).
6, =111 K JEN,
Here, N₁ is the of top-K K nearest neighbors for sample 2,. larger òG means the sample is more isolated in geometry. Such samples often lie near class boundaries or in under-represented semantic zones. Unified uncertainty. The total uncertainty is defined as:
U; (1 F..)
8
This design captures two kinds of cues: vertical cue from prediction confidence and lateral cue from structural smoothness. gives unified measure of semantic reliability. Instead of discarding uncertain samples, we calibrate their gradient impact by softly reweighting the label signal 42]
29)
Cealib
1 D
Σ IEED c=1
Σ
The log(o(4))er
ÿ. y 11-...11.00 -1-U1)
where (-) is the softmax function. This calibration reduces the effect of samples that are semantically or structurally uncertain. improves the shape of semantic manifolds and strengthens alignment inside coherent regions. The resulting gradients mainly update well-supported semantic areas and smooth the uncertain ones without forcing hard boundaries.
Theoretical Analysis. We now give theoretical explanation for our uncertainty- supervision calibration. The uncertainty score U, combines semantic confidence and structural consistency. approximates the posterior probability that the observed label is correct. With a mild conditional independence assumption, 1 U/ can be used as a proper weight in the supervision loss.
Theorem 3.1 (Uncertainty- Weighted Confidence Approximates Label Correctness). Let æ, be a training sample with observed label y; and true (latent) label y, Define the energy-based prediction confidence as E, = - log D. exp(lic), and É, max, E,-mi E,' the structure-bal divergence as 6, 1 EjEN, cos(h,, h2)) ) If we assume the probability that the label correct depends solely on the semantic and structural reliability of the sample, and these two sources are conditionally independent given Di then:
E, E.-min; E,
Pr(y Di
æi] 1 U..
(10)
This result provides a probabilistic interpretation - U, as an estimate of label correctness. By treating semantic confidence and structural consistency as independent indicators of supervision quality, 1 U, serves as a soft surrogate for the reliability of the observed label. Therefore, using 1 U, to weight the label signal helps the model reduce gradients from noisy or ambiguous samples. It keeps stronger supervision for reliable regions. This process aligns the learning dynamics with semantically meaningful structures.
3.5 Structure-Aware Interpolation for Boundary Regularization
Uncertainty-guide calibration reduces noisy gradients but does not handle semantic ambiguity near decision boundaries. Samples these areas are often uncertain and structurally isolated. To solve this, we draw inspiration from Mixup [53 59 37], which smooths class transitions and
5

---

makes representations more generalizable by mixing samples. Based on this idea, we design structure-award interpolation mechanism for refining boundaries noisy multi-label hashing.
Traditional Mixup mixes any two samples without restriction. our method, we mix only pairs that semantically similar but have different uncertainty levels. We split the clean set De into parts: a confident subset Don and an uncertain subset Dun The split is based on the mean uncertainty U:
Deon
[ E De U, S U. Pun {a, De U, U..
(11)
For each uncertain sample E Dun we find its most similar confident sample 23 E Doon The two samples must share least common label, that is, v yj O. We measure similarity by cosine distance in the normalized embedding space:
z} arg
max
cos cos(h, h2).
(12)
2, EDo.. M; y,
This rule keeps interpolation between samples that share some labels. It preserves semantic consis- tency and uses confident samples as anchors to guide uncertain ones. We then generate interpolated virtual samples and soft labels as:
I == Ax; + (1 - A)zj. ÿ = Aÿ + (1- A) A~ Beta a).
(13)
Here, Dr and 0e are the uncertainty-calibrace soft labels described in Section 3.4 The coefficient comes from a symmetric Beta distribution to provide diverse mixing strengths. We define the mixup loss follows:
miix
Σ Σ Te (oglo
(14)
D
(
where f() is the hash encoder and (-)) denotes the softmax function. This structure-aware interpola- tion transmits semantics and supervision signals from confident samples uncertain samples, which introduces additional regularity near the decision boundaries.
Theoretical Analysis. We provide theoretical analysis to justify the effectiveness of structure- interpolation. demonstrates that the interpolation keeps the consistency of the semantic structure in the hash space.
Theorem 3.2 (Structure- Preserving Interpolation Bound). Let æ; and F3 be two clean samples belonging to class i.e., Yic Yje = 1. h; and fúj represent the normalized hash embeddings of these two samples. Let Pe be the unit-norm prototype vector of class For any X E [0, 1], the interpolated embedding is defined as follows:
h = Xh; + (1- X)h;
The cosine similarity between h and Pe has the following lower bound:
cos(h. h. p.) 2 À cos((h, Pc) + - >) cos(h,:20 Pc)
This result indicates that structure- aware interpolation preserves the proximity to the class prototype along the interpolation path. It enhances the consistency of the semantic structure in uncertain regions and improves the class boundaries.
3.6 Unified Training Objective
We integrate all modules into unified training objective. It includes four complementary loss terms that jointly learn semantic anchors, maintain structural stability, calibrate noisy supervision, and improve class boundaries against label noise.
-= Lelean + Lennt Lealih + Lmix-
(15)
Each loss term contributes to shaping the semantic geometry of the hash space differently: (1) Lelean clusters clean samples around their corresponding prototypes, which ensures stable class semantics; (2) Lcomt enhances the structural consistency for noisy instances; (3) Leamb alleviates the impact of unreliable supervision using both the energy of predicted distributions and structure-based divergence; (4) Lmix facilitates the consistency of the semantic structure in uncertain regions and regularizes the class boundaries. It makes the learned hash codes more robust and generalizable to optimize the total loss objective. The overall training algorithm of our proposed SEGA is provided in Algorithm [

---

Algorithm 1: Training Procedure of SEGA
Input : Noisy dataset D; encoder f()) with parameters θ prototypes {pela=)
Hyperparameters:
Code length L; Mixup coefficient a; partition percentile dr
Output
: Trained parameters + learned prototypes {po)a=1
Initialize Pe ~ (0, I) for c L..,,,
while not converged do
Compute embeddings h, = f(x,) and logits I via Eq. 1
Compute alignment scores s via Eq. and partition D + De. Dn, using Eq. 3
Compute Lelean on De via Eq. and Levet on D,, via Eq.
For each a; DD compute U: via Eqs.
Calibrate labels DJ yi (1 Ui) and compute Lealib via Eq.
Partition D. + Don: Dun by U via Eq. 11
For each a;; un, select x; via Eq. 12:
Generate (z) Y) via Eq. 13 and compute Cmix via Eq
14
Update + {Pe Jam 1 by minimizing total loss Eq. 15
Table 1: The comparison of MAP scores four datasets under pairflip label noise.
Method
CIFAR-10
PLICKRISK
NUS-WIDE
MS COCO
Bits
128
16
128
DPSH
51.89
51,04
54.41
56.32
58.29
59.72
59.89
59.65
42.98
44.32
45.74
46.17
32.69
33:47
34,07
34.33
HashiNet
44.27
45.03
46.96
47.20
56.83
$7.77
58.34
60.63
43.61
44.17
45.21
47.01
31.93
32.09
32.58
56.17
SPQ
41.11
42.35
43.87
45.67
$2.75
53.21
54.15
54.50
40.03
41.20
41.75
42.31
27.
28.35
28.97
50.17
DCH
54.17
54.28
55.27
58.87
60:20
43.31
44.39
46.20
FSDH
$7.37
GrshdyHash
50:97
52.14
53.41
$4.18
$7.98
$8.04
59
59.77
43.78
44.26
44.49
44.69
34.17
34.21
34.31
34.66
IMLH
49.93
$1.58
$4.48
56.01
18.54
18.
58.61
$8.71
43.15
43.89
44.81
45 14
34.18
34
34.39
34.81
DPX
49.77
$1,02
$4.12
56.21
59.23
59.98
00.54
44,02
44,23
45.34
45,
34.23
35.01
WGLHH
39.72
CSQ
53.13
54.27
55,33
58.25
59.17
60.02
60.38
61.37
43.89
43.67
44,92
46.13
34.27
34,12
34,50
34.78
Orf
$4.13
55.15
$7.67
$8.37
58.39
60.29
59.5
60.19
44.4
44.74
44.45
16.61
34.84
34,90
34.88
34,97
REL
56.19
58.58
60.99
61.74
59.76
61.27
61.46
61.96
44.76
45.02
45.96
46.21
34,85
34,97
35.04
35.12
60.12
60.59
60.12
DIOR
60.96
67.8%
69.3
71.17
61.36
65:44
66.78
68.21
50.02
51.40
$2.26
$2.64
37.14
38.22
38.78
38.92
STAR
69.05
69.82
70.52
71.83
70.54
70.70
71 40
71 96
56.
56.77
$7.69
38 17
41.72
42.04
42.86
43.21
SEGA (Ours)
70,01
70059
72.34
73.56
71.19
72.14
74.49
76.48
59.11
59.52
61.86
64.61
44.37
46.08
47.21
47.58
4 Experiments
4.1 Setup
Datasets. We evaluate SEGA on four widely-used image retrieval benchmarks: CIFAR-10 28),
Flickr25k [24] NUS-WIDE [10), and MS COCO (36). CIFAR-10 is a balanced single-label dataset with 10 classes of images. Flickr25k and NUS- WIDE are multi-label web datasets with 38 and 81 categories, respectively; COCO provides multi-object annotations over 80 classes with dense labels per image. All datasets come with clean annotations. To simulate realistic training noise, inject synthetic corruption into a portion of training labels using two schemes: symmetric and pairwise noise. Noise rates are varied from 20% to 80% in steps of 20%. Symmetric noise randomly replaces each label with any other class, while pairwise noise flips labels only to semantically related categories (2).
follow prior work (55] by selecting the top 10 classes in NUS-WIDE MS
Baselines and metrics. We compare SEGA with diverse set of baselines grouped into three categories: (I) Standard Deep Hashing Methods, including DPSH [33), HashNet (5), SPQ (25). DCH 6 FSDH [19), GreedyHash [51], JMLH (88), DPN (2). WGLHH [52) CSQ [63] and
OrH 22 (2) General Label Noise-Robrr Robust Methods, including REL (7] and Jo-SRC 611: and (3)
Risse-Resilient Hashing Methods, including DIOR (55) and STAR 4T. We evaluate all methods at four code lengths: 16, 32, 64, and 128 bits. Performance is measured using the standard retrieval metric, i.e., mean average precision (MAP).
Implementation Details. All experiments are finished using PyTorch on a NVIDIA A40 GPU. We utilize stochastic gradient descent (SGD) with momentum of 0.9 and batch size of 24. The learning rate is initialized at 0.001, with weight decay set to 0.0004 and dropout rate to 0.5. The backbone network is initialized from pretrained VGG-16 [49 model, consistent with all baseline
7

---

Table 2: The comparison of MAP scores on four datasets under symmetric label noise.
Method
CIFAR-10
F1,1KKR25K
NUS-WIDN
MS COCO
Bits
16
32
61
128
16
32
61
128
16
32
61
128
32
611
128
DPSH
56.67
58.24
44.82
Huddieet
42.54
43.24
44.47
45.29
53.47
$4.69
56.03
$6.87
43.21
44.37
45.12
46.
27.17
27.69
28.54
28.89
DCH
45.29
47.75
48.19
49.24
57.02
58. 201
58.4 44
59 12
44. 44.36
44.21
44.98
45 8X
31.55
32.39
32,99
53.41
Gash
42.75
45.96
47.51
49.81
$6.89
$7.21
58.01
58.95
43.17
43,82
44.45
45,00
30.70
31.24
31.97
32.63
JMLH
DPN
44:57
47.08
48.09
48.56
56.83
57.43
58.35
59.33
44.87
45.72
46.
46
31.11
31.65
52.13
WGLHH
47.92
49.83
50.19
$2.35
57.17
$7.99
58.46
39.23
45.11
45.91
46.68
47.32
32.21
33.13
33.74
34.11
CSQ
50.08
51.75
54.21
55.39
57.38
58.15
58.88
59. 12
46.01
46.55
47.08
47.65
31.97
32.45
33.71
34,60
OrfE
49.87
50.65
51.98
53.26
57.16
58.73
59.12
60.04
46.58
47.13
47.86
49.24
31,76
32.74
33.06
33.89
REL
52.61
58.68
59.02
59.16
60.35
47.05
47.67
32.17
Jo-SRC
50.82
51.42
$1.96
53.62
58.12
58.91
59.89
00.87
47.79
48.14
48.97
49.34
32,59
33.19
34.51
35.11
DIOR
58.76
59.30
59.82
60.99
63.83
64.05
64.97
65.40
52.26
52.78
53.61
$4,06
35.
36.33
37.01
38.22
STAR
61.85
65.03
65.52
66.76
69.57
70.11
70.84
71.53
57.21
57.83
58.61
59.
59.89
40.80
41.2
41.87
42.79
SEGA (Ours)
65.87
69.01
70.23
71.50
72.66
75.04
63.91
Mcan Avergge Preasion
Predision
Prestsion
Mean Average Predsion
60
15
2714
SRC
STAR
Mean
9.00
DIC
DP%
D6OR
DPN
D.OK
DPN
40
0.3
06
0.0
0.5
0.5
0.
30 0,1 0.5
0.8
30
0.4 0.7 0.0
(a) Flickr25K (pair)
(b) Flickr25K (sym)
(c) NUS-WIDE (pair)
(d) NUS-WIDE (sym)
Figure 2: Performance comparison under different noise levels on Flickr25K and HUS-WIDE.
methods ensure fair comparison. Specifically, the first seven convolutional layers fine-tuned, while the final fully connected hashing layer is trained from scratch. More details about the datasets and implementation can be found in the Appendix.
4.2 Empirical Results
Performance Comparison. We report the retrieval performance of all methods on four benchmark datasets under two types of label noise in Tables D and 2 Noise rates of both types are 60%. The results are measured by mean average precision (MAP) across different code lengths (16, 32, 64, and 128 bits). Table shows the MAP scores under pairwise label noise. Table reports the results under symmetric noise. Several clear observations can be made from the results. (1) Overall, SEGA achieves the highest MAP scores across all datasets and noise types, showing strong robustness to both structured and unstructured label corruption. SEGA clearly outperforms both conventional deep hashing models and recent noise-robust approaches. Specifically, SEGA achieves 64.61 MAP on
with pairflip noise at 128 bits, more than 10% above DIOR and STAR. These findings confirm that SEGA can preserve semantic similarity under supervision noise. (2) Our SEGA consistently outperforms all the baselines, and the dimension of the hash codes increases, the performance gap widens generally. This shows the strong scalability and robustness our method in terms of the hash encoding dimension. (3) On MS COCO, which has dense and noisy multi-label annotations, our SEGA delivers particularly solid performance. This demonstrates its adaptability for real web data where the quality of annotations is poor.
Effects of Different Noisy Rates. Figure 2 shows the robustness of SEGA under different noise levels with 64 hash codes. Both pairwise and symmetric noise types are evaluated. As the noise rate increases from 0.2 to 0.8, most baseline methods drop sharply in performance. CSQ, Jo-SRC, and DPN degrade quickly beyond 0.4 noise rate, showing poor tolerance to heavy label corruption. In contrast, SEGA keeps consistently high MAP scores across all noise levels. The drop is smooth even when the noise rate reaches 0.8. For example, under pairfipp noise on NUS-WIDE (Figure 2(c). SEGA maintains a MAP above 60. It outperforms STAR and DIOR by large margins as noise increases. The advantage becomes most obvious at high noise levels, where other methods collapse but our structure- model stays stable. These results show that learning semantic geometry and calibrating supervision jointly gives better robustness than label correction alone.
Ablation Study. We evaluate the contribution of each module in SEGA through ablation experiments, where one loss term is removed at time. Table 3 reports the MAP scores under symmetric and pairflip noise on four datasets. Removing Letean results in clear drop across all datasets, showing that prototype-ged guidedd supervision is essential for stable learning. When Lcont is excluded, performance
8

---

Table 3: Ablation studies on four benchmarks under symmetric and pairflip label noise.
Method
CIFAR-10
Flickr25K
NUS-WIDE
MS COCO
Noise Type.Bits
[sym.52
pair.32
5yym.64
64
pair:4
sym_32
pair.32
sym,64 64
pair
1ym, 32
pair:52
sym.64
pair64
sym,3 32
pair 32
sym.64 64
pair
SEGA
60.61
68.63
61.62
72.05
62.04
70.02
62.18
67.81
51,52
54.75
51.88
58.70
41.37
41.41
41.55
40.23
SEGA wio L.er
SEGA we Luah
62.95
70.45
66.33
71.87
68.47
70.10
68.37
73.35
59.09
53.05
62.70
59.34
44.89
44.85
42.23
13.74
SEGA w/o Cap
60,48
70.12
63.05
71.39
68.26
69.94
68.46
73.38
58.72
57.54
61.65
60.70
44.20
45.25
14.81
16.90
SEGA
(Ours)
65.87
70.59
67.11
72.34
71.50
72.14
72.66
74,49
60.72
59.52
63.91
61.86
45.01
46.08
46.50
47.31
Aarragg trettion
partii
Juengg Prectton
parflip
33
75-
symmetric
65
Symmer
S-E m
N-SWE Dimensaan
70-
60
&
01
Perconcle
Percontic &: 03
-20 Simension
-40
20 20
(a) Flickr25K
(b) NUS-WIDE
(c)
Baseline
(d) Ours
Figure 3: (a)(b) Sensitivity analysis of percentile threshold qr with 64-bit hash codes. (o)() t-SNE
visualization 128-bit hash codes on CIFAR-10 under symmetric noise. The baseline for comparison is STAR, the strongest competing method.
decreases, especially under pairflip noise, indicating its necessity to maintain structural consistency on noisy areas. Excluding Lealib compromises robustness to both settings of noise and confirms that soft supervision limits the impact of annotation errors. Removing Luix also causes consistent performance loss, particularly on MS COCO, demonstrating the advantage of boundary regularization. Collectively, these results demonstrate the necessity of the elements of SEGA and their effectiveness in complementing each other. The Appendix contains additional ablation results.
Sensitivity analysis. We evaluate the sensitivity of SEGA to hyperparameter dr under pairflip and symmetric noise with 64-bit hash codes. The best performance appears when the q. value is set to 0.3 or 0.5, from Figure ((a) and 3(b) Performance drops when qr too high, as excessive filtering removes many clean samples. Denoising becomes weak when g, is too low, since noisy samples remain unfiltered. These observations imply a moderate threshold achieve the best compromise between over-filterig and under-filtriing
Visualization. To analyze the learned semantic structure, we visualize the hash embeddings using I-SNE on the CIFAR-10 10 dataset. The experiment is performed under symmetric noise with 128-bit hash codes. Specifically, SEGA generates clusters that far more separated than STAR, as shown in Figure3(c) [(d] Our SEGA presents a clearer boundary for each category, which indicates our framework learns more discriminative hash embeddings. Conversely, STAR shows evident overlaps between classes, demonstrating lower robustness to label noise. These results show SEGA learns hash codes that are semantically consistent and robust against noisy supervision.
5 Conclusion
In this paper, we present unified framework named SEGA for robust multi-label hashing under noisy supervision. The method models semantic geometry improve the reliability of learned hash representations. We first use prototype-guides guided semantic anchoring to align embeddings with class semantics and maintain structural stability. Next, we design an enrertainty-guideee calibration module to adjust the effect of unreliable labels. We also apply a structure- aware interpolation strategy to smooth decision boundaries and enhance local consistency in noisy regions. In theory, SEGA reduces semantic misalignment and improves class boundaries. Experiments on several noisy hashing benchmarks show clear robustness gains. Although built for multi-label hashing, the core ideas of SEGA can be extended to other structure-sensitive learning tasks, such multi-modal retrieval and multi-label classification under weak or noisy supervision.
Acknowledgement
Ming Zhang and Yiyang Gu are supported by grants from the National Key Research and Development Program of China with Grant No. 2023 2023YFC334120 and the National Natural Science Foundation of China (NSFC Grant Number 62276002).

---

References
[1] Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In Proceedings of the International Conference on Machine Learning, 2017.
[2] Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. Understanding and improving early stopping for learning with noisy labels. Advances in Neural Information Processing Systems, 21224322-24403 2021.
[3] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399- -4344, 2006.
[4] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Deep risual-semantic quantization for efficient image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1328-1337, 2017.
[5] Zhangjic Cao, Mingsheng Long, Jianmin Wang, and Philip Yu. Hashnet: Deep learning to hash by continuation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017.
[6] Zhangjie Cao, Ziping Sun, Mingsheng Long, Jianmin Wang, and Philip S Yu. Deep priority hashing. In Proceedings of the 26th ACM international conference on Multimedia, pages 1653-1661, 2018.
[7] Haw-Shiuan Chang, Erik Learned-Miller, and Andrew McCallum. Active bias: Training more accurate neural networks by emphasizing high variance samples. In Proceedings of the Conference on Neural Information Processing Systems, 2017.
[8] Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien. Semi-Supervised Learning. MIT Press, 2006.
[9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PmLR, 2020.
[10] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng, Nus- wide: a real-world web image database from national university of singapore. Proceedings of the ACM international conference on image and video retrieval, pages 1-9, 2009.
[11] Yawen Cui, Wanxia Deng. Haoyu Chen, and Li Liu. Uncertainty-aware distillation for semi- supervised few-shot class-incrementa learning. IEEE Transactions on Neural Networks and Learning Systems, 35(10):
2023.
[12] Lixin Fan, Kam Woh Ng. Ce Ju, Tianyu Zhang, and Chee Seng Chan. Deep polarized network for supervised learning of accurate binary hashing codes. In IJCAI, volume 825, 2020.
[13] Aritra Ghosh and Andrew Lan. Contrastive learning improves model robustness under label noise. pages 2703-2708, 2021.
[14] Aritra Ghosh, Naresh Manwani, and PS Sastry. On the robustness of decision tree learning under label noise. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 685-697, 2017.
[15] Jacob Goldberger and Ehud tation layer. In Proceedings of the International Conference on Learning Representations, 2017.
Training deep neural netrol-networks using a noise adap-
[16] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 17, pages 529-536, 2005.
10

---

[17] Yiyang Gu, Binqi Chen, Zihao Chen. Ziyue Qiao, Xiao Luo, Junyu Luo, Zhiping Xiao, Wei Ju. and Ming Zhang. Mate: Masked optimal transport with dynamic selection for partial label graph learning. Artificial Intelligence, page 104396, 2025.
[18] Yiyang Gu, Zihao Chen, Yifang Qin, Zhengyang Mao, Zhiping Xiao, Wei Ju, Chong Chen, Xian-Sheng Hua, Yifan Wang, Xiao Luo, et al. Deer: Distribution divergence-based graph contrast for partial label learning on graphs. IEEE Transactions on Multimedia, 2024.
[19] Jie Gui, Tongliang Liu, Zhenan Sun, Dacheng Tao, and Tieniu Tan. Fast supervised discrete hashing. IEEE transactions on pattern analysis and machine intelligence, 40(2):490 496, 2017.
[20] Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Proceedings of the Conference on Neural Information Processing Systems, 2018
[21] Dan Hendrycks, Kimin Lee, and Mantas Mazcika. Using pre-training can improve model ro- bustness and uncertainty. In Proceedings of the International Conference on Machine Learning, pages 2712-2721, 2019.
[22] Jiun Tian Hoe, Kam Woh Ng. Tianyu Zhang. Chee Seng Chan, Yi-Zhe Song, and Tao Xiang. One loss for all: Deep hashing with a single cosine similarity based learning objective. 2021.
[23] Peng Hu, Xi Peng, Hongyuan Zhu, Liangli Zhen, and Jie Lin. Learning cross-modal retrieval with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[24] Mark J Huiskes and Michael S Lew. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference Multimedia information retrieval, pages 39 43, 2008.
[25] Young Kyun Jang and Nam Ik Cho. Self-supervised product quantization for deep unsupervised image retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12085-12094 2021.
[26] Wei Ju, Siyu Yi, Yifan Wang, Zhiping Xiao, Zhengyang Mao, Hourun Li, Yiyang Gu, Yifang Qin, Nan Yin, Senzhang Wang, et al. A survey of graph neural networks in real world: Imbalance, noise, privacy and ood challenges. arXiv preprint arXiv:2403.0444 2024.
[27] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:186611 18:18666-16773 2020.
[28] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] Chen Li, Xiaoling Hu, Shahira Abousamra, and Chao Chen. Calibrating uncertainty for semi- supervised crowd counting. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 16685-16695 IEEE, 2023.
[30] Jingyao Li, Pengguang Chen, Zexin He, Shaozuo Yu, Shu Liu, and Jiaya Jia. Rethinking ditto--distriuttinn (ood) detection: Masked image modeling all you need. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1-578-11589, 2023.
[31] Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. In Proceedings of the International Conference on Learning Representations, 2021.
[32] Qi Li, Zhenan Sun, Ran He, and Tieniu Tan. Deep supervised discrete hashing. Advances in neural information processing systems, 30, 2017.
[33] Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.0385.. 2015.

---

[34] Wu-Jun Li, Sheng Wang, and Wang Cheng Kang. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 1711-1717, 2016.
[35] Kevin Lin, Huei-Fang Yang. Jen-Hao Hsiao, and Chu-Song Chen. Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pultern recognition workshops, pages 27-35, 2015.
[36] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pages 740-755. Springer, 2014.
[37] Hongyi Ling, Zhimeng Jiang, Meng Liu, Shuiwang Ji, and Na Zou. Graph mixup with soft alignments. In International Conference on Machine Learning, pages 21335-21349, PMLR, 2023.
[38] Sheng Liu, Jonathan learning regularization prevents memorization of noisy labels. In Proceedings of the Conference on Neural Information Processing Systems, 2020.
Narges Razavian, and Carlos Femandez-Granda. Early-
[39] Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(3):447-461, 2015.
[40] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based oit-of-distriiutionn detection. Advances in neural information processing systems, 33:21464-21473 2020.
[41] Qingqing Long, Haixin Wang, Jinan Sun, Wei Xiang, Yijia Xiao, Yusheng Zhao, and Xiao Luo. Learning resistant binary descriptors against noise for efficient image retrieval. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2699, 2025.
[42] Jie Ma, Chuan Wang, Yang Liu, Liang Lin, and Guanbin Li. Enhanced soft label for semi- supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 185--1995, 2023.
[43] Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu Making deep neural networks robust label noise: loss correction approach. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017.
[44] Himashi Peiris, Munawar Hayat, Zhaolin Chen, Gary Egan, and Mehrtash Harandi. Uncertainty- guided dial- for sumi-super volumetric medical image segmentation. Nature Machine Intelligence, 5(7):224-733)) 2023.
[45] Scott Reed, Honglak I Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. In Proceedings of the International Conference Learning Representations, 2015.
[46] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
[47] Yuming Shen, Li Liu, and Ling Shao. Unsupervised binary representation learning with deep variational networks. International Journal of Computer Vision, 127(11): 2019.
[48] Yuming Shen, Jie Qin, Jiaxin Chen, Li Liu, Fan Zhu, and Ziyi Shen. Embarrassingly simple binary representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0-0, 2019.
[49] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:/409 1556, 2014.
[50] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. arXiv preprint arXiv:2007.08199) 2020.
12

---

[51] Shupeng Su, Chao Zhang, Kai Han, and Yonghong Tian. Greedy hash: Towards fast optimization for accurate hash coding in enn. Advances in neural information processing systems, 31, 2018.
[52] Rong- Canng Tu, Xian-Ling Mao, Cihang Kong, Zihang Shao, Ze-Lin Li, Wei Wei, and Heyan Huang. Weighted gaussian loss based hamming hashing. In Proceedings of the 29th ACM International Conference on Mullimedia, pages 3409-3417, 2021.
[53] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International conference on machine learning, pages 6438-6447. PMLR, 2019.
[54] Changqi Wang, Haoyu Xie, Yuhui Yuan, Chong Fu, and Xiangyu Yue. Space engage: Col- laborative space supervision for contrastive-based semi- cemi-supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 931-942,
2023.
[55] Haixin Wang, Huiyu Jiang, Jinan Sun, Shikun Zhang, Chong Chen, Xian-Sheng Hua, and Xiao Luo. Dior: Learning to hash with label noise via dual partition and contrastive learning. IEEE Transactions on Knowledge and Data Engineering, 36(4):1502-1555
2023.
[56] Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. A survey on learning to hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):769-790, 2017.
[57] Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nannan Wang, Zongyuan Ge, and Yi Chang. Robust early-leaming: Hindering the memorization of noisy labels. In Proceedings of the International Conference on Learning Representations, 2021.
[58] Ming-Kun Xie, Jiahao Xiao, Hao-Zhe Liu, Gang Niu, Masashi Sugiyama, and Sheng-Jun Huang. dlas-distribution-avee aware pseudo-labeling for semi--upervvin multi-label learning. Advances in Neural Information Processing Systems, 36:25731 257477, 2023.
[59] Yizhe Xiong, Hui Chen, Zijia Lin, Sicheng Zhao, and Guiguang Ding. Confidence-basee visual dispersal for few-shot unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11621-11631, 2023.
[60] Junwei Yang, Hanwen Xu, Srbuhi Mirzoyan, Tong Chen, Zixuan Liu, Zequn Liu, Wei Ju, Luchen Liu, Zhiping Xiao, Ming Zhang, et al. Poisoning medical knowledge using large language models. Nature Machine Intelligence, 6(10):1156-11 1168, 2024.
[61] Yazhou Yao, Zeren Sun, Chuanyi Zhang, Fumin Shen, Qi Wu, Jian Zhang, and Zhenmin Tang. Jo-sre: A contrastive approach for combating noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[62] Jiaguo Yu, Huming Qiu, Dubing Chen, and Haofeng Zhang. Weighted contrative hashing. In Proceedings of the Asian Conference on Computer Vision, pages 3861-3876, 2022.
[63] Li Yuan, Tao Wang, Xiaopeng Zhang, Francis EH Tay, Zequn Jie, Wei Liu, and Jiashi Feng. Central similarity quantization for efficient image and video retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3083-3092, 2020.
[64] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 2021.
[65] Hai Yang Zhang, XiMing Xing, and Liang Liu. Dualgraph: A graph- based method for reasoning about label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021.
[66] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In Proceedings of the International Conference on Learning Representations, 2018
[67] Zhilin Zhao and Longbing Cao. Dual representation learning for -ittr--diitribution detection. Transactions on Machine Learning Research.
13

---

[68] Guoqing Zheng, Ahmed Hassan Awadallah, and Susan Dumais. Meta label correction for noisy label learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 2021.
[69] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. In Proceedings of the AAAI conference on Artificial Intelligence, volume 30, 2016.
[70] Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML), pages 912-919, 2003.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The contributions and scope are presented in the abstract section and introduc- tion section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification:
The limitations section is discussed in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The proofs are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The implementation details are provided in the experiment section and ap- pendix.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code link is provided in the Appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
wtweeeeuuubsssisooooooooooe for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https ctiips.cc/public//uue
Codeeuumiiii wettubbmmssiiioi ssionoolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The implementation details are provided in the experiments section and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
The variance is limited in experiments. Therefore, compared methods also do not report the variance. To ensure a fair comparison with state-of-the-eee baselines, we fix the seed and adopt the same setting with their corresponding papers.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enoor bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We use NVIDIA A40 40G GPUs to conduct all the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We sincerely read the ethics guidelines and obey this rule.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The discussion about the broader impacts is provided in the Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks.
19

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the creators of assets are properly credited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssithcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: New assets are well documented.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
20

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non stanndaddrrd components.
Please refer to our LLM policy (https://neut) / for what should or should not be described.
21

---

A More Experiment Results
We conduct ablation studies on 16-bit and 128-bit hash codes. As shown Table 4 removing any loss term leads to a clear drop in performance across all the datasets. This demonstrates that each module plays necessary role learning robust hash codes against label noise. For instance, removing Lealib causes clear drops in performance across all the datasets, especially with 16-bit hash codes. This justifies the benefit uncertainty-guide supervision calibration in noisy environments. Turning off Lmix also reduces MAP, particularly on Flickr25K, showing its effect in regularizing category boundaries. These results confirm that all modules in our framework are complementary and necessary under different code lengths.
Table 4: Ablation studies on four benchmarks under symmetric (sym.) and pairflip (pair.) label noise 16-bit and 128 bit code lengths.
Method
eifar-10
Flickr25K
NUS-WIDE
MS-COCO
Noise Type.Rits
15ym.16
16 pair 16
128
poir.128
sym.16
16
pair.16]
128
p2ir.12
16
poir.
sym. 128
pair.15
sym.16
sym. 128
128
SPGA wlo faae
59.70
64.87
62.85
70.96
60:71
63.92
63.72
68.82
52.54
54.74
55.17
62.87
37.85
38.03
45.80
44.24
SBGA
62.98
63.49
65.90
72.95
69.38
69.78
71.85
15.13
57.66
57.07
62.18
60.68
42.75
42.38
66.26
45.33
SBGA
67.30
64.34
68.19
54.19
63.12
46.38
46.88
Caa.
67.94
SEGA (Ours)
70.01
59.06
B Proofs of Theorems
Theorem 3.1 (Uncertainty- -eighttd Confidence Approximates Label Correctness). Let I; be a training sample with observed label Y: and true (latent) label y Define the energy- -ased prediction confudence as F. = log Ee_ exp(Iic), and F. max, E, E-mii E,' the structure-buc divergence as 6, = K k cos(hi, h2)) If we assume the probability that the label correct depends solely on the semantic and structural reliability of the sample, and these two sources are conditionally independent given oe then:
E. E.-min, E,
Pr|y = yi x.]] < 1 U..
(16)
Proof. The probability that sample is confidently predicted is proportional to the negative energy in energy-based learning 44]: Pr[semantic correctness|z:]] x cxp(-E:) E.).
After batch normalization, E. E [0, maintains a monotonic relationship with Es and thus Pr|semantic correctness|zg] x E.
The structural divergence score 8; reflects how well a sample aligns with its local neighborhood. Samples high-densin regions with consistent semantic structure are more likely to have correct labels under the widely adopted cluster assumption. In contrast, geometrically isolated samples, those with low average similarity to neighbors, are often located near semantic boundaries or in underrepresented regions of the data manifold and are thus more prone to label noise.
This assumption supported by several foundational approaches. In graph- semi-supervised learning (70 B), Laplacian regularization encourages neighboring nodes to share similar outputs. Label propagation methods [8] assume that labels should remain consistent within locally coherent regions. Moreover, the low-density separation principle [66] suggests that well-formed clusters of correctly labeled data tend to lie in structurally connected, high-confidence zones. Therefore, smaller divergence score (i.e., larger 1 - 6²) indicates stronger neighborhood support and a higher likelihood of label correctness. We thus approximate structural confidence as being proportional to the complement of divergence:
Prjstructural consistency xx] x
According to the assumption, we can obtain
Priyf ý úlæs x (1 (-)E;)( 6.).
On the other hand,
1- U, = 1 1(1 E.)o, (1 E,)(1 5,) -E,6.,
22

---

when E, and ó; are relatively small (i.e., low uncertainty). the second term is small, and 1 U, = (1 E1) 6.) x Pr(y = [=/==
Under the assumption that label correctness depends only on semantic and structural confidence, modeled respectively by energy-based and neighborhood-baase signals, the complement of the joint uncertainty score 1 - U; provides a proportional estimate of the posterior label correctness probability.
Theorem 3.2 Structure-Preseerving Interpolation Bound), Let ay and x be two clean samples belonging class i.e., Yic Yje 1. h, and fJ represent the normalized hash embeddings of these two samples. Let Pe be the unit-norm prototype vector of class c. For any X E [0, 11, the interpolated embedding is defined follows:
= Xh; + (1- (1-X)h3-
The cosine similarity between h and Pe has the following lower bound:
cos(h, P.) 2 Xcos((; P.) + (1 A) cos(h;,,P)
Proof. Since both h: and hj normalized to unit norm (i.e., ||h.l| ||hا|| 1), and Py is also
unit-norm vector (i.e., ||p,l 1), we begin by expanding the cosine similarity between h and py:
(Ahs + (1 (1- (1-))),, Pc
hT Pe
cos (h. p.)
|1
Applying linearity of inner product:
h_Pee Pe Ah; Pa (1 1)h3 Pc: = Acos(h; po) + (1 A) cos(h;, Po).
Thus,
cos(h, Po) Acos(h; Pa) + (1- A) cos(h,j.P.
|//
Since ||hl| < 1 by convexity of Euclidean norm (specifically, Minkowski inequality), conclude:
cos(h,po) po) Acos(h; po) + (1 A) cos(h;,Po))
as desired.
C Dataset Details
We evaluate our method on four widely used image retrieval benchmarks. All datasets are standard in deep hashing and image retrieval literature, with consistent splits across prior work (55).
This a single-label dataset containing 60,000 natural images evenly divided into 10 categories. Each image is 32 x 32 in resolution with three color channels. We sample 1,000 examples per class as queries. The remaining examples are utilized the retrieval database, from which 500 images per class are further sampled for training. The low resolution and compact structure make this dataset a challenging testbed for semantic feature extraction.
CIFAR-10
28 30
Flickr25K (24). The Flickr25K dataset collects 25,000 multi-label images from the Flickr platform. Each image in the dataset tagged with one more labels from 38 semantic concepts. We utilize the 24 most frequent concepts as the label set following [55]. We use 2,000 images as queries, and sample 10,000 images for training from the remaining retrieval set. The annotations are sparse in this dataset, where each image is assigned 4.7 labels on average.
NUS-WIDE [10. The NUS-WIDE dataset includes 269,648 web images with 81 semantic tags. We utilize the 10 most frequent tags as the target categories following [55]. We employ 5,000 images as queries, and sample 5,000 images from the remaining retrieval set for training.
The MS COCO dataset contains more than 120,000 images, where each image tagged by some of 80 categories. We adopt the 2014 release and use 5,000 images queries. From the remaining retrieval set, 10,000 are sampled for training. Its complex scenes and dense annotations make it challenging dataset for fine-grained image retrieval.
MS COCO 36 58
23

---

D Implementation Details
All experiments are implemented in PyTorch and rum on single NVIDIA A40 GPU in standard Linux environment. Two types of label noise are introduced. In symmetric noise, each label is randomly replaced by another class with equal probability. In pairwise flip noise, labels are switched to semantically related categories according to predefined mappings [55]. The noise rate ranges from 20% to 80% increments 20%. During mixup training, interpolation coefficients are sampled from symmetric Beta distribution with 0.4 as in [66]. The percentile threshold gr for selecting clean samples set to 0.3 by default. The code can found at sattps://githwn com/dllab001/SEGh
E
Broader Impacts and Limitations
This paper deals with problem of robust image retrieval under noisy supervision and proposes generic framework called SEGA to do so. With this process our approach is scalable and reliable in real-world settings where label quality is imperfect: e.g. web-based image indexing and large-scale multimedia systems. As well, although our framework is instantiated in multi-label hashing, its essential principles not domain-specific and can be readily adapted more general tasks, e.g. multi-modal retrieval, multi-label classification and other structure-sente sensstive learning tasks under weak noisy supervision. However, one limitation of current solution is that we use static retrieval database, which does not have the ability to cope with the change in the data distribution in dynamic environments. Future work could adapt SEGA continual learning online retrieval situations.
24


Reviews:
[{"review_id": "YYmaCVeRSk", "reviewer": "Reviewer_mLfB", "summary": "This paper presents a novel framework, named SEGA, for robust multi-label image hashing under noisy supervision. The main contributions include:\n- The paper presents a geometric perspective for learning under noisy supervision.\n- The proposed method considers three components during learning: (1) prototype anchoring, (2) uncertainty-calibrated soft labels, and (3) structure-aware interpolation\n- The experimental results show promising improvements compared with prior works on 4 popular benchmark.", "questions": "see weakness", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Idea: The proposed semantic geometry shaping (SEGA) considers the semantic geometry for optimization. The three proposed modules enable representations, supervision, and uncertainty to co-evolve in a close-loop. The idea of closing the loop is interesting.\n- Design of SEGA moduel: The idea of prototype anchoring could reduce the requirement of clean labels. The structure-aware MixUp is also a good design.\n- Strong Results: The paper presents experiments on CIFAR-10, Flickr25K, NUS-WIDE, MS COCO. The proposed method achieves comparable or better performance than the previous works in terms of MAP.\n- Comprehensive analysis: The paper presents ablation studies to examine the importance of each component. Sensitivity analysis is also provided. The paper also presents the robustness of SEGA under different noise levels. Overall, the experiments are compelling.\n\nWeakness:\n- Scalability: The experiments are conducted to small or medium size of datasets. There is no large-scale dataset considered, which is the real scenario for hashing. \n- Generalizability: It might be supplemented to include discussions on possible extensions to other tasks, such as zero-shot retrieval (or other tasks that also require semantic geometry)\n- Readability: The paper is not easy to follow, especially theoretical analysis.", "was_revised": true, "final_justification": "The authors have addressed my concerns. I don't have significant concerns on this paper. Given the rebuttal and their experiments, I will vote borderline accept.", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> **Q1**: Scalability: The experiments are conducted to small or medium size of datasets. There is no large-scale dataset considered, which is the real scenario for hashing.\n\n**A1**: Thanks for your comment.\nTo assess SEGA’s scalability and robustness under real-world noisy supervision, we conducted experiments on the WebVision dataset [1], which contains 2.4 million web images across 1,000 classes. We used the Google image subset for training. The results are as follows:\n\n|Method|WebVision (mAP)|\n|-|-|\n|DIOR|27.82|\n|STAR|32.64|\n|SEGA|34.71|\n\nSEGA achieves the best performance, demonstrating both scalability to large datasets and robustness under real-world web noise. We will include this result in the revised version.\n\n[1] WebVision Database: Visual Learning and Understanding from Web Data. 2017\n\n> **Q2**: Generalizability: It might be supplemented to include discussions on possible extensions to other tasks, such as zero-shot retrieval (or other tasks that also require semantic geometry)\n\n**A2**: Thanks for your comment. To demonstrate SEGA’s generalizability, we conducted a zero-shot retrieval experiment following [2].\nWe split CIFAR-10 into seen/unseen categories and trained only on seen classes. For example, \"airplane\" (\"dog\"/\"truck\") was treated as an unseen class, and 1,000 query images were retrieved from a database of 59,000 mixed samples.\n\n|Method|airplane|dog|truck|\n|-|-|-|-|\n|DIOR|22.19|30.66|33.17|\n|STAR|26.09|35.97|37.23|\n|SEGA|30.50|38.24|39.44|\n\nSEGA outperforms strong baselines, showing better generalization to unseen categories. We will include this setting and results in the revised version.\n\n[2] Zero-Shot Hashing via Transferring Supervised Knowledge. ACM MM 2016\n\n\n> **Q3**: Readability: The paper is not easy to follow, especially theoretical analysis.\n\n**A3**: Thanks for your comment. We appreciate your feedback on the readability of the paper, particularly the theoretical analysis. We understand that Theorems 3.1 and 3.2 may be difficult to follow due to dense formulations and limited intuitive guidance, and that some transitions and descriptions in the main text could be further streamlined to improve overall clarity.\n\nTo address these issues, we will revise the theoretical section and the overall narrative in the following ways:\n\n1. **Add plain-language motivation**: Before each theorem, we will briefly explain its purpose. For example, Theorem 3.1 quantifies the connection between our unified uncertainty score and label correctness, while Theorem 3.2 explains why interpolating semantically consistent samples preserves class semantics.\n\n2. **Include boxed interpretations**: After each theorem, we will summarize the key takeaway in a short “Interpretation” paragraph to help readers understand its practical implication.\n\n3. **Add diagrams and toy examples**: In the Appendix, we will illustrate uncertainty-guided reweighting and structure-aware MixUp with intuitive 2D examples to visualize the theoretical claims.\n\n4. **Improve overall narrative flow**: Beyond the theoretical sections, we will refine transitions between sections, simplify overly dense paragraphs, and introduce key terms more gradually. Additionally, we will revise Figure 1 to better reflect the motivation and pipeline of SEGA at a glance, which should help readers grasp the framework before diving into technical details.\n\nWe will incorporate these changes in the revised version. Thank you again for your constructive suggestions.\n\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}, {"review_id": "Y7JJSiC5xr", "reviewer": "Reviewer_6Ut3", "summary": "This paper introduces SEGA (Semantic Geometry Shaping), a novel framework for robust multi-label hashing, specifically addressing the challenging problem of learning under noisy supervision. The core idea revolves around explicitly sculpting the semantic geometry of the representation space rather than solely focusing on label correction. The proposed method integrates three synergistic components: prototype-guided semantic anchoring, uncertainty-guided supervision calibration, and structure-aware interpolation. The paper provides theoretical justifications and extensive empirical evaluations, demonstrating state-of-the-art performance on various benchmarks under different noise regimes.", "questions": "- The prototypes are randomly initialized. How sensitive is the final performance to this initialization, particularly when dealing with datasets that have highly imbalanced classes or very noisy initial labels?\n\n- The paper mentions that multi-label scenarios can have intrinsic semantic ambiguity. How does SEGA differentiate between true semantic ambiguity (where multiple labels are genuinely appropriate) and label noise (where a label is simply incorrect)? And how does its uncertainty estimation handle this distinction?", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strength:\n- The paper introduces a fresh and compelling perspective by shifting the focus from mere label correction to actively shaping the semantic geometry of the representation space. This is a significant conceptual contribution to robust learning under noise, especially for structured outputs like hash codes.\n\n- SEGA's three core components (prototype anchoring, uncertainty calibration, and structure-aware interpolation) are not standalone heuristics but are thoughtfully designed.\n\n- The inclusion of theoretical analyses (Theorem 3.1 and 3.2) providing probabilistic interpretations for uncertainty weighting and demonstrating structure-preserving interpolation adds significant rigor to the paper.\n\n- The paper is well-written, clear, and easy to follow. The figures (especially Figure 1) effectively illustrate the framework.\n\nWeakness:\n\n- While sensitivity analysis for q_T (percentile threshold for clean/noisy set partition) is provided, the framework still introduces several hyperparameters (e.g., m for contrastive loss, K for nearest neighbors, alpha for Beta distribution in MixUp). The robustness to these other hyperparameters is not fully explored, and their optimal selection might be dataset-dependent.\n\n- The proposed method involves maintaining dynamic prototypes, calculating neighbor-based divergence, and performing selective MixUp. While the paper mentions using a single NVIDIA A40 GPU, a more explicit discussion or comparison of the computational cost (training time, memory usage) against baselines would be beneficial.", "was_revised": false, "final_justification": "", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> **Q1**: While sensitivity analysis for q_T (percentile threshold for clean/noisy set partition) is provided, the framework still introduces several hyperparameters (e.g., m for contrastive loss, K for nearest neighbors, alpha for Beta distribution in MixUp). The robustness to these other hyperparameters is not fully explored, and their optimal selection might be dataset-dependent.\n\n**A1**: Thank you for the insightful comment. We have conducted additional sensitivity experiments (64-bit hash codes) to evaluate the robustness of SEGA with respect to hyperparameters m for contrastive loss, K for nearest neighbors, and $\\alpha$ for Beta distribution in MixUp, as shown below:\n\n\n|m|0.5|1|1.5|2|\n|-|-|-|-|-|\n|FLICKR25K|72.07|74.49|73.50|72.98|\n|NUS-WIDE|60.72|61.86|61.75|60.19|\n\n|$\\alpha$|0.1|0.4|1.0|10.0|\n|-|-|-|-|-|\n|FLICKR25K|72.80|74.49|74.08|73.74|\n|NUS-WIDE|60.54|61.86|60.22|59.79|\n\n|K|1|3|5|10|20|\n|-|-|-|-|-|-|\n|FLICKR25K|74.09|74.23|74.49|74.41|74.34||\n|NUS-WIDE|60.98|61.41|61.86|61.71|61.59|\n\nAcross a wide range of values, SEGA consistently maintains strong performance, indicating robustness to hyperparameter choices. We will include this analysis in the final version.\n\n\n> **Q2**: The proposed method involves maintaining dynamic prototypes, calculating neighbor-based divergence, and performing selective MixUp. While the paper mentions using a single NVIDIA A40 GPU, a more explicit discussion or comparison of the computational cost (training time, memory usage) against baselines would be beneficial.\n\n**A2**: Thanks for your comment. We agree that understanding the computational cost is essential for practical deployment.\nWhile SEGA introduces additional modules (e.g., dynamic prototypes, neighbor-based divergence and structure-aware MixUp), they are implemented efficiently and scale well. As shown below, SEGA’s training time per epoch and memory usage are comparable to competitive baselines, yet it achieves significantly better performance:\n\n|Method||FLICKR25K|||MS COCO||\n|-|-|-|-|-|-|-|\n||time (s)|memory (MB)|mAP|time (s)|memory (MB)|mAP|\n|CSQ|87.00|1322.00|58.88|55.00|1624.00|33.71|\n|STAR|179.00|1287.00|70.84|161.00|1425.00|41.87|\n|SEGA|160.00|1569.00|72.66|182.00|1879.00|46.50|\n\nSEGA outperforms all baselines by a large margin in mAP, while maintaining similar or even lower training time on Flickr25K. We will include these results in the revised version to clarify SEGA’s efficiency-performance tradeoff.\n\n\n\n> **Q3**: The prototypes are randomly initialized. How sensitive is the final performance to this initialization, particularly when dealing with datasets that have highly imbalanced classes or very noisy initial labels?\n\n**A3**: Thanks for your comment.\nWe conducted experiments on CIFAR-10 under 80% symmetric noise, comparing performance under both balanced and highly imbalanced class distributions (Class 1–10: 500, 1000, ..., 5000). Results with standard deviation across 5 runs are shown below:\n\n|Method|Imbalance=True|Imbalance=False|\n|-|-|-|\n|CSQ|37.38±0.72|45.30±0.59|\n|DPN|34.12±0.51|50.29±0.44|\n|STAR|56.90±0.27|61.45±0.22|\n|SEGA|61.46±0.63|63.71±0.48|\n\nDespite severe imbalance and high label noise, SEGA consistently outperforms all baselines, with significantly higher mAP and comparable standard deviation. This demonstrates its robustness to data imbalance, as the learned prototypes adapt to uneven label distributions. We will include this result in the revised version.\n\n> **Q4**: The paper mentions that multi-label scenarios can have intrinsic semantic ambiguity. How does SEGA differentiate between true semantic ambiguity (where multiple labels are genuinely appropriate) and label noise (where a label is simply incorrect)? And how does its uncertainty estimation handle this distinction?\n\n**A4**: Thanks for your comment. The key difference between label noise and true semantic ambiguity is reflected in the logit distribution, which is captured by our energy-based prediction confidence.\n\nIn the case of noisy labels, the image lacks meaningful visual features aligned with the incorrect class prototypes. As a result, its embedding shows low similarity to those prototypes, and consequently, the logits for the noisy classes are low, yielding high energy, which indicates uncertainty.\n\nIn contrast, the samples with true multiple labels contain meaningful visual features that are aligned with multiple class prototypes. Their embeddings tend to have high similarity with those class prototypes, resulting in higher logits and lower energy.\n\nThis formulation allows SEGA to effectively downweight noisy labels while preserving supervision for semantically valid co-occurrences. We will clarify this interpretation in the revision. Thanks for pointing this out.\n\n\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}, {"review_id": "2orfzmohjv", "reviewer": "Reviewer_XoCg", "summary": "The paper proposes SEGA (Semantic Geometry Shaping) for robust hashing under noisy multi-label supervision, integrating prototype-guided anchoring, uncertainty-calibrated soft labels, and structure-aware interpolation. While experiments on CIFAR-10, Flickr25K, NUS-WIDE, and MS COCO show state-of-the-art mAP gains, critical limitations in theoretical innovation and practical validation hinder its novelty.", "questions": "1.Novelty vs. Recent Baselines: Why exclude comparisons with DCMIP [2024] and PCL [2023]? Provide cross-method mAP on NUS-WIDE .\n\n2.Computational Efficiency: What is the training time/memory cost of SEGA vs. STAR on MS COCO? .\n\n3.Theoretical Assumptions: Validate Theorem 3.1’s conditional independence with real-world noise datasets .\n\n4.Small-Dataset Performance: Report results on CUB-200 with 50% symmetric noise .", "limitations": "SEGA relies on pre-trained VGG-16 features, limiting generalizability to feature-agnostic scenarios. The framework’s sensitivity to hyperparameters (e.g., K in structure-based divergence) and lack of real-world deployment cases further restrict its impact .", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1.Empirical Validation: SEGA outperforms baselines (STAR, DIOR) across noise types, with notable robustness under high noise rates.\n\n2.Closed-Loop Design: The framework jointly optimizes supervision, uncertainty, and geometry, shown via ablation studie .\n\n3.Theoretical Formulation: Theorems 3.1 and 3.2 provide probabilistic grounding for uncertainty weighting, though assumptions (conditional independence) are strong .\n\nWeaknesses\n\n1.Incremental Innovation: Prototype learning and uncertainty-based weighting replicate prior work (e.g., CSQ [50], DivideMix [23]), lacking novel architectural breakthroughs .\n\n2.Computational Overhead: Multi-component design (prototype updates, structure-aware MixUp) increases training complexity, but the paper omits inference time/memory comparisons .\n\n3.Noise Type Bias: Performance gains are more significant under structured noise (pairflip) than symmetric noise, indicating limited generalizability .\n\n4.Small-Dataset Gap: SEGA is untested on small-scale/noisy datasets (e.g., CUB-200), critical for real-world applicability .", "was_revised": true, "final_justification": "Thanks for the response. After considering the comments from the other reviewers, I have ultimately decided to increase my score.", "rebuttal": "We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.\n\n> **Q1**: Incremental Innovation: Prototype learning and uncertainty-based weighting replicate prior work (e.g., CSQ [50], DivideMix [23]), lacking novel architectural breakthroughs.\n\n**A1**: Thank you for your comment. While prototype learning and uncertainty-based weighting have been explored in prior works such as CSQ [1] and DivideMix [2], SEGA adopts a substantially different formulation and integration strategy.\n\nFor CSQ, hash centers are constructed using Hadamard matrices. These centers are then used to define pairwise similarity constraints between samples. In contrast, SEGA learns semantic prototypes via backpropagation in the embedding space, adapting them based on the evolving representation distribution during training. These prototypes are used not only for contrastive alignment but also to guide structure-aware interpolation.\n\nFor DivideMix, the method addresses noisy classification via loss distribution modeling and Gaussian mixture separation. It discards high-loss (presumed noisy) samples and adopts a semi-supervised co-training approach. SEGA, on the other hand, introduces a unified uncertainty score combining energy-based confidence and structure-based isolation. Rather than discarding uncertain samples, SEGA uses this score for soft reweighting of supervision and for guiding interpolation, which better suits multi-label and structurally ambiguous scenarios.\n\nMoreover, SEGA integrates all components into a closed-loop framework, where each part reinforces the others. This coordinated design is novel among robust multi-label hashing methods and contributes to both robustness and semantic consistency under noisy supervision.\n\nWe will include this discussion in our final version.\n\n[1] Central similarity quantization for efficient image and video retrieval. CVPR 2020\n\n[2] Dividemix: Learning with noisy labels as semi-supervised learning. ICLR 2020\n\n\n> **Q2**: Computational Overhead: Multi-component design (prototype updates, structure-aware MixUp) increases training complexity, but the paper omits inference time/memory comparisons.\n\n**A2**: Thanks for your comment. We agree that understanding the computational cost is essential for practical deployment.\nWhile SEGA introduces multi-component design  (prototype updates, structure-aware MixUp), they are implemented efficiently and scale well. As shown below, SEGA’s training time per epoch and memory usage are comparable to competitive baselines, yet it achieves significantly better performance:\n\n|Method||FLICKR25K|||MS COCO||\n|-|-|-|-|-|-|-|\n||time (s)|memory (MB)|mAP|time (s)|memory (MB)|mAP|\n|CSQ|87.00|1322.00|58.88|55.00|1624.00|33.71|\n|STAR|179.00|1287.00|70.84|161.00|1425.00|41.87|\n|SEGA|160.00|1569.00|72.66|182.00|1879.00|46.50|\n\nSEGA outperforms all baselines by a large margin in mAP, while maintaining similar or even lower training time on Flickr25K. We will include these results in the revised version to clarify SEGA’s efficiency-performance tradeoff.\n\n> **Q3**: Noise Type Bias: Performance gains are more significant under structured noise (pairflip) than symmetric noise, indicating limited generalizability.\n\n**A3**: Thanks for your comment. We agree that SEGA shows larger gains under structured noise (pairflip), which is by design: our method leverages semantic geometry and local consistency, both of which are more informative when the noise is structured.\n\nBesides, SEGA still outperforms baselines with a significant margin under symmetric noise across all benchmarks (as shown in Tables 2), demonstrating general robustness. For example, under 60% symmetric noise with 64-bit hash codes, SEGA achieves a +8.98% mAP gain over STAR on NUS-WIDE and +11.05% on MS COCO.\n\nWe will add more discussion in the revision to clarify SEGA’s behavior under different noise types and emphasize that while structured noise offers more exploitable signals, SEGA remains effective in unstructured noise settings.\n\n> **Q4**: Small-Dataset Gap: SEGA is untested on small-scale/noisy datasets (e.g., CUB-200), critical for real-world applicability.\n\n**A4**: Thanks for your comment. We conducted additional experiments on the small-scale fine-grained dataset CUB-200 under 50% symmetric noise. The results are summarized below:\n\n|Method|mAP|\n|-|-|\n|Jo-SRC|27.88|\n|DIOR|29.06|\n|STAR|31.13|\n|SEGA|37.34|\n\nSEGA outperforms all baselines by a significant margin (+19.94% mAP over STAR), demonstrating strong robustness even in small and noisy settings. We will include these results in the revised version.\n\n> **Q5**: Novelty vs. Recent Baselines: Why exclude comparisons with DCMIP [2024] and PCL [2023]? Provide cross-method mAP on NUS-WIDE.\n \n**A5**: Thanks for your comment. We have conducted additional comparisons with DCMIP and PCL on NUS-WIDE under 60% symmetric noise. The results are as follows:\n\n|Method|mAP|\n|-|-|\n|DCMIP|60.57|\n|PCL|44.05|\n|SEGA|61.86|\n\nSEGA outperforms both recent methods, showing a clear advantage in noisy multi-label retrieval. We will include these results in the revised version.\n\n\n> **Q6**: Computational Efficiency: What is the training time/memory cost of SEGA vs. STAR on MS COCO?\n\n**A6**: Thanks for your comment. Please refer to **A2** for detailed comparisons, SEGA achieves significantly higher mAP while maintaining comparable training time and memory cost on MS COCO.\n\n> **Q7**: Theoretical Assumptions: Validate Theorem 3.1’s conditional independence with real-world noise datasets.\n\n**A7**: Thanks for your comment.\n\nThe conditional independence assumption in Theorem 3.1 serves as a simplifying modeling choice to derive a tractable uncertainty weighting function. While strict independence may not always hold in real-world datasets, we provide the following empirical support:\n\nOn real-world noisy datasets such as WebVision, the learned uncertainty (via energy and structural divergence) correlates strongly with label correctness, and improves performance over baselines without such modeling.\n\nIn practice, the two signals, energy-based confidence and structure-based divergence, capture complementary aspects of noise: one from semantic consistency, the other from neighborhood alignment. This supports their use in a combined uncertainty estimation framework, even if conditional independence is approximate.\n\nWe will clarify this modeling assumption in the revised version.\n\n\n> **Q8**: Small-Dataset Performance: Report results on CUB-200 with 50% symmetric noise.\n\n**A8**: Thanks for your comment. Please refer to **A4**, SEGA outperforms strong baselines on CUB-200 with 50% symmetric noise, demonstrating strong robustness in small-scale settings.\n\n\n> **Q9**: SEGA relies on pre-trained VGG-16 features, limiting generalizability to feature-agnostic scenarios. The framework’s sensitivity to hyperparameters (e.g., K in structure-based divergence) and lack of real-world deployment cases further restrict its impact.\n\n**A9**: Thanks for your comment.\nWe acknowledge that SEGA and all baselines leverages pre-trained features (e.g., from VGG-16). This is standard practice in deep hashing, and ensures fair comparison focused on robust supervision under noisy labels. To assess the generalizability of SEGA beyond pre-trained settings, we also trained it without any pre-trained backbone and report the results:\n\n|Method|FLICKR25K|NUS-WIDE|\n|-|-|-|\n|CSQ(pre-train=True)|60.38|44.92|\n|Jo-SRC(pre-train=True)|60.42|46.12|\n|STAR(pre-train=False)|58.31|42.87|\n|SEGA(pre-train=False)|62.09|48.33|\n|SEGA(pre-train=True)|74.49|61.86|\n\nEven without pre-training, SEGA outperforms all baselines that use pre-trained features, showing strong adaptability.\n\nTo assess SEGA’s scalability and robustness under real-world noisy supervision, we conducted experiments on the WebVision dataset [1], which contains 2.4 million web images across 1,000 classes. We used the Google image subset for training. The results are as follows:\n\n|Method|WebVision (mAP)|\n|-|-|\n|DIOR|27.82|\n|STAR|32.64|\n|SEGA|34.71|\n\nSEGA achieves the best performance, demonstrating both scalability to large datasets and robustness under real-world web noise. We will include this result in the revised version.\n\n[3] WebVision Database: Visual Learning and Understanding from Web Data. 2017\n\nIn light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points."}, {"review_id": "Ll9josfGlQ", "reviewer": "Reviewer_Fzag", "summary": "SEGA is a deep hashing method for noisy multi-label data that focuses on shaping the geometry of the representation space. Instead of just cleaning or reweighting labels, it trains the model to form smooth, class-consistent manifolds even under label noise.\n\nIt introduces three components:\n\n1. Prototype-Guided Anchoring: Learnable class prototypes pull clean sample embeddings toward class centers, promoting semantic alignment.\n2. Uncertainty-Guided Calibration: Each label is weighted based on a sample-specific uncertainty score, which combines prediction confidence and local feature consistency. Noisy labels are downweighted accordingly.\n3. Structure-Aware MixUp: Interpolation is done between similar samples with differing uncertainty, smoothing boundaries without mixing unrelated classes.\n\nFinally, SEGA brings a geometric perspective to noisy-label learning and shows strong retrieval performance under heavy noise.", "questions": "1. Computational Overhead and Scalability: SEGA requires maintaining class prototypes and computing $K$-nearest neighbors per sample to estimate structural divergence $\\delta_i$. For datasets with tens of thousands of images, was this handled efficiently (e.g., batch-wise approximations, FAISS)? How does SEGA's training time compare to standard hashing baselines? If scaled to millions of samples or hundreds of classes, would these operations become a bottleneck? It would be helpful to know if any optimizations (e.g., periodic prototype updates or mini-batch $K$-NN) were used. Also the retrieval done within the training set? Would it be overfitted easily?\n2. Multi-Label Handling: SEGA normalizes multi-hot labels into a probability distribution for prototype alignment, similar to OrthoHash. Did you consider using standard multi-label losses like independent sigmoid cross-entropy? If so, how did they compare in practice?\n3. Behavior in Noise-Free or Low-Noise Settings: While SEGA targets noisy labels, how does it perform when supervision is clean? Could the added structure (e.g., prototype anchoring or guided MixUp) limit performance in noise-free scenarios? Any results or insights on 0% noise would clarify if robustness comes at a cost in clean settings.", "limitations": "1.  Most limitations are acknowledged in the appendix, but a dedicated section in the main paper would improve transparency. A key assumption is the presence of known class structure -- SEGA relies on predefined prototypes and assumes each sample has at least one true label. It may struggle with open-set noise or entirely unlabelled data.\n2. SEGA depends on tuning the clean sample threshold $q_r$, which may be difficult under very high noise or skewed alignment distributions. Early-stage prototype alignment might be unreliable when noise exceeds 90%, though the paper mitigates this via dynamic updates. In biased or asymmetric noise (e.g., label flips between visually similar classes like dog vs. wolf), prototype drift or collapse could occur. More discussion on how SEGA handles non-uniform noise or partial label missingness would be valuable.\n3. The method's complexity is also a practical limitation -- it introduces more moving parts and hyper-parameters than some simpler baselines. This might make reproduction or deployment harder, though the release of code helps alleviate that. Finally, while the authors showed robustness on synthetic noise, applying SEGA to real-world noisy datasets (with unknown noise type) would be the next step to fully validate it.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Quality:\n\n1. The paper is technically solid and thorough. The method is well-motivated, leveraging ideas from metric learning and robust training. Each SEGA module is justified both intuitively and theoretically.\n2. The inclusion of formal guarantees (Theorems 3.1 and 3.2) is a strength -- Theorem 3.1 links the uncertainty score to true label correctness, supporting label reweighting; Theorem 3.2 ensures interpolating with prototype-aligned neighbors preserves class consistency.\n3. Experiments are strong: SEGA outperforms prior work across four datasets and two noise types, especially under heavy noise (e.g., 80% on NUS-WIDE, where others drop below 50% mAP while SEGA stays above 60%).\n4. Ablations (Table 4) show that removing any component leads to performance drops, confirming each part is useful.\n5. Weakness 1: The added complexity. SEGA introduces several hyperparameters (e.g., \n$q_r$, $\\alpha$, $K$) and requires prototype updates and k-NN per epoch. While some sensitivity is analyzed, runtime/memory overhead is not discussed. The method is rigorous and effective, with minor concerns around scalability and reproducibility.\n\nClarity:\n\n1. The writing is clear and well-organized.\n2. Motivation is strong, and experiments are extensive and easy to follow.\n\nSignificance:\n\n1. Robust multi-label hashing under noise is practically important.\n2. SEGA brings a new perspective: shaping semantic geometry instead of just filtering labels. The gains in high-noise settings are substantial.\n3. The approach could extend beyond hashing (e.g., to classification or structured outputs), opening doors for future work.\n4. Weakness 2: the method is evaluated only in the multi-label retrieval setting, which is narrower but still highly relevant (e.g., web images, tag-based search).\n5. Overall, this is a meaningful contribution both scientifically and practically.\n\nOriginality:\n\n1. The shift from “fixing labels” to “shaping geometry” is a novel framing for noisy-label learning in hashing.\n2. Weakness 3: SEGA combines known tools (prototypes, uncertainty weighting, MixUp), but the way they're fused is original and purposeful.\n3. Using neighborhood consistency with energy-based confidence to form $U_i$ , and guiding MixUp and contrastive learning with that, are novel in this context.\n4. Contrastive learning applied only to noisy samples is a nice detail that helps regularize without reinforcing noise.\n5. One possible critique is that the method combines many known techniques -- but the integration is guided by a clear principle and leads to strong results. It’s an original contribution, even if composed of familiar pieces.", "was_revised": true, "final_justification": "thanks authors for their rebuttal and extensive evaluation. concerns addressed and i am keeping my score as Borderline Accept.", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> **Q1**: The added complexity. \n\n**A1**: Thanks for your comment. \nWhile SEGA introduces multi-component design, they are implemented efficiently and scale well. As shown below, SEGA’s training time per epoch and memory usage are comparable to competitive baselines, yet it achieves significantly better performance:\n\n|Method||FLICKR25K|||MS COCO||\n|-|-|-|-|-|-|-|\n||time (s)|memory (MB)|mAP|time (s)|memory (MB)|mAP|\n|CSQ|87.00|1322.00|58.88|55.00|1624.00|33.71|\n|STAR|179.00|1287.00|70.84|161.00|1425.00|41.87|\n|SEGA|160.00|1569.00|72.66|182.00|1879.00|46.50|\n\nSEGA outperforms all baselines by a large margin in mAP, while maintaining similar or even lower training time on Flickr25K. We will include these results in the revised version to clarify SEGA’s efficiency-performance tradeoff.\n\n> **Q2**: the method is evaluated only in the multi-label retrieval setting, which is narrower but still highly relevant (e.g., web images, tag-based search).\n\n**A2**: Thank you for your comment. While SEGA is evaluated primarily in the noisy multi-label retrieval setting, this scenario is highly relevant to real-world applications such as web image retrieval and tag-based search.\n\nTo further demonstrate SEGA’s generalizability, we conducted a zero-shot retrieval experiment following [1]. We split CIFAR-10 into seen and unseen categories and trained only on the seen classes. For example, “airplane” (or “dog” / “truck”) was treated as the unseen class, and 1,000 query images were retrieved from a mixed database of 59,000 images.\n\n|Method|airplane|dog|truck|\n|-|-|-|-|\n|DIOR|22.19|30.66|33.17|\n|STAR|26.09|35.97|37.23|\n|SEGA|30.50|38.24|39.44|\n\nSEGA outperforms strong baselines on all unseen categories, indicating its robust generalization to out-of-distribution concepts. We will include this setting and results in the revised version.\n\n[1] Zero-Shot Hashing via Transferring Supervised Knowledge. ACM MM 2016\n\n**Q3**: SEGA combines known tools (prototypes, uncertainty weighting, MixUp), but the way they're fused is original and purposeful.\n\n**A3**: Thank you for your comment. While prototype learning and uncertainty-based weighting have been explored in prior works, SEGA adopts a substantially different formulation and integration strategy:\n\n- SEGA learns semantic prototypes via backpropagation in the embedding space, adapting them to the evolving representation distribution during training. These prototypes are used not only for contrastive alignment but also to guide structure-aware interpolation.\n\n- Our uncertainty estimation combines energy-based prediction confidence with structure-based divergence, refining the semantic geometry by suppressing unreliable gradients.\n\n- Our structure-aware MixUp selectively interpolates between semantically similar but uncertainty-dissimilar samples, rather than applying naive input-level interpolation.\n\nThese components are not used in isolation, but are tightly coupled in a closed-loop framework that encourages mutual refinement of representation, supervision, and uncertainty.\n\nWe believe this principled design and novel formulation go beyond a simple combination of existing tools. We will make this contribution more explicit in the revision.\n\n\n> **Q4**: Computational Overhead and Scalability: SEGA requires maintaining class prototypes and computing $K$-nearest neighbors per sample to estimate structural divergence $\\delta_i$. For datasets with tens of thousands of images, was this handled efficiently (e.g., batch-wise approximations, FAISS)? How does SEGA's training time compare to standard hashing baselines? If scaled to millions of samples or hundreds of classes, would these operations become a bottleneck? It would be helpful to know if any optimizations (e.g., periodic prototype updates or mini-batch \nK-NN) were used. Also the retrieval done within the training set? Would it be overfitted easily?\n\n**A4**: Thank you for your thoughtful question.\n\n- Training efficiency: SEGA employs lightweight optimizations, including FAISS-accelerated K-NN search, to efficiently estimate structural divergence $\\delta_i$.\n\n- Training cost comparison: As shown in **A1**, SEGA achieves comparable training time and memory usage to strong baselines like STAR, while significantly outperforming them in mAP, demonstrating a favorable efficiency-performance tradeoff.\n\n- Scalability: SEGA’s design naturally scales to larger datasets. Techniques suggested by you, such as periodic prototype updates and mini-batch K-NN, can be readily integrated to support larger-scale training (e.g., millions of samples or hundreds of classes). Moreover, other key computations are local to each batch and support efficient parallelization or distributed training.\n\n- Overfitting concern: Retrieval is conducted on a large retrieval database, while the training set constitutes only a small portion of it. This setup naturally avoids overfitting and follows the standard evaluation protocol in the hashing literature.\n\n> **Q5**: Multi-Label Handling.\n\n**A5**: Thank you for your comment.\nInstead of using independent sigmoid cross-entropy, SEGA normalizes multi-hot labels into a probability distribution to guide prototype-based alignment. This formulation enables consistent semantic geometry shaping in multi-label settings.\n\nThe normalized label distribution guides the representation toward a set of semantic anchors, i.e., class prototypes, according to their relative importance. This allows SEGA to learn embeddings that are geometrically aligned with the full semantic context of each sample, rather than pulling independently toward individual labels. Such alignment promotes smooth and structured class manifolds, especially under noisy supervision.\n\nEmpirical results with 64-bit hash codes under 60% pairflip noise confirm the benefit:\n\n|Method|FLICKR25K|NUS-WIDE|\n|-|-|-|\n|SEGA with independent sigmoid cross-entropy|73.45|60.74|\n|SEGA|74.49|61.86|\n\nWe will clarify this design motivation and its advantages in the revised version.\n\n\n> **Q6**: Behavior in Noise-Free or Low-Noise Settings.\n\n**A6**: Thank you for the thoughtful question.\nWe conducted experiments under 0% label noise to assess whether SEGA’s structural components (e.g., prototype anchoring, structure-aware MixUp) introduce any overhead in clean settings. On the FLICKR25K dataset with 64-bit hash codes, we observe the following:\n\n|Method|FLICKR25K|\n|-|-|\n|DIOR|75.07|\n|STAR|76.72|\n|SEGA|78.68|\n\nSEGA achieves better performance than strong baselines in noise-free scenarios. This indicates that its geometry-aware design benefits generalization and does not rely on label noise to be effective. We will include this result in the revised version.\n\n> **Q7**: Most limitations are acknowledged in the appendix.\n\n**A7**: Thank you for your helpful suggestion.\nWe will add a dedicated limitations section in the main paper in the revised version.\nSpecifically, SEGA assumes known class structure and the presence of at least one true label per sample. As noted, this may limit performance under open-set noise or fully unlabeled data. Extending SEGA to more flexible or open-world scenarios, e.g., via prototype discovery or unsupervised pretraining, remains promising future work.\n\n> **Q8**: SEGA depends on tuning the clean sample threshold $q_r$, which may be difficult under very high noise or skewed alignment distributions. \n\n**A8**: Thanks for your comment. We agree that very high noise levels pose challenges to prototype-based methods. However, SEGA’s design, including dynamic prototype updates and structure-aware calibration, is specifically intended to address this. To evaluate robustness under extreme conditions, we conducted additional experiments on Flickr25K with 95% noise:\n\n|Method|FLICKR25K(95%)|\n|-|-|\n|STAR|62.82|\n|SEGA|65.43|\n\nSEGA still achieves strong performance, outperforming STAR by a notable margin. This suggests that the learned semantic structure remains reliable even when the clean/noisy partition becomes highly ambiguous. We will include this result and further discussion on handling non-uniform noise and partial label missingness in the revised version.\n\n\n> **Q9**: The method's complexity is also a practical limitation -- it introduces more moving parts and hyper-parameters than some simpler baselines. This might make reproduction or deployment harder, though the release of code helps alleviate that. Finally, while the authors showed robustness on synthetic noise, applying SEGA to real-world noisy datasets (with unknown noise type) would be the next step to fully validate it.\n\n**A9**: While SEGA incorporates multiple components, each is conceptually simple and grounded in well-established principles. Moreover, we have released our code, which includes default settings to facilitate reproducibility.\n\nTo further validate SEGA’s practical applicability, we conducted experiments on the WebVision dataset [1], which comprises 2.4 million web images across 1,000 categories. We used the Google image subset for training. SEGA achieves the best performance:\n\n|Method|WebVision (mAP)|\n|-|-|\n|DIOR|27.82|\n|STAR|32.64|\n|SEGA|34.71|\n\nThis demonstrates SEGA’s scalability and robustness to real-world, unknown-type noise, going beyond synthetic noise settings. We will include this result in the final version.\n\n[1] WebVision Database: Visual Learning and Understanding from Web Data. 2017\n\n\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}, {"review_id": "Du3ykjkWQm", "reviewer": "Reviewer_9nS7", "summary": "To learning from noisy multi-label supervision in hashing, this paper introduces a geometric perspective, formulating multi-label hashing as a semantic structure alignment problem rather than a conventional label recovery task. Moreover, this paper proposes a closed-loop framework termed SEGA that integrates prototype anchoring, uncertainty-calibrated soft labels, and structure-aware interpolation into a unified and geometry-aware semantic learning process. The experiments in the paper are very abundant and perfectly validate the innovations of the paper.", "questions": "In addition to the Weaknesses, I also have the following concerns:\nMore textual explanation is needed on why the contrastive learning loss is robust on noisy datasets. The origin of this contrastive learning should be correctly cited. Please confirm whether using the contrastive learning loss on the clean dataset would further enhance the performance.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1) This paper introduces a geometric perspective for learning from noisy multi-label supervision in hashing.\n\n2) The SEGA framework has a certain degree of innovativeness and has effectively solved the current problems. \n\n3) The experiments in the article are very abundant and perfectly validate the innovations of the paper.\n\n4) The writing of the article is highly logical, and the charts and graphs are also very clear.\n\nWeaknesses\n\n1) In section 3.3, the accuracy of the prototype will directly determine the quality of the division between clean and noisy samples, if its accuracy is ensured.\n\n2) It is suggested that the relationship between Energy-Based Prediction Confidence and samples could be demonstrated with figure, for example, blue represents correct samples and red represents noisy samples.\n\n3) Please explain intuitively why label soft reweighting in Eq. (9) can be beneficial for noisy learning. How would the performance be if we use Energy-based prediction confidence or Structure-based divergence alone, without employing Unified uncertainty?", "was_revised": true, "final_justification": "I appreciate the author's response, which provided ample clarification and addressed my concerns. Therefore, I'm willing to maintain my rating.", "rebuttal": "We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.\n\n> **Q1**: In section 3.3, the accuracy of the prototype will directly determine the quality of the division between clean and noisy samples, if its accuracy is ensured.\n\n**A1**: Thanks for your comment. While SEGA does not require perfectly accurate prototypes, we empirically find that the learned prototypes align well with clean samples and provide reliable signals for clean/noisy separation. This alignment yields reasonably accurate partitions, which are used in a soft manner for uncertainty calibration. Moreover, SEGA’s dynamic prototype updates and structure-aware design mitigate potential early-stage errors. We will clarify this in the revised version.\n\n> **Q2**: It is suggested that the relationship between Energy-Based Prediction Confidence and samples could be demonstrated with figure, for example, blue represents correct samples and red represents noisy samples.\n\n**A2**: Thanks for the suggestion. We computed the ratio of correct vs. noisy samples across energy score deciles and found a clear correlation: samples with lower energy scores are significantly more likely to be correct, validating the effectiveness of our confidence estimator. We will include a visualization (color-coded histograms or scatter plots) in the revised version to illustrate this relationship more intuitively.\n\n\n> **Q3**: Please explain intuitively why label soft reweighting in Eq. (9) can be beneficial for noisy learning. How would the performance be if we use Energy-based prediction confidence or Structure-based divergence alone, without employing Unified uncertainty?\n\n**A3**: Thanks for the question. Label soft reweighting in Eq. (9) helps suppress noisy supervision by down-weighting unreliable labels based on sample-specific uncertainty. Instead of relying on a single signal, SEGA fuses energy-based prediction confidence and structure-based divergence, capturing both model belief and local consistency. Empirically, this unified uncertainty yields better performance than either source alone:\n\n|Variant|Flickr25K(mAP)|\n|-|-|\n|Energy-only|73.49|\n|Structure-only|73.93|\n|SEGA|74.49|\n\nThis shows the benefit of integrating both semantic and structural cues for robust supervision. We will clarify this intuition in the revision.\n\n\n> **Q4**: In addition to the Weaknesses, I also have the following concerns: More textual explanation is needed on why the contrastive learning loss is robust on noisy datasets. The origin of this contrastive learning should be correctly cited. Please confirm whether using the contrastive learning loss on the clean dataset would further enhance the performance.\n\n**A4**: Thanks for the insightful comment. Contrastive learning is inherently robust to label noise as it relies on instance-level similarity rather than explicit label correctness. In SEGA, we apply contrastive loss selectively on samples estimated to be noisy, encouraging them to align with trusted class prototypes and avoid reinforcing incorrect labels.\n\nWe will properly cite relevant prior work (e.g., SimCLR [1], SupCon [2]) in the revision.\nAdditionally, we confirm that contrastive learning also improves performance under clean supervision:\n\n|Method|Flickr25K(mAP)|\n|-|-|\n|SEGA(without contrastive loss)|77.23|\n|SEGA(with contrastive loss)|78.68|\n\nThis suggests that contrastive regularization complements clean and noisy learning alike by enforcing semantic consistency.\n\n[1] A Simple Framework for Contrastive Learning of Visual Representations. ICML 2020\n\n[2] Supervised Contrastive Learning. NeurIPS 2020\n\nThanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions."}]

Author rebuttal:

Meta-review: The paper proposes SEGA (Semantic Geometry Shaping) for robust hashing under noisy multi-label supervision, integrating three key components: (1) prototype anchoring, (2) uncertainty-calibrated soft labels, and (3) structure-aware interpolation. The authors offer a fresh and compelling perspective on learning under noisy supervision by shifting the focus from mere label correction to actively shaping the semantic geometry of the representation space. Moreover, they provide theoretical analyses and validate its effectiveness through extensive experiments across diverse tasks, yielding notable performance improvements.

All reviewers appreciated the clear motivation, novelty, and practical value of the work. The primary concerns raised included the use of small to medium-sized datasets, insufficient discussion on the robustness of hyperparameters, and a lack of comparisons with recent methods. The authors’ detailed rebuttal effectively addressed most of these concerns, culminating in unanimous acceptance by the conclusion of the discussion. Therefore, the AC recommends acceptance.
Decision: Accept (poster)