Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Authors: Peng Wang
Yong Li
Lin Zhao
Xiu-Shen Wei
Venue: ICML 2025 poster (2025)
Keywords: Large-Scale Fine-Grained Image Retrieval
Learning-to-Hash
Attribute-Aware
Deep Learning
Primary area: applications->computer_vision

Abstract: Fine-grained hashing has become a powerful solution for rapid and efficient image retrieval, particularly in scenarios requiring high discrimination between visually similar categories. To enable each hash bit to correspond to specific visual attributes, we propose a novel method that harnesses learnable queries for attribute-aware hash code learning. This method deploys a tailored set of queries to capture and represent nuanced attribute-level information within the hashing process, thereby enhancing both the interpretability and relevance of each hash bit. Building on this query-based optimization framework, we incorporate an auxiliary branch to help alleviate the challenges of complex landscape optimization often encountered with low-bit hash codes. This auxiliary branch models high-order attribute interactions, reinforcing the robustness and specificity of the generated hash codes. Experimental results on benchmark datasets demonstrate that our method generates attribute-aware hash codes and consistently outperforms state-of-the-art techniques in retrieval accuracy and robustness, especially for low-bit hash codes, underscoring its potential in fine-grained image hashing tasks.

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Peng Wang' Yong Li
Lin Zhao Xiu-Shen Wei 2t
Abstract
1. Introduction
Fine-grained hashing has become a powerful so- lution for rapid and efficient image retrieval, par- ticularly in scenarios requiring high discrimina- tion between visually similar categories. To en- able each hash bit correspond to specific visual attributes, we propoe novel method that har- nesses learnable queries for attribute-aware hash codes learning. This method deploys a tailored set of queries to capture and represent nuanced attribute-level information within the hashing pro- cess, thereby enhancing both the interpretability and relevance of each hash bit. Building this query-based optimization framework, we incor- porate an auxiliary branch to help alleviate the challenges of complex landscape optimization of- ten encountered with low-bit hash codes. This auxiliary branch models high-order attribute in- teractions, reinforcing the robustness and speci- ficity of the generated hash codes. Experimental results on benchmark datasets demonstrate that our method generates attribute-awan hash codes and consistently outperforms state-of-the-art tech- niques in retrieval accuracy and robustness, es- pecially for low-bit hash codes, underscoring its potential in fine-grained image hashing tasks. Our code is available at https: / / /githbb..c.. com/ SEU-VIPGroup: QueryOpt
Large-scale fine-grained image retrieval aims to retrieve images from various subcategories within meta-category, such different species of animals, plant varieties, car models, or types of retail products (Wah et al., 2011; Horn et al., 2017; Wei et al., 2019). With the explosive growth of fine-grained data in real applications, fine-grane tieeggriined hashing has emerged as promising solution for large-scale retrieval tasks (Wang et al., 2018; Cakir et al., 2018; Lin et al., 2015; Hu et al., 2024; Li et al., 2025), hashing method signif- icantly reduces storage costs and increases retrieval speed by using compact binary hash code representations. Unlike fine-grained classification, retrieval task face an open-world problem with unlimited categories. This task is complicated by subtle inter-class variations among similar subcategories and intra-class variations caused by different object pos- tures (Wei et al., 202 lb; Zhao al., 2017; Wei et al., 2017). To address these challenges, it is essential to explore the nuances in object characteristics and leverage high-level Ane-grained features.
Recently, a variety of learning- to-hnsshtt methods have been developed to enhance retrieval performance. In the litera- ture, DSaH (Jin et al., 2020), ExchNet (Cui et al., 2020), SEMICON (Shen al., 2022) and AGMH (Lu et al., 2023) concentrated on the design of feature extraction modules. As for the process of generating hash codes, the final hash codes are derived by projecting intricately coupled image features through simple linear layer. Although the generated hash codes can perform retrieval tasks, a single bit will do not imply significant semantics and it is difficult to preserve all the intricately coupled global features of an image. Addi- tionally, A² 2-NET (Wei et al., 2021a) and A A--NET+ (Wei et al., 2023a) employs reconstruction task to constrain the process of generating hash codes, allowing each bit of hash codes to indicate attribute-level information. This nuanced attribute-leve information enables us to effectively identify different categories. Consequently, hash codes can perform retrieval tasks effectively while ensuring that each bit is interpretable. Following the idea of utilizing attribute-level information hash codes, we propose novel method to generate attribute-aware hash codes for fine-grained image retrieval.
'School of Computer Science and Engineering, Nanjing Uni- versity of Science and Technology, Nanjing, China. "School of Computer Science and Engineering. and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications. Southeast University. Nanjing, China. This work was upported by National Key R&D Pro- gram of China YFA1001 100), National Natural Science Foundation of China under Grant (62272231, 62172222), CIE- Tencent Robotics X Rhino-Bird Focused Research Program, the Fundamental Research Funds for the Central Universities (4009002401, (,09000990, and the Big Data Computing Cen- ter of Southeast University. Correspondence to: Xiu-Shen Wei <weixs.gns gm @ gmail.com>.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
In this paper, we model the hash problem as set prediction problem, where each element in the set represents bit of the hash codes that is able to indicate a specific visual attribute. By structuring hash problem in this manner, an image can be directly decomposed into a set of atrribut-speccffic features through learnable queries. Then each well-decomposed attribute-ppecffic features is compressed into a final bit of hash codes for retrieval. Building upon this core idea, our proposed query optimization method integrates query learn- ing to generate attribute-aware hash codes. Additionally, we incorporate an auxiliary branch for better optimization, further improving retrieval performance.
2. Related Work
Fine-Grained Deep Hashing Hashing-based retrieval is representative method for Approximate Nearest Neigh- bor Search (Wang et al., 2018; Liu et al., 2011; Li al., 2013; Ye et al., 2022). Specifically, FPH (Yang et al., 2019) and DSaH (Jin et al., 2020) were among the earliest meth- ods to incorporate hashing into fine-graine image retrieval. FPH introduced a novel two-pyramid hashing architecture to learn both semantic information and subtle appearance details. DSaH for the first time introduced the attention mechanism to the learning of fine-grained hashing codes, During the same period, ExchNet (Cui et al., 2020) investi- gated the large-scale fine-grained hashing task, proposing unified end-to-end trainable network that captures both local and global features, representing parts and wholes of fine- grained objects. A (Wei et al., 2021a) employed a similar localization module and attempted to learn high- level attribute vectors for hash code generation; its enhanced version, - (Wei et al., 2023a), further boosts the model's self-consistency. SEMICON (Shen et al., 2022) proposed suppression eenhannii mask to explore the re- lationships between discovered regions. It generates the final hash codes in a stage-by-stage manner based on fea- tures from different levels, rather than aggregating features from different levels to generate unified hash codes. Most recently, AGMH (Lu et al., 2023) introduced attention dispersion loss and a step-wise interactive external atten- tion module to group and embed category-specific visual attributes in multiple descriptors for comprehensive feature representation. In this paper, we primarily follow the setup proposed by DPSH (Li et al., 2016b). Other notable works, such as FISH (Chen et al., 2022), CNET (Zeng Zheng, 2023), DAHNET (Jiang et al., 2024) and CMBH (Chen et al., 2024) utilized classification tasks to enhance feature representation, which differ from the pairwise supervision setup and are not the primary focus of this paper.
For query learning, a set of randomly initialized learnable queries is used for querying diverse attributes. Through a de- coder based on cross-attention mechanism, these learnable queries interact with the extracted global features to directly decouple different attribute-specin specifcc features and then pro- duce bits containing visual attributes level information. This direct querying method also demonstrates effectiveness and accuracy in other Dision-related tasks (Carion et al., 2020: Kirillov et al., 2023; Jaegle et al., 2021). However, with the aforementioned process, we observe that the directly query learning strategy suffers from severely poor retrieval per- formance in low-bit hash codes scenarios. We analyze this phenomenon from the perspective of cosine similarity and discover the intrinsic reason lies in the inherent limitation of large class Numbers with low feature dimensions, resulting a challenges of complex landscape during the optimization process. This limitation makes difficult for the model to learn distinguishable features. Based on the analysis, we em- ploy auxiliary branch only for training to help alleviate this challenges without introducing additional parameters. With this strategy, the model's performance is significantly enhanced, especially in low-bit scenarios. To evaluate our models, we conduct extensive experiments that provide both quantitative and qualitative results, and we also offer further experimental analysis of our method. The contributions of this work can be concluded as:
Set Prediction and Parallel Decoding The basic set pre- diction task is multilabel classification (Rezatofighi et al., 2017; Pineda al., 2019; Li et al., 201 Furthermore, DETR (Carion et al., 2020) presented new method that viewed object detection as direct set prediction problem. It introduced an mecoder-decoder transformer architecture that significantly simplified the object detection pipeline. Given a fixed, small set of learned object queries, DETR reasoned about the relationships among objects and the global image context to directly output the final set of predictions. Since the introduction of DETR, various modifications and im- provements have been made (Zhu et al., 2021; Lin et al., 2022; Shehzadi et al., 2023; Chen et al., 2025), while the core concept of set prediction and the overall structure of the pipeline have remained intact. In our work, we treat the hashing problem as a set prediction task for distinguishable
We tailor the prevalent query learning mechanism to generate attribute-an aware hash codes for fine- one-grained image retrieval tasks.
We explain from the perspective of cosine similarity why retrieval performance is poor in low-bit hash codes scenarios. Based on our analysis, we seamlessly incor- porate an auxiliary branch with query learning mecha- nism, which significantly improved the model's 's perfor- mance.
Experiments conducted on five commonly used bench- mark datasets for fine- -raiind image retrieval illustrate the effectiveness of our proposed method.
2

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Input Image
Query for Allributes
Speeciff Features
Haah Codes
Compress
Sign(
Extractor
Decoder
out
h
Refine
Blue Head
Retricval
1
Rodupe
Staged
Orange Body
Distinguishable Attributes
PFN
Cony Stage3
Cony Stage2
Rashaae
Positional Encoding
Conv Stage1
in
Input Image
Subtle Feature Extractor
Gallery Set
Figure 1. Overall framework of our method for generating attribate-aware hash codes for fine-grained image retrieval. The top part of the figure shows the workflow the method: given group of learnable parameters, an image decoupled into set of visual attributes, and each attribute compressed into single bit, serving the hash code for retrieval. The bottom left part details the implementation of the Subtle Feature Extractor, where MHSA stands for multi head self attention and FFN stands for feed-for feed-forward network. The bottom right part illustrates motivation for using distinctive attributes hash codes for retrieval.
visual attributes which are more fine-grained than objects.
details are presented in the following sections.
Parallel decoders are widely used the field of deep learn- ing, particularly scenarios that require fast and efficient decoding, such as machine translation, text generation, and speech recognition (Vaswani al., 2017: Chan et al., 2020). DETR (Carion al., 2020) also combined with parallel decoding, achieving a suitable balance between the compu- tational cost and the ability of performing global computa- tions. The main advantage of parallel decoders was their ability process multiple data points simultaneously, rather than sequentially, which greatly enhanced the speed and ef- ficiency of decoding. For hash problems, parallel decoders handle the generation of variable-length hash codes, adding flexibility to the decoding process.
3.1. Subtle Feature Extractor
In fine- ane-grained tasks, images often contain extremely rich and subtle features (Gao et al., 2021; Haoran al., 2023; Wei et al., 2023b). To obtain richer representations, and en- sure that the decoding process is not hindered by this stage, we adopted two efficient strategies for subtle feature encod- ing, as shown in the bottom left part of Figure 1: (1) We constructed top-down architecture to capture multi-scale features, thereby enabling the model effectively capture the local characteristics of image.
(2) We incorporated self-attention mechanisms to eliminate irrelevant informa- tion, allowing the model to focus on identifying the most crucial regions.
3. Methodology
Specifically, a raw pixel input image represented as I, E ; In), where is the total number of train- ing instances. We extract the deep feature of an input image I, via CNN backbone PCNN by:
Figure illustrates the procedure for generating attribute- aware hash codes for an image, which comprised of two key components: (1) Subtle Feature Extractor (S, cf. Sec- tion 3.1) that obtains global image features for subsequent hash codes generation, and (2) decoder with Query Leam- ing mechanism (Q, cf. Section 3.2) to decouple the extracted global features into distinguishable attributes for the subse- quent generation of attribute-aware hash codes. Addition- ally, we introduce an Auxiliary Branch (A, cf. Section 3.3) which reuses the aforementioned decoder for more effective attribute decoupling and performance enhancement. The
(X/li {1, .I,....AA [.....I]) PCNN
(1)
where the feature map X Xuy Xhy is the output of the jth stage (from top to bottom) of the backbone. Then, we fuse the features by:
if 1, X - ØCon (X)) sap (X!-1) if < j $ L,
X
(2)
3

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
where a 1 x convolution layer &Couv adjusts the number of channels Cj and Oup denotes upsampling operation.
distinguishable attribute-specific features a; E RR which can be compressed into h; ||a.l| W. where W E Rdx1
Note that we use Q = [1 42. ..... qa] T E R to represent k learnable queries and abstract the aforementioned process as a function h = (I, Q. )) E IRA, where the vector is associated with parameters Q and θ
To further refine the multi-scale features, a 1 x convolution operation is first applied to reduce the channel dimension of the feature maps to d. The feature maps are then flattened and concatenated. Finally, {X j E (I,...,,,,,, L}} is trans- formed into Din EdxE, where F = Σラ w x hz. This sequence can then be processed by standard transformer block. To improve computational efficiency, we only use a single layer of multi-head self-attention (MHSA) and a fred-fowward network (FFN) to allow the model to focus on identifying significant regions.
Optimization Objective The optimization objective is defined based on the vector h. that have been obtained. As- suming that we have p test data points denoted as {pi) and d gallery set points denoted [d,) For each at- tribute vector p: and dj. the corresponding hash codes can be generated by:
Conclusively, the hybrid convolution and attention structure is utilized to enforce the model to focus on crucial fine- grained regions in the input image (Guo et al., 2022; Yang et al., 2021; Yu et al., 2024). The output Tout E RExd will be further processed.
e: = sign(p:), 2j = sign(d,).
(4)
In order to learn hash codes whose Hamming distances are minimized on similar image pairs and simultaneously maximized dissimilar image pairs, we minimize the 62 loss between the pairwise supervised information and the inner product of query-database binary code pairs. Following (Jiang & Li, 2018), the formulation of hash code learning can be expressed through inner products as:
3.2. Query Optimization
Query Learning Unlike other works that directly obtain hash codes through fully connected layers, our method lever- ages decoder and a group of learnable queries to automat- ically decouple different attribute features from the global features Dout. This decoupling process is illustrated at the middle top part of Figure 1. The automation in this pro- cess refers to the fact that it relies solely on the supervi- sion signal indicating whether image pairs are similar or not, rather than explicit local attributes labeling informa- tion. Our query learning process decouples attribute-level features from intricately coupled global features. Then, each resulting attribute-ppecific features are compressed into one dimension and considered as a bit of hash codes. The decoder is implemented through cross-attention mech- anism. To effectively decode diverse attribute- specifci fea- tures, we randomly initialize k different learnable attribute qq; E ii E {l....,k)) }}}} to interact with Fout through the decoder. This random initialization ensures the diversity of the queries, allowing queries to naturally focus on different features at the beginning of the learning process. This decoding process can be formulated as:
v. zj
min L(I) 0.Q
Sig
(5)
IEN JET
where v, = sign(h) and presents the indices of all the gallery set points while Ω C I presents the indices of the train points for we can only gain access to subset of gallery set points. S.j- E {0, 1} represents the pairwise supervised information, where Sij- = if images and j belong to the same category, and 0 otherwise (Leng et al., 2014).
However, the gradient can not be paok-propagated owing to the sign function. Hence, we substitute sign(-) for tanh(.) to relax the whole optimization process. Typically, con- verting continuous codes v to binary codes b will lead to information loss, which is also known as quantization error. Therefore, we introduce a quantization loss to mitigate the impact of the relaxation. The loss of query optimization can be reformulated as:
7
a." =sofmma
(g. Wo) (aout mn m (q.
(Fout W,)
L(T) = 3
D. 2j
k
Sis
+
[2,
V/3 12
Vd
IER JERJET JET
iE
(6)
a: :.=CCa((a...... W.
where v; = tanl(h).
(3)
where q,' E BR. Ixd represents one learnable query, d' is the dimension of each head, m 1,..., M indices the output from the m-th attention head, and Wg, WR. W. e ddxd represent the learnable projection matrices. Additionally, tout denotes the resition-augmentee Tost. We concatenate the outputs of each heads and using projection W. to get the
3.3. Auxiliary Branch
Observations of Poor Performance in Low-Bit Scenarios We conducted experiments according to the above process. As shown in Figure 2, the comparison with the baseline demonstrates the effectiveness of our method. However, our
4

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Roofin
Figure Comparison our method (without an auxiliary branch A. cf. Section 3.3) with two previous methods and baseline CUB200.
(a) of dimensions 12
(b) # dimensions 48
Figure The loss landscape under C' -200.
minimum that greater than zero, rather than the ideal zero
What kind of minimum can we optimize achieve? We can randomly select one image from each category of training samples to compute its corresponding relaxed hash codes with tanh(-). Specifically, we assemble the selected codes into matrix V E R where V can define:
07-
[U1, [U1,U2, U2 Vcl. We
Figure Lower bound of a with different values C.
U 1yy
method performs significantly worse in low-bit scenarios compared to the two previous state-of-the-art methods. Typ- ically, it is assumed that low-bit hash codes perform poorly due to their limited bit length, which restricts the amount of information they can represent. Nevertheless, we indicate that this poor performance is primarily due to optimization difficulties associated with low-bit hash codes. the fol- lowing, we provide both detailed analysis of this issue and the corresponding visualization results.
A
max IS
|0:
|0,
max 181JNN
COs asj.
(8)
When C' exceeds n, y has definite lower bound (Welch, 1974) can be formulated as:
C' 7 2 V -1)
(9)
where μ reflects the minimum state of the loss value dur- ing the optimization process. Since the loss function can be interpreted from the perspective of cosine similarity as discussed earlier, if μu is too large (i.e., cos 0xj could po- tentially tend to approach 1), the model may incorrectly perceive samples from different classes as being similar. This makes it difficult for the model to learn distinguishable features. As shown in Figure 3, when increases from 12 to 48, the minimum value of μ decreases rapidly. Due to the rapid decrease of in higher-dimensional spaces, the cosine similarity between samples of different categories can be kept lower level, enabling the model to learn more distinguishable features. In addition, visualization results of the loss landscape using the method from (Li et al., 2018) 8) in Figure 4, which clearly show that in higher dimensions, achieving lower loss values is much easier.
Conjecture the Limitation of Large Class Numbers with Low Feature Dimensions Geometrically, for b E +11,+1}% the dot product can be represented by cosine similarity as:
b,b, | |لراا cos l cos
where both ||b||| and |byl| are constant (i.e., |bb]| VT). This means that we can analyze the loss function from the perspective of cosine similarity in continuous Euclidean space (Hoe et al., 2021). In designing our optimization ob- jective, we actually facilitate distinguishable feature learn- ing by minimizing the squared cosine similarity between the relaxed hash codes of pairs of samples from different image categories. Theoretically, in an n-dimensional space, there can be up mutually orthogonal vectors. However, in practical optimization processes, the number of categories C' exceeds the spatial dimensions n, employing Stochastic Gradient Descent to minimize the sum of squared cosine similarities between all pairs becomes particularly challeng- ing. Under such circumstances, we generally achieve local
present the
Based on the above analysis, we conduct query transforma- tion and add an auxiliary branch solution.
Query Transformation and Auxiliary Branch for Bet- ter Optimization During the training process, our objec- tive is to obtain well-optimized parameter (q: E Rdji
5

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
method can be employed to generate hash codes for an input image I, follows:
Auxiliary Branch
Trais only
Decoder
Compress
= (H(
(I1)
Train and Indorznee
qi
4. Experiment
Transformation
Datasets We conducted series of ablation studies and comparisons with the latest methods on five fine-grained datasets, i.e., CUB200 (Wah et al., 201 1), Aircraft (Maji et al., 2013), Food101 (Bossard et al., 2014), VegFru (Hou et al., 2017) and NABirds (Van Horn et al., 2015). During the evaluation, we used the official training and testing sets as the retrieval and query sets. Specifically, CUB200 contains 11,788 bird images from 200 bird species and is officially split into 5,994 images for training and 5,794 images for testing. Aircraft includes 10,000 images of 100 aircraft vari- ants, with 6,667 images designated for training and 3,333 for testing. For large-scale datasets, Food101 features 101 kinds of foods with 101,000 images, each class has 250 test images that are manually checked for correctness, while the 750 training images may still contain certain amount of noise. NABirds comprises 48,562 images of north ameri- can birds across 555 sub-categories, with 23,929 images for training and 24,633 for testing. VegFru is another large-scale lineigrained dataset covering 200 kinds of vegetables and 92 kinds of fruits, with 29,200 images for training, 14,600 for validation, and 1 116,931 for testing.
41
Loss Function
Compress
Decoder
Xout
Figure 5. The N 1 auxiliary branch with query transformation strategy (illustrated in the figure with N = 2 for simplicity, where N is a hyperparameter) is used only during training.
[L] for effectively decoupling fine-grained at- tributes. With this objective and the motivation to conduct the learning process in a higher dimension to mitigate the an- alyzed limitations above, we conduct query transformation for each learnable query. We evenly slice the query vector and then perform a circular shift operation on the sliced sub-vectors. The shifted é: which shares the same group of parameters with q, is processed with an auxiliary branch. We concatenate the output of both the original branch and the auxiliary branch for optimization. As illustrated in Fig- ure 5, with one auxiliary branch, the numbers of dimensions is doubled. This query transformation and the incorpora- tion of auxiliary branch offers two significant benefits: (1) the learnable queries interact more thoroughly with the en- riched image features, and (2) the number of dimensions is increased without introducing any additional learnable parameters during training, which helps alleviate the chal- lenges of complex landscape optimization encountered in low dimensions.
Implementation Details For fair comparisons of fine- grained hashing, we follow the efficient training setting out- lined in ExchNet (Cui et al.. 2020). Concretely, for CUB200, Aireraft and Food101, we sample 2,000 images per epoch, while 4,000 samples are randomly selected for NABirds and VegFru. Following the method of ExchNet, ResNet-50 (He et al., 2016) is employed in experiments. The number of total training epoch is the same as SEMICON (Shen et al., 2022) and AGMH (Lu et al., 2023).
Mathematically speaking, given a learnable query q: E Rdd, we evenly split it into N equal-length sub-vectors lq!: q q²: :..; ".), then we perform circular shift op- eration the sliced sub-vectors to augment the new query:
For all datasets, we preprocess all images to 224 X 224. Initial learning rate is 0.0003. SGD with mini-battc size of 16 is used for training. We set the weight decay to 0.0001 and momentum to 0.90. We set L, 2 to leverage multi- scale features. d is set to 384, and N is set to 96 All experiments are conducted with one GeForce RTX 3090 GPU.
Φ = 19,
N 1+ :....:.." q.: :......:.""" N- IR. (10)
T.
jherej = 1, N -
As illustrated in Figure 5, in the auxiliary branch, the vanilla decoder is reused to generate auxiliary outputs. During in- ference stage, we only retain the original q, during inference to generate bits hash codes.
Comparison Methods To prove the superiority of our method, we compare with existing hashing- methods. Among them, DPSH (Li et al., 2016b), HashNet (Cao et al., 2017) and ADSH (Jiang & Li, 2018) are coarse-grained methods. ExchNet (Cui et al., 2020), A--NET (Wei et al., 2021 SEMICON (Shen et al., 2022), -E- (Wei et al., 2023a) and AGMH (Lu et al., 2023) are advanced hashing methods aiming at large-scale fine-grained retrieval.
3.4. Out-of-Sample Extension
After the training phase, we obtain the well-learned queries and other modules. Discarding the auxiliary branch, our
6

---

Learning Attribute-Awarc Hash Codes for Fine-Grained Image Retricval via Query Optimization
Table I. Comparisons of retrieval accuracy (% mAP) five fine-grained benchmark datasets.
Datasets
bits
DPSH
HashNet
ADSH
ExchNet A -ENET SEMICON
A
AGMH
Ours
12
8.68
12.03
20.03
25.14
33.83
37.76
37.83
56.42
72.19
24
12.51
17.77
50.33
58.98
61.01
65.41
71.73
77.44
81.38
CUB200
32
12.74
19.93
61.68
67.74
71.61
72.61
78.39
81.95
82.22
48
15.58
22.13
65.43
71.05
77.33
79.67
82.71
83.69
83.96
12
8.74
14.91
15.54
33.27
42.72
49.87
57.53
71.64
78.47
24
10.87
17.75
23.09
45.83
63.66
75.08
73.45
83.45
83.88
Aircraft
32
13.54
19.42
30.37
51.83
72.51
80.45
81.59
83.60
84.06
48
13.94
20.32
50.65
59.05
81.37
84.23
86.65
84.91
85.60
12
11.82
24.42
35.64
45.63
46.44
50.00
54.51
62.59
70.69
24
13.05
34.48
40.93
55.48
66.87
76.57
81.46
80.94
81.76
Food101
32
16.41
35.90
42.89
56.39
74.27
80.19
82.92
82.31
82.74
48
20.06
39.65
48.81
64.19
82.13
82.44
83.66
83.21
83.81
12
2.17
2.34
2.53
5.22
8.20
8.12
8.80
28.13
24
4.08
3.29
8.23
15.69
19.15
19.44
22.65
37.75
NABirds
32
3.61
4.52
14.71
21.94
24.41
28.26
29.79
41.33
48
3.20
4.97
25.34
34.81
35.64
41.15
42.94
43.71
12
6.33
3.70
8.24
23.55
25.52
30.32
30.54
43.99
69.76
24
9.05
6.24
24.90
35.93
44.73
58.45
60.56
68.05
83.30
VegFru
32
10.28
7.83
36.53
48.27
52.75
69.92
73.38
76.73
83.81
48
9.11
10.29
55.15
69.30
69.77
79.77
82.80
84.49
85.38
(a) CUB200 (b) Aircraft (c) Food101 (d) NABirds (c) VegFru Figure 6. Quality demonstrations of the learned attribute hash codes. Each column each figure strongly correspond to a certain kind of properties of the fine-grained objects, c.g., "red birds", "double-wing winl aircrafts". "fried eggs", "long beak" "layered petal-like structure", etc. (Best viewed in color and zoomed in.)
(a) CUB200 (b) Aircraft (c) Food101 (d) NABirds (e) VegFru
Figure Visualization of different queries. Each row corresponds to distinct Icarnable attribute query (g. E R.), (Best viewed in color zoomed in.)
low-bit hash codes scenarios, we significantly improve re- trieval performance. Thereby narrowing the performance gap across various lengths of hash codes. Notably, in the 12-bits setting, method shows substantial enhancements on the CUB200 and VegFru datasets, achieving 15.77% and 25.77% improvements over AGMH (Lu et al., 2023), respectively. This enhancement is especially valuable for practical applications, where fast retrieval and reduced stor- age requirements are essential. In contrast other previous methods, A--NET (Wei et al., 2021a) and A ---NET++ (Wei et al., 2023a) enhance learned bits by incorporating strong visual features through reconstruction operations. Our query optimization, as demonstrated in Section 4.2 and Figure 6, not only ensures the learned hash codes maintain the ad- vantage of attribute awareness but also achieves superior retrieval performance.
We provide a comparison with classification-base methods, such as FISH (Chen et al., 2022), CNET (Zeng & Zheng, 2023), DAHNET (Jiang et al., 2024) and CMBH (Chen al., 2024), in appendix.
4.1. Quantitative Results
Table 1 presents the mean average precision (mAP) results for fine-grained retrieval across five benchmark datasets. For each dataset, evaluate performance using four dif- ferent hash codes lengths: 12, 24, 32 and 48. As shown in Table 1, our method consistently outperforms the pre- vious methods across datasets. It's important to high- light the factor that contribute to our method's superior performance. By alleviating optimization challenges in

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Table Retrieval accuracy (% mAP) with incremental compo-
Table 3. Comparison results of hyper parameter N. Results are based on CUB200 and Aircrafi datasets under the 12- bitbits setting.
nents.
Configurations
CUB200
N
2
6
8
12
CUB200
33.33
52.44
69.46
71.57
72.19
72.36
S
A
12bits
24bits
32bits
48bits
Aircraft
39.43
65.70
74.51
75.20
78.47
78.38
X
X
X
20.03
50.33
61.68
65.43
x
x
27.43
59.19
67.69
77.07
Table 4. Parameters and computational costs (Flops) for training and inference. Methods with source codes publicly released are included. Results are hased on 12-bits setting.
x
33.33
64.94
74.79
80.52
72.19
81.38
82.22
83.96
Flops
Method
Params
training
inference
ExchNet
40.43M
7.67G
7.67G
SEMICON
42.42M
7.09G
7.09G
AGMH
35.56M
8.61G
8.61G
(a)
(b)
(c)
(d)
Ours
31.71M
5,27G
5,23G
Figure 8. Visualization results of learned queries with the 12-bits
setting. (a) is the similarity matrix of 12 queries. (b), (c) and (d) are the t-SNE (van der Maaten & Hinton, 2008) results of the input images from three different categories, respectively. Different colors represent different attribute-specifi specific features.
improving performance. Additionally, the incorporation of auxiliary branch mitigate the inherent limitations caused by the low dimensions, significantly improving performance.
4.2. Qualitative Results
Attribute Diversity of Query Learning In our method, the learnable queries are randomly initialized, which in- herently introduces diversity without requiring additional constraints. At the beginning of the training process, this di- versity naturally directs focus toward different attributes of the imput images, and we illustrate the well-learned queries in Figure 8a with the 12-bit setting. The results indicate that the 12 well-learned queries are distinct, showing that this diversity is preserved throughout training.
We hereby discuss the quality of the attribute-aware hash codes generated by our method. Similar to the method of -ETT (Wei et al., 2021a) and A (Wei al., 2023a), we visualize the retrieved images using single bit randomly selected from the hash codes highlight the strong link between visual attributes. All five datasets from our experiments serve as examples to demonstrate the qual- ity. As depicted in Figure 6, images in each column exhibit similar fine-graing grainedi object properties, i.e., visual attributes. The learned hash codes clearly exhibits a strong attribute- awareness, which indicates that this direct attributes query method is interpretable. As can be seen from Figure 7, the activated regions of different queries (highlighted in warm colors) are also semantically meaningful. These visualiza- tions qualitatively illustrate that our proposed query learning achieves strong interpretability.
Apart from the learnable queries themselves, a key ques- tion is whether our query mechanism can efficiently decou- ple visual attributes that exhibit diversity. We visualized these decoupled attribute-specifie features in Figures 8b, 8c and 8d. The images from three different categories are represented by 12 different attributes, respectively. This demonstrates that our method efficiently decouples diverse attributes, ensuring that the same image is represented by 12 bits containing diverse semantic information.
4.3. Further Analysis
Ablation Studies of Difffrrnt Modules We conduct ab- lation studies on CUB200 to verify the effectiveness of each module, including a Subtle Feature Extractor (S, cf. Section 3.1), the Query Learning mechanism (Q, cf. Sec- tion 3.2), and the Auxiliary Branch (A, cf. Section 3.3). These modules are incrementally applied to base net- work (i.e., ResNet-50). As illustrated in Table 2, by adding these modules one by one, the retrieval results are steadily improved, which justifies the effectiveness of our method. Based on the enriched features, the query learning mech- anism can generating attribute-awar hash codes, thereby
Number of Auxiliary Branches We increased the num- ber of dimensions during training to N x & by applying query transformation and adding N - 1 auxiliary branches. The retrieval results with different values of N under the 12-bits setting are presented in Table 3. As shown, when N increases from I to 6, retrieval performance improves rapidly, closely aligning with the sharp decline of shown in Figure 3. Once N exceeds 6, the changes in become more gradual, and also the performance stabilizes. Finally, we set N = 96 where 96 is the least common multiple of 12, 24, 32 and 48.
A
8

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Model Complexity and Computational Costs A poten- tial concern is whether our method sacrifices efficiency, es- pecially when optimizing in high-dimensional spaces. How- ever, as shown in Table 4, our method is more lightweight and computational efficient compared to some previous methods. Our strategy, which conducts query transforma- tion and incorporates an auxiliary branch for better optimiza- tion, only introduces negligible computational overhead.
transformers. In Proc. Eur. Conf. Comput. Vis., pp. 213- 229, 2020. 2,3
Chan, W., Saharia, C., Hinton, G., Norouzi, M., and Jaitly, N. Imputer: Sequence modelling via imputation and dynamic programming. In Proc. Int. Conf. Mach. Learn.,
pp. 1403-1413, 2020. 3
Chen, S., Sun, H., Wu, Y., Shang, L., and Ruan, X. helmet detection algorithm based transformers with deformable attention module. Chin. J. Electron., pp. 229 241, 2025. 2
5. Conclusion
In this paper, we proposed a novel method that can gener- ate attribute-aware hash codes for large-scale fine- eee-grained image retrieval task. Specifically, to obtain subtle features, we first extracted multi-scale information from a CNN back- bone and then refined these features. Then, by utilizing decoder based tross-attention mechanisms and group of learnable parameters which are treated as attribute queries, we directly decouple attribute-specific features and com- press them into hash codes for retrieval. The generated attribute-aware hash codes were able to provide interpretabil- ity. Additionally, we explored the limitations between large class numbers and low optimization dimensions during the optimization process. We proposed query transformation strategy and trained the model with an auxiliary branch, which significantly enhanced the model's performance with- out introducing any additional parameters. Furthermore, the decoder has the potential to be extended to interact with text embeddings, which encourages us to explore cross-modal hashing future research.
Chen, Z.-D., Luo, X., Wang, Y. Guo, S., and Xu, X.-S. Fine- Fine-grained hashing with double filtering. IEEE Trans. Image Process. 31:1671-1683, 2022. 2, 7, 12
Chen, Z.-D., Zhao, L.-J., Zhang, Z.-C., Luo, X., and Xu, X.- S. Characteristics matching based hash codes generation for efficient fine-grained image retrieval. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 17273-17281, 2024. 2, 7, 12
Cui, Q. Jiang, Q.-Y., Wei, X.-S,, Li, W.-.,, and Yoshie, O. ExchNet: A unified hashing network for large-scale fine-graines image retrieval. In Proc. Eur. Conf. Comput. Vis., pp. 189-205, 2020. 1,2,
Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y,, Yang, M.-H., and Torr, P. Res2Net: A new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell., 43 (2):652-662, 2021. 3
Impact Statement
Guo, M.-H., Xu, T.-.., Liu, J.-J., Liu, Z.-N., Jiang, P.-T.. Mu, T.-J., Zhang, S.-H., Martin, R. R.,, Cheng, M.-M., and Hu, S.-M. Attention mechanisms in computer vision: A survey. Comput. Visual Media, 8(3):331-36,, 2022. 4
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must specifically highlighted here.
Haoran, S., Yang, W., Haipeng, I.., and Biao, Q- Fine- grained cross-modal fusion based refinement for text-to- image synthesis. Chin. Electron.. pp. 1329-1340. 2023. 3
References
Bossard, L.., Guillaumin, M., and Van Gool, L.. Food- 101d mining discriminative components with random forests. In Proc. Eur. Conf. Comput. Vis., pp. 446-461, 2014. 6, 12
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proc. IEEE Conf. Comput. Vis. Patlern Recognit., pp. 770-778, 2016. 6
Cakir, F., He, K., and Sclaroff, S. Hashing with binary matrix pursuit. In Proc. Eur. Conf. Comput. Vis., pp. 344-361, 2018. ISBN 978-3-030-01227-4.1
Hoe, J. T., Ng, K. W., Zhang. T., Chan, C. S., Song, Y.-Z., and Xiang, T. One loss for all: Deep hashing with a single cosine similarity based learning objective. In Adv. Neural Inf. Process. Syst., volume 34, pp. 24286-2422 -42998, 2021. 5
Cao. Z., Long, M., Wang, J., and Yu. P. S. HashNet: Deep learning to hash by continuation. In Proc. IEEE Int. Conf. Comput. Vis., pp. 5608-5617, 2017. 6
Horn, G. V., Aodha, O. M., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. J. The inaturalist species classification and detection dataset. Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 8769-8778, 2017.
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Hou, S., Feng, Y., and Wang, Z. VegFiru A domain-specific dataset for fine-gr rree-graied visual categorization. In Proc. IEEE Int. Conf. Comput. Vis., pp. 541-549, 2017.
Lin, J., Mao, X., Chen, Y., Xu, L.,, He, Y., and Xue, H. D'ETR: Decoder-only detr with computationally efficient cross-scale attention. arXiv preprint arXiv:2203.00000 2022. 2
Hu, F., Zhang, C., Guo, J., Wei, X.-S., Zhao, L., Xu, A., and Gao, L. An asymmetric augmented self-supervised learn- ing method for unsupervised fine-gre aeeeggraied image hashing. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 17648-17657 2024. 1
Lin, Z.. Ding, G., Hu, M., and Wang. J. Semantics- preserving hashing for viess-view retrieval. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3864 3872, 2015.
Jaegle, A., Gimeno, F., Brock, A., Vinyals, O., Zisserman, A., and Carreira, J. Perceiver: General perception with iterative attention. In Proc. Int. Conf. Mach. Learn., pp. 4651-4664, 2021.
Liu, W., Wang, J., Kumar, S., and Chang, S.-F. Hashing with graphs. In Proc. Int. Conf. Mach. Learn., pp. 1-8, 2011. 2
Lu, X., Chen, S., Cao, Y., Zhou, X., and Lu, X. Attributes grouping and mining hashing for fine- line-grained image re- trieval. In Proc. ACM Int. Conf. Multimedia, pp. 6558- 6566, 2023. 1, 2,
Jiang, Q.-Y. and Li, W.-. Asymmetric deep supervised hashing. Proc. AAAI Conf. Artif. Intell., volume 32, pp. 3342 3349, 2018. 4, 6
Jiang X., Tang, H., and Li, Z. Global meets local: Dual activation hashing network for large-scale fine-grained image retrieval. IEEE Trans. Knowl. Data Eng., 36(11): 6266-6277 2024. 2, 7, 12
Maji, S., Rahtu, E.,, Kannala, J., Blaschko, M., and Vedaldi, A. Fine-grained visual classification of aircraft. arXiv preprint wrXiv:1306.515/ 2013. 6
Pineda, L., Salvador, A., Drozdzal, M., and Romero, A. Elu- cidating image-to-set prediction: An analysis of models, losses and datasets. arXiv preprint arXiv: urXiv:1904.0 1ee4.090499999 2019. 2
Jin, S., Yao, H., Sun, X., Zhou, S., Zhang, L.,, and Hua, X. Deep saliency hashing for fine- fine-grained retrieval. IEEE Trans. Image Process., 29:5336-53511 2020. 1,2
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollár, P., and Girshick, R. Segment anything. In Proc. IEEE Int. Conf. Comput. Vis., pp. 3992-4002 2023.
Rezatofighi, S. H., Bg, V. K., Milan, A., Abbasnejad, E., Dick, A., and Reid, I. Deepsetnet: Predicting sets with deep neural networks. In Proc. IEEE Int. Conf. Comput. Vis., pp. 5257-5266, 2017. 2
Shehzadi, T., Hashmi, K. A., Stricker, D., and Afzal, M. Z. 2D object detection with transformers: a review. arXiv preprint arXiv:2306.00 230.::23006.466700 2023.
Leng, C., Cheng, J., Wu, J., Zhang, X., and Lu, H. Super- vised hashing with soft constraints. In Proc. ACM Int. Conf. Inf. Knowl. Manage, pp. 1851-1854, 2014.
Shen, Y., Sun, X., Wei, X.-S., Jiang, Q.-Y,, and Yang, J. SEMICON: learning-to- -ooiingtt-hhshh solution for large-scale fine- fine-grained image retrieval. Proc. Eur. Conf. Comput. Vis., pp. 531 1411-48, 2022. 1, 2,2,
Li, C., Wang, B., Pavlu, V., and Aslam, J. Conditional bernoulli mixtures for multi-label classification. In Proc. Int. Conf. Mach. Learn., pp. 2482-2491, 2016a. 2
Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T. Visualizing the loss landscape of neural nets. In Adv. Neural Inf. Process. Syst., pp. 6391-64011 2018. 5
van der Maaten, L. and Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res., (86)) 2605, 2008.
Li, W.J., Wang, S., and Kang, W.-C. Feature learning based deep supervised hashing with pairwise labels. In Proc. Int. Joint Conf. Artif. Intell., pp. 1711-1717, 2016b.
Van Horn, G., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., and Belongie, S. Building bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 595-604 2015. 6
Li, X., Lin, G., Shen, C.. Hengel, A., and Dick, A. Learning hash functions using column generation. In Proc. Int. consistent fair face recognition. IEEE Trans. Pattern Anal. Mach. Intell., pp. 1-17, 2025.
Conf. Mach. Learn., pp. 142-150, 2013. 2
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. At- tention is all you need. Adv. Neural Inf. Process. Syst., volume 30, pp. 6000-6010, 2017.
Li, Y., Sun, Y., Cui, Z., Shen, P., and Shan, S. Instance-
10

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The Caltech-UCSDD birds-200-2011 dataset. Tech. Report CNS-TR-2011-001, 2011. 1, 6, 12
Zeng, X. and Zheng, Y. Cascading hierarchical networks with multi-task balanced loss for fine-grained hashing. arXiv preprint arXiv:2303. 2023. 2, 7, 12
Wang, J., Zhang, T., song, j.. Sebe, N., and Shen, H. A survey on learning hash. IEEE Trans. Pattern Anal. Mach. Intell., 40(4):769-790) 2018. 1,2
Zhao, B., Feng, J., Wu, X., and Yan, S. survey on deep Barming-baseed fine-grained object classification and se- mantic segmentation. Inl. Autom. Comput., 14(2):119 11 135, 2017.
Wei, X.-S., Luo, J..H.. Wu, J., and Zhou, Z.-H. Selective convolutional descriptor aggregation for fine-grained im- age retrieval. IEEE Trans. Image Process. 26:2868-2881, 2017.
Zhu, X., Su, W., Lu, I.,, Li, B., Wang, X., and Dai, De- formable DETR: Deformable transformers for end-to-end object detection. In Proc. Int. Conf. Learn. Represent.. pp. 1-11, 2021. 2
Wei, X.-S., Cui, Q. Yang, L.,, Wang, P., and Liu, L. RPC: A large-scale retail product checkout dataset. arXiv preprint arXiv: 0etp:/901109599 2019. 1
Wei, X.-S., Shen, Y., Sun, X., Ye, H.-J,, and Yang. J. A NET: Learning attribute-aweer aware hash codes for large-scall scale fine-grained image retrieval. In Adv. Neural Inf. Process. Syst. volume 34, pp. 5720-5730, 2021a. 2, 6, 7,
Wei, X.-S., Song, Y.-Z., Mac Aodha, O., Wu, J., Peng, Y., Tang, J., Yang, and Belongie, Fine-grained image analysis with deep learning: A survey. IEEE Trans. Puttern Anal. Mach. Intell., 44(12):8927-8944 2021b.
Wei, X.-S., Shen, Y., Sun, X., Wang, P., and Peng, Y. Attribute-wware deep hashing with self-consistency for large-scale fine-grained image retrieval. IEEE Trans. Pat- tern Anal. Mach. Intell., (1111))13994-19220) 2023a. 1, 2, 6, 7
Wei, X.-S., Xu, Y.-Y., Zhang, C.-L., Xia, G.-S., and Peng, Y.-X. CAT: coarse-to-fie attention tree for semantic change detection. Vis. Intell., pp. 1-12, 2023b.
Welch, L. Lower bounds on the maximum cross correlation of signals (corresp.). IEEE Trans. Inf. Theory, 20(3): 397-399, 1974. 5, 14
Yang, L.,, Zhang, R.-Y., Li, L.., and Xie, X. Simam: A simple, parameter-free attention module for convolutional neural networks. In Proc. Int. Conf. Mach. Learn., pp. 11863- 11874, 2021. 4
Yang, Y., Geng. L., Lai, H., Pan, Y., and Yin, J. Feature pyra- mid hashing. In Proc. Int. Conf. Multimedia Retrieval, pp. 114-122, 2019. 2
Ye, Z. He, X., and Peng. Unsupervised cross-media hashing learning via knowledge graph. Chin. J. Electron., pp. 1081-1091, 2022. 2
Yu, H., Lu, H., Zhao, M., Li, Z., and Gu, G. Gradient aggregation based fine-grained image retrieval: A unified viewpoint for CNN and Transformer. Pattern Recognit., 149: 10248 110260, 2024. 4

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
A. Appendix Overview
Table 5. Comparison of Retrieval Accuracy (% mAP) with Classification Based Methods.
In this appendix, we present additional information about our proposed method, including: (I) an explanation of en- hanced interaction in the auxiliary branch; (2) further exper- imental results that compare
Dataset
CUB200-2011
Methods
12 bits
24 bits
32 bits
48 bits
method with classification-
FISH
76.77
79.93
80.09
80.88
based methods, examples of retrieved results, and additional visualization results; and (3) a proof for the lower bound of μ.
CNET
77.10
82.11
83.09
83.92
DAHNET
61.69
79.00
81.69
83.98
CMBH
80.30
82.16
82.68
82.80
Ours! 80.79
82.02
83.34
84.27
B. Explanation of Enhanced Interaction in Auxiliary Branch
denotes CMBH without multi-region feature embedding to ensure similar training costs, and denotes our method is trained with classification.
One of the advantages of query transformation and the incor- poration of an auxiliary branch is that the learnable queries interact more thoroughly with the enriched image features. As illustrated in Figure 9, both q and q share the same pa- rameters, however, their interactions with the features differ. By including the auxiliary branch, we effectively expand the receptive field of the learnable queries, enhancing the flex- ibility and comprehensivenes. of their interactions, which enables the queries to capture complex patterns within the input images more effectively.
interaction
qi
q
X Xout
Xout
Figure 9. The interaction of learnable query with enriched image features.
C. Additional Experimental Result
Methods Apart from the methods mentioned in the main body, FISH (Chen et al., 2022), CNET (Zeng& & Zheng, 2023), DAHNET (Jiang et al., 2024) and CMBH (Chen et al., 2024) are other fine- grained hashing methods that have achieved good retrieval accuracy. For fair comparisons, we control empirical set- tings to be the same as these methods and incorporate addi- tional classification tasks into our method during the train- ing phase. The comparison results are presented in Table 5, demonstrating that our method is comparable to these meth-
Comparison with Classification-Batl
aligning well with the trends shown in Figure 3.
Larger class numbers coupled with lower feature di- mensions indeed lead to inaccessible lower local min- ima and a less smooth loss landscape.
This phenomenon aligns with our conjecture on the limita- tion of large class numbers with low feature dimensions.
ods.
Visualization of Extracted Features In Section 3.3, we conjecture that a large number of classes with low feature dimensions can make difficult for the model to learn distin- guishable features. To mitigate this limitation, we propose strategy of adding an auxiliary branch. To validate our conjecture, we employ t-SNE to visualize the features Dout- The visualization results in figure 13 demonstrate that in- corporating the auxiliary branch during training produces feature distributions exhibiting two key characteristics: (1) larger inter-class distances, manifested through more dis- tinct cluster separations between categories; and (2) smaller intra-class variations, evidenced by tighter aggregation of samples within each category. In contrast, the model with- out the auxiliary branch produces features with substantially overlapping distributions across different categories. This quantitative evidence strongly supports our claim that the auxiliary branch enhances feature discriminability.
Examples of Retrieved Results We present retrieval results on CUB200-2011 (Wah et al., 2011) and Food1O/ (Bossard et al., 2014). As shown in Figure 10 and Figure 11, our proposed method performs well in re- trieving among multiple subordinate categories. However, there are several failure cases where minimal differences, such as those caused by different views or the presence of distracting objects, require careful observation between the query image and the returned images.
Visualization of Loss Landscape We provide an ex- tended analysis with additional visualization cases across multiple C' values (C = 200, 555) in Figure 12. The com- prehensive results consistently reveal that:
The loss landscape exhibits a strong correlation with,
12

---

Learning Attribute-Awar Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Query image
Top-10 retrieved images
Figure 10. Examples of top-10 10 retrieved images on CUB200-201J of 48-bit hash codes by method.
Query image
Top- 10p-10mm retrieved images
Figure 11. Examples of top 10 retrieved images on Food101 of 48 bit hash codes by our method.
(a) C = 200, D = 12
(b) C = 200, D = 24
(c) C 200, D = 32
(d) C = 200, D = 48
(e) 555, D
(1) 555, D - 24
(g) 555, D 32
(h) C 555, D 48
Figure 12. Loss landscape across different parameter configurations: (a-d) show results with C 200. and (e-h) demonstrate with 555. Each column corresponds to specific dimension settings, where C' represents the number classes and 1) denotes the number of dimensions.
13

---

Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
(a) CUB200 with A
(b) Aireruft
(c) Food101 with
(d) NABirds with A
(e) VegFru with
(f) CUB200 without
(g) Aireraft without A
(h) Food101 without A
(i) NABirds without
(i) VegFru without
Figure 13. I-SNE Visualization of X for 10 randomly selected classes
D. A Proof for the Lower Bound
preceding inequality gives:
In our conjecture on the limitation of large class numbers with low feature dimensions, introduce lower bound for analyzing the observations of poor performance in low- bit scenarios. This lower bound is also called the Welch bound. Referring to (Welch, 1974), we provide proof.
(Tr G)
ΣΣ (a,, a,)))
(,,,,
(15)
n
As p are unit vector, so the diagonal of G are all 1, hence Tr G C and
We use the relaxed hash codes v; = (H Q θ)) to construct a matrix = [0i, U2 ..., Vel E BBxC' with C > we first normalized the columns and let x; T
C C
(ai
2,)))
-C+
((;1, -C+
(16)
v
get the correspond gram matirx:
Therefore,
(21, σ1)
11 Tc)
G
(12)
Σ (0,))
C(C - n)
(17)
(wo, x1)
(oc, ac)
where ( is the usual inner product on RT.
The mean of set of non-negative numbers is smaller than their maximum:
The trace of G is equal to the sum of its cigenvalues. Be- cause the rank of G is at most n, and positive semidef- inite matrix, G has at most n positive eigenvalues with its remaining eigenvalues all equal zero. Writing the non-zero cigenvalues of G as Ai...., Ay. with S and applying the Sauchy-Schwar inequality we can get:
1 C(C 1) in1
(( 2,),²
(18)
So.
C - n
2 V
(19)
n(C
(Tr G)
d
x²
(13)
1=1
The square of the Frobenius norm of G satisfies:
G ΣΣ (o,, 22 Σ X2
(14)
1=
1=1
Taking Equation (13) and Equation (14) together with the
14


Reviews:
[{"review_id": "xZ9s2v4vBB", "reviewer": "Reviewer_ATUG", "summary": "This paper presents a query optimization-based attribute-aware hash code generation method. First, a hybrid convolution and attention structure is utilized to obtain rich representations. Second, unlike other works that simply use fully connected layers to generate hash codes, this paper leverages a decoder and a set of learnable queries to automatically decouple different attribute features. Third, the paper incorporates an auxiliary branch to help alleviate the challenges of complex landscape optimization. Quantitative experimental results demonstrate the high retrieval performance of the method, while qualitative results show that the learned bits exhibit visual attribute-level interpretability. Additionally, the method is relatively lightweight, resulting in low computational overhead in practical applications.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment1:** *One of the main contributions of the paper is the introduction of an additional auxiliary branch. However, the description of this operation is too brief and does not sufficiently clarify its specific implementation. It is recommended that the authors provide a more detailed explanation of the auxiliary branch, including the interaction between queries and features, so that readers can better understand the significance and practical implications of this contribution.*\n\n**Reply1:**\nWe sincerely appreciate your valuable feedback. Without the auxiliary branches, the learnable queries interact with the extracted features in a fixed manner. For instance, each query can be split into $N$ segments. When $N=2$, the first segment consistently interacts with features corresponding to the first half of the image channels, while the second segment interacts with features from the latter half. However, when an auxiliary branch is introduced, the interaction pattern undergoes a significant change. In the case where $N=2$, the first segment of the query can interact with features from either the first half or the latter half of the image channels, and the same applies to the second segment. The introduction of auxiliary branches expands the receptive range of queries across the channel dimension. Since different channels typically encode distinct semantic information or visual features, this enhancement allows the optimized queries to better capture visual attributes across different images.\n\n---\n\n**Comment2:** *Figure 7 could provide a more detailed description.*\n\n**Reply2:**\nWe sincerely appreciate your valuable feedback. In Figure 7, we present the visualization results of heatmaps across different datasets. Each row corresponds to a distinct learnable attribute query ($q_i \\in \\mathbb{R}^d$). The heatmaps generated by different $q_i$ demonstrate that the learned queries effectively focus on different parts of the objects. For instance, on $CUB200$ dataset, certain queries attend to the body, while others focus on the beak of a bird. On $Aircraft$ dataset, some queries focus on the wings, while others focus on the tail of the airplane. These visualizations qualitatively illustrate that our proposed query learning achieves strong interpretability.\n\n\n---\n\n**Questions1:**\n*What is the relationship between the two decoders in Figure 4?*\n\n**Reply1:**\nFigure 4 shows the visualized results of the loss landscape under different conditions. \nIn Figure 5, the two decoders share the same weights.", "claims_and_evidence": "The claims made in the submission are clearly supported by convincing evidence. The author provides a well-rounded and methodologically sound approach. The motivation and the method of this paper are clearly described, and the extensive experimental results offer rigorous validation, demonstrating the superior efficacy and robustness of the proposed method across diverse datasets and scenarios.", "methods_and_evaluation_criteria": "The large-scale fine-grained retrieval problem studied in this paper is a fundamental and practical task, and the proposed method can be treated as an effective solution.", "theoretical_claims": "I have checked the proofs concerning the theoretical claims. The analysis and proofs are both reasonable.", "experimental_designs_or_analyses": "The authors conducted extensive experiments to compare the performance of the proposed method with other methods. Additionally, the authors performed experimental analysis on the method itself, which helps readers better understand the proposed method.", "relation_to_broader_scientific_literature": "This paper proposes an efficient attribute-aware hash code generation method. Unlike the stepwise attention proposed by AGMH for better feature extraction, this paper uses a decoder to decouple the extracted features, allowing the generated hash bits to keep key information from the images. This idea contributes a solution to the field of fine-grained image retrieval. Additionally, the introduction of an auxiliary branch is quite interesting. It incurs only a small amount of extra computational overhead, helping the model to alleviate the challenges of optimization in low-bit scenarios while enhancing performance.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strengths:\n\n1. This paper is well written and easy to follow.\n\n2. The motivation is clear, and the method is reasonable. The generated hash codes are associated with distinguishable attribute-level information, which brings interpretability to the retrieval process. Qualitative experimental results show that the proposed method offers interpretability.\n\n3. The experimental design is well-structured, and Section 4.3 provides a thorough analysis of the experiments, offering a comprehensive evaluation of the proposed method.\n\n4. The proposed method demonstrates strong performance while requiring less computational overhead.\n\nWeaknesses:\n\n1. One of the main contributions of the paper is the introduction of an additional auxiliary branch. However, the description of this operation is too brief and does not sufficiently clarify its specific implementation. It is recommended that the authors provide a more detailed explanation of the auxiliary branch, including the interaction between queries and features, so that readers can better understand the significance and practical implications of this contribution.", "other_comments_or_suggestions": "1. Figure 7 could provide a more detailed description.", "questions_for_authors": "What is the relationship between the two decoders in Figure 4?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zQcazo9oxZ", "reviewer": "Reviewer_QKWg", "summary": "This work treats the hash learning process as a set prediction problem, using a cross-attention-based decoder to decouple attribute-specific features and further compress them into hash codes. From the perspective of cosine similarity, it argues that large class numbers with low feature dimensions lead to poor retrieval performance in low-bit hash code scenarios, and a query transformation operation is employed as a solution, resulting in a well-optimized set of queries for hash codes generation. The authors provide numerous experimental results that validate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment1:** *The subtle feature extractor employs a multi-scale framework in conjunction with a multi-head self-attention (MHSA) mechanism. This method is widely adopted in deep learning, particularly for image feature extraction. For instance, architectures like Mobile-Former and ConVit exemplify this trend. While the design is reasonable, it does not offer significant innovation.*\n\n**Reply1:**\nWe would like to emphasize that:\n\n1. We model the hash problem as a set prediction problem, where each element in the set represents a bit of the hash code that can indicate a visual attribute. Specifically, our method uses $ k $ learnable queries to directly decouple distinguishable visual attributes from complex feature representations to generate the $ k $-bit hash code.\n\n2. Previous fine-grained retrieval methods have overlooked the low-bit problem. We provide an analysis from the perspective of cosine similarity and design a query transformation strategy that effectively alleviates this issue. \n   \n3. Regarding the design of the subtle feature extractor, our main objective is to use a lightweight and simple strategy to extract fine-grained features. Overly complex feature extraction modules also lead to larger model parameters and greater computational overhead. Some previous methods have primarily focused on the design of this module, while neglecting improvements in the hash code generation process.\n\nOverall, the first two key contributions have never been explored in previous work. At the same time, using our subtle feature extractor makes the method more lightweight and reduces computational overhead..\n\n---\n\n**Comment2:** *Line 58 should use 'demonstrates,' and line 75 should use 'are'.*\n\n**Reply2:**\nWe sincerely appreciate your thorough review. We have carefully addressed the identified issues and conducted further proofreading of the manuscript to ensure its quality.\n\n---\n\n**Questions1:**\n*How many layers of cross attention does the decoder part consist of?*\n\n**Reply1:**\nThe decoder consists of a single layer. Its lightweight design ensures that the introduction of auxiliary branches does not incur significant computational overhead.", "claims_and_evidence": "The paper claims that the proposed method can capture attribute-level information from images. The authors provide single bit retrieval results, demonstrating that the generated hash code can indicate nuanced visual attributes that can distinguish different fine-grained categories. The visualization results of the heatmaps also show that different queries can focus on different parts of the image.", "methods_and_evaluation_criteria": "The proposed method is well-suited for the problem of fine-grained image retrieval. It not only achieves superior retrieval performance but also brings interpretability to the generated hash codes.", "theoretical_claims": "The proof of the lower bound for μ in the paper seems to be free of issues.", "experimental_designs_or_analyses": "During the query learning process, no attribute-level annotations were provided. Interestingly, the authors present corresponding quantitative results and visualizations that showcase the attribute-aware characteristics of the generated hash codes and further analyze the diversity of query learning.", "relation_to_broader_scientific_literature": "The innovation of this paper is reflected in its practicality, as it combines semantic decoupling of fine-grained hashing with a lightweight model. This approach effectively addresses the performance limitations of existing methods (such as ExchNet and SEMICON) in low-bit scenarios. By introducing a learnable query mechanism, the proposed method differentiates from the attribute-aware learning framework of A²-NET, achieving automated attribute decoupling of complex global image features.", "essential_references_not_discussed": "The related references are discussed.", "other_strengths_and_weaknesses": "Strengths:\n\n1.The paper utilizes a query learning mechanism with learnable queries to generate attribute-aware hash codes, achieving not only performance improvements but also enhanced interpretability.\n\n2.The explanation from the perspective of cosine similarity is clearly exemplified by Figures 3 and 4. This fresh perspective substantiates its novelty.\n\n3.The experiments in this paper are sufficient, demonstrating significant performance improvements, especially in low-bit scenarios.\n\nWeaknesses:\n\n1.The subtle feature extractor employs a multi-scale framework in conjunction with a multi-head self-attention (MHSA) mechanism. This method is widely adopted in deep learning, particularly for image feature extraction. For instance, architectures like Mobile-Former and ConVit exemplify this trend. While the design is reasonable, it does not offer significant innovation.", "other_comments_or_suggestions": "1.Line 58 should use 'demonstrates,' and line 75 should use 'are'.", "questions_for_authors": "1.How many layers of cross attention does the decoder part consist of?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2rNMWTp4YE", "reviewer": "Reviewer_gSZJ", "summary": "In this paper, the authors propose a query optimization-based fine-grained image hashing method, which enables the generated hash bits to exhibit attribute-aware characteristics. From the perspective of cosine similarity, the challenges in generating effective low-bit hash codes are analyzed. Based on this analysis, the performance of the model is particularly enhanced for the low-bit case through the incorporation of auxiliary branches. Expensive experiments were conducted, showing that the proposed method achieves superior retrieval performance, and a single bit demonstrates interpretability.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment1:** *The paper lacks a comparison with the recent method ConceptHash*\n\n**Reply1:**\n| Method       | bits | CUB-200 | Aircraft | Stanford Cars |\n|--------------|------|---------|----------|---------------|\n| ConceptHash  | 16   | 83.45   | 82.76    | 91.70          |\n|              | 32   | 85.27   | 83.54    |**92.60**|\n|              | 64   | 85.50   | 84.05    | 93.01         |\n| Ours$^+$     | 16   | **84.12**   | **84.16**|**92.12** |\n|              | 32   | **86.02**  | **84.26**| 92.40  |\n|              | 64   |**86.76** | **85.76**|**93.46** |\n\n**Table 2:Comparison of Retrieval Accuracy (%mAP) with ConceptHash.$^+$ denotes our method is trained with classification and uses the ViT-Base backbone.**\n\nDue to the different experimental settings between ConceptHash and our experiment (both in the main paper and appendix). We made adjustments to our own method and re-conducted the experiment. The experimental results are shown in Table 1, demonstrating that our method is comparable to these methods.\n\n---\n\n**Comment2:** *The paper discusses the limitation of large class numbers with low feature dimensions. However, the experiments related to the hyperparameter N, which are directly relevant to this analysis, were only conducted on one dataset, which is very insufficient.*\n\n**Reply2:**\n|     $N$  | 1      | 2      | 4      | 6      | 8      | 12     |\n|----------|--------|--------|--------|--------|--------|--------|\n| CUB200   | 33.33  | 52.44  | 69.46  | 71.57  | 72.19  | 72.36  |\n| Aircraft | 39.43  | 65.70   | 74.51  | 75.20  | 78.47  | 78.38  |\n| Food101  | 43.79  | 66.16  | 71.91  | 69.67  | 70.69  | 71.08  |\n| NABirds  | 7.48   | 11.01  | 18.88  | 24.17  | 28.13  | 32.08  |\n| VegFru   | 18.16  | 27.76  | 39.08  | 49.66  | 69.76  | 71.68  |\n\n**Table 2: Comparison results of hyperparameter N. Results are based on five commonly used benchmark datasets under the 12-bits setting.**\n\nThank you for your suggestion. We have provided more experimental results in Table 2. The results on different datasets show that as $N$ increases from 1 to 6, the retrieval performance improves rapidly. Once $N$ exceeds 6, the performance stabilizes, which follows the same trend as the change of mu shown in Figure 3.\n\n---\n\n**Comment3:** *The authors provide a detailed description of the motivation behind the auxiliary branch design, but only a brief description is given regarding the specific operations.*\n\n**Reply3:**\nFor the specific implementation of the auxiliary branch, we provide a detailed description. Given a query $q_i \\in \\mathbb{R}^d$, we divide $q_i$ evenly into multiple parts. We then perform a circular shift with step sizes of $1, 2, 3, ..., N-1$. For each of the $N-1$ different step sizes, we extend the original $q_i$ into $N-1$ new queries, $\\hat{q}_i^j$, where $j = 1, 2, 3, ..., N-1$ indexes the different step sizes and all queries share the same parameters. We refer to this extension process as query transformation. We initialize a total of $k$ learnable queries. After the query transformation operation, each query is passed to the decoder for computation according to Equation 3. Finally, we obtain $\\hat{h} \\in \\mathbb{R}^{N \\times k}$, which is optimized using the loss function defined in Equation 6.\n\n**Comment4:** *The Related Work section on 'Set Prediction and Parallel Decoding' could be divided into two parts.*\n\n**Reply4:**\nWe sincerely appreciate your valuable suggestion regarding the organization of the related work section. We have restructured this part to improve clarity.\n\n---\n\n**Questions1:**\n*What is the difference between Query Optimization mentioned in the paper and c-vectors optimization in CMBH？*\n\n**Reply1:**\nOur approach differs from CMBH in the following ways:\n1. CMBH’s contribution focused on better extracting subtle image features but does not improve the process of generating hash codes. This distinction results in the hash codes generated by CMBH not exhibiting interpretability, while our approach generates hash codes that can indicate whether an image possesses a certain visual attribute.\n\n2. We primarily follow the pairwise setting, which means that we can not use the classification task for training. In contrast, CMBH’s optimization process is designed based on a classification task, which cannot be used in the pairwise setting.", "claims_and_evidence": "From the perspective of cosine similarity, this paper provides an analysis of why retrieval performance is poor in low-bit hashing scenarios within a pairwise setting. The authors also visualize the loss landscape, providing empirical evidence for their theory. Furthermore, the authors claim that the proposed method can generate attribute-aware hash codes. They showcase the results through qualitative comparisons of one bit retrieval results and heat maps. Additionally, the authors explain the attribute diversity of query learning mechanism.", "methods_and_evaluation_criteria": "The method proposed in the paper is meaningful for the application of fine-grained image retrieval. By focusing on attribute-level information and optimizing low-bit hash codes, it offers a robust and interpretable framework for this task.", "theoretical_claims": "The theoretical proof in the paper seems to be reasonable.", "experimental_designs_or_analyses": "The authors compare the results of different methods on five fine-grained datasets and present a lot of visualizations. However, a major contribution of the paper is the introduction of auxiliary branches. The key experiments regarding the hyperparameter N are only conducted on one dataset, which makes this part of the experimentation insufficient.", "relation_to_broader_scientific_literature": "The paper presents an advanced method that generates attribute-aware hash codes optimized through queries for large-scale fine-grained image retrieval. Compared to previous works such as AGMH and SEMICON, this method achieves superior retrieval performance", "essential_references_not_discussed": "No essential references not discussed", "other_strengths_and_weaknesses": "Strengths:\n1.The proposed method achieves superior retrieval performance and can generate attribute-aware hash codes.\n2.The introduction of the auxiliary branch and the corresponding analysis is interesting.\n3.The authors provide many visualization results.\n\nWeaknesses:\n1.The paper lacks a comparison with the recent method ConceptHash[1]\n2.The paper discusses the limitation of large class numbers with low feature dimensions. However, the experiments related to the hyperparameter N, which are directly relevant to this analysis, were only conducted on one dataset, which is very insufficient\n3.The authors provide a detailed description of the motivation behind the auxiliary branch design, but only a brief description is given regarding the specific operations.\n\nRef:\n[1]Ng, K. W., Zhu, X., Song, Y.-Z., and Xiang, T. Concepthash: Interpretable fine-grained hashing via concept discovery. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshop, pp. 1211–1223, 2024.", "other_comments_or_suggestions": "1. The Related Work section on 'Set Prediction and Parallel Decoding' could be divided into two parts.", "questions_for_authors": "1. For key questions, please refer to the Weaknesses section.\n2. What is the difference between Query Optimization mentioned in the paper and c-vectors optimization in CMBH[1].\n\nRef:\n[1]Chen, Z.-D., Zhao, L.-J., Zhang, Z.-C., Luo, X., and Xu, X.-S. Characteristics matching based hash codes generation for efficient fine-grained image retrieval. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 17273–17281, 2024.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DIUPBSobUD", "reviewer": "Reviewer_Zq7Z", "summary": "This paper presents a novel learn-to-hash method for large-scale fine-grained image retrieval. It introduces a query learning mechanism that can capture nuanced attribute-level information, making each bit of the hash code interpretable. The paper also introduces auxiliary branches during the training process to improve model performance. Experiments on five datasets demonstrate the effectiveness of the method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comment1:** *Figure 3 illustrates the curves at different values of c, and the authors further support their analysis with the visual results in Figure 4. However, Figure 4 only provides visualization for the specific case of C=200. Does the trend of the loss landscape at different values of c align with that in Figure 3? More visual results should be included.*\n\n**Reply1:**\nWe sincerely appreciate your constructive feedback. In response to your insightful suggestion, we have supplemented the analysis with additional visualization cases across multiple c-values (C = 200, 292, 555). These extended results consistently demonstrate that:  \n\n1. The loss landscape exhibits a strong correlation with $\\mu$, aligning well with the trends shown in Figure 3.  \n2. Larger class numbers coupled with lower feature dimensions indeed lead to:  \n   - Inaccessible lower local minima  \n   - Less smooth loss landscape  \n\n\nThis phenomenon aligns with our ​conjecture on the limitation of large class numbers with low feature dimensions. The visualization results can be found at: https://anonymous.4open.science/r/rebuttal_for_ICML2025-89B6/figure1.png\n\n---\n\n**Comment2:** *Section 3.3 mentions that without the auxiliary branch, the model may not learn discriminative features. Are there any experimental results to support this observation? The authors could consider plotting the t-SNE result of X_out to observe the feature distribution across different categories.*\n\n**Reply2:**\nWe sincerely appreciate your insightful suggestion. To validate our observation, we have conducted t-SNE visualization on the feature embeddings $X_{out}$. The results demonstrate that when training ​with the auxiliary branch, the feature distributions exhibit:  \n1. ​Larger inter-class distances: Distinct categories form more separated clusters.  \n2. ​Smaller intra-class variations: Samples within the same category show tighter aggregation.  \n\nIn contrast, the model ​without the auxiliary branch produces features with substantially overlapping distributions across different categories. This quantitative evidence strongly supports our claim that the auxiliary branch enhances feature discriminability. visualization results can be found at: https://anonymous.4open.science/r/rebuttal_for_ICML2025-89B6/figure2.png\n\n\n---\n\n**Comment3:** *There is a spelling error in line 302.*\n\n**Reply3:**\nWe sincerely appreciate your thorough review. We have carefully addressed the identified issues and conducted further proofreading of the manuscript to ensure its quality.", "claims_and_evidence": "This paper analyzes the pairwise loss used in its method from the perspective of cosine similarity, arguing that when the number of categories exceeds the feature dimensions, the model may not learn distinguishable features. In addition to the theoretical analysis, the authors also provide visual results of the loss landscape, clearly illustrating that it is more difficult to achieve lower loss values when the number of dimensions is small.", "methods_and_evaluation_criteria": "I think the method proposed in the paper is meaningful for the application of fine-grained image retrieval.", "theoretical_claims": "This paper leverages auxiliary branch to increase the number of dimension, alleviating the low-bit optimization challenge. Experiments regarding the hyperparameter N show that the experimental results are consistent with the theoretical analysis.", "experimental_designs_or_analyses": "The experiments conducted on five fine-grained datasets, covering various scenarios such as birds, airplanes, and food, demonstrate that our method significantly outperforms previous approaches in low-bit scenarios. This underscores its broad applicability. Additionally, ablation study further validate the effectiveness of each module.", "relation_to_broader_scientific_literature": "This paper innovatively models the hash problem as a set prediction task inspired by DETR. By leveraging learnable queries, the method not only enhances interpretability but also demonstrates improved retrieval capabilities. This contribution explores the intersection of hashing techniques and cross attention mechanisms, offering a fresh perspective that could inspire further research in both hashing and attribute-based image retrieval.", "essential_references_not_discussed": "No Essential References Not Discussed", "other_strengths_and_weaknesses": "Strengths:\n1，This paper is well organized, and the figures and tables are presented clearly.\n2，This paper addresses the important and practical task of large-scale fine-grained retrieval, presenting an effective method that enhances retrieval accuracy while also offering intriguing interpretability. The entire method is well-motivated.\n3，This paper analyzes why retrieval performance is poor in low-bit hash code scenarios from the perspective of cosine similarity and proposes an efficient approach that introduces an additional auxiliary branch during the training process as a solution. \n4,The authors provide many visualizations to help readers understand the method.\n\nWeaknesses:\n1. Figure 3 illustrates the curves at different values of c, and the authors further support their analysis with the visual results in Figure 4. However, Figure 4 only provides visualization for the specific case of C=200. Does the trend of the loss landscape at different values of c align with that in Figure 3? More visual results should be included.\n2.Section 3.3 mentions that without the auxiliary branch, the model may not learn discriminative features. Are there any experimental results to support this observation? The authors could consider plotting the t-SNE result of X_out to observe the feature distribution across different categories.", "other_comments_or_suggestions": "There is a spelling error in line 302.", "questions_for_authors": "Please refer to Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a query optimization-based method for generating attribute-aware hash codes in fine-grained image retrieval. It combines a hybrid CNN-attention encoder with a learnable query decoder and an auxiliary branch to enhance optimization, especially under low-bit settings. The method is well-motivated and tailored for fine-grained tasks, demonstrating good accuracy and good interpretability across multiple benchmarks.

The four reviewers (2 Accept, 2 Weak Accept) agree the work is solid, with clear contributions, effective experiments, and meaningful insights. Minor improvements could include clarifying the auxiliary branch and adding more recent baselines. Overall, a promising and well-executed paper.
Decision: Accept (poster)