Dynamic Masking and Auxiliary Hash Learning for Enhanced Cross-Modal Retrieval
Authors: Shuang Zhang
Yue Wu
Lei Shi
Yingxue Zhang
Feifei Kou
Huilong Jin
Pengfei Zhang
Meiyu Liang
Mingying Xu
Venue: NeurIPS 2025 poster ()
Keywords: Cross-modal hash retrieval
Auxiliary hash
Dynamic masking
Primary area: deep_learning

Abstract: The demand for multimodal data processing drives the development of information technology. Cross-modal hash retrieval has attracted much attention because it can overcome modal differences and achieve efficient retrieval, and has shown great application potential in many practical scenarios. Existing cross-modal hashing methods have difficulties in fully capturing the semantic information of different modal data, which leads to a significant semantic gap between modalities. Moreover, these methods often ignore the importance differences of channels, and due to the limitation of a single goal, the matching effect between hash codes is also affected to a certain extent, thus facing many challenges. To address these issues, we propose a Dynamic Masking and Auxiliary Hash Learning (AHLR) method for enhanced cross-modal retrieval. By jointly leveraging the dynamic masking and auxiliary hash learning mechanisms, our approach effectively resolves the problems of channel information imbalance and insufficient key information capture, thereby significantly improving the retrieval accuracy. Specifically, we introduce a dynamic masking mechanism that automatically screens and weights the key information in images and texts during the training process, enhancing the accuracy of feature matching. We further construct an auxiliary hash layer to adaptively balance the weights of features across each channel, compensating for the deficiencies of traditional methods in key information capture and channel processing. In addition, we design a contrastive loss function to optimize the generation of hash codes and enhance their discriminative power, further improving the performance of cross-modal retrieval. Comprehensive experimental results on NUS-WIDE, MIRFlickr-25K and MS-COCO benchmark datasets show that the proposed AHLR algorithm outperforms several existing algorithms.

Dynamic Masking and Auxiliary Hash Learning for Enhanced Cross-Modal Retrieval
Shuang Zhang .2 Yue Wu', Lei Shi Yingxue Zhang', Feifei Kou
Huilong Jin', Pengfei Zhang', MMiyy Liang", Mingying Xu
College of Engineering, Hebei Normal University HHebei Provincial Key Laboratory of Information Fusion and Intelligent Control
State Key Laboratory of Media Convergence and Communication, Communication University of China
*College of Computer and Cyber Security, Hebei Normal University
School of Computer Science (National Pilot School of Software Engineering), BUPT
6 key Laboratory of Trustworthy Distributed Computing and Service, BUPT, Ministry of Education
School of Computer Science and Engineering, Anhui University of Science of Technology Schooo of Artificial Intelligence and Computer Science, North China University of Technology *Corresponding author: leiky_ Biciky_sheeee shiicuc....ccccc edu. cn
Abstract
The demand for multimodal data processing drives the development of information technology. Cross-moda hash retrieval has attracted much attention because it can overcome modal differences and achieve efficient retrieval, and has shown great application potential in many practical scenarios. Existing cross-modal hashing methods have difficulties in fully capturing the semantic information of différent modal data, which leads a significant semantic gap between modalities. Moreover, these methods often ignore the importance differences of channels, and due to the limitation of single goal, the matching effect between hash codes is also affected to a certain extent, thus facing many challenges. To address these issues, we propose Dynamic Masking and Auxiliary Hash Learning (AHLR) method for enhanced cross-modal retrieval. By jointly leveraging the dynamic masking and auxiliary hash learning mechanisms, our approach effectively resolves the problems of channel information imbalance and insufficient key information capture, thereby significantly improving the retrieval accuracy. Specifically, we introduce dynamic masking mechanism that automatically screens and weights the key information images and texts during the training process, enhancing the accuracy of feature matching. We further construct an auxiliary hash layer to adaptively balance the weights of features across each channel, compensating for the deficiencies of traditional methods in key information capture and channel processing. In addition, we design a contrastive loss function to optimize the generation of hash codes and enhance their discriminative power, further improving the performance of cross-modal retrieval. Comprehensive experimental results on NUS-WIDE MIRFlickr-25K and MS-COCO benchmark datasets show that the proposed AHLR algorithm outperforms several existing algorithms.
Introduction
In the field of information retrieval, although traditional unimodal data representation is efficient in processing single-type data, it is difficult to capture data associations and semantic details in the face of increasingly popular cross modsl data. Cross-modal retrieval [ as emerging solution, it effectively bridges the gap between heterogeneous modalities by establishing connections
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

between different data types. In recent years, cross- moosl hashing [ has attracted widespread attention due to its advantages of fast retrieval and efficient storage. It uses hashing tech- nology to convert high-dimensional data into low-dimensional binary hash codes, thereby reducing computational complexity and storage requirements while retaining semantic information.
Currently, some scholars have proposed a variety of new cross-modal hashing retrieval methods. Neu- ral network technologies, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs), have been widely used in cross-modal hashing retrieval. CNN can effectively extract semantic information from images through its powerful feature extraction capabilities, while GAN generates robust hash codes through adversarial training of generators and discriminators, thereby improving the performance of cross- mossl retrieval. In addition, large language models (LLMs) have also been introduced into cross-modal hashing retrieval, which enhance the semantic representation of text modalities through their powerful natural language processing capabilities, thereby improving the accuracy of tross- matching.
Although many methods have achieved good results in the field of cross-modal hashing retrieval, they still face some challenges. Due to the huge semantic gap between different modalities [8] []] often
leads inconsistent cross-modal representations, many noncritical information or noise [0] may
affect the matching accuracy, resulting in similar images and texts being mismatched [.]. Secondly, when processing features. traditional hash layers often ignore the importance differences between different channels [12), which can lead to insufficient capture of key information and difficulty in effectively suppressing noise and redundant information. In addition, when hash codes are generated, they usually rely on single optimization goal, which may lead to insufficient performance of hash codes in cross-modal matching.
To effectively address these challenges, we proposed a method called auxiliary hashing learning (AHLR). significantly improves feature extraction and alignment capability by introducing dynamic mask mechanism. Specifically, the dynamic mask can automatically identify and weight key information in the image and text during the training process, effectively improving the accuracy of matching of cross cross-modal features. In addition, we also constructed an auxiliary hashing layer that can adaptively weight the features of each channel, thereby solving the problem of channel information imbalance, while enhancing the ability to capture key information and effectively suppressing noise interference. Finally, by introducing a contrastive loss function, minimizing the distance between similar samples, and maximizing the distance between heterogeneous samples, the distinguishing ability of hash codes in cross-modal retrieval is effectively enhanced, thereby improving the retrieval accuracy. The main contributions of this paper are as follows:
We propose dynamic masking and auxiliary hash learning (AHLR) method for cross- modal retrieval, which can effectively enhance feature extraction and alignment capabilities, generate more detailed hash codes, and improve the accuracy of cross-modal hashing retrieval.
We introduce a dynamic masking mechanism to automatically select key information from images and text and weight it, thereby improving the accuracy of feature alignment and matching.
We design an adaptive auxiliary hash learning cross-modal module that can adaptively weight the features of each channel, enhancing the retention of key information. Moreover, we introduce the contrast loss function to distinguish the similarity and heterogeneity of the samples and improve cross-modal semantic consistency.
Extensive experiments on three benchmark datasets show that our AHLR outperforms state-of-the the--rrrrtt art baselines, demonstrating clear performance advantages.
2 The Proposed Method
2.1 Notation and problem definition
For ease of understanding, we first introduce the following notation used this article. Assume that there is training dataset O = [(X,, X.) li [1, NJ. E NJ). where X; represents the i-th image sample, X, represents the t-th text sample and N' represents the number of samples. The size of the image is defined as X WxC where H, W, and C represent the height, width,

---

and number of channels of the image, respectively. F2 and Fr represent the features after the extraction of the image and text. The dynamic attention mask is used to process the features of the image and text data, and the generated dynamic mask matrix is defined as M. When M 0, the model calculates the attention weight normally, and when key information. The auxiliary hash layer performs channel dimensionality reduction and weighting processing on the features to obtain the final embedded features fi and f7. The binary hash code B; is generated by the maximum probability selection method. The final hash vector is represented as
-00, the model focuses only on
H (B1; B Bx) E {0, 13 and K represents the length of the hash code.
Cres-modal Feature Learning
Auriliary Hash Learning Module
Tranformer Emooder
Erdeed
Mema
Lineer Projection of leneed Paiches
Emage
Americ
Layer
Auniliay Lowrning
Laver Tnrkiaraadis Layel
Vican
engil Xi Xom
Dynemic teetarr Mack
Conteet Loss
Codae saxilatty
Steileer
white silling greea
bmnken up Frequency
Fit white? Is! sating [rass]
Rabad:
MID
Merge Updited
Angtin
Tokes
Hash Lager
Aucilay Locnaag Liber Tonnermaaie
The AHLR framework consists of two main modules: 1) cross-modal feature learning module that extracts features from images and texts. By introducing dynamic attention mask, automatically selects and weights the key information in images and texts according to the length of the input sequence; 2) an auxiliary hash learning module that adaptively weights the features each channel, optimizes the feature representation to assist in the generation of hash codes, and introduces a contrast loss function enhance the distinguishing ability of hash codes.
Figure 1:
2.2 Framework Architecture AHLR
The AHLR framework in Figure] contains two modules: Cross-modal Feature Learning Module and Auxiliary Hash Learning Module. For cross-modal feature learning, the Vision Transformer is employed as the image feature extractor, after which the features are fused via Transformer obtain the final image representation. For the text modality, Byte Pair Encoding (BPE) is first applied to segment the input text, followed by Transformer-basee fusion process generate the corresponding text representation. In order to adapt to input sequences of different lengths, the model introduces a dynamic attention mask mechanism to automatically generate the corresponding mask matrix, so that the model can effectively ignore irrelevant information and focus on key features. Traditional attention mechanisms assign continuous soft weights all tags or image patches. However, even information with minimal contribution to the task is retained rather than discarded. In contrast, the proposed dynamic masking mechanism adaptively selects semantic key information from both image and text modalities during training and assigns appropriate weights. This process effectively suppresses modality redundancy and irrelevant regions, thereby enhancing semantic consistency across modalities, strengthening feature alignment, and ultimately improving retrieval efficiency. In the hash code learning part, an auxiliary hash layer is designed to enhance representation ability of the hash code by adaptively weighting the features of each channel, and a contrast loss function used strengthen the feature aggregation between similar samples, while pulling away the hash representation of dissimilar samples to improve the retrieval performance of the model.
3

---

2.3 Cross-modal Feature Learning Module
In the AHLR model, the cross-modes feature learning module is mainly divided into two steps. namely image feature learning and text feature learning.
2.3.1 Image Feature Learning
First, use the Vision Transformer (ViT) 13 to extract deep features from the input image. Given an input image of size X |x it is split into patches of size P P. and the total number of patches generated is represented as follows:
H W HWW H.W
(I)
P P2
where, H. W. and represent the height, width, and number of channels of the image, respectively, Each patch is flattened into a vector to form matrix representation Xpatch- The specific process is as follows:
X patch E Nx...P.C)
The shape the matrix representation is P2, C, which represents the feature dimension of each patch after flattening. Subsequently, it is mapped to a feature space of dimension D through the linear transformation. The mapping process as follows:
F X patch W E + bg, WE (PP.c)) be TR.
where, We E is the linear transformation matrix and b the bias term. Then, maintain spatial information, a position code P is added to each patch. The specific input of the Vision Transformer is as follows:
= P.PER
The encoded patch features are input into the Transformer. The dynamic attention mask sets the corresponding mask matrix according to the sequence length of the image input. The definition of the dynamic attention mask matrix M is as follows:
if Si if x
Mij
(5)
DC
When M. 0, this means that the model normally calculates the attention weight. When M;j- -oo, the attention score of this position will be completely blocked, which will make the model focus the key parts while avoiding the interference of invalid information. The dynamic attention mask matrix restricts each token to attend only to its current and previous positions. Although the mask matrix is determined by the length of the input sequence and does not directly contain semantic information, during the training process, the model can model global and local information through the multi-head attention mechanism and ferw-forward network, thereby gradually learning key information under the mask restriction. Subsequently. the global and local information of the image is captured through the multi-head self-attention mechanism and feedforward network. The specific formula is as follows:
Q = F=F. K= FWW V FFWW
(6)
QK"
Attention (Q, V) = =sof -ooffmaa
M
V
(7)
Vdl.
X; = Concall
Attentionn, "==ConalAAttention,,
Attention,) w
(8)
where, Wq. W k and Wy represent the transformation matrices of query, key and value, respectively, de is the dimension of the key vector, and wo is the output projection matrix of multi-head attention. Finally, after the MLP layer of the Transformer encoder, the final feature representation F, of the image is obtained. The specific process is as follows:
F ML)(X.)
(9)

---

2.3.2 Text Feature Learning
For text data, the model uses Byte Pair Encoding (14 for subword-level encoding, splitting the input text into subword units, and gradually merging high-frequency subword pairs according to the frequency of occurrence of subword pairs in the text, thereby generating tokens with more text semantic information. After BPE encoding, the text representation is as follows:
F, RER R
(10)
where, L represents the length of the token sequence and D is the feature dimension. Next, add the corresponding position encoding P to the final tokens to ensure that the model can understand the order relationship in the word sequence. The process of adding position encoding is shown as follows:
F =
(11)
F is the final input feature after adding the position encoding. The tokens sent to the Transformer model, the following process is similar to the image processing process, and the final feature representation Fr of the text is obtained. The specific process is follows:
Fr MLP(X,)
(12)
2.4 Auxiliary Hash Learning Module
When processing features, traditional hash layers often ignore the importance differences between different channels, which may lead to insufficient capture of key information and difficulty in effectively suppressing noise and redundant information. In addition, traditional hashing methods usually rely on a single optimization goal, which may lead to poor performance of hash codes in cross modal matching. To solve this problem, we built auxiliary hash code generation module that is designed to generate hash codes that contain more information.
First, the linear hash layer performs a linear transformation on the input features F, and Fr, mapping the nigh-dimensional input of images and texts into a -iw-dimensional embedding space to generate preliminary feature representation. The specific mapping process is as follows:
f = F,W, + by, Jt = WT + bt
(13)
where W,, WT E Dxd is the weight dimension reduction matrix, which maps the original characteristic dimension D the embedding space of dimension d, and by, by g is the bias term. The mapped features are processed nonlinearly through the ReLU activation function. The main processing process as follows:
= ReLU(())
(14)
The auxiliary hash layer is used to adjust the channel weights of the embedded features fy and fr after dimensionality reduction, and the weights are adaptively assigned according to the global importance of each channel. The main process is average pool the feature maps in all feature channels to obtain the global statistical information S of the channel. The specific process is shown as follows:
N S Σ 1=1
= N
S
(15)
where j can be expressed as 7 or T, then / represents the feature J or Fr. the reduced dimension feature of the image and text. N represents the spatial dimension of the feature. The importance weight of each channel is calculated using the fully connected layers (FC, and FC2), and the global statistical information S reduced and restored so that the model can capture the nonlinear relationship between channels. FC, reduces the number of channels from C to smaller dimension (C/R), and then activates through ReLU. The process of dimensionality reduction and activation is shown as follows:
Q ReLU + byc,), Q E
(16)
5

---

where R represents the channel compression rate, which is used to control the number of channels after dimensionality reduction. Then FC2 is used to restore the channels to the original number of channels C. The specific recovery process V is shown as follows:
V FW + bFC V E E
(17)
Finally, the weight coefficient W of each channel is obtained through the Sigmoid activation function, and the activation generates the channel weight representation as follows:
W Sigmoid (V), W, E [0,
(18)
Each channel C' of features f1 and f is multiplied by the corresponding weight W; to strengthen the representation of important channels and suppress the influence of unimportant channels. The specific process is shown as follows:
f -
(19)
where @ represents the hhaanel-bby-haanee dot product, J7 and f represent the embedded features after dimension reduction and weighting. The embedded features fÿ and f are mapped using the linear transformation layer F(.) to map the hash code of each bit to the binary probability distribution. The specific mapping process is shown as follows:
F(f:) WHS" bH
(20)
where WH represents the trainable weight matrix, which is mainly used to transform f2 into new space, and by represents the bias term. After the mapping transformation, the final hash code is determined by the calculated probability distribution F.. The main process is shown as follows:
F = softmax(F(({))
(21)
where r can be represented as an image or text. The final binary is determined by the maximum probability selection method. The hash code generation process shown as follows:
0 if F F1[1] F < F1[L]
B. = {1
(22)
where B; is the binary hash code of the i-th bit, the length of the hash code is represented by K, and the vector representation of the entire hash code is shown follows:
H BB,, B BR) E (0, 13*
(23)
The auxiliary hash code generation module solves the channel neglect problem of the traditional hash layer through channel weighting, enhances the expressiveness of the hash code, and makes the matching between different modal data in the model more robust.
2.5
Loss Function
The model introduces contrast loss into the hash code module. aiming to maximize the similarity between positive sample pairs (image and text) while minimizing the similarity between negative samples. First, the mapped image feature fi and text feature ft are represented as X, and X2, and normalized so that the vector module length between different modalities is 1, thereby avoiding the similarity calculation deviation caused by different vector lengths. The specific normalization process is shown as follows:
X X1
X
X
(24)
X
2
X, and X₂ represent the normalized image and text embeddings, respectively. Then dot product calculation is performed to calculate the cosine similarity matrix between the image and the text. The main definition is as follows:
X
X
Sij
(25)
X
X2
2

---

Table 1: The mAP comparison results on three datasets
MIRFlickr-25K
NUS-WIDE
MS-COCO
Task
Method
16bits
32bits
64bits
16bits
32bits
64bits
16bits
32bits
64bits
DISRH
0.6652
0.6873
0.6987
0.5271
0.5582
0.6015
0.5257
0.5454
0.5646
JDSH
0.7276
0.7426
0.7468
0,6536
0.6601
0.6900
0.5928
0.6348
0.6517
CDTH
0.7317
0.7461
0.7477
0.6596
0.6613
0.6700
0.5853
0.6411
0.6573
1-T
UCCH
0.7606
0.7620
0.7674
0.6718
0.6738
0.6891
0.6039
0.6249
0.6398
MICAH
0.7960
0.8080
0.8150
0.6440
0.6410
0.6430
0.5700
0.5620
0,5620
DCHMT
0.8177
0.8221
0.8261
0.6711
0.6812
0.6932
0.6450
0.6331
0.6647
AHLR
0.8203
0.8233
0.8266
0.6777
0.6884
0.6994
0.6454
0.6582
0.6797
DISRH
0.6710
0.6958
0.7043
0.5575
0.5680
0.5952
0.5590
0.5591
0.5519
JDSH
0.7304
0.7326
0.7481
0.6439
0.6640
0.6921
0.5888
0.6510
0.6635
CDTH
0.7315
0.7464
0.7503
0.6788
0.6815
0.6910
0.5846
0.6427
0.6573
T-I
UCCH
0.7343
0.7342
0.7410
0.6740
0.6812
0.6945
0.6023
0.6258
0.6371
MLCAH
0.7940
0.8050
0.8050
0.6620
0.6730
0.6870
0.5440
0.5470
0,5940
DCHMT
0.8007
0.8021
0.8065
0.6852
0.6963
0.7009
0.6298
0.6176
0.6616
AHLR
0.8046
0.8052
0.8154
0.6952
0.7040
0.7144
0.6451
0.6557
0.6672
Siy represents the cosine similarity between the i-th image and the j-th text embedding. X X
represents the dot product between vectors, |Xill and X are the L2 norms of X and X
respectively, representing the length of the vector. Since X, and X₂ have been normalized, Siy can be simplified follows:
X X
(26)
The labeling matrix A marks whether each pair of image and text is positive sample or a negative sample, where positive sample refers matching pair between image and the corresponding text, and a negative sample refers to pair between an image and an irrelevant text. The loss between positive samples calculated follows:
Eposition ΣΣAN (1
(27)
where Aij the element in the label matrix, and then the loss of the negative sample pair calculated. & controls the similarity between negative samples and pushes the similarity between negative samples down the set boundary ç. The specific process is shown as follows:
N
Enegative >>>
(1 A.y) ReLU(S. - E
(28)
Experiments
In this section, assess the performance of the proposed AHLR framework across three benchmark datasets using cutting-edge techniques. We subsequently conduct an in-depth analysis through systematic ablation studies to examine the role of each component within our model.
3.1 Experimental Settings
In this paper, we use the Vision Transformer as the feature extractor of images and adopt the BPE method to perform word segmentation on text. Subsequently. Transformer is further used to fuse and process the feature representations of images and text. At the same time, the model introduces a dynamic attention mask mechanism. We experimentally analyzed the parameters involved and selected the most appropriate value, which 1. In the experiment, the batch size set to 64, the Adam optimizer 15 is used for the main optimization, and the method of dynamically adjusting the learning rate is adopted, where the initial learning rate of 1e-3, a decay schedule of 0.9 times the learning rate every 5 epochs, and weight decay of 0.2. The AHLR method is mainly implemented
7

---

I-TI MIRFIC225KK 18sts
1-T MiRFFce25KK 32xt
T MIRFFCE2 6tats
0.00
0.50
0.05
0780
018
Proiskm
Peztikon
Pretsion
0.75
075
0.70
0.70
0.70
0.85
10.0
20.0
3000
4000
5000
1000
2.00
3000
4000
5000
1000
2000
300
40.0
5000
K
tap-K
top-K
T- @0 MIRPhickr2 10hsts
T-16 MiRK-ccce 30thtt
T (6 MIRFaade22KK Glbiss
0.83
0,80
Poxiske
0.80
0,80
0.75
0.70
0.00
0.70
0.70
0.00
10.0
3.001
1000
2000
-000
4000
10001
20.0
4.00
5000
top K
top K
top K
Figure 2: The Top-K curves on the MIRFlickr-25K
I--T @ NUSIIDE 12at
I. @ NUSWIDE 32HS
1--T Ø NUSWIDE GANIS
0.80
0.80
0775
0.70
0.6
0.70
Pretsion
0.70
0.56
0.00
0.00
0.00
0.50
0.50
trp-K
top-K
tog-
T-1 @ NUSWIDE 16M0
T-1 NUSWIDE 30bits
T- @ NUSWIDE
0.75
0.75
0.80
0.70
0.70
0.75
Protson
0.65
Protsson
0.68
Prstsion
0.70
0.00
0.00
0.55
top-K
top-K
top-K
Figure 3: The Top-K curves on the NUS-WIDE
@
0.80
0.80
0.65
0.75
0.75
0.80
0.70
0.70
Prodtiin
0.75
0.53
0.60
0.50
1000
X000
3000
4001
3000
1000
-000
-000
4000
5000
72001
(000
400
500
lop-K
lay-K
lug-K
3Chits
644bits
0.00
0.00
0.89
0.70
0.80
PrecSSon
0.60
0.65
0.55
0.35
0.60
0.30
0.30
0.50
1000
28000
3000
8000
5000
1000
2000
3000
4000
3000
1000
2000
3000
41000
5000
top-K
tap-K
Figure 4: The Top-K curves the MS-COCO

---

Table 2: Ablation study the three datasets
MIRFlickr-25K
NUS-WIDE
MS-COCO
Task
Method
16bits
32bits
64bits
16bits
32bits
64bits
16bits
32bits
64bits
AHLR-M
0.8086
0.8161
0.8193
0.6753
0.6850
0.6946
0.6396
0.6516
0.6642
I
AHLR-A
0,8083
0.8115
0.8179
0.6686
0.6818
0.6894
0.6447
0.6578
0.6688
AHLR
0.8203
0.8233
0.8266
0.6777
0.6884
0.6994
0.6454
0.6582
0.6797
AHLR-M
0.7992
0.8005
0.8093
0.6929
0.7011
0.7132
0.6319
0.6525
0.6656
T-1
AHLR-A
0.7988
0.7995
0.8066
0.6865
0.6974
0.7031
0,6422
0.6540
0,6669
AHLR
0.8046
0.8052
0.8154
0.6952
0.7040
0.7144
0.6451
0.6557
0.6672
based Pytorch [6] and all experiments RTX 3080 graphics card with 40GB RAM to ensure the stability of the experiment.
equipped with NVIDIA GeForce
3.2 Comparison Methods
There are two cross-modal hashing retrieval tasks (convert image to text and text to image). We compare the performance of the AHLR method with six other cross-modal hashing methods on three datasets. The specific methods are as follows: DCHMT JDSH [L) 2CCH CDTH(8] and DISRH(2].
3.3 Performance Comparison
To demonstrate the efficacy of the AHRL algorithm, we performed comparative analysis of its performance across three datasets. Table [ shows the mAP values of the AHLR method on the MIRFlickr-25K, NUS-WIDE, and MS-COCO datasets. The best results in the table are in bold, where + T denotes image-to-text retrieval and -> I" indicates text-to-image retrieval.
From the mAP of the three datasets, we can see that the AHLR method we proposed can achieve good performance on both small-scale and large-scale datasets. Compared the baseline, our method achieves over 10% higher mAP across the three datasets than the lowest experimental method. This due the introduction of the auxiliary hash module, which enables the model to generate hash codes containing rich semantic information. When facing larger datasets, the performance between different bits gradually stabilizes. This because the dynamic mask mechanism proposed by AHLR enables the model to adaptively focus on key parts while ignoring unimportant information, thereby improving retrieval accuracy.
Figures 2 3 and show the Top-K accuracy curves of the AHLR method on the three datasets. The
value K covers multiple retrieval ranges, mainly including 50, 100, 200, 500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500 and 5000. As can be seen from the figure, as the number of retrieval samples K increases, the retrieval performance of the AHLR method always significantly higher than that of other comparison methods. As the number of retrieval samples continues to increase, the retrieval performance of the model gradually stabilizes, indicating that our method maintains good stability and reliability while improving retrieval accuracy.
3.4 Ablation Study
In order to verify the effectiveness of each module in the model, the following ablation experiments designed:
AHLR-M: This experiment removes the dynamic attention mask, and all other modules the same as in AHLR.
AHLR-A: This experiment removes the auxiliary hash learning module, and all other modules are the as in AHLR.
The results of the ablation experiment are shown in Table 2 After AHLR-M removes the dynamic attention mask, the performance of the model in the image-text cross-modal retrieval task is signifi- cantly reduced, especially in the accuracy mAP indicator. This shows that the dynamic attention mask

---

T-T
T-
0.85
0.85
0.80
0.80
mAP
0.75
mAP
0.65
0.65
NUSWVDDE
NUSWID
Figure 5: Parameter sensitivity of
can effectively capture the key information in the image and text, and flexibly weight it according to the sequence length, thereby improving the representation ability of the feature.
After AHLR-A removes the auxiliary hash learning module, the retrieval performance of the model reduced, especially under the condition of short hash code length, the performance decline more obvious. This shows that the auxiliary hash learning module helps to generate more compact and more discriminative hash codes by adaptively weighting the features of each channel, thereby improving the retrieval effect.
The results of the ablation experiment show that the dynamic attention mask and auxiliary hash learning module have an impact on the overall performance of the model and demonstrate their important role in improving mross-modal retrieval performance.
3.5 Parameter Sensitivity
We analyze the parameter & we designed and select unified 64-bit hash code to verify it on three data sets. The experimental results show that when the value the parameter is adjusted, the mAP result will fluctuate accordingly. The specific results are shown in Figure It can be seen from the figure that when the value of { 1, the mAP value of the model the highest, indicating that retrieval performance of the model is optimal this time. As the value of & continues increase, the performance of the model shows a downward trend, indicating that an excessively large & may affect the model's expression of feature information, resulting in decrease in retrieval accuracy. Therefore, based on the comprehensive consideration of the performance of the model, taking & as more in line with the performance of the model, and ensure good stability and effectiveness the three data sets.
Conclusion
We propose auxiliary hash learning (AHLR) for cross- moos- retrieval methods. By introducing dynamic mask mechanism, the key information between different modalities automatically selected and weighted to enhance the feature representation and semantic alignment between modalities. In addition, an auxiliary hash layer is constructed to adaptively weight the features of each channel, and combined with the contrast loss function, AHLR minimize the distance between similar samples and maximize the distance between heterogeneous samples, thereby improving the distinguishing ability of hash codes in cross-modal retrieval tasks and further improving the accuracy of retrieval. Comprehensive experiments have proved the effectivvness of this method. We mainly study the retrieval between images and texts, and cannot effectively process other types of multimodal data (such as video, audio, etc.), which leads to limited applicability of the model and low scalability. Therefore, how to explore semantic alignment methods between multiple different modalities, realizing mutual retrieval, and improving the versatility and applicability of the model is an important direction for future research. Cross-modal hashing retrieval can improve information retrieval efficiency and promote cultural communication and educational innovation. However, it may also leak privacy, threaten security, amplify social bias, and impact employment structure. Therefore, strengthening data privacy security is crucial to ensure its positive social impact.
10

---

Acknowledgments
This work is supported by the Science and Technology Project of Hebei Education Department (No. CXY2024050), the Joint Fund Key Program of the National Natural Science Foundation of China (No. U23B2029), the Fundamental Research Funds for the Central Universities (No. CUC25SG013), the Beijing Municipal Natural Science Foundation (No. L257023), the Agricultural Scientific and Technological Achievements transformation Funds of Hebei Province (No. 2025JNZ- S24), and the North China University of Technology 2025 Youth Research Special Project (No. 2025NCUTYRSP012),
References
[1] T. Wang, F. Li, L. Zhu, Li, Z. Zhang, and H. T. Shen, "Cross-modal retrieval: A systematic review of methods and future directions," Proceedings of the IEEE, vol. 112, no. 11, pp. 1716-17767 1754, 2024.
[2] Y. Li, W. Wang. L. Qu, L. Nie, W. Li, and T.S. Chua, "Generative cross-modal retrieval: Memorizing images in multimodal language models for retrieval and beyond." arXiv preprint arXiv:2402. 10805, 2024.
[3] M. Cao, S. Li, J. Li, L. Nie, and M. Zhang, "Image-text retrieval: A survey on recent research and development," arXiv preprint arXiv:2203 147133 2022.
[4] Z. Hu, Y-M. Cheung, M. Li, and W. Lan, "Cross-modal hashing method with properties of hamming space:
A new perspective." IEEE Trunsactions Puttern Anulysis and Machine Intelligence, vol. 46, no. 12, pp. 7636-7650, 2024.
[5] L. Xu, H. Li, B. Zheng, W. Li, and J. Lv, "Deep lifelong cross-modal hashing." IEFF Transactions on Circuity and Systems for Video Technology, vol. 34, no. 12, pp. 13478-13493, 2024.
[6] J. Qin, L. Fei, Z. Zhang. J. Wen, Y. Xu, and D. Zhang. "Joint specifies and consistency hash learning for large-scale cross modsl retrieval," IEEE Transactions on Image Processing, vol. 31, pp. 5343 5358, 2022.
[7] L. Zhu, C. Zheng, W. Guan, Li, Y. Yang, and H. T. Shen, "Multi-modal hashing for efficient multimedia retrieval: A survey," IEEE Transactions Knowledge and Data Engineering, vol. 36, no. 1, pp. 239-260, 2023.
[8] F. Li, B. Wang. L. Zhu, Li, Z. Zhang, and X. Chang, droos--domain transfer hashing for efficient cross mross-modal retrieval," IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 10, pp. 9664-9677, 2024.
[9] Z. Hu, Y.M. Cheung, M. Li, and W. Lan, "Cross-modal hashing method with properties of hamming space: A new perspective," IEEE Transactions on Pattern Analysis and Machine Intelligenee, vol. 46, no. 12, pp. 1636-7650, 2024.
[10] R. Pu, Y. Sun, Y. Qin, Z. Ren, X. Song, H. Zheng, and D. Peng. "Robust self-paced hashing for modal retrieval with noisy labels." Proceedings of the AAAI Conference Artificial Intelligence, vol. 39, 19, pp. 19 1999-19977777 Apr. 2025.
[11] Y. Sun, K. Liu, Li, Ren, J. Dai, and D. Peng. "Distribution consistency guided hashing for cross-modal retrieval," in Proceedings of the 32nd ACM International Conference on Multimedia, ser. MM '24. New York, NY, USA: Association for Computing Machinery, 2024, p. 5623-5632.
[12] F. Wu. S. Li, G. Gao, Ji, X.-Y. Jing, and Z. Wan, "Semi-supervised cross-modal hashing via modality- specific and cross-modal graph convolutional networks," Pattern Recognition, vol. 136, pp. 1-10, 2023.
[13] A. Dosovitskiy, L., Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, and S. Gelly, "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.1 1999, 2020.
[14] R. Sennrich, B. Haddow, and A. Birch, "Neural machine translation of rare words with subword units," arXiv preprint arXiv: 150.0099999 2015.
[15] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv: 1412.6900. 2014.
[16] A. Paszke, "Pytorch: An imperative style, high- regh-performance deep learning library," arXiv preprint a01iv.1912.01703,200 2019.

---

[17] J. Tu, X. Liu, Z. Lin, R. Hong, and M. Wang, "Differentiable cross- moos-modal hashing via multimodal transformers," " in Proceedings the 30th ACM International Conference on Multimedia, 2022, pp. 453- 461.
[18] S. Liu, S. Qian, Y. Guan, J. Zhan, and L. Ying, Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval," in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Virtual Event China: ACM, Jul. 2020, pp. 1379-1388
[19] X. Ma, T. Zhang, and C. Xu, "Multi-level correlation adversarial hashing for cross-modal retrieval," JEEE Transactions on Multimedia, vol. 22, no. 12, pp. 3101-31 14, 2020.
[20] P. Hu, H. Zhu, J. Lin, D. Peng, Y.-P. Zhao, and X. Peng. "Unsupervised contrastive cross-modal hashing." IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 3877-3889, 2022.
[21] S. Su, Z. Zhong, and C. Zhang, "Deep joint-semantics reconstructing hashing for large-scale unsupervised cross-modal retrieval," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3027-3035.
[22] Z. Yang. X. Deng, L. Guo, and J. Long, "Asymmetric supervised fusion-oriented hashing for cross-modal retrieval," IEEE Transactions on Cybernetics. vol. 54, no. 2. pp. 851-864, 2024.
[23] K. Han, Y. Liu, R. Wei, K. Zhou, J. Xu, and K. Long, "'Supervised Hierarchical Online Hashing for Cross-modal Retrieval," ACM Transactions on Multimedia Computing, Communications, and Applicutions. vol. 20, no. pp. 1-23, Apr. 2024.
[24] Liang, Du Z. Liang. Y. Xing, W. Huang, and Z. Xue, "Self-supervised multi multi--odal knowledge graph contrastive hashing for cross-moda search," " in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 13 744-13
[25] Z. Hu, Y.-M. Cheung, M. Li, W. Lan, D. Zhang, and Q. Liu, "Joint semantic preserving sparse hashing for cross-modal retricval," IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. pp. 2989-3002, 2023.
[26] Z. Lin, G. Ding, M. Hu, and J. Wang, "Semmatiic-preservinn hashing for cross-view retrieval," in Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3864-3872.
[27] Q.-Y. Jiang and W.-J Li, "Deep cross-modal hashing," in Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2017. pp. 3232-3240.
[28] I.. Fei, 7. Hc, W. K. Wong, Q. Zhu, S. Zhao, and J. Wen, "Semantic decomposition and enhancement hashing for deep modal retrieval," Puttern Recognition, vol. 160, pp. 1-15, 2025.
[29] J. Wang, 7.. Zeng. B. Chen, Y. Wang, D. Liao, G. Li, Y. Wang, and S.-T. Xia, "Hugs Bring Double Benefits: Unsupervised Cross-Modal Hashing with Multi granulata Aligned Transformers," International Journal of Computer Vision, vol. 132, no. 8, pp. 2765-2797, Aug. 2024.
[30] D. Yao. Z. Li, B. Li. C. Zhang, and H. Ma. "Similarity unsupervised cross-modal hashing," Expert Systems with Applications, vol. 237, pp. 1-13, 2024.
reconstruction network for
[31] R.-C Tu, J. Jiang, Q. Lin, Cai, S. Tian, H. Wang, and Liu, "Unsupervised cross modal hashing with modality-interaction IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 5296-5308, 2023.
[32] X. Xia, G. Dong. F. Li, L. Zhu, and X. Ying. "When CLIP meets cross- cooss-modal hashing retrieval: A new strong baseline," " Information Fusion, vol. 100, pp. 1-12, 2023.
[33] Song, Y. Yang, Y. Yang, Huang, and H. T. Shen, "Inter-media hashing for large-scale retrieval from heterogeneous data sources," in Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. New York New York USA: ACM, Jun. 2013, pp. 785-796.
[34] M. J. Huiskes and M. S. Lew, "The MIR flickr retrieval evaluation." " in Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. Vancouver British Columbia Canada: ACM, Oct. 2008 pp. 33.43.
[35] T.SS. Chua, J. Tang, R. Hong, H. Li, 7. Luo, and Y. Zheng, "NUS-WIDE: A real-world web image database from National University of Singapore," " in Proceedings of the ACM International Conference on Image and Video Retrieval. Santorini, Fira Greece: ACM, Jul. 2009, pp. 1-9.
[36] T.-Y. Lin, M. Maire, S. Belongie, Hays, P. Perona. D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft COCO: Common Objects in Context," in Computer Vision ECCV 2014. D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing. 2014, vol. 8693, pp. 740-755.
12

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly describe the main contributions of this paper, and propose a new cross- moodll hashing retrieval method to solve the matching accuracy and single optimization target problems caused by the semantic gap. The main contributions are as follows: We introduce a dynamic attention mask module to enhance the feature extraction and alignment capabilities between different modal data, thereby improving the matching accuracy of cross-modal features. An auxiliary hash layer is constructed to adaptively weight the features of each channel. We optimize the generation of auxiliary hash codes for feature representation. Theoretical analysis and experimental results support the above viewpoints and verify the effectiveness of this method.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize to other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the conclusion of the paper, we explain the limitations of this work in detail. The main limitation that we mainly study the retrieval between images and texts, and do not extend it to retrieval between other types, which leads to limited applicability of the model.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-ted system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
13

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
In Sections 2.3 to 2.5 of this paper, we describe in detail each step involved in the framework, including feature extraction, auxiliary hash learning, and loss function modules, ensuring that each module follows a clear and progressive logic. All theorems, formulas, and proofs are listed in these sections, with the main proof summary attached, and numbered to maintain the coherence and rigor of the argument.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Our proposed AHLR method builds a new model framework and introduces its core components in detail in Section 2.2, "Network Architecture of AHLR." The model consists of cross-modal feature extraction and auxiliary hash learning modules. In Section 3, "Experiments" we fully describe the experimental details to ensure the reproducibility of the experiments.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
14

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The source code associated with our paper can be accessed on GitHub the following link: https:: anonymous .oppn sci science/r R/HLLe////AAA--eO
The three datasets used in this study and their download links are as follows:
mirflickr25k:
NUS.WIDEhhttps://2nn. comp nus hts.ccd....p..ooottrr///p/o/o//0
research//nuvvide/NUS--I
MS-COCO: http://ttcs///cocattstttttttttt orw.oonnoooddddddddddd
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and basclines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
15

---

Justification: We use Vision Transformer as the feature extractor for images and the BPE method for word segmentation of text. Subsequently, Transformer is used to further fuse and process the feature representations of images and text. In the experiment, the batch size is set to 64, the Adam optimizer used for the main optimization, and the learning rate dynamically adjusted. Our AHLR method is mainly implemented based on Pytorch, and all experiments are run on a server equipped with an NVIDIA GeForce RTX 3080 graphics card with 40GB RAM to ensure the stability of the experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
The performance of our model is evaluated using the top-K curve. The top-K curve evaluates the performance of the model in retrieving relevant terms in the top-K predictions, thus providing a practical perspective on its effectiveness in real-world applications. is important metric our experiments.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma eтror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The AHLR method in this paper is implemented in Pytorch and experiments are performed on a server equipped with an NVIDIA GeForce RTX 3080 GPU (40GB RAM). The paper clearly specifies the computing resources required to reproduce the experiments, including the type computing device and memory size, enabling other researchers to reproduce the experiments under similar conditions.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https:
Answer: [Yes]
Justification: The code of our paper as follows:
sciencecceeee
r/ T/ //AHLR-C422
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Cross-modal hashing retrieval significantly improves the efficiency of informa- tion retrieval and promotes the communication and integration between different cultures. However, cross-modal hashing retrieval also has some potential negative effects, which may lead unauthorized use and dissemination problems and face challenges of privacy leakage and security threats. Therefore, while paying attention to the positive impact of cross-moda hashing retrieval technology, it is also necessary deal with related negative impacts and challenges.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
17

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All existing resources used in this paper, including the comparative experiments and the three datasets, have been properly cited.
The main download links for the datasets are as follows:
MIRFlickr25K
mirflickr25k
NUS- DOSS-WID,hhttpss///nn come:pppp/.mmm.coomp.p.oom nus edu. http://ww......ppp..oorrrr/oppap
research research/nuswideee wttpcc//wwwwiiewww//DOOOS html
MS-COCO
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwwe thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
18

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve the L.M as any significant, original or non-standard component.
Guidelines:
19

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (https://neurip:: /ttp://pp.ccc//offfeeeee//////// for what should or should not be described.
20

---

Appendix supplemental material
A Related Work
In this section, we introduce existing work related our study, focusing on two forms of cross- modal hashing retrieval, namely, unsupervised cross-modal hash retrieval and supervised cross-modal hashing retrieval.
A.1 Supervised Cross-Modal Hashing
Supervised cross-modal hashing retrieval 22) 24 25 optimizes the generation hash codes by leveraging labels or semantic information, thereby improving the accuracy and efficiency of retrieval. The core of supervised cross-modal hashing retrieval lies in mapping data of different modalities (such images and text) into a shared Hamming space, that the similarity between different modalities can be measured by the Hamming distance. In the early stage of research, non-deep cross-modal hashing methods were proposed to bridge the semantic gap between different modalities. For example, Semantics- Preeiicing Hashing (SePH) [26] takes the semantic affinity of given training data as supervised information and converts it into probability distribution and learns the nonlinear projection of features by minimizing KL divergence to make hash codes for each view.
Deep learning methods have gradually become the mainstream methods for supervised cross-modal hashing retrieval due to their advantages in feature extraction and nonlinear representation. For example, Deep Cross-Modal Hashing (DCMH) (7) integrates feature extraction and hash code learning into a deep convolutional neural network framework to achieve end-to-end learning. Semantic Decomposition and Enhancement Hashing (SDEH) improves the performance of cross-modal retrieval by making full use of multi-label semantic information. This method decomposes the shared semantic information between different modalities and bridges the feature and semantic gap between different modalities, thereby achieving efficient cross-modal retrieval. We adopt this supervised method to fully utilize semantic information to optimize hash code generation and enhance cross-modal retrieval performance.
A.2 Unsupervised Cross-Modal Hashing
do not require labels or semantic information, and map data from different modalities a shared Hamming space through learning. Unsupervised hashing methods do not require pre-annotated semantic labels but instead discover common representations between different modalíties by mining potential intra- modal and inter-modal connections. Early unsupervised hashing methods were mainly based on shallow models, learning hash codes, and hash functions by designing efficient algorithms. These methods usually process manually extracted features. For example, Inter-media hashing (IMH) (33 converts multimedia data from heterogeneous data sources into a common Hamming space and enables fast search through XOR and bit counting operations.
Compared to supervised hashing methods, unsupervised hashing methods (2 33 [30
Deep learning methods have also been widely used in unsupervised hashing. The Unsupervised Contrastive Multi-modal Fusion Hashing Network (UCMFH))32 uses the pre-trained CLIP model to extract features and enhances the interaction between modalities through the multimodal fusion transformer encoder and contrast loss. Recent unsupervised deep hashing methods pay more attention the construction of similarity matrices to reduce redundant information and capture the potential associations between modalities. Deep Seint-Semantics Reconstructing Hashing (DISRH) con- structs a joint semantic affinity matrix that integrates the original neighborhood information from different modalities to capture the potential associations of input multimodal instances.
B Datasets
To evaluate the proposed method, we choose to conduct experiments three widely used cross- modal datasets: MIRFlickr- 255,34,, NUS- WIDE and MS-COCO(3). As shown in the table [ the detailed description of the datasets is as follows.
MIRFlickr-25K contains 25,000 images covering 24 common categories. Each image is accompanied by multiple text labels and is annotated with at least one of the categories. We select image-text pairs
21

---

with at least 20 labels as experimental data, randomly extract 2,000 pairs as query sets, the rest as retrieval sets, and randomly select 10,000 pairs from the retrieval sets as training sets.
MUS-WIDE contains 269,648 web images with text labels covering 81 categories. Each image is annotated with at least of the 81 categories. We select 186,577 image-text pairs, all of which belong to the 10 most common classes, randomly select 2,100 pairs from the dataset as query sets, the as retrieval sets, and randomly select 10,500 pairs from the retrieval sets training sets.
MS-COCO contains 123,289 images covering categories. Each image is labeled as at least one of the 80 categories. In experiments, deleted the samples that did not contain valid instances in the text samples to improve the data quality and reliability of the experiment. 5,000 pairs were randomly selected from the dataset the query set, the rest the retrieval set, and 10,000 pairs were randomly selected from the retrieval set as training data.
Table 3: Datasets Statistics
Dataset
Size
Label
Query
raining
MIRFlickr-25 25,000
24
2,000
10,000
NUS-WIDD
269,648
81
2,100
10,500
MS-COCO
123,289
80
5,000
10,000
C Evaluation Metrics
In the experiment, we use two evaluation indicators to evaluate the performance of the model, namely the mean average precision (mAP) and the Top-K precision curve (top-K curve).
22


Reviews:
[{"review_id": "tAAs5n0ZJn", "reviewer": "Reviewer_tjNt", "summary": "The paper focuses on research in the field of cross-modal hash retrieval and proposes a novel AHLR method aimed at effectively enhancing feature extraction and alignment capabilities, generating more detailed hash codes, and improving the accuracy of cross-modal hash retrieval. This topic has important research value and practical application significance, and the proposed method has a certain degree of innovation.", "questions": "1. Please explain how the image and text features are interactively fused at which step. If not, delete the related description in the article.\n2. The dynamic masking mechanism can automatically filter and weight key information in images and text, but how does this mechanism determine which information is key?\n3. The paper introduces a contrastive loss function to optimize the generation of hash codes. May I ask, what is the design principle of this loss function? How does it ensure that the generated hash code has stronger discriminative power?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S1:A well-written academic paper offers is given. It provides a rigorous and evidence-based approach (AHLR) to solving the semantic gap between heterogeneous modalities such as images and text. \nS2: AHLR uses Vision Transformer and Transformer-based text encoders for unified cross-modal feature learning.\nS3: The dynamic mask filters irrelevant inputs by sequence length, while the auxiliary module adaptively reweights channels for better hash representations.\n\nW1: Section 2.2 mentions \"Transformer is used to further fuse and process the feature representation of images and texts\", but Section 2.3 treats image and text learning separately.\nW2: The innovation of dynamic masking mechanism lies in automatically filtering key information, but its essential difference from existing attention mechanisms is not clear. \nW3: Authors mention using an Adam optimizer with “dynamically adjusting the learning rate,” but do not specify the initial learning rate, decay schedule, or weight decay.", "was_revised": true, "final_justification": "Thank you for the author's response. My problems have been properly resolved. I have decided to keep my score.", "rebuttal": "For  Weaknesses:\n\n1.Weakness:[Section 2.2 mentions \"Transformer is used to further fuse and process the feature representation of images and texts\", but Section 2.3 treats image and text learning separately. ]\n\nResponse: Thank you for your comments. In Section 2.2, we mentioned that \"Transformer is used to further fuse and process the feature representation of images and texts\" means that images use Vision Transformer as feature extractor and then are fused through Transformer to obtain the feature representation of images, at the same time, texts use BPE encoding and are fused through Transformer to obtain the feature representation of texts, as shown in Figure 1. This is why we separate image and text learning in Section 2.3. Since our statement is not clear, it has caused ambiguity that images and texts are fused. We will add more explanations for the utilization of Transformer for images and texts in the final version.\n\n2.Weakness: [The innovation of dynamic masking mechanism lies in automatically filtering key information, but its essential difference from existing attention mechanisms is not clear.]\n\nResponse: Thank you for your comments. The difference between the dynamic masking mechanism and existing attention mechanisms does need to be further clarified. Traditional attention mechanisms (such as self-attention in Transformer) assign continuous soft weights to all tags or image patches, and even if some information contributes little to the task, it will not be truly discarded. The dynamic masking mechanism we proposed can automatically select semantic key information from image and text modalities during training and weight it, thereby suppressing modal redundancy or irrelevant areas, improving semantic consistency between modalities, enhancing feature alignment effects, and improving retrieval efficiency. We will explicitly emphasize the fundamental difference between this mechanism and traditional attention mechanisms in the final version.\n\n3.Weakness: [Authors mention using an Adam optimizer with \"dynamically adjusting the learning rate\", but do not specify the initial learning rate, decay schedule, or weight decay.]\n\nResponse: Thank you for your comments. We welcome the opportunity to clarify the optimizer settings we used in our experiments. In our experiments, we used the Adam optimizer with an initial learning rate of 1e-3, a decay schedule of 0.9 times the learning rate every 5 epochs, and a weight decay of 0.2. We will adding the initial values in the Experimental Settings in the final version.  \n\n\nFor  Questions:\n\n1.Question: [Please explain how the image and text features are interactively fused at which step. If not, delete the related description in the article.]\n\nResponse: Thank you for your comments. As we answered in weakness, we mentioned that \"Transformer is used to further fuse and process the feature representation of images and texts\" means that images use Vision Transformer as feature extractor and then are fused through Transformer to obtain the feature representation of images, while texts use BPE encoding and are fused through Transformer to obtain the feature representation of texts. We will add the above explanations in the final version.\n\n2.Question: [The dynamic masking mechanism can automatically filter and weight key information in images and text, but how does this mechanism determine which information is key?]\n\nResponse: Thank you for your comments. The dynamic masking mechanism guides the model to focus on the key information in the sequence by generating a mask matrix based on the input structure. Specifically, we construct a causal attention mask matrix (as shown in formula (5)) to limit the attention range of tokens at each position in the Transformer, that is, each token can only focus on the current position and its previous position. Although the mask matrix is determined by the length of the input sequence and does not directly contain semantic information, during the training process, the model can model global and local information through the multi-head attention mechanism and feed forward network, thereby gradually learning key information under the mask restriction. We will adding the above explanation in the final version.\n\n3.Question: [The paper introduces a contrastive loss function to optimize the generation of hash codes. May I ask, what is the design principle of this loss function? How does it ensure that the generated hash code has stronger discriminative power?]\n\nResponse: Thank you for your comments. The contrastive loss function we introduced aims to improve the discriminative ability of hash codes by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. The design principle of the contrastive loss function is as follows: As shown in formulas (27) and (28), we use the label matrix A to mark whether the sample pairs are positive samples (images and corresponding texts) or negative samples (images and irrelevant texts). For positive sample pairs, the cosine similarity S is made as close to 1 as possible; for negative sample pairs, the model compresses the similarity values between them through a threshold to ensure that they are far away from the positive sample pairs in the embedding space. Therefore, the contrastive loss function can effectively align semantically related image and text modal representations, while increasing the distance between semantically irrelevant samples, thereby ensuring that the generated hash codes are more discriminative."}, {"review_id": "NO07TmxSll", "reviewer": "Reviewer_vdAW", "summary": "The paper introduces dynamic Masking and Auxiliary Hash-Learning (AHLR) for cross-modal (image–text) retrieval. A dynamic masking block first filters out redundant visual patches and text tokens, keeping only the most informative ones. An auxiliary hash-learning layer then re-weights channel features before hashing, addressing the imbalance between modalities. Trained with a contrastive loss, the two components jointly yield markedly better retrieval accuracy on several widely-used benchmarks.", "questions": "Q1. What are the actual training and inference runtimes (or FLOPs/parameter counts) of AHLR versus DCHMT and UCCH on MS-COCO?\n\nQ2. What are the advantages of the channel weight allocation strategy of the auxiliary hash layer compared to classical methods such as SENet?\n\nQ3. Equation (5) imposes a causal-style mask. How does this pattern select “key” tokens? Would a learnable top-k pruning score be superior?\n\nQ4. Could the dynamic mask be reused for other multimodal tasks (classification, generation)? What modifications would be required?\n\nI would consider to give a higher score if the authors could address my concerns in Weaknesses and Questions.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n+ The auxiliary hash-learning (AHL) layer assigns data-dependent weights to each channel, generating more discriminative hash codes and improving cross-modal retrieval precision.\n\n+ Dynamic masking, paired with a contrastive loss, automatically highlights key regions/tokens and enforces tighter semantic alignment between images and text.\n\n+ Experiments on three public datasets show consistent, often sizeable, gains over strong baselines. Furthermore, ablations isolate the contribution of each module.\n\n** Weaknesses**\n\n- Dynamic masks plus AHL add parameters and operations, yet no analysis of training/inference latency or memory use is provided.\n\n- While addressing channel information imbalance and insufficient key information capture is a novel perspective in hash learning, there are some literatures adopting similar thoughts. For example, [A, B] model the relevance between visual and textual modalities, and [C] addresses modality imbalance across modalities. It would strengthen the paper to draw clearer connections with these related works and explicitly clarify the differences. [A] Noise-robust Vision-language Pre-training with Positive-negative Learning. [B] Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining. [C] Test-time Adaption against Multi-modal Reliability Bias.\n\n- The paper states that image and text features are fused via a Transformer, but omits the exact token construction, positional encoding, and cross-attention setup—making reproduction difficult.", "was_revised": true, "final_justification": "Thank you for the detailed responses, which have addressed all my concerns. After reading the comments from other reviewers, I believe this is a good work and hope it will be accepted.", "rebuttal": "For Weaknesses:\n\n1. Weakness: [Dynamic masks plus AHL add parameters and operations, yet no analysis of training/inference latency or memory use is provided.]\n\nResponse: Thank you for your comments. The training process is on an NVIDIA RTX 3080 GPU with 64-bit hash codes and batch size 64, our method requires about 5.75 minutes per epoch for training and 2 minutes 38 seconds for inference on MIRFLICKR (5,120 queries over 19,461 gallery items) with peak GPU memory usage of 6.95 GB and 77% GPU utilization; about 10.5 minutes per epoch for training and 5 minutes 45 seconds for inference on MS-COCO (5,120 queries over 117,098 gallery items) with peak GPU memory usage of 6.87 GB and 64% GPU utilization; and about 11 minutes per epoch for training and 6 minutes for inference on NUS-WIDE (5,120 queries over 190,714 gallery items) with peak GPU memory usage of 6.87 GB and 77% GPU utilization.\n\n2.Weakness: [While addressing channel information imbalance and insufficient key information capture is a novel perspective in hash learning, there are some literatures adopting similar thoughts. For example, [A, B] model the relevance between visual and textual modalities, and [C] addresses modality imbalance across modalities. It would strengthen the paper to draw clearer connections with these related works and explicitly clarify the differences. [A] Noise-robust Vision-language Pre-training with Positive-negative Learning. [B] Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining. [C] Test-time Adaption against Multi-modal Reliability Bias.]\n\nResponse: Thank you for your comments. We note that methods such as [A], [B], and [C] use similar ideas in modeling visual-textual correlation or dealing with modality imbalance, but our method is somewhat different from these works in terms of specific mechanisms: [A] Under the synergy of four pre-training objectives and the positive-negative contrast learning mechanism, it dynamically filters out noise correspondences, thereby capturing multi-level visual-textual correlations from token-level, pair-level, intra-modal to cross-modal, and modeling fine-grained semantic correlations between visual and textual modalities, but it is only applicable to large-scale pre-training stages and is not designed specifically for hash code learning; [B] The image-text pairs are divided into three categories and two mechanisms are designed to reconstruct their consistency. By \"refining and correcting pseudo-matches in positive samples\" and \"mining hidden consistency from negative samples\", the semantic consistency between images and texts is dynamically reconstructed in the feature spaces of the two modalities, thereby modeling the correlation between visual and textual modalities, but it focuses on robust retrieval scores rather than compact binary hashing; [C] Adaptive attention modules are used to achieve reliable cross-modal fusion, and confidence-aware loss functions are used to achieve robust adaptation, converting modal imbalance into \"reliability weights\" that can be detected online and corrected in real time, thereby solving cross-modal imbalance without relying on source domain labels, but these mechanisms are not integrated into the training of compact hash representations. Our method focuses on the problem of low matching accuracy and failure to capture key information caused by information redundancy in cross-modal hash learning. We designed a dynamic attention masking module to adaptively filter key cross-modal semantics in the image-text fusion stage, significantly alleviating modal interference and feature redundancy; at the same time, an auxiliary hash layer is introduced and the hash compactness of semantically consistent samples and the separability of heterogeneous samples are enhanced through contrast loss, thereby improving the discriminability and robustness of cross-modal hash retrieval from the source. We will add the above comparison in the related work in the final version.\n\n3.Weakness: [The paper states that image and text features are fused via a Transformer, but omits the exact token construction, positional encoding, and cross-attention setup—making reproduction difficult.]\n\nResponse: Thank you for your comments. In our model, images are divided into fixed-size patches, and positional encoding and learnable class embedding are performed through ViT before being fed into the image transformer; text is encoded through BPE and embedded into a token sequence, and positional encoding is also added before being fed into the text transformer. Image and text tokens are processed by independent transformer encoders, and are not explicitly fused at the token level. The transformer structure consists of multiple layers of multi-head self-attention and feedforward MLP, each layer of which contains residual connections and normalization functions. We will add the explanation in the final version.\n\nFor Question:\n\n1.Question: [What are the actual training and inference runtimes (or FLOPs/parameter counts) of AHLR versus DCHMT and UCCH on MS-COCO?]\n\nResponse: Thank you for your comments. We report actual training and inference runtimes on the MS-COCO dataset under the same experimental setup (NVIDIA RTX 3080 GPU, 64-bit hash codes, batch size 64):\n\nUCCH trains in 46-51 seconds per epoch, with per-step forward-backward time of 22-28 ms. Its peak GPU memory usage is 3.6 GB, and inference takes only 2-3 seconds per epoch.\n\nDCHMT requires significantly more time, with training taking 18 minutes per epoch, and each forward-backward batch taking 230-240 ms. Its inference time for full validation (10,240 queries) is approximately 6 minutes, with a peak GPU memory usage of 13.7 GB.\n\nAHLR trains in approximately 10.5 minutes per epoch, with inference time of 5 minutes 45 seconds over 5,120 queries and 117,098 gallery items. Peak GPU memory usage is 6.87 GB, and GPU utilization reaches 64% during training.\n\n2.Question: [What are the advantages of the channel weight allocation strategy of the auxiliary hash layer compared to classical methods such as SENet?]\n\nResponse: Thank you for your comments. SENet uses global average pooling and then two fully connected layers to recalibrate channel importance, but it is mainly used for classification tasks and does not consider the unique semantics of cross-modal retrieval. The auxiliary hashing layer in our model is different from classic methods such as SENet in that it focuses on cross-modal hashing tasks. Instead of using a general channel recalibration, it calculates the channel weights of image and text features separately after dimensionality reduction using the auxiliary hashing layer, and uses a fully connected layer to calculate the importance weight of each channel, allowing the model to capture the nonlinear relationship between channels. Our strategy can emphasize information channels, thereby enhancing discriminative hashing learning.\n\n3.Question: [ Equation (5) imposes a causal-style mask. How does this pattern select “key” tokens? Would a learnable top-k pruning score be superior?]\n\nResponse: Thank you for your comments. We construct a causal attention masking matrix (as shown in Equation (5)) to limit the attention scope of the token at each position in the Transformer, that is, each token can only focus on the current position and its previous position. Although the masking matrix is determined by the length of the input sequence and does not directly contain semantic information, during training, the model can model global and local information through a multi-head attention mechanism and a feedforward network, thereby gradually learning key information under masking restrictions. Regarding alternatives to learnable Top-k pruning scores, we acknowledge that this mechanism can provide a more flexible and explicit way to select semantically important tokens. However, it generally increases computational complexity and may introduce instability during training[1]. Since it is necessary to learn and optimize additional gating variables (Hard-Concrete) and add additional parameters for training, which increases the training time and backpropagation complexity. \n\nREFERENCES\n\n[1]Wang, Tiannan, et al. \"Efficientvlm: Fast and accurate vision-language models via knowledge distillation and modal-adaptive pruning.\" arXiv preprint arXiv:2210.07795 (2022).\n\n4.Question: [ Could the dynamic mask be reused for other multimodal tasks (classification, generation)? What modifications would be required?]\n\nResponse: Thank you for your comments. The dynamic masking mechanism we proposed aims to adaptively focus on key information by generating a mask matrix based on the input sequence structure. This mechanism can indeed be extended to other multimodal tasks, such as classification and generation. For classification tasks, dynamic masking can help the model selectively focus on the most relevant regions or markers in different modalities, and subtle modifications may include adjusting the mask generation criteria to adapt to the input format. For generation tasks, models usually need to follow the sequential generation principle of autoregression, so dynamic masking should continue to maintain or strengthen the restrictions of causal masks to ensure the correct sequential dependencies and information flow during generation. In summary, although dynamic masking can be used for other multimodal tasks, it needs to be adjusted according to the specific task, especially the mask generation logic needs to be modified according to the structure and length of different input modalities."}, {"review_id": "lrvZ6Eq1Yt", "reviewer": "Reviewer_ArmY", "summary": "This article proposes a cross-modal retrieval method called AHLR, which automatically filters and weights key information in images and texts through the dynamic mask mechanism, and combines the auxiliary hash layer to adaptively balance the feature weights of each channel, effectively solving the shortcomings of traditional methods in capturing key information and channel processing, and significantly improving retrieval accuracy. The experimental results show that the AHLR method performs well on multiple benchmark datasets, providing a new solution for cross-modal retrieval.", "questions": "1. Has a differentiated masking strategy been used for sequence length differences in different modalities, such as images and text?\n2. Is there a detailed design for the network structure of the cross-modal fusion module, such as the number of layers and attention heads? \n3.How to ensure semantic alignment of different modal features during the fusion process?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. A novel method of dynamic masking and auxiliary hash learning (AHLR) has been proposed, which effectively solves the problems of channel information imbalance and insufficient capture of key information by jointly utilizing dynamic masking and auxiliary hash learning mechanisms.\n2. By constructing an auxiliary hash learning layer, the problem of traditional hash layers ignoring importance differences when processing different channel features has been solved, enhancing the ability to capture key information and effectively suppressing noise and redundant information.\n3. The experiment was validated on three benchmark datasets, NUS-WIDE, MIRFlickr25K, and MS-COCO, demonstrating the performance of the method.\n\nWeaknesses:\n1. There is a typo in the scale of the NUS-WIDE dataset in Table 3, which needs to be corrected.\n2. The depth of performance experiment analysis needs to be strengthened, and the reasons for obtaining relevant experimental results need to be analyzed.\n3. In practical applications, if there are problems such as noise, missing values, or inaccurate labeling in the input data, it may affect the performance of the model and require additional data cleaning and preprocessing work to ensure data quality.", "was_revised": true, "final_justification": "The responses addressed all my previous questions and concerns. I suggest acceptance.", "rebuttal": "For Weaknesses:\n\n1.Weakness: [There is a typo in the scale of the NUS-WIDE dataset in Table 3, which needs to be corrected.]\n\nResponse: Thank you for your comments. We sincerely apologize for the typographical error regarding the dataset sizes in Table 3. We have revised the table accordingly in the updated manuscript. This correction does not affect the results or conclusions reported in the paper.\n\n2.Weakness: [The depth of performance experiment analysis needs to be strengthened, and the reasons for obtaining relevant experimental results need to be analyzed.]\n\nResponse: Thank you for your comments. We agree that a more in-depth analysis of the experimental results would help improve the paper. We will add in Section 3.1: Compared with the baseline, our method has a mAP that is more than 10% higher than the lowest experimental method on the three datasets, which is mainly attributed to the auxiliary hash learning module, which helps the model generate hash codes rich in semantic clues, thereby enhancing its generalization ability. On larger datasets, the performance difference between different hash code lengths (16/32/64 bits) gradually decreases. This trend suggests that the dynamic mask mechanism enables the model to better capture key areas and suppress irrelevant or noisy features, resulting in more stable retrieval results. We will add more experiment analysis in the final version.\n\n3.Weakness: [In practical applications, if there are problems such as noise, missing values, or inaccurate labeling in the input data, it may affect the performance of the model and require additional data cleaning and preprocessing work to ensure data quality.]\n\nResponse: Thank you for your comments. In practical applications, there may indeed be noise, missing data, or inaccurate labels in the input data. To address this issue, we perform standard preprocessing steps before feeding the dataset into the model, which alleviates these problems to a certain extent. In addition, the dynamic attention masking mechanism we proposed can adaptively filter out features with less information or more noise, thereby providing a certain degree of robustness to noisy and incomplete data. This design further mitigates the negative impact that may be caused by data quality issues.\n\nFor Question:\n\n1.Question: [Has a differentiated masking strategy been used for sequence length differences in different modalities, such as images and text?]\n\nResponse: Thank you for your comments. In our approach, we do consider the sequence length differences of different modalities when applying dynamic attention masks. Our dynamic attention mask sets the corresponding mask matrix M according to the length of the image and text sequences, and the mask matrix is adaptively set according to the sequence length after encoding and position embedding. That is, the mask strategy is dynamically applied according to the input length of each modality, thereby achieving differentiated attention control.\n\n2.Question: [Is there a detailed design for the network structure of the cross-modal fusion module, such as the number of layers and attention heads? ]\n\nResponse: Thank you for your comments. When we further use Transformer to fuse and process image and text features respectively, our network architecture is designed in detail. We use stacked Transformer layers and set the number of layers to 12. We use multi-head self-attention, and the self-attention projection dimension is 64, that is, the query, key, and value vector dimensions of each attention head are all 64. \n\n3.Question: [How to ensure semantic alignment of different modal features during the fusion process?]\n\nResponse: Thank you for your comments. We use Transformer to encode images and texts separately to obtain feature representations, and use the learned linear projection matrix to map the two modal features to a shared embedding space of the same dimension, ensuring that image and text features are compared and fused at the same semantic scale. The Transformer module uses a dynamic attention masking mechanism that can adaptively adjust attention calculations for input lengths of different modalities, improve the ability to focus on key semantic information during feature extraction, and avoid interference from irrelevant information. At the same time, before calculating the similarity, we normalize the feature vectors of both the image and text to reduce the difference in feature scales between different modalities, so that cosine similarity can more accurately reflect the semantic relevance between the two. These designs together ensure the semantic alignment of different modalities during the fusion process."}, {"review_id": "Nur8I8O0hU", "reviewer": "Reviewer_pwot", "summary": "This paper proposed a novel hash learning method for hash learning enhanced cross-modal retrieval. The main challenge in hash learning based cross-model retrieval is the mismatch problem among different modalities, thus leading to wrong retrieval. The proposed method contains two components: Dynamic Masking (DM) and Auxiliary Hash Learning (AHLR). The experimental results show that DM and AHLP could successfully work for hash learning enhanced cross-modal retrieval.", "questions": "1. Regarding the DM policy, can the mask cause a negative influence, such as slow convergence speed?\n\n2. Can the author report the computation cost for the proposed method? The motivation for this question is that hash learning suffers from the convergent problem. This paper introduces an additional loss. Thus, I guess the proposed method needs extra training epochs, hindering it from being implemented in real tasks.", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The overall method is sound.\n\n2. The proposed method achieves the SOTA results.\n\n3. The writing is clear and easy to follow.\n\nWeaknesses:\n\n1. Some necessary details about the DM strategy are missing. Based on the ablation study, DM is important for the overall method and even important than AHLR. However, this paper only describes the DM strategy with a few contents in Sec. 2.1. The author should discuss more about the DM such as the rate that should be masked for image features, which may further enhance the contribution of this paper.\n\n\nTo sum up, I think this paper is sound and well-written. The proposed method solves a challenge in hash learning. Meanwhile, i did not find any technical flaws. Thus, i rate it as accept.", "was_revised": true, "final_justification": "This paper proposes a novel hash learning method for enhanced cross-modal retrieval. The initial concerns relate to the potential negative impact of the dynamic mask. After the rebuttal, the author clarifies these concerns. I believe this paper meets the standards of NIPS and has the potential to have a broad impact on hash learning. Therefore, I rate it as strong accept.", "rebuttal": "For Weaknesses:\n\n1.Weakness:[Some necessary details about the DM strategy are missing. Based on the ablation study, DM is important for the overall method and even important than AHLR. However, this paper only describes the DM strategy with a few contents in Sec. 2.1. The author should discuss more about the DM such as the rate that should be masked for image features, which may further enhance the contribution of this paper.]\n\nResponse: Thank you for your comments. We agree that the dynamic masking strategy plays a crucial role in our framework and should be described more clearly and in depth. We will add in the final manuscript how the dynamic masking mechanism determines the selected information as key information, i.e., add the following content. The dynamic masking mechanism guides the model to focus on the key information in the sequence by generating a mask matrix based on the input structure. Specifically, we construct a causal attention mask matrix to limit the attention range of tokens at each position in the Transformer, that is, each token can only focus on the current position and its previous position. Although the mask matrix is determined by the length of the input sequence and does not directly contain semantic information, during the training process, the model can model global and local information through the multi-head attention mechanism and feed forward network, thereby gradually learning key information under the mask restriction.  \n\nFor Question:\n\n1.Question: [Regarding the DM policy, can the mask cause a negative influence, such as slow convergence speed?]\n\nResponse: Thank you for your comments. Our dynamic masking mechanism is designed to help the model select key information. It selectively \"masks\" parts of image or text features during training, thereby reducing the information available in each forward propagation, which can lead to slower convergence. This is because the mask restricts the flow of information in the attention mechanism and may initially make optimization more difficult. However, by carefully adjusting the mask rate and training schedule, we found that the benefits of using a dynamic attention mask mechanism outweigh the potential disadvantages. In our experiments, models using DM perform better than variants that do not use masks.\n\n2.Question: [Can the author report the computation cost for the proposed method? The motivation for this question is that hash learning suffers from the convergent problem. This paper introduces an additional loss. Thus, I guess the proposed method needs extra training epochs, hindering it from being implemented in real tasks.]\n\nResponse: Thank you for your comments. The training process is on an NVIDIA RTX 3080 GPU with 64-bit hash codes and batch size 64. Our method takes about 5.75 minutes per epoch for training and 2 minutes 38 seconds for inference on MIRFLICKR, about 10.5 minutes per epoch for training and 5 minutes 45 seconds for inference on MS-COCO, and about 11 minutes per epoch for training and 6 minutes for inference on NUS-WIDE, with peak GPU memory usage around 6.9 GB and GPU utilization between 64% and 77%. Despite introducing an additional loss, our proposed model does not require a significant increase in the number of epochs to converge. We will add the computation cost in the final version."}]

Author rebuttal:

Meta-review: This paper presents a novel cross-modal retrieval framework combining dynamic masking and auxiliary hash learning to address key challenges in hash code generation. The method effectively tackles channel information imbalance and insufficient key feature extraction through innovative mechanisms that dynamically filter and weight critical information across modalities. All four reviewers recognized the paper's technical foundation and meaningful contribution to cross-modal retrieval. The authors' thorough rebuttal clarified implementation specifics, provided detailed computational analysis, and demonstrated consistent superiority across multiple benchmarks.
Decision: Accept (poster)