SHE: Streaming-media Hashing Retrieval
Authors: Ruitao Pu
Yang Qin
Xiaomin Song
Dezhong Peng
Zhenwen Ren
Yuan Sun
Venue: ICML 2025 poster (2025)
Keywords: cross-modal retrieval
cross-modal hashing
streaming-media
Primary area: general_machine_learning->representation_learning
TL;DR: This paper reveals and studies a practical but less-touched problem in cross-modal hashing, i.e., streaming-media hashing retrieval.

Abstract: Recently, numerous cross-modal hashing (CMH) methods have been proposed, yielding remarkable progress. As a static learning paradigm, existing CMH methods often implicitly assume that all modalities are prepared before processing. However, in practice applications (such as multi-modal medical diagnosis), it is very challenging to collect paired multi-modal data simultaneously. Specifically, they are collected chronologically, forming streaming-media data (SMA). 
To handle this, all previous CMH methods require retraining on data from all modalities, which inevitably limits the scalability and flexibility of the model. In this paper, we propose a novel CMH paradigm named Streaming-media Hashing rEtrieval (SHE) that enables parallel training of each modality. Specifically, we first propose a knowledge library mining module (KLM) that extracts a prototype knowledge library for each modality, thereby revealing the commonality distribution of the instances from each modality. Then, we propose a knowledge library transfer module (KLT) that updates and aligns the new knowledge by utilizing the historical knowledge library, ensuring semantic consistency. Finally, to enhance intra-class semantic relevance and inter-class semantic disparity, we develop a discriminative hashing learning module (DHL). 
Comprehensive experiments on four benchmark datasets demonstrate the superiority of our SHE compared to 14 competitors.

SHE: Streaming-media Hashing Retrieval
Ruitao Pu'
Yang Qin
Xiaomin Song?
Dezhong Peng
123 Zhenwen Ren+ Yuan Sun
Abstract
1. Introduction
Recently, numerous cross-modal hashing (CMH) methods have been proposed, yielding remark- able progress. As static learning paradigm, existing CMH methods often implicitly assume that all modalities are prepared before processing. However, in practical applications (such as multi- modal medical diagnosis), it is very challenging to collect paired multi-modal data simultaneously. Specifically, they are collected chronologically, forming streaming-media data (SMA). To handle this, all previous CMH methods require retrain- ing on data from all modalities, which inevitably limits the scalability and flexibility of the model. In this paper, we propose a novel CMH paradigm named Streaming- that enables parallel training of each modality. Specifically, we first propose knowledge library mining module (KLM) that extracts prototype knowledge library for each modality, thereby re- vealing the commonality distribution of the in- stances from each modality. Then, we propose a knowledge library transfer module (KLT) that up- dates and aligns the new knowledge by utilizing the historical knowledge library, ensuring seman- tic consistency. Finally, to enhance intra-class semantic relevance and inter-class semantic dis- parity, we develop a discriminative hashing Iearn- ing module (DHL). Comprehensive experiments on four benchmark datasets demonstrate the su- periority of our SHE compared to 14 competitors.
Recently, with the rapid development of smart devices and social media, multimedia data (such as images, audio, video, etc) has shown explosive growth trend. Cross-modal retrieval (CMR) for massive and complex multimedia data has attracted widespread attention in academia and industry (Zhu et al., 2023; Liang et al., 2024). CMR aims to use data from one modality as the query to retrieve semanti- cally related samples in another modality. However, due to different feature distributions, there is an inherent hetero- geneity gap between different modalities. How to establish effective cross-modal mapping relationships achieve ac- curate semantic retrieval between different modalities is a key challenge.
To bridge the heterogeneity gap, numerous CMR methods (Sun et al., 2024; Li al., 2025b; Liu et al., 2024)have been proposed to capture the shared semantic information between different modalities, which could be roughly di- vided into unsupervised (Hu al., 2022; Li et al., 2025a; 2024) and supervised methods (Wang et al., 2024a;b; Pu et al., 2025b). Among them, unsupervised CMR methods aim to directly explore the semantic similarity relationships between multi-modal data according to the original features. However, due to the lack of available semantic labels, un- supervised methods suffer from a performance bottleneck. Thus, many supervised CMR methods proposed that utilize label information to guide the learning of semantic similarity relationships between instances, thereby obtaining the discriminative common representations. Most of these methods employ a joint learning manner to learn common representations for multi-modal data. In other words, they necessitate that data from all modalities be available simulta- neously and collaboratively incorporated to learn a common representation. Although these methods have demonstrated promising performance, their success heavily relies on the implicit assumption of complete data availability. In practi- cal application scenarios, it is difficult to collect data from modalities simultaneously, such as emergency medical aid. Due to the asynchronous characteristics of multi-modal data collection, these data from different modalities of- ten continuously collected and processed at different time points, thus forming greaming-meddin data (SMA). When dealing with SMA, the aforementioned methods must fre- quently retrain all the sub- networks for all modalities, which
Hashing rEtrieval (SHE)
'School of Computer Science, Sichuan University, Chengdu, China "Sihhuan National Innovation New Vision UHD Video Tech- nology Co., Ltd., Chengdu. China 'Tianfu Jincheng Laboratory, Chengdu, China "Southwest University of Science and Technol- ogy, Mianyang, China National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University, Chengdu, China. Correspondence Yuan Sun <sunyuan, work @ 163.com>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025, Copyright 2025 by the author(s).
https:/ //gi thub. com www..//////guu.. site/

---

SHE Streaming-media Hashing Retrieval
Diseriminative hashing learning
Knowledge library mining
Knowledge library transfer
Labels Y
||+
Hash codes 1 cB ① 4
Bound
Modality
Backbone
Labels 3/2
Image
A rabbit is standing on the lown.
Hash codes 1 !
Modality
Backbone
Similarity space
Labels V'r
Text
Hash codes 1 0
Modality
Backbone
3D
Knowledge library
Hash codes
Prototypes
+ Intra-class prototype Inter-class prototype
Figure 1. The framework of our SHE. Hash codes from different modalities are represented blue, green, and yellow. Circles and rectangles represent different categories, while pentagrams denote class prototypes. Specifically, KLM extracts essential class prototypes. thereby explicitly constructing an implicit knowledge library for each modality. KLT aligns the knowledge library from the newly arrived modality with the historical knowledge library maintain semantic consistency across modalities. DHLL ensures the discriminability and compactness of hash codes by maximizing semantic similarity within the same class and semantic disparity across different classes.
undoubtedly increases the learning cost and limits the scala- bility and flexibility of CMR.
nificantly reducing their effectiveness.
To address the above challenges, we propose a novel Streaming-media Hashing rEtrieval (SHE) method to achieve asynchronous retrieval for streaming-media data. Different from existing static CMH methods that require all modalities be prepared before processing, proposed SHE can learn hash codes of each modality in parallel to process streaming- media data without retraining all histori- cal modalities, thus reducing the training complexity. The framework of our SHE shown in Fig.1, which consists of a Knowledge Library Mining module (KIM)), Knowl- edge Library Transfer module (KLT), and a Discriminative Hashing Learning module (DHL). Specifically, first pro- pose the KLM module to extract essential class prototypes, thereby explicitly constructing an implicit knowledge library for each modality. Then, we propose the KLT module to align the knowledge of the newly arrived modality with the historical knowledge library, thereby preserving semantic consistency across modalities. Finally, we present DHL to learn compact and discriminative hash codes by maximiz- ing the semantic relevance of intra-class samples and the semantic disparity of inter-class samples. In general, the major contributions of this work are shown as follows:
To enhance the flexibility of processing SMA, the CMR community has introduced scalable paradigm (Wang & Peng. 2021; Pu al., 2025a), which independently trains data from each modality. When new modality data arrives, the paradigm only requires training the sub-network specific to the new modality without retraining all sub-networks. For instance, MARS (Wang & Peng, 2021) leverages the shared label parsing module to achieve semantic alignment without interaction across modalities, thereby providing the funda- mental conditions for independent training. Although these methods have achieved considerable performance, their low retrieval efficiency falls short of meeting the demands of large-scale data retrieval. Cross- modal hashing (CMH) has become an effective solution, due to its low storage require- ments and high efficiency. However, how to construct the CMH framework to deal with SMA still remains an open research problem. In fact, this learning framework for SMA has the following key challenges: 1) Independent training of each modality in the CMH framework could amplify quantization errors, thereby leading to significant semantic information loss. 2) The absence of cross moddl seman- tic interaction hinders the learning of common semantic knowledge from SMA, thereby potentially degrading the quality of the generated hash codes. 3) When streaming data becomes unpaired, existing CMH methods fail tablish cross-modal semantic correspondences, thereby sig-
This paper reveals and studies a practical but less-touched problem in cross-modal hashing, i.e., streaming- media hashing retrieval. To overcome this problem, we propose novel cross-modal hashing
2

---

SHE: Streaming mediaa Hashing Retrieval
(CMH) retrieval framework dubbed streaming- -eeiia hashing retrieval (SHE). To the best of our knowledge, this could be the first work that addresses the problem of streaming- media data using CMH paradigm.
the semantic multilevel advantages of the entire database and bridge the semantic and heterogeneity gaps between dif- ferent modalities, MIAN (Zhang al., 2022) explores asym- metric intra- and inter-modal similarity preservation under a probabilistic modality alignment framework. Unlike MIAN, which overlooks the relationship between the relative rank- ing of adjacent instances and the Sine-grained label-level similarity, DCH-SCR (Liu et al.. 2023) narrows the inherent gap between modalities by developing ranking alignment loss function and introduces Normalized Discounted Cumu- lative Gain (NDCG) to achieve varying optimization inten- sities for data pairs with different similarities. Despite their decent performance, insufficient consideration is given to the fact that there is a spatial gap between real-number space and Hamming space, which may lead to solution space com- pression and loss function oscillation, thereby degrading performance. For this issue, SCH (Hu et al., 2024) divides sample pairs into fully semantically similar, partially seman- tically similar, and semantically dissimilar categories based on their similarity, and applies different constraints to each category to ensure the utilization of the entire Hamming space. Additionally, SCH introduces semantic channels to alleviate the issue of loss function oscillation.
We propose a knowledge library mining module (KLM) propose and a knowledge library transfer (KLT) module that jointly extract an implicit knowledge library from the newcome data and align the commonality distribution from the new knowledge with ones from the historical knowledge library. This ensures that our SHE can process the newly arrived media data without retraining the whole historical data, thus reducing the training complexity.
Extensive experiments of streaming media retrieval on four widely used multi-modal datasets demonstrate the superiority and effectiveness of our proposed SHE compared with 14 state-of-the-art methods.
2. Related Work
Technically, cross-modal retrieval (CMR) aims at learn- ing a series of modality-speciffc sub-networks to project multimodal data into a common space, enabling the di- rect measurement of similarity between data from different modalities (Sun al.. 2023; Duan et al., 2025; Pu et al., 2025b). Among these, cross-modal hashing (CMH), as efficient and storage-friendly CMR technique, has garnered widespread attention from academia and industry. CMH methods are generally classified into unsupervised and su- pervised ones. Further, unsupervised ones (Qin et al., 2023; Liu et al., 2020; Shi et al., 2022) aim to capture the hidden semantic information by preserving the original structure of the multimodal data. For example, JDSH (Liu et al., 2020) constructs joint-modal similarity matrix to maintain the multimodal semantic correlations among instances while incorporating a sampling and weighting mechanism to en- courage the discriminative quality of hash codes. However, most existing unsupervised ones overly prioritize semantic consistency, overlooking the varying capacities of different hash functions to learn similarity. This imbalance often results in the hash function for one modality being weaker than those for others. To this end, DAEH (Shi et al., 2022) introduces an Information Mixed Similarity Estimation to estimate the similarity relations with discriminability while deploying an adaptive teacher guided enhancement opti- mization scheme to identify and strengthen weaker hash functions.
The aforementioned CMH methods are implemented on the assumption that all media are simultaneously accessible, which is often impractical in real-world scenarios. When facing streaming -ediaa data (SMD), i... the ever-emerging media, these methods must frequently retrain all modality- specific sub-networks, compromising the flexibility and scalability of CMR. In this regard, several real-valued CMR methods (Wang & Peng, 2021; Pu et al., 2025a) are proposed to allow for training only the newly added modality speecifc network when new modality data arrives, greatly facilitat- ing the flexibility of CMR. However, they are real-valued based representation methods, which involve high computa- tional complexity and memory overhead, making it difficult to meet the fast retrieval demands of large-scale datasets. CMH, technique celebrated for its high retrieval capabili- ties, presents a promising solution. However, how to design an effective CMH framework to deal with SMA remains an unresolved research gap. In this paper, we propose a novel CMH framework named streaming media hashing re- (SHE), which integrates both high retrieval efficiency and flexibility in handling SMD.
3. The Proposed Method
3.1. Problem Formulation
However, these methods still face performance bottlenecks due to the absence of semantic labels. In contrast, supervised methods (Qian et al., 2022; Qin et al., 2022; Wang et al., 2024c) enjoy a more considerable performance by leverag- ing semantic information in labels. For example, to leverage
Without loss of generality, given a multi-media dataset MI M
'D = {Xm.. Ym ) m= with M modalities, where Xm, and ym denote the data set and label set from the m-th modality respectively. Further, the data set and label set from the m-th modality are denoted as Xm, = { 2 2hh
3

---

SHE: Streaming-media Hashing Retrieval
and D' {um yem Dm respectively, where sm denotes the i-th sample from the m-th modality, V'n = 10 denotes the correspond- ing label, C' the number of categories, and n'm means the number of samples in X For a sample = if it belongs to the e-th category, w = 1, otherwise u/z = O. In our paper, the hash code of sample wi,, is denoted as byn {-', where L denotes the length of the binary code. Our main goal is to project M modalities into the Hamming space to generate hash codes so that the similarity between dif- ferent instances can be measured efficiently. To obtain dis- criminative hash codes, we first learn M modality-specifft sub-networks H = [Him Pm)) m=1 for multi-media in- put. Note here that H.n represents the sub-netword of the m-th modality and pm denotes its corresponding learnable parameters. Then, the binary code of each sample z'nn could be generated by the sign function, i.e.,
first stage, for the first available modality, we first employ KLM to extract abstract knowledge library and push hash codes close to the corresponding class prototype knowledge. Then, we adopt DHL to learn high-quality hash codes. Thus, the objective loss could be formulated as
{umn vy.. ..., 2hhhh...... E
Lm = Litm + aLtir + B.ahtt
(2)
Lkims Cktrs and Las represent the KLM loss, the KLR loss, and the DHL loss, respectively. In the second stage. for the new incoming media data, the knowledge library is updated by KLT to align the new knowledge with the historical knowledge library, thereby integrating the semantic information of the new modality into the knowledge library. Mathematically, the objective loss of the m-th modality (m 2 2) can be written as
where a and | are two
Um = Siin(Hmm
Pm)))
(1)
Lm = Litm + afLttr + LRte) + BLast-
(3)
Considering that the sign function cannot be used for gradient-based optimization due to its non-differentiabiliy, we replace the sign function with the tanh function to gen- erate binary-like codes during training.
Therefore, the overall objective loss can be summarized as follows:
c Lulm + aLCur + +Baaht,
if m 1
(4)
Culm + ai + Lute) + &Cand if m 2 2
3.2. Overview
3.3. Knowledge Library Mining
Different from traditional static learning methods that re- quire all modalities to be prepared before processing, our SHE method can handle streaming-media data while mul- timedia data is continuously collected. In this paper, we propose a new atreaming-medin hashing retrieval (SHE) method, which is specifically designed to handle such streaming data. Since class prototypes could capture the consistency and specific information, we use them as the abstract knowledge library to preserve the instance com- monality distribution of latent representations in streaming- media data. Moreover, such a library also alleviates the size pressure of historical multimedia data. Thus, the core of our SHE lies in extracting a prototype knowledge library that evolves over time. In general, SHE mainly consists of three modules, i.e., Knowledge Library Mining (KLM), Knowl- edge Library Transfer (KLT), and Discriminative Hashing Learning (DHL). To be specific, the KLM module constructs a knowledge library for every modality by extracting essen- tial prototype knowledge, thereby serving as a semantic consistency maintenance medium. The KLT module aims to adaptively align the knowledge library extracted from the newly arrived modality with the historical knowledge library, thereby capturing the semantic consistency of multi- ple modalities. The DHL module focuses on reinforcing the compactness of intra-class samples and the separability of inter-class samples.
Due to the absence of modality interaction, independently training modality-speciffc sub-networks for streaming media data is prone to semantic discrepancies. To this end, we uti- lize class prototypes to act as the abstract knowledge library, thereby preserving the semantic information of streaming data while alleviating the computational burden of main- taining historical data. When the data with new modality arrives, we hope to extract the latent semantic information and convert it into a knowledge library, thereby building a bridge to reduce the heterogeneity gap between historical data and new data. Thus, we propose Knowledge Library Mining (KLM) to extract the abstract essential knowledge, thereby constructing a knowledge library for each modality as the medium. Specifically, we first utilize class prototypes as the abstract knowledge library. Considering the vari- ability of intra-class samples, using only a single prototype for each class might result in a compromised representa- tional capacity, potentially leading to suboptimal semantic alignment across modalities. Therefore, we assign multiple (i.c., K) prototypes each class to better capture diverse semantic information, facilitating more effective consis- tency among intra-class samples. Further, we randomly initialize and construct a learnable abstract knowledge li- brary Pm. = {p P". pm" for modality m (m 2 1), where ps, = [pt.] pm? ........ph.. P.. E R K denotes K normalized and binary prototypes for the c-th class. Then, we encourage hash codes and their closest homo-category
With the arrival of streaming- media data, the training pipeline of our SHE mainly contains two stages. the
4

---

SHE: Streaming-medee Hashing Retrieval
prototype move closer each other. To this end, we denote the similarity between any sample zit from the m-th modality and its k-th corresponding prototype as follows,
3.6. Optimization
To learn the optimal hashing functions, we jointly minimize the above losses. Specifically, for the m-th modality, the overall loss function is defined follows:
s( Ik) Ee y [bmm pm)) +
(5)
Cuim + acCer + -Bahhi
if =
C
(10)
where -((---) -) is the cosine similarity function. Thus, the loss function could be formalized as follows:
Leam + aaCutr + LLu) + BLas if m 2 2
To offer a comprehensive overview of our SHE framework, we summarize the training process in Algorithm which outlines the optimization pipeline across modalities.
Ckim
1
Σ
Tly
log
max klI...,K
s (sialk))
(6)
Thyre
The above loss aims to encourage each training sample to be close to the nearest prototype with the same category. How- ever, this KLM loss could lead to a trivial solution where all prototypes collapse into one point, thus weakening the discriminative ability of hash codes. To this end, we develop Knowledge Library Regularization (KLR) mechanism to enhance the distinctiveness between inter-class prototypes while ensuring semantic consistency between intra-class prototypes. The KLR loss can be written as:
4. Experiments
4.1. Dataset
In our experiment, we evaluate the proposed SHE on four widely used multimedia datasets, namely, Wikipedia (Rasi- wasia et al., 2010), OUS-WIDE (Chua et al., 2009), XMedia (Peng al., 2015), and XMediaNet (Peng al., 2018). In Tab.1, we summarize the statistics of the datasets. Notably, to align with the streaming- media scenario, we gradually incorporate the modalities into the training process in the order they are collected within the datasets. More details about the four datasets are presented in the appendix.
ER=INE Flonh pskk) log Ee Ck= el Torint peitt
Ler
CK
3.4. Knowledge Library Transfer
Since streaming media data is constantly changing and up- dating, the model needs to have good generalization ability to adapt to such changes. To this end, we propose Knowl- edge Library Transfer (KLT) to extract knowledge from new media data and transfer its semantic information to the benchmark knowledge library. This allows our SHE to pro- cess new media without retraining the entire historical media data, thereby reducing computational complexity while en- suring semantic consistency between streaming modal data. Therefore, for the modality m (m 2 2), the KLT loss is defined as follows, i.e.,
Table f. General statistics of the four datasets, where "*pt/*" in the "Instance" column indicates the number of samples for training, testing, and retrieval database, respectively.
Dataset
Modality
Instance
Feature
Image
2,173/693/2.866
4,096d VGG
Wikipedia
Text
2,173/693/2,88
300d Vec
Image
8,000/2,000/10.000
4,096d VGG
NUS-WIDE
Text
8, 8,000/2,000/10,00
300d Dec2
Image
1.0000/1,00055000
4,096d VGG
DAi = CK Σ log [min(1, moz(0, "P a +1)]
Text
4,000/1,000/5.0000
3,000d Bow
(8)
XMedia
Audio clip
800/200/1,000
29d MFCC
where a is the similarity boundary.
3D
100/100/5(0)
4,700d LightField
Video
969/174/1,145
4,096d C3D
3.5. Discriminative Hashing Learning
Image
32,000/8,000/40.0
4,096d VGG
Text
92,,000/80000/44.000
300d Doc2Vec
To strengthen the compactness of intra-class samples and the scatter of inter-class samples, we employ a Discrimi- native Hashing Learning (DHL) module constrain the learning of all sub-networks. Specifically, DHL strives to maximize the similarity of hash codes within the same class while minimizing the similarity of hash codes from different classes. For modality m (m 2 1), the DHI. loss could be expressed as the following formula, i.c.,
XMediaNet Audio clip 8,000//2,000/0,.000
672d MFCC
3D
1,600/440/2,000
4,700d LightField
Video
8,000/2,000/10.000
4,096d C3D
4.2. Experimental Settings and Evaluation Metric
In our SHE, all modality-speciff sub-networks com- prise three fully connected layers, with the ReLU acti- vation function applied after the first two layers and fz-normalization operation applied to the last layer. Their dimensions are [d" + 4096 + 4096 + L). where d' rep- resents the input feature dimensions of the corresponding modality. For all datasets, we set the batch size n, as 256, the similarity boundary a as 0.95, the iteration number T
1 Thm
Σn I (ln) Σnm T(M Bm))
List
log
(9)
where Gi is an indicator that measures the similarity be- tween two samples, and its value is if zf and wi, are intra-class samples, otherwise O.
5

---

SHE: Streaming-media Hashing Retrieval
Algorithm 1 The training process of our SHE
and all experiments are conducted on single GeForce RTX3090TT 24GB GPU.
Multi-modal data D = [mm: ) m= a, 8, the number of prototypes K, the similarity boundary o, sub- networks 7/- {7mm( (++ Pm ] m- hash length L, batch size nb. learning rate 4,, and iteration number T.
Input:
Following previous works (Wang & Peng. 2021; Pu et al., 2025a). we evaluate the performance of SHE by perform- ing cross-modal retrieval (CMR) tasks, i.e., retrieving one modality using queries from another modality, For instance, we designate images as queries to retrieve text (i.e., 12T) and text as queries to retrieve images(i.e., T2I). Besides, the Mean Average Precision (MAP) score, widely adopted metric in the CMR community, is employed as the evalua- tion metric in our experiment since it simultancously reflects both the precision and recall of the retrieved results.
// The first stage (m =
Randomly generate a learnable abstract knowledge li- brary P1 {pl, Pi. Pi)
2: Randomly initialize 1
3: for iler ..., to T do
4: for step =
to
m n6
do
5: Get mini-batch Jn) from (21 31)-
6: Compute hash codes for Xi by the sub-network H. (H1(-. P1)}.
4.3. Comparison methods
7: Update the knowledge library P1 by tanh function and normalization.
To demonstrate the superiority of the proposed SHE, we compare it against 14 state-of-the-art (SOTA) methods, in- cluding three real-valued methods (i.e., MARS (Wang & Peng, 2021), GNN4CMR (Qian et al., 2022), and HOPE (Zhang et al., 2024) ), three unsupervised hashing methods (i.e., DGCPN (Yu et al., 2021), CIRH (Zhu et al., 2022), and UCCH (Hu et al., 2022)), and eight supervised hashing methods (i.e., HMAH (Tan et al., 2022). CMMQ (Yang et al., 2022), MIAN (Zhang et al.. 2023), DSPH (Huo et al., 2023), DHRL (Shu et al., 2024), DHaPH (Huo et al., 2024), DNpH (Qin et al., 2024), and SCH (Hu et al., 2024)). All methods except MARS and our SHE are limited training paired data from two modalities, and refer these methods as paired-oriente methods in the section. When paired-oriente methods face datasets with five modalities, we first construct pseudo-instance pairs through label-based repeat sampling and then conduct training on every two modalities, resulting in a total of x 4/2 training processes. For the Wikipedia and NUS- WIDE datasets, we only com- pare hashing methods. For the XMedia and XMediaNet datasets, compare both hashing methods and real-valued methods.
8:
Compute the loss L according to Eq.10.
9:
Compute gradients of 1 and P1 and update them:
i = 11 - 4 Oc
801
DC P1 Pi 4r OP1
10: end for
11: end for
// The second stage (m 2 2)
1: Use the mined knowledge library CK₁ to steer the semantic alignment for subsequent modalities.
2: for m = 2,,,..., to M do
3: Randomly generate a learnable abstract knowledge library Pm { P .i..... p.].
4: Randomly initialize 4 m-
5: for iter 1, to T do
6: for slep = 1, ..., to Tum do
7: Get a mini-batch (Am, m from (Ni Vn)) Compute hash codes for X1 by the sub-network = (H.m (:, {.)).
8:
9:
Update the knowledge library Pm by tanh func- tion and normalization.
10:
Compute the loss L according Eq.
4.4. Comparisons with State-of-the-Art Methods
11:
Compute gradients of Φ. and P.,. and update them:
Quantitative Comparisons: To demonstrate the superiority of our SHE, we compare it with SOTA methods in terms of MAP scores. Tab.2 reports the comparison between our SHE and SOTA methods on the Wikipedia and NUS-WIDE datasets with different bit lengths (i.c., 16, 32, 64, and 128). For the XMedia and XMediaNet datasets, we only present experimental results with 128 bits in Tab.3 and Tab.4, and a detailed analysis of the impact of bit length on performance is provided in the appendix. From these results, it can be concluded: 1) Due to the lack of label supervision, unsuper- vised methods suffer from a compromise in performance, compared with supervised ones. 2) The XMediaNet dataset, with more categories (200), presents increased learning diffi- culty, leading to a significant performance decline across all
Φ m = 1 4 ac
ac Pm = Pm - 1 OP..
12:
end for
13: end for
14: end for
Output: The optimized parameters {pm m==1'
as 300, the number of prototype vectors K as 3, and the hyperparameter a as 1. For four datasets, we set the hyper- parameter 8 as 4, 5, 1, and 6, respectively. Additionally, the SHE framework is implemented by the PyTorch toolkit,
6

---

SHEE: Streaming-medie Hashing Retrieval
Table 2. The MAP scores with different bit lengths on the Wikipedia and NUS- WIDE datascts.
Wikipedia
Method
Ref.
12T
T21
12T
T2I
16
32
64
128 Avg
16
32
64
128 Avg
16
32
64
128 Avg
16
32
64
128
Avg
DGCPN
21
21.6
23.8
29.0
29.9 26.1
21.7
24.7
30.5 30.4 26.8 50.9 50.8 51.8
52,3 51.5
48.7
49.3
50.3
51.1 49,9
CIRH
TKDE'22
29.1
31.0 29.9 25.0 28.8 27.8 31.3 28.5 24.0 27.9 41.8 45.4 46.0 46.6 45.0
43.0
47.1
47.4
47.9 46.4
UCCH
23
25.7
29.3 30.9 32.6 29.6
24.7 28.9 30.3 33.2 29.3 42.9 44,9 48.2 49.4 46.4
42.2
44.0
47.8
48.5 45,6
HMAH
TMM'22
51.9 51.7 51.2 51.3
69.9 74.0 75.7 76.2 74.0 59.7 61.0 61.9 61.9 61.1 64.4
66.7
67.8
67.8 66.7
CMMQ
CVPR'22
55.8
55.6 56.5 56.2 56.0
66.5 72.6 75.7 76.0 72.7 58.3 59.2 59.5 59.5 59.1 63.3
66.0 66.1 65.2 65.2
MIAN
TKDE'23
46.1
43.7 45,6 47.4
38.2 38.6 39.6 43.5 40.0 60.3 62.8 62.1 62.8 62.0
47.9
52.6 54.3 56.5 52,8
DSPH
23
45.3
46.2 48.2 48.7 47.1 52.0 59.4 59.1 57.0 56.9 56.1 57.7 58.7 58.2 57.7 63.0
64.5 65.1 65.8 64.6
DHRL
TBD'24
46.8 48.2 50.7 51.5 49.3 46.9 47.1 52.3 51.6 49.5 55.1 64.6 60.5 64.3 61.
54.8
61.2 61.4 62.2 59.9
DHaPH
TKDE'24 24
45.7
48.6 51.0 51.7 49.3
59.7 65.9 61.6 61.7 56.6 58.8 60.3 60.2 59.0
63.0
63.3 64.6 64. 63.8
DNpH
TMM 24
46.8
48.2 50,6 50.9 49.1 62.1 58.4 64.2 56.7 60.4 55.9 58.4 59.2 59.6 58.3
63.5
65.5 65.3 65.7 65.0
SCH
TPAMI
51.0
55.4 57.1 56.3 55.0
67.1 74.0 76.4 77.2 73.7 60.1 60.3 62.2 61.6 61.1
64.6
65.3
66.6
63.9 65.6
SHE
Ours
56.3 58.6 59.5 59.4 58.5
71.7 74.6 77.7 77.9 75.5 67.0 68.0 70.7 69.8 68.9
67.2
65.9 69.4
69.2 67.9
Table The MAP scores with 128 bits on the XMedia dataset.
XMedia
Methods
Ref.
Image
Text Audio
Audio
3D
Video
Avg
Text
Audio
3D
Video
Image
Video
Image
Text
3D
Video
Image
Text
Audio
Video
Image
Text
Audio
3D
DGCPN
21
57.0
30.9
10.1
80.0
45.8
28.2
7.1
35.1
24.5
32.3
6.0
6.7
6.2
5.7
5.7
56.3
30.6
5.6
6.1
24.3
CIRH
TKDE'22
85.9
11.6
51.3
74.4
86.3
8.4
35.7
52.2
11.0
8.2
9.8
11.6
44.5
35.8
10.0
29.0
50.7
42.3
8.4
24.0
34.6
UCCH
TPAM1'23
84.0
31.0
29.3
75.0
88.4
22.8
9.9
56.4
27.0
23.8
12.2
19.0
14.8
10.1
6.8
4.9
52.8
43.6
14.6
7.3
31.7
HMAH
TMM'22
91.4
53.9
87.3
86.5
95.8
57.6
92.0
91.4
54.2
52.4
40.4
29.1
61.6
63.8
30.1
57.4
57.9
56.0
19.1
52.6
61.6
CMMQ
CVPR'22
91.4
26.4
87.5
83,9
94.9
9.8
31.2
30.8
29.5
10.7
12.1
8.5
69.1
27.1
7.6
34.0
47.6
19.0
6.4
23.4
23.4
MIAN
TKDE'23
85.2
29.0
80.2
76.8
88.0
6.5
9.7
9.5
24.5
5.2
8.4
9.2
56.7
5.2
5.8
5.8
51.8
5.3
5.8
6.1
28.7
DSPH
TCSVT'23
91.7
41.7
24.5
86.0
95.8
41.4
20.1
90.8
50.6
47.8
8.9
21.4
14.9
10.3
7.0
6.4
56.4
57.0
14.6
11.1
39.9
DHRL.
TBD'24
90.1
74.9
81.6
84.3
94.4
81.6
86.5
84,1
60.0
61.9
45.6
37.0
61.8
63.5
43.3
45.1
56.9
51.7
29.0
38,9
63.6
DHaPH
TKDE' 24
91.9
45.2
48.5
86.3
96.2
44.7
40.6
91.4
52.3
48.5
11.7
33.7
43.8
35.3
8.1
19.8
60.2
59.6
17.4
47.6
DNPH
TMM
91.8
36.9
36.8
85,4
96.2
40.0
33.6
21.5
47.4
46.2
12.1
27.8
43.2
37.3
10.6
10.2
58.0
59.8
16.2
15.2
44,8
SCH
T4
90.1
85.9
80.5
87.2
94.4
62.2
52.6
88.3
64.4
45.2
12.8
28.7
66.0
44.8
15.3
8.9
59.7
54.9
20.0
9.6
53.6
MARS
TC
91,4
84.3
88.3
85.5
92.9
85.8
90.2
87.3
63.5
64.3
61.4
57.0
70.8
71.2
65.2
65.6
55.8
55.5
50.3
53.3
72.0
GNN4CMR
2P
90.9
40.7
86.8
84.9
92.2
36.4
87.2
87.6
46.8
44.9
40.0
37.9
62.6
62.8
26.0
57.0
54.3
55.5
19.5
47.8
58.1
HOPE
TPAMI
91.0
71.1
86.8
84,8
95.4
66.2
87.3
82.7
59.4
54.0
47.6
40.3
63.5
61.8
41.7
49.9
53.5
51.4
27.0
36.6
62.6
SHE
Ours
92.1
87.3
89.6
87.2
95.8
92.0
94.0
91.7
65.3
66.5
64.8
62.3
20.2
70.4
67.7
66.4
60.7
61.3
57.2
58.4
75.0
Table 4. The MAP scores with 128 bits on the XMediaNet dataset, where ' means out of memory. XMediaNet
Methods
Ref.
Image
Text
Audio
3D
Video
Avg
Text Audio 3D
Video Image Audio 3D Video Image
Text
3D
Video Image Text Audio Video Image Text Audio 3D
DGCPN
AAAI'21
CIRH
TKDE'22
1
1
1
1
1
1
1
1
1
1
1
1
1
/
1
1
1
1
/
/
UCCH
TPAMT'23
32.6
35.1
10.1
47.1
25.1
8.8
3.7
14.5
22.8
8.8
2.7
0.8
2.0
1.5
1.4
1.2
30.0
13.9
0.8
1.5
13.2
HMAH
T2M'22
44.2
59.6
70.2
72.0
42.5
5.1
29.6
19.1
33.1
7.3
19.4
0.6
37.6
24.4
12.7
31.6
39.4
18.4
0.6
34.5
30.T
CMMQ
CVPR'22
/
/
/
/
/
/
/
MIAN
TKDE'23
1
1
/
1
/
/
/
/
/
DSPH
TCSVT'23
18.3
23.8
14.1
52.9
33.4
4.3
4.1
12.4
26.7
3.1
1.5
0.6
3.3
1.1
0.7
1.5
35.0
6.9
0.9
1.7
12.3
DHRL
TBD'24
54.3
55.9
64.4
61.7
40.5
33.1
35.8
37.4
38.2
26.9
25.4
0.7
33.3
25.0
14.6
25.2
32.9
24.7
0.9
26.3
32.9
DHaPH
TKDE'24
24
29.0
27.3
32.3
60.9
42.0
5.1
6.6
22.4
27.2
3.6
2.2
0.7
18.8
2.3
1.0
5.6
39.3
12.3
0.6
4.2
17.2
DNPH
TMM'24
18.4
21.7
27.5
44.4
31.8
4.8
6.4
16.2
26.8
3.6
2.4
0.8
19.7
2.4
1.3
4.7
34.3
8.5
0.7
4.5
14.0
SCH
TPAMI
24
67.4
42.4
56.6
75.6
66.0
3.7
12.0
23.3
35.1
1.3
2.1
0.7
33.3
10.4
1.7
6.5
46.8
15.6
0.6
5.4
25.3
MARS
TCSVT
22
70.0
67.5
67.3
61.6
63.9
57.2
57.4
51.1
40.2
36.7
30.6
40.8
37.2
35.0
31.7
37.4
34.0
32.4
33.2
GNN4CMR TPAM 23
47.0
51.6
70.2
73.3
61.3
12.3
52.4
49.4
36.1
8.9
17.2
0.7
36.7
20.7
7.6
25.2
43.5
22.7
0.8
30.8
HOPE TPAMI24
51.8
58.3
67.5
68.2
54.0
43.8
56,2
57.0
45.0
39.2
0.6
38.7
35.2
18.7
21.1
40.1
35.9
1.1
23.8.39.
SHE Ours
75.1
70.6
70.3
72.8
70.5
62.8
63,5
65.8
42.2
39.8
36.7
38.2
41.2
39.3
34.0
37.7
43.1
41.0
36,4
51.0
methods. 3) Paired-oriented methods typically rely on true- instance pairs to bridge the cross-modal gap, which hinders
their performance on unpaired datasets (i.c., the XMedia and XMediaNet datasets), resulting in performance inferior
7

---

SHEE: Streaming- Hashing Retrieval
0.5
511
0.7
bamm
CRI
0.7
0.6
0.0
Proctiio
0.
PMAII
0.5
Procision
0.5
DWCPS
0.5
10061
0.4
1968
1861
0.4
Chaffi
0.3
0.3
D5P4
DAN
0.2
Diggh
0.2
Ch21
SC1
0.1
521
SC1
0.0
0.0
0.2
10
Recall
Rocall
(a) Wikipedia (12T)
(b) Wikipedia (T2I)
(c) NUS-WIDE (12T)
(d) SUS- (T2I)
Figure 2. Precision-recall curves with 128 bits on the Wikipedia and NUS- LUS-WIDE datasets.
to MARS and our SHE. 4) MARS simply deploys a shared label parsing module to guide the learning process across all modalities. comparison, our SHE outperforms them by mining knowledge libraries, regularizing knowledge li- braries, and aligning knowledge libraries for consistency. This holistic strategy greatly enhances the quality of the hash codes, resulting in performance that surpasses that of MARS.
DHL to enhance the discriminability of inter-class samples and the compactness of intra-class samples. In conclusion, the removal of any single component leads to performance degradation, highlighting the importance of each proposed component in addressing the cross-modal retrieval problem in streaming-media media scenarios.
Table 5. Ablation study with 128 bits.
Qualitative Comparisons: To further evaluate the retrieval performance of SHE, we compare it with SOTA methods in terms of precision-recall curves (PR-curves). F plots PR-curves with 128 bits the Wikipedia and NUS-WIDE datasets. As shown in the figure, our proposed SHE consis- tently achieves higher curves compared to SOTA methods, indicating superior precision at the same recall levels. This further confirms the superiority of SHE.
Wikipedia WEIDEEMMedii XMeiiXXMMMMMeaaaNNee
SHE w/o KLM
21.6
12.2
13.4
0.9
SHE w/o KLR
54.3
68.2
73.8
33.3
SHE w/o KLT
68.0
69.0
39.8
29.0
SHE w/o DHL SHE
56.7
64.3
38.2
31.4
68.6
69.5
75.0
51.0
4.6. Parameter Analysis
4.5. Ablation Study
To investigate the sensitivity of two parameters (i.e., a and 3), we conduct experiments with 128 bits on the XMedia and XMediaNet datasets. Specifically, we try one parame- ter with different values while keeping another parameter fixed to evaluate its individual impact on performance. As presented in Fig.3, the performance of our proposed SHE method on both datasets initially improves as the parameter value increases, and then either stabilizes or slightly de- clines. These results demonstrate that our method maintains decent performance across a relatively wide range of param- eter values. In general, the performance more stable when a is set between 4 and 8, and the best results are achieved when 3 is set to 1.
To verify the effectiveness of our components, we conduct an ablation study with 128 bits on four datasets and report the corresponding mean MAP scores of all cross-modal re- trieval tasks. The results are presented in Tab.5, where SHE w/o KLM represents the removal of KLM loss, SHE w/o KLR indicates the exclusion of KLR loss, SHE w/o KLT denotes the deletion of KLT loss, and SHE w/o DHL refers to the absence of DHL loss. From these results, we can obtain the following findings: 1) The absence of KI.M suf- fers a significant performance degradation, likely because KLM is capable of extracting essential prototype knowledge and building effective knowledge libraries. 2) The perfor- mance degradation without KLR could be attributed to the fact that KLR can enhance the discrimination of inter-class prototypes in the knowledge library, which is beneficial for learning high-quality hash codes. 3) The removal of KLT leads to a performance decline. This is likely due to the ability of KLTT to align knowledge libraries mined on different modalities, thereby preserving semantic consis- tency across modalities. 4) The exclusion of DHL similarly results in lower performance, likely due to the ability of
4.7. The Impact of Media Learning Sequence
To delve into the impact of media learning sequences with 128 bits on the XMedia and XMediaNet datasets, we se- quentially utilize the Image, Text, Audio, 3D, and Video modalities as the first available modality for knowledge li- brary mining, with the corresponding mean MAP scores of all cross- mross-modal retrieval tasks presented in Fig.4. The re- sults show that for the XMediaNet dataset, the performance
8

---

SHEE: Streaming-media Hashing Retrieval
identical prototype knowledge (i.e., o 1.0).
60-
4.9. The Impact of Knowledge Library Scale
MAP
MAP
40
50
The number of class prototypes K can influence the scale of the knowledge library, which may in turn impact the quality of the mined prototype knowledge. To investigate the impact of the knowledge library scale on performance, experiments with 128 bits on the XMedia dataset by varying K. As depicted in Fig.5(b), when K takes most values (i.e., 3, 5, and 9), the performance consistently surpasses that at 1. This observation underscores the advantage of assigning multiple prototypes each category, it can alleviate semantic information loss, strengthen semantic consistency, and ultimately enhance overall performance.
20
XMedia XMediaNet
40-
XMedia
XMediaNet
0
30
1
2 3
B
perform
(a) Different
(b) Different 3
Figure 3. Parameter analysis with 128 bits on the XMedia and XMediaNet datasets.
remains relatively stable regardless of the choice of the first available modality. However, for the XMedia dataset, while the performance is similarly stable across most modality choices, it exhibits a notable decline for the Audio modality. This discrepancy is likely attributable to the reliance of the Audio modality in the XMedia dataset on -in-iiensioonl MFCC features (i.e., 29-dimensional MFCC), which lack sufficient representational capacity. Consequently, this lim- itation results in lower-quality knowledge library, ulti- mately leading to significantly degraded performance.
74
MAP
71
0.5
0.6
0.7
0.8
0.9
1.0
9
90
90
(a) Different
(b) Different K
80
75.0
74.1
73.6
80-
66.3
60:
60
51.0
49.5
49.0
49.4
50.8
Figure 5. The impact of similarity boundary and knowledge library scale the XMedia dataset. a) The mean MAP scores of all cross-modal retrieval tasks with 128 bits under different similarity boundaries a. b) The mean MAP scores of all cross-modal retrieval tasks with 128 bits under different numbers of class prototypes K.
50.
40
30
24.9
30-
10
Image
Text
Audio
30
Video
0 Image
Text
Audio
3D
Video
(a) XMedia
(b) XMediaNet
Figure 4. The mean MAP scores of all cross modal retrieval tasks with 128 bits for different choices of the first available modality on the XMedia and XMediaNet datasets.
5. Conclusion
In this work, we reveal and study a practical but less-touched problem in cross-modal hashing (CMH), i.c., streaming- media hashing retrieval. To overcome this issue, we propose novel CMH paradigm dubbed SHE, which could train con- tinuously incoming streaming media data in parallel. The proposed SHE mainly consists of three core modules, i.e.. Knowledge Library Mining module (KLM), Knowledge Library Transfer module (KLT), and Discriminative Hash- ing Learning module (DHL). Specifically, KLM extracts essential class prototypes, thereby constructing an implicit knowledge library for each modality. KLT aligns the knowl- edge of the newly arrived modality with the historical knowl- edge library, thereby preserving semantic consistency across modalities. DHL maximizes intra-class semantic relevance and inter-class semantic disparity, resulting in compact and discriminative hash codes. Extensive experiments on four benchmark datasets highlight the superiority of our SHE compared bate-of-the--a methods.
4.8. The Impact of Similarity Boundary
In our KLT, for intra-class prototypes across different modal- ities, we focus on maintaining a certain level of semantic similarity between them by setting similarity boundary rather than forcing them to be identical. To investigate the impact of different similarity boundaries on performance and verify the effectiveness of the above strategy, we con- duct experiments with 128 bits on the XMedia dataset by varying the similarity boundary a and report the correspond- ing mean MAP scores of all cross- modal retrieval tasks. As presented in Fig.5(a), the performance follows single peak trend as a increases, reaching the optimal result at a = 0.95. The finding highlights that maintaining a certain level of semantic similarity is more effective in guiding the learn- ing of different modalities compared directly enforcing

---

SHEE: Streaming-media Hashing Retrieval
Acknowledgements
Lau, J. H. and Baldwin, T. An empirical evaluation of doc2vec with practical insights into document embedding generation. arXiv preprint arXiv: 033666, 2016.
This work supported in part by the National Natural Sci- ence Foundation of China under Grant No. 62372315, the Sichuan Science and Technology Planning Project under Grants No. 2024NSFTD0049, 2024ZDZX0004, 2024YFHZ0089, the Chengdu Science and Technology Project (Grant no. 2023-XT00-00004-GX), and the Sichuan Science and Technology Miaozi Program (Grant no. MZGC20240057).
Li, Y., Qin, Y., Sun, Y., Peng, D., Peng, X., and Hu, P. Romo: Robust unsupervised multimodal learning with noisy pseudo labels. IEEE Transactions on Image Pro- cessing, 53:5086-5097, 2024.
Li, Y., Sun, Y., Qin, Y., Peng, D., Peng, X., and Hu, P. Robust duality learning for unsupervised visible-infrared person re-identification. IEEE Transactions on Informa- tion Forensics and Security, 2025a.
Impact Statement
Most existing cross-modal retrieval (CMR) methods com- monly assume synchronous availability of media data, ig- noring the presence of streaming-medit data (SMD), i.e., continuously incoming data. Although some real-valued methods independently train each modality to improve flex- ibility in handling SMD, their low retrieval efficiency falls short of large-scale data retrieval. This paper proposes novel hashing paradigm to achieve asynchronous retrieval for SMD, offering more practical and portable solution for real-world applications.
Li, Y., Zhen, I.., Sun, Y., Peng, D., Peng, X., and Hu, P. Deep evidential hashing for trustworthy cross-modal retrieval. In Proceedings of the AAAI Conference on Artificial In- telligence, number 17, pp. 18566-185777 2025b.
Liang, K., Meng, L.,, Liu, M., Liu, Y., Tu, W., Wang, S., Zhou, S., Liu, X., Sun, F,, and He, K. A survey of knowl- edge graph reasoning on graph types: Static, dynamic, and multi-modal. IEEE Transactions on Pattern Analysis and Machine Intelligence,
2024.
References
Liu, K., Gong, Y., Cao, Y., Ren, Z., Peng, D., and Sun, Y. Dual semantic fusion hashing for multi-label cross- modal retrieval. In Proceedings of the Thirty- Thiry-Thirdda Inter- national Joint Conference on Artificial Intelligence, pp. 4569-4577, 2024.
Chua, T.-S., Tang, J., Hong. R., Li. H., Luo, Z., and Zheng, Y. Nus-wide: a real-world web image database from na- tional university of singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval, pp. 1-9, 2009.
Liu, S., Qian, S., Guan, Y., Zhan, J., and Ying, L. Joint- modal distribution- based similarity hashing for large- scale unsupervised deep cross-modal retrieval. In Pro- ceedings of the 43rd International ACM SIGIR Confer- ence on Research and Development in Information Re- trieval, pp. 1379-1388, 2020.
Duan, S., Sun, Y., Peng, D., Liu, Z., Song, X., and Hu, P. Fuzzy multimodal learning for trusted cross-modal retrieval. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2025.
Hu, P., Zhu, H., Lin, J., Peng, D., Zhao, Y.-P., and Peng. X. Unsupervised contrastive cross- cross-modal hashing, IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 45(3)33877-3889, 2022.
Liu, X., Zeng. H., Shi, Y., Zhu, J., Hsia, C.-.., and Ma, K.-K. Deep cross cross-modal hashing based on semantic consistent ranking. IEEE Transactions on Multimedia, 25:9530- 9542, 2023.
Hu, Z., Cheung, Y.-M., Li, M., and Lan, W. Cross-modal hashing method with properties of hamming space: A new perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):663-77500 2024.
Peng, Y., Zhai, X., Zhao, Y., and Huang, X. Semi-supervised cross-media feature learning with unified patch graph reg- ularization. IEEE Transactions on Circuits and Systems for Video Technology, 26(3):583-596 2015.
Huo, Y., Qin, Q. Dai, J., Wang, L., Zhang, W., Huang, L., and Wang, C. Deep semantic- proxy hashing for multi-label cross- modal retrieval. IEEE Transactions Circuits and Systems for Video Technology, 34(1):576- 589, 2023.
Peng. Y., Huang. X., and Zhao, Y. An overview of cross-
media retrieval: Concepts, methodologies, benchmarks and challenges. IEEE Transactions on Circuits and Sys- tems for Video Technology, pp. 2372-2385, Sep 2018.
Huo, Y., Qin, Q. Zhang, W., Huang, L., and Nie, J. Deep hierarchy- aware proxy hashing with self-paced learning for cross modal retrieval. IEEE Transactions on Knowl- edge and Data Engineering,
Pu, R., Qin, Y., Peng, D., Song, X., and Zheng, H. Deep reversible consistency learning for cross-modat retrieval. IEEE Transactions on Multimedia, 2025a.
2024.
10

---

SHE: Streaming-medd Hashing Retrieval
Pu, R., Sun, Y., Qin, Y., Ren, Z., Song, X., Zheng, H., and Peng, D. Robust self- self-paced hashing for cross-modal retrieval with noisy labels. Proceedings of the AAAI Con- ference on Artificial Intelligence, (1119):9999--1997) 2025b.
Wang, L., Qin, Y., Sun, Y., Peng, D., Peng, X., and Hu, P. Robust contrastive cross-modal hashing with noisy labels. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 5752-5752550 5760, 2024a.
Wang, Y. and Peng. Y. Mars: Learning modality-agnosti representation for scalable cross-media retrieval. IEEE Transactions on Circuits and Systems for Video Technol- ogy, 32(7))4776-44 4777, 2021.
Qian, S., Xue, D., Fang, Q., and Xu, C. Integrating multi- label contrastive learning with dual adversarial graph neural networks for cross-modal retrieval. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 45 (4):4794-4814 2022.
Wang. Y., Dong, F., Wang, K., Nie, X., and Chen, Z.-D. Weighted cross-modal hashing with label enhancement. Knowledge- Based Systems, 293: 2024b.
Qin, Q. Huo, Y., Huang, L., Dai, J., Zhang, H.,, and Zhang, W. Deep nrighborhood-preeervee hashing with quadratic spherical mutual information for cross-modke retrieval. IEEE Transactions Multimedia, 26:6361-6374 -6374, 2024.
Wang, Z.,Xu X., Wei, J., Xie, N., Yang. Y., and Shen, H. T. Semantics disentangling for cross-modal retrieval. IEEE Transactions on Image Processing, 33:2226-2237, 2024c.
Qin, Y., Peng, D., Peng, X., Wang. X., and Hu, P. Deep evidential learning with noisy correspondence for cross- modal retrieval. In Proceedings of the 30th ACM Interna- tional Conference on Multimedia, pp. 4948-448899 4956, 2022.
Yang. E., Yao, D., Liu, T., and Deng, C. Mutual quantization for cross-modal search with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7551-7560, 2022.
Qin, Y., Sun, Y., Peng, D., Zhou, J. T., Peng, X., and Hu, P. Cross- mooal active complementary learning with self- refining correspondence. Advances in Neural Information Processing Systems, 36:24829 28844029-2440, 2023.
Yu, J., Zhou, H., Zhan, Y., and Tao, D. Deep graph-neighbor coherence preserving network for unsupervised cross- modal hashing. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4626 4634, 2021.
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R., Levy, R., and Vasconcelos, N. A new approach to modal multimedia retrieval. In Pro- ceedings of the 18th ACM International Conference Multimedia, pp. 251 160, 2010.
Zhang, F., Zhou, H., Hua, X.-S., Chen, C., and Luo, X. Hope: A hierarchical perspective for semi- sumi-super 2d- 3d cross-modal retrieval. IEEE Transactions Pattern Analysis and Machine Intelligence, 46(12):8976-8993 2024.
Shi, Y., Zhao, Y., Liu, X., Zheng, F., Ou, W., You, X., and Peng, Q. Deep adaptively eedaaneeel-eehhaneedd enhanced hashing with discrim- inative similarity guidance for unsupervised cross-modal retrieval. IEEE Transactions on Circuits and Systems for Video Technology. 32(10):7255-7266 2022.
Zhang, Z., Luo, H., Zhu, L., Lu, G., and Shen, H. T. Modality-invariant asymmetric networks for cross-modal hashing. IEEE Transactions Knowledge and Data Engineering, 35(5):5091-51)) 5104, 2022.
Shu, Z., Bai, Y., Yong, K., and Yu. Z. Deep cross-modal hashing with ranking learning for noisy labels. JEEE Transactions Big Data, 2024.
Zhang. Z., Luo, H., Zhu, L., Lu, G., and Shen, H. T. Modality-invariant asymmetric networks for cross-modal hashing. IEEE Transactions Knowledge and Data Engineering, 35(5):5091-5104 2023.
Simonyan, K. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1555 2014.
Sun, Y., Ren, Z., Hu, P., Peng, D., and Wang, X. Hierarchi- cal consensus hashing for cross moosl retrieval. IEEE Transactions Multimedia, 26:824 836, 2023.
Zhu, L.. Wu, X., Li, J., Zhang, Z. Guan, W., and Shen, H. T. Work together: Correlation-identiy reconstruction hashing for unsupervised cross-modal retrieval. IEEE Transactions Knowledge and Data Engineering, 35 (9)8883888851, 2022.
Sun, Y., Qin, Y., Peng, D., Ren, 7., Yang, C., and Hu, P. Dual self-paced hashing for image retrieval. IEEE Transactions on Multimedia, 26:9619-9629 2024.
Zhu, L., Zheng, C., Guan, W., Li, J., Yang, Y., and Shen, H. T. Multi-modas hashing for efficient multimedia re- trieval: A survey. IEEE Transactions on Knowledge and Data Engineering, 36(1):239-260, 2023.
Tan, W. Zhu, L., Li, J., Zhang, H., and Han, J. Teacher- student learning: Efficient hierarchical message aggrega- tion hashing for cross modal retrieval. IEEE Transactions on Multimedia, 25:4520- 45322, 2022.

---

SHEE: streaming-media Hashing Retrieval
APPENDIX
50
This document provides supplementary material to our manuscript, which aims to offer comprehensive under- standing of our SHE framework. Specifically, we mainly present the dataset description, additional experimental re- sults, more detailed analysis about the media learning se- quence, the analysis about the impact of bit length, and the discussion of limitations of this work to support the main submission. For convenience, we provided jumpable links to each supplementary section in Tab.6.
60
MAP
50
30-
30
20
16
32
125 236
512
1924
15
22
128
256
512
1924
Bit Longth
Bit Length
(a) XMedia
(b) XMediaNet
Figure 6. The MAP scores with different bit lengths on the XMedia and XMediaNet datascts.
Dataset
Please click here.
The impact of bit length Precision-recaee curves
Please click here.
2021).
Please click here.
The impact of media learning sequence
Please click here.
B. The Impact of Bit Length
Limitations
Please click here.
To examine the impact of bit length on retrieval performance, we perform experiments on the XMedia and XMediaNet datasets. The corresponding mean MAP scores of all cross- modal retrieval tasks are illustrated in Fig.6. The results reveal the following insights: 1) Hash codes with longer bit lengths can carry more semantic information, thus enjoying better performance. 2) Given that the XMedia and XMe- diaNet datasets contain five modalities with considerable differences, hash codes with shorter bit lengths (e.g.. 16 and 32 bits) may fail to effectively capture the underlying semantic features, leading to subpar performance.
Table 6. The hyperlinks for each supplementary section.
A. Dataset
The following is a detailed introduction to each dataset: Wikipedia is dataset comprising 2,866 image-text pairs with 10 categories. For our experiment, we randomly al- locate 2,173 and 693 pairs for training and testing, respec- tively, while all pairs are used the retrieval database. NUS-WIDE is a dataset comprising over 260,000 image- text pairs with 81 categories. In this work, we utilize a subset of 10,000 pairs with 10 semantic labels to perform experiments. In more detail, we randomly assign 8,000 and 2,000 pairs for training and testing, respectively, while all pairs are used as the retrieval database. XMedia is dataset covering five modalities with 20 categories, com- prising 5,000 image-text pairs, 1,000 audio clips, 500 3D models, and 1,143 videos. For our experiment, we select 4,000 image-text pairs, 800 audio clips, 400 3D models, and 969 videos as the training set, while the remaining samples are used as the test set. Besides, all samples from the dataset included in the retrieval database. NetdiaNet is dataset covering five modalities with 20 categories, compris- ing 40,000 image-text pairs, 10,000 audio clips, 2,000 3D models, and 10,000 videos. For our experiment, we select 32,000 image-text pairs, 8,000 audio clips, 1,600 3D mod- els, and 8,000 videos as the training set, while the remaining samples are used as the test set. Besides, all samples from the dataset are included in the retrieval database.
C. Precision-recall Curves
To comprehensively evaluate the retrieval performance of our proposed SHE, we provide precision-recall curves with different bit lengths (i.e., 16, 32, and 64 bits) on the Wikipedia and NUS-WIDE datasets. As shown in Fig.7, SHE consistently exhibits higher precision at almost all recall rates compared to SOTA methods, exhibiting its stronger discriminative power and retrieval performance.
D. The Impact of Media Learning Sequence
In the manuscript, we have delved into the impact of me- dia learning sequences with 128 bits on the XMedia and XMediaNet datasets in terms of mean MAP scores of all cross-modal retrieval tasks. To offer a more detailed anal- ysis, we provide the MAP scores for each cross-modal re- trieval task in Tab.7. From the results, we can get the same findings as the manuscript. Besides, we also find that when the audio modality is selected as the first available modality on the XMedia dataset, the performance on the audio, 3D, and video modalities degrades significantly. This may be due to: 1) The audio modality being represented by low- dimensional (29D) MFCC features with weak expressive power, leading to poor-quality knowledge initialization. 2)
For Wikipedia and image modality are extracted using the pre-trained VGG- 19 (Simonyan, 2014) model, while the features of the text modality are obtained using the pre-trained Doc2vec (Lau & Baldwin, 2016) model. For XMedia and XMediaNet, their original features are provided by the authors (Wang& Peng,
the initial features of the
12

---

SHEE: Atreaming-media Hashing Retrieval
0.5
0.7
DGGPN
0.6
Frecsson
HIGH
RMM
0.54
MM1
0.50
0.6g
U00H
0.5
UCDI
0.31
0211
0.4
Draft
0.3
0.2
0.2
0.2
ChuFF!
0.0
0.4
0.6
0.8
1.0
0.1
0.0
0.2
0.4
0.5
0.3
1.0
0.1
0.0
02
0.4
0.6
0.5
1.0
0.1
0.0
0.2
0.4
0.5
0.8
1.0
Recall
Recall
Recall
Recall
(a) Wikipedia (12T)
(b) Wikipedia (T21)
(c) NUS-WIDE (12T)
(d) NUS-WIDE (T2I)
0.5
0.71
DGEPN
0.6
MMH
0.5
000%
Prece
LC31
0.5
UCOH
0.4
UCDI
0.3g
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.0
0.4
0.6
0.8
1.0
0.0
0.4
0.6
0.75
1.0
0.0
022
0.4
0.5
0.8
1.0
0.0
022
0.4
0.7
1.0
Recall
Recall
Recall
Recall
(e) Wikipedia (I2T)
(f) Wikipedia (T2I)
(g) NUS-WIDE (12T)
(h) NUS-WIDE (T2I)
0.8
0.5
S:E
5:5
0.0g
5:0
0.6
0.5
Precsiio
CF11
Precson
Procsin
9.5
CFI1
CEI
UC31
0.5
U001
U001
0.4
UC01
0.3
DUVO
0.4
CUMO
CAMO
DETLL
0.3
0.3
DIO
0.3
DIE.
0.2
0.1
0.1
SCH
0.1
0.1
501
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.5
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.5
0.8
1.0
Recall
Recall
(i) Wikipedia (I2T)
(i) Wikipedia (T2I)
(k) NUS-WIDE (12T)
(l) WIDE (T2I)
Figure 7. Precision-recat recall curves with different bit lengths the Wikipedia and NUS-WIDE datasets, where (a-d), (e-h), and (i-l) show the curve results under 16 bits, 32 bits, and 64 bits, respectively.
Table The MAP scores with 128 bits for different choices of the first available modality on XMedia and XMediaNet datasets.
Image
Text
Audio
3D
Video
Dataset
Avg
Text Audio 3D
Video
Image Audio 3D
Video
Image Text 3D
Video
Image Text Audio Video
Image
Text Audio 3D
Image
92.1
87.3 89.6 87.2
95.8
92.0
94.0
91.7
65.3 66.564.8
62.3
70.2 70.4
67.7
66.4
60.7 61.3 57.2
75.0
Text
91.8
86.7 88.4
87.4
95.4
91.3
93.6
92.5
63.4 62.5
60.6
68.6
69.4
64.0
64.7
60.5
62,2 56.5
58.2
XMedia
90.0
6.0 8.4 61.1
91.4
5.5
7.8
56.5
10.4
10.3 7.0
11.5
9.6
13.7 5.8
10.7
40.7
39.1 5.3 6.7 24.9
3D
91.6
71.9 84.1
84.0
94.6
75.0
88.1
86.1
52.9
55.1 49.8
47.5
59.7
58.6
51.2
54.5
59.8
60.5 47.2
54.466..
Video
91.9 85.0 88.9
87.4
95.6
89.9
93.9
92.0
63.0 63.2
59.3
68.3
69.3
63.2
64.3
60.8
61.6 55.6
57.2 73.6
Image
75.1 70.6 70.3 72.8
70.5
62.8
63.5 65.8
42.4 39.8 36.7 38.2
41.2
39.3
34.0
37.7
43.1
41.0
36.4
31.0
Text 73.4 67.7 68.1 71.4
69.0
60.8
61.2 64.0
41.1 38.4
36.9
38.2
36.6
32.8
35.3
43.3
41.0
36.9
37.949.5
XMediaNet Audio
74.9 63.8 67.8 72.6
69.9
55.4
60.2 65.0
42.4
39.535.7
38.2
38.9
36.8
27.6
35.8
43.1
41.5
33.1
37.2
3D 74.6 70.0 67.4 72.3
69.9
62.9
60.2 65.0
41.1
39.033.6
37.3
36.9
35.2
30.9
34.5
42.7
41.1
37.3
36,249.4
Video 75.4 70.4 70.4 73.3
71.0
62.8
63.5 66.0
41.1
38.8 35.8
36.9
40.4
38.7
34.1
37.6
43.6
41.1
36.7
39.0
50.8
The insufficient number of training samples in the audio, 3D, and video modalities, which causes underfitting.
retrieval. To the best of our knowledge, this could be the first work to address the problem of streaming data (SMD) using CMH paradigm. However, we have not yet accounted for the scenario that SMD may contain noisy annotations, which may be one of our future research direc- tions.
E. Limitations
This paper proposes novel cross-modal hashing (CMH) retrieval framework tailored for streaming-mede media hashing
13


Reviews:
[{"review_id": "hev0GKbvTe", "reviewer": "Reviewer_TSiu", "summary": "While most previous research in CHM have assumed that all modalities are prepared before processing, the authors propose a novel CHM paradigm named Streaming-media Hashing rEtrieval (SHE) that enables parallel training of each modality for streaming-media data, where the data is collected chronologically. \nThe paradigm includes a knowledge library mining module and a knowledge library transfer module to jointly extract an implicit knowledge library from the new incoming data and align the commonality distribution from the new knowledge with ones from the historical knowledge library and also a discriminative hashing learning module to enhance intra-class semantic relevance and inter-class semantic disparity. \nThis paradigm is tested over 4 common benchmark datasets streaming-media retrieval, and the results show its superiority over 14 state-of-the-art methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Our responses are listed below.\n\n**R-Weakness-1**: We have revised this issue, providing the relevant explanation when KLR first appears.\n\n**R-Weakness-2**: In the introduction section, we list CMH methods (Such as UCCH, DHaPH, DCH-SCR[1], SCH, and CICH[2]) to illustrate the shortcomings of existing CMH research. Here, we briefly discuss some recent CMH works below: 1) UCCH aims to implicitly capture the semantic relevance between modalities by mining the co-occurrence relationship between multimodal data (such as image-text pairs) and their initial distribution characteristics. 2) DCH-SCR[1] deploys a ranking alignment mechanism to capture the close semantic relationship between modalities, which preserves the semantic similarity between tags and feature levels. 3) CICH[2] designs a prototypical semantic similarity coordination module to globally rebuild partially-observed cross-modal similarities under an asymmetric learning scheme. Notably, these CMH methods implicitly assume that all modalities are prepared before processing and adopt joint learning to achieve cross-modal semantic alignment. In practical application scenarios, it is challenging to collect data of all modalities simultaneously, such as emergency medical aid, multi-modal search engines, and financial data analysis. More commonly, data from all modalities is collected asynchronously. To improve the practical applicability of CMH, this paper proposes a novel method termed SHE to enhance the flexibility of processing asynchronously collected multimodal data.\n\n**R-Weakness-3**: To enhance the readability and understanding of our SHE method, we explain some loss functions in detail. 1) For the Knowledge Library Regularization loss (KLR), which is formulated as:\n\n$L_{klr} = -\\frac{1}{CK} \\sum_{c=1}^{C} \\sum_{k=1}^{K} \\log \\frac{ \\sum_{k'=1,k' \\ne k }^{K}e^ {\\Gamma(p^{c,k}_m,p^{c, k'}_m)}} {\\sum _{c'=1}^{C}\\sum _{k'=1}^{C}{e^ {\\Gamma(p^{c,k}_m,p^{c',k'}_m)}}-e}.$\n\nKLR aims to escape a simple solution where all prototypes converge to a single point. As the above equation shows, we first treat inter-class prototypes as positive pairs and intra-class prototypes as negative pairs. Then, we maximize the similarity among positive pairs and minimize the similarity among negative pairs, thereby enhancing the distinctiveness between inter-class prototypes while ensuring semantic consistency between intra-class prototypes. 2) For the Knowledge Library Transfer loss (KLT), which is formulated as:\n\n$L_{klt} = -\\frac{1}{CK} \\sum_{c=1}^{C} \\sum_{k=1}^{K}\n\\log \\left [ min(1,max(0,\\Gamma(p_1^{c,k},p_m^{c,k}) - \\sigma +1)) \\right ].$\n\nKLT aims to transfer the semantic information in the newly mined knowledge to the benchmark knowledge library, thereby achieving cross-modal semantic consistency. As is shown in the above equation, we encourage the prototypes mined from the newly coming modality to align with the prototypes with the same semantics in the benchmark knowledge library by maximizing their similarity. Notably, we focus on maintaining a certain level of semantic similarity between them by setting a similarity boundary rather than forcing them to be identical. That is, the above learning objective is $\\Gamma(p_1^{c,k},p_m^{c,k}) \\ge \\sigma$, which relaxes the alignment requirement and can further improve the representation ability of multimodal data.\n\n**R-Weakness-4**: Parallel training means that the proposed SHE can independently process each modality. When facing datasets with five modalities, SHE can directly perform training without any preprocessing, resulting in $5$ training processes and a storage overhead of $5$ sub-networks. By contrast, all baselines except MARS are limited to joint training on paired data from two modalities. When these methods face datasets with five modalities, we first construct pseudo-instance pairs through label-based repeat sampling and then conduct training on every two modalities. As a result, they require $5\\times 4/2 = 10$ training processes and incur a storage overhead of $5 \\times 4 = 20$ sub-networks. Obviously, parallel training can enhance training efficiency and reduce the required storage space. Besides, to further show the advantages of parallel training, we compare the training time (TT) and storage overhead (SO) of SHE and SCH on the XMedia dataset as follows:\n\n|Method|TT|SO|\n|:-:|:-:|:-:|\n| SCH |2148 s|2315.72 MB|\n| SHE |240 s|910.66 MB|\n\nThe results show that our SHE can enhance training efficiency and reduce the required storage space by parallel training.\n\n**R-Suggestions**: We carefully check our paper and ensure that similar mistakes are not made.\n\n**Reference**\n\n[1] Liu X, Zeng H, Shi Y, et al. Deep cross-modal hashing based on semantic consistent ranking[J]. IEEE Transactions on Multimedia, 2023, 25: 9530-9542.\n\n[2] Luo H, Zhang Z, Nie L. Contrastive incomplete cross-modal hashing[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36: 5823-5834.", "claims_and_evidence": "Yes, the claims in the submission are well supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria make sense for the problem or application at hand.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes, the experimental designs and analyses are valid.", "relation_to_broader_scientific_literature": "The proposed SHE aims to solve the problem of training streaming-media data, which isn’t prepared before processing. SHE introduces an evolving knowledge library to avoid training redundancy and also improve training efficiency.", "essential_references_not_discussed": "It is suggested to list the references about the previous CMH methods in introduction.", "other_strengths_and_weaknesses": "Strengths\n- The paper is well-constructed and easy to read.\n- The experiments are conducted over 4 experiments and 14 state-of-the-art methods.  \n\nWeaknesses\n- In section 3.2, the authors use the notation KLR loss before explaining what it is. It’s better to address it with its full name and notify the readers this will be explained later.\n- In introduction, authors describe many previous CMH methods with all sorts of shortcomings but fail to list the actual methods. Also, the authors only point out emergency medical aid for practical application scenarios, which seems insufficient.\n- The loss function formulation in this paper lacks detailed explanation. It’s better to explain the loss function design more specifically.\n- The authors mention that the proposed method enables parallel training of each modality, making training more efficient and require less storage. But this is not shown in the experiments.", "other_comments_or_suggestions": "Typos\n- Line 39 right: two ‘such as’s\n- Line 99 right: ‘knowledge library mining module (KLM) module’ module is included in the abbreviation\n- Line 205 right: ‘where $p_m^c$ denotes K the normalized and binary prototype for the c-th class’ unnecessary ‘K’", "questions_for_authors": "please refer to the weakness part.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i9vSJNoRoW", "reviewer": "Reviewer_cJMB", "summary": "This work addresses a less studied but practically valuable problem in cross-modal retrieval, i.e., streaming-media hashing retrieval. The paper points out the key challenge is to preserving cross-modal interactions. Through knowledge library mining on existing modalities and knowledge transferring only on the subsequently arrived modalities, the proposed training framework requires no re-training on the historical data but successfully establishes cross-modal knowledge alignment, hence reducing training complexity. Experiments on various datasets have validated the effectiveness of the proposed method.\n\n## update after rebuttal\nThe rebuttal from authors solved my concerns regarding clarity. Therefore, I keep my score as 4 towards acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable feedback. Our responses are listed below.\n\n**R-Weaknesses**: To make the experiment more rigorous, a detailed description of the training procedure for the compared methods is provided. Specifically, there are two cases in the training. 1) For the Wikipedia and NUS-WIDE datasets, all baselines directly conduct training without any preprocessing or modification, as the data in these datasets consists of image-text pairs. 2) For the XMedia and XMediaNet datasets, all baselines except for MARS adopt label-based repeat sampling to construct pseudo-instance pairs, and then conduct training on every two modalities, as the data in these datasets consists of independent instances (unpaired and inconsistent in number). MARS directly conducts training without any preprocessing or modification, as it can process each modality independently.\n\n**R-Suggestion-1**: We carefully check our paper and ensure that similar mistakes are not made.\n\n**R-Suggestion-2**: To enhance the understanding of the loss of Eq. 8 (i.e., Knowledge Library Transfer loss, KLT), we add some explanations to further elaborate on it. The KLT loss is formulated as:\n\n$L_{klt} = -\\frac{1}{CK} \\sum_{c=1}^{C} \\sum_{k=1}^{K}\n\\log \\left [ min(1,max(0,\\Gamma(p_1^{c,k},p_m^{c,k}) - \\sigma +1)) \\right ].$\n\nKLT aims to transfer the semantic information in the newly mined knowledge to the benchmark knowledge library, thereby achieving cross-modal semantic consistency. As is shown in the above equation, we encourage the prototypes mined from the newly coming modality to align with the prototypes with the same semantics in the benchmark knowledge library by maximizing their similarity. Notably, we focus on maintaining a certain level of semantic similarity between them by setting a similarity boundary rather than forcing them to be identical. That is, the above learning objective is $\\Gamma(p_1^{c,k},p_m^{c,k}) \\ge \\sigma$, which relaxes the alignment requirement and can further improve the representation ability of multimodal data.\n\n\n**R-Questions**: It is a constructive suggestion. To alleviate negative effects caused by low-quality first modalities, we first design a score to evaluate the quality of knowledge mined from different modalities. For the $m$-th modality, the score could be formulated as follows: \n\n$S_{m}=\\frac{1}{2n_m} \\sum_{i=1}^{n_m}  \\max_{k=1, \\ldots, K} s\\left(x_m^i | k\\right) +\\frac{1}{2CK} \\sum_{c=1}^{C} \\sum_{k=1}^{K} \\frac{ \\sum_{k'=1,k' \\ne k }^{K}e^ {\\Gamma(p^{c,k}_m,p^{c, k'}_m)}} {\\sum _{c'=1}^{C}\\sum _{k'=1}^{C}{e^ {\\Gamma(p^{c,k}_m,p^{c',k'}_m)}}-e},$\n\nwhere $s(x_m^i | k)=\\frac{\\sum_{c=1}^{C} y_m^{i,c}\\cdot \\Gamma(b_m^i,p^{c,k}_m)+1}{2}$ and $\\Gamma(\\cdot,\\cdot)$ is the cosine similarity function. In the above equation, the first item is responsible for reflecting the consistency between the mined knowledge and sample representations, while the second item is responsible for reflecting the distinction among inter-class knowledge. The higher the score is, the more the mined knowledge reflects the semantic information in the samples and the more discriminable it is, that is, the higher quality the modality owns. To verify the effectiveness of the proposed score, we conduct experiments on the XMedia dataset with 128 bits. The results are shown as follows: \n\n|Indicator|Image|Text|Audio|3D|video|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|Score|0.763|0.763|0.560|0.747|0.762|\n|MAP|75.0|74.1|24.9|66.3|73.6|\n\nFrom the results, it can be observed that the proposed score can accurately reflect the quality of the modalities. Due to the word limit, a simple strategy is provided here. In real applications, if we observe that the score of the current modality is too low, we can wait for subsequent modalities to arrive. Once a modality with sufficient quality is detected, it can be selected as the initial modality. According to your suggestion, we will explore how to alleviate negative effects caused by low-quality first modalities in the future.", "claims_and_evidence": "Yes, claims regarding the effectiveness of each part of the paper, including the three modules and all the loss objectives are well-supported by clear experimental evidences, including benchmarks with various number of modalities and comprehensive ablation study on all evaluated datasets.", "methods_and_evaluation_criteria": "The proposed method is established with solid analysis on the characteristics of streaming-media data, while the used metrics (mAP, P-R Curves) are widely-used evaluation protocols in cross-modal hashing that can also effectively evaluate the studied problem. The selected datasets are widely-used for cross-modal hashing with multiple modalities, hence suitable for the evaluation of the proposed method.", "theoretical_claims": "The paper does not contain proofs/theoretical claims.", "experimental_designs_or_analyses": "The paper's method is verified comprehensively and analyzed thoroughly to establish its validity.", "relation_to_broader_scientific_literature": "The work can be applied to cross-modal hashing with increasing data. Particularly, this paper contributes to the problem of increasing modalities and can also potentially improve the similarity learning of current multi-modal hashing methods.", "essential_references_not_discussed": "Key references are well-discussed.", "other_strengths_and_weaknesses": "Strengths: \n1. The paper addresses the novel and critical problem of streaming-media hashing retrieval with clear analysis of its potential challenges.\n2. The method is clearly motivated and designed based on the characteristics of streaming-media data.\n3. Experiments compare with state-of-the-art methods and display significant advantages for retrieval with increasing modalities. Ablation study and other analyses further emphasize the validity and effectiveness of the method. Results with different media sequence verifies the general effectiveness with different initial modalities.\n\nWeaknesses:\n1. The authors may include descriptions of the training procedure for the compared methods.", "other_comments_or_suggestions": "1. Typos: a) Line16: “in practice applications”. b) Line 39 (the right column) “such as such as”.\n2. It's better to add some explanations about the Eq. 8 loss objective and the similarity boundary.", "questions_for_authors": "The model’s performance can vary upon the choice of the initial modality, which is sometimes infeasible to be simply replaced by another in real applications. What kind of improvements are significant to alleviate such effects caused by low-quality first modalities?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G4Nh3s9sQe", "reviewer": "Reviewer_UdWL", "summary": "This paper proposes a novel CMH paradigm, specifically designed for cross-modal retrieval in streaming-media scenarios. The proposed SHE framework comprises three key modules: the Knowledge Library Mining (KLM) module, the Knowledge Library Transfer (KLT) module, and the Discriminative Hash Learning (DHL) module. Specifically, KLM constructs a knowledge library for every modality by extracting essential prototype knowledge, thereby serving as a semantic consistency maintenance medium. KLT aims to adaptively align the knowledge library extracted from the newly arrived modality with the historical knowledge library, thereby capturing the semantic consistency of multiple modalities. DHL enhances the quality of hash codes by maximizing intra-class semantic relevance while amplifying inter-class semantic disparity. Extensive experimental evaluations demonstrate that the proposed method effectively handles cross-modal retrieval in streaming-media scenarios, offering a flexible and scalable solution for real-world applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive review. Our responses are listed below.\n\n**R-Weakness-1**: To evaluate the quality of knowledge mined from different modalities, we design a score, which can be found at **`R-Questions’** in [Reviewer cJMB](https://openreview.net/forum?id=JqLKV0L5hM&noteId=i9vSJNoRoW).\n\n**R-Weakness-2**: To make readers understand the work more clearly, we reorganize the writing, such as the design of losses. Specifically, we explain Knowledge Library Regularization (KLR) loss and Knowledge Library Transfer (KLT) loss in detail, which can be found at **`R-Weakness-3’** in [Reviewer TSiu](https://openreview.net/forum?id=JqLKV0L5hM&noteId=KyOMFKYOSw).\n\n**R-Weakness-3**: It is a constructive suggestion. To solve this, we first design a score to evaluate the quality of knowledge mined from different modalities, which can be found in **`R-Questions’** in [Reviewer cJMB](https://openreview.net/forum?id=JqLKV0L5hM&noteId=i9vSJNoRoW). Then, due to the word limit, we provide a simple solution here. If we observe that the score of the current modality is too low, we can wait for subsequent modalities to arrive. Once a modality with sufficient quality is detected, it can be selected as the initial modality. According to your suggestion, we will explore how to alleviate negative effects caused by low-quality first modalities in the future.\n\n**R-Suggestion-1**: We investigate the latest studies and found that there are no other CMH works to handle cross-modal retrieval (CMR) in streaming-media scenarios (SMC). Here, we discuss some real-valued representation-based CMR methods, including SDML[1] and DRCL[2], which are specifically designed to accomplish CMR in SMC. 1) These methods are real-valued based representations, which involve high computational complexity and memory overhead, making it difficult to meet the fast retrieval demands of large-scale datasets. 2) They deploy either a randomly initialized common space or an identical transformation weight matrix to guide semantic alignment, which overlooks the discrepancies among modalities. Since the randomly initialized common space or learned transformation weight matrix may retain modality-specific features that are not very relevant to the new modality, directly applying it may lead to semantic discrepancies. By contrast, our SHE transfers the baseline knowledge library to the new modality, thereby obtaining more semantically consistent representations.\n\n**R-Suggestion-2**: We reorganize the writing so that readers can understand the work more clearly, such as the design of losses. The details can be found at **`R-Weakness-3’** in [Reviewer TSiu](https://openreview.net/forum?id=JqLKV0L5hM&noteId=KyOMFKYOSw).\n\n**R-Question-1**: We have designed a score to evaluate the quality of knowledge libraries. Please see **`R-Questions’** in [Reviewer cJMB](https://openreview.net/forum?id=JqLKV0L5hM&noteId=i9vSJNoRoW).\n\n**R-Question-2**: As you suggested, we conduct experiments on the XMedia and XMediaNet datasets and found that it is effective to incorporate the above idea into existing methods (i.e., MARS). Specifically, we replace the original label parsing module in MARS with the knowledge library to guide representation learning and add the Knowledge Library Transfer (KLT) loss to maintain semantic consistency. The new objective of MARS can be formulated as:\n\n $L=L_{ori}+L_{klt},$\n\n where $L_{ori}$ is the original objective of MARS and $L _{klt}$ is the KLT loss. The results are shown as follows:\n\n|Objective|XMedia|XMediaNet|\n|:-:|:-:|:-:|\n|$L _{ori}$|72.0|46.1|\n|$L$|73.8|51.7|\n\nThe results can show that using knowledge libraries as a medium to maintain semantic consistency can be integrated into existing methods.\n\n**R-Question-3**: In fact, the KLR loss is not our main innovation, so we only provide a simple comparative Analysis. As you suggested, we provide another regularization technique, i.e., orthogonal regularization (OR). Specifically, OR aims to impose the constraint that inter-class prototypes are mutually orthogonal, which can be formulated as:\n\n$L_{or} =   \\frac{1}{CK} \\sum_{c=1}^{C} \\sum_{k=1}^{K}\\sum_{c'=1,c'\\ne c}^{C} \\sum_{k'=1}^{K} \\left | \\Gamma (p^{c,k}_m,p^{c',k'}_m) \\right |,$\n\nwhere $\\Gamma$ is the cosine similarity function. We replace Knowledge Library Regularization (KLR) loss with OR loss and conduct experiments on the XMedia and XMediaNet datasets with 128 bits, and the results are shown as follows: \n\n|Regularization|XMedia|XMediaNet|\n|:-:|:-:|:-:|\n|No|73.8|33.3|\n|OR|73.9|40.0|\n|KLR|75.0|51.0|\n\nThe results show that OR is effective but inferior to the regularization technique used in our paper. In the future, we will further explore better regularization techniques to contribute to our proposed SHE.\n\n**Reference**\n\n[1] Scalable deep multimodal learning for cross-modal retrieval, SIGIR. 2019: 635-644.\n\n[2] Deep Reversible Consistency Learning for Cross-modal Retrieval, TMM, 2025.", "claims_and_evidence": "SHE effectively handles cross-modal retrieval in streaming-media scenarios, which is supported by extensive experimental evaluations on four benchmark datasets. Besides, SHE can train each modality in parallel, which is supported by the reasonable module design. Although each modality is learned in parallel, semantic alignment can be achieved by using knowledge libraries as the bridge.", "methods_and_evaluation_criteria": "The proposed SHE framework effectively addresses the challenges of cross-modal retrieval, particularly in streaming-media scenarios. The selected benchmark datasets are well-suited for evaluating the effectiveness of SHE, as two of them contains five modalities and are unpaired, which accurately simulate the characteristics of streaming-media data. Additionally, the evaluation metrics employed in this study are widely recognized in the field of cross-modal retrieval and are highly appropriate for assessing the performance of SHE and the baseline methods.", "theoretical_claims": "SHE can handle unpaired multimodal data, which is well-grounded. SHE adopts a learning framework without modality interaction and utilizes class prototypes as an abstract knowledge library to preserve the semantic information of streaming-media data. This enables effective semantic representation learning even for unpaired multimodal data.", "experimental_designs_or_analyses": "The experimental designs are sound, with suitable benchmark datasets, appropriate evaluation metrics, well-defined experimental objectives, and reasonable experimental analysis. Extensive experiments and analyses are conducted to comprehensively evaluate assess the effectiveness of SHE, such as investigations into the impact of media learning sequences, similarity boundaries, and the scale of the knowledge library.", "relation_to_broader_scientific_literature": "This paper proposes a novel Cross-Modal Hashing (CMH) framework, termed Streaming-media Hashing rEtrieval (SHE), which is highly relevant to the domain of cross-modal retrieval. To be specific, this paper aims to analyze the limitations and shortcomings of existing research in streaming-media scenarios and propose corresponding solutions, thereby highlighting the advantages of SHE.", "essential_references_not_discussed": "Further discussion on cross-modal hashing methods, particularly techniques addressing cross-modal retrieval in streaming-media scenarios, would be beneficial.", "other_strengths_and_weaknesses": "Strengths: \n\n1)\tSound in originality. This paper proposes a novel CMH framework named SHE specifically designed for streaming-media data, which is a highly valuable application scenario in cross-modal retrieval. \n\n2)\tReasonable technical route. To enable parallel training of different modalities while maintaining semantic consistency, SHE employs the KLM module to construct a dedicated knowledge library for each modality, preserving semantic information from streaming-media. Subsequently, the KLT module aligns the newly extracted knowledge library with the baseline knowledge library, ensuring cross-modal semantic coherence. \n\n3)\tReasonable and comprehensive experimental arrangements. Extensive experiments are conducted on four datasets to verify the effectiveness and superiority of this method.\n\nWeaknesses: \n\n1)\tThis paper designates the knowledge library extracted from the first modality as the benchmark which may introduce a degree of arbitrariness. In other words, the study has not yet evaluated the quality of the knowledge libraries extracted from different modalities, which could impact the overall effectiveness of the proposed method.\n\n2)\tSome sections could benefit from clearer writing and organization.\n\n3)\tFrom Fig. 3, the media learning sequence has a significant impact on performance, but no possible solution is provided.", "other_comments_or_suggestions": "1)\tFurther discussion on cross-modal hashing methods, particularly techniques addressing cross-modal retrieval in streaming-media scenarios, would be beneficial.\n\n2)\tSome sections could be written and structured more clearly.", "questions_for_authors": "1)\tThis paper designates the knowledge library extracted from the first modality as the benchmark which may introduce a degree of arbitrariness. This paper would benefit from comparing and evaluating the quality of knowledge libraries.\n\n2)\tWould integrating the proposed idea of using a knowledge library as a medium to maintain semantic consistency into existing methods be beneficial?\n\n3)\tExcept the regularization approach applied to the knowledge library in this paper, could the authors provide a comparative analysis with other existing regularization techniques?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PysONTY7TF", "reviewer": "Reviewer_LZBc", "summary": "Existing CMH methods often implicitly assume that all modalities are prepared before processing. However, in practice applications (such as multi-modal medical diagnosis), it is very challenging to collect paired multi-modal data simultaneously. Specifically, they are collected chronologically, forming streaming-media data (SMA). To handle this, all previous CMH methods require retraining on data from all modalities, which inevitably limits the scalability and flexibility of the model. For this issue, this paper proposes a novel CMH paradigm named Streaming-media Hashing rEtrieval (SHE) that enables parallel training of each modality. Specifically, SHE proposes a knowledge library mining module (KLM) that extracts a prototype knowledge library for each modality, thereby revealing the common distribution of each modality. Then, SHE proposes a knowledge library transfer module (KLT) that uses the historical knowledge library to update and align new knowledge to ensure semantic consistency. Finally, to enhance the intra-class semantic relevance and inter-class semantic differences, SHE develops a discriminative hash learning module (DHL). In general, this paper is significantly innovative and has important value in practical applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. Our responses are listed below.\n\n**R-Weakness-1**: To enhance readability and understanding of our proposed SHE, we provide a detailed explanation of some key losses, which can be found at **`R-Weakness-3’** in [Reviewer TSiu](https://openreview.net/forum?id=JqLKV0L5hM&noteId=KyOMFKYOSw).\n\n**R-Weakness-2**: When mining the knowledge library, if all class prototypes are considered simultaneously, a sample may be influenced by multiple prototypes, leading to an unstable optimization objective. Thus, we only consider the nearest prototype. Besides, to verify the effectiveness of the above behavior, we conduct experiments on the XMedia and XMediaNet datasets with 128 bits by comparing the consideration of only the nearest prototype versus considering all prototypes. Specifically, we replace the KLM loss with the KLM* loss when mining the knowledge library, and the KLM* loss can be formulated as:\n\n$L_{klm^*} = -\\frac{1}{n_m} \\sum_{i=1}^{n_m} \\log \\left( \\frac{1}{K} \\sum_{k=1}^{K}  s\\left(x_m^i | k\\right) \\right).$\n\nThe results are shown as follows: \n\n|Loss|XMedia|XMediaNet|\n|:-:|:-:|:-:|\n|$L_{klm*}$|73.3|46.9|\n|$L_{klm} $|75.0|51.0|\n\nThe results show that only considering the nearest prototype with the same category when mining the knowledge library can enjoy a clear optimization objective and a better performance.\n\n**R-Suggestion-1**: We provide detailed explanations of key loss terms in **`R-Weakness-3’** of [Reviewer TSiu](https://openreview.net/forum?id=JqLKV0L5hM&noteId=KyOMFKYOSw).\n\n**R-Suggestion-2**: We investigate the latest studies and found that there are no other CMH works to handle cross-modal retrieval (CMR) in streaming-media scenarios (SMC). Here, we discuss some real-valued representation-based CMR methods, including SDML[1] and DRCL[2], which are specifically designed to accomplish CMR in SMC. 1) These methods are real-valued based representations, which involve high computational complexity and memory overhead, making it difficult to meet the fast retrieval demands of large-scale datasets. 2) They deploy either a randomly initialized common space or an identical transformation weight matrix to guide semantic alignment, which overlooks the discrepancies among modalities. Since the randomly initialized common space or learned transformation weight matrix may retain modality-specific features that are not very relevant to the new modality, directly applying it may lead to semantic discrepancies. By contrast, our SHE transfers the baseline knowledge library to the new modality, thereby obtaining more semantically consistent representations.\n\n**R-Question-1**: The KLM module can mine a knowledge library to preserve the semantic information of streaming data, which can serve as a medium to maintain semantic consistency while reducing the computational burden of maintaining historical data. The KLT module can transfer semantic information extracted from new modalities to the benchmark knowledge library. This allows our SHE to process new media without retraining the entire historical media data, thereby reducing computational complexity while ensuring semantic consistency between streaming modal data.\n\n**R-Question-2**: We provide the explanation of why only considering the nearest prototype when mining the knowledge library, and perform a comparison with other methods. Please see **`R-Weakness-2’** for details.\n\n**R-Question-3**: To improve the efficiency of handling streaming-media data, MARS leverages the shared label parsing module to achieve semantic alignment without interaction across modalities. However, the label parsing module may retain modality-specific features that are less relevant to the new modality, and directly using it may lead to semantic bias. By contrast, our SHE proposes a KLT module to transfer the baseline knowledge library to the new modality, thereby obtaining more semantically consistent representations. To further demonstrate the advantages of SHE, we integrate the idea of using a knowledge library as a medium to maintain semantic consistency into MARS, and found it beneficial to do so. For detailed experiments and analysis, see **`R-Question-2’** in [Reviewer UdWL](https://openreview.net/forum?id=JqLKV0L5hM&noteId=G4Nh3s9sQe).\n\n**Reference**\n\n[1] Hu P, Zhen L, Peng D, et al. Scalable deep multimodal learning for cross-modal retrieval[C]//Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval. 2019: 635-644.\n\n[2] Pu R, Qin Y, Peng D, et al. Deep Reversible Consistency Learning for Cross-modal Retrieval[J]. IEEE Transactions on Multimedia, 2025.", "claims_and_evidence": "Yes, the claim that SHE can train all modalities in parallel and improve the flexibility and scalability of cross-modal retrieval is supported by extensive experimental evidence on four datasets. The improvement in retrieval performance is shown in experimental results.", "methods_and_evaluation_criteria": "Yes, the proposed SHE is suitable for solving the cross-modal retrieval problem under streaming-media data. The selected baseline datasets and evaluation metrics such as the Mean Average Precision (MAP) score are suitable for evaluating the superiority of the proposed SHE. This paper adopts a reasonable experimental setting and compares it with the state-of-the-art methods, which is helpful to verify the effectiveness of SHE.", "theoretical_claims": "Yes, the theoretical claims that SHE mines a knowledge library for each modality as a medium for maintaining semantic consistency is feasible. This is because the knowledge library can capture the common distribution information in streaming-media data, and the knowledge libraries of different modalities are aligned through the Knowledge Library Transfer (KLT) module, and using the knowledge library as a medium can ensure the consistency of cross-modal semantics.", "experimental_designs_or_analyses": "Yes, the experimental setting is reasonable. The dataset containing five modalities is used for the experiment and compared with other state-of-the-art (SOTA) methods to fully demonstrate the superiority of SHE under streaming-media data. Additionally, The ablation experiment demonstrates the effectiveness of each component of SHE.", "relation_to_broader_scientific_literature": "This paper positions the proposed Streaming-Media Hashing rEtrieval (SHE) within the field of cross-modal retrieval (CMR), comparing SHE with numerous cross-modal hashing (CMH) methods and three real-valued CMR methods. The study underscores the limitations of existing CMH methods in handling streaming-media data and introduces SHE as a novel hashing framework to address these challenges.", "essential_references_not_discussed": "Key references are cited in this paper, but the authors are recommended to cite and discuss more cross-modal retrieval methods that specifically handle streaming-media data to enrich the research context of the current work.", "other_strengths_and_weaknesses": "Strengths: \n\n1. This paper introduces a Streaming-media Hash rEtrieval (SHE) framework specifically designed for streaming-media applications, offering a unique and innovative research perspective while demonstrating its practical value in the field of cross-modal retrieval.\n2. The experimental results show that SHE achieves superior retrieval performance across multiple benchmark datasets. Additionally, ablation studies confirm the contributions of each module to the overall model performance. \n3. This paper demonstrates a high degree of clarity in its writing style and organizational structure, with well-articulated and explicit experimental motivations.\n\nWeaknesses: \n1. Some loss function formulations are relatively complex, which may hinder the understanding of the core optimization objectives. The authors should provide more detailed explanations of key loss terms to enhance readability.\n2. The authors do not explain why only the nearest prototype is considered when mining the knowledge library.", "other_comments_or_suggestions": "1. The authors are recommended to provide more detailed explanations of key loss terms to enhance readability and help readers better grasp the design motivations.\n2. The authors are recommended to provide more discussion and comparison with more cross-modal retrieval methods that specifically handle streaming-media data.", "questions_for_authors": "1. This paper mines a knowledge library from each modality by KLM, and then aligns the knowledge libraries of different modalities through the KLT module to achieve semantic alignment. Can the author explain the benefits of doing so?\n2. When mining the knowledge library, this paper only considers the nearest prototype with the same category. What are the benefits of doing so and can it be compared with other methods?\n3. Apart from retrieval efficiency, could the authors provide other advantages of SHE compared to MARS?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces SHE, a novel cross-modal hashing framework tailored for streaming-media scenarios where multimodal data arrive asynchronously. By integrating Knowledge Library Mining, Knowledge Library Transfer, and Discriminative Hash Learning, SHE enables parallel training across modalities while preserving semantic consistency. Extensive experiments on four benchmark datasets demonstrate its advantages in retrieval performance, scalability, and storage efficiency.

Reviewers agreed that the paper addresses a significant and previously underexplored problem with a well-motivated and technically sound solution. Strengths include its practical relevance, strong empirical results, and comprehensive ablation studies. Concerns regarding the clarity of loss function definitions and the handling of low-quality initial modalities were addressed in the rebuttal.
Decision: Accept (poster)