# Multi-Label Weighted Contrastive Cross-Modal Hashing
## 1 Introduction  
With the prosperity of multimedia technology and smart devices, a tremendous number of multi-modal data (e.g., text, image, video, and audio) have been pouring into the Internet [1; 2; 3; 4; 5]. Despite differences in structures, various types of data are usually semantically related to each other. These semantic relationships could be used for data retrieval or sharing. Naturally, cross-modal retrieval technology [6; 7; 8; 9] has become a desiderata since it efficiently returns one modality as a result to queries of other modalities by effectively mining the intrinsic semantic relationships. The primary issue of cross-modal retrieval is to reduce the heterogeneity gap between modalities. Most existing approaches try to address this issue by projecting the original features of data into a common real-valued subspace in which the semantic similarity can be easily measured [10; 11; 12; 13; 14; 15; 16; 17; 18; 19]. Unfortunately, due to the explosive growth of data, the computational complexity of real-valued cross-modal retrieval has become an unavoidable challenge. A viable solution is cross-modal hashing [20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30], which maps high-dimensional multi-modal features into compact binary codes and the cross-modal similarity can be calculated by XOR operation efficiently. Depending on whether category information is used during the training stage, existing cross-modal hashing methods are mainly classified into unsupervised and supervised manners. Generally, without category information, it is difficult for unsupervised cross-modal hashing [31; 32; 33; 34; 35; 36; 37; 38] to generate cross-modal hash codes with strong semantic discrimination even though they endeavor to learn latent similarity structures among different modalities. Supervised cross-modal hashing [39; 40; 41; 42; 43; 44; 45], in contrast, is able to use category labelsto enhance cross-modal semantic discrimination so as to obtain high-quality hash codes. As a special case of supervised hashing learning, multi-label cross-modal hashing methods [29; 46; 47; 48; 49; 50] aim to efficiently handle the instances associated with multiple labels [51; 52; 53]. Unlike traditional single-label hashing methods that focus on binary similarity for individual instances, they use multiple labels to construct a semantic similarity matrix so as to learn more accurate similaritiy relationships (e.g., each similarity is defined as a real value between \(-1\) and \(1\)). Meanwhile, motivated by the remarkable achievements in contrastive learning, some contrastive learning methods [30; 54] for cross-modal hashing are introduced which aim to capture cross-modal similarities more effectively by comparing samples across modalities. **Motivation. ** Recently, some great progress has been made in cross-modal hashing with contrastive learning, such as [30; 54]. UCCH [54] represents the initial endeavor in employing contrastive learning within unsupervised cross-modal hashing. It introduces a cross-modal ranking learning loss (CRL) to alleviate the influence of false-negative pairs. Conversely, Unihash [30] leverages a contrastive label correlation learning (CLC) loss to establish connections between diverse modalities through category labels. However, they simply apply InfoNCE [55; 56] in cross-modal hashing, which treats an image-text pair as a positive sample; otherwise, it is a negative sample. This contrastive learning strategy gives rise to the false-negative problem, where samples belonging to the same class are incorrectly regarded as negative samples, resulting in the learning of erroneous relationships among cross-modal instances. Hence, a widely adopted approach in supervised contrastive learning [57] is to consider instances as similar if they share at least one common category.