# Multi-Grained Similarity Preserving and Updating for Unsupervised Cross-Modal Hashing
## 1 Introduction  
As an efficient information retrieval paradigm for big multimedia data [1; 2; 3], cross-modal retrieval [4; 5; 6; 7; 8; 9] utilizes one modality as a query to search another modal data. Among the existing techniques [10; 11; 12; 13; 14; 15; 16; 17; 18], cross-modal hashing [16; 17; 18; 19; 20] is popular for its fast retrieval speed and low storage cost. The core of cross-modal hashing is to map high-dimensional data to a low-dimensional common Hamming space, in which multi-modal instances (text, image, audio and video) with similar semantics are closer, and dissimilar instances are far apart. Notwithstanding the strong practical significance, this task is extremely challenging due to the heterogeneity gap between different modalities. A number of studies are striving to promote the progress of cross-modal hashing. In general, they can be simply divided into two groups: supervised learning methods [21; 22; 23; 24; 25] and unsupervised learning methods [26; 27; 28; 29; 30]. The supervised learning methods use human-annotated label to guide model learning, which preserves semantic discrimination well to obtain great performance. However, directly obtaining the category information is really difficult in real-world scenarios, while the cost of manual labeling is so huge as to limit the wide application. In contrast, without relying on category annotations, unsupervised learning methods are great at capturing the underlying correlation patterns between different modalities to bridge the heterogeneity gap. Thus, they have gained considerable attention currently. **Motivation. ** Although existing researches have shown remarkable results on large-scale datasets, unsupervised cross-modal hashing is yet a Herculean task that await further exploration due to the following challenges. _The first challenge is how to accurately measure inter-modal and intra-modal similarity without category information_. In the early days, severalunsupervised learning methods [31; 32] used discrete models to calculate binary similarity, namely, defining the similarity between two instances with small distances as \"1\", otherwise \"\(-\)1\". Unfortunately, the naive similarity measurement is far from accurate. To break this limitation, researchers [12; 15; 33; 34] began to use original feature distance to measure the continuous similarity. However, such similarity is still not accurate enough to represent the real relationship between instances due to the noise existing in their original features. _The second challenge is how to preserve similarity structure consistency from both local and global view during hash learning process_. Undisputedly, using multi-grained similarity relationships across modalities is crucial to narrow cross-modal heterogeneity. However, current methods [35; 36; 37] either overlook multi-grained similarity learning across various modalities or struggle to capture a consistent distribution of relations within the content details. Additionally, some state-of-the-art methods [38; 39; 40; 41] do not account for the semantic correlation between feature vectors from one modality and the corresponding hash codes from another modality. To conquer these challenges, this paper has made the following two efforts: (1) designing a novel and effective similarity updating strategy to rise the accuracy of underlying similarity measurement, and (2) combining multi-grained similarity information from local and global view to preserve similarity consistency. **Our Method. ** We propose a novel cross-modal hashing framework, named **M**ulti-**G**rained **S**imilarity **P**reserving and **U**pdating (**MGSPU**). As shown in Figure 1, this framework consists four module, i.e., feature extraction, semantic similarity matrix updating, hash code learning and hash function learning. To overcome the first challenge, a novel semantic similarity matrix update strategy is developed, which removes noises from the similarity matrix so as to obtain high-confidence supervisory signal.