# Multi-Grained Similarity Preserving and Updating for Unsupervised Cross-Modal Hashing
## 2 Related Work
### Unsupervised Cross-Modal Hashing  
Unsupervised cross-modal hashing methods do not rely on labels so as to apply in real-world scenarios easily. In the existing solutions, constructing a semantic similarity matrix is the key to guide cross-modal relationship learning.Among them, an ingenious method is Deep Joint Semantic Reconstruction Hash (DJSRH [12]), which proposes a joint semantic similarity matrix to simultaneously integrate multi-modal similarity information  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Type** & **Reference** & **Source** & **Authors** & **Description** \\ \hline \multirow{8}{*}{Unsupervised} & \multirow{2}{*}{DJSRH [12]} & \multirow{2}{*}{ICCV (2019)} & \multirow{2}{*}{[PERSON], [PERSON], et al.} & The potential semantic relationships of data are captured by constructing a joint semantic matrix, but redundant information is introduced. \\  & & & & The proposed sampling weighting scheme can generate more efficient hash codes. \\  & & & & Design information mixed similarity \\  & & & & estimation integrates distance distribution and similarity rate information. \\  & & & & Using a graph model to integrate \\  & & & & neighborhood information, the overall loss is maintained by designing three types of data similarity. \\  & & & & The semantic structure of data is mined by \\  & & & & GCN, and the structural information is \\  & & & & explored from multiple perspectives by \\  & & & & using a variety of similarity measures. \\  & & & & Cross-modal triples are used to maintain \\  & & & & data similarity, and the distance between \\  & & & & similar samples is required to be smaller \\  & & & & than that between dissimilar pairs. \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Cont._of cross-modal instances. However, the similarity matrix in DJSRH introduces redundant information from intra-modal fusion items. As an improvement, joint-modal distribution-based similarity hashing (JDSH [36]) involves distribution-based similarity decision and weighting to learn more discriminative hash codes. Deep adaptively-enhanced hashing (DAEH [27]) utilizes distance distributions and similarity ratio information to estimate comparable similarity relationships as complementarity of simple feature distance-based metrics. Although competitive performance is achieved, lack of accurate similarity measurement are these methods since the similarity matrix they rely on are constructed from the original features with noises.  
Other unsupervised learning studies make efforts on narrowing the heterogeneity gap by preserving similarity structure consistency. For example, deep graph-neighbor coherence preserving network (DGCPN [48]) explores the relationship information between data and their neighbors to capture neighborhood structure coherence. Aggregation-based graph convolutional hashing (AGCH [26]) employs a GCN to deeply explore the underlying neighborhood structure. Deep relative neighbor relationship preserving hashing (DRNPH [49]) method excavates deep relative neighbor relationships in common Hamming space via binary feature vector based intra- and inter-modal neighbor matrix reconstruction. Impressive progress had been made by these works, however, a common weakness they suffer from is lack of both global and local similarity learning from multi-modal contents.  
To address the above shortcomings, we attempt to boost cross-modal hashing learning from two aspects: (1) trying to rise the quality of similarity matrix by reducing noises from original features, and (2) inviting multi-grained similarity learning strategy to capture both global and local similarity relationships. As a results, a novel unsupervised cross-modal hashing technique called multi-grained similarity preserving and updating is developed. The detailed details of this approach will be explored in depth in the next section.