# A Cross-Modal Hash Retrieval Method with Fused Triples
###### Abstract  
Due to the fast retrieval speed and low storage cost, cross-modal hashing has become the primary method for cross-modal retrieval. Since the emergence of deep cross-modal hashing methods, cross-modal retrieval significantly improved. However, the existing cross-modal hash retrieval methods still need to effectively utilize the dataset's supervisory information and the lack of similarity expression ability. This means that the label information needs to be maximized, and the potential semantic relationship between two modalities cannot be fully explored, thus affecting the judgment of semantic similarity between two modalities. To address these problems, this paper proposes Tri-CMH, a cross-modal hash retrieval method with fused triples, which is an end-to-end modeling framework consisting of two parts: feature extraction and hash learning. Firstly, the multi-modal data are preprocessing into the form of triple groups. The data supervision matrix is constructed so that the samples with labels and their meanings are aggregated together. In contrast, the samples with labels and their opposite meanings are separated, thus avoiding the problem of the under-utilization of supervisory information in the data set and achieving the effect of efficiently utilizing the global supervisory information. Meanwhile, the loss function of the hash learning part is optimized by considering the Hamming distance loss, single-modality internal loss, cross-modality loss, and quantization loss to explicitly constrain semantically similar hash codes and semantically dissimilar hash codes and to improve the model's ability to judge cross-modality semantic similarity. The method is trained and tested on the IAPR-TC12, MIRFLICKR-25K, and NUS-WIDE datasets, and the experimental evaluation criteria are mAP and PR curve, and the experimental results show the effectiveness and practicality of the method. t 2023 20  
manual feature and hash code learning based on hash methods are independent and can lead to poor retrieval performance. With the development of deep understanding, deep neural networks are widely used for feature learning. Compared with earlier hashing methods, the deep cross-modal hashing model puts modal features and hash functions in an end-to-end framework to learn together, which solves the problem of incompatibility between manual features and hash code learning and makes the learned hash code more efficient. Thus, The hashing algorithm is suitable for single-modal and multimodal semantic similarity retrieval. Two main existing multimodal hashing algorithms exist, namely multi-source hashing [6] and cross-modal hashing [1; 7; 8], with cross-modal hashing methods performing relatively well in cross-modal retrieval tasks. The key to the cross-modal retrieval approach is to construct the latent semantic space so that each point has definite semantic information that reflects the mapping relationship between the two modalities, thus performing similarity calculations on the data in the space [9]. Most deep cross-modal hashing methods symmetrically learn hash codes, i.e., the hash codes of query instances and database instances are learned in the same way, the training process is very time-consuming, the supervisory information is difficult to make full use of during the training process, and the generalization ability of the learned hash codes is not strong. Thus, The majority of cross-modal retrieval methods currently have two issues. The first is that features are extracted directly from the data set without processing, and the second is that only the neural network's training speed is considered. As a result, the supervisory information in the data could be utilized more effectively in the training process, and the label information is not utilized to its fullest extent. Another problem is that the private representation within the modalities is eliminated to facilitate the network's training on different modalities.