# A Cross-Modal Hash Retrieval Method with Fused Triples
## 4 A Cross-Modal Hash Retrieval Method with Fused Triples
### Feature Extraction  
In the feature extraction part, the data set selection uses triples data selection. The information based on the label is used to close the distance of similar samples while pushing the distance of different samples farther to filter out similar information to the semantics of the label. Afterward, the image modality and text modality were extracted using the convolutional neural network (CNN) model [28] and multi-layer multivariate long- and short-term memory network model (Multi-LSTM) [29] for feature extraction.  
According to two excellent deep cross-modal hashing algorithms in [2,15], for the feature extraction of image modalities, we chose an eight-layer CNN model, including five convolutional layers and three fully connected layers. Figure 1 illustrates the feature extraction process of CNN for image modality. The model's convolutional layers are identified as \"conv1-conv5,\" and its fully connected layers as \"full6-full8\", Table 1 depicts its structure.  
Where \"f. num \(\times\) size \(\times\) size\" indicates the number and size of the convolution kernels. \"s\" denotes the step size of the convolution. \"pad\" shows the number of pixels to fill. \"LRN\" denotes regularization to increase generalization performance. The \"pool\" indicates the pooling process. The value in the fully connected layer indicates the number of nodes. It is also the output dimension of the layer.  
For text modal feature extraction, the text data vectorizing uses a bag-of-words model, and then feature extraction is performed using Multi-LSTM. Three LSTM layers, one fully connected layer, and one one-dimensional convolutional layer comprise the multi-layer, multi-variable, more extended short-term memory network model, known as Multi-LSTM. Each LSTM layer has an input gate, an oblivion gate, an output gate, and a memory unit. The oblivion gate discards useless information and the input gate controls the memory unit to store helpful information. The output gate controls the output of useful information. Multi-LSTM combines the advantages of each layer to avoid the gradient disappearance and explosion problem to a certain extent and to obtain the semantic feature information of text modality. Figure 1 illustrates the feature extraction process of Multi-LSTM for text modalities. Using the network model for the feature extraction of text modes is superior to simple network models, such as LSTM. The Multi-LSTM model used in this paper has  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Layer** & **Structure** \\ \hline Conv1 & f.64 \(\times\) 11 \(\times\) 11; s.4 \(\times\) 4, pad0, LRN, \(\times\)2 pool \\ \hline Conv2 & f.256 \(\times\) 5 \(\times\) 5; s.1 \(\times\) 1, pad2, LRN, \(\times\)2 pool \\ \hline Conv3 & f.256 \(\times\) 3 \(\times\) 3; s.1 \(\times\) 1, pad1 \\ \hline Conv4 & f.256 \(\times\) 3 \(\times\) 3; s.1 \(\times\) 1, pad1 \\ \hline Conv5 & f.256 \(\times\) 3 \(\times\) 3; s.1 \(\times\) 1, pad1, \(\times\)2 pool \\ \hline full6 & 4096 \\ \hline full7 & 4096 \\ \hline full8 & Hash code length c \\ \hline \hline \end{tabular}
\end{table}
Table 1: Convolutional neural network model structure.  
three LSTM layers, and the best training results and minor errors are achieved when the number of neurons in the hidden layer is 64, 256, and 64.