# Transformer-Based Distillation Hash Learning for Image Retrieval
## 4 Experiment and Analysis
### Comparison to the State of the Art  
In this session, we compared our method with the state-of-the-art methods performed using the same evaluation metrics. The contrasting methods include LSH [42], ITQ [16], SH [15], PCAH [43], SpH [44], DH [45], DeepBit [46], DSH [47], BDNN [48], DVB [49] and DOH [50]. The LSH, ITQ, SH, PCAH, SpH, and DSH are not deep learning based hashing methods. We extracted depth features from the pre-trained ResNet model [27] and used them as input for these methods in order to make a fair comparison. For the deep learning-based methods, i.e., DH, DeepBit, BDNN, and DOH, we evaluated them with the hyperparameter settings suggested in their papers and ran the source codes provided by the authors. For DVB [49], we directly refer to the results of the original paper. The MAP results calculated using different lengths of hash codes on two datasets, MIRFlickr-25K and NUS-WIDE, are reported in Table 3. The reported results show the superiority of the proposed method and validate that the motivation of the proposed method is valid. Furthermore, by incorporating the self-attention mechanism of the Transformer model, image hash codes can be made global and unique. At the same time, this advantage can be instilled into the efficient lightweight model by knowledge distillation, which reduces the computational effort and has the feature advantage of the Transformer. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Teacher** & **Train Data** & **Length** & **MAP** \\ \hline \multirow{4}{*}{Student} & None & & 24 & 0.6678 \\  & Transformer & & 24 & 0.7112 \\  & None & MIRFlickr-25K & 48 & 0.6561 \\  & Transformer & & 48 & 0.7088 \\ \hline \multirow{4}{*}{Student} & None & & 24 & 0.5321 \\  & Transformer & & 24 & 0.6473 \\ \cline{1-1}  & None & & 48 & 0.5396 \\ \cline{1-1}  & Transformer & & 48 & 0.6482 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Distillation experiment with the Transformer-based model as the teacher and the linear projection-based fast model as the student. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Methods** & \multicolumn{3}{c}{**MIR Flickr-25K**} & \multicolumn{3}{c}{**NUS-WIDE**} \\ \hline Length & 12 & 24 & 32 & 48 & 12 & 24 & 32 & 48 \\ \hline LSH [42] & 0.5763 & 0.6065 & 0.5966 & 0.6263 & 0.3523 & 0.4096 & 0.4186 & 0.4555 \\ \hline SH [15] & 0.6621 & 0.6433 & 0.6296 & 0.6225 & 0.5652 & 0.5061 & 0.4866 & 0.4546 \\ \hline SpH [44] & 0.5982 & 0.5832 & 0.5831 & 0.582 & 0.4656 & 0.4662 & 0.4473 & 0.4481 \\ \hline ITQ [16] & 0.6932 & 0.7082 & 0.6686 & 0.6991 & 0.6332 & 0.6255 & 0.5922 & 0.6481 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: MAP results for the MIR Flickr-25K and NUS-WIDE datasets, using hash codes of different lengths, were calculated using the first 5000 images retrieved. Figure 3 and 4 show the performance curves of retrieval results on the MIRFlickr-25K dataset and the NUS-WIDE dataset. It can be seen from these two figures that the proposed method outperforms all the compared methods on both datasets. The results express the superiority of our method over the compared methods.