# TDCMR: Triplet-Based Deep Cross-Modal Retrieval for Geo-Multimedia Data
## 3 Preliminaries
### Problem Definition  
The cross-modal hash algorithm solves the problem of uniform representation and mutual retrieval of multi-modal data through hash learning of multi-modal data. Assuming a given set of training data containing \(n\) sets \(R=\{T_{i},I_{i},t_{i}\}_{i=1}^{n}\), the \(i\)-th set of training data \begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Definition** \\ \hline \(O\) & Space multimedia datasets \\ \hline \(o\) & geo-multimedia Data Objects \\ \hline \(o.ulD\) & Identification code \(o\) spatial objects \\ \hline \(o.loc\) & Geographical location information \(o\) spatial objects, composed of \\  & longitude \(o.lng\) and latitude \(o.lat\). \\ \hline \(o.m\) & Multimedia Data Information \(o\) Space Objects \\ \hline \(o.v^{T}\) & Cross-modular hash code for spatial objects \(o\) text modes \\ \hline \(o.v^{l}\) & Cross-modular hash code for spatial objects \(o\) image modes \\ \hline \(q\) & geo-multimedia data \(k\) nearest neighbor query \\ \hline \(q.loc\) & Geographical location information \(q\) query objects \\ \hline \(q.m\) & Multimedia Data Information \(q\) Query Object \\ \hline \(f_{s}(q,o)\) & Spatial similarity between \(q\) and \(o\) \\ \hline \(f_{s}(q,o)\) & Semantic similarity between \(q\) and \(o\) \\ \hline \(F_{GM}(q,o)\) & \(q\) and \(o\) geo-multimedia data similarity \\ \hline \(H_{TDCMR}(\cdot)\) & TDCMR cross-modal hash function \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The summary of notations. is composed of text modal data \(T_{i}\), image modal data \(I_{i}\), and category label \(t_{i}\). Tuple class label subscripts constitute datasets \(\Gamma=\{a_{i},p_{i},n_{i}\}_{i=1}^{m}\), and the label subscript \((a_{i},p_{j},n_{i})\) of the \(j\)-th group of triples indicates that the similarity between anchor sample \(a_{i}\) and positive sample \(p_{i}\), which is higher than that between anchor sample and negative sample \(n_{i}\). That is, samples with \(a_{j}\) and \(p_{j}\) have common category labels, while samples \(a_{j}\) and \(n_{j}\) do not have common category labels. **Definition 1** (Spatial similarity). : _Given a geo-multimedia query \(q=(q.loc,q.m)\) and a space object \(o\), then, spatial similarity between \(q\) and \(o\) is defined as ratio of euclidean space distance \(\delta(q,o)\) between \(q\) and \(o\) to maximum euclidean space distance \(\delta_{max}(q,O)\), which can be expressed as:_  
\[f_{s}(q,o)=1-\frac{\delta(q.loc,o.loc)}{\delta_{max}(q,O)},\]  
_where \(\delta(q.loc,o.loc)\) represents the euclidean distance between query \(q\) and spatial object \(o\), and \(\delta_{max}(q,O)\) represents the maximum euclidean distance between a query \(q\) and any spatial object in the spatial object dataset \(O\), which can be expressed as:_  
\[\delta(q.loc,o.loc)=\sqrt{(q.lng-o.lng)^{2}+(q.lat-o.lat)^{2}}\]  
\[\delta_{max}(q,O)=max(\{\delta(q.loc,o.loc)\}|\forall o\in O\}).\]  
**Definition 2** (Cross-modal semantic similarity). : _Given a geo-multimedia Query \(q=(q.loc,q.m)\) and a space object \(o\), then, the cross-modal semantic similarity between \(q\) and \(o\) is defined as the cosine value of the \(TDCMR\) hash code and the spatial object, which can be expressed as:_  
\[f_{c}(q,o)=\frac{\sum_{i\in q,o}\overline{q}.\overline{v}^{T(i)} \times o.\overline{v}^{I(i)}}{||q.\overline{v}^{T}||\times||o.\overline{v}^{ I}||}, \tag{1}\]  
_where \(\overline{v}^{T}\) represents text mode TDCMR hash code, \(v^{I}\) represents image mode TDCMR hash code, and \(||q.\overline{v}^{T}||\) and \(||o.\overline{v}^{I}||\) are query object and space object TDCMR Hash code module._  
**Definition 3** (geo-multimedia Data Similarity).