# Structural Correlation Based Method for Image Forgery Classification and Localization
## 3 Bag-of-Features and Hamming Embedding Based Image Retrieval  
In image retrieval, images are represented by descriptive features. The features are used to evaluate similarity or dissimilarity between images. In the image forgery, since the forged regions may be rotated, scaled, and translated in different manners, the features of the images should be invariant to these transformations. The features generated by SIFT [33] have such noteworthy characteristics and the proposed algorithm utilizes the SIFT features to represent images [34; 35]. In this section, we briefly review the image retrieval method based on BOF [36; 37; 38] and HE encoding [38; 39]. Suppose that a query image \(\mathbf{Q}\) is represented by a set of \(N\) descriptors, \(\mathbb{X}^{\mathbf{Q}}=\left\{x_{1}^{\mathbf{Q}},x_{2}^{\mathbf{Q}},\ldots,x_ {N}^{\mathbf{Q}}\right\}\). All of these descriptors are mapped into a visual vocabulary set \(\mathbb{W}=\left\{w_{1},w_{2},\ldots,w_{K}\right\}\) by a \(K\)-means vector quantizer \(q\). For example, \(q\) maps \(x_{n}^{\mathbf{Q}}\)\((n=1,2,\ldots,N)\) to the closest visual word \(w_{k}\)\((k=1,2,\ldots,K)\), where \(q(x_{n}^{\mathbf{Q}})=w_{k}\in\mathbb{W}\). We define a set of descriptor indexes, which assigns descriptors of \(\mathbf{Q}\) to a particular visual word \(w_{k}\) as \(\mathbb{I}_{k}^{\mathbf{Q}}=\left\{n\left|q\left(x_{n}^{\mathbf{Q}}\right)=w_{ k}\right.\right\}\). A matching model HE is used to estimate the matching of descriptors to a visual word. HE represents each descriptor as \(D\)-dimensional binary signatures [38]. Let \(b_{n_{d}}^{\mathbf{Q}},1\leq d\leq D\), is a single bit binary code used to represent \(x_{n}^{\mathbf{Q}}\), then \(b_{n}^{\mathbf{Q}}=\left\{b_{n_{1}}^{\mathbf{Q}},b_{n_{2}}^{\mathbf{Q}},\ldots,b_{n_{d}}^{\mathbf{Q}},\ldots,b_{n_{D}}^{\mathbf{Q}}\right\}\) is a binary signature of descriptor \(x_{n}^{\mathbf{Q}}\). The Hamming distance between two descriptors, \(x_{m}^{\mathbf{Q}}\) and \(x_{n}^{\mathbf{Q}}\), is computed using their binary signatures as follows:  
\[h\left(b_{m}^{\mathbf{Q}},b_{n}^{\mathbf{Q}}\right)=\sum_{d=1}^{D}\left|b_{m_{ d}}^{\mathbf{Q}}-b_{n_{d}}^{\mathbf{Q}}\right|. \tag{1}\]  
Let us denote \(\mathbb{X}^{\mathbf{P}}\) be the set of descriptors of the database image \(\mathbf{P}\). The probability that two sets of descriptors, \(\mathbb{X}^{\mathbf{Q}}\) and \(\mathbb{X}^{\mathbf{P}}\) are assigned to the same visual word \(w_{k}\) is defined as:  
\[P_{k}(\mathbb{X}^{\mathbf{Q}},\mathbb{X}^{\mathbf{P}})=\sum_{i\in\mathbb{I}_{ k}^{\mathbf{Q}}}\sum_{j\in\mathbb{I}_{k}^{\mathbf{P}}}f\left(h\left(b_{i}^{ \mathbf{Q}},b_{j}^{\mathbf{P}}\right)\right), \tag{2}\]  
where the weighting function for a Hamming distance \(h\) is calculated as a Gaussian function [38]:  
\[f(s)= \begin{cases}e^{-h^{2}/\sigma^{2}},&\text{if }h\leq 3\sigma/2,\\ 0,&\text{otherwise}.\end{cases}  \tag{3}\]  
The number of dimensions for the binary signatures is typically set to \(D=64\), and the Gaussian bandwidth parameter [38; 40] is set to \(\sigma=D/4=16\). In order to retrieve images, an inverted index file is built in the image indexing process. The inverted file consists a list of entries. In each entry, a visual word is stored along with the identifier of associated images, descriptors of those images which are assigned to the visual word and the HE used for matching measurement. When the query of \(\mathbf{Q}\) is performed, the entries of visual words associated to \(\mathbf{Q}\) are searched in the inverted file. The score of a database image \(\mathbf{P}\) in this query is calculated by accumulating the Hamming distances between two sets of descriptors' signatures for all the shared visual words of two images.