# AutoRet: A Self-Supervised Spatial Recurrent Network for Content-Based Image Retrieval
## 3 Methodology
### Self-Supervision through AutoEmbedder  
The retrieval system is trained using Autoembedder architecture so that it can generate embeddings based on similarity. Generally, Autoembdder architecture is trained based on pairwise constraints. The policy is based on a siamese network that can be defined as follows,  
\[\mathcal{S}(x,x^{\prime})=ReLU(\|\mathcal{E}_{\phi}(x)-\mathcal{E}_{\phi}(x^{ \prime})\|,\alpha)=\mathbb{R}_{\leq\alpha}^{+} \tag{1}\]  
The \(ReLU(\cdot,\cdot)\) function used in Equation (1) is a thresholded ReLU function, such that,  
\[ReLU(x,\alpha)= \begin{cases}x&\text{if }0\leq x<\alpha\\ \alpha&\text{if }x\geq\alpha\end{cases}  \tag{2}\]In Equation (1), the \(\mathcal{S}(\cdot,\cdot)\) is a siamese network receiving a pair of input data \(x\) and \(x^{\prime}\). An embedding model is indicated by \(\mathcal{E}_{\phi}\), which generates embedding for a given input image. The architecture of the embedding model (\(\mathcal{E}_{\phi}\)) is explained in Section 3.2. The Autoembedder strategy involves training embedding models based on the pair-wise constraint. For a given training batch, half of the pairwise data contains similar image pairs, whereas the other half contains non-similar image pairs. The training target is to produce closer embeddings for a given pair of similar images. Therefore, the euclidean distance would be close to zero for a pair of a similar image. In contrast, for non-similar image pairs, the target is to produce embeddings at a minimum distance of \(\alpha\). The training policy based on similarity is generated based on randomization and augmentation. Such idea of training is termed as self-supervised learning, where each piece of data is given a pseudo label or trained based on random augmentation [15]. Autoembedder strategy has already been explored for training using pseudo labels [49]. Figure 2 explains the self-supervised training process. The data selection process of the self-supervised strategy can be explained by the following two points:  
* Can-link pair: For a given pair of similar images (containing equivalent content), the embedding system should generate closer embeddings-pairs. Image pairs with such a relationship are defined as can-link pairs. Half of the training data are randomly selected to generate a can-link pair with similar image pairs. If the data labels are unknown, a can-link pair can be generated using the raw image and an augmented version of that image. For augmentation, basic types of augmentation techniques, shear, random contrast/brightness, random crop, rotate, flip, noise is used. Figure 1: The training policy of AutoRet only includes DCNN architecture using the AutoEmbedder framework (illustrated in (a)). After completing the training, the trained DCNN architecture is directly used to generate image embeddings. The image embeddings of the query image and retrievable images are stored. Finally, a similarity search is conducted using a nearest neighbor algorithm based on the query image’s embedding (illustrated in (b)). * Cannot-link pair: For a given pair of dissimilar images (containing different content), the embedding system should generate distant embedding points. Image pairs with no content relationship are defined as cannot-link pairs. Half of the training data are randomly selected to generate a cannot-link pair representing dissimilar image pairs. For a given image, another randomly selected image is used for generating a cannot-link pair. In the training strategy, the cannot-link pair can be erroneous, as a randomly selected image can often be of a similar class. If the number of errors in the cannot-link is huge, it would be impossible for the embedding network to converge to its optimal. Let us consider a dataset \(D\) consisting of \(N_{c}\) classes where each class contains a uniform number of data \(N_{p}\). The probability of selecting an erroneous pair (\(S_{e}\)) is,  
\[