# Multi-Subject Image Retrieval by Fusing Object and Scene-Level Feature Embeddings
## 2 Related Work  
The content-based image retrival (CBIR), a steady and predominantly treated study, is a problem of finding images with visual content similar to a given query image. The scale-invariant feature transform (SIFT) [18] was combined with the bag-of-words (BoW) model and used for image retrieval and classification. BoW is the proposed model for modeling documents. Because a document is divided into words, the document can be represented as a single histogram by using the frequency of words. Bag-of-visual-words (BoVW) [13; 14; 15] is a method of dividing images into visual words and utilizing the generated histogram as an image representation in Figure 2. SIFT is used to generate local features used as visual words. Since AlexNet [19] has achieved significantly higher recognition accuracy on ILSRVC 2012, image retrieval research began to focus on deep learning-based methods [20; 21; 22; 23; 24; 25], especially using convolutional neural networks (CNNs). CNN has outperformed hand-crafted features in many computer vision tasks. For image retrieval, it also shows competitive performance while using shorter feature vectors compared to BoVW models. In this work, we use pre-trained CNNs for comparative experiments with proposed methods. Image retrieval may be classified into various types according to the purpose. This study focuses specifically on CBIR, instance retrieval, and object-based image retrieval (OBIR). Instance retrieval [26; 27; 28; 24; 29] is a problem of searching for images containing the same scenes and landmarks taken in different views, illumination, and occlusions for given query images with specific scenes and landmarks. Oxford5k and Paris6k are the most popular landmark dataset used by numerous image retrieval methods to date. Unlike previous datasets, which have only 4-5 similar images per class, such as Holidays [30] and UKBench [31], they have more than 100 similar images per class. However, the problem is that there are incorrect cases of image annotation. So, the revised version of them, the ROxford and RParis datasets [2] are presented to solve this problem. In this paper, ROxford and RParis are used for experiments. As a CNN-based global descriptor that shows excellent performance on those datasets, we select two descriptors: regional max-pooling (R-MAC) [10] and generalized mean-pooling (GeM) [9], which are applied to feature maps obtained from pre-trained CNNs, respectively. They are widely used to conduct the research and evaluation of instance retrieval on the Oxford5k and Paris6k datasets. R-MAC and R-GeM show good performance in various benchmarks [2; 3; 9]. Several image retrieval studies re-learn networks for the specific purpose and database. [PERSON] et al. [9] and [PERSON] et al. [10] use constant and triplet loss functions to re-learn the network, respectively. NetVLAD [32] uses GPS information and street-view images to re-learn the network. The combination of multiple global descriptors [33] is designed to implement the ensemble effect of several previously proposed global descriptors, and it similarly generates a descriptor through re-learning. In contrast to the fine-tuning methods as explained above, the method of using a pre-trained network on ImageNet [34] for direct image retrieval is used for our method, which is called the off-the-shelf method. Depending on the type and diversity of the database, retrieval may not be successful by simply comparing the visual features of the image once. To address this problem, several studies have been conducted on visual re-ranking methods [35; 36; 37] that correct initial image retrieval results. Query Expansion [2; 26; 38] further refines the existing expression of the query image using the information of the top image obtained from the initial search. Geometric context verification [39] compares the query and retrieved images with the geometric context of the local feature to determine the incorrectly found images.