# Multi-Subject Image Retrieval by Fusing Object and Scene-Level Feature Embeddings
## 2 Related Work
In K-nearest neighbors (K-NN)-based re-ranking [40], only those that are one of the K-NN of the query and include the query as K-NN from among the searched top images are extracted. Diffusion-based re-ranking was originally proposed for the purpose of using a manifold, but it was successfully applied to image retrieval [41] and achieved good results for several benchmarks. Recently, a method using contextual similarity aggregation with the self-attention of a transformer encoder has been proposed [36]. This method is robust to the image retrieval method that uses other feature representations because it is not directly related to the original features. [PERSON] et al. [4] approach splicing detection [6; 42] from an image search perspective. Object Embeddings for Spliced Image Retrieval (OE-SIR) was proposed to define spliced image retrieval (SIR), in which the authentic image of a query image in the database is found when given a spliced query image. If the existing image search technique focuses on finding near-duplicate images or images pointing to the same landmark, then SIR finds the authentic image used in the query image manipulation. Currently, the query image and authentic image often have completely different backgrounds; therefore, the local feature is more suitable for SIR than the previously used global feature. Because most spliced images use authentic image objects (e.g., animals, faces, and logos) for image manipulation, OE-SIR extracts object-level representations of each object in the image through object detection. The performance was improved by using classification embeddings for object-level feature extraction. In our work, we also used object-level feature embeddings to retrieve images based on existing objects without considering the background scene. Global and local features have different advantages and disadvantages. In certain situations, they are combined to increase retrieval performance complementally. The SIFT  
Figure 2: The pipeline of BoVW for image retrieval. feature [18] extracts keypoints from the image and represent them only in the gradient magnitudes and orientations. HSV-SIFT [43], HueSIFT [44], and OpponentSIFT [45] combined the conventional SIFT and color descriptors to improve the representation ability. [PERSON] et al. [46], [PERSON] et al. [47], and [PERSON] et al. [48] propose a descriptor with richer information through the fusion of local and global features. In this study, we propose an image embedding combining object and scene-level feature embeddings, which are not considered in previous studies. Similar to our approach, [PERSON] et al. [49] presented an approach that starts from a strong pre-trained model, and adapts it to tackle multiple retrieval tasks concurrently, using only unlabeled images from the different task domains. It adopts an adaptor structure that extends a pre-trained model to enable multiple adaptors to learn different domains, and integrates them into a single integrated representation suitable for multiple retrieval tasks. It employs a strategy to fuse various representations with networks for practical visual models, but a separate learning process is needed for convergence with adaptors for each representation.