# A Scene Graph Similarity-Based Remote Sensing Image Retrieval Algorithm
###### Abstract
In Figure 1, (a) is the query image \(\mathrm{Iq}\), containing three target detection results (storage tanks) arranged in a straight line, \(\mathrm{I_{1}}\), \(\mathrm{I_{2}}\), \(\mathrm{I_{3}}\) \(\in\) Gallary (a remote sensing image database), and \(\mathrm{Iq}\) has similar image features to \(\mathrm{I_{1}}\), \(\mathrm{I_{2}}\), and \(\mathrm{I_{3}}\), red box is target detection result, green line is spatial relationship. According to the CBIR method of feature matching, the retrieval results might include \(\mathrm{I_{1}}\), \(\mathrm{I_{2}}\), and \(\mathrm{I_{3}}\). However, based on the existing SBIR methods, which only consider the category and number of objects without considering spatial relationships, \(\mathrm{I_{2}}\) and \(\mathrm{I_{3}}\) might be considered as retrieval results because both of them contain three oil tanks. In reality, although \(\mathrm{I_{2}}\) has three oil tank objects, the spatial arrangement of the tanks is triangular, which does not match that of \(\mathrm{I_{3}}\) has three linearly arranged oil tank objects, making its semantic features more consistent with \(\mathrm{Iq}\). Generally, the CBIR approach has a low retrieval accuracy due to a lack of semantic information, and typically the SBIR approach ignores the relationships among object targets [6]. Therefore, this research employed scene graphs, which can better represent both the attributes and relationships of targets to capture more semantic information to further improve retrieval accuracy. Compared with ordinary images, remote sensing images have more significant characteristics. Firstly, they have a top-down view, where the relationships among objects are mainly reflected in spatial positions on a plane. Secondly, they have large fields of view, containing several objects with complex spatial relationships. Finally, they have complex backgrounds, including numerous unknown objects. Therefore, this research adopted a semantic-based approach to develop a scene-graph-based remote sensing image retrieval algorithm. The algorithm simulated the human behavior of comparing key targets to retrieve images, simplifying remote sensing image retrieval to match the multiple  
Figure 1: Examples of sensing image retrieval based on semantic features. categories and spatial relationships of the targets in image scenes, effectively using the semantic information in the images. In this work, the semantic information of targets was first obtained using a one-shot object detection algorithm [7] for the remote sensing images. Then, scene graphs [8] were constructed based on this semantic information. Finally, the similarity of scene graphs was calculated and ranked to obtain the retrieval results. The contributions of this paper include the following:  
1. A Siamese- network-based one-shot object detection algorithm was designed. A Siamese network model was proposed considering the issue of potentially unknown categories of objects in query images. The training objective of the model was to ensure that the distance in the feature representation space between any two input samples reflected their image feature similarity. In addition, to deal with the problem of complex backgrounds in a remote sensing image, a feature extraction network based on asymmetric convolution was developed in each feature extraction branch of the Siamese network to enhance feature extraction capabilities. To address the problem of missing small targets in remote sensing images, a feature pyramid structure was designed to improve detection capabilities for small targets. Finally, a candidate region generation network based on attention mechanisms was proposed to solve the problem of low positional regression accuracy in one-shot object detection tasks. 2. A remote sensing image retrieval method was developed using scene graph similarity.