## 1 Introduction  
With the advancement of remote sensing imaging technology, the spatial resolution of remote sensing images has been continuously improved. In low- and medium-resolution remote sensing images, the problem is that different objects may share the same spectral response curve, but the same object may have different spectral response curves, thus making the classification methods at the pixel or object level show many limitations. Using the image block (i.e., the scene) as the basic unit for conducting image interpretation can make effective use of the spatial context information to eliminate the ambiguity of interpretation. As a basic unit of remote sensing image interpretation, the scene is a combination of multiple objects, environments, and semantics. Remote sensing image scene understanding (RSISU) not only needs to recognize each object, but also needs to perceive the topology distribution of multiple objects in a remote sensing image scene. Consequently, RSISU benefits by improving things such as the stability of remote sensing image scene classification, remote sensing image scene retrieval, and scene-driven geospatial object detection. Until now, large amounts of methods have been proposed around RSISU. To pursue the sustainable development of RSISU, this paper mainly focuses on giving a comprehensive review of RSISU, which also discusses the difference and relationship among various sub-tasks in RSISU. As is well-known, visual abstraction is an important prerequisite for remote sensing image scene understanding (RSISU). As a classic representative of deep learning, the convolutional neural network (CNN) [1] is the milestone technique for abstracting the visual content of remote sensing image scenes. Because of this, CNN has been widely applied in remote sensing image scene classification, remote sensing image retrieval, scene-driven object detection, and so on. Among all of the sub-tasks in RSISU,deep learning outperforms the traditional technology (e.g., hand-crafted features) by a large margin. With this consideration, this paper mainly reviews and discusses RSISU driven by deep learning. As the basic tasks of RSISU, remote sensing image scene classification, remote sensing image scene retrieval, and scene-driven geospatial object detection share some communal hints (e.g., scene abstraction) but have key respective techniques, as well as different goals. More specifically, remote sensing image scene classification pursues a high-accuracy classification result by perceiving the objects and their spatial layout, whereas remote sensing image scene retrieval often maps the remote sensing image scene to an efficient representation (e.g., the low-dimensional feature vector) to address the large-scale image retrieval problem. Scene-driven geospatial object detection aims at recognizing the objects by exploiting the scene context. Around each sub-task, large amounts of methods have been proposed and summarized in a specific review. For example, [PERSON] et al. [2] summarized the remote sensing image scene classification methods, [PERSON] et al. [3] reviewed the remote sensing image scene retrieval methods, and [PERSON] et al. [4] conducted a survey on the different kinds of remote sensing image object detection methods. Currently, there are no systematic surveys available in regard to the bigger research topic--namely, RSISU. To achieve a collaborative development of remote sensing image scene classification, remote sensing image scene retrieval, and scene-driven geospatial object detection, this paper aims at summarizing the deep learning driven achievements around RSISU and systematically depicting the relationship among the sub-tasks in RSISU. As a whole, this review includes the following highlights:  
1. In contrast to the existing reviews which often review the related techniques on a specific topic, this paper clarifies the relationship and difference among the sub-tasks in RSISU, and gives a systematic review.