## 3 Remote Sensing Image Scene Understanding
### Deep-Learning-Driven Scene Classification
#### Fully Supervised Deep Networks for Remote Sensing Image Scene Classification  
Unlike pre-trained and unsupervised deep feature learning, which need classifiers such as a random forest or SVM, metric learning (ML) has gradually developed, which is able to enhance the learned features separability. ML is an end-to-end learning method mainly used to find an appropriate measure of similarity between pairs of data, which can preserve the desired distance structure very well. Existing metric learning is usually categorized into two streams: contrastive embedding and triplet embedding [49; 50; 51].  
_Contrastive embedding_: Contrastive embedding is trained on paired data \((x_{i},x_{j})\). Concretely, the cost function is defined as,  
\[J=\sum_{i,j}\ell_{ij}D^{2}(x_{i},x_{j})+\left(1-\ell_{ij}\right)h\big{(}a-D \big{(}x_{i},x_{j}\big{)}\big{)}^{2} \tag{1}\]  
where the label indicator \(\ell_{ij}\in\{1,0\}\) indicates whether the paired data \((x_{i},x_{j})\) are from the same class or not. \(h(x)=\max(0,x)\) denotes the hinge loss function, and \(D\big{(}x_{i},x_{j}\big{)}=\parallel f(x_{i})-f\big{(}x_{j}\big{)}\parallel_{2}\) is the Euclidean distance of paired data \((x_{i},x_{j})\).  
_Triplet Embedding_: Triplet embedding is trained on triplet data \((x_{a},x_{p},x_{n})\). Concretely, the cost function is defined as:  
\[J=\sum_{a,p,n}h\big{(}D\big{(}x_{a},x_{p}\big{)}-D(x_{a},x_{n})+a\big{)}^{2} \tag{2}\]  
Here, \((x_{a},x_{p},x_{n})\) are the triplet data, where \((x_{a},x_{p})\) have the same class labels and \((x_{a},x_{n})\) have different class labels.  
Compared with traditional measurement learning, which focuses on preserving interclass separability, [44] pays more attention to label consistency (LC). In order to ensure the intraclass compactness of VHR images, a discriminable distance metric learning method (DDML) is proposed in this paper. In the learning process, besides the enforced intraclass compactness and interclass separability, the global and local label consistency (LC) is constrained, which is aimed to jointly optimize the feature manifold, distance metric, and label distribution.  
As is well-known, the last layer of the network is structured as FC layers, which are typically used in a bid to better summarize the low-level feature learning for a final strategic decision. Apart from reducing cross entropy loss in FC layers, a metric learning regularization method was proposed in [21] to make the network model more discriminative. In this paper, a new objective function is proposed with three terms, including a cross entropy loss term, a metric learning regularization term, and a weight decay term. When the percentage of labeled images is 15%, the classification accuracy (AC) and standard deviation (STD) can reach \(69.33\pm 1.25\%\).  
In order to utilize the structural information between each scene, [52] proposed a special structured loss to measure pairwise distances. Hence, this paper puts forward diversity-promoting deep structural metric learning (D-DSML). Compared with the similar learned metric parameters of a traditional DSML, this method can overcome the overlapping of representations of different factors. The D-DSML method is implemented by imposing a diversity-promoting prior which can prevent the factors in DSML from being correlated, so that the redundancy caused by the similarity between the hidden factors can be decreased. D-DSML-CaffeNet's classification accuracy can reach \(96.76\pm 0.36\%\) with a slight boost, compared with CaffeNet (95.48%), GoogLeNet (\(94.31\pm 0.89\%\)), and VGG-VD16 (\(95.21\pm 1.20\%\)).