# A Deep Curriculum Learning Semi-Supervised Framework for Remote Sensing Scene Classification
## 3 Proposed Method
### Multimodal Curriculum Learning
The AID dataset, which is mainly derived from Google Earth images, contains 30 challenging scene categories and about 200 to 400 images in each category, each of which is 600 \(\times\) 600 in size. The NWPU RESICS-45 dataset contains 31,500 images. The dataset consists of 45 categories, each with 700 images. The UC Merced dataset was proposed earlier and is generally accepted by the majority of researchers, but due to the small size of the dataset and the introduction of deep learning methods, the recognition rate on the dataset has been saturated. The AID dataset and NWPU RESICS-45 dataset have been proposed in recent years, and the categories in the dataset are rich and the image data scale is large, so as to meet deep learning requirements that require a large amount of data. In particular, NWPU RESICS-45 dataset has high within-class diversity and between-class similarity. ### Experimental Setup

In the experiment, three widely studied pretrained deep networks were used to extract deep features, namely ResNet [28], Densenet [29], and Inceptionresnet [30]. The fully connected layer features of the pretrained network were extracted, and there were 1000 deep feature dimensions. In each experiment, we adjusted the input image size to match the pretrained network input size. All CNN models pretrained on the ImageNet dataset were obtained from the deep learning toolbox of Matlab2018b for deep CNN feature extraction, and then these deep features were fused by the DCA method. The three deep features after fusion were regarded as three modalities, which were then sent to the multimodal curriculum learning framework for semi-supervised classification. On three public datasets, we evaluated the classification performance of the test method under different scales of the labeled set. To reduce the impact of random selection, we executed each algorithm ten times and report the average value of the accuracy obtained. The experiments were performed on AMAX DevMax-401 with an Intel Core i7-6850K CPU, 128 GB RAM, and NVIDIA GTX 1080 Ti GPU with 11 GB memory. The method of feature fusion adopts DCA [14] and DCCA [15], respectively. ### Comparisons on UC Merced Dataset

The UC Merced dataset has a total of 2100 images, each category has 100 images, and each image is 256 \(\times\) 256 in size. The dataset has 21 scene categories: agricultural, airplane, baseball diamond, beach, buildings, chaparral, dense residential, forest, freeway, golf curriculum, harbor, intersection, medium density residential, mobile home park, overpass, parking lot, river, runway, sparse residential, storage tanks, and tennis courts. Some examples of the UC Merced dataset are shown in Figure 3.

To compare with the methods mentioned in other literature, the same experimental settings [8] are used: 10, 50, and 80 training samples per class for the UCM dataset. The overall recognition rates of the common methods are shown in Table 1. The recognition rate of our proposed semi-supervised method is better than other methods, which improved by 4.62%, 2.92%, 3.69%, and 1.04% compared with SSGA-E, Fixmatch, Mixmatch, and SS-RCSN, respectively. Different feature fusion methods have different subsequentclassification performances. The classification performance of the DCCA method is better than that of the DCA method. This is because compared to DCA, which only considers the relationship between classes, DCCA also considers connections within the class, so the performance of its feature fusion is better than that of DCA, and the dimension after fusion has not increased. The confusion matrix in Figure 4 shows the recognition rate of our proposed method. Among these categories, there is some confusion between dense residential and medium residential categories, mainly because these pairs have similar spectral or structural characteristics, and the main difference is only in the density of the buildings.