# A Deep Curriculum Learning Semi-Supervised Framework for Remote Sensing Scene Classification
## 3 Proposed Method
### Multimodal Curriculum Learning
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Method Name** & **10\% Training Set** & **20\% Training Set** & **60\% Training Set** \\ \hline \multirow{2}{*}{Traditional methods} & Sparse coding+SVM & 41.64 \(\pm\) 1.02 & 46.54 \(\pm\) 0.38 & 54.95 \(\pm\) 0.78 \\  & Dictionary learning+SVM & 42.31 \(\pm\) 0.87 & 47.23 \(\pm\) 0.56 & 55.93 \(\pm\) 1.57 \\ \hline \multirow{3}{*}{Supervised learning} & VGG-F [32] & 82.25 \(\pm\) 0.95 & - & 89.28 \(\pm\) 0.87 \\  & ResNet [28] & 85.71 \(\pm\) 1.44 & - & 93.31 \(\pm\) 1.67 \\  & GBN [34] & - & 92.20 \(\pm\) 0.23 & - \\ \hline \multirow{2}{*}{Unsupervised learning} & Attention-GAN [35] & - & 78.95 \(\pm\) 0.23 & - \\  & Self-training(ResNet) [12] & - & 89.38 \(\pm\) 0.87 & - \\ \hline \multirow{6}{*}{Semi-Supervised learning} & Co-training [11] & 90.87 \(\pm\) 1.08 & - & - \\  & SSGA-E [12] & 91.35 \(\pm\) 0.83 & - & - \\  & Fixmatch [10] & 93.63 \(\pm\) 0.60 & - & - \\  & Mixmatch [9] & 92.52 \(\pm\) 0.48 & - & - \\  & SS-RCSN [8] & 94.25 \(\pm\) 0.64 & - & - \\  & Our Method with DCA & 88.67 \(\pm\) 0.52 & 90.43 \(\pm\) 0.43 & 95.29 \(\pm\) 0.67 \\  & Our Method with DCCA & 97.95 \(\pm\) 0.78 & 98.20 \(\pm\) 0.81 & 99.14 \(\pm\) 0.56 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy (%) of different methods on AID dataset. Figure 6: Confusion matrix on AID dataset at 20% training ratio using DCA fusion method. The recognition rate for each category is indicated by a black background color. thermal power station, wetland, etc. For each scene class, the dataset shows great differences in viewpoint, object pose, illumination, and occlusion, etc. An example of the NWPU RESISC-45 dataset is shown in Figure 7.

The comparison results for the NWPU RESICS-45 dataset are shown in Table 4. Our proposed framework yields the best accuracy among all the methods. In summary, it is proved that the semi-supervised classification method combined with deep feature fusion for multimodal curriculum learning can surpass many supervised classification methods. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Method Name** & **10\% Training Set** & **60\% Training Set** \\ \hline \multirow{2}{*}{Traditional methods} & Sparse coding+SVM & 38.64 \(\pm\) 1.02 & 50.63 \(\pm\) 1.26 \\  & Dictionary learning+SVM & 39.53 \(\pm\) 0.96 & 50.91 \(\pm\) 1.78 \\ \hline \multirow{2}{*}{Supervised learning} & GoogleNet [31] & 76.00 \(\pm\) 0.99 & 82.65 \(\pm\) 2.08 \\  & VGG-F [32] & 79.40 \(\pm\) 0.80 & 86.33 \(\pm\) 0.47 \\  & ResNet [28] & 84.30 \(\pm\) 0.60 & 90.20 \(\pm\) 0.20 \\ \hline \multirow{2}{*}{Unsupervised learning} & Attention-GAN [35] & 72.21 \(\pm\) 0.21 & - \\  & Self-training(ResNet) [12] & 85.82 \(\pm\) 1.30 & - \\ \hline \multirow{6}{*}{Semi-Supervised learning} & Co-training [11] & 87.25 \(\pm\) 0.95 & - \\  & SSGA-E [12] & 88.60 \(\pm\) 0.31 & - \\  & Fixmatch [10] & 90.45 \(\pm\) 0.43 & - \\ \cline{1-1}  & Mixmatch [9] & 89.22 \(\pm\) 0.29 & - \\ \cline{1-1}  & SS-RCSN [8] & 92.03 \(\pm\) 0.16 & - \\ \cline{1-1}  & Our Method with DCA & 88.10 \(\pm\) 0.35 & 93.10 \(\pm\) 0.54 \\ \cline{1-1}  & Our Method with DCCA & 93.01 \(\pm\) 0.65 & 95.20 \(\pm\) 0.74 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy (%) of different methods on NWPU RESICS-45 dataset. Figure 7: Examples of the NWPU RESICS-45 dataset. ### Ablation Experiment

In order to better analyze our semi-supervised classification framework, an ablation study was conducted on the UC Merced dataset. We study the effect of removing components in the proposed DCLSSF. Table 5 shows the experimental results. In the proposed feature extraction module, the multimodal approach can obtain more discriminative deep features, which is helpful in improving the classification accuracy. It can be seen that the result is decreased when feature fusion is removed. This shows that deep feature fusion can obtain strong discriminatory power features.