# Semantic Multigranularity Feature Learning for
## 5 Conclusions  
In this paper, we propose a novel deep neural architecture named SMGFL-Net for RS image scene classification. At the training step, the proposed SMGFL-Net learns multigranularity features through a destruction operation on intermediate feature maps by a jigsaw puzzle generator with different sizes, which avoids the meaningless edges appearing in prior studies. The proposed method is validated on the NWPU-RESISC45 and AID datasets. Results show that SMGFL-Net can effectively learn and leverage both global and local features at different granularities, demonstrating the SOTA performance compared with several peer methods from recent literature.  
This study has the following limitations, which will be addressed in future work. First, the efficacy of the proposed SMGFL-Net is only validated on two datasets, namely, NWPU-RESISC45 and AID, and its capability on further RS image datasets has not been evaluated. Second, in the experiment of this study, the number of stages with multigranularity branches and the patch sizes are manually determined, which is not cost-effective and provides limited guidance on the choice of these parameters. It is thus essential to develop a heuristic to search for an optimal or suboptimal set of parameters suitable for a problem instance in a systematic way. Third, the challenge of intraclass variance and interclass similarity could be addressed by other CNN design strategies, such as pairwise feature learning and biattention, which can be selectively integrated into SMGFL-Net.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Method** & **Ratio = 0.1** & **Ratio = 0.2** & **Ratio = 0.5** \\ \hline \multirow{2}{*}{Global Methods} & VGG16 [3] & - & 86.6\% & - \\  & Resnet50 [4] & 89.8\% & 91.3\% & 93.7\% \\  & ResNeXt50 [41] & 90.1\% & 92.1\% & 94.2\% \\ \hline \multirow{2}{*}{Local Methods} & Bilinear-CNN [16] & 90.0\% & 92.6\% & 93.0\% \\ \hline \multirow{3}{*}{FGVC Methods} & NTS-Net [9] & 90.5\% & 94.0\% & 94.7\% \\  & DCL [17] & 86.3\% & 91.2\% & - \\  & Progressive-MG [22] & 90.0\% & 94.1\% & 95.9\% \\ \hline \multirow{2}{*}{RS Methods} & InceptionV3-CapsNet [34] & - & 89.0\% & - \\  & VGG16-MSCP [35] & - & 91.6\% & - \\  & MGML-FENet [36] & - & 95.8\% & - \\ \hline \multirow{2}{*}{Ours} & SMGFL-Net & **93.3**\% & **96.2**\% & **97.2**\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparative results on AID.  
**Author Contributions:** Conceptualization and methodology, X.M., Z.X., H.-s.Y. and S.-J.-L.; software, validation, and original draft preparation, X.M.; review and editing, supervision, funding acquisition, Z.X., H.-s.Y. and S.-J.L. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was supported by a grant (2019-MOIS33-005) of Lower-level and Core Disaster-Safety Technology Development Program funded by the Ministry of Interior and Safety (MOIS, Korea). This research was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C201231911).  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** The datasets supporting the conclusions of this article are available at [[https://www.tensorflow.org/datasets/catalog/resisc45](https://www.tensorflow.org/datasets/catalog/resisc45)]([https://www.tensorflow.org/datasets/catalog/resisc45](https://www.tensorflow.org/datasets/catalog/resisc45)) and [[https://captain-whu.github.io/AID/](https://captain-whu.github.io/AID/)]([https://captain-whu.github.io/AID/](https://captain-whu.github.io/AID/))(accessed on 2 June 2021).  
**Conflicts of Interest:** The authors declare no conflict of interest.