# Semantic Multigranularity Feature Learning for
## 4 Experiments
### Key Design Choices  
We evaluate four design choices in the proposed SMGFL-Net, including (1) the pooling operation that connects the convolutional and classification block, (2) the different granularity options, (3) different fusion strategies of granularity-specific branches, (4) with or without the rearrangement of the feature maps. When evaluating each design choice, the accuracies of the two predictions discussed in Section 3.3 are observed on both NEWPU-RESISC45 and AID. The training ratio is set to be 0.5 in NEWPU-RESISC45 and 0.3 in AID. For the pooling operation, we consider average vs. max pooling. For the granularity options, we consider four granularity combinations applied on stages three, four, and five. Let \(G_{S_{1}}\) denote the granularity at stage \(i\) in the network. For instance, \(G_{S_{i}}=8\) means that the patch size in the rearranged feature map at stage \(i\) is \(8\times 8\) during training. The four granularity options we chose are {\(G_{S_{3}}=28\), \(G_{S_{4}}=14\), \(G_{S_{5}}=7\)}, {\(G_{S_{3}}=8\), \(G_{S_{4}}=4\), \(G_{S_{5}}=4\)}, \(G_{S_{5}}=7\), \(G_{S_{4}}=7\), \(G_{S_{5}}=7\)}, and {\(G_{S_{3}}=2\), \(G_{S_{4}}=2\), \(G_{S_{5}}=2\)}. For the fusion strategies of granularity-specific branches, three fusion strategies--concatenation, summation, and multiplication--are evaluated. For evaluating the effectiveness of rearranging the feature maps, we keep the feature maps of each granularity-specific branch unchanged to compare with the situation of rearranging feature maps in the training process. We conducted a randomized search on the combinations of the design options and obtained the best combination based on the search result: { max pooling, {\(G_{S_{3}}=8\), \(G_{S_{4}}=4\), \(G_{S_{5}}=2\)}, concatenation, with rearrangement }. To present the effect of a specific design choice, we chose the optimal value for other choices. Results are reported in Tables 1-4. We provide the observations as follows. * **Pooling strategy**. As shown in Figure 3, all of the global and granularity-specific branches contain pooling operations. The average pooling can process all information in a region and the maximum pooling can obtain the maximum value in a region. Table 1 shows an ACC comparison between average and max pooling on both the NWPU-RESISC45 and AID datasets. It is observed that max pooling consistently outperforms average pooling by 0.2-0.9%. Since average pooling considers all pixels in a region, it tends to smooth out an image and sharp features may not be identified. On the other hand, max pooling works by selecting the brightest pixel in a region, which makes it useful when the image background is dark and the semantic features are in lighter pixels. On both datasets, the majority of scene images are in dark background with light semantic objects that distinguish the categories. This characteristic of the dataset justifies the better effect of max pooling. * **Branch granularities**. Table 2 reflects the effect of the four granularity options, in which {\(G_{S_{3}}=8\), \(G_{S_{4}}=4\), \(G_{S_{5}}=2\)} consistently outperform other options on both datasets. From the limited options for the branch granularities that have been tried, we have the impression that a progressively decreasing patch size is favored to accommodate the feature maps that are also decreasing stage-by-stage. However, to validate this point, more granularity options can be tried, or even a heuristic can be developed to automate the search for an optimal or suboptimal set of granularities. We leave this as a meaningful future study, as described in Section 5. * **Different fusion strategies**. Table 3 shows the effect of three different fusion strategies--concatenation, summation, and multiplication--in which concatenation consistently outperforms other options in all scenarios. The reason for this result is that the concatenation operation cannot cover the information of the original features and obtain the feature vector with higher dimension.