# Analysis of Airglow Image Classification Based on Feature Map Visualization
## 4 Discussion  
Residual neural networks (ResNets) and transformers are challenging candidates besides CNNs. ResNets address the problem of vanishing gradients in deep neural networks through the introduction of residual connections between layers and can achieve higher accuracy in large image classification tasks [40; 41], but at the risk of overfitting. Transformers have the advantage of being highly flexible and capturing long-range dependences in images [42], but their computationally intensive self-attention mechanisms make them unsuitable for this application with constrained resources. CNNs are widely utilized for image classification due to their simple but effective structure and remarkable performance [43]. Another important reason we chose a CNN is that we can select and verify the generated classification model by visualizing the feature maps of the CNN. To track the training process, we visualized the feature map of category c (light band) after every epoch (shown in Figure 6). After 10 epochs, a cross structure can be seen in most feature maps. The cross corresponds to the horizontal and vertical light bands in the original and rotated images of category c. The parts in the image that coincide with the central cross have a high weight in the evaluation of category c; hence, an input image with a light band will be determined as category c by the classification model. The cross structure becomes clearest near the 31 st epoch, which is consistent with the accuracy-loss curves in Figure 4.  
Figure 5: Feature maps of five categories: (**a**) clear night; (**b**) overcast sky; (**c**) light band; (**d**) moon; (**e**) twilight. The brightness of pixels on feature maps represents weights of classification, with brighter color corresponding to higher weight. We notice that, due to the obvious image features and suitable CNN structures, the model achieved brilliant scores (high accuracy and low loss) in training set and validation set during the training process (shown in Figure 4). However, the accuracy and loss cannot comprehensively and objectively evaluate the performance of the model. The insufficiency and unbalance of data may increase the accuracy to unrealistically high. During the training process, the model load images randomly from the training set and modify its learnable parameters to minimum the loss in different batches. If the sample characteristics of a batch are too similar to those of the previous batch, the accuracy of this batch may be too high. This is because the model may easily achieve high accuracy on these similar samples, but it cannot be well generalized to new data. In addition, the inconsistent number of samples in each category will lead to unrealistically high accuracy because the model tends to judge the ambiguous images to the category with more data. Therefore, to evaluate the effectiveness of the classification model, we visualized the feature maps to exhibit the classification criteria, which can give a direct impression. After 31 epochs, the cross structure gradually disappeared until the 40 th epoch, indicating a trend of overfitting. Overfitting is significant at the 37 th epoch, where the loss of the validation set has a peak while the loss of the training set decreases (shown in Figure 4). Accordingly, the feature structure of classification is not obvious in the feature maps of the 37 th epoch (shown in Figure 7). Except for category b ('overcast sky'), other feature maps exhibit similar weight distributions, which will lead to confusion in the actual classification. At the 40 th epoch, the significant overfitting was weakened and the feature structures were again visible in the feature maps (shown in Figure 7). However, the central region of the  
Figure 6: Feature maps for each epoch of the category c (‘light band’) of the classifier. The numbers below images represent the current epoch. Figure 7. Feature maps of different epochs. five-categorical classification results and obtained a binary classification: the category of 'clear night' and others.