# A Novel Method for Ground-Based Cloud Image Classification Using Transformer
###### Abstract  
In recent years, convolutional neural networks (CNNs) have achieved competitive performance in the field of ground-based cloud image (GCI) classification. Proposed CNN-based methods can fully extract the local features of images. However, due to the locality of the convolution operation, they cannot well establish the long-range dependencies between the images, and thus they cannot extract the global features of images. Transformer has been applied to computer vision with great success due to its powerful global modeling capability. Inspired by it, we propose a Transformer-based GCI classification method that combines the advantages of the CNN and Transformer models. Firstly, the CNN model acts as a low-level feature extraction tool to generate local feature sequences of images. Then, the Transformer model is used to learn the global features of the images by efficiently extracting the long-range dependencies between the sequences. Finally, a linear classifier is used for GCI classification. In addition, we introduce a center loss function to address the problem of the simple cross-entropy loss not adequately supervising feature learning. Our method is evaluated on three commonly used datasets: ASGC, CCSN, and GCD. The experimental results show that the method achieves 94.24%, 92.73%, and 93.57% accuracy, respectively, outperforming other state-of-the-art methods. It proves that Transformer has great potential to be applied to GCI classification tasks. ground-based cloud image (GCI) classification; convolutional neural network (CNN); local features; global features; Transformer 202214397810390/rs14163978  
11 August 2  
factors. For the above reasons, it is crucial to investigate methods that can automatically and accurately classify GCIs. Early GCI classification was based on manually extracted features, and most methods employed brightness, texture, shape, and color features to represent image content. [PERSON] et al. [17] proposed a classification algorithm based on spectral features in red-green-blue (RGB) color space and texture features extracted using the Gray-Level Co-occurrence Matrix (GLCM). [PERSON] et al. [18] generated the corresponding feature vectors by calculating the weighted frequency values of microstructures in each cloud image and then inputting them into the support vector machine (SVM) classifier for classification. [PERSON] et al. [19] proposed an improved text-based classification method that incorporates manually extracted color and texture features to improve the classification. [PERSON] et al. [20] extracted visual descriptors from color, texture, and the structure of clouds in a dense sampling manner simultaneously and then completed feature encoding using Fisher Vector, which characterized image samples by computing the log-likelihood gradient of model parameters to enhance classification performance. [PERSON] et al. [21] applied color census transform to compute cloud image statistics to extract texture and structure features, which were then fed into a conventional classifier. In recent years, with the development of deep learning, many convolutional neural network (CNN)-based methods have made significant progress in GCI classification. [PERSON] et al. [22] argued that locally rich information is more important than global layout information, and, therefore, used deep convolutional activation-based features (DCAF) and shallow convolutional layer-based features for classification. [PERSON] et al. [23] extracted multiscale feature maps from pretrained CNNs and then used Fisher Vector coding to perform spatial feature aggregation and high-dimensional feature mapping on the original deep convolutional features, which aimed to find discriminative local information to better distinguish cloud types. [PERSON] et al. [24] used a 3D-CNN model to process multiple consecutive ground-based cloud images to extract cloud features such as texture and temporal information, followed by a fully connected layer for classification. [PERSON] et al.