# Contrastive Learning Based on Transformer for Hyperspectral Image Classification  
[PERSON]  
1 The State Key Laboratory of High-Performance Computing, College of Computer, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
[PERSON]  
2 Beijing Institute for Advanced Study, National University of Defense Technology, Beijing 100020, China; [EMAIL_ADDRESS]  
[PERSON]  
1 The State Key Laboratory of High-Performance Computing, College of Computer, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
[PERSON]  
3 College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
[PERSON]  
1 The State Key Laboratory of High-Performance Computing, College of Computer, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
Footnote 1: [[https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci)]([https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci))  
Footnote 2: [[https://www.mdpi.com/journals/pdf/11/1](https://www.mdpi.com/journals/pdf/11/1)]([https://www.mdpi.com/journals/pdf/11/1](https://www.mdpi.com/journals/pdf/11/1))  
labels, so this challenge can be solved. Thus, research on unsupervised hyperspectral image classification is urgently required. Unsupervised learning aims to extract information from data without labels. According to the mainstream view, unsupervised methods can be divided into representative learning and discriminative learning. Most of the representative learning methods are based on two models: an autoencoder [12] and generative adversarial network (GAN) [13]. These two models both aim to map the training data to a certain distribution mode. According to their distribution, we may obtain fake samples that are similar to real data or useful feature extractors. For hyperspectral image classification, the stacked sparse autoencoder (SSAE) extracts sparse spectral features and multiscale spatial features using the autoencoder [14]. Additionally, GAN [15] was used for hyperspectral image classification. Furthermore, the 3D convolutional autoencoder (3 DCAE) [16], GAN-assisted CapsNet (TripleGAN) [17], and many other representative-learning-based models have been proposed for hyperspectral image classification. Except for representative learning, discriminative learning is a method for collecting information from unlabeled data. Contrastive learning is a popular discriminative learning method. Unlike representative learning, contrastive learning aims to discriminate different data instead of obtaining the data distribution feature. It requires much less computational resources than the representative learning method. After being trained by comparing different samples, the discriminator can be applied to downstream tasks, such as image recognition [18] and object detection [19]. Contrastive learning has achieved great success in the computer vision field. However, some challenges remain when applying contrastive learning to hyperspectral image classification. Firstly, the data augmentation methods commonly used in the computer vision field are not applicable to the hyperspectral image classification field. For example, color distortion is a typical data augmentation method used for general images. However, when used for hyperspectral image classification, color distortion disrupts the spectral information in hyperspectral images. Therefore, this method is unsuitable for hyperspectral image augmentation. Secondly, the models used in contrastive learning, mostly 2D CNNs, for computer vision tasks are not applicable in hyperspectral image processing. CNNs are not able to mine the sequence attributes of spectral signatures well. To introduce contrastive learning to hyperspectral image classification, we handled these two problems in this study. Firstly, we devised a useful data augmentation method for hyperspectral image classification.