# Contrastive Learning Based on Transformer for Hyperspectral Image Classification
## 4 Experimental Descriptions and Result Analysis
### Result Analysis
Some of the data in the table were obtained from [16]. The best values in each row are bolded. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Class** & \multicolumn{3}{c}{**Supervised Feature Extraction**} & \multicolumn{3}{c}{**Unsupervised Feature Extraction**} \\ \hline  & **LDA** & **1D-CNN** & **S-CNN** & **3 DCAE** & **AAE** & **VAE** & **Proposed** \\ \hline
1 & 58.54 & 43.33 & 83.33 & 90.48 & **100.00** & **100.00** & 97.05 \\
2 & 69.88 & 73.13 & 81.41 & 92.49 & 81.63 & 78.78 & **96.73** \\
3 & 65.86 & 65.52 & 74.02 & 90.37 & 95.27 & 92.37 & **95.34** \\
4 & 73.71 & 51.31 & 71. Even though our model works without label information, it still outperformed the supervised methods. Additionally, compared to the LDA results, we conclude that the deep-learning-based model is superior to the machine-learning-based method. According to the above experimental results and analysis, we conclude that our model can effectively extract the features under unsupervised conditions. It is suitable for addressing the challenge posed by the lack of labeling. The model does not contain convolutional operations. Our findings prove that convolutional operations are not necessary for hyperspectral image classification. Additionally, 2D convolutional networks for traditional computer vision are not applicable to hyperspectral image classification. A 3D convolutional network can provide a better hyperspectral image classification result than a 2D convolutional network. The transformer model we used in the experiments is the same as the transformer model used for traditional computer vision tasks in terms of model \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Class** & \multicolumn{2}{c}{**Supervised Feature Extraction**} & \multicolumn{4}{c}{**Unsupervised Feature Extraction**} \\ \hline  & **LDA** & **1D-CNN** & **S-CNN** & **3 DCAE** & **AAE** & **VAE** & **Proposed** \\ \hline
1 & 78.41 & 90.93 & 95.40 & 95.21 & 95.39 & 81.31 & **98.19** \\
2 & 83.69 & 96.94 & 97.31 & 96.06 & 98.96 & 97.65 & **99.86** \\
3 & 73.02 & 69.43 & 81.21 & 91.32 & 97.49 & 91.00 & **98.87** \\
4 & 93.68 & 90.32 & 95.83 & 98.28 & 96.94 & 94.79 & **96.99** \\
5 & **100.00** & 99.44 & 99.91 & 95.55 & 99.94 & 99.94 & 96.75 \\
6 & 88.51 & 73.69 & 95.29 & 95.30 & 99.29 & 99.91 & **100** \\
7 & 85.75 & 83.42 & 87.05 & 95.14 & **100.00** & 99.42 & 96.32 \\
8 & 74.49 & 83.65 & 87.35 & 91.38 & 97.90 & 94.77 & **98.32** \\
9 & 99.11 & 98.23 & 95.66 & **99.96** & 96.32 & 88.62 & 84.04 \\ AA(\%) & 86.29 & 87.34 & 94.75 & 95.36 & **98.03** & 94.16 & 96.59 \\ OA(\%) & 83.75 & 89.99 & 92.78 & 95.39 & 98.14 & 94.53 & **98.67** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Classification results of different models in UP. Some of the data in the table were obtained from [16]. The best values in each row are bolded. Figure 7: The classification maps of SV. (**a**–**c**) Classification results of AAE, VAE, and our proposed model, respectively. structure. We only changed the model size, which demonstrates that computer vision models have huge potential for hyperspectral image classification. The other important part of our method, contrastive learning, relies on data augmentation for the input image. Because some data argumentation methods in traditional computer vision are unsuitable for hyperspectral images, we used image flip, and deleting spectral information of some points to augment the data. With a small number of data augmentation methods, the contrastive learning method can perform much better than the representative learning method. With more data augmentation methods, the contrastive learning method may be more accurate for hyperspectral image classification.