# Boosting hyperspectral image classification with Gate-Shift-Fuse mechanisms in a novel CNN-Transformer approach

Mohamed Fadhlallah Guerri, Cosimo Distante, Paolo Spagnolo, Abdelmalik Taleb-Ahmed

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Mohamed Fadhlallah Guerri, Cosimo Distante, Paolo Spagnolo, Abdelmalik Taleb-Ahmed. Boosting hyperspectral image classification with Gate-Shift-Fuse mechanisms in a novel CNN-Transformer approach. Computers and Electronics in Agriculture, 2025, 237, Part A, pp.110489. 10.1016/j.compag.2025.110489. hal-05123205

HAL Id: hal-05123205

https://hal.science/hal-05123205v1

Submitted on 20 Jun 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Boosting hyperspectral image classification with Gate-Shift-Fuse mechanisms in a novel CNN-Transformer approach

Mohamed Fadhlallah Guerri a, b, ID
Cosimo Distante a, b, ID, *, Paolo Spagnolo b, ID
Abdelmalik Taleb-Ahmed c, ID

## ARTICLE INFO

### Keywords:

Convolutional Neural Networks (CNNs)

Hyperspectral Imaging (HSI)

Semantic features

Gate-shift fuse

Transformer

## ABSTRACT

Hyperspectral Image (HSI) classification categorizes every pixel sample into distinct land-cover types. While CNN-based methods have demonstrated significant advances in feature representation, they often struggle with extracting deep features. On the other hand, transformer models excel in capturing high-level semantic features, offering complementary strengths. This paper introduces a novel HSI classification framework that combines convolutional blocks with Gate-Shift-Fuse (GSF) and transformer blocks to leverage CNNs for local feature extraction and transformers for long-range context modelling. The GSF block enhances the integration of spatial-spectral features, while an effective attention mechanism module refines feature extraction from HSI cubes. The proposed method was evaluated on four benchmark datasets (Indian Pines, Pavia University, WHU-Hi-LongKou, and WHU-Hi-HanChuan). It achieved state-of-the-art overall accuracies (OA) of 99.47% on the Indian Pines dataset and 99.86% on WHU-Hi-LongKou, demonstrating significant improvements over existing models. These results highlight the framework's ability to address challenges like spectral variability and unbalanced datasets. Our contributions include the integration of the GSF block for enhanced spatial-spectral feature fusion and the application of transformers for robust high-level semantic understanding, setting a new benchmark in HSI classification.

## 1. Introduction

Recent advances in spectrum imaging technology have markedly elevated the precision of data collected through hyperspectral sensors. Hyperspectral imaging (HSI) is distinguished by its high-dimensional features and the richness and continuity of its spectral curves. HSI classification is used in various applications across fields such as agriculture (Gevaert et al., 2015; Sharifi and Hosseingholizadeh, 2020), biomedical imaging (Noor et al., 2016), pollution monitoring (Sharifi and Felegari, 2022; Xie et al., 2023), mineral exploration (Wang et al., 2012; Chen et al., 2023), food safety (Fong et al., 2020), and military reconnaissance (Ardouin et al., 2007; Yu et al., 2024; Sun et al., 2023).

Unlike traditional machine learning methods that rely on hand-crafted features (Li et al., 2019; Guerri et al., 2024; Cheng et al., 2022; Zhou et al., 2024), deep learning architectures such as convolutional neural networks (CNNs), transformers, and graph convolutional networks (GCNs) have demonstrated remarkable capabilities in processing

complex data (Bougourzi et al., 2023; Nejad et al., 2022). CNNs have been particularly effective in capturing local spatial patterns, making them widely used in image-based applications (Xu et al., 2022), but they often struggle with modelling long-range dependencies. To address this limitation, transformers with self-attention mechanisms have been increasingly adopted, enabling the capture of both local and global context (Tu et al., 2024).

Recent advancements in deep learning have focused on integrating multimodal data sources and designing hybrid architectures that combine CNNs, transformers, and GCNs to enhance predictive accuracy (Xu et al., 2024). Strengthened residual graph convolutional networks have been proposed to aggregate spatial features while maintaining node-level information, addressing the issue of homogeneous feature propagation. Similarly, multiscale gated guided CNNs leverage different kernel sizes to capture fine-grained spatial details at various scales, improving feature representation for classification and change

<footnote>a Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy</footnote>

<footnote>b National Research Council of Italy, Institute of Applied Sciences and Intelligent Systems, via Monteroni snc, 73100 Lecce, Italy</footnote>

<footnote>c Institute d'Electronique de Microelectronique et de Nanotechnologie (IEMN), UMR 8520, Universite Polytechnique Hauts de France, Universite de Lille, CNRS, 59313 Valenciennes, France</footnote>

<footnote>* Corresponding author at: National Research Council of Italy, Institute of Applied Sciences and Intelligent Systems, via Monteroni snc, 73100 Lecce, Italy.
*E-mail addresses:* mohamedfadhlallah.guerri@unisalento.it (M.F. Guerri), cosimo.distante@cnr.it (C. Distante), paolo.spagnolo@cnr.it (P. Spagnolo), Abdelmalik.Taleb-Ahmed@uphf.fr (A. Taleb-Ahmed).</footnote>
detection tasks. These innovations highlight the evolving landscape of deep learning, where hybrid models are increasingly employed to overcome the limitations of individual architectures and enhance performance in domains such as remote sensing, medical imaging, and autonomous systems.

New hybrid models that integrate CNNs, transformers, and state-space models (SSMs) (Gu and Dao, 2023) to enhance computational efficiency and performance have been recently introduced. The 3D-Spectral-Spatial Mamba (3DSS-Mamba) framework (He et al., 2024) leverages SSMs for efficient spectral-spatial modelling, addressing the computational challenges posed by self-attention mechanisms in transformers. By incorporating spectral-spatial token generation and selective scanning, this model improves classification accuracy while maintaining linear computational complexity, demonstrating its potential for real-world HSI applications.

Throughout the evolution of HSI classification, an extensive range of traditional methodologies have been proposed to refine classification accuracy. At their inception, HSI classification strategies were anchored in linear spectral transformations, particularly principal component analysis (PCA) (Beirami and Mokhtarzade, 2020), discriminant analysis (Imani and Ghasseman, 2014; Li et al., 2011), and independent component analysis (ICA) (Xia et al., 2016). These initial approaches tapped into the vast spectral data reservoir to categorize a variety of land covers effectively. In contrast, spectral classifier-based methods (Li et al., 2023), which are highly regarded for their enhanced generalization abilities and non-linear expression, have proven to be more effective in HSI classification. This genre includes sophisticated algorithms like k-nearest neighbour (Cariou and Chehdi, 2016), random forest (Xia et al., 2017), and support vector machine (SVM) (Chen et al., 2021). Nonetheless, owing to the prevalent high intra-class spectral variability and minimal inter-class spectral differentiation, spectral-based techniques often falter in accurately distinguishing between disparate entities and are vulnerable to unanticipated noise (Liu et al., 2024).

Traditional machine learning techniques face challenges in feature extraction, modelling nonlinear relationships, and managing high-dimensional data in HSI (Li et al., 2019; Guerri et al., 2024; Cheng et al., 2022; Zhou et al., 2024). On the other hand, Deep Learning (DL) presents benefits such as automatic feature learning, powerful nonlinear modelling, efficient data representation, and data augmentation for better generalization (Bougourzi et al., 2023; Guerri et al., 2024; Nejad et al., 2022). These advantages position DL as highly effective for the complex, high-dimensional, and non-linear nature of HSI data, significantly improving the accuracy and robustness of classification (Xu et al., 2022).

Due to their potent ability to extract both local spatial and spectral features, CNNs have exhibited remarkable efficacy in HSI classification tasks (Mohammadi and Sharifi, 2021; Esmaeili et al., 2023; Ren et al., 2024). To address the challenges posed by high-dimensional HSI data, a Multiscale 3D deep convolutional neural network (M3D-DCNN) was introduced by He et al. (2017), leveraging 3-D CNNs to mine spatial and spectral features within HSI data effectively. Additionally, the advent of more streamlined CNN models has been facilitated through the integration of group convolution techniques and attention mechanisms. Haut et al. (2019) innovatively integrated CNNs and Residual Networks (ResNets) with visual attention to enhance feature identification, pinpointing the most significant parts of the data. This approach was proven by experimental evidence to endow deep attention models with a considerable competitive edge. Sun et al. (2019) proved that the efficacy of CNN-based methods was compromised by interfering pixels, which diluted the discriminative capacity of spatial-spectral features and introduced a Spectral-Spatial Attention Network (SSAN). This network adeptly isolates discriminative spatial-spectral features within HSI's focused attention areas.

A notable development in HSI is the GroupFormer model (Khan et al., 2024), which employs group-based deep learning techniques to

improve classification performance. By grouping similar spectral bands, this approach enhances feature extraction and reduces computational complexity. However, the model's reliance on predefined groupings may limit its adaptability to diverse datasets, potentially affecting its generalizability across various HSI applications.

Another significant contribution is introducing a new band selection framework utilizing multi-objective optimization techniques (Phaneendra Kumar et al., 2024). This framework aims to select the most informative spectral bands from hyperspectral images, thereby reducing data redundancy and enhancing classifier accuracy. While this method effectively addresses the dimensionality reduction challenge, it may overlook subtle spectral features essential for certain classification tasks. Moreover, the optimization process can be computationally intensive, potentially hindering its application in scenarios requiring rapid analysis.

In this paper, we propose a CNN-Transformer approach for HSI data classification. The proposed approach exploits two types of convolutional operations (2D and 3D): a Gate-shift fuse (GSF) block and a Transformer block. The proposed approach is highly efficient for HSI data classification by employing the strengths of CNN blocks for extracting local features and the Transformer blocks for long-range context modelling. The main contributions of this paper can be summarized as follows:

* We introduced the GSF designed specifically for HSI classification; GSF can strengthen CNNs and transformers in extracting local and global spatial-spectral features.

* We propose an effective attention mechanism module which enables better extraction of local and global information from HSI cubes.

* We conduct a comparison analysis on four well-known datasets, where the proposed framework obtained superior results compared to other models.

The rest of this article is structured as follows: Section 3 introduces the proposed model. Section 4 provides details on the experimental datasets, the experimental parameter setup, and the comparison of classification results. The discussion, related work, and conclusions are presented in Sections 2, 5, and 6.

## 2. Related works

In Sun et al. (2019), a lightweight spectral-spatial attention network (LSSAN) for HSI classification was designed, achieving high accuracy with a substantial reduction in computational demand. Further, Gao et al. (2020) introduced a multi-scale residual network (MSRN) that employs mixed depth-wise separable convolution for HSI classification, offering enhanced results with reduced computation. CNNs have significantly advanced HSI classification performance, but their fixed convolutional kernel sizes pose a challenge in capturing the long-range dependencies within the intricate HSI cube. Consequently, achieving superior classification performance often necessitates additional CNN layers, which runs counter to the objective of lightweight modelling. With the introduction of the vision transformer (ViT) model (Dosovitskiy et al., 2020) and its applications in image processing, transformer-based networks have also found their way into HSI classification. The SpectralFormer (SF) network, detailed in Hong et al. (2021), adeptly learns spectrally local sequence information through group-wise spectral embeddings. There is also a growing trend among researchers to leverage CNNs to extract shallow features before deploying transformers.

The study by Cui et al. (2023) presents MADANet, a lightweight network employing multi-scale feature aggregation and a dual attention mechanism. MADANet leverages depth-wise separable convolution for extracting multi-scale features, significantly enhancing local contextual information capture. Concurrently, its dual attention mechanism
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** The proposed framework for the HSI classification. The process begins with PCA to reduce the spectral dimensionality of the HSI data. Then, the data undergo a spatial feature extraction phase using 3D and 2D convolution layers (Conv3D, Conv2D). The extracted features are then processed through the GSF block, enhancing the local and global feature representation. A tokenizer is used to convert the features into a sequence of tokens, which are fed into a transformer encoder used to capture long-range dependencies and high-level semantic features. Finally, the output passes through a linear layer, and a softmax activation to classify the hyperspectral pixels, with the classification result presented as a colour-coded map.</figcaption></figure>

optimizes channel and spatial dimensions, providing a comprehensive understanding of global semantic information. MADANet's efficacy is demonstrated in scenarios with varying complexity, from agricultural to urban settings.

In Zhang et al. (2023), a transformer-based framework, Multi-Range Spectral-Spatial Transformer (MRSST) was presented for HSI classification. A strategy for constructing composite token sequences, significantly enhancing the transformer's performance. The proposed MRSST architecture includes a convolutional feature pre-encoder with dual branches to capture localized features within each spectral band and a composite token generator that amalgamates spectral information and encoded features. This process results in token sequences enriched with multi-range information, enabling a more meticulous analysis of spectral band interdependencies. Experiments on three well-known datasets demonstrate that the multi-range composite token sequences and information exchange mechanisms notably improve transformer performance.

The authors in Gu et al. (2024) introduced an innovative deep-learning framework that aims to classify single trawl fishing vessels by integrating both structured data (AIS and radar) and unstructured data (trajectory and recurrence plot images). The authors constructed the HN-STV dataset, proposed a Double-Tower Transformer for comprehensive multi-dimensional feature extraction, and incorporated a Fusion Former visual module to effectively process image-based representations. Additionally, a Fast Attention mechanism was utilized to enhance computational efficiency, and a Dual-gate structure was implemented to dynamically optimize input weighting. Experimental results demonstrated that the MFGTN model outperformed previous methods, achieving up to a 2.95% improvement in accuracy along with significant enhancements in recall, precision, and F1-score, thus providing a robust and efficient solution for real-time monitoring of marine fishing activities.

To conclude, utilizing transformer backbone networks in HSI classification is a viable approach, thanks to their enhanced global receptive field and superior ability to extract features from sequential data. Nonetheless, a notable downside is the substantial computational expense involved, as every position needs to be calculated at each process step.

## 3. Methods

Fig. 1 shows the overall framework for the HSI classification based on the proposed model composed of four parts—the spectral-spatial feature extraction, GSF, the Gaussian weighted feature tokenizer, and the transformer-encoder (TE) module.

### 3.1. Spectral-spatial feature extraction

To handle the high-dimensional nature of HSI data, we employed Principal Component Analysis (PCA) for spectral dimension reduction, ensuring computational efficiency. The processed HSI data is then transformed into a spatial grid structure with reduced spectral bands. To extract meaningful spectral-spatial information, we implemented a hybrid convolutional architecture, combining 3D and 2D CNN layers:

* 3D Convolutional Layers: Extract deep spectral-spatial features from each patch while preserving local spectral continuity.

* 2D Convolutional Layers: Capture finer spatial details within the spectral bands.

This hybrid architecture extends traditional CNN-based methods by leveraging multi-resolution spatial-spectral representations.

The initial HSI dataset is represented as $I \in \mathbb{R}^{m \times n \times l}$, indicating a spatial grid of $m \times n$ with $l$ spectral bands. In this configuration, each pixel in $I$ encompasses $l$ spectral values, constituting a one-hot category vector $Y = (y_1, y_2, ..., y_C) \in \mathbb{R}^{1 \times 1 \times C}$ to represent the total $C$ land-cover classes. The extensive spectral bands render the HSI data computationally intensive. To mitigate this, Principal Component Analysis (PCA) is employed to streamline both computation and spectral dimensions by reducing the spectral bands from $l$ to $b$, producing a transformed dataset $I_{PCA} \in \mathbb{R}^{m \times n \times b}$, where $b$ signifies the reduced number of spectral bands.

Subsequently, a 3-D patch extraction process is applied to $I_{PCA}$, generating patches $P \in \mathbb{R}^{s \times s \times b}$, where $s \times s$ defines the patch dimensions. The position of a patch's centre pixel is marked as $(x_i, y_j)$, constrained by $0 \le i < m$ and $0 \le j < n$, with the centre pixel's label
<figure><img src="image_6.png" /><figcaption>**Fig. 2.** In the GSF framework, the integration of group gating, forward and backward spectral shift, and fusion mechanisms are employed. A single 3D convolution kernel facilitates the gating mechanism, finely tuned with tanh calibration. For the fusion process, a single 2D convolution kernel is used, refined with sigmoid calibration. As a result, the adoption of GSF introduces a negligible increase in parameters.</figcaption></figure>

<table><caption>**Table 1**

Classification results on different methods for the Indian Pines Dataset. Overall accuracy (OA), Average accuracy (AA) and Kappa Coefficient (κ × 100).</caption><thead><tr><th>NO.</th><th>SVM</th><th>EMAP (Dalla Mura et al., 2010)</th><th>1D-CNN (Hu et al., 2015)</th><th>2D-CNN (Zhao and Du, 2016)</th><th>3D-CNN (Chen et al., 2016)</th><th>SSRN (Zhong et al., 2017)</th><th>Cubic-CNN (Wang et al., 2020)</th><th>DualMamba (Sheng et al., 2024)</th><th>Ours</th></tr></thead><tbody><tr><td>1</td><td>65.63</td><td>62.50</td><td>43.75</td><td>48.78</td><td>41.46</td><td>83.15</td><td>87.86</td><td><b>96.83</b></td><td>95.12</td></tr><tr><td>2</td><td>63.44</td><td>81.57</td><td>77.93</td><td>78.13</td><td>90.51</td><td>95.31</td><td>96.35</td><td>99.07</td><td><b>99.67</b></td></tr><tr><td>3</td><td>60.25</td><td>83.19</td><td>56.72</td><td>83.51</td><td>79.36</td><td>94.23</td><td>93.65</td><td>99.00</td><td><b>99.18</b></td></tr><tr><td>4</td><td>41.11</td><td>85.89</td><td>45.18</td><td>47.42</td><td>46.01</td><td>90.68</td><td>82.54</td><td>98.29</td><td><b>98.55</b></td></tr><tr><td>5</td><td>87.05</td><td>78.61</td><td>87.57</td><td>75.12</td><td>95.17</td><td>97.79</td><td>96.69</td><td><b>97.91</b></td><td>96.32</td></tr><tr><td>6</td><td>97.21</td><td>79.08</td><td>98.63</td><td>92.99</td><td><b>99.70</b></td><td>98.67</td><td>96.69</td><td>99.54</td><td>99.54</td></tr><tr><td>7</td><td>89.47</td><td>52.63</td><td>65.11</td><td>60.00</td><td>88.00</td><td>97.92</td><td>90.16</td><td>99.85</td><td><b>100</b></td></tr><tr><td>8</td><td>96.66</td><td>91.19</td><td>97.36</td><td>98.37</td><td><b>100</b></td><td>99.26</td><td>98.46</td><td><b>100</b></td><td><b>100</b></td></tr><tr><td>9</td><td>32.26</td><td>50.12</td><td>37.14</td><td>66.67</td><td>48.89</td><td>89.49</td><td>89.93</td><td><b>100</b></td><td><b>100</b></td></tr><tr><td>10</td><td>73.84</td><td>81.32</td><td>66.03</td><td>87.77</td><td>86.06</td><td>97.48</td><td>93.94</td><td>98.29</td><td><b>99.16</b></td></tr><tr><td>11</td><td>84.36</td><td>86.91</td><td>82.49</td><td>89.09</td><td>97.51</td><td>98.16</td><td>97.45</td><td><b>99.58</b></td><td>98.69</td></tr><tr><td>12</td><td>42.89</td><td>78.43</td><td>73.49</td><td>63.67</td><td>74.91</td><td>93.07</td><td>93.18</td><td>99.01</td><td><b>99.13</b></td></tr><tr><td>13</td><td>98.58</td><td>96.35</td><td>99.30</td><td><b>100</b></td><td>99.46</td><td>98.59</td><td>99.12</td><td>99.73</td><td><b>100</b></td></tr><tr><td>14</td><td>94.02</td><td>93.91</td><td>93.78</td><td>95.33</td><td>99.74</td><td>99.72</td><td>99.39</td><td>99.72</td><td><b>99.91</b></td></tr><tr><td>15</td><td>42.65</td><td>77.36</td><td>55.39</td><td>66.76</td><td>84.10</td><td>93.31</td><td>84.26</td><td>92.35</td><td><b>98.84</b></td></tr><tr><td>16</td><td>92.19</td><td>84.38</td><td>81.54</td><td>91.57</td><td>93.98</td><td>93.79</td><td>89.69</td><td>97.98</td><td><b>99.08</b></td></tr><tr><td>OA(%)</td><td>76.39</td><td>83.69</td><td>79.37</td><td>84.47</td><td>91.03</td><td>94.78</td><td>94.90</td><td>99.23</td><td><b>99.47</b></td></tr><tr><td>AA(%)</td><td>72.18</td><td>76.53</td><td>70.87</td><td>77.83</td><td>82.18</td><td>94.67</td><td>93.85</td><td>99.11</td><td><b>99.57</b></td></tr><tr><td>κ × 100</td><td>72.85</td><td>81.40</td><td>76.28</td><td>82.24</td><td>89.68</td><td>94.08</td><td>94.17</td><td>99.08</td><td><b>99.11</b></td></tr></tbody></table>

assigning the patch's true classification. To accommodate edge pixels and ensure full patch extraction, padding of width $(s-1)/2$ is applied. This methodology yields a total of $m \times n$ 3-D patches, each spanning a width of $x_i - (s-1)/2$ to $x_i + (s-1)/2$ and a height of $x_j - (s-1)/2$ to $x_j + (s-1)/2$, inclusive of all b spectral bands.

Following the exclusion of patches with null labels, the residual patches are assorted into training and testing sets.

Two convolution layers (3-D and 2-D) are utilized to extract detailed spectral-spatial characteristics from each patch. Here, a 3-D convolution layer takes in training sample patches of dimension $s \times s \times b$, with each layer's jth feature cube's spatial position $(\alpha, \beta, \gamma)$ value being computed via a designated convolution operation.

## 3.2. Gate-shift-fuse

Originating from Sudhakaran et al. (2023), the GSF framework features a spatial gating block and a subsequent fusion block, both of which are adaptable and designed to process dual splits of incoming feature sets. The gating block acts selectively, channelling features to emphasize spectral details. The fusion block then integrates these enriched spectral features with the spatial dimensions. Both components

are dynamic, adjusting their operations based on the input data, as illustrated in Fig. 2.

For the Gate-Shift-Fuse (GSF) operation, consider an input tensor $X$ of dimensions $C \times T \times W \times H$, where $C$ represents the number of feature channels, $T$ denotes the spectral dimension, and $W$ and $H$ are the spatial dimensions.

Following spatial convolution derived from a conventional 2D CNN model, $X$ enters the gating mechanism. Here, it is divided into two segments along the channel axis (C), creating two sets of gating planes, each of shape $\frac{C}{2} \times T \times W \times H$, for the feature groups. These planes, acting as spatial weight maps, modulate the feature groups, resulting in gated features alongside their residuals. These are then spectrally shifted within their respective groups.

Let $X = [X_1, X_2]$ be the group-wise split of $X$ along the channel dimension. Given gating kernels $W_{g1}, W_{g2}$ of shape $1 \times C/2 \times 3 \times 3$, and fusion kernels $W_{f1}, W_{f2}$ of shape $1 \times 2 \times 3 \times 3$, the GSF output $Z = [Z_1, Z_2]$ is computed as follows:

$$Y_1 = \tanh(W_{g1} * X_1) \odot X_1, \qquad (1)$$

$$Y_2 = \tanh(W_{g2} * X_2) \odot X_2, \qquad (2)$$

$$R_1 = X_1 - Y_1, \qquad (3)$$
$$R_2 = X_2 - Y_2, \quad (4)$$

$$Y_1^{shift} = shift\_fw(Y_1), \quad (5)$$

$$Y_2^{shift} = shift\_bw(Y_2), \quad (6)$$

$$F_1 = \sigma(W_{f1} * [\phi(Y_1^{shift}; R_1)]), \quad (7)$$

$$F_2 = \sigma(W_{f2} * [\phi(Y_2^{shift}; R_2)]), \quad (8)$$

$$Z_1 = F_1 \odot Y_1^{shift} + (1 - F_1) \odot R_1, \quad (9)$$

$$Z_2 = F_2 \odot Y_2^{shift} + (1 - F_2) \odot R_2, \quad (10)$$

where “⊙” is the pointwise multiplication. At the fusion stage, a channel-specific weight map of C×T×1×1 for each group is generated. A weighted averaging process, leveraging these weight maps, fuses the shifted features and residuals. Through this process, GSF adeptly merges spatial and spectral data, guided by its trainable gating and fusion components.

To establish the isomorphism between the feature spaces of CNNs and Transformers, we employ feature space transformations, kernel mapping (Gretton, 2013), and manifold alignment (Correa et al., 2023) to ensure effective fusion within the GSF module.

CNNs operate in a *Euclidean feature space*, where convolutional filters extract *localized spatial features*, forming structured representations with translation equivariance. In contrast, Transformers process global dependencies via *self-attention mechanisms*, capturing relationships in a *high-dimensional, non-Euclidean space*. To address this discrepancy, we introduce a learned projection function $\phi: \mathbb{R}^{d_1} \rightarrow \mathbb{R}^{d_2}$ that ensures CNN and Transformer embeddings share a common latent space.

We define a kernel-induced transformation where feature embeddings are projected into a *reproducing kernel Hilbert space (RKHS)*. The transformation is given by:

$$K(x, y) = \langle \phi(x), \phi(y) \rangle, \quad x, y \in \mathbb{R}^{d_1}, \quad (11)$$

where $K(x, y)$ is a kernel function that preserves feature similarity between CNN-extracted and Transformer-extracted embeddings. By enforcing $\phi$ to be a *bijectional transformation*, we ensure that the feature spaces remain *structurally equivalent* and isomorphic.

To further guarantee feature space consistency, we employ *manifold alignment*, minimizing the Wasserstein distance between the CNN and Transformer feature distributions:

$$W_2(P_{CNN}, P_{Transformer}) = \inf_{\gamma \in \Pi(P_{CNN}, P_{Transformer})} \mathbb{E}_{(x,y) \sim \gamma} [\|x - y\|^2], \quad (12)$$

where $\Pi(P_{CNN}, P_{Transformer})$ represents the set of all joint distributions with marginal distributions $P_{CNN}$ and $P_{Transformer}$. Minimizing $W_2$ ensures that semantically similar features from CNN and Transformer representations are aligned.

A potential challenge in multi-scale feature extraction is the distribution shift problem, where feature distributions may drift as the network depth increases. To mitigate this, the GSF framework inherently incorporates adaptive feature weighting through a spatial gating block, ensuring feature distributions remain consistent across different scales. The forward-backward shift operation aids in aligning spectral features over different network layers, reducing inconsistencies that could affect classification accuracy. Additionally, the fusion block employs channel-wise weighting, dynamically adjusting feature integration based on spatial and spectral contexts. These mechanisms collectively ensure that multi-scale feature fusion remains effective despite distribution variations across layers, thereby preserving classification performance.

The GSF block is a key contribution that enhances feature fusion. By incorporating trainable gating and shift operations, the GSF block extends conventional feature extraction by dynamically emphasizing key spectral components while suppressing redundant information.

### 3.3. Gaussian-weighted feature tokenizer

While the GSF operation extracts features imbued with spectral and spatial attributes, these features alone fall short of comprehensively characterizing the attributes of terrestrial objects. Hence, feature maps are reinterpreted as semantic tokens to encapsulate and manipulate the advanced semantic constructs inherent to HSI feature categories. This redefinition initiates with the input feature map, denoted as $X \in \mathbb{R}^{u \times v \times w}$, where $u, v$, and $w$ symbolize height, width, and channel count, respectively. Semantic tokens are articulated as $T \in \mathbb{R}^{w \times z}$, with $w$ signifying the token count.

The objective here is to transpose $X$ into a semantic cohort, resulting in a semantic grouping represented by $A \in \mathbb{R}^{u \times v \times w}$. Subsequently, $A$ undergoes a transposition and is refined through the application of softmax(·) to accentuate segments of paramount semantic significance. The culmination of this process involves the multiplication of $A$ with $X$, thereby yielding $T$ as the designated semantic tokens.

### 3.4. Transformer encoder module

The TE module leverages semantic tokens generated earlier to discern relationships among high-level semantic features, as illustrated in Fig. 3. This module is structured into three main components. The initial component employs position embedding to indicate the location of each semantic token, represented as $[T_1, T_2, ..., T_w]$. These tokens are merged with a learnable classification token $T_{cls}^0$, aimed at classification tasks. Following this, the positional information $PE_{pos}$ is encoded and merged with the token representations. The resulting sequence of embedded semantic tokens is:

$$T_{cls}^0, T_1, T_2, ..., T_w + PE_{pos}.$$

The TE block, a pivotal component, is crafted to model intricate relationships among the semantic tokens. It includes a Multi-Head Self-Attention (MSA) block, an MLP layer, and two Normalization Layers (LN), enhanced by residual skip connections before the MSA and MLP layers.

The MSA block, the cornerstone of the transformer architecture, employs the Self-Attention (SA) mechanism to adeptly capture feature sequence correlations. It starts with defining three learnable matrices, $W_Q$, $W_K$, and $W_V$, for diverse interpretations. Mapping the tokens to these matrices forms 3-D matrices of queries $Q$, keys $K$, and values $V$. The attention score is derived from $Q$ and $K$, with the softmax function allocating the weights. SA is defined as:

$$SA = \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_K}}\right) V,$$

with $d_K$ being the key's dimension.

The MSA block maps $Q$, $K$, and $V$ using several weight matrix groups, performing the same operation to compute multi-head attention. The outcomes are concatenated:

$$MSA(Q, K, V) = \text{Concat}(SA_1, SA_2, ..., SA_h)W,$$

with $h$ indicating the head count, and $W$ the parameter matrix, sized at $h \times d_K \times d_W$ ($d_W$ is the token count).

Subsequently, this matrix enters an MLP layer with two fully connected layers and a Gaussian error linear unit in between. Following this, LN layers aid in training speed and mitigate gradient issues.

The TE module ensures the input ($T_{in}$) and output ($T_{out}$) sizes match. The classification token $T_{out}$ goes to the final classifier's linear layer, where a softmax function computes the input's category probability, with the highest value determining the category.

Our method enhances the HSI classification frameworks by integrating CNN-based local feature extraction with Transformer-based long-range modelling, resulting in a more comprehensive feature representation.
<table><caption>**Table 2**
Classification results on different methods for the Pavia university dataset.</caption><thead><tr><th>NO.</th><th>SVM</th><th>EMAP (Dalla Mura et al., 2010)</th><th>1D-CNN (Hu et al., 2015)</th><th>2D-CNN (Zhao and Du, 2016)</th><th>3D-CNN (Chen et al., 2016)</th><th>SSRN (Zhong et al., 2017)</th><th>Cubic-CNN (Wang et al., 2020)</th><th>DualMamba (Sheng et al., 2024)</th><th>Ours</th></tr></thead><tbody><tr><td>1</td><td>91.22</td><td>89.74</td><td>84.91</td><td>93.28</td><td>93.30</td><td>95.35</td><td>94.48</td><td>95.51</td><td><strong>99.33</strong></td></tr><tr><td>2</td><td>97.78</td><td>93.50</td><td>94.14</td><td>94.90</td><td>93.99</td><td>94.69</td><td>93.96</td><td>99.49</td><td><strong>99.92</strong></td></tr><tr><td>3</td><td>34.95</td><td>42.06</td><td>47.38</td><td>75.55</td><td>90.19</td><td>96.48</td><td>93.49</td><td>94.18</td><td><strong>98.29</strong></td></tr><tr><td>4</td><td>81.55</td><td>81.54</td><td>82.28</td><td>93.87</td><td>91.29</td><td>96.37</td><td>98.01</td><td><strong>99.55</strong></td><td>98.49</td></tr><tr><td>5</td><td>98.59</td><td>97.96</td><td>99.76</td><td>97.98</td><td>95.47</td><td>99.69</td><td><strong>99.76</strong></td><td>96.71</td><td>99.53</td></tr><tr><td>6</td><td>41.50</td><td>48.18</td><td>77.67</td><td>70.05</td><td>93.85</td><td>97.49</td><td>93.48</td><td>99.43</td><td><strong>100.00</strong></td></tr><tr><td>7</td><td>16.71</td><td>31.58</td><td>19.40</td><td>70.92</td><td>81.45</td><td>95.36</td><td>96.48</td><td><strong>100.00</strong></td><td>99.13</td></tr><tr><td>8</td><td>89.74</td><td>91.45</td><td>70.01</td><td>90.30</td><td>92.73</td><td>91.49</td><td>92.51</td><td>95.97</td><td><strong>98.05</strong></td></tr><tr><td>9</td><td><strong>99.89</strong></td><td>98.78</td><td>98.55</td><td>97.89</td><td>95.46</td><td>95.90</td><td>95.35</td><td>95.22</td><td>95.44</td></tr><tr><td>OA(%)</td><td>82.76</td><td>84.89</td><td>83.50</td><td>90.19</td><td>92.01</td><td>95.87</td><td>95.68</td><td>98.16</td><td><strong>99.21</strong></td></tr><tr><td>AA(%)</td><td>72.44</td><td>76.36</td><td>74.90</td><td>87.31</td><td>90.45</td><td>95.86</td><td>95.28</td><td>97.35</td><td><strong>98.69</strong></td></tr><tr><td>κ × 100</td><td>76.28</td><td>80.41</td><td>77.90</td><td>87.52</td><td>90.87</td><td>95.78</td><td>95.55</td><td>97.57</td><td><strong>99.15</strong></td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** (a) The main Transformer Encoder framework. (b) Multi-Head Self-Attention (MSA). (c) Self-Attention (SA) module.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 4.** Ground-truth image of WHU-Hi-LongKou dataset.</figcaption></figure>

### 3.5. Implementation

This model distinguishes itself from traditional CNN frameworks by minimizing the layer count and introducing the GSF, a tokenizer, and TE to interpret the semantic levels of image patches. We showcase this with the Pavia University dataset, sized 610 × 340 × 103 (see Table 2).

<table><caption>**Table 3**
Groundtruth classes for the WHU-Hi-LongKou dataset.</caption><thead><tr><th>No.</th><th>Class name</th><th>Samples</th><th>Legend</th></tr></thead><tbody><tr><td>C1</td><td>Corn</td><td>34511</td><td></td></tr><tr><td>C2</td><td>Cotton</td><td>8374</td><td></td></tr><tr><td>C3</td><td>Sesame</td><td>3031</td><td></td></tr><tr><td>C4</td><td>Broad-leaf soybean</td><td>63212</td><td></td></tr><tr><td>C5</td><td>Narrow-leaf soybean</td><td>4151</td><td></td></tr><tr><td>C6</td><td>Rice</td><td>11854</td><td></td></tr><tr><td>C7</td><td>Water</td><td>67056</td><td></td></tr><tr><td>C8</td><td>Roads and houses</td><td>7124</td><td></td></tr><tr><td>C9</td><td>Mixed weed</td><td>5229</td><td></td></tr></tbody></table>

Post-PCA reduction and patch extraction, patches are 13 × 13 × 30. The foremost 3-D convolution layer produces eight 11 × 11 × 28 feature cubes, using eight 3 × 3 × 3 cube kernels on each patch, harnessing the extensive spectral information present. These cubes are merged into one 11 × 11 × 224 cube. Then, 2-D convolution with 64 plane kernels of 3 × 3 generates 64 9 × 9 feature maps, each flattened into a 1-D 81-element vector, culminating in $X \in \mathbb{R}^{81 \times 64}$.

Next, utilizing the Xavier normal distribution, we derive the initial weight matrix $W_a \in \mathbb{R}^{64 \times 4}$ to normalize the feature distribution. This matrix is applied to the feature vectors, producing the semantic array $A \in \mathbb{R}^{81 \times 4}$. The transposition of $A$ with $X$ forms semantic tokens $T \in \mathbb{R}^{4 \times 64}$. Integrating a zero vector into $T$ as an adjustable token with position markers, we obtain $T_{in} \in \mathbb{R}^{5 \times 64}$, which, after processing of the TE module, visualizes the semantic features. The primary output of the classification token, $T_{out}^0 \in \mathbb{R}^{1 \times 64}$, is used as the input of the classifier to finalize the label (see Fig. 4).
<table><caption>**Table 4**
Classification results on different methods for the WHU-HI LongKou.</caption><thead><tr><th>NO.</th><th>2DCNN (Zhao and Du, 2016)</th><th>3DCNN (Chen et al., 2016)</th><th>FC3DCN (Ahmad et al., 2020)</th><th>HybridSN (Roy et al., 2019)</th><th>S3EResBoF (Roy et al., 2020a)</th><th>A2S2KRes (Roy et al., 2020c)</th><th>FuSENet (Roy et al., 2020b)</th><th>DualMamba (Sheng et al., 2024)</th><th>Ours</th></tr></thead><tbody><tr><td>1</td><td>99.95</td><td>99.89</td><td>99.90</td><td>99.99</td><td>99.97</td><td>99.31</td><td>99.83</td><td><b>99.99</b></td><td>99.96</td></tr><tr><td>2</td><td>98.89</td><td>99.20</td><td>99.26</td><td>99.87</td><td>99.01</td><td>99.30</td><td>99.42</td><td>99.89</td><td><b>99.90</b></td></tr><tr><td>3</td><td>97.48</td><td>99.83</td><td>99.76</td><td>98.98</td><td>100</td><td>99.13</td><td>99.82</td><td>99.42</td><td><b>100</b></td></tr><tr><td>4</td><td>99.67</td><td>99.71</td><td>99.62</td><td>99.46</td><td>99.88</td><td>99.81</td><td>99.31</td><td>99.85</td><td><b>99.94</b></td></tr><tr><td>5</td><td>97.07</td><td>98.46</td><td>97.76</td><td>97.59</td><td>99.46</td><td><b>99.84</b></td><td>99.54</td><td>98.34</td><td>98.86</td></tr><tr><td>6</td><td>99.57</td><td>99.86</td><td>99.92</td><td>99.88</td><td>99.99</td><td>99.63</td><td>99.97</td><td>99.87</td><td><b>100</b></td></tr><tr><td>7</td><td>99.98</td><td>99.97</td><td>99.98</td><td>99.96</td><td>99.88</td><td>99.98</td><td><b>99.99</b></td><td>99.98</td><td>99.97</td></tr><tr><td>8</td><td>95.45</td><td>98.16</td><td>98.41</td><td>98.16</td><td>92.85</td><td>98.26</td><td>95.79</td><td>98.36</td><td><b>99.02</b></td></tr><tr><td>9</td><td>94.75</td><td>94.08</td><td>97.06</td><td>98.13</td><td><b>99.55</b></td><td>97.64</td><td>98.91</td><td>97.08</td><td>98.05</td></tr><tr><td>OA(%)</td><td>99.42</td><td>99.59</td><td>99.64</td><td>99.63</td><td>99.59</td><td>99.63</td><td>99.54</td><td>99.76</td><td><b>99.86</b></td></tr><tr><td>AA(%)</td><td>98.09</td><td>98.79</td><td>99.07</td><td>99.11</td><td>98.96</td><td>99.21</td><td>99.18</td><td>99.20</td><td><b>99.52</b></td></tr><tr><td>κ × 100</td><td>99.24</td><td>99.46</td><td>99.64</td><td>99.52</td><td>99.46</td><td>99.51</td><td>99.40</td><td>99.68</td><td><b>99.81</b></td></tr></tbody></table>

# 4. Experiment and analysis

This section presents a comprehensive evaluation of our proposed method through rigorous experimentation and analysis. We begin by describing the HSI datasets used for training and validation, followed by a detailed explanation of the experimental setup, including hardware configurations and evaluation metrics. Subsequently, we provide an in-depth comparison of our method's classification performance against state-of-the-art techniques. The results are analysed to highlight the strengths and potential limitations of our approach, demonstrating its effectiveness in HSI classification, which is also supported by an ablation study.

## 4.1. Data description

Our method's performance was gauged through tests conducted on four Hyperspectral Imaging (HSI) datasets: Indian Pines, Pavia University, WHU-WHU-Hi-LongKou, and WHU-Hi-HanChuan. Captured by the AVIRIS Sensor in 1992 over Indiana, USA, the Indian Pines dataset contains 224 spectral bands ranging from 0.4 to 2.5 µm, covering 145 × 145 pixels at 20 m resolution, featuring 16 land-cover classes. After eliminating 24 bands due to noise and water absorption, we selected 200 bands for experimentation.

The Pavia University dataset, collected by the ROSIS Sensor in 2001 over Northern Italy, includes 115 spectral bands (0.43 to 0.86 µm) across a 610 × 340 pixel area with a 1.3 m resolution, categorizing nine land-cover types. Here, we discarded 12 noise bands, working with 103 bands.

Sourced from Hubei, China, using the Headwall NanoHyperspec sensor, the WHU-Hi datasets (WHU-Hi-LongKou and WHU-Hi-HanChuan) focus on agricultural lands. WHU-Hi-LongKou, with 9 classes and 270 spectral bands, covers 550 × 400 pixels at a 0.463 m resolution (see Table 3 and Fig. 4). WHU-Hi-HanChuan offers 16 classes across 274 spectral bands, spanning 1217 × 303 pixels with a resolution of 0.109 m (see Table 5 and Fig. 5).

## 4.2. Experimental setting

To ensure a fair and thorough evaluation of our proposed method, we designed a series of experiments leveraging multiple hyperspectral datasets. The experimental setup includes specific configurations for training, hyperparameter tuning, and implementation details within a deep learning framework. Additionally, we employed robust evaluation metrics to assess the classification performance, allowing for direct comparisons with existing state-of-the-art methods. The following subsections provide an overview of the evaluation criteria, computational resources, and training configurations used in our experiments.

<figure><img src="image_9.png" /><figcaption>**Fig. 5.** Ground-truth image of WHU-Hi-HanChuan dataset.</figcaption></figure>

### 4.2.1. Evaluation indicators

For a quantitative evaluation of our proposed method alongside others, we have employed four key quantitative indices: overall accuracy (OA), average accuracy (AA), kappa coefficient (κ × 100), and classification accuracy across individual land-cover categories. Higher values in these metrics denote improved classification outcomes.

### 4.2.2. Configuration

The verification experiments for our proposed approach were conducted within the PyTorch framework, utilizing an NVIDIA DGX A100 Workstation equipped with Dual AMD Rome 7742 CPUs, 2TB RAM, and eight NVIDIA A100 80 GB GPUs. We opted for the Adam optimizer as the primary optimizer, setting the initial learning rate at 1e-3. For batch processing, minibatches were configured to contain 64 samples each. We applied 200 training epochs to each dataset.

### 4.2.3. Classification results

To illustrate the proposed model's efficiency, a selection of representative methods was employed for comparison in experimental trials: SVM, EMAP (Dalla Mura et al., 2010), 1D-CNN (Hu et al., 2015), 2D-CNN (Zhao and Du, 2016), 3D-CNN (Chen et al., 2016), SSRN (Zhong et al., 2017), Cubic-CNN (Wang et al., 2020), HybridSN (Roy et al., 2019) for Indian Pines and Pavia University Datasets, and 2DCNN (Zhao and Du, 2016), 3DCNN (Chen et al., 2016), FC3DCN (Ahmad et al., 2020), HybridSN (Roy et al., 2019), S3EResBoF (Roy et al., 2020a), A2S2KRes (Roy et al., 2020c), FuSENet (Roy et al., 2020b), DLEM (Iyer et al., 2021) and our proposed method.
<table><caption>**Table 5**
Groundtruth classes for the WHU-Hi-HanChuan dataset.</caption><thead><tr><th>No.</th><th>Class name</th><th>Samples</th><th>Legend</th></tr></thead><tbody><tr><td>C1</td><td>Strawberry</td><td>44735</td><td></td></tr><tr><td>C2</td><td>Cowpea</td><td>22753</td><td></td></tr><tr><td>C3</td><td>Soybean</td><td>10287</td><td></td></tr><tr><td>C4</td><td>Sorghum</td><td>5353</td><td></td></tr><tr><td>C5</td><td>Water spinach</td><td>1200</td><td></td></tr><tr><td>C6</td><td>Watermelon</td><td>4533</td><td></td></tr><tr><td>C7</td><td>Greens</td><td>5903</td><td></td></tr><tr><td>C8</td><td>Trees</td><td>17978</td><td></td></tr><tr><td>C9</td><td>Grass</td><td>9469</td><td></td></tr><tr><td>C10</td><td>Red roof</td><td>10516</td><td></td></tr><tr><td>C11</td><td>Gray roof</td><td>16911</td><td></td></tr><tr><td>C12</td><td>Plastic</td><td>3679</td><td></td></tr><tr><td>C13</td><td>Bare soil</td><td>9116</td><td></td></tr><tr><td>C14</td><td>Road</td><td>18560</td><td></td></tr><tr><td>C15</td><td>Bright object</td><td>1136</td><td></td></tr><tr><td>C16</td><td>Water</td><td>75401</td><td></td></tr></tbody></table>

The performance of our proposed method and comparative methods across the Indian Pines, Pavia University, and WHU-HI datasets is detailed in Tables 1, 2, 4, and 6, assessing OA, AA, $κ$, and individual class accuracies. The optimal results are emphasized in bold, under- scoring our method's superior performance in achieving the highest OA, AA, $κ$, and class accuracy values. Particularly, Table 1 illustrates the challenges of unbalanced classification in specific categories like “alfalfa”, “grass-pasture-mowed”, and “oats” within SVM, EMAP, and 1-D-CNN methods due to limited sample sizes and the consequent sample imbalance from percentage sampling. Our method’s uniform class accuracy highlights its efficacy in classifying small samples based on regional semantic concepts.

In SVM classification, the RBF kernel was used, determining the best hyperparameters $λ$ and $λ$ via cross-validation. The EMAP method set the number of principal components to three and selected attribute profiles based on area, moment of inertia, and standard deviation, with predetermined scales for each. The 1-D-CNN consisted of five layers, including 20 1-D convolutional filters in the convolution layer. The 2-D-CNN architecture featured three convolutional blocks and two linear layers.

Within the 3-D-CNN model, the network is composed of three 3-D convolutional blocks and two linear layers, mirroring the structure of the 2-D-CNN. Each block within the 3-D-CNN setup includes a 3-D convolutional layer, a batch normalization (BN) layer, and a ReLU activation function, housing eight, 16, and 32 3-D filters sized $3 \times 3$, respectively.

Specific configurations for SSRN, Cubic-CNN, and HybridSN are detailed in their individual references. For our approach, we designated 30 principal components. The network’s 3-D convolution layer features eight kernels of $3 \times 3 \times 3$ dimensions, and the 2-D convolution layer includes 64 kernels of $3 \times 3$. Patch dimensions were set to $13 \times 13$, with the sample token count and the number of TE heads both fixed at four. A random selection process was employed to compile training and test sets across all methods, ensuring an equitable comparison.

Table 7 presents a qualitative comparison of classification results across three benchmark hyperspectral datasets: Indian Pines, Pavia University, and WHU-HI LongKou. The table reports the overall accuracy (OA), average accuracy (AA), and Kappa coefficient for various classification methods, including traditional machine learning models (SVM Chen et al., 2021, EMAP Dalla Mura et al., 2010), convolutional neural network-based approaches (1D-CNN Hu et al., 2015, 2D-CNN Zhao and Du, 2016, 3D-CNN Chen et al., 2016, SSRN Zhong et al., 2017, Cubic-CNN Wang et al., 2020), and more recent advanced models such as DualMamba (Sheng et al., 2024). The results indicate that our proposed method outperforms all other approaches across all datasets, achieving the highest OA, AA, and Kappa scores. Specifically, our approach demonstrates superior classification performance

on WHU-HI LongKou, with an OA of 99.86%, reinforcing its robustness in hyperspectral image classification. These results highlight the efficacy of the proposed model in capturing spectral-spatial dependencies and improving classification accuracy compared to state-of-the-art techniques.

### 4.3. Ablation experiments

In order to thoroughly showcase the effectiveness of our method, we undertook ablation studies on the WHU-HI HanChuan dataset, exploring five different configurations of component combinations. This analysis aimed to discern the impact of various components on the model’s overall classification accuracy, with all results detailed in Table 8. The analysis deconstructed the model into five principal components: the 3-D convolution layer, the 2-D convolution layer, the GSF, the tokenizer, and the TE. The absence of the 2-D convolution layer led to the least effective classification performance. Removing the 3-D convolution layer slightly improved outcomes compared to the absence of the 2-D layer. The model achieved 90.98% accuracy in the third configuration. In the fourth, replacing GSF with PE and keeping both convolution layers intact resulted in a commendable 97.51% accuracy, just shy of our method’s outcome, highlighting the GSF’s role in classification enhancement. The fifth configuration, relying solely on the convolution layers for spectral-spatial feature analysis, achieved 93.52% accuracy, demonstrating the significant contribution of the TE module towards performance improvement. These ablation study outcomes further validate our model’s robustness.

### 4.4. Time cost comparison

A comparison of training and testing time for 2-D-CNN, 3-D-CNN, SSRN, Cubic-CNN, HybridSN, and our method is shown in Table 9.

Our approach outpaces the competition in terms of speed, barring the 2-D-CNN, thus cutting down on computation time and boosting classification efficiency. Due to their more complex and deeper network structures, SSRN and Cubic-CNN experience longer durations in both training and testing phases, consuming extensive computational cycles for each iteration. The marginally longer computation time of our method than that of the 2-D-CNN can be credited to the employment of a 3-D convolution layer for extracting spectral features. Additionally, the time invested in residual calculations within the TE structure is noteworthy.

## 5. Discussion

In this study, we embarked on a journey to enhance HSI classification by ingeniously intertwining the strengths of CNNs and transformers within a unified architecture. Our proposed model, fortified with two convolutional blocks, a Gate-Shift-Fuse (GSF) block and a transformer block, has unveiled a new horizon in HSI classification, demonstrating remarkable prowess in harnessing local and global spatial-spectral features.

A pivotal cornerstone of our approach is introducing the GSF block, a novel mechanism meticulously designed to bolster the extraction of intricate spatial-spectral features. This innovation has proven instrumental in navigating the complexities and nuances inherent in HSI data, facilitating a more nuanced and robust feature extraction process. Coupled with an effective attention mechanism module, our model has exhibited a remarkable aptitude to recognize and leverage critical local information within HSI cubes. Our model has shown particular resilience in grappling with the challenges posed by unbalanced classification scenarios, showcasing its ability to maintain performance integrity even in the face of categories marked by smaller sample sizes.

In dissecting the performance landscapes, it becomes evident that the synergistic interplay between CNNs and transformers within our model is a powerful catalyst, driving enhanced classification outcomes.
<table><caption>**Table 6**
Classification results on different methods for the WHU-HI HanChuan.</caption><thead><tr><th>NO.</th><th>2DCNN (Zhao and Du, 2016)</th><th>3DCNN (Chen et al., 2016)</th><th>FC3DCN (Ahmad et al., 2020)</th><th>HybridSN (Roy et al., 2019)</th><th>S3EResBoF (Roy et al., 2020a)</th><th>A2S2KRes (Roy et al., 2020c)</th><th>FuSENet (Roy et al., 2020b)</th><th>DLEM (Iyer et al., 2021)</th><th>Ours</th></tr></thead><tbody><tr><td>1</td><td>96.61</td><td>96.80</td><td>98.46</td><td>97.19</td><td>91.61</td><td>95.26</td><td>97.49</td><td>98.41</td><td><strong>98.98</strong></td></tr><tr><td>2</td><td>93.65</td><td>93.14</td><td>95.73</td><td>94.38</td><td>95.07</td><td>97.44</td><td>97.16</td><td>96.69</td><td><strong>99.62</strong></td></tr><tr><td>3</td><td>86.85</td><td>91.03</td><td>89.49</td><td>89.96</td><td>92.12</td><td>94.71</td><td><strong>99.09</strong></td><td>94.38</td><td>98.82</td></tr><tr><td>4</td><td>98.21</td><td>97.82</td><td>98.46</td><td>98.28</td><td>98.14</td><td>97.67</td><td>99.61</td><td>98.77</td><td><strong>99.94</strong></td></tr><tr><td>5</td><td>69.59</td><td>67.18</td><td>74.74</td><td>61.43</td><td>96.94</td><td>89.86</td><td>87.30</td><td>90.55</td><td><strong>99.91</strong></td></tr><tr><td>6</td><td>68.41</td><td>67.36</td><td>69.82</td><td>70.00</td><td>79.46</td><td>78.51</td><td>57.36</td><td>75.96</td><td><strong>95.18</strong></td></tr><tr><td>7</td><td>88.02</td><td>90.24</td><td>87.53</td><td>87.63</td><td>93.61</td><td>91.80</td><td>93.58</td><td>93.24</td><td><strong>99.43</strong></td></tr><tr><td>8</td><td>87.54</td><td>90.41</td><td>94.72</td><td>91.68</td><td>94.11</td><td>92.96</td><td>92.87</td><td>95.30</td><td>98.55</td></tr><tr><td>9</td><td>89.13</td><td>89.01</td><td>90.58</td><td>92.75</td><td>92.31</td><td>92.99</td><td>91.03</td><td>94.28</td><td><strong>98.01</strong></td></tr><tr><td>10</td><td>99.01</td><td>97.80</td><td>98.13</td><td>98.99</td><td>98.33</td><td>98.84</td><td>96.78</td><td>98.67</td><td><strong>99.35</strong></td></tr><tr><td>11</td><td>96.74</td><td>96.96</td><td>98.85</td><td>98.25</td><td>92.37</td><td>96.42</td><td>94.90</td><td>97.51</td><td>98.83</td></tr><tr><td>12</td><td>66.74</td><td>87.31</td><td>80.44</td><td>69.36</td><td>84.98</td><td>84.95</td><td>57.56</td><td>82.18</td><td><strong>97.85</strong></td></tr><tr><td>13</td><td>77.10</td><td>75.30</td><td>81.08</td><td>75.67</td><td>76.54</td><td>86.51</td><td>67.25</td><td>84.27</td><td><strong>97.46</strong></td></tr><tr><td>14</td><td>90.05</td><td>91.82</td><td>87.45</td><td>91.65</td><td>91.31</td><td>98.42</td><td>98.74</td><td>94.68</td><td><strong>99.61</strong></td></tr><tr><td>15</td><td>57.89</td><td>73.23</td><td>84.94</td><td>72.41</td><td>96.10</td><td>91.42</td><td>97.43</td><td>84.93</td><td><strong>99.99</strong></td></tr><tr><td>16</td><td>99.33</td><td>99.73</td><td>99.71</td><td>99.38</td><td>99.95</td><td>99.78</td><td>99.65</td><td>99.76</td><td><strong>99.84</strong></td></tr><tr><td>OA(%)</td><td>93.40</td><td>94.28</td><td>95.09</td><td>94.41</td><td>94.27</td><td>96.87</td><td>94.05</td><td>96.47</td><td><strong>99.23</strong></td></tr><tr><td>AA(%)</td><td>85.30</td><td>87.82</td><td>89.38</td><td>86.94</td><td>92.06</td><td>92.97</td><td>89.24</td><td>92.47</td><td><strong>98.47</strong></td></tr><tr><td>κ × 100</td><td>93.30</td><td>94.25</td><td>93.45</td><td>93.28</td><td>96.34</td><td>93.05</td><td>95.87</td><td>95.09</td><td><strong>99.03</strong></td></tr></tbody></table>

<table><caption>**Table 7**
Qualitative comparison of classification results across three datasets (OA, AA, and Kappa score).</caption><thead><tr><th rowspan="2">Method</th><th colspan="3">Indian Pines</th><th colspan="3">Pavia university</th><th colspan="3">WHU-HI LongKou</th></tr><tr><th>OA (%)</th><th>AA (%)</th><th>Kappa</th><th>OA (%)</th><th>AA (%)</th><th>Kappa</th><th>OA (%)</th><th>AA (%)</th><th>Kappa</th></tr></thead><tbody><tr><td>SVM (Chen et al., 2021)</td><td>76.39</td><td>72.18</td><td>72.85</td><td>82.76</td><td>72.44</td><td>76.28</td><td>99.42</td><td>98.09</td><td>99.24</td></tr><tr><td>EMAP (Dalla Mura et al., 2010)</td><td>83.69</td><td>76.53</td><td>81.40</td><td>84.89</td><td>76.36</td><td>80.41</td><td>99.59</td><td>98.79</td><td>99.46</td></tr><tr><td>1D-CNN (Hu et al., 2015)</td><td>79.37</td><td>70.87</td><td>76.28</td><td>83.50</td><td>74.90</td><td>77.90</td><td>99.64</td><td>99.07</td><td>99.64</td></tr><tr><td>2D-CNN (Zhao and Du, 2016)</td><td>84.47</td><td>77.83</td><td>82.24</td><td>90.19</td><td>87.31</td><td>87.52</td><td>99.63</td><td>99.11</td><td>99.52</td></tr><tr><td>3D-CNN (Chen et al., 2016)</td><td>91.03</td><td>82.18</td><td>89.68</td><td>92.01</td><td>90.45</td><td>90.87</td><td>99.59</td><td>98.96</td><td>99.46</td></tr><tr><td>SSRN (Zhong et al., 2017)</td><td>94.78</td><td>94.67</td><td>94.08</td><td>95.87</td><td>95.86</td><td>95.78</td><td>99.63</td><td>99.21</td><td>99.51</td></tr><tr><td>Cubic-CNN (Wang et al., 2020)</td><td>94.90</td><td>93.85</td><td>94.17</td><td>95.68</td><td>95.28</td><td>95.55</td><td>99.54</td><td>99.18</td><td>99.40</td></tr><tr><td>DualMamba (Sheng et al., 2024)</td><td>99.23</td><td>99.11</td><td>99.08</td><td>98.16</td><td>97.35</td><td>97.57</td><td>99.76</td><td>99.20</td><td>99.68</td></tr><tr><td>Ours</td><td><strong>99.47</strong></td><td><strong>99.57</strong></td><td><strong>99.11</strong></td><td><strong>99.21</strong></td><td><strong>98.69</strong></td><td><strong>99.15</strong></td><td><strong>99.86</strong></td><td><strong>99.52</strong></td><td><strong>99.81</strong></td></tr></tbody></table>

<table><caption>**Table 8**
Ablation analysis of the proposed model on WHU-HI HanChuan dataset.</caption><thead><tr><th rowspan="2">Cases</th><th colspan="4">Component</th><th colspan="3">Results</th></tr><tr><th>2D Conv</th><th>3D Conv</th><th>GSF</th><th>TE</th><th>OA (%)</th><th>AA (%)</th><th>κ × 100</th></tr></thead><tbody><tr><td>1</td><td>×</td><td>√</td><td>√</td><td>√</td><td>94.03</td><td>90.60</td><td>93.33</td></tr><tr><td>2</td><td>√</td><td>×</td><td>√</td><td>√</td><td>85.54</td><td>69.83</td><td>80.58</td></tr><tr><td>3</td><td>×</td><td>×</td><td>×</td><td>√</td><td>90.98</td><td>85.55</td><td>88.06</td></tr><tr><td>4</td><td>√</td><td>√</td><td>×</td><td>√</td><td>97.51</td><td>94.55</td><td>95.51</td></tr><tr><td>5</td><td>√</td><td>√</td><td>×</td><td>×</td><td>93.52</td><td>92.19</td><td>91.33</td></tr><tr><td>6</td><td>√</td><td>√</td><td>√</td><td>√</td><td><strong>99.23</strong></td><td><strong>98.47</strong></td><td><strong>99.03</strong></td></tr></tbody></table>

The CNN components, renowned for their local feature extraction prowess, seamlessly intertwine with the transformers' capacity for long-range context modelling, culminating in a harmonized architecture that resonates with the multifaceted nature of HSI data.

The proposed method for HSI classification holds significant potential across various real-world applications, leveraging its enhanced spatial-spectral feature extraction capabilities. In precision agriculture, HSI is widely employed for crop monitoring, disease detection, and soil analysis, allowing for early intervention and improved yield predictions (Gevaert et al., 2015). Remote sensing applications benefit from HSI for land cover classification and environmental monitoring, including deforestation tracking and pollution assessment (Jalayer et al., 2023; Virghileanu et al., 2020). Additionally, the mining industry utilizes hyperspectral imaging for mineral exploration, facilitating accurate identification of mineral compositions in different terrains (Wang et al., 2012). In the biomedical field, HSI is instrumental in detecting skin cancer, assessing tissue oxygenation, and enhancing medical diagnostics with non-invasive imaging techniques (Noor et al., 2016). Furthermore, in food safety and quality control, hyperspectral imaging

is used for detecting contaminants, assessing food freshness, and improving sorting mechanisms in agricultural produce (Fong et al., 2020). Military and defence applications also exploit hyperspectral imaging for surveillance, target detection, and camouflage material identification, providing strategic advantages in security operations (Ardouin et al., 2007). These diverse applications highlight the broad utility and impact of HSI classification, underscoring the importance of advancements in classification frameworks like the one proposed in this study.

## 6. Conclusion

This paper presents a novel method designed to boost HSI classification performance by organically fusing a backbone CNN with a transformer architecture. The convolution layers, enhanced by the GSF block, excel at extracting low-level spectral-spatial features. Subsequently, these features are transformed into semantic tokens. The method employs the TE structure to process the high-level semantic attributes of the tokens, facilitating a comprehensive analysis of land-cover traits. Experimentally, this approach has been shown to increase classification efficacy significantly. Furthermore, it convincingly illustrates the feasibility of expanding the scope of HSI classification into the realm of local high-level semantic classification.

The impact of different attention mechanisms, dimensionality reduction methods, and parameter settings on classification accuracy was also evaluated in this study. The choice of attention mechanism significantly influences feature extraction efficiency; for instance, self-attention in transformers effectively captures long-range dependencies, while local spatial attention mechanisms enhance fine-grained feature representation. Our model integrates a GSF mechanism, which dynamically refines spectral-spatial features, reducing redundancy and improving classification performance. Similarly, the dimensionality reduction approach plays a crucial role in managing high-dimensional
<table><caption>**Table 9**
Training time in minutes and test time is seconds between our method and other methods on three datasets.</caption><thead><tr><th rowspan="2">Methods</th><th colspan="2">Indian Pines</th><th colspan="2">Pavia university</th><th colspan="2">WHU-HI HuanChan</th></tr><tr><th>Train (m)</th><th>Test (s)</th><th>Train (m)</th><th>Test (s)</th><th>Train (m)</th><th>Test (s)</th></tr></thead><tbody><tr><td>2D-CNN (Zhao and Du, 2016)</td><td><strong>1.45</strong></td><td><strong>3.32</strong></td><td><strong>1.91</strong></td><td><strong>4.82</strong></td><td><strong>2.19</strong></td><td><strong>4.34</strong></td></tr><tr><td>3D-CNN (Chen et al., 2016)</td><td>7.26</td><td>6.90</td><td>11.48</td><td>14.62</td><td>8.29</td><td>9.10</td></tr><tr><td>SSRN (Zhong et al., 2017)</td><td>11.74</td><td>8.83</td><td>14.46</td><td>24.43</td><td>8.93</td><td>11.75</td></tr><tr><td>Cubic-CNN (Wang et al., 2020)</td><td>10.54</td><td>9.60</td><td>13.58</td><td>29.91</td><td>9.73</td><td>15.18</td></tr><tr><td>HybridSN (Roy et al., 2019)</td><td>6.61</td><td>7.93</td><td>9.32</td><td>28.40</td><td>6.73</td><td>8.49</td></tr><tr><td>Ours</td><td><strong>2.41</strong></td><td><strong>5.20</strong></td><td><strong>2.03</strong></td><td><strong>9.61</strong></td><td><strong>2.62</strong></td><td><strong>4.49</strong></td></tr></tbody></table>

HSI data. PCA was employed to extract the most relevant spectral features while maintaining computational efficiency. Alternative techniques such as Independent Component Analysis (ICA) or Variational Autoencoders (VAE) could be explored in future work to assess their impact on feature preservation and classification accuracy. Moreover, different parameter settings, including the number of convolutional layers, transformer heads, and kernel sizes, were systematically fine-tuned to optimize performance. Ablation studies in Table 8 revealed that removing key components such as GSF or transformers resulted in a significant decline in accuracy, underscoring their importance in achieving state-of-the-art results. These findings highlight the necessity of a carefully designed hybrid architecture that balances local and global feature extraction while maintaining computational efficiency.

Some limitations can be pinpointed from this study that will drive future research paths on this topic. One key limitation is the reliance on a fixed patch size for feature extraction, which may not optimally capture spatial relationships in all cases. Additionally, the transformer-based architecture, while effective, introduces higher computational complexity compared to traditional CNN approaches, potentially limiting its deployment on resource-constrained devices.

Future research should explore optimizing the model's computational efficiency, particularly by investigating lightweight transformer architectures or hybrid attention mechanisms that reduce memory overhead and expanding the model's applicability to real-time HSI processing scenarios, such as onboard satellite classification or drone-based agricultural monitoring, which remains an open challenge. Incorporating self-supervised learning techniques to reduce dependency on labelled data could also enhance the robustness and adaptability of the proposed method.

### CRediT authorship contribution statement

**Mohamed Fadhlallah Guerri:** Writing – original draft, Software, Investigation. **Cosimo Distante:** Writing – review & editing, Funding acquisition, Conceptualization. **Paolo Spagnolo:** Validation, Supervision, Investigation. **Abdelmalik Taleb-Ahmed:** Validation, Formal analysis.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Acknowledgements

The authors thank Mr. Arturo Argentieri from CNR-ISASI Italy for his technical contribution to the multi-GPU computing facilities. This research was funded in part by the Italian Ministry of University and Research with the grant "Future Artificial Intelligence Research"—FAIR CUP B53C22003630006 grant number PE0000013, and in part by the Ministry of Enterprises and Made in Italy with the grant ENDOR "ENabling technologies for Defence and mOnitoring of the foRests" - PON 2014–2020 FESR - CUP B82C21001750005.

### Data availability

used data is publicly available, but the architecture will be disclosed on Github (if paper is accepted) to allow other researchers to reproduce our experiments and use our method for further research.

### References

Ahmad, M., Khan, A.M., Mazzara, M., Distefano, S., Ali, M., Sarfraz, M.S., 2020. A fast and compact 3-D CNN for hyperspectral image classification. IEEE Geosci. Remote. Sens. Lett. 19, 1–5.

Ardouin, J.-P., Lévesque, J., Rea, T.A., 2007. A demonstration of hyperspectral image exploitation for military applications. In: 2007 10th International Conference on Information Fusion. IEEE, pp. 1–8.

Beirami, B.A., Mokhtarzade, M., 2020. Band grouping SuperPCA for feature extraction and extended morphological profile production from hyperspectral images. IEEE Geosci. Remote. Sens. Lett. 17 (11), 1953–1957.

Bougourzi, F., Distante, C., Dornaika, F., Taleb-Ahmed, A., 2023. Pdatt-unet: Pyramid dual-decoder attention Unet for Covid-19 infection segmentation from CT-scans. Med. Image Anal. 86, 102797.

Cariou, C., Chehdi, K., 2016. A new k-nearest neighbor density-based clustering method and its application to hyperspectral images. In: 2016 IEEE International Geoscience and Remote Sensing Symposium. IGARSS, IEEE, pp. 6161–6164.

Chen, Y., Jiang, H., Li, C., Jia, X., Ghamisi, P., 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 54 (10), 6232–6251.

Chen, J., Song, Y., Li, D., Lin, X., Zhou, S., Xu, W., 2023. Specular removal of industrial metal objects without changing lighting configuration. IEEE Trans. Ind. Inform.

Chen, Y.-N., Thaipisutikul, T., Han, C.-C., Liu, T.-J., Fan, K.-C., 2021. Feature line embedding based on support vector machine for hyperspectral image classification. Remote. Sens. 13 (1), 130.

Cheng, D., Chen, L., Lv, C., Guo, L., Kou, Q., 2022. Light-guided and cross-fusion U-Net for anti-illumination image super-resolution. IEEE Trans. Circuits Syst. Video Technol. 32 (12), 8436–8449.

Correa, A.F.D., Lizotte, M., Wolf, G., Moon, K.R., 2023. Manifold alignment with label information. In: 2023 International Conference on Sampling Theory and Applications. SampTA, IEEE, pp. 1–6.

Cui, B., Wen, J., Song, X., He, J., 2023. Madanet: A lightweight hyperspectral image classification network with multiscale feature aggregation and a dual attention mechanism. Remote. Sens. 15 (21), 5222.

Dalla Mura, M., Villa, A., Benediktsson, J.A., Chanussot, J., Bruzzone, L., 2010. Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis. IEEE Geosci. Remote. Sens. Lett. 8 (3), 542–546.

Doosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.

Esmaeili, M., Abbasi-Moghadam, D., Sharifi, A., Tariq, A., Li, Q., 2023. Hyperspectral image band selection based on CNN embedded GA (cnnega). IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 16, 1927–1950.

Fong, A., Shu, G., McDonogh, B., 2020. Farm to table: Applications for new hyperspectral imaging technologies in precision agriculture, food quality and safety. In: CLEO: Applications and Technology. Optica Publishing Group, pp. AW3K–2.

Gao, H., Yang, Y., Li, C., Gao, L., Zhang, B., 2020. Multiscale residual network with mixed depthwise convolution for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59 (4), 3396–3408.

Gevaert, C.M., Suomalainen, J., Tang, J., Kooistra, L., 2015. Generation of spectral-temporal response surfaces by combining multispectral satellite and hyperspectral UAV imagery for precision agriculture applications. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 8 (6), 3140–3146.

Gretton, A., 2013. Introduction to rkhs, and some simple kernel algorithms. Adv. Top. Mach. Learn. Lect. Conduct. Univ. Coll. Lond. 16 (5–3), 2.

Gu, A., Dao, T., 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752.
Gu, Y., Hu, Z., Zhao, Y., Liao, J., Zhang, W., 2024. MFGTN: A multi-modal fast gated transformer for identifying single trawl marine fishing vessel. Ocean Eng. 303, 117711.

Guerri, M.F., Distante, C., Spagnolo, P., Bougourzi, F., Taleb-Ahmed, A., 2024. Deep learning techniques for hyperspectral image analysis in agriculture: A review. ISPRS Open J. Photogramm. Remote. Sens. 100062.

Haut, J.M., Paoletti, M.E., Plaza, J., Plaza, A., Li, J., 2019. Visual attention-driven hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 57 (10), 8065-8080.

He, M., Li, B., Chen, H., 2017. Multi-scale 3D deep convolutional neural network for hyperspectral image classification. In: 2017 IEEE International Conference on Image Processing. ICIP, IEEE, pp. 3904-3908.

He, Y., Tu, B., Liu, B., Li, J., Plaza, A., 2024. 3DSS-Mamba: 3D-spectral-spatial mamba for hyperspectral image classification. arXiv preprint arXiv:2405.12487.

Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., Chanussot, J., 2021. SpectralFormer: Rethinking hyperspectral image classification with transformers. IEEE Trans. Geosci. Remote Sens. 60, 1-15.

Hu, W., Huang, Y., Wei, L., Zhang, F., Li, H., 2015. Deep convolutional neural networks for hyperspectral image classification. J. Sensors 15, 1-12.

Imani, M., Ghassemian, H., 2014. Principal component discriminant analysis for feature extraction and classification of hyperspectral images. In: 2014 Iranian Conference on Intelligent Systems. ICIS, IEEE, pp. 1-5.

Iyer, P., Sriram, A., Lal, S., 2021. Deep learning ensemble method for classification of satellite hyperspectral images. Remote. Sens. Appl.: Soc. Environ. 23, 100580.

Jalayer, S., Sharifi, A., Abbasi-Moghadam, D., Tariq, A., Qin, S., 2023. Assessment of spatiotemporal characteristic of droughts using in situ and remote sensing-based drought indices. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 16, 1483-1502.

Khan, R., Arshad, T., Ma, X., Zhu, H., Wang, C., Khan, J., Khan, Z.U., Khan, S.U., 2024. GroupFormer for hyperspectral image classification through group attention. Sci. Rep. 14 (1), 23879.

Li, W., Prasad, S., Fowler, J.E., Bruce, L.M., 2011. Locality-preserving discriminant analysis in kernel-induced feature spaces for hyperspectral image classification. IEEE Geosci. Remote. Sens. Lett. 8 (5), 894-898.

Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., Benediktsson, J.A., 2019. Deep learning for hyperspectral image classification: An overview. IEEE Trans. Geosci. Remote Sens. 57 (9), 6690-6709.

Li, Y., Song, L., Hu, Y., Lee, H., Wu, D., Rehm, P., Lu, N., 2023. Load profile inpainting for missing load data restoration and baseline estimation. IEEE Trans. Smart Grid.

Liu, K., Jiao, S., Nie, G., Ma, H., Gao, B., Sun, C., Xin, D., Saha, T.K., Wu, G., 2024. On image transformation for partial discharge source identification in vehicle cable terminals of high-speed trains. High Volt. 9 (5), 1090-1100.

Mohammadi, M., Sharifi, A., 2021. Evaluation of convolutional neural networks for urban mapping using satellite images. J. Indian Soc. Remote. Sens. 49 (9), 2125-2131.

Nejad, S.M.M., Abbasi-Moghadam, D., Sharifi, A., Farmonov, N., Amankulova, K., László, M., 2022. Multispectral crop yield prediction using 3D-convolutional neural networks and attention convolutional LSTM approaches. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 16, 254-266.

Noor, S.S.M., Michael, K., Marshall, S., Ren, J., Tschannerl, J., Kao, F.-J., 2016. The properties of the cornea based on hyperspectral imaging: Optical biomedical engineering perspective. In: 2016 International Conference on Systems, Signals and Image Processing. IWSSIP, IEEE, pp. 1-4.

Phaneendra Kumar, B., Vaddi, R., Manoharan, P., Agilandeeswari, L., Sangeetha, V., 2024. A new band selection framework for hyperspectral remote sensing image classification. Sci. Rep. 14 (1), 31836.

Ren, W., Jin, N., OuYang, L., 2024. Phase space graph convolutional network for chaotic time series learning. IEEE Trans. Ind. Inform.

Roy, S.K., Chatterjee, S., Bhattacharyya, S., Chaudhuri, B.B., Platoš, J., 2020a. Lightweight spectral-spatial squeeze-and-excitation residual bag-of-features learning for hyperspectral classification. IEEE Trans. Geosci. Remote Sens. 58 (8), 5277-5290.

Roy, S.K., Dubey, S.R., Chatterjee, S., Baran Chaudhuri, B., 2020b. FuSENet: fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification. IET Image Process. 14 (8), 1653-1661.

Roy, S.K., Krishna, G., Dubey, S.R., Chaudhuri, B.B., 2019. HybridSN: Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geosci. Remote. Sens. Lett. 17 (2), 277-281.

Roy, S.K., Manna, S., Song, T., Bruzzone, L., 2020c. Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59 (9), 7831-7843.

Sharifi, A., Felegari, S., 2022. Nitrogen dioxide (NO2) pollution monitoring with sentinel-5P satellite imagery over during the coronavirus pandemic (case study: Tehran). Remote. Sens. Lett. 13 (10), 1029-1039.

Sharifi, A., Hosseingholizadeh, M., 2020. Application of sentinel-1 data to estimate height and biomass of rice crop in Astaneh-ye Ashrafiyeh, Iran. J. Indian Soc. Remote. Sens. 48, 11-19.

Sheng, J., Zhou, J., Wang, J., Ye, P., Fan, J., 2024. DualMamba: A lightweight spectral-spatial mamba-convolution network for hyperspectral image classification. arXiv preprint arXiv:2406.07050.

Sudhakaran, S., Escalera, S., Lanz, O., 2023. Gate-shift-fuse for video action recognition. IEEE Trans. Pattern Anal. Mach. Intell.

Sun, L., Wang, Q., Chen, Y., Zheng, Y., Wu, Z., Fu, L., Jeon, B., 2023. Crnet: Channel-enhanced remodeling-based network for salient object detection in optical remote sensing images. IEEE Trans. Geosci. Remote Sens.

Sun, H., Zheng, X., Lu, X., Wu, S., 2019. Spectral-spatial attention network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 58 (5), 3232-3245.

Tu, B., Ren, Q., Li, J., Cao, Z., Chen, Y., Plaza, A., 2024. NCGLF2: Network combining global and local features for fusion of multisource remote sensing data. Inf. Fusion 104, 102192.

Virghileanu, M., Săvulescu, I., Mihai, B.-A., Nistor, C., Dobre, R., 2020. Nitrogen Dioxide (NO2) pollution monitoring with sentinel-5P satellite imagery over Europe during the coronavirus pandemic outbreak. Remote. Sens. 12 (21), 3575.

Wang, J., Song, X., Sun, L., Huang, W., Wang, J., 2020. A novel cubic convolutional neural network for hyperspectral image classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 13, 4133-4148.

Wang, J., Zhang, L., Tong, Q., Sun, X., 2012. The spectral crust project—Research on new mineral exploration technology. In: 2012 4th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing. WHISPERS, IEEE, pp. 1-4.

Xia, J., Bom Brun, L., Adali, T., Berthoumieu, Y., Germain, C., 2016. Spectral-spatial classification of hyperspectral images using ICA and edge-preserving filter via an ensemble strategy. IEEE Trans. Geosci. Remote Sens. 54 (8), 4971-4982.

Xia, J., Ghamisi, P., Yokoya, N., Iwasaki, A., 2017. Random forest ensembles and extended multiextinction profiles for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 56 (1), 202-216.

Xie, D., Huang, H., Feng, L., Sharma, R.P., Chen, Q., Liu, Q., Fu, L., 2023. Aboveground biomass prediction of arid shrub-dominated community based on airborne Lidar through parametric and nonparametric methods. Remote. Sens. 15 (13), 3344.

Xu, H., Li, Q., Chen, J., 2022. Highlight removal from a single grayscale image using attentive GAN. Appl. Artif. Intell. 36 (1), 1988441.

Xu, S., Xia, X., Li, H., Zhang, Y., Sheng, R., Gao, H., Zhang, B., 2024. Strengthened residual graph and multiscale gated guided convolutional fusion network for hyperspectral change detection. IEEE Trans. Geosci. Remote Sens.

Yu, S., Guan, D., Gu, Z., Guo, J., Liu, Z., Liu, Y., 2024. Radar target complex high-resolution range profile modulation by external time coding metasurface. IEEE Trans. Microw. Theory Tech.

Zhang, L., Wang, Y., Yang, L., Chen, J., Liu, Z., Wang, J., Bian, L., Yang, C., 2023. A multi-range spectral-spatial transformer for hyperspectral image classification. Infrared Phys. Technol. 104983.

Zhao, W., Du, S., 2016. Spectral-spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach. IEEE Trans. Geosci. Remote Sens. 54 (8), 4544-4554.

Zhong, Z., Li, J., Luo, Z., Chapman, M., 2017. Spectral-spatial residual network for hyperspectral image classification: A 3-D deep learning framework. IEEE Trans. Geosci. Remote Sens. 56 (2), 847-858.

Zhou, Z., Li, Z., Zhou, W., Chi, N., Zhang, J., Dai, Q., 2024. Resource-saving and high-robustness image sensing based on binary optical computing. Laser Photonics Rev. 2400936.