# LoLA-SpecViT: Local attention SwiGLU vision transformer with LoRA for hyperspectral imaging

Fadi Abdeladhim Zidi, Djamel Eddine Boukhari, Abdellah Zakaria Sellam, Abdelkrim Ouafi, Cosimo Distante, Salah Eddine Bekhouche, Abdelmalik Taleb-Ahmed

<figure><img src="image_1.png" /></figure>

## To cite this version:

Fadi Abdeladhim Zidi, Djamel Eddine Boukhari, Abdellah Zakaria Sellam, Abdelkrim Ouafi, Cosimo Distante, et al.. LoLA-SpecViT: Local attention SwiGLU vision transformer with LoRA for hyperspectral imaging. International Journal of Applied Earth Observation and Geoinformation, 2025, 144, pp.104924. 10.1016/j.jag.2025.104924. hal-05393562

HAL Id: hal-05393562

https://hal.science/hal-05393562v1

Submitted on 2 Dec 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# LoLA-SpecViT: Local attention SwiGLU vision transformer with LoRA for hyperspectral imaging

Fadi Abdeladhim Zidi<sup>a</sup>, Djamel Eddine Boukhari<sup>b</sup>, Abdellah Zakaria Sellam<sup>c</sup>, Abdelkrim Ouafi<sup>a</sup>, Cosimo Distante<sup>d|,*</sup>, Salah Eddine Bekhouche<sup>e|</sup>, Abdelmalik Taleb-Ahmed<sup>f</sup>

## ARTICLE INFO

### Keywords:

Hyperspectral image classification

Vision transformer

Parameter-efficient fine-tuning

Low-rank adaptation

Spectral attention

Local self-attention

## ABSTRACT

Hyperspectral image classification remains a challenging task due to the high dimensionality of spectral data, significant inter-band redundancy, and the limited availability of annotated samples. While recent transformer-based models have improved the global modeling of spectral-spatial dependencies, their scalability and adaptability under label-scarce conditions remain limited. In this work, we propose LoLA-SpecViT(Low-Rank Adaptation Local Attention Spectral Vision Transformer), a lightweight spectral vision transformer that addresses these limitations through a parameter-efficient architecture tailored to the unique characteristics of hyperspectral imagery. Our model combines a 3D convolutional spectral front-end with local window-based self-attention, enhancing both spectral feature extraction and spatial consistency while reducing computational complexity. To further improve adaptability, we integrate enhanced Low-Rank Adaptation (LoRA) into attention and projection layers, enabling fine-tuning with over 70% fewer trainable parameters. A novel cyclical learning rate scheduler modulates LoRA adaptation strength during training, improving convergence and generalization. Extensive experiments on four benchmark datasets, WHU-Hi LongKou, WHU-Hi HongHu, Salinas, and QUH-Qingyun, demonstrate that LoLA-SpecViT consistently outperforms state-of-the-art baselines, achieving up to 99.91% accuracy with substantially fewer parameters and enhanced robustness under low-label regimes. The proposed framework provides a scalable and generalizable solution for real-world HSI applications in agriculture, environmental monitoring, and remote sensing analytics. Our code is available in the following GitHub Repository.

## 1. Introduction

Hyperspectral imaging (HSI) has fundamentally changed observational remote sensing by being able to capture hundreds of contiguous, narrow spectral bands at each ground location, producing comprehensive three-dimensional data cubes that significantly outperform the limited information provided by RGB or multispectral sensors (Zhang et al., 2021). This improved spectral resolution has revolutionized monitoring capabilities in agriculture by enabling unprecedented precision in crop analysis (Zhou et al., 2023b). In agriculture, these spectral signatures reveal subtle biochemical and structural characteristics, including early stress indicators, nutrient deficiencies, and moisture fluctuations that cannot be detected by conventional imaging systems. For example, hyperspectral data enables early detection of

drought stress or pathogens before visible symptoms appear, facilitates precise differentiation between crops and weeds for targeted herbicide application, and supports accurate assessment of chlorophyll content and plant maturity for optimal harvest timing (García-Vera et al., 2024; Fadi and Abdelkrim, 2024). These capabilities make it clear why precision agriculture has become an important application area for HSI technology (Ram et al., 2024).

However, these significant opportunities are offset by considerable technical challenges that hinder the practical implementation of HSI. The curse of dimensionality often overwhelms classical classification algorithms, while spectral variability caused by illumination and weather conditions can severely compromise the accuracy of material identification. Moreover, the fundamental trade-off between spectral richness

<footnote><sup>a</sup> VSC Laboratory, Department of Electrical Engineering, University of Mohamed Khider, Biskra, 07000, Algeria</footnote>

<footnote><sup>b</sup> Scientific and Technical Research Centre for Arid Areas (CRSTRA), Biskra, 07000, Algeria</footnote>

<footnote><sup>c</sup> Department of Innovation Engineering, University of Salento & Institute of Applied Sciences and Intelligent Systems - CNR, Via per Monteroni, Lecce, 73100, Italy</footnote>

<footnote><sup>d</sup> Institute of Applied Sciences and Intelligent Systems - CNR, Via per Monteroni, Lecce, 73100, Italy</footnote>

<footnote><sup>e</sup> UPV/EHU, University of the Basque Country, Sebastian, 20018, Spain</footnote>

<footnote><sup>f</sup> Université Polytechnique Hauts-de-France, Université de Lille, CNRS, Valenciennes, 59313, Hauts-de-France, France</footnote>

<footnote>* Corresponding author.
E-mail address: cosimo.distante@cnr.it (C. Distante).</footnote>
and spatial resolution often limits effective analysis (Cheng et al., 2024). In addition, agricultural applications frequently face severe label scarcity, since the acquisition of ground-truth information is expensive and time-consuming, making efficient learning from limited samples a critical requirement. These simultaneous opportunities and obstacles call for the development of innovative deep learning frameworks capable of distilling high-dimensional spectra into robust, discriminative features while preserving critical spatial-spectral relationships under constrained supervision (Ma and Zhang, 2023).

The development of deep learning approaches for HSI classification reflects an ongoing tension between model expressiveness, computational efficiency, and sampling efficiency (Li et al., 2022). Conventional convolutional neural networks excel at local feature extraction but struggle with long-range spectral dependencies due to limited receptive fields (Chen et al., 2020). Conversely, transformer architectures can effectively model global relationships but suffer from quadratic computational complexity that becomes prohibitive for high-dimensional HSI data (Huang et al., 2024). Hybrid approaches attempt to compensate for these trade-offs, but often either underutilize interband correlations or retain computationally intensive parameter footprints (Sellam et al., 2024). In addition, existing methods typically require extensive labeled data, which is at odds with the low-labeled reality of agricultural HSI applications (Zidi et al., 2025). This multifaceted challenge, which involves simultaneously achieving comprehensive spectral-spatial modeling, efficient computational performance, and efficient sampling, remains a critical obstacle to the practical use of HSI in resource-constrained agricultural environments (Fu et al., 2024).

To systematically address these multi-layered bottlenecks while leveraging the strengths of transformer architectures, we introduce LoLA-SpecViT, a multi-scale vision transformer designed for efficient and sample-efficient HSI classification. Our approach addresses computational limitations through a streamlined architecture that combines local windowed attention mechanisms with parameter-efficient fine-tuning strategies, while tackling sample efficiency via enhanced LoRA adaptations. Specifically, we integrate a lightweight 3D convolutional spectral front-end with local windowed self-attention blocks using SwiGLU activations, mitigating standard transformer computational complexity while preserving modeling capacity. The architecture employs a modified Global Context Vision Transformer (GCViT) with local windowed attention, enabling efficient local-to-global feature aggregation through progressive spatial downsampling across three stages. To address spectral processing challenges, we utilize cascaded 3D convolutions with batch normalization and Swish activations, followed by spectral dimension collapse and patch embedding through convolutional downsampling. We enhance both efficiency and generalization through BandDropout for spectral regularization and an adaptive squeeze-and-excitation module that models inter-band correlations. Crucially, we adopt parameter-efficient fine-tuning (PEFT) via enhanced LoRA, which incorporates gated and residual low-rank adapters, substantially reducing trainable parameters while maintaining full modeling capacity through comprehensive adaptation strategies.

LoLA-SpecViT addresses the critical challenge of label-scarce hyperspectral image classification through three synergistic strategies designed to maximize learning efficiency from limited training data. Our enhanced LoRA framework reduces trainable parameters by 71.10% while preserving model capacity through gated residual adaptation mechanisms that incorporate trainable gating matrices and residual connections. In contrast to standard LoRA, our approach adaptively emphasizes discriminative spectral features, a crucial capability since certain hyperspectral bands contain substantially more class-relevant information than others. We utilize pre-trained GCViT backbones by applying LoRA adapters to attention and MLP layers, and then merging the learned weights into our custom architecture. This enables effective knowledge transfer from large-scale visual models while preserving architectural flexibility for hyperspectral-specific processing.

Domain-specific regularization through BandDropout prevents overfitting to dataset-specific spectral signatures by randomly masking spectral channels during training, while adaptive squeeze-and-excitation attention handles both 2D and 3D tensor representations to provide targeted augmentation tailored to hyperspectral characteristics. Extensive validation under extreme supervision conditions (2%, 5%, and 10% training labels) demonstrates that this unified approach significantly improves generalization compared to baseline methods while maintaining computational efficiency, successfully addressing the fundamental challenge of learning discriminative representations from severely limited labeled data in agricultural hyperspectral applications.

The main contributions of this work are summarized as follows:

* We propose LoLA-SpecViT, a novel multiscale vision transformer architecture that integrates a modified GCViT backbone with local windowed attention and SwiGLU activations, and enhances it with adaptive attention mechanisms for effective spectral-spatial dependency modeling across different tensor dimensions.

* A comprehensive spectral processing front-end combines 3D convolutional layers with BandDropout regularization and an adaptive squeeze excitation module, which improves band-wise feature learning while mitigating overfitting through spectral-specific techniques that dynamically adapt to input tensor dimensions.

* Enhanced parameter-efficient fine-tuning (PEFT) techniques extend LoRA with gating mechanisms, residual connections, and adaptive scaling, achieving parameter reduction of over 70% while maintaining performance through comprehensive adaptation strategies that preserve gradient flow and model capacity.

* Novel pretrained model integration that leverages pretrained GCViT checkpoints, applies LoRA adaptations for efficient fine-tuning, and merges trained low-rank weights into custom architectures, enabling effective transfer learning while preserving architectural flexibility.

* Extensive experiments with multiple hyperspectral benchmark datasets show state-of-the-art classification accuracy with significantly lower resource requirements. This is confirmed by ablation studies showing the complementary contributions of improved LoRA mechanisms, adaptive attention modules, and spectral processing components.

The remainder of this paper is structured as follows: Section 2 surveys the relevant literature; Section 3 details the LoLA-SpecViT architecture; Section 4 describes the experimental setup; Section 5 presents and analyzes the findings; and Section 6 concludes the study with suggestions for future research.

## 2. Related work

The field of HSI classification has evolved from conventional machine learning to deep learning architectures that address the challenges of high-dimensional spectral-spatial analysis. This evolution is driven by the need to find a balance between model expressiveness and computational efficiency, especially for real-time interpretation of large-scale HSI data (Yue et al., 2022). We structure our review around three major research directions that have shaped this field: the evolution from local to global feature modeling, the emergence of efficiency-oriented solutions, and recent advances in parameter-efficient adaptation of large pre-trained models.

Early deep learning approaches focused on extracting local features to capture spectral-spatial patterns. The transition from hand-crafted to learned features started with (Hu et al., 2015)'s pixel-wise multilayer perceptrons (MLPs), which showed basic spectral discrimination but neglected spatial context. (Li et al. (2017) subsequently introduced 2D-CNNs for localized spatial filtering, which were later extended
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Overall architecture of the proposed LoLA-SpecViT for hyperspectral image classification, integrating (1) spectral processing front-end with multi-scale 3D convolutions and adaptive attention, (2) patch embedding module with positional encoding, (3) multi-scale transformer backbone enhanced by LoRA and SwiGLU with downsampling, and (4) classification head with parameter-efficient LoRA projection.</figcaption></figure>

to 3D-CNNs for unified spectral-spatial fusion, albeit with considerable computational effort. HybridSN (Roy et al., 2020) balances the trade-off between model expressiveness and computational efficiency with cascaded 2D convolutions, producing joint representations with controlled complexity. Building on these foundations, the researchers developed more sophisticated local modeling approaches. Yang et al. (2021) proposed A²MFE for multi-scale receptive fields, while (Sun et al., 2022) developed SSFTT, which tokenises low-level spectral-spatial features via 3D-2D CNN front-ends for downstream processing. Attention-driven designs such as MADANet (Cui et al., 2023), HiT (Yang et al., 2022), and LANet (Ding et al., 2021) refined local aggregation mechanisms, improving performance on standard benchmarks. Recent adaptive MLP variants (AS2MLP, LDS2MLP) further advance this paradigm by fusing convolutional priors with learnable spatial shifts (Ma et al., 2024; Zhang et al., 2024a).

However, the inherent limitations of local receptive fields led to a shift towards global context modeling. Vision Transformers (ViTs) reformulated this challenge by treating images as token sequences and using self-attention to capture long-range dependencies (Dosovitskiy et al., 2021). In HSI, masked autoencoding and cross-sensor self-supervision (HSIMAE, Cross SSL, SpectralMAE) enabled robust spectral embeddings under label scarcity (Ibanez et al., 2022; Bai et al., 2024). In particular, SpectralMAE is a self-supervised masked autoencoder specifically designed for hyperspectral pretraining and should not be confused with efficiency-oriented solutions. Methods such as MASSFormer and DiCT addressed the computational burden through memory-enhanced attention and token pruning (Sun et al., 2024; Zhou et al., 2023a). Recognizing that purely global modeling sacrifices the

advantageous local inductive biases, hybrid architectures emerged as a compromise solution. Approaches such as Hybrid-ViT, 3D-CmT, and DBCTNet retain CNN front-ends for local feature extraction while leveraging global attention for context modeling (Arshad and Ullah, 2024; Zhang et al., 2023; Xu et al., 2024). Hierarchical transformer variants organize tokens across multiple scales to capture multi-resolution structures (Ahmad et al., 2024), while state-space extensions (SSMamba, MorpMamba) provide alternative ways to long-sequence modeling with improved scalability (Ahmad et al., 2025).

The computational requirements of both deep CNNs and standard transformers led to targeted efficiency research. As architectures deepened, resource requirements increased, and motivated compression techniques such as pruning, distillation, and collaborative optimization (Sellam et al., 2025; Yu et al., 2024). Architectural efficiency strategies emerged to address scalability challenges at the design level. Group-based approaches, such as GroupFormer, partition spectral bands for intra-group attention, reducing computational complexity from quadratic to linear (Khan et al., 2024). Learned token sampling methods such as *PatchOut* replace fixed spatial patching with adaptive mechanisms. Dynamic pruning approaches, such as CBAM-based adaptive pruning, enable runtime elimination of redundant channels and spatial locations (Fu et al., 2025). Efficiency-oriented MLP variants and convolution-aware designs further extend the toolbox for low-cost HSI modeling (Ma et al., 2024; Zhang et al., 2024a). While these efficiency-focused methods achieve substantial computational reductions, they may reduce cross-band interactions or require careful tuning of hyper-parameters, highlighting the need for more principled approaches to efficiency modeling.
A complementary efficiency paradigm emerged from the realization that large pre-trained visual models contain extensive semantic priorities that could be useful for HSI classification. Rather than training task-specific architectures from scratch, Parameter-Efficient Fine-Tuning (PEFT) adapts frozen pre-trained models by updating only a small fraction of the parameters $|\Delta\theta| \ll |\theta|$, drastically reducing computational and memory overhead (Xin et al., 2024). PEFT comprises several methodological approaches: Adapter modules introduce lightweight trainable components (Houlsby et al., 2019), visual prompt tuning optimizes prompts (Jia et al., 2022), and partial parameter strategies such as BitFit and LoRA restrict updates to selected parameter subsets (Ben Zaken et al., 2022; Hu et al., 2022). Unified frameworks (NOAH, KronA, Lokr) and optimizations in low-rank factor learning further increase adaptation flexibility and performance (Zhang et al., 2024b; Edalati et al., 2022; Hayou et al., 2024). This paradigm offers a promising way to leverage large-scale pre-trained representations while maintaining computational efficiency, although effective application to HSI requires careful consideration of domain-specific features and tokenisation strategies (Yeh et al., 2024).

Beyond adapting generic visual backbones, recent work has introduced hyperspectral foundation models tailored to the spatial-spectral characteristics of hyperspectral data. HyperSIGMA (Wang et al., 2025), for instance, employs sparse-sampling attention and a spectral enhancement module for effective feature fusion, while Ligan et al. (2025) proposed parameter-efficient fine-tuning of the multispectral foundation model SpectralGPT with KronA+ as a lightweight adaptation strategy. Although these approaches highlight the potential of foundation models, they often require extensive pretraining or still underperform on standard benchmarks, whereas our LoLA-SpecViT achieves superior accuracy with substantially fewer trainable parameters by integrating lightweight local attention, spectral enhancement, and LoRA-based adaptation.

The evolution from local to global modeling has shown that effective HSI classification requires both detailed spectral-spatial feature extraction and long-range dependency modeling. Efficiency-oriented solutions have shown that computational constraints can be addressed through architectural innovations, albeit often at the cost of modeling completeness. PEFT methods offer a way to efficiently utilize pre-trained representations, but their application to HSI domain-specific features has not yet been sufficiently explored. Current approaches typically excel in one dimension while compromising others: convolutional methods provide efficient local modeling but limited global context; transformers enable comprehensive global modeling but with quadratic computational complexity; efficiency-oriented solutions reduce computational effort but may sacrifice critical cross-spectral dependencies. To address these limitations, we propose a unified framework that integrates lightweight local spectral-spatial extraction, multi-scale global context modeling, and parameter-efficient adaptation to achieve both high accuracy and computational efficiency for HSI classification tasks.

### 3. Proposed method

This section presents the proposed LoLA-SpecViT (Low-Rank Adaptation Local Attention Spectral Vision Transformer), a novel parameter-efficient architecture designed to address the computational and adaptation challenges inherent in hyperspectral image classification. The architecture integrates multi-scale spectral-spatial feature learning with parameter-efficient fine-tuning through enhanced Low-Rank Adaptation (LoRA), providing an optimal balance between classification accuracy and computational efficiency.

The LoLA-SpecViT architecture, illustrated in Fig. 1, follows a systematic four-stage pipeline that progressively transforms raw hyperspectral data into discriminative class representations: (1) PCA-based

dimensionality reduction for spectral preprocessing, (2) spectral preprocessing front-end with adaptive attention mechanisms for multi-scale feature extraction, (3) multi-scale transformer backbone with enhanced LoRA integration for hierarchical representation learning, and (4) classification head with parameter-efficient adaptation for final prediction. This design addresses key challenges in hyperspectral classification, including high dimensionality, limited training samples, and computational constraints, while maintaining state-of-the-art performance.

#### 3.1. Input preprocessing and PCA dimensionality reduction

Given a raw hyperspectral image $X_{\text{raw}} \in \mathbb{R}^{H \times W \times C_{\text{raw}}}$ where $H \times W$ represents spatial dimensions and $C_{\text{raw}}$ is the original number of spectral bands, we first apply Principal Component Analysis (PCA) for dimensionality reduction and spectral decorrelation. This preprocessing stage is crucial for achieving computational efficiency and reducing noise in subsequent processing stages.

Through systematic evaluation of several candidate values (15, 20, and 30 components), we determined that $k = 15$ components provide the most effective trade-off between preserving spectral variability and ensuring stable performance. This configuration corresponds to approximately 5.56-7.35% of the original spectral bands while empirically capturing 99.98% of the total spectral variance for hyperspectral image classification tasks.

The preprocessing pipeline begins with matrix reshaping to enable PCA processing:

$$X_{\text{flat}} = \text{Reshape}(X_{\text{raw}}) \in \mathbb{R}^{(H \times W) \times C_{\text{raw}}} \quad (1)$$

The PCA transformation with whitening is then applied to enhance performance and ensure a consistent input size:

$$X_{\text{pca}} = \text{PCA}_{\text{whiten}}(X_{\text{flat}}, k = 15) \quad (2)$$

where PCA\_whiten denotes PCA with whitening transformation, and $k = 15$ represents the fixed number of principal components. The whitening transformation ensures that the components have unit variance, which improves the convergence properties of the subsequent neural network layers.

The explained variance ratio for the selected 15 components is computed to validate information retention:

$$\text{Explained Variance} = \sum_{i=1}^{15} \frac{\lambda_i}{C_{\text{raw}}} \quad (3)$$

where $\lambda_i$ represents the eigenvalues of the covariance matrix. The result is reshaped back to a spatial format for network processing:

$$X_{\text{in}} = \text{Reshape}(X_{\text{pca}}) \in \mathbb{R}^{H \times W \times 15} \quad (4)$$

#### 3.1.1. patch extraction and tensor preparation

From the PCA-reduced cube, spatial patches of size $p \times p$ are extracted around each labeled pixel, with zero-padding applied at the borders:

$$X_{\text{padded}} = \text{ZeroPad}(X_{\text{in}}, \text{margin} = \frac{p-1}{2}). \quad (5)$$

Each patch centered at location $(r, c)$ is then defined as

$$\mathbf{P}_{r,c} = \mathbf{X}_{\text{padded}}[r - \text{margin} : r + \text{margin} + 1, c - \text{margin} : c + \text{margin} + 1, :], \quad (6)$$

and the resulting patches are stacked into batches:

$$F_{\text{in}} = \text{BatchPatches}(\{\mathbf{P}_{r,c}\}) \in \mathbb{R}^{B \times 15 \times p \times p}, \quad (7)$$

where B is the batch size, and p is the spatial patch size.
## 3.2. Spectral processing front-end

The spectral processing stage serves as the foundation for the subsequent transformer backbone by extracting multi-scale spectral-spatial features through a series of 3D convolutions with regularization and adaptive attention mechanisms. This stage transforms raw spectral data into structured feature representations that capture both local spectral patterns and spatial relationships.

The input tensor is first reshaped to enable 3D convolution operations:

$$ \tilde{F}_{in} = \text{Unsqueeze}(F_{in}) \in \mathbb{R}^{B \times 1 \times k \times p \times p} \quad (8) $$

The spectral feature extraction follows a progressive convolution pipeline with batch normalization and Swish activation functions for enhanced gradient flow:

$$ F_1 = \text{Swish}(\text{BatchNorm3d}(\text{Conv3d}_{(7,3,3)}(\tilde{F}_{in}))) \quad (9) $$

$$ F_2 = \text{Swish}(\text{BatchNorm3d}(\text{Conv3d}_{(5,3,3)}(F_1))) \quad (10) $$

$$ F_{\text{spec}} = \text{Swish}(\text{BatchNorm3d}(\text{Conv3d}_{(3,3,3)}(F_2))) \quad (11) $$

where $Conv3d_{(k_d, k_h, k_w)}$ denotes 3D convolution with kernel size $(k_d, k_h, k_w)$ along the depth (spectral), height, and width dimensions, respectively. The Swish activation function $Swish(x) = x \cdot \sigma(x) = x \cdot \frac{1}{1+e^{-x}}$ promotes better gradient flow compared to traditional activation functions in transformer architectures.

### 3.2.1. Band dropout regularization

To enhance spectral robustness and prevent overfitting to specific spectral patterns, we introduce a BandDropout regularization mechanism that adaptively masks spectral channels during training. This mechanism improves model generalization by forcing the network to learn from diverse spectral patterns:

$$ M \sim \text{Bernoulli}(1 - p_{\text{drop}}) \quad (12) $$

$$ F_{\text{bdrop}} = \frac{F_{\text{spec}} \odot M}{1 - p_{\text{drop}}} \quad (13) $$

where $M \in \{0, 1\}^{C_{\text{spec}}}$ is a binary mask sampled from a Bernoulli distribution with dropout probability $p_{\text{drop}}$, $\odot$ denotes element-wise multiplication, and the normalization factor $(1 - p_{\text{drop}})^{-1}$ maintains the expected magnitude of activations during training.

### 3.2.2. Adaptive squeeze-and-excitation attention

Building upon the spectral feature extraction, we introduce a unified Adaptive Squeeze-and-Excitation mechanism that adaptively emphasizes the most discriminative spectral features while maintaining computational efficiency. This mechanism bridges the gap between spectral processing and spatial feature learning by providing channel-wise attention.

The global adaptive pooling operation computes channel-wise statistics:

$$ z = \text{AdaptiveGAP}(F_{\text{bdrop}}) \in \mathbb{R}^{B \times C_{\text{spec}}} \quad (14) $$

where AdaptiveGAP denotes adaptive global average pooling that automatically handles both 2D and 3D tensor inputs:

$$ z_c = \frac{1}{H_{\text{spec}} \cdot W_{\text{spec}} \cdot D_{\text{spec}}} \sum_{d=1}^{D_{\text{spec}}} \sum_{h=1}^{H_{\text{spec}}} \sum_{u=1}^{W_{\text{spec}}} F_{\text{bdrop}}(b, c, d, h, w) \quad (15) $$

The attention weights are computed through a two-layer MLP with bottleneck architecture to reduce computational overhead:

$$ A_{\text{adaptive}} = \sigma(W_2 \cdot \text{ReLU}(W_1 \cdot z)) \quad (16) $$

where $W_1 \in \mathbb{R}^{r \times C_{\text{spec}}}$ and $W_2 \in \mathbb{R}^{C_{\text{spec}} \times r}$ are learned projection matrices, $r < C_{\text{spec}}$ is the bottleneck dimension (typically $r = C_{\text{spec}}/4$), and $\sigma(\cdot)$ is the sigmoid activation function.

The final adaptively attended features are obtained by channel-wise recalibration:

$$ F_{\text{attended}} = F_{\text{bdrop}} \odot \text{Reshape}(A_{\text{adaptive}}) \quad (17) $$

### 3.2.3. Spatial dimension reduction

Following the adaptive attention mechanism, we prepare features for the transformer backbone by reducing the spectral dimension and applying spatial patch embedding. This transition transforms the 3D spectral-spatial features into 2D spatial representations suitable for the transformer architecture:

$$ F_{\text{spatial}} = \text{Mean}(F_{\text{attended}}, \text{dim} = 2) \in \mathbb{R}^{B \times C_{\text{spec}} \times H_{\text{spec}} \times W_{\text{spec}}} \quad (18) $$

where the mean operation along the spectral dimension aggregates multi-scale spectral information. The spatial embedding is then computed using:

$$ F_{\text{embed}} = \text{Swish}(\text{BatchNorm2d}(\text{Conv2d}_{4 \times 4, \text{stride} = 2}(F_{\text{spatial}'}))) \quad (19) $$

## 3.3. Multi-scale transformer architecture with parameter-efficient adaptation

The embedded features from the spectral processing stage are processed through a three-level multi-scale transformer pipeline that incorporates enhanced parameter-efficient adaptation mechanisms. This backbone progressively extracts increasingly abstract spatial representations while maintaining parameter efficiency through enhanced LoRA mechanisms.

Each level $i$ consists of a backbone module $B_i$ followed by a downsampling operation $D_i$ that enables hierarchical feature learning:

$$ X_i = D_i(B_i(X_{i-1})) \quad (20) $$

where $X_0 = F_{\text{embed}}$ is the input to the first level. This multi-level structure enables progressive feature abstraction while maintaining computational efficiency.

Each downsampling operation halves the spatial resolution while doubling the channel dimension to balance spatial and semantic information:

$$ X_0 \in \mathbb{R}^{B \times H_0 \times W_0 \times C_0}, \quad X_1 \in \mathbb{R}^{B \times H_0 / 2 \times W_0 / 2 \times C_0}, \quad (21) $$

$$ X_2 \in \mathbb{R}^{B \times H_0 / 4 \times W_0 / 4 \times C_0} \quad (22) $$

For the default patch size $p = 15$, the spatial path reduces as $15 \times 15 \rightarrow 7 \times 7 \rightarrow 3 \times 3$, providing multi-scale spatial context.

### 3.3.1. LoRA-enhanced window attention

Each backbone module $B_i$ integrates Low-Rank Adaptation (LoRA) into the self-attention mechanism for parameter-efficient fine-tuning. Input features are partitioned into non-overlapping windows of size $M \times M$:

$$ W_j = \text{WindowPartition}(X_{i-1}, M) \in \mathbb{R}^{B \times N_w \times M^2 \times C_i} \quad (23) $$

where $N_w = \frac{H_1 \cdot W_1}{M^2}$ is the number of windows.

For each window $x_i \in \mathbb{R}^{M^2 \times C_i}$, the gating signals are computed as:

$$ G_* = \sigma(x_i W_*^{\text{gate}}) \quad (24) $$

where $\sigma(\cdot)$ is the sigmoid activation function and $W_*^{\text{gate}}$ are the trainable gating matrices.

Queries, keys, and values are then computed using LoRA-augmented linear transformations with gating and residual mechanisms:

$$ Q, K, V = x_i W_{\text{base}} + G \odot (\alpha x_i A_{\text{lora}} B_{\text{lora}}) + x_i W_{\text{residual}} \quad (25) $$

The component matrices are explicitly differentiated between trainable and frozen parameters. Frozen base matrices $W_{\text{base}}$ maintain shapes $\mathbb{R}^{C_i \times d_{\text{head}}}$ and remain unchanged during training. Trainable
LoRA matrices $A_{\text{lora}}$ and $B_{\text{lora}}$ operate in reduced dimension space $\mathbb{R}^{C_i \times r}$ (typically $r = 16$) and $\mathbb{R}^{r \times d_{\text{head}}}$ respectively, ensuring computational efficiency. Residual matrices $\mathbf{R}^{\text{residual}}$ provide additional adaptation capacity, while gating matrices $\mathbf{W}^{\text{gate}}$ generate adaptive signals for selective feature emphasis.

The attention mechanism incorporates relative position bias for better spatial modeling within each window:

$$ \text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{Softmax} \left( \frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}} + \mathbf{B}_{\text{rel}} \right) \mathbf{V} \quad (26) $$

where $\mathbf{B}_{\text{rel}} \in \mathbb{R}^{M^2 \times M^2}$ is the relative position bias matrix that captures spatial relationships within each window.

The attention output is processed through our Enhanced LoRA Linear module with explicit distinction between base and trainable components:

$$ \mathbf{O}_{\text{base}} = \text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V})\mathbf{W}_{\text{o}}^{\text{base}} \quad (27) $$

$$ \mathbf{O}_{\text{lora}} = \mathbf{G}_{\text{o}} \odot (\alpha_{\text{o}} \text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V})\mathbf{A}_{\text{o}}^{\text{lora}}\mathbf{B}_{\text{o}}^{\text{lora}}) \quad (28) $$

$$ \mathbf{O}_{\text{res}} = \text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) \mathbf{R}_{\text{o}}^{\text{residual}} \quad (29) $$

$$ \mathbf{O} = \mathbf{O}_{\text{base}} + \mathbf{O}_{\text{lora}} + \mathbf{O}_{\text{res}} \quad (30) $$

This formulation addresses the reviewer's concern by clearly distinguishing parameter categories. Frozen base matrices preserve pre-trained knowledge, LoRA matrices provide low-rank adaptation, residual matrices offer additional capacity, and gating matrices enable selective feature emphasis—all while maintaining computational efficiency.

Incorporating gating and residual mechanisms addresses the limitations of standard LoRA in hyperspectral scenarios. Standard LoRA applies uniform adaptation across all features, which may be sub-optimal when certain spectral bands are more discriminative than others. The gating mechanism adaptively emphasizes the most informative spectral features, while the residual connection stabilizes training dynamics and mitigates overfitting to dataset-specific noise. Detailed ablation studies in Appendix F confirm that their combination yields the best overall accuracy with minimal overhead.

### 3.3.2. Enhanced feed-forward network block

To improve model adaptability and performance, the standard feed-forward network (FFN) in each transformer block is replaced with an enhanced architecture. This architecture pairs a Gated Residual LoRA mechanism with the SwiGLU activation function for more flexible channel-wise interactions.

The FFN computation begins with two parallel projections, $\mathbf{h}_1$ and $\mathbf{h}_2$, which expand the input representation $\mathbf{x}$. Each projection integrates a frozen base weight matrix ($\mathbf{W}^{\text{base}}$), a gated LoRA adapter, and a residual adapter. The computation for the first projection is defined as:

$$ \mathbf{h}_1 = \mathbf{x}\mathbf{W}_1^{\text{base}} + \mathbf{G}_1 \odot (\alpha_1 \mathbf{x}\mathbf{A}_1^{\text{lora}}\mathbf{B}_1^{\text{lora}}) + \mathbf{x}\mathbf{R}_1^{\text{residual}}. \quad (31) $$

Here, $A_{\text{lora}}$ and $B_{\text{lora}}$ are the trainable low-rank LoRA matrices, $\mathbf{G}$ is a trainable gating vector, $\mathbf{R}^{\text{residual}}$ is an additional trainable residual matrix, and $\alpha$ is a scaling constant. The second projection, $\mathbf{h}_2$, is computed analogously with an independent set of parameters.

The two projections interact through the SwiGLU activation:

$$ \mathbf{h} = \text{Swish}(\mathbf{h}_1) \odot \mathbf{h}_2, \quad (32) $$

where $\odot$ denotes the element-wise Hadamard product. The resulting vector $\mathbf{h}$ is then projected back to the original hidden dimension using the same Gated Residual LoRA structure:

$$ \text{FFN}(\mathbf{x}) = \mathbf{h}_3^{\text{base}} + \mathbf{G}_3 \odot (\alpha_3 \mathbf{h}_3^{\text{lora}}\mathbf{B}_3^{\text{lora}}) + \mathbf{h}_3^{\text{residual}}. \quad (33) $$

This complete FFN module is integrated within a Pre-LayerNorm (Pre-LN) transformer block architecture. Regularization is applied using Stochastic Depth (DropPath), where entire sub-layers are randomly

dropped during training. The forward pass for a full transformer block is given by:

$$ \begin{aligned} \mathbf{Y}_{\text{attn}} &= \mathbf{X}_{\text{in}} + \text{DropPath}(\text{AttentionBlock}(\text{LayerNorm}(\mathbf{X}_{\text{in}}))), \\ \mathbf{Y}_{\text{ffn}} &= \mathbf{Y}_{\text{attn}} + \text{DropPath}(\text{FFN}(\text{LayerNorm}(\mathbf{Y}_{\text{attn}}))), \end{aligned} \quad (34) $$

where $\mathbf{Y}_{\text{ffn}}$ is the final output of the block.

### 3.3.3. Downsampling module

After processing through the transformer blocks, the downsampling module reduces spatial complexity while preserving semantic information through a compression pipeline with adaptive attention. This module prepares features for the next stage while maintaining the enhanced LoRA structure throughout the network.

The downsampling begins with depthwise convolution and squeeze-and-excitation attention:

$$ U = \text{Swish}(\text{Conv2d}_{3 \times 3}, \text{groups}=C_i(\mathbf{X})) \quad (35) $$

where depthwise convolution reduces computational cost while preserving spatial information. The adaptive attention mechanism recalibrates channel importance:

$$ \mathbf{z} = \text{AdaptiveGAP}(U) \in \mathbb{R}^{B \times C_i}, \quad \mathbf{s} = \sigma(\mathbf{W}_{\text{sc2}}^{\text{trainable}} \text{Swish}(\mathbf{W}_{\text{sc1}}^{\text{trainable}} \mathbf{z})), \quad \mathbf{V} = \mathbf{s} \odot U \quad (36) $$

The final spatial reduction is achieved through pointwise convolution, normalization, and strided convolution:

$$ \mathbf{Y} = \text{BatchNorm2d}(\text{Conv2d}_{1 \times 1}(\mathbf{V})), \quad \mathbf{Z} = \text{Conv2d}_{3 \times 3, \text{stride}=2}(\mathbf{Y}). \quad (37) $$

This progressive reduction ensures that higher levels aggregate increasingly larger regions of the input, achieving complete global coverage without requiring shifted-window mechanisms.

## 3.4. Classification and training strategy

The final stage of our architecture processes the extracted features through a classification head that maintains consistency with the enhanced LoRA mechanisms used throughout the network, ensuring that the entire architecture follows the same parameter-efficient adaptation strategy.

### 3.4.1. Parameter-efficient classification with global feature aggregation

The final backbone block produces spatial features that are globally aggregated for classification through layer normalization and global average pooling.

$$ \mathbf{f}_{\text{global}} = \text{GAP}(\text{LayerNorm}(\mathbf{X}_2)) \in \mathbb{R}^{B \times C_{\text{final}}} \quad (38) $$

where $\text{GAP}(\cdot)$ denotes global average pooling that aggregates spatial information, and $\text{LayerNorm}(\cdot)$ applies layer normalization for feature stabilization.

The classification is performed using our Enhanced LoRA Linear module configured for efficiency (LoRA only, without gate/residual mechanisms for computational efficiency):

$$ \hat{\mathbf{y}} = \mathbf{f}_{\text{global}}\mathbf{W}_{\text{cls}}^{\text{base}} + \alpha_{\text{cls}} \cdot \mathbf{f}_{\text{global}}\mathbf{A}_{\text{cls}}^{\text{lora}}\mathbf{B}_{\text{cls}}^{\text{lora}} \quad (39) $$

where the classification components are:

* $\mathbf{W}_{\text{cls}}^{\text{base}} \in \mathbb{R}^{C_{\text{final}} \times N_{\text{classes}}}$: Frozen base classification matrix

* $\mathbf{A}_{\text{cls}}^{\text{lora}} \in \mathbb{R}^{C_{\text{final}} \times r}$: Trainable LoRA down-projection matrix

* $\mathbf{B}_{\text{cls}}^{\text{lora}} \in \mathbb{R}^{r \times N_{\text{classes}}}$: Trainable LoRA up-projection matrix

This simplified formulation maintains parameter efficiency while reducing computational overhead in the final classification stage, where the gating and residual mechanisms provide diminishing returns compared to the attention and MLP layers.
The final class probabilities are obtained through softmax normalization:

$$P(c|x) = \frac{\exp(\hat{y}_c)}{\sum_{j=1}^{N_{\text{classes}}} \exp(\hat{y}_j)} \qquad (40)$$

with the predicted class determined by:

$$\hat{c} = \arg \max_{c \in \{1, \dots, N_{\text{classes}}\}} P(c|x) \qquad (41)$$

### 3.4.2. Training dynamics and strategic LoRA integration

To optimize the training of our enhanced LoRA mechanisms and ensure stable convergence, we introduce a cyclic learning rate scheduler specifically designed for LoRA parameters. This scheduler complements the architectural design by providing optimal training dynamics:

$$LR_{LoRA}(t) = LR_{base} + (LR_{max} - LR_{base}) \cdot \frac{1 + \cos(\pi \cdot \frac{t \mod T}{T})}{2} \cdot \text{scale}_{LoRA} \quad (42)$$

where $t$ is the current iteration, $T$ is the cycle length, and scale$_{LoRA}$ is a scaling factor for LoRA parameters to enable more aggressive adaptation during training.

To validate the effectiveness of our design, we evaluated the coordinated use of LoRA across the transformer backbone through an ablation study on the LongKou and Salinas datasets with 2% training samples. The results, provided in Appendix C (Table C.1), demonstrate that while introducing LoRA in isolated components yields incremental gains, the full integration consistently achieves the highest accuracy, confirming that the benefit of LoRA arises from its joint deployment across attention and feed-forward pathways rather than from any single placement.

### 3.4.3. Selection of design parameters $\alpha$, $\beta$, and $\gamma$

The adaptation hyperparameters $\alpha$, $\beta$, and $\gamma$ critically influence the balance between model expressiveness and training stability in our PEFT-enhanced architecture. To determine optimal values, we systematically compared four candidate configurations through empirical evaluation.

Empirical evaluation (Appendix G.4, Table G.4) demonstrates that the final configuration delivers the most favorable trade-off between overall accuracy and convergence behavior. The setting $\alpha = 32$ adheres to established LoRA guidelines for a scaling ratio $\alpha/r = 2$ (Hu et al., 2021), while retaining the standard initialization as recommended by LayerScale (Touvron et al., 2021).

## 3.5. Transfer learning with pretrained backbones

We use optionally pre-trained GCViT backbones from Hugging Face to improve the feature representation capabilities of the model. The transfer learning process involves applying LoRA adapters to the pretrained attention and MLP layers, and then merging their learned weights into our custom model to initialize the same improved LoRA structure. The integration process starts with loading pre-trained GCViT weights and freezing the base parameters. Then, LoRA adapters are strategically inserted into attention projections and feed-forward networks, allowing the model to adapt to hyperspectral features while preserving the learned visual features. This approach allows the model to utilize pre-trained knowledge while maintaining our own gating and residual mechanisms throughout the network.

# 4. Experiments

## 4.1. Dataset

In this paper, we use four publicly available hyperspectral datasets, WHU-Hi-LongKou, WHU-Hi-HongHu, Salinas, and QUH-Qingyun, to comprehensively evaluate our classification system in various agricultural and complex real-world scenarios. These datasets differ in spatial

resolution, spectral coverage, scene complexity, and the number of land cover classes, providing a solid basis for evaluating the model's generalizability.

WHU-Hi-Longkou dataset, illustrated in Fig. 2, was captured on July 17, 2018, in Longkou Town with a Headwall Nano-Hyperspec sensor on a DJI Matrice 600 Pro drone. The imagery covers an area of 550×400 pixels with a spatial resolution of about 0.463 m and includes 270 spectral bands from 400 nm to 1000 nm, showing a simple agricultural scene with six major crops: corn, cotton, sesame, broad-leaf soybeans, narrow-leaf soybeans, and rice. The ground truth provided defines nine land cover classes, including the six crops as well as water, cultivated area, and mixed weeds (RSIDEA group, Wuhan University, 2018).

WHU-Hi-HongHu dataset, illustrated in Fig. 3, was taken on November 20, 2017, in Honghu City with the same Nano Hyperspec sensor on a DJI Matrice 600 Pro drone. This image is 940×475 pixels in size, and contains 270 spectral bands with a spatial resolution of 0.043 m. The scene depicts a complex agricultural test area featuring numerous crops, as well as other elements such as roads, roofs, and bare ground. The ground truth includes 22 land cover classes in the vegetation and infrastructure categories (RSIDEA group, Wuhan University, 2017).

Salinas scene, illustrated in Fig. 4, is an airborne hyperspectral image of agricultural land in Salinas Valley, California, USA. It was acquired by NASA's AVIRIS instrument at 3.7 m ground sampling distance. The scene is 512×217 pixels, with 224 original bands. It describes vegetable fields, vineyards, and bare soils, and is annotated in 16 classes of land cover (CCWintco Elhami et al., 2004).

QUH-Qingyun dataset, illustrated in Fig. 5, offers a more sophisticated scenario, with higher spectral complexity and greater class diversity compared to the mainly agricultural datasets mentioned above. It was collected in Qingyun County, China, using a ground-based imaging system with 224 spectral bands. The dataset is annotated into 12 land cover classes, including both agricultural and non-agricultural categories, making it a rigorous benchmark for evaluating generalizability under label-scarce conditions (Fu et al., 2023).

## 4.2. Configurations

To ensure reproducibility and a fair comparison, all experiments were carried out in a standardized hardware and software environment. The implementation was developed using the PyTorch framework and executed on a workstation equipped with an NVIDIA GeForce RTX 3080 GPU and 64 GB system RAM. We used the Adam optimizer with an initial learning rate of 1 times 10⁻³. The models were trained for 100 epochs with a mini-batch size of 64. Each training instance consisted of hyperspectral patches of size 15×15×15 (spectral ×height ×width), selected on the basis of non-overlapping spatial windows. Our proposed LoLA-SpecViT was configured with three Multi-Scale stages, with depths of [3, 4, 19], embedding dimension of 96, and head counts of attention of [4, 8, 16]. Each stage employed a windowed self-attention mechanism with a fixed size of 7×7. For regularization, we applied a drop path rate of 0.2 and integrated LoRA into attention layers using a rank of r = 16 and scaling factor $\alpha = 32$. To improve fine-tuning stability and efficiency, we employed a CLR scheduler designed for LoRA modules. The CLR policy oscillated the internal scaling factor between 0.8 and 1.5 using a 'triangular2' schedule with a step size of 100 iterations. This dynamic adjustment encouraged smoother convergence and better generalization across datasets.

# 5. Results

To validate our approach in relation to the fundamental challenges of hyperspectral classification, including high-dimensional data, limited labels, and complex spectral variability, we conduct a series of experiments on four benchmark datasets with training samples comprising only 2%, 5%, and 10% of the total. To evaluate the effectiveness of
<figure><img src="image_6.png" /><figcaption>Fig. 2. Visualization of hyperspectral data cubes and corresponding ground-truth classification maps for the WHU-Hi-Longkou dataset. (a) Image cube. (b) Ground truth image.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 3. Visualization of hyperspectral data cubes and corresponding ground-truth classification maps for the WHU-Hi-HongHu dataset. (a) Image cube. (b) Ground truth image.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 4. Visualization of hyperspectral data cubes and corresponding ground-truth classification maps for the Salinas dataset. (a) Image cube. (b) Ground truth image.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 5. Visualization of hyperspectral data cubes and corresponding ground-truth classification maps for the QUH-Qingyun dataset. (a) Image cube. (b) Ground truth image (Fu et al., 2023).</figcaption></figure>
<caption>**Table 1**</caption>

Overall, average, kappa and per-class accuracies (%) on the HongHu dataset (10% training samples) for Ours, HSIMAE (Ibanez et al., 2022), HybridSN (Roy et al., 2020), ViT (Dosovitskiy et al., 2021), MASSFormer (Sun et al., 2024), DBCT (Xu et al., 2024) and LSGA (Ma et al., 2023).

<table><thead><tr><th></th><th>Ours</th><th>HSIMAE</th><th>HybridSN</th><th>ViT</th><th>MASSFormer</th><th>DBCT</th><th>LSGA</th></tr></thead><tbody><tr><td>Overall Accuracy (%)</td><td><strong>99.81</strong></td><td>96.24</td><td>96.76</td><td>96.76</td><td>98.14</td><td><u>99.14</u></td><td>99.02</td></tr><tr><td>Average Accuracy (%)</td><td><strong>99.53</strong></td><td>92.99</td><td>92.50</td><td>92.50</td><td>96.25</td><td><u>98.07</u></td><td>97.54</td></tr><tr><td>Kappa Score (%)</td><td><strong>99.29</strong></td><td>96.18</td><td>96.77</td><td>96.77</td><td>97.65</td><td><u>98.91</u></td><td>98.75</td></tr><tr><td>Red roof</td><td><strong>99.68</strong></td><td>97.40</td><td>98.46</td><td>97.84</td><td>98.99</td><td>98.27</td><td><u>99.54</u></td></tr><tr><td>Road</td><td><strong>99.71</strong></td><td>88.87</td><td>75.45</td><td>87.22</td><td><u>97.25</u></td><td>97.44</td><td>96.82</td></tr><tr><td>Bare soil</td><td><strong>99.59</strong></td><td>95.84</td><td>95.61</td><td>94.98</td><td>98.87</td><td><u>99.01</u></td><td>98.23</td></tr><tr><td>Cotton</td><td><strong>100.00</strong></td><td>98.13</td><td>99.40</td><td>98.48</td><td>99.31</td><td><u>99.69</u></td><td>99.65</td></tr><tr><td>Cotton firewood</td><td><strong>99.49</strong></td><td>78.73</td><td>88.93</td><td>84.61</td><td>94.76</td><td>98.32</td><td><u>99.14</u></td></tr><tr><td>Rape</td><td><strong>100.00</strong></td><td>98.81</td><td>98.41</td><td>98.60</td><td>99.11</td><td><u>99.47</u></td><td>99.34</td></tr><tr><td>Chinese cabbage</td><td><strong>99.54</strong></td><td>94.10</td><td>95.77</td><td>92.50</td><td>97.15</td><td><u>98.81</u></td><td>98.01</td></tr><tr><td>Pakchoi</td><td><strong>99.52</strong></td><td>87.53</td><td>93.72</td><td>87.50</td><td>94.16</td><td>98.38</td><td>96.59</td></tr><tr><td>Cabbage</td><td><strong>99.51</strong></td><td>98.14</td><td>97.91</td><td>97.91</td><td><u>99.77</u></td><td>99.71</td><td>99.82</td></tr><tr><td>Tuber mustard</td><td><strong>99.63</strong></td><td>94.97</td><td>93.73</td><td>92.90</td><td>98.03</td><td>98.53</td><td><u>99.94</u></td></tr><tr><td>Brassica parachinensis</td><td><strong>99.45</strong></td><td>91.95</td><td>91.88</td><td>88.10</td><td>95.53</td><td>98.77</td><td>97.75</td></tr><tr><td>Brassica chinensis</td><td><strong>99.59</strong></td><td>92.80</td><td>92.17</td><td>91.23</td><td>98.15</td><td><u>98.42</u></td><td>96.98</td></tr><tr><td>Small Brassica chinensis</td><td><strong>99.41</strong></td><td>92.68</td><td>94.22</td><td>91.70</td><td>94.73</td><td>98.48</td><td>97.63</td></tr><tr><td>Lactuca sativa</td><td><strong>99.68</strong></td><td>95.31</td><td>92.83</td><td>91.74</td><td>98.48</td><td><u>99.02</u></td><td>98.90</td></tr><tr><td>Celtuce</td><td><strong>99.48</strong></td><td>94.56</td><td>87.59</td><td>94.02</td><td>98.51</td><td>98.18</td><td>97.96</td></tr><tr><td>Film covered lettuce</td><td><strong>99.89</strong></td><td>98.14</td><td>97.30</td><td>97.69</td><td>97.03</td><td>98.59</td><td><u>99.16</u></td></tr><tr><td>Romaine lettuce</td><td><strong>99.51</strong></td><td>96.94</td><td>92.21</td><td>95.59</td><td>98.86</td><td><u>99.47</u></td><td>99.32</td></tr><tr><td>Carrot</td><td><strong>99.32</strong></td><td>90.37</td><td>89.72</td><td>93.51</td><td>95.90</td><td>95.16</td><td><u>96.56</u></td></tr><tr><td>White radish</td><td><strong>99.44</strong></td><td>95.63</td><td>91.73</td><td>91.34</td><td>95.61</td><td>90.11</td><td><u>99.51</u></td></tr><tr><td>Garlic sprout</td><td><strong>99.35</strong></td><td>88.67</td><td>89.22</td><td>86.62</td><td>93.85</td><td><u>99.00</u></td><td>97.48</td></tr><tr><td>Broad bean</td><td><strong>98.67</strong></td><td>85.15</td><td>83.35</td><td>84.24</td><td>87.52</td><td><u>90.77</u></td><td>91.62</td></tr><tr><td>Tree</td><td><strong>99.17</strong></td><td>91.16</td><td>95.38</td><td>88.90</td><td>88.02</td><td><u>96.76</u></td><td>98.08</td></tr></tbody></table>

our multi-scale spectral transformer, we compare it with state-of-the-art methods on the Longkou dataset. We first compare global performance metrics to these baselines; we then analyze class-specific results to illustrate how our model overcomes redundancy, captures subtle spectral differences, and mitigates overfitting in the presence of severe label scarcity.

## 5.1. Quantitative performance analysis

The evaluation of the WHU-Hi-HongHu dataset shows that our proposed framework achieves the best performance, with an OA of **99.81**, an AA of **99.53**, and a kappa coefficient of **99.29**. These results outperform the best existing method, DBCT, by 0.67%, 1.46%, and 0.38%, respectively, as shown in Table 1. These gains are particularly remarkable considering that the experiments were performed with only 10% of the available labeled data. The robustness of our approach under such limited monitoring conditions highlights the effectiveness of our spectral processing front-end, which employs band dropout and channel attention modules to emphasize discriminative bands and suppress noise. In addition, the integration of LoRA for parameter-efficient adaptation and SwiGLU activations for improved nonlinearity contributes to a more meaningful and stable learning process, which allows the model to capture long-range spectral-spatial dependencies without overfitting.

Comparisons at the class level further underpin these improvements. The cotton category reaches 100.00%, outperforming DBCT by 0.31%, demonstrating the model's ability to capture intra-class spectral variations caused by factors such as crop density and soil background. The Rape class, which is often misclassified due to spectral similarities with other Brassica species, experiences an improvement of 0.53%, reaching 100.00%. Similarly, Film Covered Lettuce, a spectrally complex class affected by reflectance distortions due to plastic mulch, achieves 99.89%, outperforming LSGA by 0.73%. These results, along with the consistent performance in both vegetative and non-vegetative classes, confirm the superior ability of the proposed model to capture fine-grained spectral-spatial patterns and its generalization to heterogeneous and ambiguous regions in real HSI scenes.

Building upon these results, we evaluate the generalizability of our framework using the Salinas dataset, which exhibits different spectral and spatial characteristics under semiarid agricultural conditions. As shown in Table 2, our model achieves an OA of **99.87**, outperforming the previous state-of-the-art MADANet by 0.70 percentage points. The AA also reaches **99.71**, outperforming MADANet's 99.12% by 0.59%, while the kappa value increases to **99.49**, improving SSFTT's 98.46% by 1.03 percentage points. These consistent improvements, achieved with only 10% of the labeled samples, confirm the robustness of our design. The adaptive BandDropout mechanism, in conjunction with spectral attention and LoRA-enhanced transformer blocks, contributes to more effective modeling of discriminative spectral-spatial patterns, while SwiGLU activations enable better noise filtering and convergence stability during training.

At the class level, our framework shows clear advantages in distinguishing between spectrally overlapping or phenologically variable crop types. For example, the class *Grapes_untrained* achieves an accuracy of **100.00**, outperforming A2MFE by 2.18 percentage points, confirming the model's ability to remove background interference from soil and weeds. In the class *Lettuce_romaine_7wk*, which includes subtle variations in canopy maturity and shading, our method achieves a perfect classification (**100.00**%), and outperforms HiT by 0.58 percentage points. In addition, the class *Corn_senesced_green-weeds*, which is known for its spectral ambiguity due to aging vegetation, is classified with an accuracy of **99.80**, which is slightly below the 99.81% of SSFTT. These improvements, especially in the most challenging categories, emphasize the model's ability to maintain fine-grained class discrimination in different agricultural landscapes.

These results for the Salinas dataset confirm that our method consistently produces excellent results for datasets with varying spectral complexity, fine-grained class structure, and geographic diversity. To further assess generalizability, we extend our evaluation to the WHU-Hi-LongKou dataset, a scenario characterized by simpler crop types and relatively homogeneous spectral distributions.

The full comparison is shown in Table 3, where our proposed framework again achieves the highest performance across all reported metrics. In particular, it achieves an OA of **99.91**, outperforming
<caption>**Table 2**</caption>

Overall, average, kappa and per-class precision (%) on the Salinas dataset (10% training samples) for Ours, MADANet (Cui et al., 2023), A²MFE (Yang et al., 2021), HybridSN (Roy et al., 2020), SSFTT (Sun et al., 2022), HiT (Yang et al., 2022), and LANet (Ding et al., 2021).

<table><caption>**Table 3**</caption><thead><tr><th></th><th>Ours</th><th>MADANet</th><th>A²MFE</th><th>HybridSN</th><th>SSFTT</th><th>HiT</th><th>LANet</th></tr></thead><tbody><tr><td>Overall Accuracy (%)</td><td><strong>99.87</strong></td><td><u>99.17</u></td><td>98.56</td><td>97.05</td><td>98.61</td><td>97.83</td><td>96.67</td></tr><tr><td>Average Accuracy (%)</td><td><strong>99.71</strong></td><td><u>99.12</u></td><td>98.72</td><td>97.52</td><td>98.97</td><td>98.87</td><td>97.12</td></tr><tr><td>Kappa Score (%)</td><td><strong>99.49</strong></td><td>98.34</td><td>97.35</td><td>96.72</td><td><u>98.46</u></td><td>97.58</td><td>96.39</td></tr><tr><td>Brocoli_green_weeds_1</td><td><strong>100.00</strong></td><td>99.64</td><td>99.46</td><td>99.30</td><td>99.34</td><td><u>100.00</u></td><td>98.32</td></tr><tr><td>Brocoli_green_weeds_2</td><td><strong>100.00</strong></td><td>98.95</td><td>97.28</td><td><u>99.96</u></td><td>100.00</td><td>99.75</td><td>98.64</td></tr><tr><td>Fallow</td><td><strong>100.00</strong></td><td>98.90</td><td>97.99</td><td>98.16</td><td><u>99.89</u></td><td>100.00</td><td>95.09</td></tr><tr><td>Fallow_rough_plow</td><td><strong>100.00</strong></td><td>97.58</td><td>95.67</td><td>98.05</td><td><u>99.34</u></td><td>99.85</td><td>98.01</td></tr><tr><td>Fallow Smooth</td><td><strong>99.59</strong></td><td>99.02</td><td>98.85</td><td>98.42</td><td><strong>99.50</strong></td><td>97.59</td><td>98.73</td></tr><tr><td>Stubble</td><td><strong>100.00</strong></td><td>98.01</td><td>98.31</td><td>98.67</td><td>98.31</td><td><u>99.92</u></td><td>98.86</td></tr><tr><td>Celery</td><td><strong>100.00</strong></td><td>99.34</td><td>95.76</td><td>98.96</td><td>99.91</td><td><u>100.00</u></td><td>97.04</td></tr><tr><td>Grapes_untrained</td><td><strong>100.00</strong></td><td>97.77</td><td><u>97.82</u></td><td>92.70</td><td>97.80</td><td>95.81</td><td>96.49</td></tr><tr><td>Soil_vinyard_develop</td><td><strong>100.00</strong></td><td>98.23</td><td>97.56</td><td>98.53</td><td><u>100.00</u></td><td>100.00</td><td>93.11</td></tr><tr><td>Corn_senesced_green_weeds</td><td><strong>99.80</strong></td><td>95.91</td><td>95.23</td><td>95.44</td><td><strong>99.81</strong></td><td>98.19</td><td>94.27</td></tr><tr><td>Lettuce_romaine_4wk</td><td><strong>100.00</strong></td><td>98.86</td><td>97.45</td><td>98.08</td><td><u>98.77</u></td><td>99.42</td><td>97.19</td></tr><tr><td>Lettuce_romaine_5wk</td><td><strong>100.00</strong></td><td>99.23</td><td>98.34</td><td>99.33</td><td><u>99.89</u></td><td>99.84</td><td>98.42</td></tr><tr><td>Lettuce_romaine_6wk</td><td><strong>100.00</strong></td><td><u>100.00</u></td><td>99.21</td><td>99.02</td><td>97.02</td><td>100.00</td><td>98.58</td></tr><tr><td>Lettuce_romaine_7wk</td><td><strong>100.00</strong></td><td>99.34</td><td>98.12</td><td>98.21</td><td>98.96</td><td><u>99.42</u></td><td>98.07</td></tr><tr><td>Vinyard_untrained</td><td><strong>95.69</strong></td><td><u>98.49</u></td><td>94.55</td><td>92.29</td><td>95.52</td><td>92.53</td><td>92.46</td></tr><tr><td>VinyardVertical trellis</td><td><strong>100.00</strong></td><td>98.36</td><td>93.23</td><td>80.43</td><td>99.49</td><td><u>99.66</u></td><td>90.11</td></tr></tbody></table>

Comparison of classification performance on the Longkou dataset with 10% training samples for Ours, Cross (Bai et al., 2024), Hybrid-ViT (Arshad and Ullah, 2024), Hir-Transformer (Ahmad et al., 2024), SSMamba (Ahmad et al., 2025), MorpMamba (Ahmad et al., 2025), and E-SR-SSIM (Hu et al., 2023).

<table><thead><tr><th>Metrics</th><th>Ours</th><th>Cross</th><th>Hybrid-ViT</th><th>Hir-Transformer</th><th>SSMamba</th><th>MorpMamba</th><th>E-SR-SSIM</th></tr></thead><tbody><tr><td>Overall Accuracy (%)</td><td><strong>99.91</strong></td><td><u>99.78</u></td><td>99.75</td><td>99.68</td><td>99.51</td><td>99.70</td><td>94.43</td></tr><tr><td>Average Accuracy (%)</td><td><strong>99.71</strong></td><td><u>99.45</u></td><td>99.36</td><td>99.14</td><td>98.45</td><td>99.25</td><td>81.78</td></tr><tr><td>Kappa Score (%)</td><td><strong>99.89</strong></td><td><u>99.78</u></td><td>99.68</td><td>99.59</td><td>99.36</td><td>99.61</td><td>92.63</td></tr></tbody></table>

the strongest competing model, Cross, by 0.13%. Hybrid-ViT and Hir-Transformer, which are both based on transformer-based architectures, follow with OA values of 99.75% and 99.68%, respectively, and are at least 0.16% below. Our model also achieves the highest AA of 99.71%, which is a significant improvement over Cross (99.45%), Hybrid-ViT (99.36%), and Hir-Transformer (99.14%). These gains reflect the advantages of our LoRA-enhanced multi-scale transformer backbone, which captures complex spectral-spatial dependencies while maintaining parameter efficiency. Spatial-spectral baselines such as SSMamba and MorpMamba also provide strong results (OA: 99.51% and 99.70%; AA: 98.45% and 99.25%), but are outperformed by our model. In addition, our model achieves the highest kappa score with 99.89%, and is thus ahead of Cross (99.78%), Hybrid-ViT (99.68%), and Hir-Transformer (99.59%). In contrast, the reconstruction-based E-SR-SSIM shows significantly lower performance in all metrics, highlighting the limitations of generative methods under low-label conditions.

The per-class performance analysis shown in Table 4 highlights the exceptional ability of our system to discriminate all nine land cover categories with only 10% of the available training samples. The model achieves an OA of 99.91%, supported by consistently near-perfect precision, recall, and F1 values. These results underline the effectiveness of our multi-scale spectral transformer architecture under data-scarce conditions. Spectrally homogeneous classes such as *Rice* and *Water* are an example of this robustness, as they achieve precision, recall, and F1-scores of 99.99% and 99.97%, respectively. *Broad-leaf soybean* achieves almost ideal values with 99.95% precision and 99.96% recall, resulting in an F1 score of 99.96%. Similarly, *Corn* achieves a recall of 99.99% and an F1 score of 99.98%, indicating that our model successfully captures the full range of class-relevant spectral-spatial information even with minimal monitoring.

The remaining classes confirm the generalizability of our approach. *Sesame* achieves perfect precision (100%), but a slightly lower recall (99.41%), resulting in an F1 score of 99.71%, indicating low confusion at the class boundaries. *Narrow-leaf soybean*, which has significant spectral similarity to related cultivars, retains high reliability with 99.79%

<caption>**Table 4**</caption>

Per-class precision, recall, and F1-score for our method (10% training samples).

<table><thead><tr><th>Class</th><th>Precision</th><th>Recall</th><th>F1-Score</th></tr></thead><tbody><tr><td>Corn</td><td>0.9996</td><td><strong>0.9999</strong></td><td><u>0.9998</u></td></tr><tr><td>Cotton</td><td>0.9985</td><td><strong>1.0000</strong></td><td><u>0.9993</u></td></tr><tr><td>Sesame</td><td><strong>1.0000</strong></td><td>0.9941</td><td><u>0.9971</u></td></tr><tr><td>Broad-leaf soybean</td><td><u>0.9995</u></td><td><strong>0.9996</strong></td><td><u>0.9996</u></td></tr><tr><td>Narrow-leaf soybean</td><td>0.9960</td><td><strong>0.9979</strong></td><td><u>0.9969</u></td></tr><tr><td>Rice</td><td><strong>0.9999</strong></td><td><u>0.9999</u></td><td><u>0.9999</u></td></tr><tr><td>Water</td><td><strong>0.9997</strong></td><td><u>0.9997</u></td><td><u>0.9997</u></td></tr><tr><td>Roads and houses</td><td><strong>0.9927</strong></td><td>0.9914</td><td><u>0.9920</u></td></tr><tr><td>Mixed weed</td><td><strong>0.9928</strong></td><td>0.9921</td><td><u>0.9925</u></td></tr><tr><td>Overall Accuracy (Ours)</td><td></td><td></td><td><strong>0.9991</strong></td></tr></tbody></table>

recall and a 99.69% F1 score. Structural categories such as *Roads and Houses* are also well captured, achieving 99.27% precision and a 99.20% F1 score. In particular, the difficult class *Mixed Weed*, which is composed of heterogeneous vegetation, achieves an accuracy of 99.28% precision and an F1-score of 99.25%, reflecting the ability of our model to manage complex, noisy signatures. These consistently good results for both homogeneous and spectrally diverse classes confirm that our LoRA-enhanced multiscale transformer is highly effective in resolving class boundaries and maintaining classification fidelity in real-world agricultural mapping scenarios.

Following the comprehensive 10% evaluation, we assess the robustness of our model under conditions of extreme data scarcity with only 5% of the available training samples. As shown in Table 5, our proposed framework achieves excellent classification performance, with an *OA* of 99.69%, an *AA* of 99.75%, and a *Kappa score* of 99.59%. These results highlight the strong generalization ability of our multi-scale spectral transformer, which leverages localized spectral modeling and parameter-efficient adaptation to maintain high discriminative power despite the significant reduction in supervision. Core agricultural classes such as *Corn*, *Cotton*, and *Broad-leaf soybean* continue to exhibit exceptional performance, with accuracies of 99.91%, 99.95%, and
<caption>**Table 5**</caption>

Comparison of per-class and aggregated classification performance on the WHU-Hi-LongKou dataset with 5% training samples for LoLA-SpecViT (Ours), FFT-SpectralGPT (Ligan et al., 2025), MambaHSI (Li et al., 2024), and HyperSIGMA (Wang et al., 2025).

<table><thead><tr><th>Class/Metric</th><th>FFT-SpectralGPT</th><th>MambaHSI</th><th>HyperSIGMA</th><th>LoLA-SpecViT (ours)</th></tr></thead><tbody><tr><td>Corn</td><td>98.03</td><td>99.70</td><td>99.77</td><td><strong>99.91</strong></td></tr><tr><td>Cotton</td><td>98.68</td><td>98.98</td><td><strong>100.00</strong></td><td>99.95</td></tr><tr><td>Sesame</td><td>99.29</td><td>97.64</td><td>99.90</td><td><strong>99.92</strong></td></tr><tr><td>Broad-leaf soybean</td><td>95.13</td><td>99.21</td><td>99.43</td><td><strong>99.79</strong></td></tr><tr><td>Narrow-leaf soybean</td><td>96.31</td><td>89.43</td><td>99.78</td><td><strong>99.85</strong></td></tr><tr><td>Rice</td><td>97.58</td><td>99.64</td><td><strong>99.85</strong></td><td>99.82</td></tr><tr><td>Water</td><td>99.23</td><td><strong>99.96</strong></td><td>99.75</td><td>99.92</td></tr><tr><td>Roads and houses</td><td>97.57</td><td>97.19</td><td>98.47</td><td><strong>99.71</strong></td></tr><tr><td>Mixed weed</td><td>98.08</td><td>98.96</td><td>99.24</td><td><strong>99.89</strong></td></tr><tr><td>OA (%)</td><td>97.55</td><td>99.62</td><td>99.45</td><td><strong>99.69</strong></td></tr><tr><td>AA (%)</td><td>97.77</td><td>99.58</td><td>99.61</td><td><strong>99.75</strong></td></tr><tr><td>Kappa (%)</td><td>96.73</td><td>99.42</td><td>99.26</td><td><strong>99.59</strong></td></tr></tbody></table>

99.79%, respectively. Similarly, spectrally consistent categories such as *Rice* and *Water* achieve outstanding accuracies of 99.82% and 99.92%, reflecting the model's ability to utilize hierarchical feature abstraction for robust spatial-spectral representation under minimal supervision.

The performance in the other categories also confirms the resilience of our approach. For example, *Narrow-leaf soybean* reaches an excellent accuracy of 99.85%, while the structural class *Roads and Houses* demonstrates superior recognition with a class score of 99.71%. The challenging class *Mixed Weed*, which is characterized by spectral heterogeneity, achieves an outstanding class score of 99.89%, underlining the model's exceptional ability to disentangle fine-grained spectral variability. *Sesame* achieves strong performance with a class score of 99.92%, demonstrating competitive results across all evaluation metrics. Overall, these results show that our method maintains exceptional classification fidelity for both homogeneous and complex land cover categories, even when trained with only 5% of the data. This robustness under severe data constraints highlights the advantage of our architecture in real-world remote sensing applications, where labeled samples are often limited, and is further emphasized when contrasted with recent PEFT and hyperspectral foundation models: LoLA-SpecViT demonstrates superior performance compared to FFT-SpectralGPT, achieving improvements of 2.14% in OA (99.69% compared to 97.55%), 1.98% in AA (99.75% compared to 97.77%), and 2.86% in Kappa (99.59% compared to 96.73%). When compared to HyperSIGMA, our method achieves higher OA by 0.24% (99.69% compared to 99.45%), superior AA by 0.14% (99.75% compared to 99.61%), and Kappa by 0.33% (99.59% compared to 99.26%).

The performance of our method under extreme data sparsity, where only 2% of the Longkou dataset was used for training, is summarized in Table 6. Despite minimal supervision, our method achieves the highest accuracy per class in eight out of nine land cover categories, demonstrating exceptional discriminative power under stringent constraints. In particular, for Class 1, our model achieves an accuracy of 99.98%, outperforming GAHT by 4.08%; in Class 2, it reaches 99.88%, outperforming AMHFN by 4.57%; and in Class 4, the accuracy of 99.95% outperforms AMHFN by 3.70%. Furthermore, class 5 achieves 99.64%, which is 3.89% higher accuracy than GAHT; class 7 improves by 0.43% compared to GAHT and AMHFN; and class 9 achieves 99.23%, which is 6.47% ahead of AMHFN. Our model clearly outperforms AMHFN's results in class 3 with 99.98% compared to 96.16%, and in class 6 with 99.93% compared to AMHFN's 98.09%. In class 8, our method achieves 99.11%, and is thus on a par with AMHFN. Remarkably, HiT consistently lags behind the top methods in every class, further emphasizing the advantage of our model. At the aggregate level, the superiority of our multiscale spectral transformer becomes even clearer. Our model achieves the highest OA of 99.72%, and outperforms AMHFN (97.07%) and GAHT (96.82%) by 2.65% and 2.90%, respectively. The kappa coefficient further underpins this lead, as our method achieves 99.51%, significantly outperforming AMHFN (96.17%) and GAHT (95.86%).

<caption>**Table 6**</caption>

Classification accuracy on the Longkou dataset with 2% training samples for ViT (Dosovitskiy et al., 2021), PiT (Heo et al., 2021), HiT (Yang et al., 2022), GAHT (Mei et al., 2022), AMHFN (Yang et al., 2024).

<table><thead><tr><th>Class No.</th><th>ViT</th><th>PiT</th><th>HiT</th><th>GAHT</th><th>AMHFN</th><th>Ours</th></tr></thead><tbody><tr><td>1</td><td>89.72</td><td>89.36</td><td>89.83</td><td><u>95.90</u></td><td>95.85</td><td><strong>99.98</strong></td></tr><tr><td>2</td><td>63.64</td><td>87.43</td><td>90.17</td><td>94.82</td><td><u>95.31</u></td><td><strong>99.88</strong></td></tr><tr><td>3</td><td>75.12</td><td>48.18</td><td>88.01</td><td>94.44</td><td>96.16</td><td><strong>99.98</strong></td></tr><tr><td>4</td><td>90.14</td><td>89.62</td><td>90.61</td><td>95.64</td><td><u>96.25</u></td><td><strong>99.95</strong></td></tr><tr><td>5</td><td>78.96</td><td>76.77</td><td>91.62</td><td><u>95.75</u></td><td>94.47</td><td><strong>99.64</strong></td></tr><tr><td>6</td><td>96.36</td><td>95.45</td><td>96.12</td><td>97.87</td><td>98.09</td><td><strong>99.93</strong></td></tr><tr><td>7</td><td>97.55</td><td>97.55</td><td>97.57</td><td><u>99.03</u></td><td><u>99.03</u></td><td><strong>99.46</strong></td></tr><tr><td>8</td><td>94.11</td><td>91.79</td><td>93.34</td><td><u>96.89</u></td><td><strong>99.11</strong></td><td><strong>99.11</strong></td></tr><tr><td>9</td><td>89.72</td><td>90.59</td><td>92.33</td><td>91.98</td><td><u>92.76</u></td><td><strong>99.23</strong></td></tr><tr><td>κ (%)</td><td>88.93</td><td>89.20</td><td>91.20</td><td>95.86</td><td><u>96.17</u></td><td><strong>99.51</strong></td></tr><tr><td>OA (%)</td><td>91.45</td><td>91.65</td><td>93.18</td><td>96.82</td><td><u>97.07</u></td><td><strong>99.72</strong></td></tr><tr><td>AA (%)</td><td>86.15</td><td>85.19</td><td>92.18</td><td>95.81</td><td>96.12</td><td><strong>99.59</strong></td></tr></tbody></table>

Our AA of 99.59% outperforms both AMHFN (96.12%) and GAHT (95.81%) by 3.47% and 3.78% respectively. Importantly, conventional transformer models, including ViT, PiT, and HiT, show significant drops in performance in this low data range, with OA falling to 91.45%, 91.65%, and 93.18%, respectively, and kappa values falling to 88.93%, 89.20%, and 91.20%. These pronounced drops highlight the inherent limitations of vanilla transformer architectures when faced with label scarcity. In contrast, the consistent top results of our method on both class-based and global metrics set a new benchmark for hyperspectral classification under extremely constrained training scenarios.

Beyond accuracy, we also assess computational efficiency. Appendix A (Table A.1) reports parameter counts, LoRA ratio, and memory usage, confirming that the enhanced LoRA design maintains high accuracy while substantially reducing the footprint of training parameters.

## 5.2. Visual classification analysis

Visual evaluation of classification maps offers crucial insight into the spatial coherence and semantic integrity of hyperspectral models, especially in applications where precise delineation and uniformity within classes are essential. Fig. 6 shows the results of our framework along with four baselines for each dataset. In all cases, our method produces clean, well-structured maps with sharp transitions and minimal misclassification, overcoming the spatial artifacts, boundary blurring, and salt-and-pepper noise of competing approaches.

This visual superiority results from our multi-scale spectral transformer, which is complemented by two architectural innovations: Enhanced LoRA and the activation function SwiGLU. LoRA's projections suppress noise caused by overfitting in data-poor regions, while SwiGLU's gated nonlinearity refines subtle spectral transitions and sharpens object boundaries. In addition, the CLR scheduler stabilizes
<figure><img src="image_10.png" /><figcaption>**Fig. 6.** Comparative classification performance on standard hyperspectral benchmarks: (I) Salinas, (II) WHU-Hi-LongKou, and (III) WHU-Hi-HongHu datasets. Ground truth and method predictions are presented with challenging regions highlighted (red boxes).</figcaption></figure>

training by cyclically adjusting the learning rate, which promotes convergence to flatter minima and provides smoother segmentation results.

### 5.3. Salinas dataset visual performance

The Salinas dataset presents a major classification challenge because the complex agricultural landscape is characterized by spectrally similar vegetable crops, irregular field boundaries, and subtle phenological variations that create ambiguous spectral signatures between adjacent classes. The semi-arid environmental conditions lead to further spectral variability due to soil background disturbance and seasonal changes in illumination, making precise delineation using conventional methods particularly difficult. In the Salinas classification maps (Fig. 6.I), our LoLA-SpecViT (f) retains contiguous crop parcels with uniform interior labeling, whereas the baseline methods show distinct artifacts. In Inset I-1, HybridSN (b) and A²MFE (c) show pronounced salt-and-pepper noise, whereas the LoRA regularization of LoLA-SpecViT reduces these misclassifications by 71% compared to HybridSN. LANet (d) and MADANet (e) produce smoother regions, but blur critical field boundaries, while the SwiGLU activation of LoLA-SpecViT sharpens the spectral transitions at the edges, and achieves an overall accuracy gain of 0.34 compared to MADANet (Table 2).

Our model addresses these inherent dataset complexities through two complementary mechanisms that work synergistically to overcome the classification challenges. Improved LoRA adaptation with gating mechanisms selectively highlights discriminative spectral features while suppressing background soil interference, enabling robust classification even when crop signatures are contaminated by environmental variability. The multiscale transformer backbone with local, window-based attention captures fine-grained spatial-spectral dependencies within individual fields while maintaining global context awareness, ensuring that subtle phenological differences between crops are preserved and not smoothed away during the classification process.

#### 5.3.1. WHU-Hi-LongKou dataset visual performance

In the LongKou maps (Fig. 6.II), our LoLA-SpecViT (f) produces an exceptionally clean delineation of the cotton fields, which are highlighted by the red-boxed inset. Unlike competing methods, which exhibit blurring at the edges and misclassification around cotton parcels, LoLA-SpecViT provides homogeneous interior labeling with sharp, uninterrupted contours that faithfully follow the ground-truth field shapes. Hybrid-ViT (b) and Hir-Transformer (c), for example, introduce speckle noise along the field edges, while SSMamba (d) and MorpMamba (e) occasionally break up contiguous cotton regions into
isolated patches. In contrast, the Enhanced LoRA of LoLA-SpecViT suppresses unwanted low-confidence mappings, and the gated nonlinearity of SwiGLU accentuates subtle spectral transitions at the crop edges, resulting in a clear, artifact-free segmentation.

This superior visual coherence is confirmed by our class-related metrics on the 10% LongKou test split: For the “cotton” class, LoLA-SpecViT achieves 99.85% precision, 100.00% recall, and 99.93% F1 score (Table 4). These values reflect near-perfect internal consistency and boundary accuracy, with virtually no pixels mismatched between cotton and neighboring classes. By combining low-rank regularization to filter out noise with gated activation to sharpen spectral contrasts, our framework maintains contiguous, cotton plots with high accuracy, even under sparse supervision.

### 5.3.2. WHU-Hi-HongHu dataset visual performance

In the HongHu maps (Fig. 6.III), the red-boxed inset contains a dense mosaic of small, spectrally similar parcels that make fine-grained differentiation difficult: HybridSN (b) creates a pervasive salt-and-pepper noise, that falsely identifies bare soil and trees as neighboring crops; LSGA (c) smooths the boundaries, too much by mixing Chinese cabbage with cabbage and blurring the edges between foil-covered and romaine lettuce; DBCT (d) confuses rapeseed and pakchoi strips, and places narrow field segments into incorrect classes; and MassFormer (e) preserves the overall shape of the plots but introduces staircase artifacts and fails to suppress false positives in homogeneous regions. In contrast, our LoLA-SpecViT (f) provides a clean, noise-free segmentation that correctly separates film-covered lettuce from Romaine lettuce and preserves the thin road filament without misclassification. This visual fidelity is driven by LoRA regularization that mitigates overfit artifacts in small data sets and the CLR scheduler that drives convergence to flatter minima for stable activations of features, resulting in fewer than 1% pixels in this segment deviating from ground truth, consistent with the lowest error rates per class in Table 1.

### 5.3.3. Aggregate visual performance across datasets

Taken together, these classification maps (Fig. 6) demonstrate the robustness of our method across different landscapes and labeling regimes. We observe > 95% contiguous intraclass regions with less than 0.5% speckle noise in all three datasets, directly reflecting the quantitative gains shown in the Tables 2–4. The combination of the regularization of LoRA, the nonlinear improvement of SwiGLU, and the optimization stability of CLR leads not only to high numerical accuracy but also to interpretable, semantically coherent results, which are crucial for precision agriculture, environmental monitoring, and other remote sensing applications with sparse surveillance.

### 5.4. Ablation study

To systematically evaluate the contribution of each architectural component within our proposed framework, we perform comprehensive ablation studies for three hyperspectral reference datasets. This analysis isolates the effects of: (i) multi-scale transformer depth, (ii) parameter-efficient fine-tuning via Low-Rank Adaptation (LoRA), (iii) activation function selection, and (iv) cyclic learning rate scheduling. The results, presented in Table 7, provide quantitative insights into the synergistic effects of these design decisions in scenarios with limited supervision.

The ablation results in Table 7 provide consistent evidence for the effects of individual design choices across the Longkou, Salinas, and HongHu datasets. The baseline configuration, which uses the original four-stage GCViT with GELU activation but without parameter-efficient fine-tuning or scheduling, performs poorly on all benchmarks (OA of 90.83% on Longkou, 66.37% on Salinas, and 56.88% on HongHu). These results confirm that training a deep transformer from scratch under limited supervision leads to pronounced overfitting and unstable convergence, especially on datasets with high intra-class variability.

While the four-stage architecture is theoretically very informative, it leads to an excessive number of parameters that cannot be effectively optimized with only 10% of the available labels, resulting in the model memorizing training patterns instead of learning generalizable spectral-spatial representations. This overfitting phenomenon is most severe in HongHu, as the multi-stage transformer architecture suffers from inefficient gradient flow and is unable to properly capture discriminative spectral-spatial features due to the complex class distributions and limited supervision.

Simplifying the architecture to a three-stage GCViT with only local attention leads to moderate improvements in all datasets. For example, OA increases from 66.37% to 66.72% for Salinas and from 56.88% to 66.72% for HongHu, while Longkou improves from 90.83% to 94.05%. This reduction in architectural depth mitigates overfitting by reducing the total number of parameters and improving the gradient flow throughout the network, thereby reducing the model's susceptibility to memorizing the small training set. However, the truncated architecture with only local attention mechanisms loses some of its ability to capture long-range spectral dependencies and deep spatial-spectral correlations, as evidenced by stagnant or slightly reduced AA values. The local attention constraint limits the receptive field of the model, resulting in uneven performance per class and a reduced ability to represent fine-grained variations across certain land cover categories, especially those with subtle spectral differences.

The integration of LoRA-based, parameter-efficient fine-tuning delivers the most significant performance gains across all experimental conditions. With LoRA, the truncated architecture achieves near-perfect classification in all datasets: for Longkou, OA achieves 99.91%, AA 99.76%, and Kappa 99.89%; for Salinas, OA 99.76%, AA 99.64%, and Kappa 99.42%; and for HongHu, OA 99.65%, AA 99.42%, and Kappa 99.11%. This dramatic jump in performance demonstrates that LoRA effectively uses pre-trained weights to adapt to dataset-specific distributions while maintaining generalization in the presence of severe label scarcity, addressing the fundamental limitations of training from scratch. By only updating low-rank attention projections and not the entire parameter space, LoRA acts as an implicit regularization mechanism that prevents overfitting while maintaining a stable gradient flow throughout the network. The parameter-efficient nature of LoRA allows the model to use already trained weights from the large-scale pre-training and fine-tune them to the specific spectral properties of each hyperspectral dataset with minimal additional parameters, resulting in a robust fit without compromising the learnt representations.

Further refinements are achieved by incorporating SwiGLU activation and cyclic learning rate scheduling. The final configuration achieves the best overall results on Salinas (OA 99.87%, AA 99.70%, Kappa 99.49) and HongHu (OA 99.81%, AA 99.53%, Kappa 99.29), while the performance on Longkou remains competitive. SwiGLU, a gated activation unit, increases nonlinearity and improves the sensitivity of the model to subtle spectral variations by increasing the expressive power of the network, which is particularly beneficial when modeling complex spectral sequences with subtle variance between classes. The gating mechanism in SwiGLU allows more selective activation patterns, allowing the model to focus on the most discriminative spectral features while suppressing irrelevant information. Meanwhile, CLR contributes to more robust convergence dynamics by allowing the learning rate to oscillate periodically, which facilitates escape from shallow local minima and accelerates convergence. The cyclic adjustment of the learning rate helps the optimizer to avoid sub-optimal solutions and converge more effectively, which is especially beneficial when optimizing deep transformers under sparse supervision, where the loss landscape tends to be more complex and has multiple local minima. Nonetheless, the improvements introduced by these components remain incremental compared to the substantial gains achieved by LoRA, suggesting that although optimization refinements are valuable,
<table><caption>**Table 7**
Ablation study results across multiple datasets.</caption><thead><tr><th>Dataset</th><th>Model</th><th>4-stage GCViT</th><th>3-stage GCViT (Local)</th><th>LoRA (PEFT)</th><th>CLR Scheduler</th><th>GELU</th><th>SwiGlu</th><th>OA (%)</th><th>AA (%)</th><th>Kappa</th></tr></thead><tbody><tr><td rowspan="4">Longkou</td><td>1</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>90.83</td><td>95.25</td><td>88.28</td></tr><tr><td>2</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>94.05</td><td>95.26</td><td>92.27</td></tr><tr><td>3</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td><strong>99.91</strong></td><td><strong>99.76</strong></td><td><strong>99.89</strong></td></tr><tr><td>4</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td><strong>99.91</strong></td><td><strong>99.71</strong></td><td><strong>99.88</strong></td></tr><tr><td rowspan="4">Salinas</td><td>1</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>66.37</td><td>72.00</td><td>62.82</td></tr><tr><td>2</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>66.72</td><td>70.00</td><td>63.30</td></tr><tr><td>3</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td><strong>99.76</strong></td><td><strong>99.64</strong></td><td><strong>99.42</strong></td></tr><tr><td>4</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td><strong>99.87</strong></td><td><strong>99.70</strong></td><td><strong>99.49</strong></td></tr><tr><td rowspan="4">Honghu</td><td>1</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>56.88</td><td>78.33</td><td>53.56</td></tr><tr><td>2</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>66.72</td><td>70.73</td><td>63.30</td></tr><tr><td>3</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td><strong>99.65</strong></td><td><strong>99.42</strong></td><td><strong>99.11</strong></td></tr><tr><td>4</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td><strong>99.81</strong></td><td><strong>99.53</strong></td><td><strong>99.29</strong></td></tr></tbody></table>

<table><caption>**Table 8**
Performance validation on QUH-Qingyun dataset under different training sample ratios (10% follows original dataset paper configuration, 2% demonstrates enhanced data efficiency).</caption><thead><tr><th rowspan="2">Method</th><th colspan="3">2% Training</th><th colspan="3">10% Training</th></tr><tr><th>OA</th><th>AA</th><th>Kappa</th><th>OA</th><th>AA</th><th>Kappa</th></tr></thead><tbody><tr><td>Standard GCViT</td><td>88.47</td><td>87.23</td><td>84.66</td><td>93.58</td><td>92.43</td><td>90.74</td></tr><tr><td>GCViT + Basic LoRA</td><td>81.70</td><td>79.45</td><td>80.18</td><td>92.10</td><td>90.68</td><td>91.54</td></tr><tr><td>LoLA-SpecViT</td><td>98.12</td><td>97.89</td><td>97.92</td><td>99.20</td><td>99.12</td><td>99.08</td></tr></tbody></table>

the fundamental challenge lies in effective parameter adaptation rather than activation or scheduling strategies.

The ablation analysis establishes LoRA as the dominant factor driving the remarkable improvements in classification accuracy, with its ability to enable parameter-efficient adaptation without overfitting serving as a key breakthrough for hyperspectral classification in label scarcity. SwiGLU and CLR provide complementary refinements that optimize the training dynamics and improve the expressiveness of the model, but their contributions are secondary to the foundational role of LoRA. Dataset-specific results also show that Longkou, characterized by relatively homogeneous agricultural classes with distinct spectral signatures, achieves near-saturation performance even with simplified configurations, while Salinas and HongHu, with higher intra-class variability and more complex spectral-spatial patterns, benefit most from the fully integrated framework combining all architectural innovations.

### 5.4.1. Generalization and robustness evaluation

To evaluate the robustness of our framework beyond agricultural scenarios, we perform experiments on the QUH-Qingyun dataset following the standard 10% training configuration established in the literature, while showing improved robustness under the more demanding 2% training regime. This evaluation provides a rigorous test of the generalizability of our method in various hyperspectral classification scenarios with different data availability constraints.

We compare three configurations to demonstrate the progressive benefits of our design components: (1) standard GCViT baseline (Hatamizadeh et al., 2023), (2) GCViT with basic LoRA adaptation, and (3) our complete LoLA-SpecViT framework. The experimental results for both training configurations are shown in Table 8.

The results show the superior performance of our system in both training configurations. Under the standard training regime of 10%, LoLA-SpecViT achieves 99.20% OA, reaching a competitive performance comparable to existing methods. More significantly, under the challenging 2% training scenario, our method maintains robust performance with 98.12% OA, demonstrating exceptional data efficiency, which is particularly valuable for practical hyperspectral applications with limited labeled data.

The deterioration in performance observed with basic LoRA adaptation (81.70% OA with 2% training) underlines the importance of

our integrated design approach. This counterintuitive result shows that parameter-efficient adaptation requires careful integration with domain-specific processing components, as LoRA's constraints alone are not sufficient to capture the full complexity of challenging hyperspectral scenarios without supporting architectural elements. The consistent superior performance of our complete LoLA-SpecViT framework across both training regimes confirms that the integrated design combining enhanced spectral processing, a multiscale transformer backbone, and adaptive attention mechanisms works synergistically to maintain robust classification performance regardless of data availability constraints.

## 6. Conclusion

This work presents LoLA-SpecViT, a parameter-efficient spectral transformer architecture that addresses critical challenges in the classification of hyperspectral images under large data scarcity. By systematically integrating advanced Low-Rank Adaptation with multiscale spectral processing, our method achieves consistent improvements of up to 0.5% in overall accuracy, average accuracy, and Cohen's kappa across four benchmark datasets (WHU-Hi LongKou, WHU-Hi HongHu, Salinas, and QUH-Qingyun) while using only 2%–10% of the available labeled samples.

Comprehensive experimental validations confirm the synergistic effectiveness of our architectural innovations. LoRA adaptation proves particularly crucial in mitigating overfitting phenomena under extreme label scarcity, while comprehensive ablation studies demonstrate the individual and combined contributions of each component. Qualitative analyses show that LoLA-SpecViT produces well-defined classification maps with sharp inter-class boundaries, homogeneous intra-class regions, and effective noise reduction, which directly support its use in precision agriculture, environmental monitoring, and resource management.

Our investigation yields several critical design considerations for parameter-efficient hyperspectral classification architectures. The enhanced LoRA formulation with gating and residual connections significantly increases model expressiveness, though optimal performance requires conservative parameter initialization to ensure training stability. The standalone architectural design deliberately prioritizes computational efficiency and deployment simplicity, making it well-suited
<caption>**Table A.1**</caption>

Efficiency analysis of the Enhanced LoRA GCViT model architecture across datasets and training splits. We report total/trainable parameters, parameter reduction, memory usage, and wall-clock times.

<table><thead><tr><th>Dataset (train %)</th><th>Total Params (M)</th><th>Trainable Params (M)</th><th>Param Reduction. (%)</th><th>Memory Usage (MB)</th><th>Training Time</th><th>Eval. Time</th></tr></thead><tbody><tr><td>Honghu (10%)</td><td>254.63</td><td>73.59</td><td>71.10</td><td>699.37</td><td>1 m 48 s</td><td>4 m 58 s</td></tr><tr><td>Salinas (10%)</td><td>254.63</td><td>73.59</td><td>71.10</td><td>699.37</td><td>3 m 11 s</td><td>10 m 21 s</td></tr><tr><td>Longkou (10%)</td><td>254.63</td><td>73.59</td><td>71.10</td><td>699.37</td><td>2 m 10 s</td><td>1 m 06 s</td></tr><tr><td>Longkou (2%)</td><td>254.63</td><td>73.59</td><td>71.10</td><td>699.37</td><td>2 m 10 s</td><td>0 m 40 s</td></tr><tr><td>Qingyun (2%)</td><td>254.63</td><td>73.59</td><td>71.10</td><td>699.37</td><td>10 m 25 s</td><td>1 h 15 m 00 s</td></tr></tbody></table>

for resource-constrained operational environments. When leveraging pretrained visual backbones for LoRA initialization, the approach maintains strict parameter efficiency while incorporating complementary visual knowledge without costly parameter unfreezing. However, the 3D spectral convolution followed by spatial averaging, while computationally efficient, introduces potential information loss that may impact performance on tasks requiring fine-grained spectral discrimination. These findings establish principled guidelines for robust implementation and reproducible training protocols.

Although our experimental evaluation focuses on agricultural VNIR hyperspectral images, the underlying architectural principles show broad applicability in different spectral domains and sensing modalities. Future research will explore domain-adaptive pretraining strategies for large-scale HSI corpora, the development of highly efficient spectral embedding mechanisms, the investigation of prompt-based tuning paradigms for improved cross-domain generalization, and the exploration of advanced model compression techniques to enable real-time inference on edge computing platforms.

## CRediT authorship contribution statement

**Fadi Abdeladhim Zidi:** Writing – original draft, Software. **Djamel Eddine Boukhari:** Writing – original draft, Methodology, Investigation. **Abdellah Zakaria Sellam:** Investigation, Data curation. **Abdelkrim Ouafi:** Validation, Supervision. **Cosimo Distante:** Writing – review & editing, Supervision. **Salah Eddine Bekhouche:** Writing – original draft, Data curation. **Abdelmalik Taleb-Ahmed:** Supervision.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgments

The authors thank Mr. Arturo Argentieri from CNR-ISASI, Italy, for his technical contribution to the multi-GPU computing facilities. This research was partially funded by the Italian Ministry of University and Research (MUR) with the project “Future Artificial Intelligence Research – FAIR” Grant number PE0000013 CUP B53C22003630006.

## Appendix A. Runtime and efficiency metrics

To complement the main accuracy tables, we report practical efficiency indicators—parameter counts, reduction ratio, memory footprint, and wall-clock training/evaluation time—measured under the experimental setup described in Section 5. In addition to these metrics, we further analyze in Appendix D (Table D.1) how alternative LoRA configurations influence parameter reduction, where the adopted setting reflects a balance between maximizing accuracy and preserving substantial efficiency gains.

<caption>**Table B.1**</caption>

Parameter counts of representative models, comparing compact CNNs, large-scale transformers, hyperspectral foundation models, and the proposed LoLA-SpecViT, which achieves a balanced trade-off between lightweight and very large architectures.

<table><caption>**Table C.1**</caption><thead><tr><th>Model</th><th>Dataset/Setting</th><th>Params (M)</th></tr></thead><tbody><tr><td>PiT-B (Heo et al., 2021)</td><td>Longkou (2%)</td><td>73.8</td></tr><tr><td>ViT-Huge (Dosovitskiy et al., 2021)</td><td>Longkou (2%)</td><td>632.0</td></tr><tr><td>LANet (Ding et al., 2021)</td><td>Salinas</td><td>5.3</td></tr><tr><td>Madanet (Cui et al., 2023)</td><td>Salinas</td><td>0.16</td></tr><tr><td>SpectralGPT (FFT) (Ligan et al., 2025)</td><td>Longkou</td><td>237.0</td></tr><tr><td>HyperSIGMA (Wang et al., 2025)</td><td>Longkou</td><td>182.6</td></tr><tr><td>LoLA-SpecViT (ours)</td><td>All datasets</td><td>73.59</td></tr></tbody></table>

Ablation study on the *LongKou* and *Salinas* datasets (2% training samples) showing the effect of progressively integrating LoRA across backbone components.

<table><thead><tr><th>Configuration</th><th>(Longkou) Acc. (%)</th><th>(Salinas) Acc. (%)</th></tr></thead><tbody><tr><td>Spectral Only</td><td>99.37</td><td>98.70</td></tr><tr><td>+ Attention</td><td>99.55</td><td>98.49</td></tr><tr><td>+ MLPs</td><td>99.58</td><td>98.57</td></tr><tr><td>All Components</td><td><strong>99.72</strong></td><td><strong>99.15</strong></td></tr></tbody></table>

## Appendix B. Model size comparison

While runtime and memory usage capture practical efficiency, parameter count provides a complementary view of model scale. Table B.1 situates LoLA-SpecViT among representative baselines, ranging from lightweight designs with only thousands of parameters to very large transformer models with hundreds of millions. This comparison highlights that our model occupies a balanced middle ground: significantly smaller than extremely large transformers such as ViT-Huge, yet more expressive than compact architectures like Madanet or LANet.

## Appendix C. Strategic lora integration ablation

We conducted an additional ablation study to isolate the effect of progressively introducing LoRA into different components of the backbone. The experiments were performed on the *LongKou* and *Salinas* datasets with 2% training samples. As shown in Table C.1, accuracy improves as LoRA modules are added to attention and MLP layers, with the best performance achieved when all components are equipped with LoRA.

## Appendix D. Configuration for parameter reduction

In order to complement the efficiency analysis, we report in Table D.1 the effect of varying LoRA rank and scaling parameters on reduction ratio, proportion of LoRA parameters, and memory footprint. This comparison highlights the trade-off between efficiency and accuracy: although lower-rank settings yield marginally higher parameter reduction and smaller memory usage, they consistently degrade classification performance. Consequently, the adopted configuration (Rank = 16, $\alpha$ = 32) reflects a principled balance between accuracy preservation and computational efficiency, which we consider critical for reliable deployment in hyperspectral image analysis.
<caption>**Table D.1**</caption>

Summary of LoRA configurations for parameter reduction. We report the impact of different rank and scaling settings on reduction ratio, proportion of LoRA parameters, and memory footprint.

<table><caption>**Table E.1**</caption><thead><tr><th>Configuration</th><th>Rank</th><th>Alpha</th><th>Reduction (%)</th><th>LoRA (%)</th><th>Trainable memory (MB)</th></tr></thead><tbody><tr><td>Original (High Rank)</td><td>16</td><td>32</td><td>71.10</td><td>75.91</td><td>280.71</td></tr><tr><td>Reduced Rank</td><td>8</td><td>16</td><td>71.42</td><td>75.58</td><td>276.62</td></tr><tr><td>Low Rank</td><td>4</td><td>8</td><td>71.59</td><td>75.41</td><td>274.58</td></tr><tr><td>Ultra Low Rank</td><td>2</td><td>4</td><td>71.68</td><td>75.32</td><td>273.55</td></tr><tr><td>Minimal LoRA</td><td>1</td><td>2</td><td>71.72</td><td>75.28</td><td>273.04</td></tr></tbody></table>

Details of the pre-trained nvidia/Global-Context-Vision-Transformer-Tiny model used in our framework. Hatamizadeh et al. (2023).

<table><caption>**Table F.1**</caption><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td>Source</td><td>NVIDIA NGC/Hugging Face Hub (nvidia/GCViT-Tiny)</td></tr><tr><td>Architecture</td><td>Global Context Vision Transformer (Tiny variant)</td></tr><tr><td>Parameters</td><td>~28M trainable parameters</td></tr><tr><td>Input Resolution</td><td>224 × 224</td></tr><tr><td>Pre-training Dataset</td><td>ImageNet-1K</td></tr><tr><td>Strengths</td><td>Captures local &amp; global dependencies with reduced FLOPs</td></tr><tr><td>Integration</td><td>Backbone initialized from checkpoint, final head replaced; LoRA adapters + BandDropout integrated for fine-tuning</td></tr></tbody></table>

Effect of adding residual and gating mechanisms to LoRA adapters, evaluated on LongKou and Salinas. The combined Gate+Residual variant consistently achieves the best accuracy across metrics.

<table><thead><tr><th>Dataset</th><th>Base LoRA</th><th>+ Residual</th><th>+ Gate</th><th>+ Gate+Residual</th></tr></thead><tbody><tr><td>LongKou (2%)</td><td>98.66</td><td>98.24</td><td>99.18</td><td><b>99.72</b></td></tr><tr><td>Salinas (2%)</td><td>98.86</td><td>97.04</td><td>98.83</td><td><b>99.15</b></td></tr></tbody></table>

### D.1. Parameter reduction calculation

The reduction ratio is computed as:

$$ \rho = \frac{P_{\text{total}} - P_{\text{trainable}}}{P_{\text{total}}} \times 100\%, \qquad (\text{D.1}) $$

where $P_{\text{total}}$ denotes the total number of parameters in the backbone and $P_{\text{trainable}}$ the parameters updated during fine-tuning.

For example, full fine-tuning requires updating all $P_{\text{total}} = 254.63M$ parameters, whereas our LoRA configuration updates only $P_{\text{trainable}} = 73.59M$ parameters. This corresponds to

$$ \rho = \frac{254.63 - 73.59}{254.63} \approx 71.10\%, $$

which confirms that LoRA reduces trainable parameters by more than 70% while maintaining competitive performance.

## Appendix E. Pre-trained model details

To enhance clarity and ensure reproducibility, we document the details of the pre-trained backbone nvidia/Global-Context-Vision-Transformer-Tiny. Specifically, we provide its source, architectural specifications, and the integration workflow within our framework. A comprehensive summary of this information is presented in (Table E.1).

## Appendix F. Ablation of gate and residual in enhanced lora

We evaluated the effect of adding a residual branch and a sigmoid gate to the LoRA adapters. Four variants were tested under the same backbone and training setup: Base LoRA, Residual only, Gate only, and Gate+Residual. Table F.1 reports the test performance in terms of OA, AA, and Kappa.

The results show that the gate alone improves stability, while the residual alone gives mixed effects. Their combination consistently

delivers the best accuracy with only a modest increase in parameters and memory (Table F.2).

## Appendix G. Architecture complexity

### G.1. Enhanced computational complexity analysis

The enhanced LoLA-SpecViT architecture introduces several computational optimizations through advanced LoRA integration and adaptive mechanisms:

where $B$ is batch size, $L$ is number of levels, $N$ is number of tokens, $M$ is window size, $d$ is feature dimension, $r$ is LoRA rank, and $k_i$ represents different kernel sizes in spectral processing. The enhanced complexity accounts for the additional gating and residual mechanisms in the advanced LoRA implementation (see Table G.3).

### G.2. SwiGLU activation function integration

The enhanced architecture incorporates SwiGLU (Swish-Gated Linear Unit) activation functions in one strategic location:

$$ \text{SwiGLU}(x) = \text{Swish}(xW_1 + b_1) \odot (xW_2 + b_2) \qquad (\text{G.1}) $$

where $\text{Swish}(x) = x \cdot \sigma(x)$ is the Swish activation function, and $\odot$ denotes element-wise multiplication. SwiGLU is applied in:

* **Feed-Forward Networks:** In the MLP blocks of GCViT transformer layers

The SwiGLU activation provides better gradient flow and has been shown to outperform traditional activation functions in transformer architectures, contributing to the enhanced performance of our hyperspectral classification model.

### G.3. Enhanced parameter efficiency analysis

The enhanced parameter efficiency with selective LoRA application and advanced gating mechanisms:

$$ P_{\text{Enhanced LoRA}} = \sum_{i \in \mathcal{L}_{\text{LoRA}}} \left[ r \cdot (d_{\text{in},i} + d_{\text{out},i}) + d_{\text{in},i} \cdot d_{\text{out},i} \cdot \mathbb{I}_{\text{gate}} + d_{\text{in},i} \cdot d_{\text{out},i} \cdot \mathbb{I}_{\text{residual}} \right] \qquad (\text{G.2}) $$

where $\mathcal{L}_{\text{LoRA}}$ denotes the set of layers with LoRA adaptation, $\mathbb{I}_{\text{gate}}$ and $\mathbb{I}_{\text{residual}}$ are indicator functions for gating and residual mechanisms respectively.

The enhanced parameter reduction ratio becomes:

$$ \rho_{\text{enhanced}} = \frac{P_{\text{Enhanced LoRA}}}{P_{\text{full}}} = \frac{2r \cdot |\mathcal{L}_{\text{LoRA}}| \cdot d + 2d^2 \cdot |\mathcal{L}_{\text{gate+residual}}|}{d^2 \cdot N_{\text{layers}}} \ll 1 \qquad (\text{G.3}) $$

For our enhanced configuration with $r = 16$, $d = 96$, $|\mathcal{L}_{\text{LoRA}}| = 26$, $|\mathcal{L}_{\text{gate+residual}}| = 26$, and $N_{\text{layers}} = 26$, $\rho_{\text{enhanced}} \approx 0.67$, achieving a 33% parameter reduction while maintaining enhanced model capacity through gating and residual mechanisms.
<caption>**Table F.2**</caption>

Computational overhead of LoRA enhancements in terms of parameter count, trainable parameters, and memory usage. The Gate+Residual design yields the best performance with only a modest increase in complexity.

<table><caption>**Table G.3**</caption><thead><tr><th>Variant</th><th>Total Params (M)</th><th>Trainable (M)</th><th>Memory usage (MB)</th><th>Notes</th></tr></thead><tbody><tr><td>Base LoRA</td><td>254.00</td><td>73.10</td><td>620.00</td><td>Lowest footprint</td></tr><tr><td>Residual only</td><td>254.40</td><td>73.30</td><td>622.00</td><td>Slight increase</td></tr><tr><td>Gate only</td><td>254.50</td><td>73.40</td><td>624.00</td><td>Slight increase</td></tr><tr><td>Gate + Residual</td><td>254.63</td><td>73.59</td><td>699.37</td><td>Best performance</td></tr></tbody></table>

Enhanced computational complexity analysis with advanced LoRA.

<table><caption>**Table G.4**</caption><thead><tr><th>Component</th><th>Time Complexity</th><th>Space Complexity</th></tr></thead><tbody><tr><td>Enhanced Spectral Conv3D</td><td>O(BCHW · Σ<sub>i</sub> k<sub>i</sub><sup>3</sup>)</td><td>O(BdHW)</td></tr><tr><td>Adaptive Spectral Attention</td><td>O(BCHW)</td><td>O(BCHW)</td></tr><tr><td>Enhanced Band Dropout</td><td>O(BCHW)</td><td>O(BCHW)</td></tr><tr><td>PEFT Window Attention</td><td>O(N · M<sup>2</sup> · d)</td><td>O(M<sup>2</sup> · d)</td></tr><tr><td>Enhanced LoRA Adaptation</td><td>O(d · r + d<sup>2</sup>)</td><td>O(d · r + d<sup>2</sup>)</td></tr><tr><td>Adaptive Squeeze-Excitation</td><td>O(BCHW)</td><td>O(BCHW)</td></tr><tr><td>Gate-Residual LoRA</td><td>O(d · r + d<sup>2</sup>)</td><td>O(d · r + d<sup>2</sup>)</td></tr><tr><td>Overall Enhanced Model</td><td>O(B · L · N · M<sup>2</sup> · d + d<sup>2</sup>)</td><td>O(BdHW + d<sup>2</sup>)</td></tr></tbody></table>

Configuration of LoRA adaptation hyperparameters on the WHU-LongKou dataset.

<table><thead><tr><th>Configuration</th><th>α</th><th>β, γ</th><th>OA (%)</th><th>AA (%)</th><th>Kappa (%)</th></tr></thead><tbody><tr><td>alpha_low</td><td>8</td><td>Learnable</td><td>99.77</td><td>99.29</td><td>99.70</td></tr><tr><td>alpha_high</td><td>64</td><td>Learnable</td><td>99.72</td><td>99.11</td><td>99.64</td></tr><tr><td>fixed Descent</td><td>32</td><td>Fixed (1, 0)</td><td>99.75</td><td>99.34</td><td>99.67</td></tr><tr><td>optimal</td><td>32</td><td>Learnable</td><td>99.91</td><td>99.71</td><td>99.88</td></tr></tbody></table>

### G.4. Advanced LoRA with cyclical learning rate

The enhanced cyclical scaling mechanism for LoRA parameters with triangular2 mode:

$$ \text{cycle} = \left[ 1 + \frac{t}{\text{step_size_up} + \text{step_size_down}} \right] \qquad (\text{G.4}) $$

$$ x = t \mod (\text{step_size_up} + \text{step_size_down}) \qquad (\text{G.5}) $$

$$ \text{scale} = \begin{cases} \frac{x}{\text{step_size_up}} & \text{if } x < \text{step_size_up} \\ 1 - \frac{x-\text{step_size_up}}{\text{step_size_down}} & \text{otherwise} \end{cases} \qquad (\text{G.6}) $$

$$ \gamma_t = \text{base\_lr} + (\text{max\_lr} - \text{base\_lr}) \cdot \text{scale} \cdot 2^{-(\text{cycle}-1)} \qquad (\text{G.7}) $$

where *t* is the current iteration, and the triangular2 mode reduces the amplitude by half each cycle, providing enhanced convergence stability.

### G.5. Configuration study of LoRA adaptation hyperparameters

See Table G.4.

## Appendix H. Algorithms

H.1. Enhanced PEFT-GCViT forward pass with advanced LoRA

H.2. Enhanced LoRA with advanced gating and residual mechanisms

H.3. Enhanced GCViT block with comprehensive PEFT integration

H.4. Enhanced spectral processing with adaptive attention

H.5. Enhanced cyclical learning rate scheduler for LoRA

### Algorithm 1 Enhanced PEFT-GCViT Forward Pass with Advanced LoRA Integration

Require: Hyperspectral input $X \in \mathbb{R}^{B \times C \times H \times W}$

Ensure: Class predictions $Y \in \mathbb{R}^{B \times K}$

1. **Phase 1: Enhanced Spectral Feature Extraction**

2. $X' \leftarrow \text{Unsqueeze}(X, \text{dim} = 1)$ ▷ $\mathbb{R}^{B \times 1 \times C \times H \times W}$

3. $X' \leftarrow \text{Conv3D}_{7 \times 3 \times 3}(X') \circ \text{BN3D} \circ \sigma_{\text{swish}}(X')$ ▷ $1 \to 32$ channels

4. $X' \leftarrow \text{Conv3D}_{5 \times 3 \times 3}(X') \circ \text{BN3D} \circ \sigma_{\text{swish}}(X')$ ▷ $32 \to 64$ channels

5. $X' \leftarrow \text{Conv3D}_{3 \times 3 \times 3}(X') \circ \text{BN3D} \circ \sigma_{\text{swish}}(X')$ ▷ $64 \to \text{dim}$ channels

6. $X' \leftarrow \text{EnhancedBandDropout}(X', p = 0.1)$ ▷ Advanced spectral regularization

7. $F_{\text{spec}} \leftarrow \text{AdaptiveSpectralAttention}(X')$ ▷ Adaptive channel-wise attention

8. $F_{\text{spec}} \leftarrow \text{Mean}(F_{\text{spec}}, \text{dim} = 2)$ ▷ Spectral dimension pooling

9. **Phase 2: Enhanced Patch Embedding & Positional Encoding**

10. $P \leftarrow \text{EnhancedPatchEmbed}(F_{\text{spec}})$ ▷ $\mathbb{R}^{B \times \frac{H}{2} \times \frac{W}{2} \times d}$

11. $P \leftarrow P + E_{\text{pos}}$ ▷ Add learnable positional encoding

12. $P \leftarrow \text{Dropout}(P, p = 0.1)$ ▷ Positional dropout

13. **Phase 3: Enhanced Multi-Scale GCViT Processing**

14. **for** $\ell = 1$ to $L$ **do** ▷ $L = 3$ Multi-Scale levels

15. **for** $b = 1$ to $D_{\ell}$ **do** ▷ $D_{\ell}$ blocks per level $\ell$

16. $P \leftarrow \text{EnhancedPEFTGCViTBlock}_{\ell, b}(P)$

17. **end for**

18. **if** $\ell < L$ **then**

19. $P \leftarrow \text{EnhancedReduceSize}_{\ell}(P)$ ▷ Spatial downsampling with Swish activation and adaptive SE

20. **end if**

21. **end for**

22. **Phase 4: Enhanced Classification Head**

23. $F \leftarrow \text{LayerNorm}(\text{Flatten}(P))$

24. $F \leftarrow \text{GlobalAvgPool}(F)$ ▷ Global feature aggregation

25. $Y \leftarrow \text{EnhancedLoRALinear}(F)$ ▷ Advanced LoRA-enhanced classifier

26. **return** $Y$
**Algorithm 2 Enhanced LoRA Linear Transformation with Gating and Residual**

Require: Input features $x \in \mathbb{R}^d$, base weights $W \in \mathbb{R}^{d_{\text{base}} \times d}$

Require: LoRA matrices $\mathbf{A} \in \mathbb{R}^{r \times d}$, $\mathbf{B} \in \mathbb{R}^{d_{\text{base}} \times r}$

Require: Gating weights $\mathbf{G} \in \mathbb{R}^{d_{\text{gate}} \times d}$, residual weights $\mathbf{R} \in \mathbb{R}^{d_{\text{gate}} \times d}$

Require: Scaling parameters $\alpha, r$, cyclical factor $\gamma_t$

Require: Dropout rate $p_{\text{lora}}$, enable\_gate\_residual flag

Ensure: Enhanced output $y \in \mathbb{R}^{d_{\text{out}}}$

1: $y_{\text{base}} \leftarrow Wx \quad \triangleright \text{Base transformation (frozen)}$

2: $h \leftarrow \text{Dropout}(\mathbf{Ax}, p = p_{\text{lora}}) \quad \triangleright \text{LoRA down-projection with dropout}$

3: $y_{\text{lora}} \leftarrow \mathbf{B}h \quad \triangleright \text{LoRA up-projection}$

4: $s_t \leftarrow \frac{\alpha}{r} \cdot \gamma_t \quad \triangleright \text{Cyclical scaling factor}$

5: **if** enable\_gate\_residual **then**

6: $g \leftarrow \sigma(Gx) \quad \triangleright \text{Gating mechanism}$

7: $r \leftarrow \mathbf{R}x \quad \triangleright \text{Residual connection}$

8: $y \leftarrow y_{\text{base}} + g \odot (s_t \cdot y_{\text{lora}}) + \mathbf{r} \quad \triangleright \text{Enhanced combination}$

9: **else**
10: $y \leftarrow y_{\text{base}} + s_t \cdot y_{\text{lora}} \quad \triangleright \text{Standard LoRA behavior}$

11: **end if**

12: **return** y

**Algorithm 3 Enhanced PEFT-GCViT Block Processing with Advanced LoRA**

Require: Input tokens $X \in \mathbb{R}^{B \times H \times W \times d}$

Require: Window size $M$, number of heads $h$, LoRA rank $r$, LoRA alpha $\alpha$

Ensure: Output tokens $X' \in \mathbb{R}^{B \times H \times W \times d}$

1: Enhanced Multi-Head Window Attention with Advanced LoRA:

2: $X_{\text{norm}} \leftarrow \text{LayerNorm}(X)$
3: $W \leftarrow \text{WindowPartition}(X_{\text{norm}}, M) \quad \triangleright \mathbb{R}^{N_{\text{norm}} \times M^2 \times d}$

4: $\mathbf{QKV} \leftarrow \text{EnhancedLoRALinear}(W, r, \alpha, \text{enable\_gate\_residual}=\text{True}) \quad \triangleright$
Advanced LoRA QKV projection

5: $\mathbf{Q}, \mathbf{K}, \mathbf{V} \leftarrow \text{Split}(\mathbf{QKV}, 3) \quad \triangleright \text{Separate Q, K, V matrices}$

6: **for** head $i=1$ to $h$ **do**

7: $A_i \leftarrow \text{Softmax} \left( \frac{Q_i K_i^T}{\sqrt{d_k}} + \mathbf{B}_{\text{rel}} \right)$

8: $O_i \leftarrow A_i V_i$

9: **end for**

10: $O \leftarrow \text{Concat}(O_1, \dots, O_h)$

11: $W' \leftarrow \text{EnhancedLoRALinear}(O, r, \alpha, \text{enable\_gate\_residual}=\text{True}) \quad \triangleright$
Advanced LoRA output projection

12: $X_{\text{attn}} \leftarrow \text{WindowReverse}(W', H, W)$

13: $X_{\text{res1}} \leftarrow X + \text{DropPath}(X_{\text{attn}})$

14: Enhanced SwiGLU Feed-Forward Network:

15: $X_{\text{norm2}} \leftarrow \text{LayerNorm}(X_{\text{res1}})$

16: $G_1 \leftarrow \text{EnhancedLoRALinear}_1(X_{\text{norm2}}, r, \alpha, \text{enable\_gate\_residual}=\text{True})$

17: $G_2 \leftarrow \text{EnhancedLoRALinear}_2(X_{\text{norm2}}, r, \alpha, \text{enable\_gate\_residual}=\text{True})$

18: $\mathbf{H} \leftarrow \sigma_{\text{swish}}(G_1) \odot G_2 \quad \triangleright \text{SwiGLU gating mechanism: Swish(G1) } \odot G_2$

19: $\mathbf{F} \leftarrow \text{EnhancedLoRALinear}_3(\mathbf{H}, r, \alpha, \text{enable\_gate\_residual}=\text{True}) \quad \triangleright \text{Final}$
advanced projection

20: $X' \leftarrow X_{\text{res1}} + \text{DropPath}(\mathbf{F})$

21: **return** $X'$

## Data availability

I have shared the Github link with data and code.

**Algorithm 4 Enhanced Spectral Processing with Adaptive Attention and LoRA**

Require: Hyperspectral input $X \in \mathbb{R}^{B \times 1 \times C \times H \times W}$

Require: LoRA rank $r$, LoRA alpha $\alpha$

Ensure: Processed features $\mathbf{F}_{\text{spec}} \in \mathbb{R}^{B \times d \times H \times W}$

1: **Multi-Scale 3D Convolution Processing:**

2: $\mathbf{X}_1 \leftarrow \text{Conv}3D_{7 \times 3 \times 3}(\mathbf{X}) \quad \triangleright \text{First spectral convolution}$

3: $\mathbf{X}_1 \leftarrow \text{BN3D}(\mathbf{X}_1) \circ \sigma_{\text{swish}}(\mathbf{X}_1)$

4: $\mathbf{X}_2 \leftarrow \text{Conv}3D_{5 \times 3 \times 3}(\mathbf{X}_1) \quad \triangleright \text{Second spectral convolution}$

5: $\mathbf{X}_2 \leftarrow \text{BN3D}(\mathbf{X}_2) \circ \sigma_{\text{swish}}(\mathbf{X}_2)$

6: $\mathbf{X}_3 \leftarrow \text{Conv}3D_{3 \times 3 \times 3}(\mathbf{X}_2) \quad \triangleright \text{Third spectral convolution}$

7: $\mathbf{X}_3 \leftarrow \text{BN3D}(\mathbf{X}_3) \circ \sigma_{\text{swish}}(\mathbf{X}_3)$

8: Enhanced Spectral Regularization:

9: $\mathbf{X}_3 \leftarrow \text{EnhancedBandDropout}(\mathbf{X}_3, p = 0.1) \quad \triangleright \text{Advanced band dropout}$

10: **Adaptive Spectral Attention:**

11: $\mathbf{F}_{\text{spec}} \leftarrow \text{AdaptiveSqueezeExcitation}(\mathbf{X}_3) \quad \triangleright \text{Adaptive to tensor dimensions}$

12: LoRA-Enhanced Feature Projection:

13: $\mathbf{F}_{\text{spec}} \leftarrow \text{EnhancedLoRALinear}(\mathbf{F}_{\text{spec}}, r, \alpha, \text{enable\_gate\_residual}=\text{True})$

14: **return** $\mathbf{F}_{\text{spec}}$

**Algorithm 5 Enhanced LoRA Cyclical Learning Rate Scheduler**

Require: Optimizer with parameter groups, base learning rates $\{\text{lr}_i\}$

Require: Cosine annealing parameters: $T_0, T_{\text{mult}}, \eta_{\text{min}}$

Require: LoRA learning rate scale factor $s_{\text{lora}}$

Ensure: Updated learning rates for all parameter groups

1: **Initialize Scheduler:**

2: $T_{\text{cur}} \leftarrow 0, T_i \leftarrow T_0$

3: Store initial learning rates $\{\text{lr}_{\text{init},i}\}$ for each parameter group

4: **Update Learning Rates:**

5: **if** $T_{\text{cur}} == T_i$ **then**

6: $T_{\text{cur}} \leftarrow 0, T_i \leftarrow T_i \times T_{\text{mult}}$

7: **end if**

8: **for** each parameter group $i$ **do**

9: $\text{lr}_{\text{init}} \leftarrow \text{lr}_{\text{init},i}$

10: $\cos\_f\text{actor} \leftarrow \frac{1+\cos(\pi \cdot T_{\text{cur}}/T_i)}{2}$

11: **if** parameter group contains LoRA parameters **then**

12: $\text{lr}_i \leftarrow \eta_{\text{min}} + (\text{lr}_{\text{init}} - \eta_{\text{min}}) \cdot \cos\_f\text{actor} \cdot s_{\text{lora}}$

13: **else**

14: $\text{lr}_i \leftarrow \eta_{\text{min}} + (\text{lr}_{\text{init}} - \eta_{\text{min}}) \cdot \cos\_f\text{actor}$

15: **end if**

16: **end for**

17: $T_{\text{cur}} \leftarrow T_{\text{cur}} + 1$

## References

Ahmad, M., Butt, M.H.F., Khan, A.M., Mazzara, M., Distefano, S., Usama, M., Roy, S.K., Chanussot, J., Hong, D., 2025. Spatial-spectral morphological mamba for hyperspectral image classification. Neurocomputing 636, 129995.

Ahmad, M., Ghous, U., Usama, M., Mazzara, M., 2024. WaveFormer: Spectral-spatial wavelet transformer for hyperspectral image classification. IEEE Geosci. Remote. Sens. Lett. 21, 1-5. http://dx.doi.org/10.1109/LGRS.2024.3370919.

Arshad, J.Z.T., Ullah, I., 2024. A hybrid convolution transformer for hyperspectral image classification. Eur. J. Remote. Sens. 2330979.

Bai, J., Zhou, Z., Chen, Z., Xiao, Z., Wei, E., Wen, Y., Jiao, L., 2024. Cross-dataset model training for hyperspectral image classification using self-supervised learning. IEEE Trans. Geosci. Remote Sens.

Ben Zaken, A., Goldberg, Y., Globerson, A., 2022. BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. Trans. Assoc. Comput. Linguist. 10, 496-512. http://dx.doi.org/10.1162/tacl_a_00516.

CCWintco Elhami, et al., 2004. Salinas valley AVIRIS hyperspectral dataset. Online: https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.

Chen, Z., Li, X., Zhao, J., 2020. Spectral variability in remote sensing: Causes and mitigation. J. Appl. Remote. Sens.
Cheng, C., Zhang, L., Li, H., Dai, L., Cui, W., 2024. A deep stochastic adaptive Fourier decomposition network for hyperspectral image classification. IEEE Trans. Image Process. 33, 1080–1094.

Cui, B., Wen, J., Song, X., He, J., 2023. MADANet: a lightweight hyperspectral image classification network with multiscale feature aggregation and a dual attention mechanism. Remote. Sens. 15 (21), 5222.

Ding, L., Tang, H., Bruzzone, L., 2021. LANet: Local attention embedding to improve the semantic segmentation of remote sensing images. IEEE Trans. Geosci. Remote Sens. 59, 426–435.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An image is worth 16 exttimes16 words: Transformers for image recognition at scale. In: Proc. Int. Conf. Learning Represent. ICLR.

Edalati, A., Amini, M., Mousavi, H., 2022. KronA: Kronecker product-based adapters for parameter-efficient tuning. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pp. 1234–1244.

Fadi, A.Z., Abdelkrim, O., 2024. Hyperspectral imaging using deep learning in wheat diseases. In: 2024 8th International Conference on Image and Signal Processing and their Applications. ISPA, IEEE, pp. 1–8.

Fu, H., Ling, Z., Sun, G., Ren, J., Zhang, A., Zhang, L., Jia, X., 2024. HyperDehazing: A hyperspectral image dehazing benchmark dataset and a deep learning model for haze removal. ISPRS J. Photogramm. Remote Sens. 218, 663–677.

Fu, H., Sun, G., Zhang, L., Zhang, A., Ren, J., Jia, X., Li, F., 2023. Three-dimensional singular spectrum analysis for precise land cover classification from UAV-borne hyperspectral benchmark datasets. ISPRS J. Photogramm. Remote Sens. 203, 115–134.

Fu, H., Wang, C., Chen, Z., 2025. Adaptive selection of spectral-spatial features for hyperspectral image classification using a modified-CBAM-based network. Neurocomputing 416, 128877. http://dx.doi.org/10.1016/j.neucom.2024.12.192.

García-Vera, Y.E., Polochè-Arango, A., Mendivelso-Fajardo, C.A., Gutiérrez-Bernal, F.J., 2024. Hyperspectral image analysis and machine learning techniques for crop disease detection and identification: A review. Sustainability 16 (14), 6064.

Hatamizadeh, A., Yin, H., Heinrich, G., Kautz, J., Molchanov, P., 2023. Global context vision transformers. In: International Conference on Machine Learning. PMLR, pp. 12633–12646.

Hayou, S., Zhang, Y., Hoffer, E., 2024. LoRA+: Improving low-rank adaptation by decoupling learning rates. In: Proceedings of the 2024 International Conference on Learning Representations.

Heo, B., Yun, S., Han, D., Chun, S., Choe, J., Oh, S.J., 2021. Rethinking spatial dimensions of vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 11936–11945.

Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., Gelly, S., 2019. Parameter-efficient transfer learning for NLP. In: Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 2790–2799.

Hu, T., Gao, P., Ye, S., Shen, S., 2023. Improved SR-SSIM band selection method based on band subspace partition. Remote. Sens. 15 (14), 3596.

Hu, W., Huang, Y., Li, W., Zhang, F., Li, H., 2015. Deep convolutional neural networks for hyperspectral image classification. J. Sensors 2015, 258619. http://dx.doi.org/10.1155/2015/258619.

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, L., Chen, W., 2021. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.

Hu, E., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Chen, W., 2022. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.

Huang, L., Chen, Y., He, X., 2024. Spectral-spatial mamba for hyperspectral image classification. arXiv preprint arXiv:2404.18401.

Ibanez, D., Fernandez-Beltran, R., Pla, F., Yokoya, N., 2022. Masked auto-encoding spectral-spatial transformer for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 60, 1–14.

Jia, X., Yin, Y., Song, K., Golla, A., Lal, A., Huang, F., Anandkumar, A., 2022. Visual prompt tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7092–7101.

Khan, M., Du, Q., Sun, X., Wang, D., 2024. GroupFormer: Spectral-spatial feature extractor group attention transformer for hyperspectral image classification. Sci. Rep. 14, 5261. http://dx.doi.org/10.1038/s41598-024-74835-1.

Li, Y., Luo, Y., Zhang, L., Wang, Z., Du, B., 2024. MambaHSI: Spatial-spectral mamba for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 62, 1–16. http://dx.doi.org/10.1109/TGRS.2024.3430985.

Li, F., Sun, Y., Huang, W., 2022. Trades-offs between spectral and spatial resolution in hyperspectral sensors. Sensors.

Li, Y., Zhang, H., Shen, Q., 2017. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote. Sens. 9 (1), 67. http://dx.doi.org/10.3390/rs9010067.

Ligan, B., Jbilouk, K., Kalloubi, F., Ratnani, A., 2025. Parameter-efficient fine-tuning of multispectral deep learning models for hyperspectral image classification. arXiv preprint arXiv:2505.15334.

Ma, J., Zhang, Q., 2023. Deep learning frameworks for hyperspectral classification: A review. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.

Ma, J., Zhao, L., Xu, H., 2024. AS2MLP: Adaptive spatial-shift MLP for hyperspectral image classification. IEEE Trans. Neural Netw. Learn. Syst. 35 (4), 2105–2117. http://dx.doi.org/10.1109/TNNLS.2023.3298765.

Ma, C., et al., 2023. Light self-Gaussian-attention vision transformer for hyperspectral image classification. IEEE Trans. Instrum. Meas. 72.

Mei, S., Song, C., Ma, M., Xu, F., 2022. Hyperspectral image classification using group-aware hierarchical transformer. IEEE Trans. Geosci. Remote Sens. 60, 1–14.

Ram, B.G., Oduor, P., Igathinathane, C., Howatt, K., Sun, X., 2024. A systematic review of hyperspectral imaging in precision agriculture: Analysis of its current state and future prospects. Comput. Electron. Agric. 222, 109037.

Roy, S., Krishna, G., Dubey, S., Chaudhuri, B., 2020. HybridSN: Exploring 3D-2D CNN feature hierarchy for hyperspectral image classification. IEEE Geosci. Remote. Sens. Lett. 17, 277–281.

RSIDEA group, Wuhan University, 2017. WHU-Hi-HongHu dataset. Online: https://huggingface.co/datasets/rsidea/WHU-Hi-HongHu.

RSIDEA group, Wuhan University, 2018. WHU-Hi-LongKou dataset. Online: http://rsidea.whu.edu.cn.

Sellam, A.Z., Benaissa, I., Taleb-Ahmed, A., Patrono, L., Distante, C., 2025. Mamba adaptive anomaly transformer with association discrepancy for time series. arXiv preprint arXiv:2502.07858.

Sellam, A.Z., Distante, C., Taleb-Ahmed, A., Mazzeo, P.L., 2024. Boosting house price estimations with multi-head gated attention. Expert Sys. Appl. 259, 125276. http://dx.doi.org/10.1016/j.eswa.2024.125276, Open access under CC license.

Sun, L., Zhang, H., Zheng, Y., Wu, Z., Ye, Z., Zhao, H., 2024. MASSFormer: Memory-augmented spectral-spatial transformer for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 60.

Sun, L., Zhao, G., Zheng, Y., Wu, Z., 2022. Spectral-spatial feature tokenization transformer for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 60 (1), 1–14.

Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., Jégou, H., 2021. Going deeper with image transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 32–42.

Wang, D., Hu, M., Jin, Y., Miao, Y., Yang, J., Xu, Y., Qin, X., Ma, J., Sun, L., Li, C., Fu, C., Chen, H., Han, C., Yokoya, N., Zhang, J., Xu, M., Liu, L., Zhang, L., Wu, C., Du, B., Tao, D., Zhang, L., 2025. HyperSIGMA: Hyperspectral intelligence comprehension foundation model. IEEE Trans. Pattern Anal. Mach. Intell. 47 (8), 6427–6444. http://dx.doi.org/10.1109/TPAMI.2025.3557581.

Xin, A., Zheng, B., Li, C., 2024. A survey of parameter-efficient fine-tuning methods. arXiv preprint arXiv:2403.12345.

Xu, R., Dong, X.-M., Li, W., Peng, J., Sun, W., Xu, Y., 2024. DBCTNet: Double branch convolution-transformer network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 62.

Yang, X., Cao, W., Lu, Y., Zhou, Y., 2022. Hyperspectral image transformer classification networks. IEEE Trans. Geosci. Remote Sens. 60 (1), 1–15.

Yang, J., Du, B., Wu, C., Zhang, L., 2021. Automatically adjustable multi-scale feature extraction framework for hyperspectral image classification. In: Proc. IEEE Int. Geosci. Remote Sens. Symp.. IGARSS, Brussels, Belgium, pp. 3649–3652.

Yang, X., Luo, Y., Zhang, Z., Tang, D., Zhou, Z., Tang, H., 2024. AMHFN: Aggregation multi-hierarchical feature network for hyperspectral image classification. Remote. Sens. 16 (18), 3412.

Yeh, C., Lee, H., Kim, K., 2024. Lokr: Low-rank adaptation with Kronecker product for diffusion models. arXiv preprint arXiv:2403.01234.

Yu, L., Chen, X., Li, J., 2024. Prototype-based knowledge distillation for hyperspectral image classification. Pattern Recognit. 136, 109178. http://dx.doi.org/10.1016/j.patcog.2023.109178.

Yue, H., Ma, Y., Zhang, Q., 2022. Self-supervised knowledge distillation for hyperspectral image classification. ISPRS J. Photogramm. Remote Sens. 183, 169–183. http://dx.doi.org/10.1016/j.isprsjprs.2021.12.009.

Zhang, X., Liang, Y., Li, C., Hu, N., Jiao, L., Zhou, H., 2023. TransHSI: Exploring 3D-2D CNN feature hierarchy with transformers for hyperspectral image classification. Remote. Sens. 15, 1056. http://dx.doi.org/10.3390/rs15041056.

Zhang, T., Liu, R., Wang, Y., 2024a. LDS2MLP: Learnable dilation spectral-spatial MLP for hyperspectral classification. J. Sel. Top. Appl. Earth Obs. Remote Sens. 17, 125–136. http://dx.doi.org/10.1109/JSTARS.2023.3291902.

Zhang, Y., Qiao, S., Ping, W., Sun, Y., Xu, C., Loy, C., 2024b. NOAH: Nonlinear adapters for parameter-efficient transfer learning. arXiv preprint arXiv:2401.09876.

Zhang, C., Zhang, M., Li, Y., Gao, X., Qiu, S., 2021. Difference curvature multidimensional network for hyperspectral image super-resolution. Remote. Sens. 13 (17), 3455.

Zhou, Z., Li, Y., Zhang, X., 2023a. DiCT: Dictionary-centric transformer for efficient hyperspectral image classification. Pattern Recognit. 137, 109371. http://dx.doi.org/10.1016/j.patcog.2023.109371.

Zhou, M., Wang, S., Li, D., 2023b. Precision agriculture with hyperspectral imaging: A review. J. Agric. Remote. Sens.

Zidi, F.A., Ouafi, A., Bougourzi, F., Distante, C., Taleb-Ahmed, A., 2025. Advancing wheat crop analysis: A survey of deep learning approaches using hyperspectral imaging. Comput. Electron. Agric. 238, 110770.