NeurIPT: Foundation Model for Neural Interfaces
Authors: Zitao Fang
CHENXUAN LI
Zhou Hongting
Shuyang Yu
Guodong DU
Ashwaq Qasem
Yang Lu
Jing Li
Junsong Zhang
Sim Kuan Goh
Venue: NeurIPS 2025 poster ()
Keywords: BCI
transformer
EEG
Primary area: neuroscience_and_cognitive_science
TL;DR: NeurIPT, advancing scalable and generalizable EEG decoding across diverse BCI tasks.

Abstract: Electroencephalography (EEG) has wide-ranging applications, from clinical diagnosis to brain-computer interfaces (BCIs). With the increasing volume and variety of EEG data, there has been growing interest in establishing foundation models (FMs) to scale up and generalize neural decoding. Despite showing early potential, applying FMs to EEG remains challenging due to substantial inter-subject, inter-task, and inter-condition variability, as well as diverse electrode configurations across recording setups. To tackle these open challenges, we propose **NeurIPT**, a foundation model tailored for diverse EEG-based **Neur**al **I**nterfaces with a **P**re-trained **T**ransformer by capturing both homogeneous and heterogeneous spatio-temporal characteristics inherent in EEG signals. Temporally, we introduce Amplitude-Aware Masked Pretraining (AAMP), masking based on signal amplitude rather than random intervals, to learn robust representations across varying signal intensities beyond local interpolation. Moreover, this temporal representation is enhanced by a progressive Mixture-of-Experts (MoE) architecture, where specialized expert subnetworks are progressively introduced at deeper layers, adapting effectively to the diverse temporal characteristics of EEG signals. Spatially, NeurIPT leverages the 3D physical coordinates of electrodes, enabling effective transfer across varying EEG settings, and develops Intra-Inter Lobe Pooling (IILP) during fine-tuning to efficiently exploit regional brain features. Empirical evaluations across nine downstream BCI datasets, via fine-tuning and training from scratch, demonstrated NeurIPT consistently achieved state-of-the-art performance, highlighting its broad applicability and robust generalization. Our work pushes forward the state of FMs in EEG and offers insights into scalable and generalizable neural information processing systems.

NeurIPT: Foundation Model for Neural Interfaces
Zitao Fang'
Chenxuan Li
Hongting Zhou'
Shuyang Yu² Guodong Du?
Ashwaq Qasem'
Yang Lu
Jing Li
Junsong Zhang Sim Kuan Goh
Xiamen University Malaysia "Columbia University
The Hong Kong Polytechnic University *Xiamen University
Harbin Institute of Technology (Shenzhen)
ait22090710xmi 0xmou edu. my zhangjs@xmi edu. simkuanghh@gmail.com com
Abstract
Electroencephallogaapy (EEG) has wide-ranging applications, from clinical diag- nosis to brain-computer interfaces (BCIs). With the increasing volume and variety of EEG data, there has been growing interest in establishing foundation models (FMs) to scale up and generalize neural decoding. Despite showing early potential, applying FMs EEG remains challenging due substantial inter-subject, inter- task, and inter-condition variability, as well as diverse electrode configurations across recording setups. To tackle these open challenges, propose NEURIPT, a foundation model developed for diverse EEG-based Neural Interfaces with Pre-trained Transformer by capturing both homogeneous and heterogeneous spatio-temporal characteristics inherent in EEG signals. Temporally, we intro- duce Amplitude-Aware Masked Pretraining (AAMP), masking based on signal amplitude rather than random intervals, to learn robust representations across varying signal intensities beyond local interpolation. Moreover, this temporal representation is enhanced by a Progressive fixture-of-Expert (PMoE) architec- ture, where specialized expert subnetworks are progressively introduced at deeper layers, adapting effectively to the diverse temporal characteristics of EEG signals. Spatially, NeurlPT leverages the 3D physical coordinates of electrodes, enabling effective transfer of embedding across varying EEG settings, and develops Intra- Inter Lobe Pooling (IILP) during fine-tuning to efficiently exploit regional brain features. Empirical evaluations across eight downstream BCI datasets, via fine- tuning, demonstrated NeurlPT consistently achieved state-of-the-ar performance, highlighting its broad applicability and robust generalization. Our work pushes forward the state of FMs in EEG and offers insights into scalable and generalizable neural information processing systems. Our project is available this https URL.
1 Introduction
Elletrreeneephalograpyy (EEG) has been widely adopted as a proxy for brain activity and dynamics, due to its non- -on-invasiveeess, portability, and high temporal resolution for real-time monitoring [1]. EEG facilitates investigations into brain function, assists in neurological diagnoses [2], provides objec- tive biomarkers for cognitive and affective states [3], and enables the development of brain-computer interfaces (BCIs) [4]. The growing availability of large-scale EEG datasets (spanning diverse popula- tions, recording configurations, and experimental paradigms) has spurred the development of wide range of computational approaches, including machine learning and deep learning models (CNN [5], RNN [6]. GNN [7], Transformers [8]) for learning representations. However, these models are often tailored to specific tasks and settings, limiting their generalizability and cross-setting applicability. Hence, a paradigm shift is necessary establish generalizable models that can effectively leverage
Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Delta-6
Theta-0 0
Alpha-(
Sigma-d
Beta-B
Gamma-y
Deep-sleep
Droway
Relased
Spindles
Focused,
Conscionsnes, Epilepay
[P:000 os4
SeeptDex
PhysaaP30
StEDM
SleeptDFX
TUEP
BCC 2A
TUEP
Mumita2206
BCC-2A
Nummta2016
Frequency (Hiz)
Figure 1: Spectrograms of the nine EEG datasets reveal both homogeneous and heterogencous spectral patterns, with some datasets showing higher power spectral density (PSD) in specific EEG frequency bands. Thus, they demand neural representations capable of adapting to input variability.
EEG signals across diverse setups and settings, including inter-subject variability, differences between healthy and patients, electrode configurations, and variations in cognitive behavioral tasks.
Recent advances in foundation models (FMs), large-scale neural architectures pre-trained using transformers on diverse and unlabeled datasets through self-supprvised learning [9], followed by fine- tuning on downstream datasets, have demonstrated remarkable success in natural language processing (NLP) and computer vision (CV). In NLP, models [10] such as Claude, DeepSeek, Gemini, GPT, and Llama, pre-trained on massive text corpora, have learned general-purpose representations that transfer effectively to wide range of downstream tasks [11]. Similarly, pre-trained Vision Transformers (ViT') [12), Masked Autoencoder (MAE) [13] have become foundational components in modern CV pipelines. These developments highlight the potential of FMs to produce robust, transferable representations of both images and language. Building on this progress, multimodal large language models (MLLMs) (e.g.. SORA [14]) further extend these capabilities by integrating information across multiple data modalities, such as text, images, and audio. Hence, is intriguing to investigate FMs' generalizability brain signals by developing EEG-based FMs.
Initial efforts develop foundation models (FMs) for neural decoding showed early promise. BENDR [15] adapted techniques from language modeling to learn compressed representations of raw data signals, enabling the model to generalize across different tasks and datasets. GeG2Vec [16] learned generative-discriinnat emotion classification tasks. BIOT [17] addressed the challenges of cross-dataset EEG learning by to-
representations for
kenizing EEG signals into fixed-length segments, accommodating variable-lengle sequences and mis- matched channels. By learning from diverse biosig- nal datasets, improved performance can be achieved in tasks such seizure detection. To encode EEG signals into discrete tokens, LaBraM [18] introduced neural tokenizer with vector-quantized spectrum prediction, facilitating the pretraining of transform- ers predict masked segments and enhancing the model's s ability to learn from large-scale EEG data. NeuroLM [19] proposed a universal multi-task foun- dation model that bridged the gap between language and EEG signals. Treating BEG as a foreign lan- guage, Neurol M adopted text-aligned neural tok- enizer and leveraged large language models (LL.Ms) to perform multi-task learning across various EEG-
(a) BER TERRTT-ike like random musking.
(b) Amplitude-aware musking.
Figure 2: BERT-like random masking v.s. our proposed amplitude-aware masking.

---

related tasks. EEGPT [20] introduced a dual self-supervised learning approach for feature alignment, capturing spatio-temporal representations in EEG data. This method enhanced the model's ability to learn robust features that generalized across different tasks and datasets. CBraMod [21] presented a criss-cross transformer architecture designed to model spatial and temporal dependencies sepa- rately in EEG signals from diverse datasets, facilitating linear probing for fine-tuning on numerous downstream tasks. These methods provided evidence of the applicability of FMs for EEG.
Despite the encouraging preliminary findings, current foundation model (FM) methods for EEG have largely adopted pretraining strategies from language and time-series domains without accounting for several unique properties of EEG. First, existing positional encodings treat electrode channels as interchangeable and ignore their physical diree-dimensional arrangement, losing critical spatial relationships, which can significantly impair transferabilityy Second, prevalent masked pretraining strategies are typically based on randomly masking contiguous signal segments like BERT [22], unin- tentionally guiding the model toward local interpolation rather than meaningful global representation learning, as illustrated in Figure 2. Third, conventional neural architectures rely heavily on fully connected layers or global pooling mechanisms when fine-tuned to downstream tasks and thus do not explicitly utilize regional brain features. Finally, due to the diversity and complexity of EEG data patterns, ranging from slow-wave oscillations during sleep to rapid spikes during seizures [23], models must be capable of adaptively capturing heterogeneous temporal dynamics recorded in diverse setups. The diverse spectral patterns are shown in Figure 1. Addressing these challenges is critical to fully realizing the potential of foundation models for EEG.
To address the aforementioned challenges, we introduce NEURIPT, a novel EEG foundation model developed to learn robust and generalizable representations across diverse BCI applications. Our NEURIPT include: (i) 3D-Aligned Spatial Encoding, a flexible positional encoding scheme that integrates the actual three-dimensional coordinates of EEG electrodes, enabling seamless adaptation across varying electrode montages without re-training; (ii) Amplitude-Aware Masked Pretraining (AAMP), a novel masking strategy guided by EEG signal amplitude rather than random intervals, shown in Figure 2, compelling the model to capture underlying EEG patterns with amplitude serves as proxy for signal energy instead of trivial local interpolations; (iii) Progressive Mixture-of of-Eeerrss (PMoE), architectural innovation wherein the number of specialized subnetworks increases with model depth, effectively accommodating diverse temporal EEG patterns; and (iv) Intra-Inter Lobe Pooling (IIP)), a spatial aggregation method that performs hierarchical pooling within and across brain lobes, leveraging distinctive regional brain features for downstream tasks. Empirically, NEURIPT achieves consistent state-of-the-art performance across eight diverse EEG benchmarks, including seizure detection, cognitive state decoding, sleep stage classification etc., summarized in Figure S. Our comprehensive evaluations highlight NEURIPT enhanced scalability, improved robustness to spatial and temporal variations, and progress toward building truly universal EEG representation models. These findings mark an advancement for foundation models in EEG and offer practical insights for future neural interface development.
The main contributions of this paper are summarized as follows:
1. We propose NEURIPT, a foundation model developed for EEG-based neural interfaces, designed learn robust and generalizable representations by capturing the intrinsic spatio- temporal heterogeneity of EEG signals across diverse settings.
2. Temporally, we introduce Amplitude-Aware Masked Pretraining (AAMP), which masks segments based on signal amplitude rather than random intervals, enabling the model to learn robust features across varying signal intensities. In addition, we design a Progressive Mixture-of-Experte (PMoE) architecture that adaptively introduces specialized experts at deeper layers to capture diverse temporal dynamics in EBG.
3. Spatially, NEURIPT incorporates the 3D physical coordinates of electrodes for embedding, facilitating spatial generalization across datasets with varying sensor montages. During fine-tuning, we introduce Intra-Inter Lobe Pooling (IILP) to explicitly model and leverage region-specific brain activity patterns for downstream tasks.
4. We conducted extensive empirical evaluations on eight benchmark BCI datasets. NEURIPT consistently achieved sfate-of-the-ard performance, demonstrating strong generalization and broad applicability across diverse EEG-based tasks.
3

---

Inte-liter Lobe Pooling
Dassller Black
Reconstrud
Promall
Ooopitl
Tompent
Torporai
Paistal
In with
Progresive Moll
wm A Aalie
Block
Spaiil
Expent
Exper
Expert
Sharod Expert
Tears
Eaame
Progesstv
Aumpiiaale Aweare Mussing
Tenporal
Block
Laver Nern
Expert
Export
Steeed Expert
10 Elettode Unnledding
Figure 3: Overview of our NEURIPT, which comprises Amplitude- Aware Masked Pretraining (AAMP), 3D Electrode Embedding. Progressive Mixture-of-Expert (PMoE). and Intra-Inter Lobe Pooling (IILP) for fine-tuning, See Figure 4 for details on the IILP module.
2 Method
The framework of NEURIPT is illustrated Figure 3. In Section 2.1, we provided the details on how NEURIPT integrates spatial and temporal EEG representations by embedding 3D spatial electrode coordinates and employing Amplitude-Awel Aware Masking enhance sulf-supe learning. Subsequently, Section 2.2 describes our Progressive cexture-of-Experee architecture to dynamically manage EEG variations, complemented by Intra-Inter Lobe Pooling effectively capture regional brain features for downstream tasks.
2.1 Embedding Spatial Context into Temporal Representations
We pretrain our NEURIPT using self-supervised learning learn robust representations of EEG signals across diverse setups and settings, without relying on annotations or incorporating any downstream EEG task data. Specifically, given of unannotated EEG dataset D(u) {xx() }
where each sample x()) E TD represents multivariate EEG signals with T time steps and D electrode channels.
3D Electrode Embedding This designed to leverage the ttree-dimensional spatial relationships among EEG electrodes, which typically arranged according to the international 10-20 electrode placement system. Specifically, for the d-th electrode channel positioned at (Id, Vd. zd), we encode its three spatial coordinates separately and concatenate them follows:
PE()) Concat (PE, (xa) PE (wa), (E)()))
Each coordinate embedding uses sinusoidal function defined PE, (pos)2j sin(pos (2/00)2)/da)) and PE. (pos)2j+1 cos(pos 10000 [0000-2j/d.) where
x. denotes the spatial axes, pos represents the position along the corresponding spatial coordinate, j indexes the embedding dimensions, and dg = dmmodet /3 is the dimensionality of the embedding for axis a.
Building upon pixel-level embeddings in CV [12], we adopt single -oint embeddings to preserve temporal details and effectively capture sharp spikes crucial for brain activity analysis. We represent each EEG sample as a collection of data points (i) = {- 11st T, < < D}. each point is then embedded with both temporal and spatial information:
Ex + PE PE
Here, Todme denotes learnable linear projection vector, (F) represents temporal positional encoding following the sinusoidal formulation from Vaswani et al. [9]. After embedding, the encoder

---

Bird View
Pooliigg
Inter Concatenati
Left
Hight
Hund
Right
75
Frustal Labe
Occipiaal
Temporal
Lake (skal1)
Paricial Lake
Atteation Score
Figure 4: (Left) Intra-Inter Lobe Pooling (II.P) leverages regional brain features during fine-tuning. (Right) Visualization of attention scores from the temporal attention module and analysis of Pearson correlation between class logits and channel perturbation using Gaussian multiplicative noise. Note that colors in the right panel correspond to the brain regions depicted on the left.
input is represented as follows:
S (1) r.
T, d D).
1
(3)
where each vector s encapsulates both spatial and temporal features, which are crucial for capturing EEG dynamics in subsequent modeling stages. Both temporal and spatial encodings natively support varying temporal lengths T and spatial dimensions D, enabling seamless adaptation to diverse EEG electrode placement systems, such as the 10-05 and 10-20 standards, without additional convolutional components or padding as used in EEGPT [20] and CBraMod [21]. This design is also naturally consistent with the proposed AAMP method introduced below.
Amplitude-Aware Masked Pretraining (AAMP) Self-supervised learning techniques like BERT [22] and MAE [13], which reconstruct randomly masked data, have shown promising results in multivariate time series (MTS) [15, 18, 20]. However, random masking often degrades to simple interpolation between unmasked points in time series data [24], thereby limiting the extraction of meaningful structural representations. To address this, we propose a novel Amplitude-Awe Aware Masking Pretraining (AAMP) approach, explicitly designed to capture more informative features while avoiding mere interpolation, illustrated Figure 2. After obtaining the embedding array S, we generate our Amplitude-Awar Aware Masking:
(1) 11 ai E [La It 1,...,
T, 1....,,,,. D).
M
(4)
where the interval Ua] covers T P points, centered around the (1)) T-hh point in sorted ())
Specifically, probability P denotes the masking ratio and e(t) ~ u(0, 1) is a randomly sampled percentile for dimension of data instance Subsequently, we derive the EEG embedding S. partitioned into the masked set S.M (st,al = 1} and the unmasked set S {st,a/me,d = 0}
according to the corresponding St,d and m4,a-
Hierarchical Attention Modules In complex MTS tasks, i.e., EEG signals processing, capturing both temporal and spatial dimensions is significant [21, 25]. To effectively extract these spatio- temporal characteristics at multiple granularities, we adopt Crossformer [26] as part of our backbone architecture, which hierarchically captures alternating temporal and spatial dependencies by using TSA module. Further implementation details of our modified Crossformer are provided in Appendix C.I.
Specifically, our hierarchical attention model comprises encoder ENC,, and decoder IDECpp each equipped with trainable parameters 0g and 0p. respectively. These components jointly capture ppatio-tempora relationships and reconstruct masked portions of the input data follows:
* IBEC,, (ENCa, (Sene), Seace).
and the decoder input & dde is formulated as:
Sdec_ = E (11 3 x()) + PE( + PE(.)
(6)
5

---

The encoder ENC,, processes the unmasked input S to extract meaningful features while set- ting the attention scores of the masked set S.M zero, thereby preventing unintended access to masked tokens. Thereafter, the decoder IDRCo, reconstructs the signals in S through a specific [mask] token, leveraging both the encoder outputs and unmasked signals enriched by comprehensive positional information. We then optimize the parameters 8g and 0D by minimizing the bp f--norm reconstruction loss: AAMP (0g,
1/P
Σ x
$001p)
72
2.2 Integrating Temporal Dynamics into Spatial Representations
EEG signals exhibit complex, heterogeneous information across various frequency bands, transient events, and even inevitable artifacts, making their representations challenging for single feed- forward network (FFN). To address this, we leverage Progressive cexture-of-Expeee (PMoE) architecture, enabling distinct sub-networks as specialized experts capture specific EEG features, combined with a shared expert to ensure stable generalization. Subsequently, we fine-tune with Intra-Inter Lobe Pooling (IILP), which effectively captures cross-channel connectivity patterns.
Progressive Mixture-of- -eppe-ests (PMoE) Given Ż' E xgTxDxxdoode the output after the attention layer and subsequent layer normalization, a gating mechanism computes token-level routing weights:
E g -1
= TopKSofimaxRRutte"" (Z')),
(8)
where Ei denotes number of experts the 1-th encoder layer and each expert applies its private sub-network Y! "FN!!(22) Introduced by [27], TopKSoftmax applies the softmax function exclusively to the top-k logits, representing the sparse activation that serves to save computation. These representations aggregated through gating weights g and further produced the output of transformer block I:
PMoB (1 (Z)) = Σ
ge + Y + FFN(1)
shared (Z'),
(9)
c=1
Z' = PMoE (ž') Z'
(10)
where O denotes element-wise multiplication, FFN shared denotes the shared expert, and Ž' denotes the residual connection from the attention output. Then we incorporate an auxiliary loss Laxx to encourage balanced expert utilization and robust routing behavior [27]. Please refer to Appendix C.1 and C.3 for more details about specific architecture and MoE, respectively.
The shared expert captures generalizable patterns, while the progressively introduced experts handle increasingly specialized signal features [28], effectively adapting to the inherent complexity and variability of EEG signals.
Finetuning with Intra-Inter Lobe Pooling (IIPP) Following the pretraining phase, we performed fine-tuning using downstream labeled datasets D = ((x0), y()) }-= comprising n instances. (1....,
Each instance x0) E I x is associated with a class label (l) E ECC c= .......)) C}. indicating the
downstream classification category.
To better capture EEG functional connectivity patterns while suppressing redundant information across channels, we propose III., two-step pooling strategy, Inira-lobe Pooling followed by Inter- lobe Concatenation, illustrated in Figure Given an encoder block output Zenel TR. x D Xdmodel where T. D, and dmodet denote the temporal length, number of EEG channels, and embedding dimension, respectively.
We first average-pool Zene along the temporal axis to aggregate information from all time steps:
eno t,d
d = 1, I....,.. D.
(11)

---

Intra-lobe Pooling. We partition EEG channels into n functional brain lobes, such as frontal and -. To suppress redundancy within each channel group
occipital lobes, denoted P .Pi..... ).) P, we compute lobe-level embeddings by averaging corresponding channel embeddings:
V =
V dEP.
k == ==
(12)
PA
Inter-lobe Concatenation. To leverage discriminative features across different brain lobes, concatenate the lobe embeddings into joint representation:
concat (Vi..... V.).
(13)
where of Rndddde indicates the aggregation vector for encoder block 1. Repeating the above IILP process across all encoder blocks and stacking the results yields the final representation:
concat (v',...,o')
(14)
where L is the number encoder blocks. Finally, we use multilayer perceptron as the classifier on the representation obtained at different granularities predict the task-specific class from the set C.
3 Experiments
3.1 Pre-training
Datasets NEURIPT is pre-trained using more than 2,000 hours of data collected from public datasets, with the eight downstream datasets explicitly excluded. For more details on pre-training datasets, please refer to Appendix E.
Preprocessing To ensure fair comparison, we follow the data pre-processing pipeline in CBraMod [21]. EEG recordings with a total duration less than 5 minutes were removed, and we discarded the first and last minute of each remaining session mitigate boundary artifacts. Signals were re-referenced using 20 bipolar channels in the canonical "double banana" montage (e.g.. FP1-F7, F7-T3, .... P4 02). For further details on preprocessing, please refer to Appendix E.1.2.
Settings We implemented NEURIPT using Python 3.9.19 and PyTorch 2.3.0 with CUDA 12.1 and cuDNN 8902. Pre-training stage trained using the AdamW optimizer combined with the OneCycle learning rate strategy [29] (upper learning rate 3e-4, divided factor 25, final divided factor le4, and cosine annealing strategy). The pre-training process was conducted for approximately 400K steps, employing an effective batch size of 480 and bfloat 16 mixed-precision training on eight NVIDIA GefFrce RTX 4090 GPUs. For more details on implementation settings and hyperparameters, please refer to Appendix D and Table 13.
Table 1: Overview of downstream BCI tasks and datasets.
Tasks
Datasets
Rate
Channels Used
Duration
Samples
Label
1. Mental Stress Detection
AritheAei
500Hz
20
Ss
1,707
2-class
I1. Mental Disorder Diagnosis
Mumtaz2016
256Hz
20
5s
6,963
2-class
III. P300
PhysioNetP300
2048Hz
20
2s
21.179
2-class
Sleep Staging
Sleep-EDFx
100Hz
30s
457.652
5-class
V. Emotion Recognition
SEFD-V
1000Hz
20
Is
115.001
5-class
Motor Imagery Task
BCIC-IV-2A
250Hz
4s
5,184
4-class
VII. Abnormal Detection
TUAB
250Hz
10s
409,455
2-class
VIII. Event Type Classification
TUEV
250Hz
20
Ss
112,491
6-class
3.2 Downstream BCI Tasks
BCI Tasks and Datasets To comprchensively assess the effectiveness of NEURIPT, we evaluated it across eight diverse BCI datasets spanning multiple downstream task categories. A summary of all tasks and corresponding datasets is provided Table and detailed in Appendix E.2.1. To ensure fair comparison, adopt the same data processing protocols CBraMod [21]. Additional details on each downstream dataset and preprocessing pipeline are presented in Appendix E.2.
Baselines and Metrics All the baselines and metrics detailed in Appendix D.2.
7

---

Table The results of NEURIPT on various datasets. Figure demonstrates the visual comparison.
Datasets
Methods
Balanced Accuracy
Cohen's Kappa /AUC-PR
Weighted F1 /ARROC
BIOT [NeurIPS23][17
68.75
60.04
75.36
LaBraM [CLR24][11
69.09
$9.99
77.21
MentalArithmtti
CBeaMod [CCR25) (21)
72.56
62.67
79.05
NEURIPT (Ours)
86.46 (+193)90
78.27 (415.60)
91.11 (+12.06)
BIOT [NeadPS23]]
93.58
97.36
97.58
LaBraM [CCB244 [18]
94,09
97.98
97.82
Mumtaz2016
CBraMod [ICLR25] [21]
95.60
99.23
99.21
NEURIPT (Ours)
98.03
(+2.43)
99.81 (+IL58)
99.79 (+0.58)
BENDR [15)
61.14
22 22.27
BIOT [Ne3
54.85
9,68
53,08
-PhysioP300
LaBraM (ICI R24 [18]
64.77
29. 3,
70.68
EEGPT [Ne4 (20)
65.02
29.99
71.68
NEURIPT (Ours)
67.31 (+2.29)
34.26 (+4.277
76.83 (+5.15)
BENDR [15]
66.55
66.59
BIOT [NeurIP223]
66.22
64.61
74.15
Sleep-EDFx
LaBraM [CCL244 [18]
67.71
67.10
75.92
EEGPT [Nea]IP224
69.17
68.57
76.54
NEURIPT (Ours)
70.47 (+1,30)
77.57 (49.000)
87.39 (+10.85)
BIOT [NeudPP223
38.37
22.61
38.56
LaBraM [[CR24][18] [18]
39,76
23 36.86
39,74
SEED
CBraMod [CCR25] [21]
40,91
25.69
41.01
NEURIPT (Ours)
41.04 (+0.13)
26.29 (+0.66)
41.58 (+0.57)
BIOT [NeuIIPS23|
47.48
46.07
LaBraM [14]
48.69
31.59
47.58
BCIC-IV-2A
CBraMod [CCR225 [21]
51.38
35.18
49.84
NEURIPT (Ours)
55.04
(+3.66)
40.04 (+4.866
53,76 (+3.92)
BIOT [Ncu1PS223| [17]
79.59
87. 07.92
88.15
LaBraM [CLR24]
92.04
91.62
EEGPT [Neu [20]
79.83
87.18
Me [CLR25] [21]
79.69
72.19
78.84
CBraMod [CCR225] [21]
82.89
92.58
92.27
NEURIPT (Ours)
82,93 (+0.04)
90.40 (-2.18)
89.49 (-2.78)
BIOT [NcurIPS23]11]]
52.81
52.73
74,92
LaBraM [CCR24][188 [18]
66.16
67 .7.45
83.29
TUEV
EEGPT [NearIP524]220 [20]
62.32
63355
$1.87
NeuroLM [CCR25]
73.59
CBraMod [CLR25]
66.71
67.72
83.42
NEURIPT (Ours)
67.61 (+0.90)
69.70 (+1.98)
84.28 (+0.86)
3.3
Main Results
We present comparative analysis of our method against base- lines across eight downstream datasets in Table 2, with the best results highlighted in bold. Moreover, we report the perfor- mance difference between our method and the best-perforning baseline. Compared the baselines, method generally out- performs them on the majority of datasets, with the exception of Cohen's Kappa and AUROC metrics on TUAB. Notably, our method achieves significant improvements on the Mental Arithmetic, PhysioP300, and BCIC-IV-2A datasets. While the performance gains are smaller on other datasets, they remain consistent, demonstrating the robustness and general applicabil- ity of our approach. Extensive empirical evaluations on eight benchmark BCI datasets consistently demonstrate strong gener- alization and broad applicability across diverse EEG-based tasks. Appendix F.2 gives the specific experimental results on each task, including experimental configuration and results analysis.
E
NeurIPT
Lofte
Neusaan
Figure S: Models performance on various BCI downtream tasks.

---

103%
2339
123h
100%
m8hh
150h
tespocs Ragent
Tippesa Expert
Rembori Diart
Tencou Equat
01.9
150h
75th
01.9%
26.0
150%%
78th
2035
160h
2M05
144h
Spapaal Exper
Scasi Eqpen
2p Speta Exper
to So:Sputa Export
Figure 6: Analysis of expert participation (temporal and spatial) when EEG data from different classes of the BCIC-IV-2A dataset is input to the model.
3.4 Additional Results and Analysis
Analysis of Spatial Relationships between EEG Channels To analyze spatial relationships, we presented in Figure 4 the attention scores from spatial attention head, along with an analysis the Pearson correlation between class logits and channel perturbations using Gaussian multiplicative noise. The attention score visualization reveals both inter- and intra-lobe interactions, which aligns with the strengths of IIL.. From the perturbation analysis, we observe contralateral activation patterns channels C3 and C4 for hand-related tasks, and more symmetrical pattern for foot and tongue movements, in line with findings from the existing study [30]. Refer to Figure 8 and 16 in Appendix G for more visualization about the attention score and Pearson correlation various datasets.
Analysis of Progressive Mixture-of-Experts (PMoE) We analyzed how temporal and spatial experts in the PMoE architecture contribute to the prediction of the four classes in the BCIC-IV-2A dataset, with the statistics presented in Figure 6. is observed that different classes engage varying numbers of experts, with some classes receiving attention from a larger number of experts. Refer to Figure 9-15 in Appendix G for more expert contributions across different datasets.
Table 3: Different MoE strategies across various datasets.
MoF. Strategies
No. of Experts
TUEV
MentalArithmetic Mumlaz2016
SEED-V
PhysioP300
BCIC-2A
w/o Expert
[0,0, 0, 0,
65.83
72.92
93.41
39.14
64.53
44.44
Uniform
44,4,4,4, 4,4]
65.91
70.49
95.00
39.33
65.66
41.93
Shrinking
[6. 4,
65.80
73,96
93.08
39.21
65,99
Progressive (Ours)
2,
68.94
75.69
97.07
39.34
66.58
44.01
Table 4: Different PMoE configurations across various datascts.
PMoE Configurations
TUEV
MentalArithmetic Mumtaz2016
SEED-V
PhysioP300
BCIC-2A
[0.0.2.4.4.6]
68.94
75.69
97.07
39.34
66.58
44.01
[0.0.2.3.4.5]
67.87
76.39
96.81
39,88
66.34
41.58
[0.0,2.4.6.8]
66.41
75.69
96.57
39.11
67.70
41.15
(0.0.3.6.9.12]
66.08
74.83
97.15
39.35
66.63
38.19
Ablation of MoE Strategies and PMoE Configurations Tables 3 and 4 show the ablation results of MoE strategies and PMoE configurations, respectively. When varying the number experts across layers, our PMoE, which increases the number of experts with depth, consistently outperforms both the non-MoE baseline, the uniform-expert variant, and the shrinking MoE variant, where the number of experts decreases with depth. The alternative configs in Table 4 also yield competitive performance, which suggests that the effectiveness of the PMoE approach stems from its inherent progressive strategy, rather than relying any specific expert allocation, demonstrating robustness across diverse progressive configurations.
Table 5: Different pooling strategies across various datasets.
Pooling Strategies
TUEV
MentalArithmetie
Mumtaz2016
SEED-V
PhysioP300
BCIC-2A
w/o Pooling
62.33
75.69
78.21
38,90
67.82
45.14 14.14
Mean Pooling
64.74
79.51
96.22
37.62
66.72
37.24
Hemispheres
64.45
81.94
97.82
39.22
67.11
43.49
Coronal
68.77
73.26
96.99
39.35
66.98
43.75
Sagittal
67.21
80.21
91.41
39.42
65.66
45.31
III. (Ours)
68.94
86.46
98.03
41.04
67.31
55.04

---

Table 6: Ablation study on each individual component in NEURIPT. Results are based
on models trained from scratch, without the intenintessive pre- training stage.
3D PE
PMoE
IILP
TUEV
MentalArithmetic
Mumlaz2016
SEED-Y
BCIC-IV-2A
X
51.80
73.36
91.83
37.82
32.64
X
X
59.64
73.61
86.07
38.54
40.19
X
X
$2.79
74.65
85.58
37.82
33.59
59.10
73.96
91.55
35.66
37.15
62.33
75.69
78.21
38.90
45.14
/
X
V
65.83
72.92
93.41
39.14
44.44
X
J
V
67.72
74.65
96.56
35.03
37.59
68.94
75.69
97.07
39.34
44.01
Ablation of Pooling Strategies We compared our Intra-Inter Lobe Pooling (IILP) with pooling as well as pooling strategies based on coronal, sagittal, and hemispheric brain region groupings. The results are presented in Table 5, demonstrating the consistent effectiveness and superiority of IILP.
Ablation of Each Individual Component The ablation Table demonstrates that performance across datasets generally improves with the inclusion of each component, achieving peak accuracy when all components are combined. Notably, the IILP module significantly enhances results on the TUEV and Mumtaz2016 datasets. This improvement is presumably because epilepsy and depression classification tasks require capturing distinctive regional signal variations across different brain areas.
For BCIC-IV-2A, which primarily involves motor imagery tasks and was recorded using EEG chan- nels located centrally with limited coverage of other brain lobes, we observed a performance decline when the 3D PE component was excluded. This suggests that motor imagery tasks are particularly sensitive spatial information, as further supported by our channel perturbation analysis Figure Given the low-data scenario (BCIC-IV-2A includes only nine subjects), explicitly encoding physical spatial relationships becomes especially important. Moreover, excluding only IILP may degrade performance due to incomplete brain lobe coverage, limiting the model's utilization of lobe-specific information, yet incorporating it during pretraining enhances generalization (see Table 5).
Table 7: Different activation functions for EEG FMs (trained from scratch, same as Table 6).
Activation Function
TUEY
MentalArithmeits Mumtaz2016
SEED-V
PhysioP300
BCIC-2A
ReLUU
64.01
73.96
96.98
38.37
66.29
47.31
GELU
64.02
73.61
97.14
38.69
64.38
46.35
SwiGLU
68.94
75.69
97.07
39.34
66.58
44.01
Exploration of Activation Function for EEG Foundation Models (FMs) Different datasets exhib-
ited distinct preferences (Table 7): ReLU performed particularly well on BCIC-2A, whereas GELU achieved competitive results on Mumtaz2016. Nonetheless, SwiGLU demonstrated the most consis- tent performance across multiple downstream tasks, indicating its robustness in FMs for EEG data.
Further Results and Analysis
For further results and analysis, please refer to Appendix B.
Conclusion
This study introduces NEURIPT, foundation model established for diverse EEG-based neural interfaces, overcoming the challenges in learning generalizable spatio-temporal representations of EEG signals from diverse sources. Temporally, we propose Amplitude-Aware Masked Pretraining (AAMP), which selects masked segments based on signal amplitude rather than random intervals, encouraging the model capture robust features across varying signal intensities and avoiding reliance on local interpolation. This is complemented by Progressive Mixture-of- architecture, which progressively incorporates specialized expert subnetworks deeper layers to better adapt to the temporal variability inherent in EEG data. Spatially, NEURIPT utilizes the 3D physical coordinates of electrodes support transferable embeddings across different EEG configurations and introduces Intra-Inter Lobe Pooling (IILP) during fine-tuning to effectively leverage region-specific brain activity. Comprehensive evaluations on eight benchmark EEG datasets demonstrate NEURIPT 's consistent state-of-the-are performance and strong generalization. These findings highlight NEUR IPT's potential as scalable foundation model for EEG, moving toward universal neural decoding systems.
(PMoE)
10

---

Acknowledgments
This work supported in part by the Ministry of Higher Education Malaysia through the Fun- damental Research Grant Scheme (FFGS///2222//CC722XXX//0
National Natural Science Foundation of China (62476070, 62376233); Shenzhen Science and Technology Program (JCYJ202
41202123503005, GXWD20231 128 128103232001, ZDSYS2023062609 1902300660092000088 KQTD202407291 02154066); Department of Science and Technology of Guangdong (2024A1 1540); Natural
Science Foundation of Fujian Province (2024J09001); Collaborative Innovation Platform Project for Fuzhou-Xiamen- Quanzhou National Independent Innovation Demonstration Zone (2022-P-028); Xiaomi Young Talents Program: Xiamen University Malaysia Research Fund (XMUMRF/2002--113)
IECE/0049, XMUMBEE2044-CL//EEEEEN
References
[1] Neethu Robinson, Ravikiran Mane, Tushar Chouhan, and Cuntai Guan. Emerging trends in bei-robotics motor control and rehabilitation. Current Opinion in Biomedical Engineering, 20:100354, 2021. ISSN wtt.
2468-45111 doi: http:////oi..rg/11.. httpa//ceeeeertcceee////////aaa/
URL https:/ //wps:////wwwww//// sciencedreen....
1
[2] Iyad Oheid and Joseph Picone. The temple university hospital eeg data corpus. Frontiers in Neuroscience, 10:196, 2016. doi: 10.3389/fnins2226600013 URL https://doi..rrg 1rg//..33/////mmmm...... www.333999/nnn./ 2016. 00096.
1, 27, 30
[3] Wei Liu, Jie-Lin Qiu, Wei-Long Zheng, and Bao-Liang Lu. Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition. IEEE Transactions on Cognitive and Developmental Systems, 14(2):715-729, 2022. doi: http:://doi.org//1.1111///111 3071170. 2626,29 29
[4] Michael Tangermann, Klaus-Robert Müller, Ad Aertsen, Niels Birbaumer, Christoph Braun, Clemens Brunner, Robert Lech, Carsten Mehring, K. J. Miller, Gernot MüällerPPtzz Guido Nolte, Gert Pfurtscheller, Hubert Preissl, Gerwin Schalk, Alais Schlögl, Carmen Vidaurre, Stephan Waldert, and Benjamin Blankertz. Review of the bci competition iv. Frontiers in Neuroscience, 6:55, 2012. doi: 26.300
https://doi.org/ www.oorg//......//. wtt.9//niin..22..00000..00
[5] Sim Kuan Goh, Hussein Abbass, Kay Chen Tan, Abdullah Al-Mamun, Nitish Thakor, Anastasios Bezerianos, and Junhua Li. Spatio spectral representation learning for electronneephalorrppiic gait- pattern classification, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(9): 1867, 2018, doi: 10.1 201 8.20018888811111 19. URI https: //tpp//////eeee//r///////// ieee. orccdentmttt/ 8428659. 17
[6] Hongli Li, Man Ding. Ronghua Zhang, and Chunbo Xiu. Motor imagery eeg classification algorithm based on cnn-Istm feature fusion network. Biomedical Signal Processing and Control, 72: 10334,2 2022. ISSN 1746-8094, doi: https///oo.org//11 1016/j htt.:/.......2........... 20.1..032141 1, 17, 24, 27, 28, 29, 30, 31
[7] Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. supervupe graph neural networks for improved electroencephalo- graphic seizure analysis. In International Conference on Learning Representations (ICLR), 2022. URL https: //operrie/pee
1,
[8] Yonghao Song, Xueyu Jia, Lie Yang, and Longhan Xie. Transformer-baseel spatialleee feature learning for eeg decoding. Xrviv preprint arXiv:2106. 1170, 2021. URI. https//v//// httpg//bb/2106... 11170.
1. 17. 24, 27, 28, 29, 30, 31
[9] Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https:://prcceeeeeheee www.eeiwinnn.i.innn.oeeee newripe....... cepppppe_filees /ttp:////....eeee2222//2/ee22110
Paper. pdf. 2, 4. 17, 19,
[10] Tianchen Gao, Jiashun Jin, Zheng Tracy Ke, and Gabriel Moryoussef. A comparison of deepseek and other llm.. arXiv preprint 2xxiv2502.03666 2025. URI. httpsttpss//arii www.or/g//baaeeaeeeeeeeeeee 006888. 2, 17
[11] Zitao Fang, Guodong Du, Shuyang Yu, Yifei Guo, Yiwei Zhang, Yiyao Cao, Jing Li, Ho-Kin Tang, and Sim Kuan Goh. To see a world in spark of neuron: Disentangling multi-task interference for training free model merging. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2025. URL https; / //tp///xxvvv///////////////a/a orrg/ abs/2503. 05320.
11

---

[12] Duy Kien Nguyen, Mido Assran, Unnat Jain, Martin R. Oswald. Cees G. M. Snoek, and Xinlei Chen. An image is worth more than 16x16 patches: Exploring transformers individual pixels. In The Thirteenth International Conference on Learning Reprexentations (ICLR), 2025. URL http:tt::/ httpa/////eeeeeeeeeeeeeeee net/ forum? forum?id-tan tj.0 2,
[13] Kaiming Ile, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 16000-16009, June 2022. doi: AALL:0/CVVPR22682222205555 2,5
URL
.eeeeieeess leee. org/ wttg//ocumentt/8799000
[14] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujic Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al. Sora: A review on background, technology. limitations, and opportunities of large vision models. arXiv preprint arXiv:240..11177 2024. URL https : http///axxvvvrrr// abs/2402 2402.1777777
[15] Demetres Kostas, Stéphanc PreaaOuulllette, and Frank Rudzicz. Bendr: Using transformers and contrastive sulf-sup learning task Iearn from massive amounts of eeg data. Frontiers in Human Neuroscience, 15:653659, 2021. doi: 10. 10.3389/fnhumm2200 200220222166999 URL htttps://dos org/10. /3399/ fnhum. 2021 653659. 2, 5, 8, 17, 24, 28, 29
[16] David Bethge, Philipp Hallgarten, Tobias Grosse-Puppeenee
Mohamed Kari, Lewis L Chuang, Ozan Özdenizei, and Albrecht Schmidt. Eeg2vec: Learning affective ceg representations via variational au- toencoders. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 3150-3157. IEEE, 2022. URL
ieee. org/ //ccumet//9955517 /994999995551999 2, 17
(17] Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Ad- vances in Neural Information Processing Systems (NeurIPS), volume 36, pages 78240-787887 78260. Curran As- sociates, Inc., 2023. URL https: //troceedg//.ooeiiing.... neuripa. cwtp.cceppppeeeeeeeeeeeeeee files/ twwes/paper/////eeeeeppaaaaaaaat 2023/filee (a030002222$55535522244420222220 Porp Conferenceeeeecccmmmm pdf. 2, 8, 17, 24, 27, 28, 29, 30, 31
[18] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for leamning generie representations with tremendous EEG data in BCL. In The Twelfth International Conference on Learning Representations (ICLR), 2024. URL https: //pp//eeeeveeeeeeeeeeeeeeeeeeeee new. / http://fommmmddddppppeee QeTTTtd.V....RT..... 2, 5, 8, 17, 19, 24, 27, 28, 29, 31
[19] Weibang Jiang. Yansen Wang, Bao liang Lu, and Dongsheng Li. NeuroLM: universal multi-task foundation model for bridging the gap between language and EEG signals. In The Thirteenth International Conference on Learning Representations (ICLR), 2025. URL ht / http:///ppeeeeeeeeeeeeeee evine .nt// formmidd I09yFt7XH7. 2, 17, 24, 31
[20] Guagnyu Wang. Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pre- trained transformer for universal and reliable representation of ceg signals. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Ad- vances in Neural Information Processing Systems, volume 37, pages 39249 39280. Curran Asso- ciates, Inc., 2024. URL attps://pot https://pproceedingss neurips wwwurips.cc/pppee wwtppcppppeppeeeeeeeeeeeeeeee files/ pttpee/papprr///2////// //lee/ 240000087eee4eebbbaaee4444080..0 Pwt.eeo0br9---aaer---e--- wwwperenoeeeeeeeeeeaeee pdf. 3, 5, 8, 17, 24, 28, 29,31
[21] Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. CBramod: A criss-cross brain foundation model for EEG decoding. In The Thirteenth International Conference on Learning Representations (ICLR), 2025. URL MPNUHgHF2s. 3, 5, 7, 8, 17, 24, 27, 28, 29, 30, 31
net/forum?id
[22] Jacob Devlin, Ming-We Chang. Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v 118633/11/119/ 1423. URL htttpe://aclanthologg. org/N199 0etg/Ni9-1423/....
[23] Chaoqi Yang. Danica Xiao, Brandon Westover, and Jimeng Sun. supersup eeg representation learning for automatic sleep staging. arXiv preprint arXiv:2110.15272 152788 2021. URL https://arxis org/abs/2110 15278. 3, 24, 27, 28, 29, 30, 31
[24] Dongwei Jiang, Wubo Li, Ruixiong Zhang, Miao Cao, Ne Luo, Yang Han, Wei Zou, Kun Han, and Xiangang Li. further study of unsupervised pretraining for transformer based speech recognition. In ICASSP 2021 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pages 6538 6542, 2021. doi: J0.1109/CCASSPP9722222222 910.: 9414539. URL https: http::///eeeeepeeeeeeee ieee. Att//bccument/4445539
12

---

[25] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers effective for time series forecasting. In The Twelfth International Conference on Learning Representations (ICLR), 2024. URL HtRLtttp::// /tttps://openreviee www.i/ewweeee/eeeeeeeeeeeeeeeeee net/forummid JePfAI8fah. ah.
[26] Yunhao Zhang and Junchi Yan. Crossforer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations (ICLR), 2023. URL https: httLhhttt////peeeee //ppr//eeeee.eeeeeeeeeeee wwwp//eeetttieeeee www..
21
[27] Noam Shazeer, *Azalia Mirhoseini. *Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The pparsely-gated mixture-offeeppert layer. In International Conference on Learning Representations (ICLR). 2017. URL https: http://ppppeeeeeeeeeeeeeeeeeeeee BickMDqlg. 6, 22
[28] Ka Man Lo, Zeyu Huang, Zihan Qiu, Zili Wang. and Jie Fu. A closer look into mixture-of-eexpers in large language models. In Lais Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Computational Linguistics: NAACL 2025, pages 4427-4447, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195.4 URL https: htttps://allatthology org/2025. Findings-naaae
6
[29] Leslie Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications. volume 1 11006, pages 369-386. SPIE, 2019. 7
[30] Yang An, Hak Keung Lam, and Sai Ho Ling. Multiclassification for eeg motor imagery signals using data evaluation-baset auto-selected regularized fhesp and convolutional neural network. Neural Computing and Applications, 35(16):12001-22)3 500521-023-08336- 9
June 2023. ISSN 1433-3058. doi: https://doi..rg// 10. 1007/
[31] Brendan Z. Allison, Jing Jin, Yu Zhang, and Xingyu Wang. A four-choice hybrid p300/ssvep bci for improved accuracy. Bruin-Compute Computer Interfaces, 1(1):17-26, 2014. doi: 10.108002326666X22133.322
17
[32] Chao Jiang, Yingjie Li, Yingying Tang. and Cuntai Guan, Enhancing eeg-hased classification of depression patients using spatial information. IEFE Transactions on Neural Systems and Rehabilitation Engineering, 29:566-575, 2021. doi: 10. 10.1109/NNSF..221..00 3059994 URL https: //wee////eeeeeeeeeeeeeeee ieee. org/
(ecument/9364655. 17
[33] Qingshan She, Chenqi Zhang, Feng Fang, Yuliang Ma, and Yingchun Zhang. Multisource associate eeg emotion recognition. IEEE Transactions on
domain adaptation for cross-subject and Instrumentation and Measurement, 72:1-12, 2023. doi: (2.10//TI..22333277777 //ieeexplore.c .ew...eeeeeee..........e...ee..e wwt.eeeeeeeeeeeeeeeeeeeeeeeeeeee org//documet//0129988 17
URL https:
[34] Zhongke Gao, Weidong Dang, Xinmin Wang. Xiaolin Hong, Linhua Hou, Kai Ma, and Matjaž Perc. Complex networks and deep learning for eeg signal analysis. Cognitive Neurodynmmics, 15(3):369- 388, 2021. ISSN 1871-4099. doi: 10.1007/s1 e11571-020-09626 17
1. URL atttpa://doi org/10. 1007/
[35] Lukas Wolf, Ard Kastrati, Martyna B Plomecka, Jie- Jic-Ming Li, Dustin Klebe, Alexander Veicht, Roger Wattenhofer, and Nicolas Langer. A deep learning approach for the segmentation of electroeneephalography data in eye tracking applications. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Machine
Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference Learning (ICML.), volume 162 of Proceedings of Machine Learning Research, pages 23912-23932. PMIR, 17-23 Jul 2022. URI. https: / http//peeeennga..c mml. ppress/v62//a/ wwtp///2l2.a...../ html. 17
[36] Jin Xie, Jie Zhang, Jiayao Sun, Zheng Ma, Liuni Qin, Guanglin Li, Huihui Zhou, and Yang Zhan. A transformer- based approach combining deep learning network and spatial-tempont information raw eeg classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30:2126-211 213662116222220 2022. URL https: /ttp://eeeeeeee //te//eeeeeeeeeeee// leee. org/ 99844477.9/9444999 17
doi: 10.1 202.TNS.....22.111422
[37] Soyiba Jawed, Ibrahima Faye, and Aamir Saeed Malik. Deep learning based assessment model for real-time identification of visual learners using raw eeg. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 32:378-390, 2024. doi: 10. 00.1109/TSSRE.2224..333 169444 URI https:
basedd
//ieeexplore.c ieee. org//ocument/10087700
17
[38] Aaqib Saeed, David Grangier, Olivier Pietquin, and Neil Zeghidour. Learning from heterogeneous eeg signals with differentiable channel reordering. In ICASSP 2021 2021 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), pages 1255 1259,, 2021. doi: http:///di.org/1..111
ICASSP39728. 001.99778..222.9999 202.2022199 1913712. 17
13

---

[39] Aaqib Saeed, David Grangier, and Neil Zeghidour. Contrastive learning of general-purpose audio rep- resentations. In ICASSP 2021 2021 IEEE International Conference Acoustics, Speech and Signal Prucessing (ICASSP), pages 3875-3879, 2021. doi: https:://ooi.or//1... 109/CASSP33972. 94115588. 17
[40] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang. et al. Deepscek-rl incentivizes reason- ing Ilms through reinforcement learning. Nature, 645:633-638, 2025. doi: 10. 1038/6441566-0055099422 URL https: / www. nature. cww/ www.m//tticeee/eeeeeeaee
025 09422-2. 17
[41] An Yang, Anfeng Li. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, al. Qwen3 technical report. arXiv preprint arXiv:2505. 2025. 17, 21
[42] Ruibin Xiong, Yunchang Yang. Di He, Kai Zheng, Shuxin Zheng. Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tic-Yan Yan Liu. On layer normalization in the transformer architecture. In Proceedings of the 37th International Conference Machine Learning (ICML), ICML' 20. JMLR.org, 2020. URL
ptrpss///procedinggs mlr. wwwpps/wssee/eepeeepepppppa.aale www111x/xooogegee wwg.o0boo.bbbooo/oo html. 21
[43] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint 1rXiv:1606.00415. 2016. URL https: /ttpe////aaavaaeeeeeeeeeeeeeeeee org/ abs/abbs//0 08415. 21
[44] Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. URL https:
// //arxivv . org/ 2002 05202. 21
[45] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09220 2023. URL https: 09288. 21
arg/aba/2307
[46] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint wiXiv:2407.21780 2024, URL https: //axxiv.//////////aa///a/ar org/ 2t1.///777.2........ 21
[47] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, al. Qwen2 technical report. arXiv preprint arXiv:2407. 2024. URL bttps://arx:ar org/ abs////a///////// 10671.
[48] Qwen, An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15113 2025. URL
https://arx:: /aarivv///xvvvv//a org/ abaaabs//211 15115.
[49] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, al. Deepseck-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024. URL https://arxii org/abs/ ab//ab2/2412 19437. 21
[50] Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In The Twelfth International Conference on Learning Representations (ICLR), 2024. URL /ttpes///operreieet net/
1 forum?id forum?2d-=2222XyhhE... 21, 23
[51] Zhengzhuo Xu, Ruikang Liu, Shuo Yang, Zenghao Chai, and Chun Yuan. Learning imbalanced data with vision transformers. In 2023 IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1.5793115803, 2023.
doi: 10.1 (D0/CCVPR52729.2223.0055
22
[52] Vernon Lawhern, Amelia Solon, Nicholas R Waytowich. Stephen Gordon, Chou Hung. Brent Lance. Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces. Journal of Neural Engineering, 1555):056113, jul 2018. doi: http:://doi..rg/O..oo 111111/////11552//aaa/a 24, 27, 28,
29, 30, 31
[53] Yonghao Song, Qingqing Zheng, Bingehuan Liu, and Xiaorong Gao. Eeg conformer: Convolutional transformer for eeg decoding and visualization. IEEE Transactions on Neurul Systems and Rehabilitation Engineering, 31:710-719, 2023. doi: https://doi.org/10. wtw.:////......0/TTTE.EEE.. 2002.2S5 24, 27, 28, 29, 30, 31
[54] Jin Jing, Wendong Ge Shenda Hong, Marta Bento Fernandes, Zhen Lin. Chaoqi Yang. Sungtae An. Aaron F Struck, Aline Herlopian, loannis Karakis, et al. Development of expert-level classification of seizures and rhythmic and periodic patterns during eeg interpretation. Newrology, 100(17)ee150--e)) c1762, 2023, doi: https://doi.org/100 http://0....2.2.....000000000000
24, 27, 28, 29, 30, 31
14

---

[55] Wei Yan Peh, Yuanyuan Yao, and Justin Dauwels. Transformer convolutional neural networks for automated artifact detection in scalp eeg. In 2022 44th Annual International Conference of the IEEE Engineering Medicine and Biology Suciety (EMBC). pages 3599-3602, 2022. doi: https://doi.or//1... 10000////00.010000 992299. 24, 27, 28, 29, 30, 31
[56] Arman Savran, Koray Ciftci, Guillame Chanel, Javier Cruz Mota, Luong Hong Viet. Bülent Sankur, Lale Akarun, Alice Caplier, and Michele Rombaut. Emotion detection in the loop from brain signals and facial images. In Summer Workshop on Multimodal Interfaces (eINTERFACE 2006), pages 69-80, 2006. URL https: //ww://wwwwwa/ isca- ar chive.ee..... org/ einterfaccccccccm 2006 swwran06..0.00.. einterface html. 25
[57] Matthew D. Luciw, Ewa Jarocka, and Benoni B. Edin. Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and friction. Scientific Data, 1(1):140047, 2014. ISSN 2052-4463. doi: 10. 1011000data. 2014.47. URL https:://///. //tp::://doooooooo org/ 1038/ sdata. 2014 47. 25
[58] Margaux Perrin, Emmanuel Maby, Sébastien Daligault, Olivier Bertrand, and Jérémie Mattout. Objective and subjective evaluation of online emoor correction during P300-based spelling. Advances in Human- Computer Interaction, 2012:578295, 2012. doi: 10.1155/202/5578295 URL wiley. www//ooooo//oo/ocoo/oooooooooooe 21015552////2999 25
[59] G. Schalk, D.J. McFarland, T. Hinterberger, N. Birbaumer, and J.R. Wolpaw. Bci2000: general-purpose brain-computer interface (bci) system. IEEE Transactions on Biomedical Engineering, 51(6): 1043, 2004. doi: 10. (19/TBME.2004.827022 URL hittps:/iieeexplors leee. www..eeee......eeeeeeeee....eeee htt////oocmmm//////n 25
[60] Logan Trujillo. Raw EEG Data, 2020. URL. htRps:ttss//doi.ooo http::///ooooooo/////// http://orrr//...11////////////1/
25
[61] Logan T. Trujillo, Candice T. Stanfield, and Ruben D. Vcla. The effect of clecrrenceehallogram (ceg) reference choice on information-theorrtic measures of the complexity and integration of eeg signals. Frontiers in Neuroscience, 1 11:25,, 2017. doi: 00.3389//inns.201.00433,,, URI https::/doii org/ 10.
3389/fminng 2017 00425.0255.. 25
[62] Wei-Long Zheng and Bao-Liang Lu. Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks. IEEE Transactions Autonomous Mental Development, 7 (3):162-175, 2015. doi: 10.1 (099/AA...005.2433
URL https: //doi. 111/////..///TAMDD.. 2015.
2431497. 25
[63] Wei-Long Zheng, Wei Liu, Yifei Lu, Bao-Liang Lu, and Andrzej Cichocki. Emotionmeter: multimodal framework for recognizing human emotions. IEEE Transactions on Cybernetics, 49(3):1110) 10- 1100-122,,,0001111111 2019. doi: 10.1 1119000C../20010 8.27871777777 166.. URL httph:
org/10. 10. 1109/TCYB 211182797176.. 25
[64] Wei Liu, Wei-Long Zheng. Ziyi Li, Si- Yuan Wu, Lu Gan, and Bao- BaonLiang Lu. Identifying similarities and differences in emotion recognition with ceg and eye movements among chinese, german, and french people. Journal of Neural Engineeringg 19(2):22012, 2, mar 2022. doi: 10. 00.100//711120 241-2552//nc2 URL https://dod org/// /0///11...0 1018/008111111 1777411255///cce ac5cdd. 25
[65] Paolo Detti, Giampaolo Vatti, and Garazi Zabalo Manrique de Lara. Eeg synchronization analysis for seizure prediction: A study on data of noninvasive recordings. Processes, 8(7), 2020. ISSN 2227-9717. doi: 00.33000 URL https:/ //doi. hww.doi.oo...////eeeeeeoooeeeee.
org/10 110. p3000// 25
[66] Mastaneh Torkamani-AAzin Sumeyra Demir Kanik, Serap Aydin, and Mujdat Cetin. Prediction of reaction time and vigilance variability from spatio-spectral features resting-state eeg a long sustained attention task. IEEE Journal of Biomedical and Health Informatics, 24(9):2550- 2550-258,, 2020, doi: 10. 109/JBHL. 2020.2980056 URL https: / /dooi org/10. 10. 1111111...0 1119//1B1..0000 2020.2980056. 2980056. 25
[67] Louis Korezowski, Martine Cederhout, Anton Andreev, Grégoire Cattan, Pedro Luiz Coelho Rodrigues, Violctte Gautheret, and Marco Congedo. Brain Invaders calibration-less P300-based BCI with modulation of flash duration Dataset (bi2015a), 5a). Research report, GIPSA-lab, July 2019. URL httas://hal. science/hal- 02172347. howeehh21101122.7...... 26
[68] G. Buckwalter, S. Chhin, S. Rahman, I. Obeid, and J. Picone. Recent advances in the tuh eeg corpus: Improving the interrater agreement for artifacts and epileptiform events. In 202/ IEEE Signal Processing in Medicine and Biology Symposium (SPMB). pages 1-3, 2021. doi: 10. 00.100//PPBB52240200009222222229 URL https: / /doi. org/ 10. 11w11 /MA.1////MM22200000. 20....99..2.... 26
[69] L. Veloso. J. McHugh, E. von Weltin, S. Lopez. Obeid, and J. Picone. Big data resources for eegs: Enabling deep learning research. In 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB). pages 1 1-3, 2017, doi: 10.1109//MBB.00111 2017 82570444 URL https: //doii htti/di..ogg///// org/ /0. 100./11099PP SPMB.
2017. 8247044. 26
15

---

[70] Vinit Shah, Eva von Weltin, Silvia Lopez, James Riley McHlug,, Lillian Veloso. Meysam Golmohammadi, lyad Obeid, and Joseph Picone. The temple university hospital seizure detection corpus. Frontiers in Neuroinformaties. 12:83, 2018. ISSN 1662-5196. doi: 10.3389/fiif22208800005... URL https: / //doi. //doi.org//tl 10. 3389/fninf 2018.00083. 26
[71] E. von Weltin. T. Ahsan, V. Shah, D. Jamshed, Golmohammadi, Obeid, and Picone. Electroen- cephalographic slowing: A primary source of error in automatic seizure detection. In 2017 JEEE Signal Pro- cessing in Medicine and Biology Symposium (SPMB), pages 1-5, 2017. doi: 10. 1009//PB..01..8255555
URL https: /tt../oiiioo///o org/ 10. 1109/SPMB 201. 8227018. 26
[72] Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mictus, George B Moody, Chung Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. Circulation, 101 (.11//.CIR.11.. 101..3.e21..... 26, 27
(23)xc215-2220, 2000.
doi: https://doi.org//
[73] Igor Zyma, Sergii Tukacv, Ivan Seleznov, Ken Kiyono, Anton Popov, Mariia Chemykh, and Oleksii Shpenkov. Eleetrenneehaalggaam during mental arithmetic task performance. Data, 4(1), 2019. ISSN 2306-5729. doi: https://oi..or// https//dbb.or/103.339////oooo 0.3390/datt440100010 URL https: / //ww///ww/////// mmpii wtt.:///-////////////// 1/14. 26,27 27
[74] Wajid Mumtaz. MDD Patients and Hcalthy Controls EEG Data (New). figshare, November 2016. doi: www./00//....///////eee 4244177..2 URL https: com/ http///ricclessstseee/////
EEG Seata_lew//22000 4244171. 26, 27
[75] Luca Citi, Riccardo Poli, and Caterina Cinel. Documenting, modelling and exploiting p300 amplitude changes due variable target delays in donchin's 's speller. Journal of Neural Engineering, 755)0056006.
sep 2010, doi: https://di..rrg 10. 1088/1741-111 25558//77775507770550000666 26, 28
[76] B. Kemp, A.H. Zwinderman, Tuk, HLA.C. Kamphuisen, and J.J.L.. Oberye. Analysis of sleep-dependent neuronal feedback loop: slow-wave microcontinuity the eeg. IEEE Transactions on Biomedical Engineering, 11855 1194, 2000. doi: https://doi.orr//000 wttp////...11000.. 109/1008879988. 26, 28
16

---

A Related Work
Existing literature on EEG-based neural decoding can be broadly categorized according to the evolution from EEG setting-specifer models to generalizable foundation models, as outlined below:
A.1 Decoding EEG Signals with Task- and Setup-Specific Models
EEG decoding has evolved from classical BCI pipelines, that develop computational approaches based on the well-known EEG signatures (e.g.. P300, event-related (de)synchronizatioo, Steady-
Visual Evoked Potential (SSVEP), [31]) to feature engineering and statistical classifiers such as, linear discriminant analysis (LDA), or support vector (SVM), random forest, common spatial pattern (CSP) [5, 32], more sophisticated machine learning and deep learning models tailored for specific tasks and recording setups. These models are often trained on specific datasets and optimized for particular paradigms, such as motor imagery [32], gait recognition [5], emotion recognition [33], and seizure detection [7], under fixed channel layouts, devices, or recording conditions, building upon prior knowledge of EEG characteristics. Deep learning approaches such as CNN [5], RNN [6], GNN [7],
Transformers [8], and their hybrids have been widely applied to model EEG's complex spatiotemporal dynamics using spectrogram, time-frequency representation, and brain connectivity [34].
Recent work has shifted end-to-end learning paradigms that learn representations directly on raw EEG signals [35, 36, 37], reducing the dependence on handcrafted preprocessing. However, these methods still struggle with generalization across subjects, devices, or experimental conditions and require retraining when used different settings.
A.2 Unified EEG Signal Decoding with Pretrained Foundation Models
Motivated by the success of foundation models in natural language processing (NLP) [9, 10], recent trends in EEG research focus developing large-scale, pretrained neural architectures that scale up and generalize across diverse EEG tasks, conditions, experimental setups, and subjects. These models are typically pre-trained using self-supervised learning on unlabeled EEG data from varied sources, enabling the extraction of task and subject-invariant representations that can be fine-tuned for a wide range of downstream EEG applications. Broadly, three main lines of approaches have emerged: (i) modeling EEG as a multivariate time series using transformer based architectures adapted from the speech or time-series domains, such as BENDR [15] and EEG2Vec [16]: (ii) tokenizing continuous EEG signals into discrete tokens to enable large-scale pretraining analogous to language modeling, as LaBraM [18] and NeuroLM [19]; and (iii) preserving EEG's spatiotemporal structure using specialized architectures that consider both between EEG channels and temporal dynamics from diverse datasets through neural architectural design, exemplified by recent EEG-based FMs, BIOT [17], EEGPT [20]. and CBraMod [21]. These FMs reduce the need for task-specific calibration and enhance robustness and scalability in real-world BCI and EEG-based applications. CHARM [38] addressed inconsistent input channels through learnable channel-level masking approach, and COLA [39] employed contrastive learning rather than masking- based self-supervised pretraining large-scale audio data. These related works offer valuable complementary perspectives on enhancing EEG modeling.
While these pioneering FMs showed early promise, few important EEG characteristics are not considered in the current FMs, such as the physical three-dimensional arrangement of electrodes, diverse EEG patterns and intensity across diverse datasets, and brain regional features are not fully taken into account. Moreover, recent successes of Mixture-of-Expert (MoE) [40, 41] based foundation models have demonstrated impressive scalability and generalization in fields like LLMs, by designing a larger network (for more parameters) with sub-networks that are only activated by relevant task (for less computation); however, their application and potential benefits remain largely unexplored in EEG decoding. This work attempts to fill the research gap for developing a robust BEG-based FM.
17

---

B Additional Results and Analysis
This section continues from Section 3.4 of the main text and provides further results and analysis.
Table 8: Comparison between different masking strategies (masking ratio: 50%).
Masking Strategy
Bal. Acc.
BERT-style random masking
0.6320
Amplitud-awaae masking
0.6845
Ablation of Masking Strategies We compare and analyze our proposed amplitude-awar masking pretraining (AAMP) strategy with BERT-style random masking in Table 8. We directly evaluate the pretrained representations by performing downstream classification task using Support Vector Machine (SVM)² aiming assess the quality of the learned representations obtained during the pretraining stage. Our amplitude-basel masking improves 5.25% compared to random masking, proving the effectiveness of masking based on amplitude. Figure 7 illustrates the training loss between to strategies, After 1000 steps, random masking has lower reconstruction loss, demonstrating that random masking is better for learning reconstruction. This implies that the model is better at interpolation under random masking, but ultimately performs poorly in downstream classification tasks. Refer to Figure 18, 19 and 20 of Appendix for more visualization comparisons.
Random
A
235
244
8
1.4
0.8
2000
3600
Figure 7: Training loss between different masking strategies.
Analysis of Masking Ratio We then also tested the performance of AAMP different masking ratios. As Table shows, SVM works best at a masking ratio of 50%. Figure 17 of Appendix G shows the loss curves for different masking ratios.
Table 9: Performance of NEURIPT with different masking ratios.
Masking Ratio (%)
Bal. Acc.
30
0,5523
40
0.6321
50
0.6845
60
0.5596
70
0.5254
"For the classification task in Appendix B, we selected the TUEV due to its complexity and variety, consisting of six distinet classes. To manage the computational complexity of SVM, which scales quadratically with the number of data points, randomly sampled 5000 data points from the training and 500 data points from the test set, using fixed random seed (520) to ensure reproducibility. Hyperparameters were optimized via grid search with 5-fold cross-validation on the sampled training data, and the best-performing combination was subsequently evaluated on the test set. Detailed ranges of hyperparameters explored during the grid search are presented in Appendix D.I and Table 14.
'We use smaller models for quick tests. Specifically, we set dmonte to 256, expert hidden FFN FFNeaperd to 128, shared expert hidden FFN to 256. The rest of the parameters remain the same as the original pre-trained model.
18

---

Table 10: Comparison between different masking strategies across diverse downstream tasks.
Datasets
Masking Strategies
Bal. Acc.
Cohen't Kappa
Weighted
Random Masking
67.35
67.28
83.37
TUEV
AAMP (Ours)
67.61
69.70
84.28
Random Masking
84.38
77.89
90.98
MethalArithmetic
AAMP (Ours)
86.46
78.27
91.11
Random Masking
99.84
99.84
Mumtaz2016
AAMP (Ours)
98.03
99.81
99.79
Random Masking
33.89
17.07
33.82
SEED-V
AAMP (Ours)
41.04
26.29
41.58
PhysioP300
Random Masking
65.64
28.99
72.75
AAMP (Ours)
67.31
34.26
76.83
Random Masking
69.68
76.82
87.11
Sleep-EDFx
AAMP (Ours)
70.47
77.57
87.39
Further Ablation of Masking Strategies We analyzed the classification performance in the preceding using SVM classifiers trained directly on representations generated under different masking strategies. Table 10 compares the downstream task performances between BERT-like random masking and our proposed AAMP, further demonstrating the advantages of our method.
Table 11: Different positional encoding strategies across various tasks. Alternative encoding strategies are included: trigonometric functions and 1D learnable embed- dings employed by vanilla Transformer [9], and 2D learnable embeddings introduced in LaBraM [18].
Datasets
Encoding Strategies
Bal. Acc.
Cohen's Kappa
Weighted F1
Trigonometric Functions
67.72
68.85
84.03
Learnable ID
64.78
67.92
83.13
TUEV
Learnable 2D
63.81
66.63
82.57
3D PE (Ours)
68.94
71.55
85.17
Trigonometric Functions
74.65
71.84
83.80
Learnable ID
71.53
74.23
83.17
MentalArithmetic
Learnable 2D
71.52
64.81
80.83
3D PE (Ours)
75.69
74.37
86.83
Trigonometric Functions
96.56
99.59
99.61
Learnalle ID
96.63
99,21
99.23
Mumtaz2016
Learnable 2D
95.56
99.55
99.54
3D PE (Ours)
97.07
99.83
99.84
Trigonometric Functions
35.03
18.58
35.08
Learnable ID
35.52
18.81
34.34
SEED-V
I Learnable 2D
37.40
21.54
37.34
3D PE (Ours)
39.34
23.93
39.67
Ablation of Positional Encoding Strategies Table 11 presents ablation studies on different position encoding methods. Our proposed 3D PE consistently outperforms both the trigonometric function embedding and the learnable ID and 2D embeddings. underscoring its effectiveness in explicitly leveraging physical electrode positions inherent in EEG data.
Table 12: Low resource scenario. Metries are shown percentages relative to the full-data baseline.
Datasets
Data Used
Bal. Acc.
Cohen's Kappa
Weighted F1
1%
64.16
72.44
88,08
75.39
81.78
91.67
TUEV
10%
80.30
88.64
95.75
100%
100.00
100.00
100.00
1%
80.97
83.86
91.33
5%
86.01
79.82
90.41
Sleep-EDFx
10%
92.51
92.74
96.13
100%
100.00
100.00
100.00
19

---

Low Data Scenario The performance of NEURIPT under low-resource scenarios is presented in Table 12. Our approach achieved notable classification accuracy, maintaining performance in the range of 80% 90% with merely 10% of the original data, and even with an extremely limited 1% of data, performance remains competitive at 64% 80%. These results further confirm the robustness of our pretraining strategy and underscore the model's 's potential in few-shot and low-resource settings.
For the ablation studies in Table and 7 in Section 3.4, due time and resource constraints, the results presented are based on models trained from scratch, without the time-intensive pre-training stage.
20

---

(Z.a ((imee)) t.me. C Additional Methodological Details
C.1 Backhone Architecture
We adopt Crossformer [26] as our backbone architecture, which leverages the TSA (Two-stage attention, including cross-time attention and cross-dimension attention) module hierarchically capture alternating temporal and spatial dependencies. Additionally, the hierarchical structure of TSA, especially deeper layers with coarser granularity, significantly reduces sequence lengths and computational demands, enabling efficient resource utilization during large-scale pre-training. Furthermore, we replace the Post-LN with Pre-LN [42] and switch the activation function from GELU [43] to SwiGLU [44] for a more stable training process, aligning with mainstream large language models such as the Llama [45, 46]. Qwen [41, 47, 48], and Deepseek [40, 49] series. The mathematical formulation of our modified backbone is as follows:
After embedding, Senc E TxDdmoodd is the encoder block ENC's input, where T and D are the number of time steps and electrode channels, respectively. Consistent with and Z' the input of the layer / after merging. Thus
Temporal Stage
Section 2.2, Z'-1 the output of the layer at the first layer, Z = Senc, For convenience, in the following, we use Zi to denote the vectors of all dimensions time step i, L for those of all time steps in dimension d. In the temporal stage, we directly apply multi-head self self-attention MSA to each dimension:
Z time, LayerNorm ( (
(15)
timed LayerNorm 2d
Z
+ MSA
(timed
stimed 2iime.) !
2
(16)
:.
Z time. PMOE\"
(17)
:,
where 1 d D and LayerNorm denotes pre-layer normalization [42], MSA Q, K, V) denotes the multi-head self-attention layer [9] where Q, K, V serve as queries, keys and values, PMoE denotes the Progressive Mixture-of-Expee Exprresss introduced in Section 2.2, and fimee denotes the residual connection result from Equation but without LayerNorm. All dimensions (1 S d S D) share the same MSA layer. timed denotes the output of the cross-time stage.
Spatial Stage
Similarly to the temporal stage, we also apply the above architecture to each time step:
dim, LayerNorm time.
(18)
LayerNorm
Z
MSA
Zddimd
(19)
z' "MMEE"' (1)
(Zdim).)
+ Z dim.l
(20)
where 1 < S T and Z.dimu denotes the residual connection result from Equation 19 but without LayerNorm. Finally, Z the output of the entire TSA layer I. Notably, in practice, at the first layer, we split Equation into the following two parts embed the temporal and spatial information separately:
time.1 LayerNorm
[ (P(()
(21)
Z dim. LayerNorm
Z time,1 + PE()))
(22)
Equations 21, 22 correspond Equations 15, 18. respectively, and represent special cases at the
first layer. Here, multi-head self-attention MSA is implemented through FlashAttention-2 by [50] to achieve faster speeds.
Hierarchical Attention Modules In each encoder layer (except the first layer), every adjacent my vectors are merged produce representations progressively coarser temporal scales. Then, TSA layer captures dependencies at this new scale. Formally, the encoder process ENC for layer 7 is defined
1 == 1: zenc. e
(23)
1 1>1:
Zene. M
Zened enn
(1-1mm+t.d Z Zene,
Zene.
(1-1mm+22d
21

---

where Si
Ti
d < Zene! denotes the output of the /-th encoder layer, M
mi
Rdmnddl Xmidmadde is a learnable matrix for dynamic segment merging, "." denotes the concatenation operation, Ti-1 denotes the number of segments in layer - 1, and if not divisible by my padding is applied to Zenell1 accordingly.
C.2 Additional Masking Strategy
Besides Amplitue-AAare Masking Pretraining (AAMP), we incorporate an additional basic masking strategy proposed BERT [22]: 80% of the selected tokens are replaced with the [mask] token, 10% are replaced with randomly sampled embedding from a predefined set S to mitigate overfitting, and the remaining 10% are left unchanged to prevent the model from overly relying on the [mask] token.
C.3 Mixture-of-Experre
We incorporate an auxiliary loss consisting of an importance loss c()) importance and a hal-balanning
(4) loss hhlancc of layer proposed by Google Brain [27]. This auxiliary loss promotes equitable expert allocation, improving model efficiency and generalization:
Laux
importance
+
(
balance
(24)
C.4 Balanced Binary Cross Entropy
We applied Bal-BCE [51], logit-adjusted balanced binary cross-entropy loss that has achieved promising results long-tailed image datasets. Here, we explore similar approaches for imbalanced BCI downstream datasets. Let Ty, Tly,NN /N represent the class distribution for y;. where the bias term of the logit Zy; is given by B log "y; log(1 Ty,). Based on this logit bias adjustment, the loss function we adopt during the fine-tuning stage as follows:
LBa--BCE = Σ we [1(y:) -log a (zy. + By.e) + +(1 (1
1(y,)) log (1 o (zy. + By: )
N:EC
(25)
where 1 is if the condition is true and ((x) 11/(1 + e-" indicates the sigmoid operation.
22

---

D
Implementation Details
D.1 Hyperparameters and Settings
Table 13: Hyperparameters for NEURIPT pre- mre-training.
Hyperparameters
Settings
Channels
EEG sample
Data length
Dynamic masking ratio
[20,35,50]
Input dimension
Output dimension
Model Architecture
Feed-forward dimension Heads Merge layers
768
[1,4.1,2,1,2]
Hidden dimension of expert Shared expert hidden dimension
512
768
Encoder expert
[0.2,2.4.4.6]
Decoder expert
[0.0,0,0.0.0]
PMoÉ Configuration
Cross expert
[0,0.0.0.0.0]
Top-4%%
0.5
Noise
0.001
W importance
0.008
Auxiliary loss weight
0.8
Epochs Batch size
Dropout
0.1
Optimizer
AdamW
Maximum learning rate
3e-4
Diy factor
25
Final div factor
le4
Warm up ratio
0.15
Pre-training
Adam (81,8) 82)
(0.9, 0.98)
Weight decay Scheduler
5c-3
CosineAmeellnggL.
Cosine cycle epochs
40
Minimal learning rate
le-5
Clipping gradient
100
Weights init
Kaiming normalization
Seed
Table 14: SVM grid search hyperparameters (ablation study only).
Hyperparameters
Settings
kemel
['lincar", "rbf"]
[0.1, 1, 10, 100]
gamma (only rbf) ["scule", "auto", 0.001, 0.005. 0.01. 0.05, 0.1. 0.5) tol class weight
[]e-4, le-3]
[None, "balanced")
Hyperparameters of pre-training are reported in Table 13, while Table 14 presents the parameters used in SVM grid search discussed in Appendix B. For complete details of the settings used downstream datasets, please refer to Appendix E.2. During fine-tuning, we conducted grid search over the hyperparameters within the ranges specified in Table 15, Owing to the inherent variability and heterogeneity of EEG datasets, we observed that, for certain datasets, reinitializing model weights was essential for NEURIPT to effectively adapt and fine-tune downstream tasks (i.e.. SEED-V. and BCIC-IV-2A). This behavior is likely attributed to a representational mismatch between the pretrained model's and the target task. To address this issue, we performed reinitialization when validation performance showed early saturation during fine-tuning. By resetting the model parameters, this enables the network to escape suboptimal representational basins. Benefiting from distributed data parallelism and FlashAttention 30 hours. The subsequent fine-tuning on downstream tasks typically required only a few minutes to several hours, depending the scale of the specific datasets involved.
[50], the entire pre-training procedure was completed within
23

---

Table 15: Hyperparameter tuming ranges used for diverse BCI tasks.
Hyperparameters
Settings
Dropout
0-0.5
Effective batch
[4, 64]
Train Epochs
10 150
Maximum leamning
10-d 10
Div factor
1-5
Final div factor
le5
Warm up ratio
Class layers
1-6
Merge layers
21
Encodor expeet
1100000 |00 (002446))
Auxiliary loss weight
Noise sid
0.001 0.1
imguriance
0,001 0.1
BCE Frocze opochs
0-0.3 0 -0
Table 16: The number of parameters in different methods.
Activated
EEGNet [52)
0.005M
EEGConformer [53]
SPaRCNot [54]
0.79M
ContraWR [23]
1.6M
CNN-Transformer [55]
3.2M
IFCL [6]
2.4M
ST-Transformer
BIOT peetrsa
[17]
3.2M
LaBraM Base
18]
5.8M
LaBraM-Huge ucuse
188
369.0M
S anoo 20
25.0M
CBraMod
4.2M
Neurol a
[19]
1696.0M
NEURIPT (Ours)
D.2 Baselines and Metries
Baselines We compare NEURIPT against eight non-foundation methods BENDR [15], EEG-
Net [52], EEGConformer [53], SPaRCNet [54], Contra [23], CNN-Transformer [55], FFCL [6].
and ST-Transformer [8] -and five foundational model baselines: BIOT [17], LaBraM [18],
EEGPT [20], NeuroL [19], and CBraMod [21]. For any model without published downstream
results, we directly followed the results reported in CBraMod [21] and EEGPT [20]. Strietly fol-
lowing the settings of CBraMod, we fine-tune BIOT and LaBraM based their open-source code
and pre-trained weights, unless their experimental results have already been reported the original papers. Table 16 demonstrates the number of activated parameters for different baselines.
Metrics For binary classification tasks, we report Balanced Accuracy, Area Under the Preci- RicnlRecall Curve (AUC-PR), and Area Under the ROC curve (AUROC), For multi-class tasks, use Balanced Accuracy, Cohen's Kappa, and Weighted score. Note that for PhysioP300, we replace AUC-PR with Cohen's Kappa to match the evaluation protocol of our EEGPT baseline [20]. More details about the metrics we used:
Balanced Accuracy: the average recall across all classes, mitigating class imbalance;
AUC-PR: the area under the precision-recall curve;
AUROC: the area under the receiver operating characteristic curve;
Cohen's Kappa: measure of inter-rater agreement that accounts for chance, computed based on observed versus expected agreement in confusion matrix;
Weighted F1 Score: the harmonic mean of precision and recall, weighted by class frequency.
24

---

E
Dataset Description
E.1 Pre-training Datasets
Table 17 presents the statisties of the datasets used for pre- pre-training NEURIPT.
Table 17: Overview of pretraining datasets.
Datasets
Subject
Total Time
Emobrain
16
4.94h
Grasp and Lift BEG challenge
12
11.72h
Inria BCI Challenge
26
29.98h
Movement/Inagery
Raw EEG Data
58
34.35h
Resting State EEG Date
22
3.04h
SEED-Series
46
166.75h
Siena Scalp EEG Dutabuse
14
30.47h
SPIS Resting Dataset
0.83h
Target Versus Non-Turge
50
16,00h
TUAR
213
92.22h
TUEP
200
591.22h
TUSZ
315
1,138.53h
TUSL
E.1.1 Description of Pre-training Datasets
multimodal emotion dataset contains fNIRS and 64-channel EEG recordings at sampling rate of 1024 Hz. EEGs are recorded by the Biosemi Active acquisition system, including 16 subjects. Emotional responses were induced using subset of the IAPS dataset.
Emobrain [56]
Grasp and Lift EEG challenge [57] A dataset containing 32-channel EEG recordings at sampling rate of 500 Hz. It includes data from 12 subjects performing grasp-and-lift (GAL) trials. EEG signals were recorded using an EEG cap in conjunction with a BrainAmp EEG signal amplifier.
Inria BCI Challenge [58] A P300-Speller dataset that includes 26 subjects with 56-channel EEG recordings sampling rate of 600 Hz using Ag/AgCI EEG sensors (VSM-CTF compatible system).
EEG Motor Movement/Imagery [59] A motor imagery dataset comprises EEG recordings from 109 subject performing baseline tasks (eyes-open and eyes-closed), motor movement and motor imagery (both fists both The EEGs were collected using 64 channels sampling rate of 160 Hz with the BCI2000 system.
Raw EEG Data [60] A dataset containing 64-channel EEG recordings sampled at 256 Hz, recorded during the reported Information- Integrationnn categorization task and the reported multidimensional Rule-Based categorization task.
Resting State EEG Date [61] An EEG dataset (64 channels, 256 Hz) containing 22 subjects for a resting task of 8 mins with 4 mins of eyes closed and 4 mins of eyes open using active Ag/AgCI electrodes either mounted in BioSemi electrode cap via freestanding electrodes.
SEED-Series [62, 63, 64] A series of datasets, including SEED, SEED-IV, LEED-GER SEED- FRA. All EEG signals were recorded from 62 channels at a sampling rate of 1000 Hz with the ESI NeuroScan System in response videos.
A database consists of 31 31-channel EEG recordings from 14 patients, collected at a sampling rate of 512 Hz using EB Neuro and Natus Quantum LTM amplifiers, along with reusable silver/gold cup electrodes.
Siena Scalp EEG Database [65]
SPIS Resting State Dataset [66] A dataset including recordings from 10 subjects, with 2.5-minute EEG segments collected in both eyes-closed and eyes-open resting states, prior to 105-minute session of Sustained Attention to Response Task (SART) using fixed-sequence and varying ISIs. Monopolar EEG activity (64 channels, 2048 Hz) recorded using 64 Ag/AgCI active electrodes.
25

---

Target Versus
[67] A P300 dataset including 32-channel EEG signals at a sampling
rate 512 Hz from 50 subjects.
TUAR [68] This subset of TUEG contains EEG recordings annotated with different artifacts, recorded from 23 channels at sampling rate of 256 Hz.
TUEP [69] This subset of the TUEG comprising EEG recordings from 100 subjects with epilepsy and 100 subjects without epilepsy, as determined by a certified neurologist. The EEG was recorded using 19-23 channels at sampling rate of 256 Hz.
TUSZ [70] This corpus contains EEG signals that have been manually annotated for seizure events (including start time, stop time, channel, and seizure type). The EEG recorded using 19-23 channels at sampling rate 256 Hz.
TUSL [71] This another subset of the TUEG containing annotations of slowing events, recorded from 23 channels at 256 Hz. This corpus has been used to study common error modalities in automated seizure detection.
E.1.2 Preprocessing of Pre-training Datasets
In preprocessing, 0.1-30 Hz band-pass filter was applied to suppress low- and high-frequency noise, followed by a notch filter to eliminate powerline interference. Data were resampled to Hz and segmented into non-overlapping 4-second windows.
To further improve data quality, we removed samples with any signal exceeding 100 μV absolute amplitude, or with values consistently below μV or above 3 μV across all time points in any channel, as these likely reflect artifacts flatlining. Subsequently, data were normalized using A-law companding (from digital signal processing) with A 0.25 was used to adjust the dynamic range all EEG signals to ensure consistent scaling. After pre mee-rocesssig, we retained 2,219,455 EEG segments, totaling over 2,100 hours of clean data for pre-training
E.2
Downstream Datasets
E.2.1 Description of Downstream Datasets
Mental Stress Detection This task focuses on identifying an individual's stress level using EEG signals. The MentalArithmetic dataset [72, 73] contains EEG recordings from 36 subjects of varying genders and ages, collected both before and during the performance of mental arithmetic tasks.
II. Mental Disorder Diagnosis This task aims categorize mental health states based on EEG activity. The Mumtaz2016 dataset [74] comprises EEG recordings from 34 patients diagnosed with major depressive disorder and 30 healthy control subjects.
III. P300 Task This task involves detecting the P300 wave, an event-related potential reflecting cognitive processes such attention, stimulus evaluation, and target recognition. The PhysioNetP300 dataset [75] provides EEG recordings commonly used for benchmarking P300-based brain computrr interface studies.
IV. Sleep Staging Detection This task aims to classify sleep stages based on polysomnographic EEG recordings. The SleepEDFx dataset [76] contains 197 whole-night recordings and widely used as benchmark for automated sleep staging.
V. Emotion Recognition This task concerns the detection and interpretation of emotional states from EEG data. The SEED-V dataset [3] provides EEG recordings collected while subjects watched emotionally evocative videos.
VI. Motor Imagery Task This task focuses on decoding motor imagery actívities from EEG signals. The BCI Competition IV-2A dataset [4] consists of EEG recordings from healthy subjects performing four types of motor imagery tasks.
26

---

VII. Abnormal Detection This task involves identifying abnormal neural patterns in EEG data. We followed CBraMod [21], TUAB [2] was chosen as the downstream dataset for this task. The TUAB dataset [2], a large-scale clinical EEG corpus, provides recordings annotated for abnormal events and has been widely used for automated abnormality detection.
VIII. Event Type Classification This task focuses on categorizing EEG segments into distinct event types. Similarly, following CBraMod [21], TUEV [2] was chosen as the downstream dataset for this task. The TUEV dataset [2] includes EEG recordings annotated for epileptic and non-epileptic events, making it a standard resource for event-type classification studies.
E.2.2 Mental Stress Detection: MentalArithmetts (2-class)
MentalArrihmmttic [72, 73] is an EEG dataset that contains recordings of 36 subjects of different genders and ages before and during the performance of mental arithmetic tasks. The EEGs are recorded from 20 silver/silver chloride electrodes placed according to the international 10-20 system at 500Hz sampling rate. All EEG recordings during mental arithmetic are labeled "under stress". The ones that are not during mental arithmetic are labeled "no stress". We present the analysis of expert participation on this dataset in Figure 10. The channel relationships are visualized in Figure 8.
Table 18: Comparison of different methods on mental stress detection (MentalArithmettic, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EECNet [52]
0.6770
0.5763
0.0102
0,7321 0.0108
EEGConformer [53]
0.6805 0.0123
0,5829
0.0134
0,7424 = 0.0128
SPaRCNet [54]
0.6879 = 0.0107
0.5825
0.0193
0,7418 = 0.0132
ContraWR [23]
0.6631 0.0097
0.5787
0.0164
0.7332 0.0082
CNN-Transformer [55]
0.6779 0.0268
0.5777
0.0285
0.7258 0.0336
FFCL [6]
0.6798 0.0142
0,5786
0.0266
0,7330 = 0.0198
ST-Transformer [8]
0.6631
0.0173
0.5672
0.0259
0,7132 + 0.0174
BIOT [Neuteesn) [17]
0.6875
0.0186
0.6004
0.0195
0,7536 + 0.0144
LaBraM (1r [18]
0.6909 0.0125
0.5999
0.0155
0.7721 + 0.0093
CBraMod pctaas [21]
0.7256
0.0132
0.6267
0.0099
0.7905 0.0073
NEURIPT (Ours)
0.8646 0.0107
0.7827
0.0197
0.9111 = 0.0163
Preprocessing All data were first converted uniform units. A 50 Hz notch filter was applied to attenuate power-line noise, followed by resampling the data to 64 Hz. A global average reference was then used. And following CBraMod [21], the data were segmented into 5-seconds.
Experimental Configuration Following CBraMod [21], subject 28 are set to training set, subject 29 to 32 are set to validation set and subject 33 to 36 are set to test set.
Results As shown in Table 18, NEUR achieves the state-of-the-art performance and gets a great improvement. NEURIPT performs 13.9 % better than the best baseline in balanced accurary, 15.6 % better than the best baseline in AUC-PR and 12.06 % better than the best baseline in AUROC.
E.2.3 Mental Disorder Diagnosis: Mumtaz2016 (2-class)
Mumtaz2016 [74] consists of EEG recordings from 34 patients diagnosed with major depressive disorder (MDD) and 30 healthy subjects (H). All EBGs recorded from 19 electrodes placed according to the international 10-20 system at 256Hz sampling rate. The dataset collects three sessions, including eyes-open session, eyes-closed session, and task session. We present the analysis of expert participation on this dataset in Figure 11. The channel relationships are visualized in Figure 8.
Preprocessing
The data were first converted to uniform units. A 50 Hz notch filter was then applied to attenuate power-line noise. After that, the data were resampled to 64 Hz. and re-referenced using a global average reference. Following CBraMod [21], the data were segmented into 5-seconds.
Experimental Configuration In our experiment, we only uses eyes-open session and eyes-closed session. Similarly, following CBraMod [21], 24 MDD and 19 H are used for training.5 MDD and H are used for validation, and 5 MDD and 5 H are used for test.
27

---

Table 19: Comparison of different methods on mental disorder diagnosis (Mumtaz2016, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNet [52]
0.9232 0.0104
0.9626
0.0095
0,9639 + 0.0093
EEGConformer [53]
0.9308 0.0117
0,9684
0.0105
0,9702 0.0101
SPaRCNet [54]
0.9316 0.0095
0,9754
0.0065
0,9781 = 0.0083
ContraWR [23]
0.9195 = 0.0115
0.9589
0.0102
0,9621 = 0.0092
CNN- TuN-rranformerr [55]
0.9305 0.0068
0.9757
0.0074
0.9742 0.0059
FFCL [6]
0,9314 0.0038
0.9717
0.0021
0,9753 0.0033
ST-Transformer [8]
0.9135 0.0103
0.9578
0.0086
0,9594 = 0.0059
BIOT (Noudi522) [17]
0.9358 0.0052
0,9736
0.0034
0,9758 = 0.0042
LaBraM (Cca))) [18]
0,9409 = 0.0079
0,9798
0.0093
0,9782 + 0.0057
CBraMod DCLa25 [21]
0.9560 0.0056
0.0032 0.9923
0.9921 = 0.0025
NEURIPT (Ours)
0.9803 0.0062
0.9981 0.0044
0.9979 = 0.0045
Results As shown in Table 19, NEURIPT achieves the state-of-the-ard performance. NEURIPT performs 2.43 % better than the best baseline in balanced accuracy.
E.2.4 P300 Task: PhysioNetP300 (2-class)
PhysioNetP300 [75] is typical P300 task dataset. Each record in the dataset contains the signals, triggers and annotations corresponding to a single run. In this dataset, each subject was asked to spell a total of 20 characters using a Donchin speller. The target characters were randomly selected before the start of the run. Each row and column of a standard 6x6 character matrix was randomly augmented for 100 ms at 50 ms intervals with approximately 20 flashes. During this time, subjects need focus on the target and count the number of times the target was highlighted. When the target was highlighted, we labeled it as "Target". Otherwise, we label it as "Non-target". We present the analysis of expert participation on this dataset in Figure 12. The channel relationships are visualized in Figure 8.
Table 20: The results of different methods on P300 task (PhysioNedP300, 2-class).
Methods
Balanced Accuracy
Cohen's Kappa
AUROC
BENDR [15]
0.6114 + 0.0118
0,2227 0.0237
0.6588
0.0163
HIOT pieuttern [17]
0.5485 0.0325
0.0968 0.0647
0.5308
0.0333
LaBraM puLaze [18]
0.6477 = 0.0110
0.2935
0.0227
0.7068
0.0134
EEGIT (ounttssn) [20)
0.6502 + 0.0063
0.0139 0.2999
0.7168
0.0051
NEURIPT (Ours)
0.6731 + 0.0045
0.3426 0.0074
0.7683
0.0039
Preprocessing The data were first converted to uniform units. A 60 Hz notch filter was applied to attenuate power-line noise. The signals were then resampled to 64 Hz and re-referenced using global average reference. Following the preprocessing steps used in EEGPT [20], the data were segmented into 2-seconds starting at 0.7 seconds before the onset of the flicker stimulus.
Experimental Configuration Following EEGPT [20), subjects 8, 10, and 12 were removed and the data from the remaining subjects were retained. We split the subjects into training set and testing set randomly.
Results As shown in Table 20, NEURIPT achieves the state-of-the-an performance. NEURIPT performs better than the best baseline all evaluation metrics.
E.2.5 Sleep Stage Detection: SleepEDFx (5-class)
SleepEDFx [76] a dataset that contains 197 (78 healthy subjects) whole-night polysomnographic sleep recordings, including EEG, EOG, chin EMG, and event markers. The sampling rate are 100Hz of the EEG and EOG channels, the ohter channels are lHz. We present the analysis of expert participation on this dataset in Figure 13. The channel relationships are visualized in Figure 8.
28

---

Table 21: The results of different methods on sleep stage detection (SleepEDFx, 5-class).
Methods
Balanced Accuracy
Cohen's Kappa
Weighted
BENDR [15)
0.6655 +: 0.0043
0.6659 = 0.0043
0.7507
0.00299
BIOT pountesnn [17]
0.6622 0.0013
0.6461 0.0017
0.7415
0.0010
LaBraM nCURon [18]
0.6771 +: 0.0022
0,6710
0.0006
0.7592
0.0005
EEGIT pountsson [20)
0.6917 + 0.0069
0.6857
0.0019
0.7634
0.0023
NEURIPT (Ours)
0.7047 # 0.0041
0.7757
0.0015
0.8739
0.0013
The data were first converted to uniform units. They were then resampled to 64 Hz and re-referenced using a global average reference. Following EEGPT [20], the data were segmented into 30-seconds.
Preprocessing
Experimental Configuration We use the whole sleep-cassette (SC) and slcep-telemetry (ST) dataset that includes 197 subjects. Following EEGPT [20], subjects were randomly divided according to the ratio 60.
Results As shown in Table 21, NEURIPT achieves the state-of-the -f--th--ttt--ttt performance. NEURIPT performs better than the best baseline in all 3 evaluation metrics. In particular, NEURIPT gets great improvement in Cohen's Kappa and Weighted F1.
E.2.6 Emotion Recognition: SEED-V (4-class)
The SEED-V dataset [3] comprises EEG and eye movement recordings from 16 participants during emotion elicitation tasks. Each participant completed three sessions on separate days, with each session containing 15 trials thhre trials for each of five emotion categories: happy, sad, neutral, disgust, and fear. EEG signals were recorded using a 62-channel ESI NeuroScan system at sampling rate of 1000 Hz. The dataset provides both raw EEG data and extracted differential entropy (DE) features across five frequency bands, facilitating various analyses in emotion recognition research. We also present the analysis of expert participation on the this dataset in Figure 14, and report the Pearson correlation between class logits and channel perturbations induced by Gaussian multiplicative noise Figure 16. The channel relationships are visualized in Figure 8.
Table 22: The results of different methods on emotion recognition (SEED-V, 55-lass
Methods
Balanced Accuracy
Cohen's Kappa
Weighted Fl
EEGNet [52]
0.2961 0.0102
0.1006
0.0143
0.2749 = 0.0098
EBGConformer [53]
0.3537 0.0112
0.1772 0.0174
0.3487 + 0.0136
SPaRCNet [54]
0.2949 0.0078
0.1121
0.0139
0.2979 0.0083
ContraWR [23]
0.3546 0.0105
0.1905
0.0188
0,3544 0.0121
Crn-Transformer [55]
0.3678 0.0078
0,2072
0.0183
0,3642 = 0.0088
FFCL [6]
0,3641 0.0092
0.2078
0.0201
0.3645 = 0.0132
ST-Transformer [8]
0.3052 0.0072
0.0121 0.1083
0,2833 0.0105
BIOT (Nomeest) [17]
0.3837
0.0187
0.2261
0.0262
0.3856 + 0.0203
LaBraM pCca24 [18]
0.3976
0.2386
0.0209
0,3974
0.0111
CBraMod peraes [21]
0.4091
0.0097
0.0143 0.2569
0.4101 0.0108
NEURIPT (Ours)
0.4104 0.0021
0.2629
0.0039
0.4158
0.0037
Preprocessing ERG signals were band-pass filtered between 0.1 Hz and 30 Hz and downsampled to 64 Hz. A 50 Hz notch filter was applied to remove power line interference. Each trial was segmented into non-overlapping I-second epochs, resulting in a total of 115,001 samples. Following CBraMod [21], data from subjects 1-10 were used for training, while subjects 11-15 were reserved for testing. Detailed results are reported in Table 22.
Experimental Configuration Each I-second segment was treated as an independent sample labeled according to the corresponding emotion category. The model was trained for 10 epochs using a batch size of 256. We use both bipolar and non-bipolar methods, and the non-bipolar one is slightly better than the bipolar one.
29

---

Our proposed NEURIPT model achieves state-of-the-an performance on SEED-V emotion recognition task, outperforming existing models across the evaluation metries. These results highlight the effectiveness and robustness of our approach in modeling complex emotional dynamics from EEG signals.
Results
E.2.7 Motor Imagery Task: BCIC-IV-2A (4-class)
The BCIC-IV-2A dataset [4] comprises EEG recordings from 9 healthy subjects performing 4 motor- imagery tasks: left hand (Class 1), right hand (Class 2), feet (Class 3) and tongue (Class 4). Each subject completed two sessions on separate days; each session contains six runs of 48 trials (12 trials per class), yielding 288 trials per session. Signals were acquired at 250 Hz from 22 channels positioned according to the international 10-20 system and band-pass filtered between 0.5 Hz and 100 Hz. We also present the analysis of expert participation on the this dataset in Figure 6, and report the Pearson correlation between class logits and channel perturbations induced by Gaussian multiplicative noise in Figure 4. The channel relationships are visualized in Figure 8.
Table 23: The results of different methods motor imagery classification (BCIC-IV-2A, 4-class).
Methods
Balanced Accuracy
Cohen's Kappa
Weight F1
EEGNet [52]
0.4482 0.0094
0.2693
0.0121
0,4226 = 0,0108
EEGConformer [53]
0.4896 0.0106
0,2924
0.0141
0,4533 = 0.0128
SPaRCNet [54]
0.4635 0.0117
0.2847
0.0147
0.4432 + 0.0126
ContraWR [23]
0.4678 0.0125
0,2905
0.0160
0.4413 0.0142
CNN- Transornnorrrrr [55]
0.4600 0.0108
0,2800
0.0148
0.4460 0.0114
FFCI [6]
0.4470 = 0.0143
0.2627
0.0176
0.4238 + 0.0139
ST-Transfomer [8]
0.4575
0.0145
0.0198 0.2733
0.4471 = 0.0142
BIOT (NeuttP22 [17]
0.4748 = 0.0093
0,2997
0.0139
0.4607 = 0.0125
LaBraM (CUR24 [18]
0.4869 0.0085
0.3159
0.0154
0.4758 0.0103
CBraMod DCL825 [21]
0.5138 0.0066
0.3518
0.0094
0.4984 0.0085
NEURIPT (Ours)
0.5504 0.0072
0.4005 0.0121
0.5376 0.0086
Preprocessing Raw EEG signals were band-pass filtered between 0.1 Hz and 30 Hz, followed by downsampling to 64 Hz. For each trial, we extracted the segment from 2 s to 6 after cue onset and assigned labels according to the instructed motor imagery class. This procedure yielded 288 non-overlapping 4 s segments per session. Signals were re-referenced using 16 predefined bipolar channel pairs. The model input comprised the 16 bipolar channels. Following CBraMod [21]. data from subjects 1-5 were used for training, subjects 6-7 for validation, and subjects 8-9 for testing. The detailed channel configuration and corresponding sample counts are provided in Table 23.
Experimental Configuration Optimization employed a vosine-annealed learning rate schedule, warming up linearly from 1.7 x 10 to 3.5 x 10 10-5, 5, then decaying back to 1.7 x 10 by epoch 150. We trained with a batch size of 16 for 150 epochs.
Results Our proposed NEURIPT model achieves state-of-the-art performance on BCIC-IV-2A motor imagery classification, outperforming existing baselines across all evaluation metrics. These results demonstrate the effectiveness and robustness of our approach.
E.2.8 Abnormal Detection: TUAB (2-class) and Event Type Classification: TUEV (6-class)
The TUEV dataset [2] is a curated subset of the Temple University Hospital EEG Corpus (TUEG), enriched with expert annotations for epileptiforn and non-epileptiform events. Each EEG segment is labeled into one of six categories: spikes and sharp waves (Class 1), generalized periodic epileptiform discharges (Class 2), periodic unilateral epileptiform discharges (Class 3), eye movements (Class 4), artifacts (Class 5), and background (Class 6). Recordings were acquired using 23 EEG channels with a sampling rate of 250 Hz. We present the analysis of expert participation on this dataset in Figure 15. The channel relationships are visualized in Figure
The TUAB dataset [2] is a large-scale EEG corpus annotated for abnormality detection. Each recording is labeled as either normal or abnormal based on clinical reports. Similar to TUEV, the EEG signals were recorded using 23 channels with a sampling rate of 250 Hz. The dataset serves as a benchmark for evaluating automated EEG abnormality detection methods. We present the analysis of expert participation on this dataset in Figure 9. The channel relationships are visualized in Figure 8.
30

---

Table 24: Comparison of different methods on TUAB (2-class) and TUEV (6-class).
TUEV
Method
Balanced Acc.
AUC-PP
AUROC
Balanced Acc.
Cohen's
Kappa
Weighted F1
EBGNe: [52]
0.7642 0.0036
0.8299
0.0043
0.8412 + 0.0031
0.3876
0.0143
0.3577
0.0155
0.6539
0.0120
EEGConformer [53]
0.7758
0.0049
0.8427
0.0054
0.8445
0.0038
0.4074
0.0164
0.3967
0.0195
0,6983
0.0152
SPaRCNet [54]
0.7896
0.0018
0.8414 + 0.0018
D.8676 + 0.0012
0.4661 + 0.0262
0,233
000181
0.7024
0.0104
ContraWR [23]
0.7746
MON-Trasformmr [55]
0.7777
0.0022
0.8433 + 0.0039
0.3461
0.0013
0.4087
0.0161
0.3815
0.0134
0.6544
0.0293
FFCL [6]
0.7848
0.0038
0,8448
0.0065
0.8569
0.0051
0.3979
0.0104
0.5732
0.0188
0.783
0.0120
ST-Tansformer [8]
0.7966 0.0023
0,8521 0.0026
0.8707
0.0019
0.3984
0.0228
0.0306 0.3768
0.6823
0.0190
BIOT seirsm [17]
0,7959
0.0057
0,8792 + 0.0023
0.8815 = 0.0043
0.5281
0.0225
0.5273
0.0249
0.7492
D 0,0022
MuaBraM-Hugoccuue
0,8258
0.0011
0.6616
06745
0.0195
0.0086
Docerss (20)
0.7983
0.0030
0.8718 = 0.8718
0.6232 = 0.0114
0.6351
0.0134
0.8187
0.0063
NeuroL
MeuroLM-XLuuuee XLauu [19]
0,7969 0.0091
0.7219
0.0082
0.7884 = 0.0194
0.4679
0.0356
0.4570
0.0498
0.7359
0.0219
CDraModaruen [21]
0.0008
0.0096
NEURIPT (Ours)
0,8293 0.0016
0.9040
0.0022
0.8949
0.0021
0.6761 0.0133
0.6970
0.0185
0.8428
0.0089
Preprocessing EEG recordings were Sand-pass filtered between 0.1 Hz and 30 Hz to attenuate low- and high-frequency noise, and 60 Hz notch filter applied to remove power-line interference. Signals were then resampled to 64 Hz and segmented into fixed-length epochs: 5 s non-overlapping segments for TUEV, and 10 segments for TUAB. We maintained the original training/test partitions provided by each dataset to ensure fair comparison with the three Beeeinne-LaBraM]]1 EEGPT (20) and CBraMMo[[2]] The training subjects further divided into training and validation subsets in an 80% / 20% ratio.
Experimental Configuration For TUEV, each 5-second segment was treated an independent sample labeled according to the corresponding category. The model was trained for 20 epochs using batch size of 16. The learning rate peaked x 10-5 and then gradually decayed minimum of I x 10-? For TUAB, each 10-second segment was treated as an independent sample labeled according to the corresponding category. The model was trained for 10 epochs using batch size of 16. The learning rate peaked at 5 x 10 and then gradually decayed to a minimum of 3 x 10
Results Our proposed NEURIPT model achieves saateoff-the-a performance on the TUEV dataset, consistently outperforming existing baselines across all evaluation metrics. These results highlight the effectiveness and robustness of our method. On the TUAB abnormality detection task, NEURIPT attains higher balanced accuracy compared to prior methods. While the performance on the other two metrics slightly lower than the best results, still exceeds most baseline models. This indicates that NEURIPT is both competitive and effective, demonstrating strong generalization ability in learning transferable EEG representations.
31

---

Discussion
E1
Limitation
While NEURIPT incorporated several important characteristics of EEG signals into its neural ar- chitecture learn more generalizable representations, there remain several open challenges and opportunities for future work. (1) For instance, additional EEG-specific aspects, such as brain con- nectivity representations, have not yet been fully explored and could enhance decoding performance. (2) Moreover, as with many foundation models, NEURIPT requires large number of parameters, resulting in increased memory usage and higher computational costs during both training and infer- ence compared to non-foundation models. (3) Our ability to further scale the model and investigate scaling laws was constrained by the limited GPU resources available. (4) Lastly, although NEUR IPT achieved SOTA performance on average across multiple datasets, there is still room to improve neural decoding accuracy to fully meet the practical demands of BCI systems for real-world and clinical applications.
F.2 Broader Impacts
This work advances neural decoding for general HEG-based neural interfaces, with potential benefits across a wide range of BCI applications, including clinical diagnosis, emotion recognition, and user intent control. These advancements have broader implications for enhancing human well-being by expanding access to brain-computer interface (BCT) technologies and enabling more diverse real-world applications. While our method is primarily developed for foundation models (FMs), is designed to leverage key EEG characteristics, making it also adaptable to task- and setup-specific models for wide range of spplication-specific scenarios. Furthermore, our findings provide valuable insights for the development of more advanced foundation models that incorporate other EEG-specific characteristics, such as brain connectivity.
F.3 Potential Misuse
As EEG foundation models become increasingly powerful for healthcare and commntcomputer interface (BCT) applications, is crucial to acknowledge their potential for misuse, for example, unauthorized cognitive surveillance profiling. The ability to infer mental states cognitive intent, if applied without proper oversight, raises profound ethical concerns. Addressing these risks will require the development of neurotechnology regulations, data protection policies, and ethics frameworks to ensure responsible use and safeguard individual neural privacy.
G Further Visualization
Further visualizations start the next page.
32

---

(a) BCIC-IV-2A
(b) Mental Arithmetic
(c) Mumtaz 2016
(d) PhysioNeetP300
(c) TUEV
(i) TUAB
(g) SEED-V
(h) Sleep-EDFx
Figure 8: Inter-channel relationships for eight downstream tasks.
33

---

30%
13%
19
Her
Labals
35%
35%
10%
Mew3
190
Mer3
LAbes
Lobals
Labels
Figure 9: Analysis of expert participation on the TUAB dataset.
11.6
0.2%
4000
65.0%
A2th
6:3%
Langran
0.3%
Labes
Label
Libels
Libel terprra Expan
Merpor
01.9%
4.3%
460h
4.5%
#2th
Langiand
Labea Spatal
Lebdl
Sontaa
SpetN Eqper)
Figure 10: Analysis of expert participation on the MritalArithmetic dataset.
11.0
17.0%
40.%%
45.5%
404h
Label
Libels
Labels tempoo
422%
A0ch
Labea
Lacal Sqaibal
Spatte
Figure 11: Analysis of expert participation on the Mumtaz dataset.
1805
0..%%
194
425%
4.5%
Libels
Labals
10.%
10.6%
190
Arge
Sputial Epen:
Sparta 2
Spedte Enpen)
Sostall
Figure 12: Analysis expert participation on the P300 dataset.
34

---

180h
150%
645
10 THRM bxpat1
1O Tomoooel Eisot3
2.90
157%1
207
1505
15.%%
15th
Lebels Spatial Supert
oitadel Expec
10 Spaee Fapert
PO PuSipaa Papntt
Figure 13: Analysis of expert participation the SleepEDF dataset.
250
2075
300%
30%
Labels
10 TERAK Expal1
Rupart
2606
10.60
1006
20m
2505
15.5%
280h
2.
Labes
Lands
Labels
SpMbAl Expar
Scatiaal
10 Spadk
DV EiSipata Expar
Figure 14: Analysis of expert participation the SEED-V dataset.
2806
17
2505
300%
365h
210%
labex
Labda
Labels
Terpon ExpaT
Skmpoal Equer
Der Supar
de Sqgae
11.0
2606
15.0
100%
2605
15.5%
28th
ONL
Jabes
Lands
Labels
SpMtA Expert
Scattal Expen
10 Spadi Eqper
Figure 15: Analysis of expert participation on the TUEV dataset.
Happy
Fear
Neutral
Sad
Disgust
Figure 16: Pearson correlation between class logits and channel perturbation using Gaussian multi- plicative noise on SEED-V.
35

---

2.
1.
Figure 17: Loss with different mask ratios
MAr M
ground S1LA7 mesked peedict
50
100
150
200
250
tme
Figure 18: Visualization of AAMP based reconstruction (a).
7.0
w wowalwwhhiii of $ Wini
-1.0
ground muuhh masked
-1.3
prodict
Figure 19: Visualization of AAMP based reconstruction (b).
1
MM
4p
truth
prodict
150
2000
tme
Figure 20: Visualization of reconstruction with BERT-style masking.
36

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Double checked carefully.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
is provided the Appendix Section F.I.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
37

---

Justification:
This work does not make theoretical assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The information detailed in Section and Appendix D with source code provided.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
38

---

Answer: [Yes]
We provide our project link in the Abstract, which includes the source code and instructions.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (uttps://nips.. ec/
Policy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //niss. /t/pcc/ppllco scc/public/ppe Seriiies/CoeeSuuee wwsiissssiccyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: It is detailed in Section 3.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
The numerical results are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
39

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: is Section and Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp:s:///uurs..../// cttppppscccpppccttetiiace httpppullc/iicccuueeeleleel.eeee csGuidel ines?
Answer: [Yes]
Justification: Yes, we followed it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
is in Appendix F.2.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
40

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve the described risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We the original owners of our assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
41

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code is well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Not relevant to our work.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Not relevant to this work we use open dataset.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
42

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification:
We do not use LLM for any core method development in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy thttps://a /ttpa//:///eu...peeeee wwt.c/ooeeeenncc//22////1/. for what should or should not be described.
43


Reviews:
[{"review_id": "Xcr8LBlVrA", "reviewer": "Reviewer_qMmx", "summary": "This paper introduces NEURIPT, a novel EEG foundation model for cross-task generalization, combining a Transformer architecture with an Amplitude-Aware Masked Pretraining (AAMP) module, a Progressive Mixture-of-Experts (PMoE) mechanism and an Intra-Inter Lobe Pooling (IILP) strategy. Evaluated across multiple datasets, NEURIPT outperforms existing methods in downstream tasks.", "questions": "The authors adopt the PMoE strategy with expert allocations [0, 0, 2, 4, 4, 6] across encoder layers (Table 3). However, an intuitively more gradual progression (e.g., [0, 2, 3, 4, 5, 6]) might better align with hierarchical feature learning in deeper layers. Could the authors clarify:\n\n1. The design rationale for omitting experts in the first two layers and repeating identical expert counts in layers 4–5?\n\n2. Whether empirical comparisons were conducted against alternative PMoE patterns (e.g., [0,2,3,4,5,6]), and if so, what trade-offs were observed in terms of performance?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. This paper presents a well-structured and clearly written introduction that effectively contextualizes the challenges in EEG foundation model development. \n\n2. The authors propose a novel amplitude-aware masked pretraining paradigm (AAMP), which departs from conventional random masking strategies to address signal heterogeneity. \n\n3. The empirical evaluation demonstrates the model's competitiveness through comprehensive comparisons with diverse EEG foundation models (e.g., BIOT, EEGPT, CBraMod), achieving superior performance across eight downstream tasks. \n\n4. The authors meticulously document training protocols, hyperparameters, and implementation details, ensuring strong reproducibility.\n\n**Weaknesses**\n\n1. The method section requires improved exposition, particularly regarding the symbolic notation. Ambiguous definitions of key components hinder readability and precise understanding of the model’s architecture.\n\n2. The proposed AAMP lacks rigorous theoretical grounding. While empirically effective, the amplitude-based masking risks disrupting temporal continuity in EEG signals. As described in Eq. 4, masking intervals centered on random amplitude thresholds could lead to prolonged contiguous signal mask (contrary to the idealized example in Fig. 3b), raising concerns about the reliability of reconstruction and its alignment with EEG dynamics.\n\n3. The ablation study in Appendix B (comparing AAMP to BERT-style random masking) is limited to a single downstream dataset (TUEV) with SVM-based evaluation. The observation that random masking achieves lower reconstruction loss (Fig. 7) but inferior downstream performance requires deeper analysis( the author's explanation fails to convince me). A more convincing validation would involve swapping only the masking strategy (keeping other components fixed) and evaluating across multiple tasks to isolate its impact.\n\n4. While Fig. 2 highlights heterogeneous spectral patterns across downstream tasks, the model architecture does not explicitly incorporate frequency-band-specific mechanisms to exploit these differences. The IILP pooling strategy, though spatially informed, does not directly address the spectral heterogeneity emphasized in the introduction, limiting the interpretability of the proposed design choices.\n\n5. The paper lacks comprehensive ablation studies to validate the contribution of each key component in NeurIPT (e.g., 3D positional encoding, PMoE, IILP).\n\n6. The right panel of Fig. 5 (channel-wise analysis) suffers from insufficient font size and resolution, obscuring critical details in the visualization", "was_revised": true, "final_justification": "All of my concerns have been adequately addressed in author's rebuttal.", "rebuttal": "**Dear reviewer qMmx:**\n\n**We appreciate your insightful comments. Below, we respond to each of your concerns.**\n\n---\n\n> [W1, Paper Formatting Concerns] The method section requires clearer symbolic notation and definitions.\n\n[A1] Thank you for requesting clarity on symbolic notation and definitions. We have now revised the manuscript, summarised as follows, for clarity and readability:\n\n- Removed the redundant definition $P_d$ at line 149 and explicitly clarified its relationship to the electrode channel $d$.\n- Revised the notation for positional embedding $PE$ in Eqs. 1 and 6 from boldface to italic, differentiating it from the bold matrix $E$, ensuring consistency with [1].\n- Simplified and rewrote the descriptions of the two sets presented at lines 179–180.\n- Inserted the term \"labeled\" at line 215 to explicitly correspond with the earlier description at line 144.\n- Refined the phrasing and terminology at lines 147, 159, 167, and 201.\n\nWe hope the revision improves clarity. Should you have any additional suggestions, we would be grateful to receive them.\n\n&nbsp;\n\n> [W2, W3: AAMP] Explaination and ablation with random masking.\n\n[A2] We first clarify that the example in Fig. 3 is not just an idealized illustration but an actual sample from the TUEV dataset, reflecting the practical masking patterns produced by AAMP. Regarding temporal continuity concerns, the amplitude-based masking strategy is intentionally designed to produce longer contiguous masked segments with a similar amplitude range compared to random masking. \n\nOur amplitude-aware masking is a context-aware strategy that encourages the model to learn signal-specific characteristics and structural patterns, with amplitude serving as a proxy for signal energy. This approach reduces the risk of trivial reconstruction via simple interpolation methods (e.g., linear), which models might otherwise exploit. While prolonged masking may occur within individual channels, our model adopts a multivariate design, allowing it to reconstruct masked regions by leveraging complementary, unmasked information across channels. This preserves learning effectiveness even under dense intra-channel masking conditions.\n\nAs highlighted in Fig. 7, although random masking achieves lower reconstruction loss, its inferior downstream performance strongly supports our central argument: random masking inherently promotes interpolation rather than encouraging deeper structural feature learning. To further substantiate this claim, we conducted additional experiments where only the masking strategy was swapped while keeping all other model components fixed:\n\n|Datasets|Masking Strategies|Bal. Acc.|Cohen’s Kappa|Weighted F1|\n|:-:|:-:|:-:|:-:|:-:|\n|TUEV|Random Masking|67.35|67.28|83.37|\n||AAMP (Ours)|**67.61**|**69.70**|**84.28**|\n|MentalArithmetic|Random Masking|84.38|77.89|90.98|\n||AAMP (Ours)|**86.46**|**78.27**|**91.11**|\n|Mumtaz2016|Random Masking|97.14|**99.84**|**99.84**|\n||AAMP (Ours)|**98.03**|99.81|99.79|\n|SEED-V|Random Masking|33.89|17.07|33.82|\n||AAMP (Ours)|**41.04**|**26.29**|**41.58**|\n|PhysioP300|Random Masking|65.64|28.99|72.75|\n||AAMP (Ours)|**67.31**|**34.26**|**76.83**|\n|Sleep-EDFx|Random Masking|69.68|76.82|87.11|\n||AAMP (Ours)|**70.47**|**77.57**|**87.39**|\n\nThese results demonstrate consistent improvements of AAMP across multiple downstream datasets, demonstrating the effectiveness and generalization capability of our proposed masking strategy.\n\n&nbsp;\n\n> [W4] IILP doesn't directly address the heterogeneous spectral patterns mentioned in Fig. 2.\n\n[A3] We would like to clarify that, as stated in Lines 196-198 and 211-213, our Progressive Mixture-of-Experts (PMoE) architecture with a hierarchical structure in the backbone model is specifically designed to handle the heterogeneous and diverse information across various EEG signals, rather than the IILP pooling strategy. Figures 6 and 9-15 further substantiate this design, illustrating distinct activation patterns by different experts across multiple categories, thereby confirming that our specialized experts effectively capture the spectral heterogeneity present in EEG signals.\n\n&nbsp;\n\n> [W5] Ablation studies on 3D positional encoding, PMoE, IILP.\n\n[A4] We have expanded our ablation analyses across more downstream datasets. Specifically, comprehensive experiments for the component AAMP have been detailed in [A2]. Below, we provide comparisons focusing on the 3D positional encoding, PMoE, and the IILP strategies. \n\n|Datasets|Encoding Strategies|Bal. Acc.|Cohen’s Kappa|Weighted F1|\n|:-:|:-:|:-:|:-:|:-:|\n|TUEV|[1]|67.72|68.85|84.03|\n||[2]|64.78|67.92|83.13|\n||[3]|63.81|66.63|82.57|\n||IILP (Ours)|**68.94**|**71.55**|**85.17**|\n|MentalArithmetic|[1]|74.65|71.84|83.80|\n||[2]|71.53|74.23|83.17|\n||[3]|71.52|64.81|80.83|\n||IILP (Ours)|**75.69**|**74.37**|**86.83**|\n|Mumtaz2016|[1]|96.56|99.59|99.61|\n||[2]|96.63|99.21|99.23|\n||[3]|95.56|99.55|99.54|\n||IILP (Ours)|**97.07**|**99.83**|**99.84**|\n|SEED-V|[1]|35.03|18.58|35.08|\n||[2]|35.52|18.81|34.34|\n||[3]|37.40|21.54|37.34|\n||IILP (Ours)|**39.34**|**23.93**|**39.67**|\n\nalternative methods are included:\n\n- [1] trigonometric functions employed by vanilla Transformer, \n- [2] 1D learnable embeddings, \n- [3] 2D learnable embeddings introduced in LaBraM,\n\nThe results above clearly demonstrate that our proposed 3D electrode embedding achieves consistently superior performance across multiple datasets, substantiating the notable advantage of explicitly leveraging physical electrode positions.\n\n| MoE Strategies / Datasets |   TUEV    | MentalArithmetic | Mumtaz2016 |  SEED-V   | PhysioP300 |  BCIC-2A  |\n| :-------------------: | :-------: | :--------------: | :--------: | :-------: | :--------: | :-------: |\n|      w/o Expert       |   65.83   |      72.92       |   93.41    |   39.14   |   64.53    |   44.44   |\n|        Uniform        |   65.91   |      70.49       |   95.00    |   39.33   |   65.66    |   41.93   |\n|       Shrinking       |   65.80   |      73.96       |   93.08    |   39.21   |   65.99    | **44.62** |\n|  Progressive (Ours)   | **68.94** |    **75.69**     | **97.07**  | **39.34** | **66.58**  |   44.01   |\n\nWe have expanded Table 3 by evaluating different MoE strategies across multiple datasets, demonstrating the robustness of our PMoE approach.\n\n| Pooling Strategies / Datasets |   TUEV    | MentalArithmetic | Mumtaz2016 |  SEED-V   | PhysioP300 |  BCIC-2A  |\n| :-------------------: | :-------: | :--------------: | :--------: | :-------: | :--------: | :-------: |\n|      w/o Pooling      |   62.33   |      75.69       |   78.21    |   38.90   | **67.82**  |   45.14   |\n|     Mean Pooling      |   64.74   |      79.51       |   96.22    |   37.62   |   66.72    |   37.24   |\n|      Hemispheres      |   64.45   |      81.94       |   97.82    |   39.22   |   67.11    |   43.49   |\n|        Coronal        |   68.77   |      73.26       |   96.99    |   39.35   |   66.98    |   43.75   |\n|       Sagittal        |   67.21   |      80.21       |   91.41    |   39.42   |   65.66    |   45.31   |\n|      IILP (Ours)      | **68.94** |    **86.46**     | **98.03**  | **41.04** |   67.31    | **55.04** |\n\nWe have expanded Table 4 by incorporating comparisons between IILP and alternative classification methods across multiple datasets, further demonstrating the consistent superiority of our approach.\n\n&nbsp;\n\n> [W6] Fig. 5 clarity.\n\n[A5] To enhance readability, we have revised the figure by increasing the font size and repositioning the channel names radially along the outer edge of the circular visualization. Additionally, we have updated the caption to emphasize that a consistent color scheme is applied in both left and right panels, enhancing the clarity of attention scores across brain regions.\n\n&nbsp;\n\n> [Q1, Q2] PMoE ablation and discussion.\n\n[A6] Indeed, assigning zero experts in the initial two encoder layers implies the direct use of standard FFNs rather than specialized experts. The rationale behind this choice is grounded in the principle that lower layers should acquire generalized representations beneficial across all tasks, while higher layers incrementally introduce specialization to handle more differentiated features. Regarding the repetition of expert counts in layers 4 and 5, it was not intentionally designed for specific benefits but resulted from a random initial allocation. Nevertheless, we have empirically compared our chosen PMoE pattern against alternative, more gradual configurations. \n\n| PMoE Strategies / Datasets |   TUEV    | MentalArithmetic | Mumtaz2016 |  SEED-V   | PhysioP300 |  BCIC-2A  |\n| :-------------------: | :-------: | :--------------: | :--------: | :-------: | :--------: | :-------: |\n|     [0,0,2,4,4,6]     | **68.94** |      75.69       |   97.07    |   39.34   |   66.58    | **44.01** |\n|     [0,0,2,3,4,5]     |   67.87   |    **76.39**     |   96.81    | **39.88** |   66.34    |   41.58   |\n|     [0,0,2,4,6,8]     |   66.41   |      75.69       |   96.57    |   39.11   | **67.70**  |   41.15   |\n|    [0,0,3,6,9,12]     |   66.08   |      74.83       | **97.15**  |   39.35   |   66.63    |   38.19   |\n\nThese alternative patterns also yield competitive performance. This empirical observation suggested that the effectiveness of the PMoE approach stems from its inherent progressive strategy, rather than relying on any specific expert allocation, demonstrating robustness across diverse progressive configurations.\n\n&nbsp;\n\nReferences:\n\n[1] [2] Attention is All You Need (NeurIPS 2017)\n\n[3] Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI (ICLR 2024)\n\n---\n\n**Thank you again for your valuable comments. We hope our responses have addressed your concerns and strengthened your impression of our work. We remain available throughout the rebuttal period to clarify any further questions you may have. Looking forward to hearing from you :-) !**"}, {"review_id": "SX7OTFx9Iq", "reviewer": "Reviewer_aqFB", "summary": "Paper introduces a foundation model for EEG signals designed to handle the high variability across subjects, tasks, and recording setups. The model uses a transformer architecture with several novel components: 3D-aligned spatial encoding to handle different electrode montages, an amplitude-aware masking, and a MoE architecture to capture diverse temporal dynamics. Evaluation demonstrate good performance by fine-tuning the pre-trained model on several downstream tasks.", "questions": "Provide a more precise algorithmic description of how the masking interval [La, Ud] is defined based on the randomly sampled percentile g(i)? Is it a fixed-size window centered on that value, or does its width vary?\n\n\nMethod demonstrates strong fine-tuning performance, but a key promise of foundation models is few-shot or even zero-shot learning. Have you considered evaluating the model in a low-data regime (e.g., fine-tuning with 1% or 10% of the data) or in a few-shot setting?\n\n3D electrode embedding is presented as a core spatial contribution, but its direct impact is not isolated in an ablation study. Could you compare its performance to a simpler learnable 1D channel embedding or something like in [1] to quantify the benefit of explicitly encoding physical coordinates?", "limitations": "Pre-training requires significant resources.\n\nModel is evaluated on classification tasks.\n\nMay be discuss potential for misuse of powerful neural decoding models.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Amplitude-Aware Masking and Progressive Mixture-of-Experts (PMoE) combination is good to handle the unique characteristics of neural signals.\n\nEvaluation across eight diverse datasets is another major strength, providing some evidence for the model's generalization capabilities.\n\nOriginality of some components is largely incremental, building heavily on existing work like Crossformer and MoE. Key implementation details, particularly for the AAMP masking strategy, are not sufficiently clear to ensure reproducibility. Furthermore, the performance gains on some datasets are marginal compared to baselines. Also how the masking is any different than what proposed earlier in [1], e.g., fixed cutout and random masking?\n\n[1] Saeed, A., Grangier, D., Pietquin, O., & Zeghidour, N. (2021, June). Learning from heterogeneous EEG signals with differentiable channel reordering. In ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 1255-1259). IEEE.", "was_revised": true, "final_justification": "Main concerns have been addressed.", "rebuttal": "**Dear reviewer aqFB:**\n\n**We appreciate your insightful comments. Below, we respond to each of your concerns.**\n\n---\n\n> [W1] Originality of some components is largely incremental, building heavily on existing work like Crossformer and MoE. Key implementation details, particularly for the AAMP masking strategy, are not sufficiently clear to ensure reproducibility. Furthermore, the performance gains on some datasets are marginal compared to baselines. Also how the masking is any different than what proposed earlier in [1], e.g., fixed cutout and random masking?\n\n[A1] Our primary objective is to develop a more generalizable foundation model tailored for EEG data. While it is true that some components, such as Crossformer and Mixture-of-Experts (MoE), are inspired by existing work, it is noteworthy that these methods have not yet been demonstrated to be effective on the EEG foundation model. Our contribution lies in adapting, integrating, and validating the generalizability of these techniques for EEG representation learning to fill this critical gap. Rather than incremental reuse, we demonstrated that such architectures can significantly advance the state of the art in the EEG foundation model when used effectively with other components tailored for EEG.\n\nTo ensure reproducibility, we have submitted our source code in the supplementary materials .zip file and on our anonymous GitHub. Moreover, we have improved writing on AAMP for better readability and clarity. Furthermore, a detailed description of the AAMP algorithm is provided in [A2].\n\nRegarding [1], the proposed CHARM addresses inconsistent input channels through a learnable channel-level masking approach, fundamentally different from our token-level AAMP method, which is tailored explicitly for representation learning in self-supervised foundational modeling. Specifically, Fixed Cutout masks out half the channels entirely, focusing solely on transfer learning with the remaining channels, whereas AAMP masks contiguous intervals of tokens based on amplitude thresholds for robust representation learning. \n\nWe also notice COLA [2], which employs contrastive learning rather than masking-based self-supervised pretraining on large-scale audio data. These related efforts offer valuable complementary perspectives on enhancing time-series modeling. We have incorporated discussions of these references into our section on Related Work (Section A.1) to cover these important related works.\n\n[1] Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering (ICASSP 2021)\n\n[2] Contrastive Learning of General-Purpose Audio Representations (ICASSP 2021)\n\n&nbsp;\n\n> [Q1] Provide a more precise algorithmic description of how the masking interval $[\\mathcal{L}_d, \\mathcal{U}_d]$ is defined based on the randomly sampled percentile $\\xi_d^{(i)}$? Is it a fixed-size window centered on that value, or does its width vary?\n\n[A2] Given the randomly sampled percentile $\\xi_d^{(i)}$ from the amplitude distribution of the EEG signal, we first identify its corresponding index in the sorted sequence $\\textit{sorted}(x_d^{(i)})$, defined as $c = \\lfloor \\xi_d^{(i)} \\cdot T \\rfloor$. Subsequently, we define the amplitude interval $[\\mathcal{L}_d, \\mathcal{U}_d]$ symmetrically around this center point $c$, covering precisely $T \\cdot \\mathcal{P}$ points:\n\n$$\\mathcal{L} _d = x _{(c - \\lfloor T\\mathcal{P}/2 \\rfloor)}^{(i)}, \\quad \\mathcal{U} _d = x _{(c + \\lceil T\\mathcal{P}/2 \\rceil)}^{(i)}$$\n\nHence, while the masking proportion $\\mathcal{P}$ remains fixed, the amplitude-based interval width $\\mathcal{U}_d - \\mathcal{L}_d$ dynamically varies depending on the local distribution of the amplitude values.\n\n&nbsp;\n\n> [Q2] Method demonstrates strong fine-tuning performance, but a key promise of foundation models is few-shot or even zero-shot learning. Have you considered evaluating the model in a low-data regime (e.g., fine-tuning with 1% or 10% of the data) or in a few-shot setting?\n\n[A3] While low-data scenarios have not been explored either in our original submission or in previous baseline works, we follow your suggestion and have conducted additional experiments with reduced training data:\n\n|Datasets|Data Used|Bal. Acc.|Cohen’s Kappa|Weighted F1|\n|:-:|:-:|-:|-:|-:|\n|TUEV|$1\\\\%$|64.16|72.44|88.08|\n||$5\\\\%$|75.39|81.78|91.67|\n||$10\\\\%$|80.30|88.64|95.75|\n||$100\\\\%$|100.00|100.00|100.00|\n| Sleep-EDFx |$1\\\\%$|80.97|83.86|91.33|\n||$5\\\\%$|86.01|79.82|90.41|\n||$10\\\\%$|92.51|92.74|96.13|\n||$100\\\\%$|100.00|100.00|100.00|\n\n\\* *Metrics are shown as percentages relative to the 100% data baseline.*\n\nNeurIPT achieved notable classification accuracy, maintaining performance in the range of 80%-90% with merely 10% of the original data, and even with an extremely limited 1% of data, performance remains competitive at 64%-80%. These results further confirm the robustness of our pretraining strategy and underscore the model’s potential in few-shot and low-resource learning settings.\n\n&nbsp;\n\n> [Q3] 3D electrode embedding is presented as a core spatial contribution, but its direct impact is not isolated in an ablation study. Could you compare its performance to a simpler learnable 1D channel embedding or something like in [1] to quantify the benefit of explicitly encoding physical coordinates?\n\n[A4] We have now included a dedicated ablation study, comparing:\n\n- [3] trigonometric functions employed by vanilla Transformer, \n- [4] 1D learnable embeddings, \n- [5] 2D learnable embeddings introduced in LaBraM,\n\n|Datasets|Encoding Strategies|Bal. Acc.|Cohen’s Kappa|Weighted F1|\n|:-:|:-:|:-:|:-:|:-:|\n|TUEV|[3]|67.72|68.85|84.03|\n||[4]|64.78|67.92|83.13|\n||[5]|63.81|66.63|82.57|\n||IILP (Ours)|**68.94**|**71.55**|**85.17**|\n|MentalArithmetic|[3]|74.65|71.84|83.80|\n||[4]|71.53|74.23|83.17|\n||[5]|71.52|64.81|80.83|\n||IILP (Ours)|**75.69**|**74.37**|**86.83**|\n|Mumtaz2016|[3]|96.56|99.59|99.61|\n||[4]|96.63|99.21|99.23|\n||[5]|95.56|99.55|99.54|\n||IILP (Ours)|**97.07**|**99.83**|**99.84**|\n|SEED-V|[3]|35.03|18.58|35.08|\n||[4]|35.52|18.81|34.34|\n||[5]|37.40|21.54|37.34|\n||IILP (Ours)|**39.34**|**23.93**|**39.67**|\n\nAcross all evaluated datasets, our method yields consistent and substantial improvements. This result confirms the tangible benefit of our spatial design while keeping the rest of the architecture unchanged, demonstrating that directly encoding the physical coordinates of electrodes provides stronger spatial information for the model.\n\n&nbsp;\n\n> [L1] Pre-training requires significant resources.\n\n[A5] We acknowledge that this is an inherent constraint shared by all contemporary foundation models. As detailed in [A3], NeurIPT has great potential in a low-data regime. Developing more efficient pre‑training strategies remains an important avenue for future work.\n\n&nbsp;\n\n> [L2] Model is evaluated on classification tasks.\n\n[A6] We acknowledge that our model is assessed on classification tasks, which are similar to all baseline methods. Expanding beyond classification, we consider it an essential direction for future research. \n\n&nbsp;\n\n> [L3] May be discuss potential for misuse of powerful neural decoding models.\n\n[A7] We have added a discussion on the potential misuse of powerful neural decoding models in Appendix G Discussion (Section G.3 Potential Misuse):\n\nAs EEG foundation models become increasingly powerful for healthcare and brain–computer interface (BCI) applications, it is crucial to acknowledge their potential for misuse, for example, unauthorized cognitive surveillance or profiling. The ability to infer mental states or cognitive intent, if applied without proper oversight, raises profound ethical concerns. Addressing these risks will require the development of neurotechnology regulations, data protection policies, and ethics frameworks to ensure responsible use and safeguard individual neural privacy.\n\n&nbsp;\n\nReferences:\n\n[1] Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering (ICASSP 2021)\n\n[2] Contrastive Learning of General-Purpose Audio Representations (ICASSP 2021)\n\n[3] [4] Attention is All You Need (NeurIPS 2017)\n\n[5] Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI (ICLR 2024)\n\n---\n\n**Thank you again for your valuable comments. We hope our responses have addressed your concerns and strengthened your impression of our work. We remain available throughout the rebuttal period to clarify any further questions you may have. Looking forward to hearing from you :-) !**"}, {"review_id": "x14WtxPLl1", "reviewer": "Reviewer_L8YY", "summary": "This paper presents NeurIPT , a new foundation model for EEG-based neural interfaces. It addresses key challenges in generalizing across diverse EEG data — such as varying subjects, tasks, and electrode setups — by introducing an amplitude-aware masked pretraining strategy and a progressive Mixture-of-Experts architecture. The model also incorporates 3D electrode coordinates and a novel pooling mechanism to better capture spatial brain patterns. Evaluated across eight BCI datasets, NeurIPT achieves strong results under fine-tuning, showing promising generalization and broad applicability in EEG modeling.", "questions": "- The paper adopts several training techniques commonly used in large language models, such as the SwiGLU activation function. While these components have shown benefits in NLP tasks, it is unclear whether they bring measurable improvements to EEG foundation model training compared to more traditional choices like ReLU or GELU. Some ablation or discussion on this would be valuable.\n- The authors mention that additional experimental results and details are included in the appendix, but it is not attached directly to the main manuscript. Instead, it appears to be submitted as a separate file. This format makes it less convenient for reviewers and readers to access supplementary material. Why not follow the standard NeurIPS practice of appending it directly after the main text?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The proposed Amplitude-Aware Masked Pretraining (AAMP) introduces a novel perspective on how to design pretraining objectives for EEG foundation models. It moves beyond traditional random masking by focusing on signal amplitude, which could inspire future work in task-specific representation learning.\n\n- The introduction of Progressive Mixture-of-Experts (PMoE) is conceptually interesting and offers a promising way to model diverse temporal patterns in EEG signals by gradually increasing the specialization of expert subnetworks.\n\n- The use of 3D physical coordinates of electrodes for embedding provides a more structured and biologically grounded approach to channel encoding, which may help improve cross-dataset generalization.\n\n- The model was evaluated across a wide range of downstream BCI tasks, including motor imagery, emotion recognition, and seizure detection. This variety adds credibility to the claim that NeurIPT has broad applicability.\n\n\n### Weaknesses\n\n- While the use of 3D electrode coordinates is a nice design choice, the paper does not clearly show how this compares with other existing spatial encoding strategies, such as those used in LaBraM or other EEG foundation models. A direct comparison or ablation would help clarify its added value.\n\n- The authors adopt Crossformer as the backbone architecture but do not provide sufficient discussion or analysis regarding its advantages over standard Transformer architectures in the context of EEG modeling. This leaves an important question about architectural choice unaddressed.\n\n- Although the ablation study shows that AAMP leads to better performance on downstream classification tasks, it's unclear why that is the case. In particular, the authors note that random masking results in lower reconstruction loss — suggesting better interpolation ability — yet performs worse in classification. A deeper analysis of *why* AAMP helps classification would strengthen the contribution.\n\n- Similarly, while the PMoE mechanism appears beneficial, the link between expert activation patterns and actual performance gains remains speculative. The observation that different classes activate different experts is interesting, but it's not shown whether this is a cause or a side effect of improved accuracy.\n\n- The paper reports that IILP outperforms other pooling strategies based on anatomical brain region groupings such as Coronal, Sagittal, and Hemispheric divisions. However, the reason for this improvement is not clearly explained — is it due to better functional alignment, more balanced regional aggregation, or something else?\n\n- Some minor issues include the lack of clarity around certain experimental details — for instance, Table 5 does not specify which dataset the results are based on. Additionally, the paper lacks information on training time and computational resources, which are important for reproducibility and assessing feasibility.", "was_revised": true, "final_justification": "The author's response addressed most of the reviewer's concerns, and the reviewer decided to maintain the positive rating while raising the clarity score to 3.", "rebuttal": "**Dear reviewer L8YY:**\n\n**We appreciate your insightful comments. Below, we respond to each of your concerns.**\n\n---\n> [W1] Ablation of 3D electrode coordinates.\n\n[A1] To rigorously examine the benefit of using 3D electrode coordinates, we conducted a comprehensive ablation study comparing against several existing spatial encoding methods:\n\n- [1] trigonometric functions employed by vanilla Transformer,\n- [2] 1D learnable embeddings,\n- [3] 2D learnable embeddings introduced in LaBraM,\n\n|Encoding Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|\n|:-:|:-:|:-:|:-:|:-:|\n|[1]|67.72|74.65|96.56|35.03|\n|[2]|64.78|71.53|96.63|35.52|\n|[3]|63.81|71.52|95.56|37.40|\n|IILP (Ours)|**68.94**|**75.69**|**97.07**|**39.34**|\n\nThe results above demonstrate the effectiveness of our proposed 3D electrode embedding across multiple datasets, substantiating the advantage of explicitly leveraging physical electrode positions.\n\n&nbsp;\n\n> [W2] Reasons for choosing Crossformer as the backbone architecture.\n\n[A2] Thank you for requesting clarification on our backbone. We have now revised lines 181-185 and 582-583 to explicitly address the rationale behind selecting Crossformer [1].\n\nSpecifically, we build upon the Two-Stage Attention (TSA) from Crossformer [1], which sequentially performs cross-time and cross-dimension computations with a hierarchical architecture, and could be especially useful for capturing the intricate spatio-temporal dependencies inherent in multi-channel, multi-granularity EEG data.\n\nMoreover, TSA addresses computational efficiency compared to standard multivariate time series architectures, significantly reducing resource consumption during large-scale pre-training. Additionally, the hierarchical structure proposed in [1] naturally captures EEG signal representations at multiple granularities, providing both detailed and broader-level information. At deeper layers with coarser granularity, the reduced sequence lengths lead to lower computational demands. These advantages collectively informed our decision to adopt [1] as the backbone architecture.\n\n&nbsp;\n\n> [W3] AAMP further analysis and ablation.\n\n[A3] Our amplitude-aware masking is a context-aware strategy that encourages the model to learn signal-specific characteristics and structural patterns, with amplitude serving as a proxy for signal energy. This approach reduces the risk of trivial reconstruction via simple interpolation methods (e.g., linear), which models might otherwise exploit. Moreover, our model benefits from its multivariate design, allowing it to reconstruct masked regions by leveraging complementary, unmasked information across channels. This preserves learning effectiveness even under dense intra-channel masking conditions.\n\nAlthough random masking achieved lower reconstruction loss, its inferior downstream performance strongly supports our central argument: random masking inherently promotes interpolation rather than encouraging deeper structural feature learning. To further substantiate this claim, we conducted additional experiments where only the masking strategy was swapped while keeping all other model components fixed:\n\n|Masking Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|Sleep-EDFx|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Random Masking|67.35|84.38|97.14|33.89|65.64|69.68|\n|AAMP (Ours)|**67.61**|**86.46**|**98.03**|**41.04**|**67.31**|**70.47**|\n\nThese results demonstrate consistent improvements of AAMP across multiple downstream datasets, demonstrating the effectiveness and generalization capability of our proposed masking strategy.\n\n&nbsp;\n\n> [W4] PMoE exploration and ablation.\n\n[A4] Thank you for acknowledging our PMoE mechanism. To address the reviewer’s concern regarding the connection between performance gains and expert activation patterns (Fig.6, 9-15), we have conducted two comprehensive ablation studies to empirically validate the effectiveness of PMoE. Specifically, we first expanded our experiments in Table 3 across multiple downstream tasks:\n\n|MoE Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|w/o Expert|65.83|72.92|93.41|39.14|64.53|44.44|\n|Uniform|65.91|70.49|95.00|39.33|65.66|41.93|\n|Shrinking|65.80|73.96|93.08|39.21|65.99|**44.62**|\n|Progressive (Ours)|**68.94**|**75.69**|**97.07**|**39.34**|**66.58**|44.01|\n\nResults consistently demonstrate that PMoE outperforms other expert-allocation strategies. Furthermore, we explored the alternative progressive configurations:\n\n|PMoE Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|[0,0,2,4,4,6]|**68.94**|75.69|97.07|39.34|66.58|**44.01**|\n|[0,0,2,3,4,5]|67.87|**76.39**|96.81|**39.88**|66.34|41.58|\n|[0,0,2,4,6,8]|66.41|75.69|96.57|39.11|**67.70**|41.15|\n|[0,0,3,6,9,12]|66.08|74.83|**97.15**|39.35|66.63|38.19|\n\nThese alternative configurations also yield competitive performance, confirming that performance improvements indeed correlate with PMoE and its progressive configurations, rather than just being side effects. This empirical observation suggests that the effectiveness of the PMoE approach stems from its inherent progressive strategy.\n\n&nbsp;\n\n> [W5] Reasons for IILP improvement.\n\n[A5] We attribute the improvement from IILP primarily to the functional and structural alignment of brain regions, as EEG datasets are typically collected to examine different aspects of brain functionality, and brain lobes are anatomically separated by sulci.\n\nIn our brain region-specific analysis (Figs. 5, 8, and 16) using attention scores, we observe consistent patterns of interaction within each brain region and clear distinctions between regions. In Fig. 5, we apply a consistent color scheme to each brain region across both the left and right panels, which we have now explicitly clarified in the revised figure caption. For instance, substantial attentional connections from the occipital lobe (red) to the right temporal lobe (yellow) indicate meaningful cross-regional interactions. Moreover, we also observe contralateral control, where voluntary movement is lateralized mainly to the motor cortex on the opposite side of the brain. For instance, the scalp heatmap on the right side of Fig. 5 highlights regional, rather than sagittal or hemispheric, patterns. Specifically, the *Left Hand* label prominently activates the right temporal lobe, whereas the *Right Hand* label activates the left temporal lobe. These observations are consistently reinforced in Figs. 8 and 16. Furthermore, we have expanded the ablation studies in Table 4 to comprehensively demonstrate IILP's comparative advantages across multiple datasets:\n\n|Pooling Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|w/o Pooling|62.33|75.69|78.21|38.90|**67.82**|45.14|\n|Mean Pooling|64.74|79.51|96.22|37.62|66.72|37.24|\n|Hemispheres|64.45|81.94|97.82|39.22|67.11|43.49|\n|Coronal|68.77|73.26|96.99|39.35|66.98|43.75|\n|Sagittal|67.21|80.21|91.41|39.42|65.66|45.31|\n|IILP (Ours)|**68.94**|**86.46**|**98.03**|**41.04**|67.31|**55.04**|\n\n&nbsp;\n\n> [W6] Lack clarity of certain experimental details.\n\n[A6] We respectfully clarify that the dataset used in Table 5 is explicitly stated in the footnote following line 579, indicating that the experiments were conducted on the TUEV dataset. Additionally, this footnote includes the rationale behind the dataset choice and detailed experimental setups. To enhance clarity and readability, we incorporated this dataset information directly into the caption of Table 5 in the revised manuscript. Regarding computational details, we have already discussed training resources and durations in lines 246–249 and 629–631. We have now included the fine-tuning duration details at line 631, indicating that the process typically takes from a few minutes to several hours, depending on the downstream dataset size.\n\n&nbsp;\n\n> [Q1] Exploration of activation functions for EEG foundation models.\n\n[A7] Thank you for raising this insightful question. We conducted additional experiments examining the impact of activation functions:\n\n|Activation Function / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|ReLU|64.01|73.96|96.98|38.37|66.29|**47.31**|\n|GELU|64.02|73.61|**97.14**|38.69|64.38|46.35|\n|SwiGLU|**68.94**|**75.69**|97.07|**39.34**|**66.58**|44.01|\n\nWe observed that different datasets exhibited distinct preferences: ReLU performed particularly well on BCIC2A, whereas GELU achieved competitive results on Mumtaz2016. Nonetheless, SwiGLU demonstrated the most consistent performance across multiple downstream EEG tasks, indicating its robustness for EEG data. Due to time constraints, these ablation experiments were conducted using models trained without the time-consuming pre-training stage. Inspired by your question, we plan to systematically explore and evaluate the impact of different activation functions during pre-training in future work.\n\n&nbsp;\n\n> [Q2] The appendix is provided separately.\n\n[A8] We apologize for any inconvenience caused by the separate submission of the appendix. When we submitted the main paper, we were still refining the presentation and structure of the appendix to ensure clarity and completeness of our extensive results from our downstream task experiments. Hence, we opted to include it as supplementary material .zip. We sincerely appreciate your patience and careful attention in reviewing the supplementary details.\n\n&nbsp;\n\nReferences:\n\n[1], [2] Attention is All You Need (NeurIPS 2017)\n\n[3] LaBraM (ICLR 2024)\n\n[4] Crossformer (ICLR 2023)\n\n---\n**Thank you again for your valuable comments. We hope our responses have addressed your concerns and strengthened your impression of our work. We remain available throughout the rebuttal period to clarify any further questions you may have. Looking forward to hearing from you :-) !**"}, {"review_id": "PhSM8S39V6", "reviewer": "Reviewer_LoU3", "summary": "This paper proposes a foundation model, NeurIPT, developed for EEG-based neural interfaces. NeurIPT could capture both homogeneous and heterogeneous spatio-temporal characteristics inherent in EEG signals. The designed techniques like Amplitude-Aware Masked Pretraining, PMoE are impressive and well-motivated. Sufficient experiments and analysis are conducted to demonstrate the effectiveness of the model.", "questions": "* What is the numerical range of spatial coordinates?\n* What is the difference between amplitude-aware masking and random patch masking? A larger patch may avoid interpolation.\n* Are the experimental settings in downstream tasks cross-subject, i.e., the test subjects are excluded from the fine-tuning data? It is crucial in tasks like mental disorders diagnosis.\n* Does IILP gathers the representations from all encoder layers? What is the motivation behind this? Most of the methods only use the representations from the last encoder layer.\n* Why the Top k in MoE sets to 0.5?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strength\n\n* The designed techniques are well-motivated. \n* The downstream tasks are sufficient.\n\nweakness\n\n* No information about the statistical significance of the experiments (e.g., error bars, p-value) is provided.\n* Some crucial information in experimental settings is missed.\n* The analysis experiments are insufficient. The analysis of different model components should be conducted on all downstream datasets to varify the effectiveness. The results on only one or two datasets are not convincing.\n* The IILP pooling strategy should be compared with other pooling methods like generalized mean pooling.", "was_revised": true, "final_justification": "The ablation analysis and some clarification issues are addressed in rebuttal.", "rebuttal": "**Dear reviewer LoU3:**\n\n**We appreciate your insightful comments. Below, we respond to each of your concerns.**\n\n---\n\n> [W1] No information about the statistical significance of the experiments.\n\n[A1] We had presented comprehensive details for each downstream task, including statistical metrics (mean, standard deviation), in Tables 11–17 of Appendix E.2 (Downstream Datasets) in the supplementary materials, as we could not include them in the main text due to space constraints and our extensive experiments. Consistent with our baselines (e.g., LaBraM, CBraMod), all reported results are averages computed over five independent runs with different random seeds, accompanied by standard deviation values to facilitate meaningful and rigorous comparisons.\n\n&nbsp;\n\n> [W2] Some experimental settings are missed.\n\n[A2] We respectfully clarify that detailed information on the experimental settings was comprehensively provided across both the main text and the appendix. Should any additional details or clarifications be required, please do not hesitate to let us know if you cannot find any setting. \n\nSpecifically, Sections 3.1 (Pre-training) and 3.2 (Downstream BCI Tasks) introduce the datasets, preprocessing methods, experiment configurations, and evaluation metrics. Additionally, Appendix D (Hyperparameter and Implementation Details) supplements with parameter choices and implementation specifics. Furthermore, Section E.1.2 and Table 10 elaborate on the filtering process and statistics of the pre-training dataset, while Sections E.2.2–E.2.8 deliver extensive descriptions of each downstream dataset. For complete reproducibility, the experimental scripts and hyperparameter configurations are accessible in the directory *./scripts/*, and all preprocessing steps are transparently documented in the folders *./preprocessing/pretrain/* and *./preprocessing/downstream/*. The provided code fully aligns with the manuscript's contents. \n\nWe sincerely apologize for any inconvenience this may have caused, and kindly note that the complete appendix and source code are included within the Supplementary Material zip file.\n\n&nbsp;\n\n> [W3] Insufficient analysis experiments of different model components.\n\n[A3] We have expanded our ablation analyses across more downstream datasets. Specifically, comprehensive experiments for the component IILP have been detailed in [A4]. Below, we provide comparisons focusing on the AAMP, 3D positional encoding, and the PMoE strategies. \n\n|Masking Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|Sleep-EDFx|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Random Masking|67.35|84.38|97.14|33.89|65.64|69.68|\n|AAMP (Ours)|**67.61**|**86.46**|**98.03**|**41.04**|**67.31**|**70.47**|\n\nThese results demonstrate consistent improvements of AAMP across multiple downstream datasets, demonstrating the effectiveness and generalization capability of our proposed masking strategy.\n\n|Encoding Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|\n|:-:|:-:|:-:|:-:|:-:|\n|[1]|67.72|74.65|96.56|35.03|\n|[2]|64.78|71.53|96.63|35.52|\n|[3]|63.81|71.52|95.56|37.40|\n|IILP (Ours)|**68.94**|**75.69**|**97.07**|**39.34**|\n\nalternative methods are included:\n\n- [1] trigonometric functions employed by vanilla Transformer, \n- [2] 1D learnable embeddings, \n- [3] 2D learnable embeddings introduced in LaBraM,\n\nThe results above clearly demonstrate that our proposed 3D electrode embedding achieves consistently superior performance across multiple datasets, substantiating the notable advantage of explicitly leveraging physical electrode positions.\n\n|MoE Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|w/o Expert|65.83|72.92|93.41|39.14|64.53|44.44|\n|Uniform|65.91|70.49|95.00|39.33|65.66|41.93|\n|Shrinking|65.80|73.96|93.08|39.21|65.99|**44.62**|\n|Progressive (Ours)|**68.94**|**75.69**|**97.07**|**39.34**|**66.58**|44.01|\n\nWe have expanded Table 3 by evaluating different MoE strategies across multiple datasets, demonstrating the robustness of our PMoE approach. Furthermore, we have also empirically compared other alternative and gradual configurations:\n\n|PMoE Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|[0,0,2,4,4,6]|**68.94**|75.69|97.07|39.34|66.58|**44.01**|\n|[0,0,2,3,4,5]|67.87|**76.39**|96.81|**39.88**|66.34|41.58|\n|[0,0,2,4,6,8]|66.41|75.69|96.57|39.11|**67.70**|41.15|\n|[0,0,3,6,9,12]|66.08|74.83|**97.15**|39.35|66.63|38.19|\n\nThese alternative patterns also yield competitive performance, suggesting that the effectiveness of the PMoE approach stems from its inherent progressive strategy, rather than relying on any specific expert allocation, demonstrating robustness across diverse progressive configurations.\n\n&nbsp;\n\n> [W4] IILP pooling strategy should be compared with alternative methods.\n\n[A4] We have expanded Table 4 (IILP ablation) by incorporating comparisons between IILP and alternative classification methods, including mean pooling across multiple datasets:\n\n|Pooling Strategies / Datasets|TUEV|MentalArithmetic|Mumtaz2016|SEED-V|PhysioP300|BCIC-2A|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|w/o Pooling|62.33|75.69|78.21|38.90|**67.82**|45.14|\n|Mean Pooling|64.74|79.51|96.22|37.62|66.72|37.24|\n|Hemispheres|64.45|81.94|97.82|39.22|67.11|43.49|\n|Coronal|68.77|73.26|96.99|39.35|66.98|43.75|\n|Sagittal|67.21|80.21|91.41|39.42|65.66|45.31|\n|IILP (Ours)|**68.94**|**86.46**|**98.03**|**41.04**|67.31|**55.04**|\n\nThe above results further demonstrate the consistent superiority of our approach.\n\n&nbsp;\n\n> [Q1] Numerical range of spatial coordinates.\n\n[A5] The spatial coordinate ranges (in $mm$) of the two EEG standard systems involved in downstream tasks are as follows:\n\n- EEG 10-20 system: $x\\in[-86.076,86.169],\\quad y\\in[-118.565,88.247],\\quad z\\in[-68.031,100.244]$\n- EEG 10-05 system: $x\\in[-87.362,88.625],\\quad y\\in[-119.343,88.247],\\quad z\\in[-68.031, 101.269]$ \n\nSpecifically, the coordinates correspond to the fsaverage MRI coordinate system, which is identical to the Montreal Neurological Institute (MNI) coordinate framework, with the origin located at the anterior commissure. These coordinates are extracted from the widely used MNE-Python library. \n\n&nbsp;\n\n> [Q2] Difference between amplitude-aware masking and random patch masking, and impact of patch size\n\n[A6] Our amplitude-aware masking is a context-aware strategy that encourages the model to learn signal-specific characteristics and structural patterns, with amplitude serving as a proxy for signal energy. This approach reduces the risk of trivial reconstruction via simple interpolation methods (e.g., linear), which models might otherwise exploit. \n\nWe appreciate this suggestion regarding the impact of larger patch sizes, which could mitigate trivial interpolation issues inherent in random masking. However, increasing the patch size substantially escalates the complexity faced by the embedding layers, typically implemented as linear transformations. This issue could impose a significant burden on the model's capacity to effectively capture detailed local signal characteristics, thereby negatively affecting downstream performance. \n\nFor instance, in CV, ViT [1] demonstrated notable performance improvements when reducing the patch size from 32 to 16, and subsequent studies [2] found further gains by decreasing the patch size even further down to 1. Similarly, in NLP, mainstream models benefit from fine-grained, byte-level or sub-word-level embeddings. \n\nIn contrast, our proposed amplitude-aware masking strategy (AAMP) intrinsically circumvents the performance-interpolation trade-off, directly guiding the model to learn meaningful signal representations. Empirical evidence supporting our claim has been further validated by experiments in [A3], reinforcing the practical effectiveness of AAMP compared to conventional random masking.\n\n&nbsp;\n\n> [Q3] Are downstream tasks cross-subject?\n\n[A7] Yes, all downstream datasets used in our study adopt a cross-subject evaluation setting, where the test subjects are strictly excluded from the fine-tuning process. This setting is consistent with other baselines as well.\n\n&nbsp;\n\n> [Q4] Does IILP gather all layers? Why?\n\n[A8] Yes, IILP aggregates representations from all layers rather than only the last one. This design choice leverages the hierarchical architecture of our backbone model, which inherently captures EEG signal features at multiple granularities. By combining fine-grained details from early layers with broader-level representations from deeper layers, IILP enriches the final representation, offering a comprehensive, multi-scale perspective beneficial for the downstream classification tasks.\n\n&nbsp;\n\n> [Q5] Why sets topk to 0.5?\n\n[A9] The choice of setting Top-k to 0.5 was a default design decision rather than the outcome of an explicit hyperparameter tuning process. We maintained this default setting uniformly across all downstream tasks and realized it worked satisfactorily without the need for tuning. Nevertheless, we have made this parameter available as a configurable hyperparameter, allowing flexibility for adjustment according to specific downstream requirements or datasets.\n\n&nbsp;\n\nReferences:\n\n[1] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021)\n\n[2] An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025)\n\n---\n\n**Thank you again for your valuable comments. We hope our responses have addressed your concerns and strengthened your impression of our work. We remain available throughout the rebuttal period to clarify any further questions you may have. Looking forward to hearing from you :-) !**"}]

Author rebuttal:

Meta-review: After the rebuttal, most of the concerns and questions were resolved, leading two reviewers to raise their scores by one point. As a result, all four reviewers gave a borderline accept rating. While it is true that none of the reviewers expressed particularly enthusiastic support, the performance across a wide range of benchmarks is solid, and no major concerns remain. On this basis, acceptance is considered a reasonable decision.
Decision: Accept (poster)