CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Authors: Yuchen Zhou
Jiamin Wu
Zichen Ren
Zhouheng Yao
Weiheng Lu
Kunyu Peng
Qihao Zheng
Chunfeng Song
Wanli Ouyang
Chao Gou
Venue: NeurIPS 2025 spotlight ()
Keywords: Brain Foundation Model
Cross-scale
Generalized EEG Decoding
Primary area: neuroscience_and_cognitive_science
TL;DR: We propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding, which effectively captures diverse spatiotemporal neural patterns and achieves state-of-the-art performance on 11 tasks across 16 public datasets.

Abstract: Understanding and decoding human brain activity from electroencephalography (EEG) signals is a fundamental problem in neuroscience and artificial intelligence, with applications ranging from cognition and emotion recognition to clinical diagnosis and brain–computer interfaces. While recent EEG foundation models have made progress in generalized brain decoding by leveraging unified architectures and large-scale pretraining, they inherit a scale-agnostic dense modeling paradigm from NLP and vision. This design overlooks an intrinsic property of neural activity—cross-scale spatiotemporal structure. Different EEG task patterns span a broad range of temporal and spatial scales, from brief neural activations to slow-varying rhythms, and from localized cortical activations to large-scale distributed interactions. Ignoring this diversity may lead to suboptimal representations and weakened generalization ability. To address these limitations, we propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain introduces two key components: (i) Cross-scale Spatiotemporal Tokenization (CST), which aggregates multi-scale features within localized temporal windows and anatomical brain regions into compact scale-aware token representations; and (ii) Structured Sparse Attention (SSA), which models cross-window and cross-region dependencies for diverse decoding tasks, further enriching scale diversities while eliminating the spurious dependencies. CST and SSA are alternately stacked to progressively integrate cross-scale spatiotemporal dependencies. Extensive experiments across 11 representative EEG tasks and 16 datasets demonstrate that CSBrain consistently outperforms both task-specific models and strong foundation baselines. These results establish cross-scale modeling as a key inductive bias for generalized EEG decoding and highlight CSBrain as a robust backbone for future brain–AI research.

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Yuchen Zhou!:2 Jiamin Wu 1., Zichen Ren', Zhouheng Yao', Weiheng Lu', W Kunyu Peng'. Qihao Zheng', Chunfeng Song', Wanli Ouyang! Chao Gou
SShanghai Artificial Intelligence Laboratory Sun Yat-een University The Chinese University of Hong Kong KKarlsruher Institut fur Technologie
Abstract
Understanding and decoding human brain activity from electreencephalograpyy (EEG) signals is fundamental problem in neuroscience and artificial intelligence, with applications ranging from cognition and emotion recognition to clinical di- agnosis and brain computer interfaces. While recent EEG foundation models have made progress in generalized brain decoding by leveraging unified archi- tectures and large-scale pretraining, they inherit scale-agnostic dense modeling paradigm from NI.P and vision. This design overlooks an intrinsic property of neu- ral activity-crree cross-scale spatiotemporal structure. Different EEG task patterns span broad range temporal and spatial scales, from brief neural activations to slow-varying rhythms, and from localized cortical activations to large-scale dis- tributed interactions. Ignoring this diversity may lead to suboptimal representations and weakened generalization ability. To address these limitations, propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain introduces two key components: (i) Cross-scale Spa- tiotemporal Tokenization (CST), which aggregates multi-scale features within localized temporal windows and anatomical brain regions into compact scale-aware token representations; and (ii) Structured Sparse attention (SSA), which models cross-window and cross-region dependencies for diverse decoding tasks, further enriching scale diversities while eliminating the spurious dependencies. CST and SSA are alternately stacked to progressively integrate cross-scale spatiotemporal dependencies. Extensive experiments across 11 representative EEG tasks and 16 datasets demonstrate that CSBrain consistently outperforms both task-specific mod- els and strong foundation baselines. These results establish cross-scale modeling key inductive bias for generalized EEG decoding and highlight CSBrain as a robust backbone for future brain-Al research. The code and model are available at http:/hww....oooccccceee///////.
Introduction
Understanding and decoding human brain activity is a long-standing ambition of neuroscience and artificial intelligence, with implications ranging from cognitive science medical diagnostics and brain-computer interfaces (BCI) [16]. Among various neural recording modalities, electroen- cephalography (EEG) particularly attractive due its non-invasive nature, high temporal resolution, and affordability 17 BEG signals are high-dimensional, multichannel time series that reflect the brain's dynamic electrical activity and are widely applied abnormal identification [T [2]. motor
imagery [3 [1], emotion analysis [15 16]. sleep stage analysis [7 [88, seizure detection [19 20)
and various neurological disorders 21 23 Over the past decade, variety of task-specific deep
"Correspondigg Author (wuj iamin1 n 8p job.b.boo cng.cn)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

35 and Transformers learning Saadells-spannnnng CNNs [2)42], RNNs 29) GNNs [3
36 have been proposed for EEG decoding in the aforementioned applications. Although effective in narrow settings, these models are typically tightly coupled to specific datasets and task formats, limiting their generalizability and scalability [39 40].
Motivated by the paradigm shift in AI from task-specific to foundation models, recent efforts have explored brain foundation models (40 that aim to learn universal EEG representations across diverse brain decoding tasks via unified architectures and large-scale self-supervised pretraining. These models represent a significant step toward generalizable EEG decoding, and most adopt a pipeline transplanted from natural language processing and computer vision 47 32] (as shown in Figure[Ta)):
Figure Ta)): EEG signals are first segmented into fixed-scale tokens, and then dense attention is applied across tokens to model dependencies. However. such acale-agnostic and dense modeling strategies fundamentally mismatch the intrinsic cross-scale spatiotemporal nature of EEG signals, which is critical for accurately capturing task-specific neural patterns.
Specifically, different EEG decoding tasks inherently exhibit distinct spatiotemporal scales. From a temporal perspective, distinct EEG decoding tasks exhibit significant differences in neural dynamics [53 [54] For instance, motor and speech imagine tasks involve transient bursts of activity over short time windows [55] 156, whereas sleep staging relies on slower oscillatory cycles across longer timescales [57]. From a spatial perspective, tasks differ in the activation patterns of brain regions due to the functional specialization of human brains [55] [58). Motor imagery typically involves localized or focal neural activations [59]] while emotion recognition requires coordinated activity across widely distributed brain regions [60], reflecting longer-range spatial dependencies. Such variability spatiotemporal scales across tasks places strong demands on token representations and modeling capacity. Most existing foundation models face three key limitations [61 [2]: (1)
Scale-agnostic tokenization: They rely on a one-time fixed-scale tokenization strategy, resulting in token representations that fail to accommodate the diverse neural patterns across tasks. These tokens often suffer from scale mismatch and semantic dilution, which undermines their effectiveness as the most fundamental modeling units. (2) Structure-agnostic dense attention: Due to the lack of spatiotemporal scale awareness in tokenization, existing foundation models rely on dense attention across all tokens to uncover task-specific neural patterns underlying noisy signals. However, indiscriminately applying attention to suboptimal tokens, without considering the intrinsic cross-scale structures, tends to introduce spurious dependencies due the inherently high noise of EEG signals 14 63, ultimately degrading representation quality and increasing computational overhead. (3) Limited generalization: These limitations collectively hinder the generalization and adaptability of unified EEG foundation models across diverse BCI tasks with heterogeneous spatiotemporal scales.
To address these limitations, we argue that EEG foundation models must move beyond scale-agnostic, dense modeling paradigms and embrace cross-scale, structure-aware architectures that respect the intrinsic regional organisation and temporal dynamics of neural activity. In this paper. we propose CSBrain, a Cross-scale Spatiotemporal Brain foundation Model for generalized EEG decoding. as illustrated in Figure lb). Guided by neurophysiological priors, CSBrain decomposes EEG signals along spatial and temporal axes into distinct brain regions and windows, and captures cross- scale dependencies both within and across these structures, yielding robust and highly adaptable representations. Specifically, we propose a Cross-scale Spatiotemporal Tokenization (CST) module, which captures EEG features at multiple temporal and spatial scales within localized regions. By integrating these patterns into compact token representations, CST enables adaptable alignment with diverse spatiotemporal requirements of EEG decoding tasks. Building upon these scale-aware tokens, we propose a Structured Sparse Attention (SSA) module to further enrich the diversity of modeling scales for EEG. SSA captures long-range dependencies across temporal windows and brain regions in a structured and efficient manner. By replacing costly dense attention. SSA reduces spurious dependencies and computational overhead, yielding more discriminative representations for noisy EEG signals. Finally, CST and SSA are alternately stacked and mutually reinforcing, progressively integrating cross-scale dependencies across both temporal and spatial dimensions. This hierarchical design enables CSBrain to build robust EEG representations that reflect the complex patterns of neural activity, thereby achieving superior generalization across EEG decoding tasks with inherently diverse spatiotemporal demands. CSBrain fundamentally differs from prior EEG foundation models by explicitly addressing and leveraging the intrinsic cross-scale spatiotemporal structure of neural activity, a property often overlooked by previous scale-agnostic dense modeling paradigms. Overall, our contribution can be summarized as follows:

---

Previous Brain Foundation Models
(c) Performance Comparison on 11 Tasks (16 Datasets)
A MY
CA
Masir Pemgory
Allrcci dlgedii
F4
Chusphhioe
Pichpabon
Scale-agnoscic Tokenúsation
Dense Spotiotemporal Attention
(b) CSBrain
Intro-window
staag
Erepe Tyee Conidiaaton
P4
Inter window
Abnaypic
Inten-ragion
Seaar
MentaGoriia
Drew
F4
ECCNe
DAT
Cross-ccnle Spatiotemporel Tokenication
Structured Attention
Comparison between previous brain foundation models and proposed CSBrain. (a) Prior models adopt a scale-agnostic and densely connected architecture, which fails to reflect the intrinsic cross-scale spatiotemporal nature of EEG signals. (b) To address these limitations, we propose a cross-scale, sparsity-aware architecture that enables efficient and robust EEG representation learning, supporting generalization across brain decoding tasks with diverse spatiotemporal demands. (c) CSBrain outperforms both task-specific and foundation baselines on 11 representative tasks across 16 datasets, demonstrating consistent and superior performance.
Figure I:
We propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized brain decoding, offering cross-scale, structure-awans architecture that effectively captures multi-scale spatiotemporal neural patterns across diverse tasks.
We introduce Cross-scale Spatiotemporal Tokenization (CST) to explicitly integrate multi- scale neural patterns within temporal windows and brain regions into compact, scale- aware tokens. Additionally, Structured Sparse Attention (SSA) is designed to efficiently capture long-range dependencies across windows and regions, thereby enriching modeling scale diversity. Alternately stacked, CST and SSA progressively integrate cross-scale scale dependencies, enabling robust EEG representations across tasks.
Extensive experiments across 11 representative EEG decoding tasks and 16 public datasets demonstrate CSBrain's superiority over both task-specific and foundation models. Further analysis reveals diverse cross-scale patterns across tasks, establishing cross-scale modeling key inductive bias for generalized EEG decoding.
2 Model Architecture
The overall framework of CSBrain is illustrated in Figure [ The model first applies standardized preprocessing module to extract initial feature representations from raw EEG signals. These features then fed into the Cross-scale Spatiotemporal Tokenization (CST) module, which constructs robust cross-scale tokens by aggregating multi-resolution information within localized temporal windows and anatomically defined brain regions. Built upon these semantically enriched tokens, the Structured Sparse Attention (SSA) module expands the modeling scope, capturing long-range dependencies across time and brain regions while avoiding redundant interactions. CST and SSA alternately stacked for L layers, progressively integrating cross-scale spatiotemporal dependencies. Finally, lightweight task-specific head attached to support various objectives. including masked reconstruction during pretraining and classification or regression for downstream tasks.
2.1 EEG Signal Preprocessing
Initially, we formalize the input EEG signals as F. E RRCG/ where Cn C C denotes the set of electrodes used, and T denotes the number of timestamps. C corresponds the standardized

---

(a) CSBrain
(b) Cross-scale Spatiotemporal Tokenization (CST)
Temporal Tokenization
Spatial Tokenization
Coftii
& +
XII
Gratto
EEG Signal Preprocessing
Intra-window
Intra-region
Cross-scale Spatiotemporal Tokenization
(c) Structured Sparse Attention (SSA)
Structured Sparse Attention
Grouping
Temporal
Attention
Inter-window
Grouping
Spatial
Attention
inter-region
Layer Norm
Feed Fonward
Task Head
Concatenation
Addition
Figure 2: Overview of the CSBrain architecture. After EEG signal preprocessing, the Cross- scale Spatiotemporal Tokenization (CST) module encodes multi-resolution features within localized temporal windows and brain regions to produce robust, scale-aware tokens. The Structured Sparse Attention (SSA) module then captures long-range dependencies across windows and regions in a structured and efficient manner. CST and SSA alternately stacked for L layers to progressively integrate cross-scale spatiotemporal dependencies to build unified and robust representations for diverse BCI tasks with varying spatiotemporal scales. Finally, lightweight task head is appended for reconstruction, classification, or regression.
international 10-20 system. To enhance signal quality and ensure feature consistency across datasets, we follow prior works 41 40) and apply a standardized signal preprocessing pipeline.
Signal Standardization. The raw signals E are processed with a band-pass filter to remove low- frequency drifts and fighfrreuueccy noise, followed by notch filter eliminate power line interfer- ence. The signals are then resampled to uniform rate of 200 Hz and scaled to 100 μV to normalize the amplitude range [-1, 1J. To enable representation learning. the preprocessed EEG signal is temporally segmented into non-overlapping segments of length i, resulting in E, E R CCxxtt where = [F] is the number of segments per electrode.
Preliminary Feature Encoding. We extract both temporal and spectral features from each segment in Ep. Local temporal dynamics are captured using ID convolutions with normalization, while spectral information is obtained by applying Fast Fourier Transform (FFT) followed by fully connected layer to encode frequency energy distributions. The two feature types are fused to form unified feature embedding. Finally, add learnable positional encoding [46), resulting in the initial EEG representation z cCxxxd, where d denotes the feature embedding dimension.
2.2 Cross-scale Spatiotemporal Tokenization (CST)
Due to the inherent cross-scale spatiotemporal nature of EEG signals, informative neural patterns emerge heterogeneous temporal and spatial resolutions. Previous EEG decoding methods adopting a fixed token granularity cannot effectively capture such patterns, limiting its generalizability across diverse BCI tasks. To address this, propose Cross-scaa Spatiotemporal Tokenization (CST), which encodes multi-resolution information within localized temporal windows and anatomically defined brain regions into unified token representations. These tokens form the basic modeling units for subsequent attention computation. In the following, we detail the two components of CST: Temporal Tokenization, which captures multi-scale temporal patterns within local time windows (i.e., intra-window). and Spatial Tokenization, which extracts region-specific features within anatomical brain regions (i.e., intra-region).
Temporal Tokenization. To enable tokenization at various temporal resolutions, we introduce multi- scale temporal convolution kernels Cf {Conv? (k) (k) Ik=1, where K denotes the number of kemels

---

and each temporal kernel ?
(k) has kernel size st and output embedding dimension dy. Given the preprocessed EEG feature r((-1) E RCxxnd, for each location (i,j), where i E {I,...,n.}
and j E {1....,y) CJ denotes the temporal and electrode channel indices, respectively, we define localized temporal window (/ (t) centered at i for the k-th scale with size (k), Subsequently, the multi-scale temporal convolution is applied over these windows to extract temporal patterns at varying resolutions for each location. The process can be formulated as:
Concat
{Coonv (8) (W! ())) Proj (r.,
(1)
where Proj, is a residual projection for dimension alignment. The outputs from different temporal
kernels are concatenated to form temporally cross-scale token "i.." which captures rich temporal dynamics within each channel. The resulted token representations could enhance model's robustness to signal noise and scalability to diverse decoding tasks with varying temporal patterns.
Spatial Tokenization. Similar to Temporal Tokenization, we design multi-scale spatial convolution kernels Cs {Cony? (k) K Jkmi with each spatial kernel Cony
(k) has a kernel size st and output embedding dimension dg. To encode spatial context across functionally related electrodes in each brain region at multiple scales, we utilize these convolution kernels to aggregate electrode features within anatomically defined brain regions. Specifically, given EEG feature a(() E IR Cxxxd after Temporal Tokenization, we first divide electrodes into a set of brain regions R {R, } based on the 10-20 system, where each region R, comprises spatially adjacent electrodes. For each electrode
&
j E R,,, we define localized spatial neighborhoods w!()) (j) S R, with size (2) within brain region for the k-th scale. Finally, the spatial tokenization is performed by applying multi-scale spatial convolution kernels their corresponding neighborhoods:
i. Concat ((oonv (k) (W! (M)) Proj, (roj. (2
where Proj, is a residual projection for dimension alignment. The outputs from different spatial kernels are concatenated form the final cross-scale token representation a capturing localized spatial dependencies within each brain region. Figj encapsulates spatiotemporal characteristics across diverse scales, serving as the final output of CST module and providing robust scale-aware features for downstream attention computation.
To balance representational capacity and computational efficiency, we allocate embedding dimensions across scales in an exponentially decaying scheme: dy о 2 Σk= d4 d, i.e., assigning higher dimensions smaller kernels retain fine-graines features, while allocating lower dimensions to larger kernels that summarize coarse context. This strategy aligns representational capacity with information density at each scale for balanced and efficient EEG representation.
2.3 Structured Sparse Attention (SSA)
Prior approuches typically apply dense attention uniformly across fixed-scale tokens. While effective in some cases, such indiscriminate attention introduce spurious dependencies, reduce repre- sentational quality, and increases computational costs. Meanwhile, our CST provides structured, cross-scale token representations within localized temporal windows and brain regions, laying robust representational space for further modeling. Building upon this, we propose Struetured Sparse Attention (SSA), which efficiently captures long-range dependencies across temporal windows and spatial regions while avoiding redundant interactions. SSA complements CST along both temporal and spatial scales, and together they enable structured cross-scale modeling of EEG representations. SSA consists of two components: wnier-window Altention, responsible for capturing long-range temporal dependencies across local windows, and Inter-region Attention, which models spatial dependencies across distinct brain regions.
wntdrw Attention. Given the cross-scale tokens =!!) Cxxxd we first perform temporal grouping operation to form wonsww groups. Specifically, for each relative index g {1,..., w) we collect all tokens that occupy the same position in each window to form a temporal group g(o). Self-attention is then computed within each group to model structured long-range dependencies across time:
((.ii))
(1) Attn(G Ja.s Z..
(3)
5

---

Reconstruction Head
Musked
CST
UST
M
MWM VVV
wwwm
M
wwm
EEG Signals
Masking
Learned EEG Representations
Masked EEG Reconstruction
Figure 3: The overview of CSBrain pre-training process.
This design enables efficient and structured temporal interaction while avoiding redundancy.
Inter-region Attention. To ensure structured sparsity while maintaining coverage across all elec- trodes, we perform spatial grouping operation to construct multiple spatial groups across brain regions. Unlike temporal windows (which have fixed lengths), the number electrodes varies across brain regions. In regions with more electrodes, CST might not fully capture all information. By introducing the regional descriptor, we ensure complete information flow between regions. For each group, sequentially sample a token x,, from each region Ry, and combine it with the average feature R, to form covering partition of the regional descriptor:
i J.CP + (P(())),
I...,, R.
where P(R,) denotes the mean-pooled feature region R,, and Φ(-) is learnable linear transforma- tion. Notably, this ensures that each token is enriched by its region's context while preserving original information before inter-region attention and does not physically rearrange token positions. These
descriptors are assembled into spatial group g! {ē1, -2. ..... .n). over which self- attentio is applied to capture structured dependencies across anatomical regions:
(f.eg) Autn(g!)) + i.j ijj
((.win))
(5)
Finally, we apply layer normalization and feedforward network with residual connection to refine the output:
(1) +
FFN LLN
((.rg) + 15
Together, SSA enables CSBrain to efficiently model long-range dependencies across both temporal and spatial dimensions by leveraging the robust cross-scale token layout provided by CST. This design progressively builds global context while preserving scale-awarengs and mitigating spurious correlations in noisy signals, thereby enhancing generalization across diverse EEG decoding tasks.
2.4 Pretraining with Masked Autoencoding
To learn generalizable EEG representations. we adopt a self-supervised pretraining strategy based on masked autoencoding [5T] 64 shown in F This process encourages the model to capture meaningful spatiotemporal dependencies from unlabeled EEG signals, laying the foundation for effective downstream transfer.
Masking Strategy. Given a preprocessed EEG signal Ep E _Cxxxt we randomly mask a fixed ratio r (0, 1) of segments along the temporal axis using Bernoulli sampling scheme. This results in two subsets: visible segments E,, E C'x, xt and masked segments Em E RR Cx where n, + 7'm
and Pm r. The masked EEG sequence is encoded by alternately stacked CST and SSA modules, which jointly model neural dependencies across multiple spatiotemporal scales. Masked segments are represented by learnable embeddings and integrated alongside visible tokens during encoding.
Reconstruction Objective. The reconstruction head consists of a lightweight fully connected layer, which takes as input the encoded visible tokens and learnable masked embeddings, and projects them into the original EEG signal space. Let Ê., E C' xt denote the predicted EEG segments and Em the corresponding ground truth. The reconstruction loss is defined the mean squared error (MSE) over all masked positions:
L - f - E.
This masked autoencoding objective drives the model to recover meaningful spatiotemporal depen- dencies from context, yielding robust and transferable EEG representations.

---

Table 1: Overview of Downstream BCI datasets and tasks used for evaluation.
Task
Dataset
Rate (Hz)
#Channels
Duration
#Samples
#Subjects
Label
BCIC-IV-2a
5.184
Motor Imagery Classification
PlysioNet-Mi
160
9,837
109
4-class
SIU--MI 68
250
32
11,988
2 2-clasa
Emetion Recognition
FACED 69
SEED-V 70
1000
250
62
32
10s
1s
117,744
10,332
123
16
5-class
9-class
Seizure Detection
Siema 7 kk
256
326.993
51,307
2-class
Sleep Staging
ISRUC 74
HMC 16
256
200
30s
30s
137.243
89,240
151
100
5-class
4-class
Imagined Speech Classification
BCIC2020-3
256
6,000
15
5.class
Vigilance Estimation
VIG
20,355
regression
Mental Stress Detection
MentalArilmmttic
X1 kk
500
1,707
36
2-class
Mental Disorder Diagnosis
Mumtaz2016
78
256
5s
7,143
64
2-class
Eveut Type Classification
256
112.491
370
6-class
Abnormal Detection
TUAB
79
256
409.455
2,383
2-class
Slowing Event Classification
TUSL
256
23
10s
245
28
3-class
3 Experiments
3.1 Pre-training Setup
Data Preprocessing, We pre-train CSBrain on the Temple University Hospital EEG corpus (TUEG) dataset, which has been shown effective for foundation model studies [46 65). We apply standard preprocessing steps: signals are band-pass filtered between 0.3-75 Hz to remove low- and high- frequency noise, and notch filter at 60 Hz is used to eliminate power line interference. All signals are resampled to 200 Hz. and segmented into 30-second EEG samples. The amplitude is normalized
to 100 μVV to ensure the signal range falls within [-1, 1). consistent with prior work 41 46) After cleaning, a total of 1,109,545 5,45 EEG segments (over 9,000 hours) are used for pre-training.
Pre-training Settings. We adopt masked autoencoding objective, with masking ratio of 50% applied to randomly sampled EEG patches. Pre-training is conducted using Python 3.11. and PyTorch 2.5.1 with CUDA 12.4. The model is trained with the AdamW optimizer, learning rate of 5e-4, weight decay of 5e-2, and cosine annealing learning rate scheduling. We use batch size of 128 and train for 40 epochs on 4 NVIDIA A 100 GPUs, with total pre-training time of approximately 101 hours. More details can be found in the supplementary material.
3.2 Experimental Setup of Downstream BCI Tasks
Tasks & Datasets. To comprchensively evaluate the generalizability of our model, we conduct experiments on 11 representative BCI tasks spanning 16 publicly available EEG datasets, as summarized in T These tasks include Motor Imagery Classification (BCIC-IV-2a, PhysioNet- MI, SHU-MI), Emotion Recognition (FACED, SEED-V V)), Seizure Detection (CHB-MIT -MIT, Siena), Sleep Staging (ISRUC, HMC), Imagined Speech Classification (BCIC2020-3), Vigilance Estimation (SEED- VIG), Menial Stress Detection (Mental Type Classification (TUEV), Abnormal Detection (TUAB), and Slowing Event Classification (TUSL). In all experiments, we strictly follow the training. validation, and test splits to ensure fair and consistent evaluation. To the best of our knowledge, this among the most comprehensive evaluations conducted EEG foundation models to date. See supplementary for details.
Menial Disorder Diagnosis (Mumtaz2016), Event
Baselines & Metries. We extensively compare CSBrain with two major categories of baselines: (1) representative task-specific EEG decoding models, including EEGNet (24), EEGConformer
(Conformer) [6) SPaRCNet (25. ContraWR (26), CNN-Transformer (-Tarnn)) 38), FFCL 27].
and ST-Transformer (rann) 37]: and (2) recent EEG foundation models, including BIOT 40
[40] 46), we report Balanced Accuracy, Cohen's 's Kappa, and Weighted F1 for multiclass classification tasks; Balanced Accuracy, AUC-PR,
LaBraM [1) and CBraMod [46]. Following prior works
and AUROC for binary classification; and Pearson correlation, R2 score, and RMSE for regression tasks. See supplementary for details.
7

---

Table 2: Performance comparison on 11 BCI tasks Using 16 datasets.
Datasel
Metries
EEGNet
Conformer SPaRCNet
ContraWRCNN-Tiaa
FFCL
ST-Traas
BIOT
LaBraM
CbraMMod
BCIC-IV-2a
B-Acc
FL-W
0.4226
0.4482
0.4533
0.46%6
0.4432
0,4635
0.4413
0,4678
0.4560
0.4500
0.4570
0.4288
0.4571
0.4755
0.4607
0,4748
0.4758
0.4869
0:5138
0.4984
0.5687
0.5637
SHU-MI
B-Ace
AUROC
0.5889
0.6283
0.5900
0.6351
0.6431
0.5978
0.5873
0.6273
0.6343
0.5975
0.6326
0.5692
0.5992
0.6431
0.6179
0.6609
0.6604
0.6166
0.6570
0.6988
0.7200
0.6417
PhysioNer-MI
FL-W
B-Acc
0.5814
0.5796
0.6049
0.6062
0.5932
0.5937
0.5892
0.5918
0.6053
0.6041
0.5726
0.5701
0.6035
0.6053
0.6153
0.6158
0.6133
0.6177
0.6174
0.6179
0.6348
0.6344
BCIC2020-3
B.Acc
0.4413
0.4506
0.4426
0.4257
0.5333
0.4678
0.4126
0.4920
0.5060
0.5373
0.6004
FI-W
0.4413
0,4488
0.4420
0,4407
0.4066
0.4689
0.4247
0.4917
0.5054
0.5363
0.6003
SEED-V
FL-W
B-Acc
0,2749
0966
0.3487
0.3537
0.2979
0.2944
0.3546
0.3544
0.3678
0.3642
0.3645
0.3641
0.3052
0.2833
0.3837
0,3856
0.3976
0.3974
0.4091
0.4101
0,4197
0.4280
FACED
FL-W
B-Acc
0.4090
0.4124
0,4589
0.4514
0.4673
0.4729
0.4887
0,4884
0.4897
0,4702
0.4673
0.4599
0.4810
0.4795
0.5118
0,5136
0.5273
0.5288
0.561
0.5509
0.5752
0.57%
TUEV
B-Ace
0.3876
0.6539
0.7024
0.4844
0.6893
0.6854
0 0.3979
0.6823
0.3984
0.749/
0.6409
0.8312
0,8342
0.6903
TUAB
B.Acc
AUROC
0.7642
0.8412
0.8445
0.8676
0.7896
0.7746
0.8456
0.8461
0.7848
0.8569
0.7966
0.8707
0.7950
0.8815
0.140
0.9022
0.8606
MentalAArihhmetic
B-Ace
AUROC
0.7521
0.6770
0.7424
0.68
0.7418
0.6879
0.6631
0.7392
0.7258
0.679
0.7330
0.6798
0.7132
0.6631
0.7536
0.6575
0.7721
066999
D.7256
0.7905
D.755N
0.8478
CHB-MIT
B-Ace
AUROC
0.8048
0.5658
0.8226
0.5976
0.8143
0.5876
0.8097
0.6344
0.8662
0.6389
0.622
0.8271
0.8237
0.5915
0.8761
0.7068
0.8679
0.7075
0.8892
0.7398
0.8915
0.7262
Siena
B-Acc
AUROC:
0.8687
0.7487
0.8159
0.7536
0.7534
0.6522
0.6546
0.7819
037199
0.6982
0.6616
0.8144
0.8844
0.7527
0:5029
0.7552
0.8814
0.7082
D.7517
D.9098
0.7662
0.9076
SEED-VIG
R2
Con.
0.5127
0,1960
D S800
0.2065
0.2185
0.5709
0.0727
0,5235
0.1796
0.5829
0.1740
0.4925
0.1138
0.6020
0.1232
0.6114
0.1808
0.6347
D.5502
0.0737
DA514
0,2363
ISRUC
FL-W
B-Asc
0,7513
0.7154
0.7400
0.7634
0.7624
0.7487
0.7402
0,7610
0.7719
0.7363
0.7277
0.7614
0.7681
0.7381
0.7790
0.7527
0.7810
0.7633
0.8011
0.7865
0.79%0
0.7925
HMC
FI-W
B-Aoc
06634
0.6536
0.7149
0.7080
0.4566
0.4108
0,2987
0.4242
0.6573
0.6896
0.2902
0.4427
0.2559
0.1428
0.7091
0.6862
0.7277
0.7454
D.7269
0.7395
0L7345
Dl7506
TUSL
FL-W
B-Ase
0.4562
0.4105
0.5512
0.1895
0.4185
0.3500
0.5857
0,5155
0.2235
0.3575
0.3159
0.2120
0,4000
0.3793
0.5785
0,2391
0.7625
0.761
0.7388
0,7453
0.8568
0.8571
Mumtaz2016
R-Acc
AUROC
0.9232
0.9639
0.9308
0.9702
0.9316
0.9781
0.9195
0.9621
0.9305
0.9742
0:9314
0.9753
0.9135
0.9594
0.9358
0.9758
0.9782
0.9409
0.9560
0.9921
0.9956
Macro-average
0,5886
0.5817
0,58/19
0.6007
0
0.5686
0.6322
0.6697
0.6760
0.7095
3.3 Experimental Results
Performance Comparison. To fairly and comprehensively evaluate the effectiveness of CSBrain, we benchmark it against 10 strong baselines across l representative BCI tasks covering 16 public EEG datasets, as summarized in Table 2
All results are averaged over five runs with different random seeds. Due to space limitations, we report two representative metrics per dataset in the main paper: Balanced Accuracy (B-Acc) and Weighted F1 (F1-W) for multi-class classification, B-Ace and AUROC for binary classification, and Pearson correlation (Corr.) and R2 score for regression tasks. Full results, including standard deviations and detailed analyses, are provided in the supplementary material.
As shown in Table 2 we highlight two key observations: (1) CSBrain consistently achieves state-of- the-art performance across nearly all tasks and metrics. In particular, it significantly outperforms all baselines on datasets such as BCIC-IV-2a, BCIC2020-3, Siena, and TUSL. Even in cases where CSBrain ranks second on certain metries, the performance gap to the best-performing method remains marginal. From the macro-average in the last row of Table 2 CSBrain achieves the highest overall score across all tasks, outperforming strong foundation models such as CbraMod, LaBraM, and BIOT by 3.35%, 3.98%, and 7.73%, respectively. These results demonstrate CSBrain's 's strong generalization and adaptability to diverse EEG decoding scenarios with varying spatiotemporal demands. (2) Foundation models outperform task-specific models by significant margin. Across all 16 datasets, EEG foundation models--ncluuing BIOT, LaBraM, CbraMod, and DiBrain-coossstentty outperform task-specific models. This also validates the advantage of unified model architectures trained with large-scale pretraining, which can extract generalizable neural representations across heterogencous EEG domains.
Tokenization Comparison. To investigate how spatiotemporal scale affects EEG representation from the perspective of token construction, we vary the number of kernels K in the CST module to control token granularity. Specifically, we compare single-scale (K 1), dual-scale (K 2), and full cross- scale (K- = 3) configurations. Additionally, to evaluate the effect of stacking, we compare our full design (with CST applied before each SSA module) against a variant where cross-scale tokenization (K = 3) is applied only once before the first SSA. For efficiency. all variants are pre-trained and fine-tuned using 30% of the training data. As shown in Fig. our SSA (K = 3, stacked) consistently achieves the best performance across four representative tasks: emotion recognition, motor imagery,
8

---

Padkeroogg 1980
muggr 20c
3.30
K-1
K-3 (deelad)
10-3 (oree)
Figure 4: The performance comparisons of tokenization.
Ever Toe Cororlade
Care PiPh
Clevltaatoe (T.Pa
Sprec I9C)
Inidhtt Sornh PCC)
Inodect Soorch 2000
Atterin
Disse Atantion
Souctored Sparee Attettion
Figure 5: The performance comparisons of attention mechanism.
event classification, and imagined speech. In contrast, the single-scale variant (K 1) performs the worst. Notably, on the motor imagery task, SSA outperforms the single-scale counterpart by 10.8%, 20.7%, and 12. % on three evaluation metrics, respectively. Moreover, observe that alternatly stacking SSA modules throughout the network consistently outperforms applying SSA only once. This is because single application of SSA before attention computation may fail to fully capture the broader cross-scale dependencies. These results highlight the importance of the proposed SSA for capturing diverse EEG dynamics across tasks.
Attention Mechanism Comparison. Furthermore, we compare different types of attention mech- anisms to assess their impact on EEG decoding performance and computational efficiency. (1) Dense Attention performs full self-attention over all tokens with quadratic complexity O(V2), where N = x C. While theoretically expressive, it introduces spurious dependencies. weakens rep- resentation quality in noisy EEG scenarios, and incurs substantial computational overhead. (2) Criss-cross Attention, which restricts attention to shared temporal spatial indices, yielding approxi- 1.5):
mate complexity O(N1.9); )): and (3) Our Structured Sparse Attention (SSA) limits attention to grouped tokens across temporal windows and brain regions, achieving linear complexity O(N k) with small
< VN. As shown Figure 5 SSA consistently achieves the best performance across Balanced Accuracy, Cohen's Kappa, and Weighted F1. For instance, on the TUEV dataset, SSA improves Balanced Accuracy by +8.1% and Weighted F1 by +5.3% compared to Criss-cross Attention, and maintains low computational cost. These results highlight SSA as an effective and scalable inductive bias for EEG modeling.
Topography Visualization. To further examine the spatial dynamics captured by CSBrain, we visu- alize activation topographies across different EEG decoding tasks. Specifically, we apply Gradient- weighted Class Activation Mapping (Grad-CAM) [80 to compute the contribution of each EEG channel to the model's predictions, as shown in Figure Different tasks elicit distinct activation patterns and scales. Vigilance states primarily activate the temporal and occipital lobes, indicating continuous engagement of auditory and visual systems. Motor imagery evokes highly localized activations over the contralateral motor cortex, reflecting ERD/ERS (event-related desynchronization synchronization) phenomenon [T]. In contrast, emotion recognition and imagined speech exhibit broad, distributed activations. Emotion tasks elicit significant activation across the frontal and occipi- tal lobes, consistent with the findings [32]. Speech imagery evokes significant activation in frontal and temporal lobes [82), which are critically involved in imagined speech processing [3]] These observations further validate that different EEG decoding tasks exhibit distinct activation regions and scales, reinforcing the necessity of cross-scale modeling for EEG foundation models. By explicitly modeling cross-scale structure, CSBrain effectively adapts to task-specific neural patterns across diverse brain decoding scenarios.

---

Vgliener Eatimation (SEEDWIG, viglance)
Moter Imagery Clavsfication IBCHWV2,, left hand)
Emotion Aecognition ISEED v sadi
Imapined Speech Cionflcation (SCK22020 3, hell'l
Figure 6: Topography visualization on four different tasks. More in the supplementary materials.
4 Conclusion
In this work, we demonstrate that modeling the cross-scale spatiotemporal structure of EEG signals is essential for building generalizable brain foundation models. Through unified architecture composed of Cross-scale Spatiotemporal Tokenization and Structured Sparse Attention, our proposed CSBrain effectively captures neural dependencies spanning multiple spatiotemporal scales. Experimental results across 11 EEG decoding tasks and 16 datasets validate the effectiveness of CSBrain. These findings establish cross-scale spatiotemporal modeling as critical inductive bias for robust, scalable, and mhyssologically-alignee EEG representation learning. We believe this work lays solid foundation for future advancements brain-Al integration and generalized neural decoding.
Acknowledgements
This work was done during Yuchen Zhou's 's internship at Shanghai Artificial Intelligence Laboratory. This work was supported by Shanghai Artificial Intelligence Laboratory, the JC STEM Lab of AI for Science and Engineering. funded by The Hong Kong Jockey Club Charities Trust, the Research Grants Council of Hong Kong (Project No. CUHK14213222 14213224). National Natural Science Foundation of China under Grant 62373387, Natural Science Foundation of Guangdong Province (No. 2033A1515030264), Beijing Natural Science Foundation (4252048), and the Shenzhen Fundamental Research Program (Grant No. JCYI202408
References
[1] Gert Pfurtscheller and Christa Neuper. Motor imagery and direct brain-computer communication. Proceedings of the IEEE, 89(7):113-11144, 2001.
[2] John-Dylan Haynes and Geraint Rees. Decoding mental states from brain activity in humans. Nature reviews neuroscience, 7(7):523-534, 2006.
[3] Célia Lorictte, Julian Amengual, and Suliann Ben Hamed. Beyond the brain computer interface: Decoding brain activity as a tool to understand neuronal mechanisms subtending cognition and behavior. Frontiers in Neuroscience, 16:81 1736, 2022.
[4] Yuqin Dai, Zhouheng Yao, Chunfeng Song, Qihao Zheng, Weijian Mai, Kunyu Peng, Shuai Lu, Wanli Ouyang, Jian Yang, and Jiamin Wu. Mindaligner: Explicit brain functional alignment for cross-subject visual decoding from limited fmri data. arXiv preprint arXiv:2502.00 050044 2025.
[5] Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, and Chunfeng Song. Adabrain-hench: Renchmarking brain foundation models for brain-computer interface applications. arXiv preprint 2500:2:2000002222 2025.
[6] Weiheng Lu, Chunfeng Song. Jiamin Wu, Pengyu Zhu, Yuchen Zhou, Weijian Mai, Qihao Zheng, and Wanli Ouyang. Unimind: Unleashing the power of lims for unified multi-task brain decoding. arXiv preprint 2rXiv:2506. 11.550688990 18962, 2025.
[7] Gernot R Müller-Pute: Electreencehhaloguphy. Handbook of clinical neurology, (68:249-262, 2020.
[8] Sharlene Flesher, John E Downey, Jeffrey M Weiss, Christopher Hughes, Angelica Herrera, Elizabeth Kabara, Michael L Boninger, Jennifer L Collinger, and Robert A Gaunt. A brain- computer interface that evokes tactile sensations improves robotic arm control. Science, 372(6544):8311 836, 2021.
10

---

[9] Hongbian Li, Hyonyoung Shin, Luis Sentis. Ka-Chun Siu, José del R Millán, and Nanshu Lu. Combining with electroeneephalogripyy frontier of brain-computer interfaces. Device, 2(6). 2024.
[10] Zhangiang Guo. Jiamin Wu, Yonghao Song. Jiahui Bu, Weijian Mai, Qihao Zheng, Wanli Ouyang, and Chunfeng Song. Neuro-3d: Towards 3d visual decoding from eeg signals. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 23870-23880, 2025.
[11] Subhrajit Roy, Isabell Kiral-Koreek, and Stefan Harrer. Chrononet: A deep recurrent neural network for abnormal eeg identification, In Artificial Intelligence Medicine: Conference Artificial Intelligence Medicine. pages 47-56. Springer, 2019.
[12] Boualem Boashash and Samir Ouelha. Automatic signal abnormality detection using tmee-requenny features and machine learning: A newborn cog scizure case study. Knowledge- Based Systems, 106:38-50 50, 2016.
[13] Ali AI-Saegh, Shefa A Dawwd, and Jassim Ahdul-Jabbar. Deep learning for motor imagery eeg-based classification: A review. Biomedical Signal Processing and Control, 63:102172, 2021.
[14] Xianlun Tang. Jing Zhang, Yidan Qi. Ke Liu, Rui Li, and Huiming Wang. A spatial filter temporal graph convolutional network for decoding motor imagery eeg signals. Expert Systems with Applications. 238: 121915, 2024.
[15] Nazmi Sofian Suhaimi, James Mountstephens, and Jason Teo. Eeg-based emotion recognition: state- of-the-art review current trends and opportunities. Computational intelligence and neuroscience, 2020(1):8875466, 2020.
[16] Sheng Dai, Ming Li, Xu Wu, Xiangyu Ju, Xinyu Jun Yang. and Dewen Hu. Contrastive learning of ceg representation of brain area for emotion recognition. IEEE Transactions on Instrumentation and Measurement, 2025.
[17] Chaoqi Yang. Danica Xiao, Brandon Westover, and Jimeng Sun. Self-supervised eeg representation learning for automatic sleep staging. JMIR AI, 2023.
[18] Jiquan Wang. Sha Zhao, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. Generalizable sleep staging via multi-level domain alignment. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 38, pages 365-273, 2024.
[19] Ijaz Ahmad, Xin Wang, Danish Javeed, Prabhat Kumar, Oluwarotimi Williams Samuel, and Shixiong Chen. A hybrid deep learning approach for epileptic seizure detection in eeg signals. IEEE Journal of Biomedical and Health Informatics, 2023.
[20] Sheng Wong. Anj Simmons, Jessica Rivera- Villicana, Scott Barnett, Shobi Sivathamboo, Piero Perucca, Zongyuan Ge. Patrick Kwan, Levin Kuhlmann. Rajesh Vasa, et al. Eeg datasets for seizure detection and prediction review. Epilepsia Open, 8(2):252-22) 267, 2023.
[21] Md Nurul Ahad Tawhid, Siuly Siuly, Kate Wang, and Hua Wang. Automatic and efficient framework for identifying multiple neurological disorders from eeg signals. IEEE Transactions on Technology and Society, 4(1):76-86, 2023.
[22] Ying Yan. Guanting Liu, Haoyang Cai, Edmond Qi Wu, Jun Cai, Adrian David Cheok, Na Liu, Tao Li, and Zhiyong Fan. A review of graph theory- based diagnosis of neurological disorders based eeg and mri. Neurocomputing, page 128098, 2024.
[23] Yihe Wang. Nan Huang, Taida Li, Yujun Yan, and Xiang Zhang. Medformer: A multi quatilgranuulaiity patching transformer for medical time-series classification. In The Annual Conference on Neural Information Processing Systems, 2024.
[24] Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung. and Brent J Lance. Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces. Journal of neural engineering, 15(5)0056013, 2018
[25] Jin Jing, Wendong Ge, Shenda Hong, Marta Bento Fernandes, Zhen Lin, Chaoqi Yang, Sungtae An, Aaron Struck, Aline Herlopian, loannis Karakis, et al. Development of expert-level classification of seizures and rhythmic and periodic patterns during eeg interpretation. Neurology, 105(17)ee750- 2023.
[26] Chaoqi Yang, Cao Xiao, M Brandon Westover, and Jimeng Sun. Sulf-supervised electrenccehaaloraan representation learning for automatic sleep staging: Model development and evaluation study. JMIR AI, 2, 2023.
11

---

[27] Hongli Li, Man Ding, Ronghua Zhang, and Chunbo Xiu. Motor imagery eeg classification algorithu based on cnn-lstm feature fusion network. Biomedical signal processing and control, 72:103342, 2022.
[28] Salma Alhagry, Aly Aly Fahmy, and Reda A EI- El-Khoribi. Emotion recognition based eeg using Istm recurrent neural network. International Journal of Advanced Computer Science and Applications, 8(10), 2017.
[29] Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng. Yong-Jin Liu, and Hongan Wang. An efficient Istm network for emotion recognition from multichannel cog signals. IEEE Transactions on Affective Computing, 13(3): 1528-1105,,, 2020.
[30] Andac Demir, Toshiaki Koike-Akina, Ye Wang, Masaki Haruma, and Deniz. Erdogmus. Feg-gnn: Graph neural networks for classification of electroencephalogram (eeg) signals. In Annual International Conference the IEEE Engineering in Medicine & Biology Society (EMBC), pages 1061-1067. IEEE, 2021.
[31] Ziyu Jia, Youfang Lin, Jing Wang, Ronghao Zhou, Xiaojun Ning, Yuanlai He, and Yaoshuai Zhao. Graphslcepnet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification. In Ijcai, volume 2021, pages 1324-1330, 2020.
[32] Peixiang Zhong, Di Wang. and Chunyan Miao. Beg-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing, 13(3):1290-1300 2020.
[33] Zhengdao Li, Kai Hwang, Keqin Li, Jie Wu, and Tongkai Ji. Graph-generative neural network for eeg-based epileptic seizure detection via discovery of dynamic brain functional connectivity. Scientific reports, 12(1))18998, 2022.
[34] Dominik Klepl, Min Wu, and Fei He. Graph neural network-based eeg classification: A survey. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 32:493 503, 2024.
[35] Chenyu Liu, Xinliang Zhou, Yihao Wu, Ruizhi Yang, Zhongruo Wang, Liming Zhai, Ziyu Jia, and Yang Liu. Graph neural networks ceg-hased emotion recognition: a survey, arXiv preprint 2402::2402.01 138, 2024.
[36] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolutional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 31:710 719, 2022.
[37] Yonghao Song, Xueyu Jia, Lie Yang, and Longhan Xie. Transformer based learning for eeg decoding, arXiv preprint arXiv:2106.11 11170, 2021.
feature
[38] Wei Yan Peh, Yuanyuan Yao, and Justin Dauwels. Transformer convolutional neural networks for automated artifact detection in scalp eeg. In Annual International Conference of the IEFE Engineering Medicine & Biology Sociery (EMBC), pages 3599-3602. IEFE, 2022.
[39] Xia Chen, Xiangbin Teng, Han Chen, Yafeng Pan, and Philipp Geyer. Toward reliable signals decoding for electroencephallgram:: A benchmark study to eegnex. Biomedical Signal Processing and Control, 87:105475, 2024.
[40] Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. Advances in Neural Information Processing Systems, 36:78240-78260, 2023.
[41] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for learning generic representations with tremendous eeg data in bei. In The International Conference Learning Representations, 2024.
[42] Guangyu Wang, Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals. In Advances in Neural Information Processing Systems, volume 37, pages 39249-39280. 2024.
[43] Wenhui Cui, Woojae Jeong. Philipp Thölke, Takfarinas Medani, Karim Jerbi, Anand A Joshi, and Richard M Leahy. Neuro-gpi gpt: Towards a foundation model for eeg. In IEEE International Symposiun on Biomedical Imaging (ISBI). pages 1 1-5, IEEE, 2024.
[44] Zhizhang Yuan, Fanqi Shen, Meng Li, Yuguo Yu, Chenhao Tan, and Yang Yang. Brainwave: A brain signal foundation model for clinical applications. arXiv preprint 2rXii:2402.10255 2024.
[45] Junhong Lai, Jiyu Wei, Lin Yao, and Yueming Wang. A simple review of eeg foundation models: Datasets, advancements and future perspectives. arXiv preprint XXX::2504420069, 2025.
12

---

[46] Jiquan Wang. Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang. Shijian Li, Tao Li. and Gang Pan. CBramod: A criss-cross brain foundation model for EEG decoding. In The International Conference on Learning Representations, 2025.
[47] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[48] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold. Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, In International Conference on Learning Representations, 2021.
[49] Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self-supervised learning: Generative or contrastive. IEEE transactions on knowledge and data engineering, 35(1):857- 876, 2021.
[50] Demetres Kostas, Stephane Aroca-Ouellette, and Frank Rudzicz. Bendr: Using transformers and contrastive self-supervised learning task to learn from massive amounts eeg data. Frontiers in Human Neuroscience, 15:653659, 2021.
[51] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16009, 2022.
[52] Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. Cswin transformer: A general vision transformer backbone with cross-shaped windows, In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 12124-122 122442,,,, 2022.
[53] Eric R Kandel, James H Schwartz, Thomas M Jessell, Steven Siegelbaum, A James Hudspeth, Sarah Mack, al. Principles of neural science, volume 4. McGraw-hill New York, 2000.
[54] György Buzsáki. Rhythms of the Brain. Oxford university press, 2006.
[55] Nikola T Markov, Maria M Fresey-Ravasz, AR Ribeiro Gomes, Camille Lamy, Loic Magrou, Julien Vezoli, Pierre Misery, Arnaud Falchier, Rene Quilodran, Maric-Alice Gariel, et al. weighted and directed interareal connectivity matrix for macaque cerebral cortex. Cerebral cortex, 24(1):17-36, 2014.
[56] Ji-Hoon Jeong, Jcong-Hyun Cho, Young-Eun Eun Lee, Sco-Hyun Lee, Gi-Hwan Shin, Young-Scok Kweon, José del R Millán, Klaus-Robert Müller, and Seong-W Whan Lee. 2020 international brain-computer interface competition: A review. Frontiers in human neuroscience, 16:898300, 2022.
[57] Marcello Massimini, Reto Huber, Fahio Ferrarelli, Sean Hill, and Giulio Tononi. The sleep slow oscillation as a traveling wave. Journal of Neuroscience, 24(31 24(31)x6862-66 (6862)6767, 2004.
[58] David C Van Essen, Stephen M Smith, Deanna M Barch, Timothy EJ Behrens, Essa Yacoub, Kamil Ugurbil, Wu-Minn HCP Consortium, et al. The wu-minn human connectome project: an overview. Neuroimage, 80:62-79, 2013.
[59] Jörn Munzert, Britta Lorey, and Karen Zentgraf. Cognitive motor processes: the role of motor imagery in the study of motor representations. Brain research reviews, 60(2):306-326, 2009.
[60] Xiang Li, Yazhou Zhang, Prayag Tiwari, Dawei Song, Bin Hu, Meihong Yang, Zhigang Zhao, Neeraj Kumar, and Pekka Marttinen, Eeg based cmotion recognition: A tutorial and review, ACM Computing Surveys, 5-(4) 2022.
[61] Christian Szegedy. Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[62] Zhe Wang, Yongxiong Wang, Chuanfei Hu, Zhong Yin, and Yu Song. Transformers for ceg- based emotion recognition: A hicrarchical spatial information learning model. JEEE Sensors Journal, 22(5):4359443 2022.
[63] Jose Antonio Urigüen and Begoña Garcia Zapirain. Eeg artifact removal state-of of the-art and guidelines. Journal of neural engineering, 12(3):331011, 2015.
13

---

[64] Zhenda Xie, Zheng Zhang. Yue Cao. Yutong Lin, Jianmin Bao. Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9653-9663, 2022.
[65] Weibang Jiang. Yansen Wang. Bao liang Lu, and Dongsheng Li. MeuroLM: universal multi-task foun- dation model for bridging the gap between language and EEG signals. In The International Confereence on Learning Representations, 2025.
[66] Clemens Brunner, Robert Leeb, Gemot Müller-Purt Alois Schlögl, and Gert Pfurtscheller. Bei competi- tion 2008-graz data set Institute for knowledge discovery (laboratory of brain-computerr interfaces). Graz University of Technology, 16(1-6):1 2008.
[67] Gerwin Schalk, Dennis McFarland, Thilo Hinterberger, Niels Birbaumer, and Jonathan R Wolpaw. Bei2000: a general-purpose brain-computer interface (hci) system. IEFE Transactions on biomedical engineering, 5 (16)):1034410443 2004.
[68] Jun Ma, Banghua Yang, Wenzheng Qiu, Yunzhe Li, Shouwei Gao, and Xinxing Xia. A large ceg dataset for studying cross-session variability motor imagery brain-computer interface. Scientific Data, 9(1):331, 2022.
[69] Kaiyuan Zhang, Ziyi Ye, Qingyao Ai, Xiaohui Xie, and Yiqun Liu. Gnn4eeg: A benchmark and toolkit for clectreencephalograhyy classification with graph neural network. In Companion of the ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 612-617, 2024.
[70] Wei Liu, Jic-Lin Qiu, Wei-Long Zheng, and Bao-Liang Lu. Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition. IEEE Transactions on Cognitive and Developmental Systems, 2021.
[71] Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulation, 101(23)ee15-e222, 2000.
[72] Ali Hossam Shoeb. Application of machine learning to epileptic seizure onset detection and treatment. PhD thesis, Massachusetts Institute of Technology, 2009.
[73] Paolo Detti, Giampaolo Vatti, and Garazi Zabalo Manrique de Lara. Eeg synchronization analysis for seizure prediction: A study on data of noninvasive recordings. Processes, 8(7), 2020.
[74] Sirvan Khalighi, Teresa Sousa, José Moutinho Santos, and Urbano Nunes. Isruc-slepp: A comprehensive public dataset for sleep researchers. Computer methods and programs in biomedicine. 124:180-192, 2016.
[75] Diego Alvarez-Estevez and Roselyne Rijsman. Inter-database validation a deep learning approach for automatic sleep scoring. PlaS one, 16(8):e02561 11, 2021.
[76] Wei-Long Zheng and Bao-Liang Lu. A multimodal approach to estimating vigilance using ceg and forchcad cog. Journal of Neural Engineering, 14(2):026017, 2017.
[77] Igor Zyma. Sergii Tukaev, Ivan Seleznov, Ken Kiyono, Anton Popov, Mariia Chemykhh, and Oleksii Shpenkov. Electreencephalorrams during mental arithmetic task performance. Data, 4(1):4,, 2019.
[78] Wajid Mumtaz. MDD Patients and Healthy Controls EEG Data (New), |attps://Eigshare com/
art articles/ aset EEG Dat New/4244177 71 2016. Figshare.
[79] Iyad Obeid and Joseph Picone. The temple university hospital eeg data corpus. Frontiers in neuroxcience, 196, 2016.
[80] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626, 2017.
[81] Gert Pfurtscheller and A Aranibar, Event-related cortical desynehronization detected by power measure- ments of scalp eeg. Ellecmoencephalorgupyy and clinical neuropyssioloyy, 42(6))817-88) 1977.
[82] S Catrin Blank, Sophie K Scott, Kevin Murphy, Elizabeth Warburton, and Richard JS Wise. Speech production: Wernicke, broca and beyond. Brain, 125(8):1829-1888 2002.
14

---

[83] Wei Zhang, Muyun Jiang, Kok Ann Colin Teo, Raghavan Bhuvanakantham, LaiGuan Fong, Wei Khang Jeremy Sim, Zhiwei Guo, Chuan Huat Vince Foo, Rong Hui Jonathan Chua, Parasuraman Padmanabhan, et al. Revealing the spatiotemporal brain dynamics covert speech compared with overt speech: A simultancous cog-fmri study, Neurolmage, 13:190 2024.
[84] Mingyang Li and Wanzhong Chen. Fft-based deep feature learning method for eeg classification. Biomedical Signal Processing and Control, 66: 1024922222222 2021.
[85] Kaveh Samiee, Peter Kovacs, and Moncef Gabbouj. Epileptic seizure classification of ceg time-series using rational discrete short-time fourier transform. IEEF transactions on Biomedical Engineering, 62(2):541-552, 2014.
[86] Mingkan Shen, Peng Wen, Bo Song, and Yan Li. An eeg based real-time epilepsy seizure detection approach using discrete wavelet transform and machine learning methods. Biomedical Signal Processing and Control, 77:103820, 2022.
[87] Benjamin Blankertz. Ryota Tomioka, Steven Lemm, Motoaki Kawanabe, and Klaus-Robert Muller. Optimizing spatial filters for robust eeg single-trial analysis. IEEE Signal processing magazine, 25(1):41- 56, 2007.
[88] Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. Multiclass brain- computer interface classification by riemannian geometry. IEEE Transactions on Biomedical Engineering, 59(4):920-9)) 928, 2011.
[89] Muhammad Zeeshan Baig, Nauman Aslam, Hubert PH Shum, and Li Zhang. Differential evolution algorithm as a tool for optimal feature subset selection in motor imagery eeg. Expert Systems with Applications, 90:184-195, 2017.
[90] Inan Guler and Elif Derya Ubeyli. Multiclass support vector machines for eeg- eeg-signals classification. JEEE transactions on information technology in biomedicine, 11(2):117-126. 2007.
[91] Jasmin Kevric and Abdulhamit Subasi. Comparison of signal decomposition methods in classification of eeg signals for motor-imagery bei system. Biomedical Signal Processing and Control, 31:398-4000 406, 2017.
[92] Ryota Tomioka, Kazuyuki Aihara, and Robur-Rober Müller. Logistic regression for single trial eeg classification. Advances in neural information processing systems, 19, 2006.
[93] Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer. Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep lcarning with convolutional neural networks for ceg decoding and visualization. Human brain mapping, 38(1 005391-54200 2017.
[94] Zhengqing Miao, Meirong Zhao, Xin Zhang, and Dong Ming. Lmda-net: A lightweight multi-dimensionn attention network for general eeg-based brain-computer interfaces and interpretability. Neurolmage, page 120209, 2023.
[95] Ping Wang, Aimin Jiang, Xiaofeng Liu, Jing Shang, and Li Zhang. Lstm-based eeg classification in motor imagery tasks. IEEE transactions neural systems and rehabilitation engineering, 26(11):208622 2095. 2018.
[96] Xiaobing Du, Cuixia Ma, Guanhua Zhang, Jinyao Li, Yu-Kun Lai, Guozhen Zhao, Xiaoming Deng, Yong-Jin Liu, and Hongan Wang. An efficient Istm network for emotion recognition from multichannel cog signals. IEEE Transactions on Affective Computing, 13(3): 1544111,5 2020.
[97] Huy Phan, Kaare Mikkelsen, Oliver Y Chén, Philipp Koch, Alfred Mertins, and Maarten De Vos. Sleeptransformer: Automatic sleep staging with interpretahillity and uncertainty quantification. IEFF Transactions on Riomedical Engineering, 69(8)22456-246 24667 2022.
[98] Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies. volume (long and short papers), pages 4171-4186 2019.
[99] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam. Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
15

---

[100] Alec Radford, Jong Wook Kim. Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry. Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 3748-8763. PmLR, 2021.
[101] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 2023.
[102) Zhanwei Zhang. Shizhao Sun, Wenxiao Wang, Deng Cai, and Jiang Bian. FlexCAD: Unified and versatile controllable CAD generation with fine-tuned large language models. In The Thirteenth International Conference Learning Representations, 2025.
[103] Yuchen Zhou, Linkai Liu, and Chao Gou. Learning from observer gaze: Zero-shot attention prediction oriented by human-object interaction recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages -840028400, 2024.
[104] Xinshun Wang. Zhongbin Fang. Xia Li, Xiangtai Li, Chen Chen, and Mengyuan Liu. Skeleton-in- context: Unified skeleton sequence modeling with in-context learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2436-2446, 2024.
[105] Zipeng Guo, Yuchen Zhou, and Chao Gou. Drivinggen: Efficient safety-critical driving video generation with latent diffusion models. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2024.
[106] Yuchen Zhou, Jiayu Tang, Shuo Yang. Xiaoyan Xiao, Yuqin Dai, Wenhao Yang. Chao Gou, Xiaobo Xia, and Tat- Ten- Chua. Logic unseen: Revealing the logical blindspots of vision-language models. arXiv preprint arXiv:2508. 11117, 2025.
[107] Yuchen Zhou, Jiayu Tang. Xiaoyan Xiao, Yueyao Lin, Linkai Liu, Zipeng Guo, Hao Fei, Xiaobo Xia, and Chao Gou, Where, what, why: Towards explainable driver attention prediction. arXiv preprint 1.Xi:2550.23088 2025.
[108] Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. Brainbert: Self-supervised representation learning for intracranial recordings. arXiv preprint arXiv:2302. 14367, 2023.
[109] Daoze Zhang, Zhizhang Yuan, Yang Yang, Junru Chen, Jingjing Wang, and Yafeng Li, Brant: Foundation model for intracranial neural signal, Advances in Neural Information Processing Systems, 36:26304- 26321, 2023.
[110] Navid Mohammadi Foumani, Geoffrey Mackellar, Soheila Ghane, Saad Irtza, Nam Nguyen, and Mahsa Salchi. Eeg2rep: enhancing self-supervised ceg representation through informative masked inputs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5544-5555, 2024.
[111] Herbert II Jasper. Ten-twenty electrode system of the international federation. Electroencephalogr Clin Neurophysiol, 10:371-375, 1958.
[112] Gian Emilio Chatrian, Ettore Lettich, and Paula I. Nelson, Ten percent electrode system for topographic studies of spontaneous and evoked eeg activities. American Journal of EEG technology. 25(2):83 92. 1985.
[113] American Electroencephallgrappicc Society. Guideline thirteen: guidelines for standard electrode position nomenclature. Clin Neurophysiol, 11(1):11-113, 1994.
[114] Masako Okamoto, Haruka Dan, Kuniko Sakamoto, Kazuhiro Takeo, Koji Shimizu, Satoru Kohno, Ichiro Oda, Seichhro Isobe, Tateo Suzuki. Kaoru Kohyama, et al. Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping. Neuroimage, (1))999111,, 2004.
[115] J Craig Henry. Electroencephalogrpphyy: basic principles, clinical applications, and related fields. Neurol- ogy, 67(11))2092-2000 2092, 2006.
[116] C lber, S Ascoli-Isracl, C Chesson, and S Quan. The american academy of sleep medicine manual for the scoring of sleep and associated events. Westchester, IL: American Academy of Sleep Medicine, 2007.
[117] Huy Phan and Kaare Mikkelsen. Automatic sleep staging of eeg signals: recent development, challenges. and future directions. Physiological Measurement, 43(4)04TTR1, 2022.
16

---

[118] Weibang Jiang, Yansen Wang, Bao liang Lu, and Dongsheng Li. NeuroLM: A universal multi-task foun- dation model for bridging the gap between language and EEG signals. In The International Conference on Learning Representations, 2025.
[119] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.088. 003611, 2020.
[120] Laurens Van der Maaten and Geoffrey Hinton, Visualizing data using t-sne. Journal of machine learning research, 9111), 2008.
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction summarize the core contributions and scope accurately, with claims supported by method and results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of work are discussed in the supplemental material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

The paper does not include any theoretical results that require formal assump- tions or proofs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide sufficient details of the model architecture, training protocols, data preprocessing, and evaluation settings to ensure reproducibility of our main results. Further implementation details are included in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer:[Yes
Justification: We use publicly available datasets. We are currently cleaning the code and will release as soon as possible.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
pub public//a guides /ooeeSuommmm
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ wwwppc/cwbuci...ee.eep.pp.p..ppp
Code Suwmissionnsnn...........s..ii.e onP ciooooicyiyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We report all necessary training and test details, with full settings described in supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report mean and standard deviation for all key experiments to reflect statistical variability, Due space constraints, standard deviation values for Tab, included in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide details on GPU hardware, training time, and memory usage in Sec.3.1, with additional analysis in the supplemental material, which together offer sufficient information to estimate the compute required to reproduce our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://:ppseeeeeerrrrp...pppppap
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work advances generalized EEG decoding and has potential positive impacts in areas such as medical diagnosis, brain-computer interfaces, and attention moni- toring, A more detailed discussion is provided in the supplemental material.
21

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external datasets and codebases used in this work are properly credited, with licenses and usage terms respected. Citations and links are provided in the main paper and supplemental material.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
22

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papeess thcode com / dat set. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce model architecture and plan to release the code with documentation upon publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any crowdsourcing or human subject research conducted by the authors.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve any new human subject research conducted by the authors and relies solely on publicly available, ethically approved EEG datasets.
Guidelines:
23

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not used as part of the core research methodology. Their use was limited to writing assistance and did not impact the methodology, scientific rigor, or originality of the work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy petttps:/neurige: http://cccccoooeeeeeeeee////////
LLM) for what should should not be described.
24

---

A Related Work
Eleetroencepphlogrrahyy (EEG) provides a direct, non-invasive window into human brain activity, and plays a pivotal role in cognitive science, medical diagnostics, and brain-computer interface (BCI) systems [T.3]. By placing electrodes on the scalp, EEG captures and amplifies electrical signals produced by coordinated neuronal activity. Its high temporal fidelity, affordability, and portabilityy make EEG especially attractive for large-scale deployment in both clinical and everyday settings (7-9). As BCI applications expand across emotion recognition [15] [6]. seizure detection [19] (20], sleep staging [I7 [8), and various neurological disorders 211 there is a growing need for scalable, general-purpose EEG models that robustly decode neural signals across diverse tasks and recording conditions.
A.1 Task-Specific EEG Decoding Models
Conventional EEG decoding methods typically follow a two-stage pipeline: handcrafted feature extraction followed by shallow classification. A wide range of handcrafted features have been utilized, including time-domain statistics, frequency-domait power spectra (e.g.. via Fast Fourier Trans- form (44). fineufrry features (e.g.. STFT [85]. DWT (6)). and spatial filters such as Common Spatial Pattern [87 or Riemannian geometry baasd methods 88 These features are typically fed into classical classifiers including linear discriminant analysis [89]. support vector machines [90), k-nearest neighbors [91], or logistic regression
[92]. Despite their simplicity, these methods often struggle with generalization due to strong reliance on task-specific priors. limited capacity to model nonlinear dynamics, and poor robustness to inter-subject and inter-session variability.
With the rise of deep learning, end-to-end models have increasingly replaced handcrafted pipelines by directly learning representations from raw EEG signals. CNN- architectures (4 93
94
have
proven effective at capturing local spatiotemporal patterns, while RNNs, particularly LSTMs 28
95 96,, are commonly adopted to model temporal dependencies. Transformers 33 have recently gained popularity due to their superior ability to capture long-range interactions and global context. To combine local and global modeling, hybrid models such as CNN-LSTM [2 and CNN-Transformer [38] 336 have been explored. In parallel, GNN-based methods [31
35 aim to
incorporate topological priors by modeling spatial relationships among electrodes. While these deep models have achieved promising results in tasks such motor imagery, emotion recognition, and sleep staging, they are typically tightly coupled to specific datasets and task formats. Adapting them to new tasks often requires retraining and architectural redesign. These limitations have motivated shift toward EEG foundation models, which aim to learn robust and generalizable representations from unified architectures and large-scale self-supervised pretraining. Such models hold the potential to enhance generalization and enable effective transfer across tasks and domains.
A.2 Brain Foundation Model
Foundation models have transformed fields such as natural language processing (NLP) and computer vision (CV), enabling strong generalization across tasks through unified architectures and large-scale pretraining. Models like BERT 98 GPT 99) CLIP [00], MAE [51], LLaVA [TOT), and Sim- MIM [64] demonstrate that unified architectures trained on large-scale data can be efficiently adapted wide range of downstream applications [02 [07]. Inspired by these successes, researchers have begun to explore foundation models for neural signal processing, particularly EEG.
number of EEG foundation models have emerged with the goal of learning generalized represen- tations from large-scale data using unified model architectures. Early efforts such as BENDR (50),
BrainBERT [08]. Brant [109. EEG2Rep [100] and BIOT [40] adopt masked modeling or con- trastive learning objectives to encode general-pune purpose neural features. LaBraM [4]] further enhances cross-dataset and multi- palltiiprrdigmm generalization through structured tokenization and vector-quantized neural codes. More recently, EEGPT 42 introduces spatiotemporal alignment, while CBraMod [46] incorporates parallel spatial-temporal attention and dynamic positional encoding to better capture signal structure. These models represent a significant step toward generalizable EEG decoding, and most adopt a pipeline transplanted from NLP and CV, which EEG signals are divided into fixed-scale tokens and processed via dense attention across all tokens. However, this scale-agnostic and fully dense modeling paradigm fails to align with the intrinsic cross-scale spatiotemporal nature of EEG signals, an essential property for accurately capturing task-specific neural dynamics.
25

---

To address these limitations, we argue that EEG foundation models must move beyond dense modeling and embrace cross-scale, structure-aware architectures that respect the inherent regional organization and temporal dynamics of brain activity. In this paper, we propose CSBrain, Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain achieves superior generalization across diverse EEG tasks with varying spatiotemporal demands.
B
Additional Pretraining Details
B.1 Hyperparameter
We summarize the detailed hyperparameter settings of CSBrain in Table 3 configuration, model architecture, optimization settings, and training strategy.
including the input
Table 3: Pretraining hyperparameters for CSBrain.
Component
Setting
Input Channels
19
Time Points per Sample
6000
Mask Ratio
0.5
Mask Token
Full-zero vector
Kemel Sizes (CST)
(1, 3, 5)
Number of Layers
12
Hidden Dimension
200
Atlention Heads
8
Feed-forward Dimension
800
Dropout
0.1
Weight Initialization
Kaiming Normalization
Optimizer
AdamW
Learning Rate
5e-4
Adam /
(0.9, 0.999)
Adam
1e-8
Weight Decay
5e-2
Gradient Clipping
1.0
Training Epochs
40
Batch Size
128
Learning Rate Scheduler
ConialAAnn
Cosine Cycle Length
epochs
Minimum Learning Rate
le-5
B.2 Training Loss Curve
Figure shows the training loss curve of CSBrain during pretraining over 40 epochs. The loss decreases rapidly in the initial phase and gradually stabilizes, demonstrating effective convergence under the masked reconstruction objective.
C Additional Downstream Experimental Details
C.1 Hyperparameter Settings
For downstream finetuning, we load the pre-trained weights of CSBrain and replace the reconstruction head with a task-specific head composed of multilayer perceptron. The learned EEG representations flattened and fed into this head to perform downstream classification finetune the entire model on the target dataset. Table lists the default hyperparameter settings used in downstream finetuning. We adopt binary cross-entropy (BCE) loss for binary classification, cross-entropy loss for multi-class classification, and mean squared error (MSE) for regression.
regression. We then
26

---

0.005
0.003
0.001
10
15
20
25
Epoch
Figure 7: Training loss curve during CSBrain pretraining,
Table 4: Default hyperparameter settings for downstream finetuning.
Component
Setting
Training Epochs
50
Batch Size
64
Optimizer
Adam W
Learning Rate
le-4
Adam
(0.9, 0.999)
Adam
1e-8
Weight Decay
1c-2
Dropout
0.1
Learning Rate Scheduler
CosineAmeallngL..R
Cosine Cycle Epechs
50
Minimum Learning Rate
1e-6
Label Smoothing (for multi-class)
0.1
C.2 Baselines
Task-specific Models:
EEGNet (24) is lightweight CNN that employs depthwise and separable convolutions for decoding EEG signals from various BCI paradigms.
EEGConformer 366 is hybrid architecture that combines CNN and transformer for EEG decoding. Specifically, the CNN module captures local spatiotemporal features, while the transformer models long-range dependencies for global temporal representation.
SPaRCNet 225 is ID CNN with dense residual connections for BEG decoding, incorpo- rating adaptive depth and channel selection mechanisms.
ContraWR [26] converts EEG signals into classified using a ResNet-style 2D CNN.
spectrograms, which are then
CNN-Transformer 38 is CNN enhanced with transformer layers and trained using belief matching (BM) loss to detect channel-wise EEG artifacts.
integrates parallel CNN and LSTM branches, where the CNN extracts spatial features and the LSTM captures temporal dynamics. Features from the different branches are fused through a fully connected layer to enhance EEG classification.
FFCL 27
ST-Transformer 37 is transformer- based architecture that applies attention mecha- nisms along both the feature-channel and segmented time dimensions to capture global spatiotemporal dependencies from EEG signals.
Foundation Models:
27

---

is a linear transformer designed for universal biosignal representation learning through a hybrid supervised uttrsssrddduuuueiisseeeeee pre-training strategy. Notably. the pre-trained BIOT accepts maximum of 18 EEG channels as input. Therefore, for signals with more than 18 channels, a convolution is applied project them into an 18-channel format.
BIOT 40
LaBraM [4
is EEG foundation model that learns generic representations by predicting masked neural tokens using a full-attention Transformer.
an EEG foundation model designed address the heterogeneous spa- tiotemporal structure of EEG signals through criss-cross Transformer backbone.
CBraMod 46
C.3 Metrics
evaluate performance using the metrics follows: Balanced Accuracy addresses class imbalance by averaging the recall across all classes, offering a more reliable evaluation metric than standard accuracy in imbalanced datasets:
Consistency with prior work 40
41
46
1 - B-ACC = = 3m1 TP++
TP.
(8)
where C' is the number of classes, TP; and FN, denote true positives and false negatives for class i.
Cohen's Kappa measures the agreement between predicted and true labels agreement while correct- ing for chance, offering insight into classifier performance beyond random predictions:
Po
Pe.
Pe
where Po is the observed agreement and Pe the expected agreement.
Weighted Fi-score the harmonic mean of precision and recall for each class, weighted by the number of true instances (i.e., class support). It provides performance measure that accounts for label imbalance in multi-class classification:
2 Precission Recall;
F1-W
wi Precision, + Recall,
(10)
where is the weight of class based on its support. The precision is defined as:
TP:
TP:
Precision; = TP: + FP,
Recall,
(11)
TP: + FN,
where FP; denotes false positives for class
AUC-PR evaluates the trade-off between precision and recall across various decision thresholds. It is computed as the area under the curve plotted with precision on the y-axis and recall on the x-axis:
AUC-PR [
Precision(r)) dr,
Recall.
(12)
A higher AUC-PR indicates better performance in correctly identifying positive instances with fewer false positives. In practice, AUC-PR is typically estimated by numerical integration of discrete precision-recall points.
AUROC measures the model's ability to distinguish between positive and negative classes by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) at various thresholds. These are defined as:
TP
FP
TPR TP + FN where T'N denotes true negatives. The AUROC is the area under this curve:
FPR
(13)
FP + TN
AUROC /.
TPR(F) df, = FPR.
(14)
28

---

The AUROC is the area under the ROC curve, typically computed using numerical integration of discrete (FPR, TPR) points. An AUROC of 0.5 indicates random guessing, while 1.0 indicates perfect discrimination.
Pearson Correlation measures the linear dependence between predicted and true values, ranging from - (perfect negative correlation) to +1 (perfect positive correlation):
r Σ (v: (v: 1)
(15)
EN (y: 8)2 E (9: 7)
where is the i-th ground truth value, Di is the corresponding predicted value, y and ÿ denote the mean of true and predicted values, respectively, and N the total number of samples.
R2 Score represents the proportion of variance in the dependent variable that explained by the model, where value of 1 indicates perfect predictive performance:
-;)²
R² =
(16)
Σ
(vi 7)
An R2 score less than 0 indicates that the model performs worse than simply predicting the mean of the target variable.
Root Mean Squared Error (RMSE) measures the standard deviation of prediction errors, quantify- ing the average magnitude of differences between predicted and actual values, where lower values indicate better model fit:
N Σ
1
RMSE
U
9.)3.
(17)
In this work, we report Balanced Accuracy, Cohen's Kappa, and Weighted F1 for multiclass clas- sification tasks; Balanced Accuracy, AUC-PR, and AUROC for binary classification; and Pearson correlation, R2 score, and RMSE for regression tasks.
D Additional Results on Downstream Tasks
To comprehensively evaluate the generalizability of our model, we conduct experiments on 11
representative BCI tasks across 16 publicly available EEG datasets. These tasks cover a wide range of
applications, including Motor Imagery Classification (BCIC-IV-2a (6). PhysioNet-MI [67, SHU-MI
(68)). Emotion Recognition (FACED 69), SEED-V V (70). Seizure Detection (CHB-M CHB-MIT 71 72).
Siena [7T 73)), Sleep Staging (ISRUC 74), HMC (75)), Imagined Speech Classification (BCIC2020-
3 (56]), Vigilance Estimation (SEED-V VIG 176). Mental Stress Detection (MentalArithheet) (7).
Mental Disorder Diagnosis (Mumtaz2016 (78), Event Type Classificution (TUEV 779 ) Abnormal
Detection (TUAB (79). and Slowing Event Classification (TUSL (79). For all tasks, we strictly follow the established train/validation//esst splits to ensure fair and reproducible evaluation. This region assignment follows a standard practice in the literature [1111115], where the 10-20 system's 's electrode labels are used as proxies for anatomical brain regions, specifically including the Frontal, Central, Parietal, Temporal, and Occipital areas. When existing baselines are unavailable incompatible due to inconsistent settings, we re-implement and re-train them under unified protocol for consistent comparison. Note that ensure stable training, apply minor hyperparameter adjustments (e.g.. learning rate) for a few datasets when necessary. the following sections, We detail the dataset setups for each task and analyze the corresponding results.
D.1 Motor Imagery Classification
Motor Imagery (MI) classification aims to decode user's 's intention by recognizing EEG patterns associated with imagined movements of specific body parts. It serves as a key component in BCT systems, enabling direct neural control in assistive technologies. We evaluate our model on three widely used MI datasets, BCIC-IV-2a [66), PhysioNet-MI 67], and SHU-MI 68, covering both multiclass and binary classification settings.
29

---

BCIC-IV-2a comprises EEG recordings from 9 subjects performing 4 motor imagery tasks: imagining movements of the left hand, right hand, both feet and tongue. Recordings were collected over two sessions on separate days using 22 electrodes at sampling rate of 250 Hz. Each session contains 288 EEG trials (72 per class). Following [46], we extract the [2. 6]-second interval from each trial, yielding a total of 5,088 samples. The signals are subsequently resampled at 200 Hz. We adopt a strict cubject-independen split: subject 1-5 for training. 6-7 for validation, and 8-9 for testing. We train our model with learning rate of 0.0001 and a weight decay of 0.05. Table Salle5Ssmmm summarizes the performance of our model and various baselines on BCIC-IV-2a. CSBrain achieves the highest performance across all three metrics: 0.5657 balanced accuracy, 0.4209 Cohen's 's kappa, and 0.5637 weighted F1. CSBrain outperforms the second-best CBraMod by +5.2% in balanced accuracy and +6.9% in kappa. These results highlight the effectiveness of our cross scale spatiotemporal modeling design in capturing nuanced motor imagery dynamics.
Table 5: Results on Motor Imagery Classification (BCIC-IV-2a, 4-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 24
0.4482 0,0094
0.2693 + 0.0121
0.4226 = 0.0108
spse-speeiife
EFGConformer
66
0.4696 # 0.0106
0.2924 = 0.0141
0.4533 0.0128
Models
SPaRCNet 25
0.4635 = 0.0117
0.2847 = 0.0147
0.4432 = 0.0126
ContraWR (26
0.4678 # 0.0125
0.2905 # 0,0160
0.4413
CNN- NNN-Transformer
38
0.4600 = 0.0108
0,2800 # 0.0148
0.4460
FFCL 27
0.4470 = 0.0143
0.2627 # 0.0176
0.4238
ST-Transformer
0.4575 = 0.0145
0.2733 = 0.0198
0.4471
Foundation
BIOT 440
0,4748 + 0.0093
0.2997 + 0.0139
0.4607 + 0.0125
Models
LaBraM-Base
0.4869 = 0.0085
0.3159 = 0.0154
0.4758 = 0.0103
CBraMod 46
0.5138 # 0.0066
0,3518 # 0.0094
0.4984 = 0.0085
CSBrain (Ours)
0.5657 # 0.0071
0.4209 + 0.0093
0.5637 = 0.0087
SHU-MI is a binary motor imagery dataset focused on classifying left-hand versus right-hand motor imagery. It contains EEG recordings from 25 subjects using 32 channels at sampling rate of 250 Hz. SHU-MI dataset contains [4,8]-second segments extracted from each trial, corresponding to the motor imagery execution period. All signals are resampled to 200 Hz and yield total of 11,988 samples. We adopt a subject-independent split with subjects 1-15 for training. 16-20 for validation, and 21-25 for testing. Model training is performed using learning rate of 0.0001 and weight decay of 0.05. presents the results on SHU-MI. CSBrain again achieves the best performance across all metrics: 0.6417 balanced accuracy, 0.7230 AUC-PR, and 0.7200 AUROC. Compared to CBraMod, which is the strongest baseline, CSBrain improves by +0.47% in balanced accuracy and +2.12% in AUROC. The consistent gains highlight the general applicability of our model across both fine-grained and boarse-grained MI tasks.
Table 6: Results on Motor Imagery Classification (SHU-MI, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNet 24
0,5889 0.0177
0.6311 # 0.0142
0.6283 # 0.0152
Task specific Models
EEGConformer
36
0,5900 = 0.0107
0.6370 0.0093
0.6351 0.0101
SPaRCNet (25)
0,5978 = 0.0097
0.6510 0.0062
0.6431
ContraWR (66
0.5873 # 0.0128
0.6315 0.0105
0.6273
CNN-Transformer
38
0.5975 # 0.0169
0.6412 0.0076
0.634300.000
FFCI. 27
0.5692 # 0.0252
0.5943 0.0171
0.6326 0.0082
ST-Transformer
37
0,5992 = 0.0206
0.6394 0.0122
0.6431 + 0.0111
Foundation Models
BIOT 40
0.6179 # 0.0183
0.6770 = 0.0119
0.6609 = 0.0127
LaBraM-BB Base
0.6166 = 0.0192
0.6761 = 0.0083
0.6604 = 0.0091
CBraMod 46
0.6370 # 0.0151
0.7139 + 0,0088
0.6988 # 0.0068
CSBrain (Ours)
0.6417 # 0.0037
0.7230 + 0.0079
0.7200 + 0.0058
Pey is a large-scale motor imagery dataset comprising EEG recordings from 109 subjects, each performing four MI tasks: imagining the movement of the left fist, right fist, both fists, and both feet. The signals were collected using 64 electrodes at a sampling rate of 160 Hz. We segment the EEG into 4-sccond windows, resample to 200 Hz, and obtain total of 9,837 samples. We adopt
30

---

subject-independen split with subjects 1-70 for training. 71-89 for validation, and 90-109 for testing. For this dataset, we use learning rate of 0.0001 and a weight decay of 0.05 for model training. As shown in Table 7 our CSBrain model achieves top performance on all three metrics: 0.6304 balaneed accuracy, 0.5071 Cohen's i kappa, and 0.6308 weighted F1. Compared to the best-performing baseline CBraMod, CSBrain yields consistent improvements (+1.3% in balanced accuracy and +1.7% in kappa). Furthermore, CSBrain surpasses strong task-specific models like EEGConformer and CNN-Transformer by large margins, further validating the advantage of foundation model pretraining and our cross-scale architecture.
Table 7: Results Motor Imagery Classification (PhysioNet-Mi, 4-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 24
0,5814 = 0.0125
0.4468 # 0.0199
0.57% = 0.0115
Yask- specificc Models
EEGConformer
36
0.6049 0.0104
0,4736 0.0171
0.6062 = 0.0095
SPaRCNet 25
0,5932 = 0.0152
0.4564 # 0.0234
0.5937 = 0.0147
ContraWR [26]
0,5892 : 0.0133
0,4527 = 0.0248
0.5918
CNN- Transssanorrrrr
38
0,6053 = 0.0118
0,4725 # 0.0223
0.6041 = 0.0105
FFCL [27]
0.5726 0,0092
0.4323 = 0.0182
0.5701 = 0.0079
ST-Transformer
37
0.6035 0.0081
0.4712 = 0.0199
0.6053 = 0.0075
Foundation Models
BIOT 40
0,6153 0,0154
0.4875 + 0.0272
0.6158 = 0.0197
LaBraM- Base
0.6173 = 0.0122
0.4912 + 0.0192
0.6177 = 0.0141
CBraMod 46
0.6174 + 0.0036
0.4898 + 0.0048
0.6179 = 0.0035
CSBrain (Ours)
0.6304 = 0.0090
0.5071 = 0.0120
0.6308 = 0.0095
D.2 Emotion Recognition
Emotion recognition from EEG signals aims to decode subject's affective state by analyzing brain activity patterns associated with different emotional experiences. EEG offers a direct. real-time and non-invasive approach to capturing internal emotional responses. However, this task remains highly challenging due to the subjective nature of emotions, significant inter-subject variability, and the inherently noisy and non-stationary characteristics of EEG signals. To evaluate the effectiveness and generalizability of our model in decoding emotional states, we conduct experiments on two widely-used used EEG emotion datasets: SEED-V [70), which involves five emotion categories and high-resolution short-segment EEG recordings across multiple sessions, and FACED (99, which focuses fine-grained nine-class classification with long EEG segments.
SEED-V benchmark dataset for EEG-based emotion recognition. includes five categories (happy, sad, neutral, disgust, fear). with EEG collected from 16 subjects over three sessions each, using 62 channels a high sampling rate of 1000 Hz. The data segmented into 117,744 1-second samples, resampled to 200 Hz. We divide each 15-trial session into three equal parts (5:5:5) for training, validation, and testing. The model trained using the same settings as FACED (learning rate 0,0001, weight decay 0.05). Table & shows the performance on SEED-V. CSBrain again surpasses all baselines, achieving 0.4091 balanced accuracy, 0.2569 Cohen's 's kappa, and 0.4101 weighted F1. Compared to CBraMod, which previously achieved the best results, CSBrain improves the kappa score by +2.1 1 %, indicating better discriminative capacity across short-time emotional states.
FACED is a large-scale EEG dataset for fine-grained emotion recognition, comprising 32-channel EEG signals recorded at 250 Hz from 123 subjects. It covers nine emotion categories: amusement, inspiration, joy, tenderness, anger, fear, disgust, sadness, and neutral. The raw signals are segmented into 10-second samples and resampled to 200 Hz, resulting in 10.332 instances. We split the data by subject: subjects 1-80 for training, 81-100 for validation, and 101-123 for testing. A learning rate of 0.0005 and weight decay of 0.05 are used for model training. As shown in Table 9 CSBrain achieves state-of-the art performance on this challenging 9-class task, with 0.5752 balanced accuracy, 0.5204 Cohen's kappa, and 0.5796 weighted F1. Compared to the best-performing baseline CBraMod, CSBrain yields consistent improvements across all metrics (+2.4% in balanced accuracy and +1.6% in kappa). These consistent gains across both long and short windowed EEG emotion datasets highlight the robustness and adaptability of CSBrain's 's cross-scale representation.
31

---

Table 8: Results on Emotion Recognition (SEED-V, 5-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
ERGNet
0.2961 = 0.0102
0.1006 = 0.0143
0.2749 = 0.0098
Task- Task-specify specifeecife Models
EFGGonformer
66
0.3537 # 0.0112
0.1772 # 0.0174
0.3487 # 0.0136
SPaRCNet 25
0.2949 = 0.0078
0.1121 = 0.0139
0.2979 0.0083
ContraWR [26]
0.3546 = 0,0105
0.1905 = 0.0188
0.3544 0.0121
NNN-Transformer
0.3678 = 0.0078
0.2072 = 0.0183
0.3642 = 0.0088
FFCL [27
0.3641 = 0.0092
0.2078 # 0.0201
=
ST-Transformer
77
0,3052 # 0.0072
0.1083 # 0.0121
0.2833 0.0105
Foundation Models
BIOT 40
0.3837 = 0.0187
0.2261 # 0.0262
0.3856 0.0203
LaBraM-Base
41
0.3976 = 0.0138
0.2386 # 0.0209
0.3974 0.0111
CBraMed 46
0.4091 0.0097
0,2569 + 0.0143
0.4101 = 0.0108
CSBrain (Ours)
0.4197 0.0033
0.2785 + 0.0034
0,4280 = 0.0023
Table 9: Results on Emotion Recognition (FACED, 9-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 2
0,4090 + 0.0122
0,3342 # 0.0251
0,4124 = 0.0141
Task-specific
EEGConforner 36
0.4559 = 0.0125
0.3858 # 0.0186
04014
Models
SPaRCNet 25
0.3978 # 0.0289
0.4729
ContraWR 26
0.4887 = 0.0078
0.4231 = 0.0151
0.4884
CNN-Transformer
38
0.4697 = 0.0132
0.4017 # 0,0166
0.0122
FFCL 227
0.4673 + 0.0158
0.3987 = 0.0338
0.469900.014
ST-Transformer
37
0.4810 0.0079
0.4137 = 0.0133
0.4795 0.0096
Foundation
BIOT 40
0.5118 # 0.0118
0.4476 # 0.0254
0.5136 = 0,0112
Models
LaBraM-Baae
0.5273 0.0107
0.4698 = 0.0188
0.5288 = 0.0102
CBraMod
0.5509 0.0089
0,5041 - 0.0122
0.5618 + 0.0093
CSBrain (Ours)
0.5752 + 0.0042
0.5204 + 0.0036
0.5796 = 0.0031
D.3 Seizure Detection
Seizure detection aims to identify pathological brain activity associated with epileptic seizures by analyzing abnormal EEG patterns. Accurate and timely seizure detection critical for clinical diagnosis, long-tern terg monitoring, and closed-loop therapeutic systems. We evaluate our model on two benchmark datasets: CHB-MIT 71 72 and Siena 71 73
CHB-MIT is widely-used clinical EEG benchmark collected from 23 pediatric patients with intractable epilepsy at the Children's Hospital Boston. EEG recordings span multiple days per subject, captured under stizuue-monitoring protocols following withdrawal of anti-scizure medication. All signals are recorded using the international 10-20 electrode placement system at 256 Hz, and labeled into binary classes: seizure and non-seizure. Following prior work (40, 46), we retain the 16 commonly used bipolar montage channels and resample all EEG signals to 200 Hz. The data then segmented into 10-second windows, resulting in 326,993 samples. To ensure fair and consistent evaluation, we adopt a inneeeeeindeen split: subjects 1-19 for training. 20-21 for validation. and 22-23 for testing. The model is trained with learning rate of 0.0001 and weight decay of 0.05. Table shoss the performance on CHB-MIT. Our CSBrain achieves the best performance in terms of AUC-PR and AUROC, outperforming all baselines by large margin. Specifically, it obtains an AUC-PR of 0.5164, exceeding the sccond-best CBraMod (0.3689) by + +14.75%, and an AUROC of 0.8915, higher than CBraMod's 's 0,8892 (+0.23%). Although its balanced accuracy (0.7262) is marginally lower than CBraMod (0.7398), the substantial gain in AUC-PR, an especially informative metric under class imbalance, demonstrates the robustness of CSBrain for seizure detection.
Siena is a clinical EEG database collected from 14 adult patients the Unit of Neurology and Neurophysiology. University of Siena. EEG recordings were acquired via Video-EEG monitoring at 512 Hz using the international 10-20 electrode placement system. Data annotation follows the International League Against Epilepsy (ILAE) criteria, with seizure and non-seizure labels rigorously verified by expert clinicians based on combined electrophysiologiaal and clinical review. We retained the 29 EEG channels that were consistently available across all 14 subjects. All signals
32

---

Table 10: Results on Seizure Detection (CHB-MIT, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNet 24
0,5658 = 0.0106
0.1914 0.0182
0.8048 0.0136
Task Task- specific Models
EEGConformer 36
0.5976 = 0.0141
0.2209 # 0.0215
0.8226
SPaRCNet [25
0,5876 = 0.0191
0.1247 0.0119
08143
ContraWR [26
0.6344 # 0.0002
0.8097 0.0114
CNN-Transfrrmer
38
0.6389 = 0.0067
0.2479 = 0.0227
0.8662 = 0.0082
FFCL 227
0.6262 = 0.0104
0.2049 = 0.0346
0.8271 = 0.0051
ST-Transformer
37
0.5915 = 0.0195
0,1422 # 0.0094
0.8237 = 0.0491
Foundation Models
BIOT 40
0.7068 = 0.0457
0.3277 # 0.0460
0.8761 = 0.0284
LaBraM-Ban Basc 41
0.7075 # 0.0358
0.3287 2 0,0402
0.8679 = 0.0199
CBraMod [4
0.7398 0.0284
0.3689 + 0.0382
0.8892 = 0.0154
CSBrain (Ours)
0.7262 0.0115
0.5164 = 0.0449
0.8915 = 0.0321
are resampled to 200 Hz. We segment the data into 10-second samples, yielding total of 51,307 samples. For fair evaluation, we employ a subject-independent split: all data from the last two subjects (PN16 and PN17) are held out as the test set, while the remaining 12 subjects' data are split into train and validation sets at an 8:2 ratio, maintaining this proportion for each subject and across all label categories. Table I1 summarizes the results the Siena dataset. CSBrain achieves the best performance across all evaluation metrics, with a balanced accuracy of 0.7662, an AUC-PR of 0.4871, and an AUROC of 0.9076. Compared to the strongest baseline CBraMod, CSBrain improves AUC-PR by 7.6 points (0.4871 vs. 0.4107) and AUROC by 0.38 points (0.9076 vs. 0.9038), while also achieving higher balanced accuracy. These results highlight the robustness of CSBrain in detecting seizures, validating the effectiveness of cross-scale spatiotemporal modeling learning discriminative neural patterns across diverse seizure morphologies.
Table 11: Results Seizure Detection (Siena, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNet 24
0.7487 # 0.0521
0,3753 # 0.0867
0.8687 = 0.0527
Task-specifie
EEGConformer 36
0.7556 0.0210
0.2091 # 0.0786
0.8159 # 0.0261
Models
SPaRCNet 25
0.6572 0.0381
0.3164 0.06599
0.7334 0.0857
ContraWR 26
0.6546 = 0.0311
0.3711 = 0.0405
0.7819 = 0.0596
CNN-Transformer
38
0.6982 # 0.0560
0,3835 # 0.0709
0.8719 = 0.0403
FFCL 227
0,6616 # 0.0391
0.3938 # 0.0903
0.8154 = 0.1
ST-Transformer
37
0.7527 0.0381
0.3636 + 0.0252
0.8884 = 0.0091
Foundation
BIOT 40
0.7352 # 0.0669
0.3809 # 0.0892
0.9029 = 0.0304
Models
LaBraM- Base
0.7082 0.0329
0.3122 # 0.0976
0.8814 = 0.0328
CBraMod 46
0.7317 # 0.0647
0.4107 + 0.0720
0.9038 + 0.0218
CSBrain (Ours)
0.7662 # 0.0471
0.4871 + 0.0343
0.9076 + 0.0119
D.4 Sleep Staging
Sleep staging refers to the task of classifying EEG signals into standard sleep stages (Wake, NI, N2, N3, REM), providing essential information for sleep quality assessment and clinical diagnosis of sleep disorders. Compared to conventional classification tasks, sleep staging involves strong temporal dependencies and stage transition patterns across time, making sequence modeling particularly important. We evaluate model on two benchmark datasets: ISRUC 74,, and HMC (75).
ISRUC contains overnight polysomnographic (PSG) recordings from 100 adult subjects. We use Sub-group 1, where EEG signals are recorded using six standard channels (F3-A2, C3-A2, O1-A2, F4-A1, C4-A1, O2-A1) at 200 Hz. All signals are segmented into 30-second samples and annotated by sleep experts according to the American Academy of Sleep Medicine (AASM) criteria [116], resulting in 89, 89,240 labeled samples across five classes. Following prior work [177, we treat sleep staging as gequence-t-sequenee samples, and models are trained to predict one sleep stage per sample. Following prior work [46) adopt subject-independee split for evaluation: subjects 1 1-80 are used for training, 81-90 for
task: EEG segments are grouped into sequences of 20 30-second
33

---

validation, and 91-100 for testing. For consistency across models, all sample encoders (including CSBrain and baselines) are followed by a one-layer Transformer as a shared sequence encoder. Table 12 reports the performance on ISRUC. CSBrain achieves the highest balanced accuracy of 0.7925 and strong overall performance across Cohen's kappa (0.7406) and weighted F1 (0.7990). Compared to CBraMod, CSBrain improves balanced accuracy by 0.6 points (0.7925 vs. 0.7865), while maintaining highly competitive scores in the other two metrics.
Table 12: Results on Sleep Staging (ISRUC, 5-class).
Method
Balanced Accuracy
Cohen's 's Kappa
Weighted F1
EEGNet 24
0.7154 # 0.0121
0.7040 = 0.0173
0.7513 0.0124
Task- specific Models
EEGConformer
0,7400 = 0.0133
0.7143 = 0.0162
0.7634 0.0151
SPaRCNet 25
0.7487 +: 0.0075
0.7097 = 0.0132
0.7624 0.0092
ContraWR (26]
0.7402 = 0.0126
0.7178 # 0.0156
0.7610 0.0137
CNN- Transsornorrerr
38
0.7363 0,0087
0.7129 = 0.0121
0.7719 0.0153
FFCI. 27
0.7277 = 0.0182
0.7016 0.0292
0.7614 0.0197
ST-Transformer
0.7381 = 0.0205
0.7013 0.0352
0.7681 = 0.0175
Foundation Models
BIOT 40
0,7527 = 0.0121
0.7291 + 0.0230
0.7790 + 0.0146
LaBraM- Base
0.7633 = 0.0102
0.7231 # 0.0182
0.7810 = 0,0133
CBraMod 44
0.7865 = 0.0110
0.7442 # 0.0152
0.8011 = 0.0099
CSBrain (Ours)
0.7925 # 0.0030
0,7406 # 0.0102
0.7990 = 0.0091
HMC contains overnight polysomnography (PSG) recordings of 151 adult subjects. EEG signals were recorded from four channels (F4-MI, C4-M1, O2-M1, and C3-M2) at 256 Hz. All signals were segmented into 30-second epochs and annotated by sleep experts according to the AASM guidelines. resulting in 137,243 labeled samples across five classes. All signals are resampled to 200 Hz. Following prior work [118]. we adopt subject-independent evaluation split: the first 100 subjects are used for training, the middle 25 for validation, and the remaining 26 for testing. Table [33 summarizes the performance of all methods on HMC. CSBrain achieves the highest balanced accuracy of 0.7345, along with top scores in Cohen's kappa (0.6818) and weighted F1 (0.7506), outperforning all baselines. Compared to the best-performing baseline LaBraM- CSBrain improves balanced accuracy by 0.68 points and weighted F1 by 0.52 points, while maintaining a comparable Cohen's kappa score.
Table 13: Results on Sleep Staging (HMC, 5-class).
Method
Balanced Accuracy
Cohen's 's Kappa
Weighted F1
EEGNet Z
0.6534 # 0.0122
0.5886 + 0.0201
0.6536 = 0.0168
Yask- Task-specifee specifie Models
E.GGCofformer
R6
0.7149 # 0.0086
0.6432 # 0.0055
0.7080 = 0.0039
SPaRCNet 25
0.4756 109
0.3147 # 0315
0.4108 = 0310
ContraWR [26]
0.4242 # 0.0541
0.2340 + 0.0554
0.2987 = 0.0288
ONN-Transformer
38
0.6573 # 0.0141
0.5961 = 0.0105
0.6896 0.0065
FFCL [27
0.4427 = 0.0702
0.2542 + 0.0654
0.2902 = 0.0485
ST-Transformer
37
0.2559 # 0.0141
0.0503 # 0.0183
0.1428 = 0.0122
Foundation Models
BIOT 40
0.6862 # 0.0041
0.6295 + 0.0113
0.7091 # 0.0147
LaBraM-Base
41
0.7277 # 0.0101
0.6813 0.0053
0.7454 0.0027
CBraMod
0.7269 = 0.0041
0.6685 + 0.0104
0.7395 # 0.0089
CSBrain (Ours)
0.7345 = 0.0047
0.6818 # 0.0046
0.7506 = 0.0042
D.5 Imagined Speech Classification
Imagined speech classification aims to decode phonological representations embedded in neural activ- ity. This task enables individuals with speech impairments resulting from conditions such as stroke, trauma, or amyotrophic lateral sclerosis (ALS) to communicate without overt articulation. Moreover, this approach enhances the understanding of the neural mechanisms underlying speech production and perception, facilitating advances in cognitive neuroscience and augmentative communication technologies. We evaluate our model on the BCIC2020-3 dataset (56). released as part of the 2020 International BCI Competition. During the data collection experiment, 15 subjects were instructed to
34

---

imagine five speech-related words or phrases ("hello," "help me," "stop, "thank you," and "yes"), and EEG signals were recorded from 64 channels a sampling rate of 256 Hz. BCIC2020-3 dataset is split into training, validation, and test sets as defined by the competition. Specifically, for each subject, 60 trials per class are provided for training, 10 for validation, and 10 for testing. Fach sample consists of a 3-second EEG recording. Therefore, we obtained 6000 64-channel 3-second EEG trials, which are resampled to 200 Hz. A learning rate of 0.0001 and a weight decay of 0.01 are used for our model training. Table [4 reports the performance on BCIC2020-3. CSBrain achieves the highest balanced accuracy (0.6004), along with strong overall performance in terms of Cohen's 's kappa (0.5006) and weighted F1 (0.6003). Compared to CBraMod, CSBrain improves the balanced accuracy by 6.3 points (0.6004 vs. 0,5373), while maintaining competitive performance on the other two metrics.
Table 14: Results on Imagined Speech Classification (BCIC2020-3, 5-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 2
0.4413 = 0.0096
0.3016 # 0.0123
0,4413 = 0.0102
Task- specific Modely
EEGConformer
3D
0.4506 # 0.0133
0.3133 0.0183
0.4488 = 0.0154
SPaRCNet 25
0,4426 = 0.0156
0.3033 2 0.0233
0.4420 0.0108
ContraWR [26
0.4257 0.0162
0.3078 + 0,0218
0.4407 0.0182
ToN-Transfrrmer
38
0,4533 # 0,0092
0.3166 # 0.0118
0.4506
FFCI. 27
0.4678 = 0.0197
0.3301 # 0.0359
0.4689
ST-Transformer
37
0.4126 = 0.0122
0.2941 # 0.0159
0.4247 0.0138
Foundation
BIOT 40
0,4920 0,0086
0.3650 + 0.0176
0.4917 = 0.0079
Models
LaBraM- Base
0,5060 = 0.0155
0.3800 + 0.0242
0.5054 + 0.0205
CBraMod 46
0.5373 + 0.0108
0.4216 + 0.0163
0.5383 = 0.0096
CSBrain (Ours)
0.6004 # 0.0187
0.5006 # 0.0233
0.6003 = 0.0192
D.6 Vigilance Estimation
Vigilance estimation plays a critical role in monitoring cognitive states in real-world scenarios such as driving, aviation, and industrial operations. By measuring fatigue levels from EEG signals, this approach can enhance safety by reducing the risk of accidents in high-stakes environments. Advances in vigilance estimation may facilitate the development of more robust and reliable monitoring systems. We evaluate our model on the SEED-VIG dataset (76). SEED- SEED-VIG simulates a real-world driving environment to induce a gradual transition from a vigilance to a fatigued state. The dataset was collected while subjects operated the simulated driving system. Vigilance levels were labeled using the PERCLOS indicator obtained from SMI eye- tracking glasses. EEG signals were recorded from 21 subjects using 17 channels at 200 Hz. and segmented into 20,355 8-second samples. We adopt subject-independen evaluation split: subject 1 to 13 for training, subject 14 to 17 for validation and subject 18 to 21 for test. To stabilize training, we use learning rate of 0.0005 and weight decay of 0.05, slightly differing from the default settings. As shown in Table 15| CSBrain achieves the best overall performance, outperforming all baselines in both R² score (0. (023333) and RMSE (0.2774), while achieving a competitive Pearson's 's correlation coefficient (0.6314 14) close to the strongest baseline (LaBraM, 0.6347). Compared to the best-performing foundation model LaBraM, CSBrain improves R2 by a substantial margin (+5.6 points) and reduces RMSE by 0.97, highlighting the effectiveness of its ccoss-scale modeling strategy in capturing subtle fluctuations in cognitive state.
D.7 Mental Stress Detection
Mental stress detection holds significant potential for both clinical and real-world applications. It enables objective assessment of psychological stress levels, overcoming the limitations of subjective self-reports. Therefore, it is particularly valuable for workplace mental health monitoring and the diagnosis of stress-related disorders. We evaluate our model on the MentalArithmeticc dataset 77 [7). MentalArithmetic comprises EEG recordings from 36 subjects collected before and during a mental arithmetic task. EEG trials were recorded before the task were labeled as "no mental stress," " while those were recorded during the task were labeled as "mental stress." All EEG signals were recorded from 20 electrodes at a sampling rate of 500Hz. The signals were resampled to 200Hz and segmented into 1,707 samples of 5-second trails. We adopt a subject-indeppnddnt evaluation
35

---

Table 15: Results on Vigilance Estimation (SEED-VIG, regression).
Method
Pearson's Correlation
R Score
RMSE
EEGNet 24
0,5127 e 0,0357
0.1960 + 0.0427
0.2847
0.0076
Task- speeeffee Models
EEGConformer 33
0,5800 0.0174
0.2065 0.0230
0.2829 0.0041
SPaRCNet 25
0.5709 0.0362
0.2185 0.0601
0.2806 0.0108
ContraWR 26
0.5235 = 0.0335
0.0727 = 0.0540
0.3057 0.0090
CNN-Transformer
38
0.5829 = 0.0246
0.1796 # 0.0105
0.2877 # 0.0018
FFCL [27]
0,4923 = 0,0313
0.1740 + 0.0530
0.2885 0.0093
ST-Transformer
37
0.6020 0.0327
0.1138 + 0.1151
0.2983 0.0199
Foundation Models
BIOT 40
0.6114 0.0169
0.1232 0.0778
0.2971
0.0128
LaBraM-Base
0.6347 0.0135
0.1808 0.0958
0.2871 0.0166
CBraMod 46
0,5502 = 0.01
0.0737 = 0.0167
0.3057 0.0027
CSBrain (Ours)
0.6314 # 0.0356
0.2363 # 0.0519
0,2774 = 0.0094
split: subject to 28 for training, subject 29 to 32 for validation, and subject 33 to 36 for testing. Table 16 reports the performance accuracy (0.7558), along with strong overall performance in terms of AUC-PR (0.6696) and AUROC (0.8478). Compared to CBraMod, CSBrain improves the balanced accuracy by 3.0 points (0.7558 vs. 0.7256), while maintaining comparable performance on the other two metrics. The superior results of CSBrain can be attributed its cross-scale spatiotemporal modeling, which effectively captures the short-term neural fluctuations and longer-range contextual dependencies underlying stress-induced brain dynamics. These findings highlight the importance of structure-awan foundation models in cognitive state classification tasks.
Mental CSBrain achieves the highest balanced
Table 16: Results Mental Stress Detection (MentalArrithmetti, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNct 24
0.6770 = 0.01 16
0,5763 # 0.0102
0.7321 # 0.0108
Task-specifie specifie Models
EFGConformer 36
0.6805 0.0123
0.5829 = 0.0134
0.7424 = 0.0128
SPaRCNet 25
0.6879 0.0107
4.5825
0.7418
ContraWR [26]
0.6631 = 0.0097
0.5787
0.7332 0.0082
Task
CNN-TranSCormer
38
0.6779 = 0.0268
2.7777
0.7258 = 0.0336
FFCL [27
0,6798 # 0.0142
0,5786 # 0.0266
0.7330 # 0.0198
ST-Transformer
37
0.6631 # 0.0173
0.5672 0.0259
0.7132 0.0174
Foundation Models
BIOT 40
0,6875 # 0,0186
0.6004 = 0.0195
0.7536 = 0.0144
LaBraM-Base
0.6909 0.0125
0.5999 # 0.0155
0.7721 = 0.0093
CBraMod [6
0.7256 = 0.0132
0.6267 = 0.0099
0.7905 = 0.0073
CSBrain (Ours)
0.7558 + 0.0106
0.6696 + 0.0221
0.8478 = 0.0297
D.8 Mental Disorder Diagnosis
Mental disorder diagnosis serves as a valuable diagnostic tool for conditions such as depression, schizophrenia, and bipolar disorder. These disorders are often associated with characteristic ab- normalities in EEG spectral features or connectivity patterns across different episodes or disease stages, reflecting underlying neural dysfunction. Owing its non-invasive nature and low cost, ERG expected to partially compensate for the subjectivity and limitations of traditional diagnostic approaches. We evaluate our model on the Mumtaz2016 dataset (78). Mumtaz2016 includes EEG recordings from 34 patients diagnosed with major depressive disorder (MDD) and 30 healthy control (HC) participants. The data collection experiment includes three sessions: eyes-open, eyes-closed, and task. Following prior work (46), only the eyes-open and eyes-closed sessions are used in this work. EEG signals were recorded from 19 electrodes at sampling rate of 256 Hz. All signals are resampled to 200 Hz. Subsequently, the signals are segmented into 5-second 7,143 trails. We adopt subject-independent evaluation split: data from MDD patients and 19 HCS are used for training, MDD patients and 4 HCs for validation, and 5 MDD patients and 5 HCs for testing. Table [7 reports the performance on Mumtaz2016. CSBrain achieves the highest balanced accuracy (0.9643), along with strong overall performance in terms of AUC-PR (0.9936) and AUROC (0.9956). Compared
36

---

to CBraMod, CSBrain improves the balanced accuracy by 0.8 points (0.9643 vs. 0.9560), while maintaining comparable performance on the other two metries.
Table 17: Results on Mental Disorder Diagnosis (Mumtaz2016, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EFGNet 24
0.9232 0.0104
0,9626 = 0.0095
0.9639 = 0.0093
BFGConformer 36
0.9308 0.0117
0.9684 # 0.0105
0,9702 = 0.0101
Task-specifte Models
SPaRCNet 25
0,9316 0.0095
0.9754 0.0065
0.9781 = 0.0063
ContraWR [26]
0,9195 = 0.0115
0,9589 # 0.0102
0.9621 0.0092
ANN-Transformer
38
0.9305 = 0.0068
0,9757 = 0.0074
0.0050
FFCL (27
0.9314 0.0083
0,9717 = 0.0021
0.9753 0.0033
ST-Transformer
0,9135 = 0.0103
0.9578 # 0.0086
0,9594 + 0.0059
Foundation Models
BIOT 40
0.9358 0.0052
0.9736 = 0.0034
0.9758 = 0.0042
LaBraM-Bass
0,9409 = 0.0079
0.9798 # 0.0093
0,9782 = 0.0057
CBraMod 46
0.9560 = 0.0056
0.9923 + 0,0032
0.9921 = 0.0025
CSBrain (Ours)
0.9643 # 0.0155
0.9936 + 0.0034
0.9956 = 0.0020
D.9 Event Type Classification
Event type classification holds significant elinical and research potential, particularly in neurological disorder diagnosis and monitoring. TUEV 779 is an EEG dataset commonly used to evaluate the event type classification performance of decoding models. It contains annotations of EEG segments categorized into six classes: (1) spike and sharp
(SPSW), (2) generalized periodic epileptiform discharges (GPED), (3) periodic lateralized epileptiform discharges (PLED), (4) eye movement (EYEM). (5) artifact (ARTF), and (6) background (BCKG). The EEG signals are recorded from 23 channels at 256 Hz. Following prior work (46) we use 16 commonly adopted bipolar montage channels based on the international 10-20 system for TUEV. All signals are resampled 200 Hz. Subsequently, the signals are segmented into 1 112,491 5-second samples. The original dataset provides predefined training and test splits. We further split the training subjects into training and validation sets using an 8:2 ratio. Table 18 reports the performance comparison on TUEV. CSBrain achieves the highest balanced accuracy (0.6903) and Cohen's 's kappa (0.6833), while maintaining competitive weighted F1 score (0.8333) close to CBraMod (0.8342). Compared to LaBraM and CBraMod, CSBrain yields consistent improvements in overall classification robustness, particularly in capturing minority classes, as reflected by gains in balanced accuracy. By explicitly modeling these cross- scale dependencies, CSBrain offers a more reliable decoding paradigm for real-world clinical EEG interpretation.
Table 18: Results on Event Type Classification (TUEV, 6-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 24
0.3876 + 0.0143
0.3577 + 0.0155
0.6539 + 0.0120
Task-specffic
EEGConformer 36
0.4074 # 0.0164
0.3967 = 0.0195
0.6983 # 0.0152
Models
SPaRCNet
25
0.4161 # 0.0262
0.4233 # 0.0181
0.7024 0.0104
ContraWR 26
0.4384 # 0.0349
0.3912 0.0237
0.6133
TeN-Trrasfrrmer
38
0.4087 = 0.0161
0.3815 + 0.0134
468542000299
FFCL 27
0.3979 # 0.0104
0.3732 + 0.0188
0.6783
ST-Transformer
37
0.3984 # 0.0228
0.3765 # 0.0306
0.6823 = 0.0190
Foundation
BIOT 40
0,5281 # 0.0225
0,5273 # 0.0249
0.7492 # 0.0082
Models
LaBraM Base
41
0,6409 0.0065
0.6637 1 0.0093
0.8312 # 0.0052
CBraMod 46
0.6671 0.0107
0.6772 + 0.0096
0.8342 # 0.0064
CSBrain (Ours)
0.6903 = 0.0059
0.6833 + 0.0047
0.8333 + 0.0057
D.10 Abnormal Detection
Abnormal detection enables the identification of pathological neuroelectric activity, which can significantly reduce the workload of medical personnel while ensuring timely alerts for various
37

---

abnormal EEG events through continuous monitoring. TUAB 79 is an EEG dataset commonly used to evaluate the abnormal detection performance of decoding models. All EEG signals were recorded from 23 channels at 256 Hz. and annotated as either normal or abnormal. Following prior work [46], we use 16 commonly adopted bipolar montage channels based on the international 10-20 system for TUAB. All signals are resampled to 200 Hz. Subsequently, the signals are segmented into 409,455 10-second samples. The original dataset provides predefined training and test splits. We further split the training subjects into training and validation sets using an 8:2 ratio. As shown in Table 19 CSBrain achieves the highest balanced accuracy of 0.8172 and the best AUC-PR of 0.9005, while maintaining strong AUROC of 0.8957. Compared to the strongest baseline LaBraM, CSBrain improves AUC-PR by 0.4 points and balanced accuracy by 0.32 points, with slightly lower AUROC. This performance demonstrates that CSBrain consistently enhances the model's 's ability to discriminate subtle abnormalities across patients, despite class imbalance and recording variability.
Table 19; Results on Abnormal Detection (TUAB, 2-class).
Method
Balanced Accuracy
AUC-PR
AUROC
EEGNet 22
0.7642 # 0.0036
0.8299 + 0.0043
0.8412 = 0.0031
Task specific Models
EEGConformer 36
0.7758 # 0.0049
0.8427 : 0.0054
0.8445 # 0.0038
SPaRCNet 225
0.7896 = 0.0018
0.8414 0.0018
0,8676 = 0.0012
ContraWR [26]
0.7746 = 0.0041
0.8421 = 0.0104
0.8456 = 0.0074
CNN-Transformer
38
0.7777 = 0.0022
0.8433 # 0.0039
0.8461 = 0.0013
FFCL. 27
0.7848 # 0.0038
0.8448 # 0.0065
0.8569 = 0.0051
ST-Transformer
37
0.7966 0.0023
0.8521 = 0.0026
0,8707 = 0.0019
Foundation Models
BIOT 40
0.7959 0.0057
0.8792 + 0.0023
0.8815 = 0.0043
LaBraM-BB M-Base
41
0.8140 # 0.0019
0.8965 + 0.0016
0,9022 # 0.0009
CBraMod 46
0.7891 = 0.0030
0,8636 # 0.0063
0.8606 = 0.0057
CSBrain (Ours)
0,8172 # 0.0043
0.9005 # 0.0066
0.8957 = 0.0046
D.11 Slowing Event Classification
TUSL 79 is a clinically relevant benchmark for slowing event classification, containing three classes: seizure, slowing, and complex background. Following previous work [188], the raw EEG signals from 23 channels were retained, segmented into 10-second samples, and resampled to 200 Hz, resulting in 245 samples. The dataset is split into training, validation, and test sets in a 6:2:2 ratio. Table 20lle200pap presents the performance comparison on TUSL. CSBrain outperforms all baselines by a notable margin, achieving a balanced accuracy of 0.8571. 1, Cohen's 's kappa of 0.7828, and weighted F1 of 0,8568. Compared to the strongest baseline LaBraM, CSBrain improves balanced accuracy by 9.46 points and Cohen's ' kappa by 14.21 points. These results indicate that CSBrain's 's cross-scale spatiotemporal modeling strategy enables more reliable discrimination of subtle, lhort-lived slowing patterns from background and seizure activity, which is key requirement in clinical EEG event classification.
Table 20: Results on Slowing Event Classification (TUSI, 3-class).
Method
Balanced Accuracy
Cohen's Kappa
Weighted F1
EEGNet 24
0.4562 + 0.0729
0.1955 4 0.1115
0.4405 + 0.0960
Task-specific
EEGConforner 36
0.5512 # 0.0666
0,3072 = 0.0877
0.4895 # 0.0733
Models
SPaRCNet 25
0.4185 # 0.0452
0.1399 # 0.0799
0.3500 # 0.0968
ContraWR 266
0.5857 # 0.0662
0.3567 = 0.0968
0.5458 = 0.0798
CNN-Transformer
38
0.3575 = 0.0151
0.0306 + 0.0179
0.2235 0.0251
FFCL [27
0.3159 = 0.0688
0.0628 + 0.0880
0.2120 = 0.0786
ST-Transformer
37
0.4000 # 0.0329
0.0866 + 0.0449
0.3793 + 0.0459
Foundation
BIOT 40
0.5785 = 0.0303
0.2012 + 0.0212
0.2394 # 0.0404
Models
LaBraM-BB Base
41
0.7625 + 0.0131
0,6407 + 0,0304
0.7614 + 0.0210
CBraMod 46
0.7388 # 0.0320
0.6149 + 0.0550
0.7453 = 0.0360
CSBrain (Ours)
0.8571 0.0240
0.7828 # 0.0270
0.8568 + 0.0180
38

---

E Additional Analysis and Ablations
E.1
The Effectiveness Pretraining
3.0
8.00
C.0
640
640
903
Figure Performance comparison between CSBrain without and with pretraining.
We evaluate the effectiveness of large-scale pretraining by comparing CSBrain with and without pretraining across four representative EEG decoding tasks. The non-pree non-pre-trained model uses the same architecture but is trained from scratch the downstream dataset only. Figure X summarizes the performance across three standard metrics. Pretraining leads to consistent performance gains across all datasets, confirming its general effectiveness. The improvement is especially pronounced in data- limited scenarios. On BCIC-IV-2a (5,000 samples), pretraining improves balanced accuracy from 0.39 to 0.57, Cohen's kappa from 0.23 to 0.42, and weighted Fl from 0.39 to 0.56. This demonstrates that pretraining is especially beneficial when downstream data is scarce and task difficulty is high. On SEED-V and TUEV, both of which contain improvements. For TUAB, which contains over 400,000 labeled EEG samples, the gains from pretraining are relatively modest. This suggests that when downstream training data is sufficiently large and diverse, training from scratch becomes more competitive. Nevertheless, pretraining still brings consistent improvements.
100,000 samples, pretraining still yields substantial
In summary, large-scale pretraining is universally beneficial across tasks. The performance gain is especially prominent in data-limited settings, while still being meaningful larger-scale datasets. These results support the core motivation of EEG foundation models: leveraging unlabeled EEG at scale to improve downstream performance, particularly when labeled data limited.
E.2 Scaling Analysis
As foundation model for EEG decoding, CSBrain is expected to benefit from larger pretraining datasets and higher model capacities. examine this, we perform scaling analysis from two perspectives: (i) the amount of pretraining data, and (ii) the model capacity (number of parameters).
Effect of Pretraining Data Volume. We vary the scale of unlabeled EEG data used for pretraining, ranging from 10 hours to 9000 hours. As shown in Figure 9 model performance improves consistently with increased pretraining data. Accuracy, Cohen's 's kappa, and Fl score all show steady gains as data size increases, particularly between 500 and 5000 hours. Beyond 5000 hours, the improvement begins to plateau, suggesting diminishing returns under fixed model capacity.
Effect of Model Capacity. We investigate the impact of model size by varying the number of Transformer layers in CSBrain from 2 to 12, which corresponds to model sizes from 1.4M to 4.9M parameters. As illustrated in Figure downstream performance improves steadily with larger model capacity, indicating that deeper networks provide stronger representational power for EEG decoding.
Together, these results follow the general trend neural scaling laws [T99), where both data and model scale contribute to performance improvements. While our current exploration is constrained
39

---

1M01 trbed
I8n 4rbre)
0333
020
0.4
0423
Stute
Tea110
Figure 9: Effect of pretraining data volume CSBrain performance.
0.45
8.53
0.45
Figure 10: Effect of model parameter size CSBrain performance.
by computational resources and may not reach the scaling limits, we believe further scaling both dimensions will continue to yield gains. Determining how much EEG data is required to effectively pretrain large-scale EEG foundation models remains a critical open question. We call for broader community efforts to collaboratively explore this direction, in order to unlock the full potential of unified architectures and large-scale pretraining for EEG-based foundation modeling.
E.3
Ablation on Regional Descriptor
We conduct an ablation study to evaluate the impact of different regional descriptor designs in the inter-region attention module. In our default configuration, for each region Ry, we construct a descriptor by combining two components: sequentially sampled token 2 and the mean- -eean-pooled feature across all tokens in Ry. This hybrid design aims to preserve both localized temporal structure and global regional context. Here, we compare four variants: (1) Token only: Using only the
sequentially sampled token jop from each region. (2) Mean only: Using only the average feature of all tokens in each region. (3) Random Sample: Randomly sampling a token from each region, and
combining it with the regional average. (4) Ours: Sequentially sampling token from each region and combining with the average feature.
0.00
Notpo Subov
Lmatie Necgg GEEDAA
Pdermaeey BOGIAIM
regery BOCKAjM
How mapor BOC-nOa
3.70
0.70
0.6
000
650
1.9
0.52
Melo Only
Micie Only
Ramitci Samplic
Du5
Figure 11: Ablation on regional descriptor design.
Figure 11 reports the results of our ablation study on regional descriptor design across four repre- sentative tasks. The default setting consistently achieves the best performance across all tasks and metrics. Using only the token only the mean leads to noticeable drops in accuracy and robustness,
40

---

indicating that both local and regional information is essential. Random sampling performs slightly better than the single-source variants but still underperforms the default design. These results confirm that combining local token features with region-level context enhances inter-region attention, while sequential sampling (covering partition) ensures consistent spatial coverage.
Additional Visualizations
F.1 Topography Visualization
In this section, we present topography visualizations for four representative datasets: SEED-VIG BCIC-IV-2a, SEED-V, and BCIC2020-3. Specifically, apply Gradient- Class Activa- tion Mapping (Grad-CAM) [80] to estimate the contribution of each EEG channel to the model's predictions. Across different EEG tasks, the visualizations reveal distinct activation topographies, which reflect task-specific activation patterns and demonstrate the model's ' ability to capture spatially meaningful representations.
For the SEED-VIG dataset, as shown Figure vigilance states primarily activate the temporal and occipital lobes, indicating sustained engagement of the auditory and visual systems. In contrast, fatigued state exhibits reduced activation these lobes, suggesting marked decline auditory and visual information processing.
vigilance
fatigue
Figure 12: Topography visualization of SEED-VIC
left- or right-hand motor imagery induces localized activation in the contralateral motor cortex, whereas motor imagery involving both feet or the tongue results more symmetrical activation across the bilateral motor cortex. These findings suggest that motor imagery effectively modulates neural activity in the motor cortex, as reflected by event-related desynchromization (ERD) and synchronization (ERS) 88.
For the BCIC-IV-2a dataset, as shown in Figure 13
left hand
right hand
both feet
tongue
Figure 13: Topography visualization of BCIC-IV-2a.
For the SEED-V dataset, shown in Figure 14 emotional tasks clicit significant activation across the frontal and occipital lobes, consistent with the finding in [2]. Notably, different emotional states produce distinct activation patterns, providing critical neurophysiological basis for their identification.
For the BCIC2020-3 dataset, as shown in Figure 15 speech imagery evokes significant activation the frontal and temporal lobes [82), which are crucial for imagined speech processing [83] In particular, strong activation is observed in fefthhemisphere electrodes F7, FT7, FC5, and F5, which correspond primarily Broca's area. This finding further supports the involvement Broca's area in speech processing [83]. Overall, these visualizations confirm that different EEG decoding tasks
41

---

happy
sad
disgust
neutral
fear
Figure 14: Topography visualization of SEED-V.
'hello'
'help me'
'stop'
'thank you'
'yes*
Figure 15: Topography visualization of BCIC2020-3.
elicit distinct activation regions and spatial scales, reinforcing the importance of modeling cross-seat scale neural dynamics. By explicitly capturing such multi-scale patterns, CSBrain effectively adapts to the heterogeneous spatial characteristics of brain activity across diverse cognitive domains.
F.2 Representation Visualization
To qualitatively assess the discriminative capability of the representations learned by our model, we conduct t-SNE visualizations [120] on two representative datasets: BCIC-IV-2a (motor imagery classification) and BCIC2020-3 (imagined speech classification). For each dataset, we compare the feature distributions of raw EEG signals and those produced by CSBrain after encoding. As shown in Figure and Figure 17 the features learned by CSBrain exhibit significantly improved clustering effects compared to the raw EEG input. In BCIC-IV-2a (Figure [6), different motor imagery classes form more compact and well-separated clusters, indicating that CSBrain effectively captures task-relevant discriminative patterns. Similarly, for BCIC2020-3 (Figure 17). imagined speech categories that are difficult to distinguish in the raw EEG space become more distinguishable after CSBrain encoding.
42

---

Row EBG Signab of BDC-V33
CSBrasn on SCCC
furd
not Luke
Figure 16: t-SNE visualization of feature representations BCIC-IV-2a. Left: raw EEG signals; Right: CSBrain representations.
bignats
Idd
hd
Figure 17: t-SNE visualization of feature representations on BCIC2020-3. Left: raw EEG signals; Right: CSBrain representations.
F.3 Attention Visualization
The attention visualizations highlight the cross-scale modeling capability of CSBrain. As shown in Fig. [8] attention patterns across multiple layers reveal the model's ability to capture temporal dependencies at various scales. For example, in the lower layers, attention focuses more on local features, while in deeper layers, attention spans broader time windows, demonstrating CSBrain's proficiency in modeling both local and long-range dependencies. The hierarchical attention across multiple layers enhances its ability capture complex spatiotemporal relationships in EEG data, key advantage of the cross-scale approach.
E4 Channel Relationship Visualization
To further investigate the spatial dependencies captured CSBrain, visualize the inter-channel relationships learned on two representative datasets: BCIC-IV-2a and BCIC2020-3. Specifically, we first compute the average token embedding for each EEG channel from the final layer representations. We then calculate the cosine similarity between every pair of channel embeddings to quantify inter- channel relationships and visualize those with similarity greater than 0.5. As shown in Figure 19Iad Figure 20, the learned channel similarity patterns reveal two notable characteristics. First, channels within the same anatomical brain region tend exhibit high similarity, indicating that CSBrain captures strong intra-region dependencies. Second, we observe meaningful long-range inter-region connections, such as between frontal and parietal regions or temporal and occipital regions, suggesting that our model also learns physiologically plausible global interactions across distant brain areas. These results validate the effectiveness of our cross-scale modeling approach in capturing both local and global spatial dynamics, which essential for robust and generalizable EEG decoding.
43

---

1909
Dun Layer3
Drg
Jeg Dan Loyer
Deg Eat--too wwg.
Figure 18: Attention Visualization.
FC3
PZ
FC1
RCZ
PZ
C24
222
FC4
22
C5
CP3
a
C6
Cz
C4
Figure 19: Channel similarity relationships of BCIC-IV-2a.
G Discussion
G.1
Implication
Our work emphasizes the critical role of cross-scale spatiotemporal modeling in the design of brain foundation models, factor often overlooked in prior efforts. CSBrain represents paradigm shift from scale-agnostic, densely connected modeling to a cross-scale, structure-awart architecture. By explicitly encoding multi-resolution dynamics through Cross-scale Spatiotemporal Tokenization (CST) and Structured Sparse Attention (SSA), our approach more effectively reflects the complex patterns of neural activity, thereby achieving superior generalization across EEG decoding tasks with inherently diverse spatiotemporal demands.
This modeling perspective provides reusable architectural template for future EEG foundation models and has the potential to generalize to other neurophysiological modalities. The strong performance of CSBrain across 1I representative tasks and 16 public datasets suggests that unified
44

---

ooo aa D0 DE 103 09 AOD P07 RAR TmL
me 122
17
Figure 20: Channel similarity relationships of BCIC2020-3.
architectures when guided by neurophysiological priors, can scale effectively across heterogeneous brain decoding scenarios.
G.2 Limitation
Despite its strong generalization ability, CSBrain still faces several limitations. Firstly, our current exploration of model and data scaling is constrained by computational resources. Following the general trend observed in neural scaling laws [119), model performance tends to improve predictably with increased model capacity and training data size. While our results suggest similar patterns in EEG modeling, we are unable to empirically assess scaling behavior at the scale of vision- language or language-only models (c.g., CLIP or GPT). Moreover, an open question remains: how much EEG data is sufficient to pretrain high-quality foundation models? Due to the sparse, noisy, and heterogeneous nature of EEG datasets, further work needed to address large-scale dataset unification, normalization, and institinsitul collaboration. We believe such efforts will be essential to unlocking the full potential of EEG-based foundation modeling and call for broader community involvement to collectively advance this direction.
G.3 Future Work
natural extension of this work is to investigate the scaling behavior of EEG foundation models further. This includes expanding pretraining datasets, increasing model capacity, and generalization across downstream tasks. Understanding the efficiency and limits of scaling in EEG foundation models will be key maximizing their representational capacity and real-world applicability. In response to insightful comments from the reviewers, we recognize the importance of exploring EEG foundation models across different model architectures. Specifically, we plan investigate arge-scale pretraining using architectures beyond CSBrain, such as GNN and Mamba models. GNNs excel at capturing local dependencies due to their graph-based structure. On the other hand, Mamba models, which leverage multi-scale learning and attention mechanisms, have demonstrated strong potential handling multi-dimensional data efficiently. A comparative analysis these models with CSBrain will help illuminate the strengths and limitations of each, contributing to more robust EEG foundation modeling. Additionally, we believe that incorporating multiple optimization objectives during pretraining could capture more complex, nuanced patterns in EEG signals. Combining different
45

---

objectives could enhance model robustness and its ability to generalize across diverse scenarios. Additionally, the cross-scale modeling paradigm in CSBrain opens the door unified cross-modal brain foundation models. By integrating signals beyond EEG, such MEG or fMRI, which offer complementary properties in spatiotemporal scales, we can construct richer and more generalizable neural representations and facilitate real-world applications in brain-computer interfaces, clinical diagnostics, and cognitive augmentation.
46


Reviews:
[{"review_id": "f5gHR2eaHj", "reviewer": "Reviewer_5sTW", "summary": "This paper proposes CSBrain, a foundation model for EEG decoding that addresses a key limitation of existing approaches: their failure to capture the cross-scale spatiotemporal structure inherent in neural activity. CSBrain introduces two main components: Cross-scale Spatiotemporal Tokenization (CST) aggregates multi-scale features within temporal windows and brain regions into compact tokens, while Structured Sparse Attention (SSA) efficiently models long-range dependencies across windows and regions. These modules are alternately stacked to progressively integrate cross-scale dependencies. The authors demonstrate CSBrain's effectiveness across 11 EEG tasks and 16 datasets, showing consistent improvements over both task-specific models and foundation baselines. The work establishes cross-scale modeling as a crucial inductive bias for generalized EEG decoding.", "questions": "1.\tHow does CST specifically handle varying temporal scales and channel numbers across datasets? Please provide technical details on the adaptation mechanism for different sampling rates and electrode configurations.\n2.\tWhy use uniform 30s windows and 19 channels during pretraining if cross-scale modeling is key? Please justify this design choice and provide ablation studies on scale diversity impact.\n3.\tHow are 19 channels selected from the Temple University Hospital corpus with various configurations? How does the model handle missing/additional channels in practice? \n4.\tPlease provide detailed complexity analysis and runtime comparisons for SSA versus dense attention across different temporal lengths and channel numbers.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written with clear logical flow and comprehensive method descriptions. The motivation for cross-scale modeling is well-articulated, and the technical components (CST and SSA) are described with sufficient detail for understanding and reproducibility.\n2. The experimental evaluation is thorough and convincing, spanning 11 diverse EEG decoding tasks across 16 datasets with consistent improvements over both task-specific models and strong foundation baselines, demonstrating the generalizability of the approach.\n3. The paper introduces a principled architectural innovation that explicitly captures the inherent multi-scale spatiotemporal structure of neural activity, moving beyond scale-agnostic dense modeling paradigms to establish cross-scale modeling as a key inductive bias for EEG decoding.\n\nWeakness:\n\n1.\tThe paper lacks detailed explanation of how CST handles differences across temporal scales and varying channel numbers, with the cross-scale adaptation mechanism not thoroughly described in the main text.\n2. Despite proposing cross-scale modeling, CSBrain uses uniform 30s temporal windows and 19 channels during pretraining, only encountering scale diversity in downstream tasks, which may limit learning of truly scale-invariant representations.\n3. The methodology for selecting 19 channels from the Temple University Hospital EEG corpus with various channel configurations is not clearly described, affecting reproducibility and understanding of generalization capabilities.", "was_revised": true, "final_justification": "The authors have provided clear explanations and new experimental results that address several of the concerns raised in my initial review. The paper is now a more complete work, as the authors have resolved the main points of uncertainty. Therefore, I am maintaining my score of **4: Borderline accept**.", "rebuttal": "Dear Reviewer 5sTW,\n\nThank you for your encouraging feedback and constructive suggestions! We are greatly encouraged by your recognition of our **principled architectural innovation, the thorough and convincing experimental evaluation**, and **the paper's well-written and comprehensive method descriptions**.  We are also pleased that you found **the motivation for cross-scale modeling well-articulated**, and that **the CST and SSA components are described with sufficient detail for understanding and reproducibility**. Due to time and page limitations, we have made every effort to address your questions through the clarifications provided. We hope these responses resolve most of your concerns and will be considered in your final assessment.\n\n> W1&Q1. Handling of Varying Spatiotemporal Settings.\n\n1. **Different Sampling Rates&Temporal Scales:**\nAs mentioned in Sec.2.1 (L120), all raw EEG signals undergo a standardized preprocessing pipeline, including resampling to a uniform 200 Hz. This ensures that temporal differences across datasets are normalized before entering the CST, **aligning with standard practices in foundation models like [ICLR2024] LaBraM, [NIPS2024] BIOT, and [ICLR2025] Cbramod.** \nSubsequently, CST further adapts to different temporal scales through its multi-scale temporal convolutions, using kernels of different sizes to capture temporal patterns at various scales. CST and the subsequent SSA module closely collaborate and complement each other to jointly capture diverse temporal scales from local to global. This combined setup collectively fulfills the varying temporal scale requirements of different datasets.\n\n2. **Different Electrode Configurations:**\nIn CSBrain, we first partition electrodes based on the standardized 10-20 system into brain regions (Frontal, Central, Parietal, Temporal, Occipital). Then, within the electrodes of each defined brain region, CST applies local multi-scale spatial convolutions. CST enables the model to learn localized yet multi-scale patterns robustly, irrespective of channel count variations within a region. This design ensures that while electrode numbers and spatial distributions may vary across different datasets, the unified 10-20 system-based spatial region partitioning and the application of local multi-scale spatial convolutions allow CST to effectively adapt to diverse electrode configurations, ensuring robust compatibility.\n\n*To make our design motivation and process even clearer, we will modify Sec. 2.1 and 2.2. Specifically, Sec. 2.1 will further emphasize the role of data preprocessing in handling diverse datasets. Sec. 2.2 (L141-153) will add clarifications to highlight CST's adaptation to different temporal scales and channel configurations. Additionally, we will open-source our code and models to facilitate further research and community use.*\n\n> Q2. Pretraining Data Setting.\n\n**The cross-scale capability of CSBrain comes from the intricate model architecture, not from the variations in input signal format.** Through the synergistic combination of the CST and SSA modules, we effectively capture cross-scale spatiotemporal dependencies, as demonstrated by our experiments across 11 tasks and 16 datasets. Speciffcally:\n\n**1. Design Justification:** The main objective of pretraining is to learn robust, generalizable EEG representations from a large, unlabeled corpus. To achieve this efficiently, we chose a uniform data setting to simplify the data pipeline, ensuring stability and efficiency during large-scale training. Using varying input settings, such as varying EEG segment lengths or channel counts, would complicate preprocessing and model training, leading to instability. This is consistent with other widely validated models like CBraMod, which also use uniform input for pretraining. Additionally, *as shown in Supp. Sec. E.1, we have demonstrated that our pretraining improves performance across datasets with various spatiotemporal settings.*\n\n**2. More Ablation Studies:** we further conduct ablation studies to evaluate the impact of pretraining diversity and model configurations on downstream task performance. Due to time and resource constraints, the total pretraining data is fixed at 2000 hours.\n\n(1) Multiple Pretraining (MP): We pretrain on diverse data configurations, including 5s, 20s, and 30s windows with the TUEG dataset (19 channels), and datasets like FACED (10s, 32 channels), SEED-VIG (8s, 17 channels), and BCIC-IV-2a (4s, 22 channels), totaling 2000 hours.\n\n(2) Single Pretraining (SP): We pretrain on a subset of the TUEG dataset (30s, 19 channels), totaling 2000 hours.\n\n(3) Single-Scale CSBrain: A single-scale configuration by adjusting CST convolution kernels to [1, 1, 1].\n\n|Config|TUEV B-A |TUEV Kappa|TUEV F1|BCIC2020-3 B-A|BCIC2020-3 Kappa|BCIC2020-3 F1|\n|-|-|-|-|-|-|-|\n|MP+Full CSBrain|0.5390±0.0229|0.4399±0.0290|0.5501±0.0198|0.6465± 0.0178|0.6264±0.0169|0.7892±0.0180|\n|SP+Full CSBrain|0.5443±0.0232|0.4254±0.0301|0.5527±0.0297|0.6414±0.0187|0.6236±0.0198|0.7971±0.0183|\n|MP+Single-Scale CSBrain|0.5041±0.0208|0.3821±0.0223|0.5019±0.0202|0.6122±0.0193|0.5596±0.0212|0.7713±0.0205|\n\n*Results show that, under the same pretraining data volume, variations in data format has a minimal impact on downstream performance. The cross-scale model architecture (CST and SSA) is the primary factor in improving performance, further validating the effectiveness of our pretraining setup and the core idea of cross-scale spatiotemporal modeling.*\n\n> W3&Q3 Pretrained Channels Selection & Handling of Diverse Downstream Channel Settings.\n\n**Pretrained Channel Selection.**\nAs noted in Sec.3.1 L225, we follow established practices by [ICLR2025]CBraMod using the TUEG dataset for pretraining. The EEG signals in TUEG range from 19 to 23 channels. We select 19 common EEG channels (Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, O2) that correspond to a subset of the 10-20 international electrode placement system to ensure clean and uniformly formatted pretraining data. The dropped channels include A1, A2, T1, and T2. A1 and A2 are located near the auricular region, which contain less brain information and are typically used for reference electrodes. T1 and T2 are obsolete and no longer utilized.\n\n**Handling of Diverse Downstream Channel Settings.**\n\n- **Architecture.** CSBrain's architecture is inherently designed to adapt to the actual channel count of each downstream dataset. During fine-tuning, for any given downstream dataset, we map its electrodes to a set of anatomically predefined brain regions $\\mathcal{R}$ based on the 10-20 system. Within each region ($R_r$), CST applies multi-scale spatial convolution kernels. This allows CST to aggregate cross-scale features from the available electrodes within that region, regardless of their exact number. Subsequently, SSA computes attention based on regional descriptors formed via sequentially sampled tokens $x_r^{rep}$ and average features of $R_r$, establishing long-range dependencies between these regions. *Thus, despite varying electrode configurations, our brain-region-based assignment allows diverse downstream setups to benefit from large-scale pretraining.*\n- **Implementation.** From an implementation perspective, *except for the task-specific heads with fixed input-output dimensions, all model weights are inherited directly from pretraining.* This is because the underlying convolutional and Transformer components of CSBrain are flexible enough to handle varying input sizes and dynamically grouped region information.\n- **Experiment Validation.** Furthermore, *as stated in supp. mat. Sec. E.1 (L418-435), our work, consistent with prior foundation models like Cbramod and BIOT, clearly demonstrates the effectiveness of pretraining*. These pre-trained models significantly improve performance on downstream tasks, even those with different channel settings.\n\n*To further enhance transparency, new Sec. B.3 and C.4 will be added to the Supp., detailing the channel settings and region assignments for the pretraining data and each downstream task dataset, respectively.* Our code and models for pretraining and downstream fine-tuning will be **open-sourced** to facilitate community use.\n\n> Q4. Complexity Analysis.\n\nWe compare the runtime, computational complexity, and performance of SSA versus Dense Attention (DA) on TUEV (C=16, T=5s), BCIC2020-3 (C=64, T=3s), and FACED (C=32, T=10s) datasets. Runtimes are averaged over 500 runs on a single A100 GPU with a batch size of 64. For DA, complexity is $O(N^2)$, where $N = T \\times C$. For SSA, complexity is $O(N \\cdot k)$, where $k$ = max(electrodes within a brain region) + window length. \n\n|Model|Runtime|Complexity|B-Acc|Kappa|F1-W|\n|-|-|-|-|-|-|\n|DA-TUEV|0.0675s|$(16\\times5)^2$|0.5957|0.5781|0.7655|\n|SSA-TUEV|0.0603s|$(16\\times5)\\times8$|0.6901|0.6839|0.8339|\n|DA-BCIC|0.0778s|$(64\\times3)^2$|0.5075|0.3856|0.5156|\n|SSA-BCIC|0.0695s|$(64\\times3)\\times26$|0.6001|0.5018|0.6008|\n|DA-FACED|0.0921s|$(32\\times10)^2$|0.5272|0.4798|0.5231|\n|SSA-FACED|0.0799s|$(32\\times10)\\times13$|0.5754|0.5202|0.5802|\n\n*We observe that SSA consistently exhibits lower runtime and complexity across various time and channel settings.* It should be noted that while SSA offers clear efficiency gains, the absolute runtime difference between SSA and DA is not as dramatically pronounced as the complexity analysis suggests, owing to PyTorch's highly optimized parallel computations for attention mechanisms. *Crucially, SSA consistently outperforms DA across all performance metrics.* This advantage stems from SSA's respect for the intrinsic cross-scale spatiotemporal structure of neural activity, effectively capturing meaningful dependencies without indiscriminately associating all tokens. *We will briefly add this analysis to L293-305 of the main paper and provide the full details in Supp. Sec. E.*"}, {"review_id": "WoSChJlXrf", "reviewer": "Reviewer_Yw1n", "summary": "The authors present CSBrain, a foundation model for EEG decoding that combines cross-scale spatiotemporal tokenization (CST) and structured sparse attention (SSA) to capture multi-scale, anatomically-aware dynamics in EEG signals. They pretrained their model on ~9,000 hours of EEG and evaluated across 11 tasks on 16 public datasets.", "questions": "1.\tWhy was concatenation chosen for feature fusion in 2.1 rather than addition or another method?\n2.\tWhat guided the specific design of SSA’s formulation? Can the authors provide deeper motivation or derivation?\n3.\tCan the authors clarify whether aggregation in 155–156 refers to functional or anatomical context? If functional, how is it designed in the model to do so?\n4.\tCould SSA’s formulation (185–203) be explained in greater depth or with additional derivation?\n5.\tCould the authors provide qualitative brain map analyses for different attention variants or for baselines?\n\nMinor: \n\nThere are minor typos, such as in line 154, where the superscript appears as t but should likely be s (referring to spatial rather than temporal context).", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper is well-written, with a clear structure and logical flow. Most sections are supplemented by additional material in the appendix.\n2.\tThe authors perform a comprehensive evaluation, applying CSBrain to a wide range of tasks and datasets.\n\nWeaknesses:\n\n1.\tIn Section 2.1, the choice to concatenate features into a unified embedding is not justified — why not addition or another fusion strategy?\n2.\tIn Section 2.2, the formulation of SSA lacks deeper derivation or motivation. How was this design chosen? What theoretical or empirical reasoning supports it?\n3.\tThe explanation of SSA (lines 185-203) is hard to follow — more derivation or intuition would strengthen this section.\n4.\tDetails on baseline training (e.g., whether competing foundation models were re-trained with the same objective or used published weights) are sparse, making it harder to assess comparison fairness.\n5.\tAblations focus more on SSA variations but less on CST: no analysis of token granularity choices, window/region design, or their effect on generalization.\n6.\tAs GNN architectures have shown strong performance in EEG domain for purposes such as emotion recognition and seizure detection by explicitly modeling electrode connectivity, including a GNN baseline would be valuable. For instance, GGN[1] for seizure detection, and HetEmotionNet[2] for emotion classification. Adding such baselines would provide meaningful context for evaluating CSBrain’s approach to spatial dependency modeling.\n7.\tQualitative analyses of attention maps across attention types (dense, criss-cross, SSA) are missing; this would enhance interpretability.\n8.\tThe paper provides brain maps for CSBrain but not for baselines — including these would better contextualize the model’s interpretability gains.\n\n[1] Li, Zhengdao, et al. \"Graph-generative neural network for EEG-based epileptic seizure detection via discovery of dynamic brain functional connectivity.\" Scientific reports 12.1 (2022): 18998.\n\n[2] Jia, Ziyu, et al. \"HetEmotionNet: two-stream heterogeneous graph recurrent neural network for multi-modal emotion recognition.\" Proceedings of the 29th ACM international conference on multimedia. 2021.", "was_revised": true, "final_justification": "The authors addressed several of my concerns with new analyses and clarifications. However, I am still not convinced of their claim regarding GNNs being inherently limited in capturing long-range dependencies due to their receptive field, as the receptive field can be extended or adapted through graph construction strategies. Additionally, I still believe that a more comprehensive qualitative analysis of the attention mechanisms were needed to assess whether the proposed method truly offers improved interpretability. Overall, I will keep my borderline accept score.", "rebuttal": "Dear Reviewer Yw1n,\n\nThank you for your encouraging feedback and constructive suggestions! We are greatly encouraged by your recognition of our **comprehensive evaluation** and that the paper is **well-written, with a clear structure and logical flow**. Due to time and page limitations, we have made every effort to address your questions and suggestions through the clarifications provided. We hope these responses resolve most of your concerns and will be considered in your final assessment.\n\n> W1&Q1. Feature Fusion Strategy\n\nThanks for the suggestion. In EEG analysis, both concatenation and addition are commonly used to fuse temporal and spectral information. We opted for concatenation as it preserves the integrity and independence of each feature type, thereby providing a richer input space. The choice of concatenation-based feature fusion is also widely supported by prior validated studies [1-2].\n\nNotably, the core contribution of our work lies in the CSBrain architecture itself, particularly the Cross-Scale Spatiotemporal Tokenization (CST) and Structured Sparse Attention (SSA) modules. These components build cross-scale dependencies upon these fused initial features, capturing intricate spatiotemporal patterns. The effectiveness of our work has been validated across 11 representative tasks and 16 datasets.\n\nWe agree that examining their impact on EEG representation learning is valuable and will consider this in future work, given the limited rebuttal timeframe. *Your insightful comment will be included in Sec. G.3 (Future Work) to guide further research.*\n\n[1] Li, Yang, et al. \"A temporal-spectral-based squeeze-and-excitation feature fusion network for motor imagery EEG decoding.\" IEEE T-NSRE2021, Cited by 142\n\n[2] Jia, Ziyu, et al. \"Sst-emotionnet: Spatial-spectral-temporal based attention 3d dense network for eeg emotion recognition.\" ACM MM2020,  Cited by 170\n\n> W2/3&Q2/4. Design Motivation of SSA\n\n- **Motivation:** SSA is designed to model long-range spatiotemporal dependencies across time windows and brain regions via efficient sparse attention. It complements CST that captures structured, cross-scale representations within local temporal and spatial contexts. This synergy enables CSBrain to effectively model brain activity from local to global scales, aligning with the cross-scale nature of neural dynamics.\n\n- **Details:** CST aggregates contextual information within local temporal windows and brain regions, endowing each token with window/region-level awareness. Based on this, SSA performs sparse attention computation across windows and regions. Specifically, cross-scale tokens are sampled and grouped based on the same indices, and attention is then computed within each group, effectively having a representative from each window/region interact across regions/windows. Multiple groups are set up to ensure every token is included in at least one group for computation, preventing information loss.\nAdditionally, in inter-region attention, we introduce a regional descriptor, which adds the average feature of the region to the sampled token. Unlike temporal windows (which have fixed lengths), the number of electrodes varies across brain regions. In regions with more electrodes, CST might not fully capture all information. By introducing the regional descriptor, we ensure complete information flow between regions.\n\nAs shown in Sec. 2.2 (L293–305), SSA achieves superior performance and efficiency compared to other attention mechanisms. *We will integrate this explanation into L185–203 of the main paper for clarity, and release our code and models to support future research.*\n\n> W4. Details on Foundation Model Baseline\n\nFor all compared foundation models, we utilized their officially published weights from their respective GitHub repositories. This is a common and validated practice adopted by prior works including [ICLR24]Labram and [ICLR25]CbraMod for evaluating foundation models, ensuring fair comparison against their intended performance.\n\nThe specific evaluation protocols for each task and dataset are outlined in Supp. Sec. C&D, following practices of CbraMod and other well-established studies. *To further enhance transparency, we will include an additional note in Sec.C.2 to specify the use of the published weights and provide weight download links.*\n\n> W5. More Ablations on CST\n\n- As shown in L278-292 and Fig. 4 of the main paper, we have already provided an analysis of token granularity choices in CST, as well as the effect of CST stacking strategy on performance. These results demonstrate CST's role in learning richer, more flexible EEG representations and the importance of token granularity in effectively capturing diverse EEG dynamics across tasks.\n- To further address your concerns and provide deeper insights into token granularity and window/region design, we conduct additional experiments to explore the configuration of using multiple kernels of the same scale: [1,1,1], [3,3,3], and [5,5,5], in contrast to our multi-scale [1,3,5] design. Due to time and computational constraints, these experiments were pre-trained on a subset of the TUEG dataset (2000 hours, excluding TUEV) and evaluated on two representative tasks.\n\n|Config|TUEV B-Acc|TUEV Kappa|TUEV F1-W|BCIC2020-3 B-Acc|BCIC2020-3 Kappa|BCIC2020-3 F1-W|\n|-|-|-|-|-|-|-|\n|[1,3,5]|0.6414±0.0187|0.6236±0.0198|0.7971±0.0183|0.5443±0.0232|0.4254±0.0301|0.5527±0.0297|\n|[1,1,1]|0.6122±0.0193|0.5596±0.0212|0.7713±0.0205|0.5041±0.0208|0.3821±0.0223|0.5019±0.0202|\n|[3,3,3]|0.6338±0.0184|0.6137±0.0179|0.7870±0.0188|0.5298±0.0247|0.4354±0.0298|0.5399±0.0280|\n|[5,5,5]|0.6188±0.0190|0.5765±0.0209|0.7732±0.0195|0.4998±0.0237|0.3809±0.0242|0.4931±0.0279|\n\nThese results further validate CST’s effectiveness in cross-scale modeling. Our proposed [1,3,5] multi-scale configuration consistently outperforms single-scale alternatives, showing that modeling diverse scales is more effective than simply increasing uniform kernels. This reinforces the importance of CST’s design for capturing EEG’s complex multi-scale spatiotemporal patterns and enhancing generalization. *We will update Supp. Sec. E to reflect these insights.*\n\n> W6. GNN Baselines\n\n*We will reference and discuss the two classic GNN works you mentioned in both the Introduction and Related Work, and include them as baselines in our evaluation.* Due to time constraints, we conducted tests on 4 representative tasks and *will include the full results in the revised paper.* Note that HetEmotionNet was originally designed for heterogeneous multimodal inputs, so when applied to EEG-only datasets, the graph is simplified, which may reduce its performance.\n\n|Model|BCIC-IV-2a(B-Acc,Kappa,F1-W)|TUEV(B-Acc,Kappa,F1-W)|MentalArithmetic(B-Acc,AUC-PR,AUROC)|FACED(B-Acc,Kappa,F1-W)|\n|-|-|-|-|-|\n|HetEmotionNet|0.4479,0.2639,0.4295|0.4771,0.4356,0.7105|0.6424,0.5924,0.7857|0.3928,0.3134,0.3849|\n|GCN|0.4653,0.2870,0.4436|0.5778,0.5669,0.7631|0.6250,0.4836,0.7254|0.4274,0.3541,0.4277|\n|CSBrain|0.5657,0.4209,0.5637|0.6903,0.6833,0.8333|0.7558,0.6696,0.8478|0.5752,0.5204,0.5796|\n\nAs shown, CSBrain consistently outperforms GNNs, demonstrating its superior capability in capturing spatiotemporal dependencies.\n\n> W7/8&Q5. More Visualization Results\n\nWe greatly appreciate your valuable suggestions. We fully agree that such analyses would significantly enhance model interpretability and highlight CSBrain's advantages. However, *due to NeurIPS conference committee regulations, we are unable to submit new PDFs or external links to showcase additional visualization results during the rebuttal phase.*\n\nNevertheless, we have already provided a rich set of visualizations in the main paper and supplementary materials, designed to illustrate CSBrain's interpretability and the effectiveness of its cross-scale modeling:\n- **Topography Visualizations:** Presented in the main paper (Fig. 6, L337-345) on four representative tasks. Supp. F.1 (L474-501, Figs.6/7/8/9) provides more detailed topography visualizations for various categories within the four different tasks.\n- **Representation Visualizations (t-SNE):** In Supp. F.2 (Fig. 10/11, L502-512), we visualize the feature distributions of raw EEG signals versus CSBrain's learned representations on two tasks. These demonstrate CSBrain's ability to learn highly discriminative and well-separated representations.\n- **Channel Relationship Visualizations:** Supp. F.3 (Fig. 12/13, L513-525) includes visualizations and analysis of learned inter-channel relationships, highlighting our model's capacity to capture both local and global global spatial dynamics.\n\nThese existing visualizations collectively demonstrate CSBrain's strong interpretability and the effectiveness of its cross-scale modeling. We believe this constitutes one of the most comprehensive sets of visualizations in published EEG foundation model papers to date. *We sincerely hope you understand our inability to upload more visualization results due to NeurIPS regulations, and we look forward to your support.*\n\n> Q3. Aggregation Context\n\nTo clarify, the aggregation in L155-156 refers specifically to  anatomically defined brain regions. We do not explicitly perform functional aggregation as precise functional connectivities are difficult to define. Instead, our spatial tokenization explicitly incorporates anatomical priors by assigning electrodes to five brain regions (e.g., frontal, central, parietal, temporal, occipital lobes) anatomically defined in the standardized 10-20 system. These lobes broadly reflect the distribution patterns of functional regions and provide a physiologically meaningful framework for spatial grouping. As evidenced by Fig. 6 and visualizations in the supp, our model effectively learns functional characteristics across tasks in a coarse-grained manner. by facilitating the modeling of cross-scale neural structures that respects brain anatomy, our model allows effective adaptation to multiple downstream EEG tasks."}, {"review_id": "DmhbafBVkM", "reviewer": "Reviewer_yshg", "summary": "The paper argues that cross-scale modeling is lacking from LLM-inspired brain models. Introduces cross-scale spatiotemporal tokenization and structured sparse attention. Problems with current models include scale-agnostic tokenization and dense attention.", "questions": "In section 2.1 you mention FFT for spectral information, why not use learned 1D convolutions, aren’t those enough?\n\nSSA: How is this different from combining temporal and spatial attention layers?\n\nKernel stacking comparison: Does this mean than even more scales would increase performance?\n\nMore comparisons with other foundational models on the pre-training task.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strength\n- large pre-training dataset\n- very comprehensive downstream evaluation\n\n### Weaknesses\nI think the term tokenization is misleading, as these are simply convolutional layers applied between attention layers.\n- Temporal tokenization is really just a stack of convolutions\n- Spatial tokenization is just standard convolution, but nearby regions are correlated anyway so I don't see what's the point.\n- The masked learning uses MSE which has fundamental issues, such as focusing on low-frequency features.\n- Is the pre-training amount used for the other foundation models similar? Otherwise comparison might not be fair\n- Lack of comparisons with other foundational models in terms of evaluating why yours is better. Not just downstream performance.\n- No direct evaluation of the pre-trained model itself", "was_revised": true, "final_justification": "After carefully reviewing the authors' responses I decided to raise my final score by 1. The authors thoroughly explained how their propose model works, and these clarifications will be integrated into the paper.", "rebuttal": "Dear Reviewer yshg,\n\nThank you for your encouraging feedback and constructive suggestions! We are greatly encouraged by your recognition of **our very comprehensive downstream evaluation** and the use of a **large pre-training dataset**. Due to time and page limitations, we have made every effort to address your questions and suggestions through the clarifications provided. We hope these responses resolve most of your concerns and will be considered in your final assessment.\n\n> W1-3. Clarification of Tokenization\n\n- In the filed of EEG foundation model, which commonly employ Transformer architectures, tokenization is not just a convolutional operation but the essential process of converting continuous spatiotemporal signals into discrete, Transformer‑friendly units (i.e., EEG tokens). This terminology is well established in prior works (e.g., [ICLR 2024] LaBraM; [NeurIPS 2024] BIOT), and since each EEG token serves as the minimal modeling unit, effective tokenization is a fundamental prerequisite for any EEG foundation model.\n- In CSBrain, our Cross-Scale Spatiotemporal Tokenization (CST) module aggregates multi-scale features across both temporal and spatial dimensions into compact, structured token representations. This design is guided by the intrinsic property of neural activity — the cross-scale spatiotemporal structure — and aligns with the underlying structure of EEG signals. The key distinction of our design is that tokenization in CSBrain is not simply a stack of convolutions, but a hierarchical process that aggregates information across scales and spatial regions. This allows the model to capture richer, more granular representations of EEG signals. The resulting tokens, which encapsulate both local and global spatiotemporal information, are passed to the attention layers where the model learns to capture dependencies across different time windows and brain regions.\n\n*We will rewrite L131-140 of the main paper to more explicitly clarify the role of tokenization and its contribution to the CSBrain architecture.*\n\n> W4. About MSE loss\n\nFor masked pretraining, MSE is a widely used and intuitive loss function in signal and image reconstruction tasks, as seen in prior works like MAE and Cbramod.. While MSE does prioritize low-frequency features due to its focus on minimizing larger amplitude errors, this characteristic aligns well with EEG signal properties, where energy is predominantly concentrated in lower frequency bands.\n\nImportantly, in CSBrain, input EEG segments undergo preliminary encoding to extract both temporal and spectral (FFT) features (Sec.2.1 L124-129). During pretraining, the model reconstructs the raw EEG based on these spectral-rich features, compelling it to utilize high-frequency FFT components for accurate reconstruction. Additionally, CSBrain's CST module extracts features across multiple temporal scales, capturing information from a range of EEG frequencies. Therefore, our masked reconstruction task could enforce the model to learn beyond low-frequency features, enabling a more comprehensive understanding of EEG signals.\n\nOur extensive experiments across 11 EEG tasks and 16 datasets demonstrate CSBrain’s strong performance, including tasks sensitive to high-frequency or transient features, such as Motor Imagery Classification and Event Type Classification. *To enhance clarity, we will integrate this discussion into Sec. 2.4 (L205-215). Furthermore, inspired by your insightful comment, we agree that augmenting pretraining with multiple objectives could capture more complexity, and we will incorporate this into our “Future Work” section (Sec. G.3) to encourage broader community efforts.*\n\n> W5-7&Q4. About Pre-trained Model Evaluation\n\n**1. Pretraining Data Amount.**\n\n- **Fairness of evaluation protocols:** In the EEG community, *comparing pretrained models in downstream task performance under varying pretraining data is a standard practice adopted by prior works such as [ICLR2024]LaBraM, and [ICLR2025]Cbramod*. Similarly, in broader AI, foundation models like CLIP, BLIP, and GPT are rarely pretrained on identical datasets, as uniform pretraining with large-scale data is impractical. Thus, our evaluation protocol is NOT unfair. \n\n- **Comparison under the same pretrained data amount:** CSBrain uses more pretraining data than LaBraM but less than BIOT, and uses the **SAME pretraining data** as the previous SOTA model Cbramod (using the TUEG dataset). Across 11 representative tasks and 16 datasets, CSBrain consistently outperforms Cbramod, achieving a macro-average score of 0.7095, surpassing Cbramod’s score of 0.6760. This demonstrates that even with the same amount of pretraining data, our architecture holds a clear advantage. *We will add a comparison of the pretraining data amount in L263 of the main paper for greater clarity and transparency.*\n\nYour insightful comments inspire us to consider future work in building sufficiently large-scale, high-quality open-source EEG databases for unified pretraining. *We will incorporate this direction into Sec. G.3 of our future work to encourage community efforts toward these advancements.*\n\n**2. Regarding Why We Focus on Downstream Performance For Comparisons.**\n\nEvaluation on downstream performance aligns with the core goal of EEG foundation models, which is to achieve performance gains and generalization on downstream tasks through large-scale unsupervised pretraining and a unified model architecture. Since pretraining data and objectives vary across models, prior foundational models such as LaBraM, BIOT, and Cbramod have **followed similar evaluation pipelines**, focusing on downstream performance after pretraining, without directly evaluating the pretraining task itself. In line with these models, we prioritize the evaluation on downstream tasks. \n\n**3. More Validation on Pretraining.**\n\nIn Supp. Mat., we have provided further validation of the quality of our pre-trained model's representations through several approaches:\n- **Pre-training Effectiveness:** In Supp. Mat. E.1, we show significant performance improvements on downstream tasks by comparing models with and without pre-training, validating the effectiveness of our pre-training strategy.\n- **Pre-training Scale Analysis:** Pre-training Scale Analysis: In Supp. Mat. E.2, our experiments demonstrate that increasing pre-training data leads to better performance, proving our compliance with the Neural Scaling Laws for foundation models and good scalability.\n- **Pre-training Process Transparency:** In Supp. Mat. B.2, we provide the Training Loss Curve and hyperparameters, demonstrating effective convergence under our pretraining objective.\n\n> Q1. FFT for Spectral Information\n\nThe use of Fourier spectrum information has been widely adopted in previous EEG foundational models ([ICLR2024]LaBraM, [NIPS2024]BIOT, and [ICLR2025]CBraMod). These models consistently show that integrating frequency-domain information with temporal features significantly enhances decoding performance. Following their practice, our model integrates FFT‑based spectral extraction with 1D convolutions, harnessing their complementary strengths. and the physiological relevance of EEG frequency‑domain information.  While 1D convolutions excel at learning local temporal dependencies, FFT provides an explicit representation of the signal's frequency components, enabling the model to capture essential frequency-based patterns that 1D convolutions might miss. By combining both approaches, our model effectively leverages both temporal and spectral information, providing a comprehensive representation of the EEG signal.\n\n> Q2. The difference of SSA and spatiotemporal Attention\n\nWe compare SSA with Temporal and Spatial Attention (TSA) from both architectural and experimental perspectives.\n- **Architectural Comparison:** The key difference between SSA and TSA lies in how they model long-range spatiotemporal dependencies. TSA typically focuses on independently modeling dependencies along the time and space axes, without distinguishing between intra- and inter-region, or short-term and long-term cross-scale structures. In contrast, SSA respects the intrinsic property of neural activity, accounting for cross-scale interactions across brain regions and time windows. It works in close conjunction with CST to integrate these spatiotemporal dependencies in a more structured and efficient manner. By leveraging the inherent cross-scale structure of EEG signals, our model effectively captures both local and global patterns.\n- **Experimental Comparison:** We compare SSA with Cbramod [ICLR2025] that explores TSA by a Criss-Cross Transformer for modeling dependencies along the time and space axes. As shown in Section 3.3 (L258-277) and Table 2 of our paper, our model achieves a 0.7095 macro-average across 11 representative EEG tasks and 16 datasets, significantly outperforming Cbramod (0.6760 macro-average). Additionally, in L293-305 and Figure 5, our proposed SSA demonstrates superior efficiency compared to TSA (Criss-Cross Attention).\n\n*We will open-source the relevant code and models to facilitate community use and further advancements in this area.*\n\n> Q3. Kernel Stacking Comparison\n\nAs discussed in L278-292, our \"Tokenization Comparison\" ablation study clearly demonstrates that full cross-scale tokenization (K=3) significantly outperforms both single-scale (K=1) and dual-scale (K=2) variants. Specifically, our SSA (K=3, stacked) consistently achieves the best performance across four representative tasks. This suggests that explicitly modeling multiple temporal and spatial scales contributes to learning richer EEG representations, further validating the effectiveness of our core insight—cross-scale spatiotemporal modeling."}, {"review_id": "eseSiQCKjG", "reviewer": "Reviewer_6YFP", "summary": "The paper introduces CSBrain: a foundation model used in the electroencephalography signals (EEG) domain of functional neuroimaging. The authors combine cross-scale spatiotemporal tokenization (CST) with structured sparse attention (SSA) to capture neural patterns on multiple scales and with sparsity in a hierarchical framework. Since EEG tasks exhibit signals at varying scales, this stacked cross-scale spatiotemporal structure is used to capture this diversity which is important for building a foundation model that can work in many settings and environments.\n\nThe introduced CST module starts by applying convolutions with varying kernel sizes across a temporal neighborhood which are then concatenated across scales to yield a temporal token. The temporal tokens are then split into localized spatial neighborhoods based on spatial adjacency in the 10-20 system. Spatial convolutions with varying kernels sizes are then applied to these neighborhoods and concatenated as before to yield the spatiotemporal tokens as output.\n\nThe introduced SSA module takes the spatiotemporal tokens and applies attention in two rounds. First, inter-window attention is applied between tokens which share the same relative index in the chosen window neighborhoods of the temporal convolutions. Second, inter-region attention is performed where each region is summarized through a region descriptor using a projection of the average token together with a randomly sampled token, then attention is applied across all of the regions' descriptor tokens.\n\nSelf-supervised learning is performed using masked reconstruction loss where whole regions of the input are erased for the model to reconstruct. The model is pre-trained on the TUEG dataset.\n\nThe authors present a very comprehensive evaluation of the EEG foundation model evaluating on 16 EEG datasets with 11 different BCI tasks against 10 baseline models. Baseline models are reimplemented and retrained where results are not available on the specific dataset. The proposed model achieves on-par or better fine-tuning performance on all downstream tasks. An ablation study is applied to the number of kernels, showing that the repeated and stacked model with three kernel sizes is the best choice of architecture. An ablation study on the chosen attention method is performed against dense attention and criss-cross attention which showed that the proposed SSA performs well across datasets.\n\nThe authors use Grad-CAM to explore the changes of model prediction by variation in input EEG channels, yielding findings consistent with prior works.", "questions": "### Questions regarding the number of kernels comparison\nFor better comparison, the number of parameters and/or compute should be kept constant between ablations in the number of kernels comparison.\n- Can the authors provide insight into the number of parameters and FLOPS used in this part of the study?\n- Is it just an increase in compute or parameters that gives the perceived performance when increasing the number of kernels used? How much of the performance increase is because of a stronger model (more filters, parameters etc.)?\n- Have the authors experimented with using multiple kernels but keeping the scale the same? Using three kernels of the form [3, 3, 3], for example? Here we are adding more parameters (more kernels) but forgoing the multiple scales.\n- What about [3, 3, 3] + Dense Attention? This would seem like the naïve baseline.\n\n### Questions for the cross-scale spatiotemporal tokenization procedure\nThe inter-region self attention has been proposed before (in Wang et al.), and the proposed alternating stacks of CST and SSA follows the conventional methodology of hierarchical models.\nUsually, in hierarchical transformer models, instead of a convolution, attention is applied intra-regionally and then inter-regionally, e.g. in SWIN transformers.\n- Have the authors considered a model where an intra- region/window attention module replaces the convolutions?\n\nIt is a bit unclear how inter-region attention provides inputs to the next stack in the model. The inter-window attention mechanism makes sense since the attention is just limited using a periodic mask. However, in the inter-region attention, after pooling and randomly sampling tokens from the region, I assume the signal is reduced to having only $R_r$ tokens in total over which the attention is performed.\n- How are these tokens then added back to $x_{i,j}$ in the next part?\n- The authors state that the multi-scale kernels are only applied to the relevant neighborhood. Do you then extract each of these and use zero-padding to perform the convolution (e.g. in regions where there are less than the kernel scale electrodes)?\n\nThe attention mechanism of the inter-window attention seems a bit arbitrary.\n- What is the justification for relative indices being grouped together? As I see it, these are not necessarily similar unless there is a periodicity of the size of the window in the signal.\n- Why not use the same token descriptor method as in inter-region attention? Does it perform worse?\n\n### Questions regarding the spatial assignment into brain regions\nIt would be helpful if the authors provided the brain region assignment in the 10-20 system, and justify why the chosen assignment is chosen as it is.\n- What is the chosen assignment? Is it based on prior works?\n- Why have the authors chosen this assignment?\n- Have the authors experimented with other assignments?\n\n### References\nSzegedy, Christian, et al. \"Going deeper with convolutions.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.\n\nWang, Zhe, et al. \"Transformers for EEG-based emotion recognition: A hierarchical spatial information learning model.\" IEEE Sensors Journal 22.5 (2022).", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- The paper is well written and technically sound with only minor spelling mistakes.\n- The proposed method is thoroughly justified and the claims are comprehensively evaluated.\n- The present paper tackles a timely and significant problem in EEG research paving the way for a generalized framework for EEG representation.\n- Importantly, using natural inductive biases yields a more structured, cost‐effective model with no loss in performance, a prospect that is considerably valuable for future directions in EEG modeling research.\n### Weaknesses\n- All of the TUEx datasets (TUEV, TUEB, TUSL) are subsets of the TUEG dataset used for pretraining. There is thus data contamination in the evaluation pipeline since some of the evaluation datasets are also part of the pretraining set. The results on these are still valuable, but I think the authors should note and discuss this, and possibly keep them separate in the results Table 2. (Also leaving it out of the macro average for the same reason).\n- It is unclear which regions are grouped together. The authors state that the grouping is based on \"anatomically defined brain regions\" and that each region comprises \"spatially adjacent electrodes\", but the assignment of electrodes into regions is not specified.\n- Although limited to spatial neighborhoods in the present paper, the idea of using multiple kernel sizes has already been explored in prior work (Szegedy et al.), where they also indeed used kernels of size [1, 3, 5] for each layer in the hierarchical model. This prior work would serve as a good reference for the importance of multi-scale processing.\n- It is unclear from the main manuscript precisely how the present work differs from previous state-of-the-art foundation models for EEG.", "was_revised": true, "final_justification": "The paper presents a technically sound and well-motivated EEG foundation model that leverages cross-scale spatiotemporal tokenization and structured sparse attention to achieve strong performance across diverse EEG decoding tasks. The rebuttal and discussion convincingly addressed all major concerns. In particular, the authors provided TUEx-excluded macro-average results and additional kernel configuration experiments that clearly supported their claims that the performance gains come from principled multi-scale modeling rather than increased model capacity. Based on the strength of the technical contribution, thorough evaluation, and the clear evidence provided in response to my concerns, I recommend acceptance.", "rebuttal": "Dear Reviewer 6YFP,\n\nThank you for your encouraging feedback and constructive suggestions! We are greatly encouraged by your recognition that **the paper tackles a timely and significant problem in EEG, paving the way for a generalized framework for EEG representation, and is considerably valuable for future directions in EEG research**. We are also pleased that you found **our proposed method thoroughly justified** and **technically sound**. Due to time and page limitations, we have made every effort to address your questions and suggestions through the clarifications provided. We hope these responses resolve most of your concerns and will be considered in your final assessment.\n\n> W1. About the TUEx Datasets\n\nWe clarify that our pretraining task, masked signal reconstruction, is purely unsupervised, involving no downstream task labels. Thus, there is no label leakage. The aim is to learn general spatiotemporal EEG representations. In prior foundation model work, such as [NeurIPS2024] BIOT and [ICLR2025] Cbramod, TUEx datasets were also used for pretraining, and they reported fine-tuned results on TUEV and TUAB datasets. To further enhance transparency, we will explicitly mark the TUEx results and compute an additional macro-average excluding TUEx datasets in Table 2. On this new macro-average, CSBrain achieves 0.6829, outperforming CBraMod (0.6537) and LaBraM (0.6430), consistently surpassing competitors by 3+%.\n\n> W2&Q3. Spatial Assignment into Brain Regions\n\nOur fundamental goal is to develop a generalizable EEG foundation model applicable to diverse EEG tasks. To ensure broader applicability, the spatial region assignment adheres to the standard 10-20 system compatible with all used datasets. In this system, electrode labels serve as proxies for anatomical brain regions, and electrodes are mapped to five primary regions: Frontal (F), Central (C), Parietal (P), Temporal (T), and Occipital (O). This information was briefly introduced in Sec D.1 (L 727-728). To further enhance readability, we will expand this explanation into a dedicated subsection (e.g., C.5) in the supp. mat., detailing electrode assignments for each dataset. We will also release the code and model to support community use of our work.\n\n> W3. Classic Work on Multi-Scale Modeling in CV\n\nWe appreciate your insightful suggestion regarding Szegedy et al. as a key early work that introduced multi-scale convolutions in the CV domain. We will gladly include this classic work as a reference in the main paper's Introduction (L76-80) to further elaborate on the significance of multi-scale modeling.\n\n> W4. Differences to Prior Works\n\nAs shown in Fig. 1 of the main paper, CSBrain fundamentally differs from prior EEG foundation models by explicitly addressing and leveraging the intrinsic cross-scale spatiotemporal structure of neural activity, a property often overlooked by previous scale-agnostic dense modeling paradigms.\n- **Difference in Tokenization:** Prior models typically rely on one-time, fixed-scale tokenization strategy, yielding token representations that struggle with diverse neural patterns across tasks. In contrast, our proposed CST aggregates multi-scale features within localized temporal windows and anatomical brain regions via various convolution kernels. This produces compact, scale-aware neural tokens, enabling adaptable alignment with diverse spatiotemporal requirements of EEG decoding tasks.\n- **Difference in Spatio-Temporal Modeling:** Previous works commonly adopt structure-agnostic attention (e.g., dense attention or Criss-Cross Attention) for spatio-temporal modeling, which can introduce redundancies and computational overhead. In contrast, our proposed SSA models cross-window and cross-region dependencies. This structure-aware design enhances scale diversity, reduces spurious dependencies, and lowers computational overhead, yielding more discriminative and efficient representations.\n- **Hierarchical Architecture:** Unlike prior works, CSBrain establishes a hierarchical architecture by alternately stacking CST and SSA, enabling synergistic enhancement and robust EEG representations that capture complex neural patterns. This leads to strong generalization across diverse EEG decoding tasks, as evidenced by significant performance gains on 16 datasets.\n\n\nWe greatly appreciate your suggestion and will revise the Introduction by adding a separate paragraph after Line 88 to more clearly emphasize the differences between our approach and prior state-of-the-art EEG foundation models.\n\n> Q1. Comparison of Number of Kernels\n- **Computational Cost Comparison**: For kernel configurations K=1, K=2, and K=3, parameters are 4.88M, 4.92M, 4.94M respectively, and FLOPs are 246.22M, 265.42M, 275.02M. The parameter count increases only slightly, and FLOPs show a modest increase from K=1 to K=3. This is because we keep the total feature embedding dimension (=200) constant via an exponentially decaying scheme for individual kernel dimensions, balancing representational capacity and efficiency (main paper L168-172). Despite these constrained increases, our model demonstrates significantly improved performance at K=3 across multiple tasks (main paper Fig. 4, L248-253). This outcome strongly supports the effectiveness of cross-scale spatiotemporal modeling, proving performance gains stem from respecting EEG's intrinsic cross-scale properties, not solely from capacity increase.\n\n- **Ablation with Additional Kernel Variants**: To address your concerns further, we added experiments with the kernel configurations [1,1,1], [3,3,3], and [5,5,5], as well as a baseline with [3,3,3]+Dense Attention. Due to time and computational constraints, these experiments were performed using a subset of the TUEG dataset (2000 hours, excluding TUEV) and evaluated on two downstream tasks:\n|Kernel Config|TUEV B-Acc|TUEV Kappa|TUEV F1-W|BCIC2020-3 B-Acc|BCIC2020-3 Kappa|BCIC2020-3 F1-W|\n|-|-|-|-|-|-|-|\n|[1,3,5]|0.6414±0.0187|0.6236±0.0198|0.7971±0.0183|0.5443±0.0232|0.4254±0.0301|0.5527±0.0297|\n|[1,1,1]|0.6122±0.0193|0.5596±0.0212|0.7713±0.0205|0.5041±0.0208|0.3821±0.0223|0.5019±0.0202|\n|[3,3,3]|0.6338±0.0184|0.6137±0.0179|0.7870±0.0188|0.5298±0.0247|0.4354±0.0298|0.5399±0.0280|\n|[5,5,5]|0.6188±0.0190|0.5765±0.0209|0.7732±0.0195|0.4998±0.0237|0.3809±0.0242|0.4931±0.0279|\n|[3,3,3]+Dense Attention|0.5901±0.0178|0.5380±0.0199|0.7531±0.0212|0.5076±0.0221|0.3844±0.0269|0.5023±0.0222|\n\nThese results further validate the effectiveness of CST for cross-scale modeling, where the [1,3,5] configuration achieves the best performance, while [3,3,3] provides the second-best results. This demonstrates that explicitly modeling diverse scales is more effective than simply adding more filters of the same scale. This comprehensive analysis reinforces that CSBrain's performance gains are indeed attributable to its principled cross-scale spatiotemporal modeling. *We will update Supp. Sec. E to reflect these insights.*\n\n> Q2. CST Procedure\n\n**Q2-1. Replacement of intra-region/window attention**:\n- We considered replacing CST's convolutions with intra-region/window attention modules during the early stages of our project. However, preliminary experiments showed negligible performance improvements (~1% on key metrics for Seed-V, TUEV) while substantially increasing parameters due to doubled attention computations.\n- We will gladly cite and discuss the classic work by Wang et al. in the Introduction and Related work. \n- We also explored Swin-style models early in our project. However, we found that their shifted window partitioning for efficient computation (as shown in Fig. 4 of their paper) would likely disrupt the temporal and spatial order inherent in EEG signals. While highly effective for images, transference of such mechanisms to EEG data poses fundamental challenges, as preserving sequential and local spatial dependencies is paramount. *We will update the manuscript to reflect these considerations more explicitly.*\n\n**Q2-2. Inter-region Attention**:\n- To clarify, as stated in L193, tokens are sequentially sampled from each region, not randomly. In Supp.E.3, we compare four strategies for constructing Regional Descriptors, including random sampling. Results show that sequential sampling with the average feature gives the best performance, ensuring each token represents its region’s spatial context.\n- In inter-region attention, token count is not reduced. As in Equation 5, each position $x_{i,j}$ is assigned to a spatial group for attention, ensuring that no information is lost during sampling. This design preserves the spatial and temporal relationships between regions and guarantees that tokens flow consistently throughout the model. *We will rewrite L191-198 for further clarity.*\n\n**Q2-3. Zero-padding**: Yes. For regions with fewer electrodes than the kernel size, we apply zero-padding during the convolution operation.\n\n**Q2-4. Inter-window Attention**: \n- In inter-window attention, each token is enriched with information from other tokens in the same window via CST, ensuring tokens share a common spatiotemporal context. This allows relative indices to be grouped together, as windowed tokens have a consistent temporal structure. \n- In contrast, for inter-region attention, varying electrode counts prevent tokens from fully representing their regions. Therefore, we use token descriptors, adding the average feature of the region to each token. \n- Our preliminary experiments showed that adding a token descriptor to windowed tokens did not significantly impact performance (~<1% difference).\n\n*Due to rebuttal space constraints, some brief experimental results will be included in revised versions of the paper. We will expand L191-198 in the main paper to clarify the specific implementation details and design motivations for SSA. Furthermore, we will open-source the relevant code to facilitate community use of our work.*"}]

Author rebuttal:

Meta-review: The authors present CSBrain, a foundation model for EEG decoding that introduces cross-scale spatiotemporal tokenization (CST) and structured sparse attention (SSA) to capture the multi-scale, anatomically grounded dynamics of neural activity. CST aggregates temporal and spatial features across multiple scales into compact tokens, while SSA efficiently models long-range dependencies across time windows and brain regions. By stacking these modules, CSBrain progressively integrates diverse cross-scale structures, addressing a key limitation of existing models that rely on scale-agnostic tokenization and dense attention. Pretrained on ~9,000 hours of EEG and extensively evaluated across several public datasets, CSBrain consistently outperforms task-specific architectures and foundation baselines. Extensive ablations further validate the effectiveness of cross-scale modeling and structured sparse attention, establishing them as crucial inductive biases for generalized EEG decoding. The findings are consistent with prior works.

The work features clear and well-structured writing, presenting a logical flow of ideas alongside detailed methodological descriptions that enhance reproducibility. It is strongly motivated by the need for cross-scale modeling, effectively addressing a major limitation in existing EEG analysis approaches. The paper introduces novel architectural contributions, specifically the CST and SSA modules, which embed natural inductive biases to better capture EEG characteristics. Its performance is validated through a comprehensive evaluation across 11 tasks and 16 datasets, consistently outperforming baseline methods. Overall, the work demonstrates strong generalizability, computational efficiency, and holds substantial potential to impact future EEG research.

There are some concerns regarding the design of the analysis for model pretraining, which could be further refined to strengthen the validity of the conclusions. Additionally, minor issues were noted related to the feature fusion approach, suggesting that alternative strategies might improve integration. There are also minor concerns about the interpretability of the model, indicating a need for clearer explanations.

The discussion and interaction between the authors and reviewers were very productive. This process enabled several improvements to the work, including: (i) an additional ablation study, (ii) a runtime analysis of the SSA module, (iii) clarification of the CST method, and (iv) more comprehensive literature references.
Decision: Accept (spotlight)