Title: CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Venue: NeurIPS 2025
Accepted: yes
Keywords: Brain Foundation Model

Cross-scale

Generalized EEG Decoding
Abstract: Understanding and decoding human brain activity from electroencephalography (EEG) signals is a fundamental problem in neuroscience and artificial intelligence, with applications ranging from cognition and emotion recognition to clinical diagnosis and brain–computer interfaces. While recent EEG foundation models have made progress in generalized brain decoding by leveraging unified architectures and large-scale pretraining, they inherit a scale-agnostic dense modeling paradigm from NLP and vision. This design overlooks an intrinsic property of neural activity—cross-scale spatiotemporal structure. Different EEG task patterns span a broad range of temporal and spatial scales, from brief neural activations to slow-varying rhythms, and from localized cortical activations to large-scale distributed interactions. Ignoring this diversity may lead to suboptimal representations and weakened generalization ability. To address these limitations, we propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain introduces two key components: (i) Cross-scale Spatiotemporal Tokenization (CST), which aggregates multi-scale features within localized temporal windows and anatomical brain regions into compact scale-aware token representations; and (ii) Structured Sparse Attention (SSA), which models cross-window and cross-region dependencies for diverse decoding tasks, further enriching scale diversities while eliminating the spurious dependencies. CST and SSA are alternately stacked to progressively integrate cross-scale spatiotemporal dependencies. Extensive experiments across 11 representative EEG tasks and 16 datasets demonstrate that CSBrain consistently outperforms both task-specific models and strong foundation baselines. These results establish cross-scale modeling as a key inductive bias for generalized EEG decoding and highlight CSBrain as a robust backbone for future brain–AI research.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces CSBrain: a foundation model used in the electroencephalography signals (EEG) domain of functional neuroimaging. The authors combine cross-scale spatiotemporal tokenization (CST) with structured sparse attention (SSA) to capture neural patterns on multiple scales and with sparsity in a hierarchical framework. Since EEG tasks exhibit signals at varying scales, this stacked cross-scale spatiotemporal structure is used to capture this diversity which is important for building a foundation model that can work in many settings and environments.\n\nThe introduced CST module starts by applying convolutions with varying kernel sizes across a temporal neighborhood which are then concatenated across scales to yield a temporal token. The temporal tokens are then split into localized spatial neighborhoods based on spatial adjacency in the 10-20 system. Spatial convolutions with varying kernels sizes are then applied to these neighborhoods and concatenated as before to yield the spatiotemporal tokens as output.\n\nThe introduced SSA module takes the spatiotemporal tokens and applies attention in two rounds. First, inter-window attention is applied between tokens which share the same relative index in the chosen window neighborhoods of the temporal convolutions. Second, inter-region attention is performed where each region is summarized through a region descriptor using a projection of the average token together with a randomly sampled token, then attention is applied across all of the regions' descriptor tokens.\n\nSelf-supervised learning is performed using masked reconstruction loss where whole regions of the input are erased for the model to reconstruct. The model is pre-trained on the TUEG dataset.\n\nThe authors present a very comprehensive evaluation of the EEG foundation model evaluating on 16 EEG datasets with 11 different BCI tasks against 10 baseline models. Baseline models are reimplemented and retrained where results are not available on the specific dataset. The proposed model achieves on-par or better fine-tuning performance on all downstream tasks. An ablation study is applied to the number of kernels, showing that the repeated and stacked model with three kernel sizes is the best choice of architecture. An ablation study on the chosen attention method is performed against dense attention and criss-cross attention which showed that the proposed SSA performs well across datasets.\n\nThe authors use Grad-CAM to explore the changes of model prediction by variation in input EEG channels, yielding findings consistent with prior works.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The paper is well written and technically sound with only minor spelling mistakes.\n- The proposed method is thoroughly justified and the claims are comprehensively evaluated.\n- The present paper tackles a timely and significant problem in EEG research paving the way for a generalized framework for EEG representation.\n- Importantly, using natural inductive biases yields a more structured, cost‐effective model with no loss in performance, a prospect that is considerably valuable for future directions in EEG modeling research.\n### Weaknesses\n- All of the TUEx datasets (TUEV, TUEB, TUSL) are subsets of the TUEG dataset used for pretraining. There is thus data contamination in the evaluation pipeline since some of the evaluation datasets are also part of the pretraining set. The results on these are still valuable, but I think the authors should note and discuss this, and possibly keep them separate in the results Table 2. (Also leaving it out of the macro average for the same reason).\n- It is unclear which regions are grouped together. The authors state that the grouping is based on \"anatomically defined brain regions\" and that each region comprises \"spatially adjacent electrodes\", but the assignment of electrodes into regions is not specified.\n- Although limited to spatial neighborhoods in the present paper, the idea of using multiple kernel sizes has already been explored in prior work (Szegedy et al.), where they also indeed used kernels of size [1, 3, 5] for each layer in the hierarchical model. This prior work would serve as a good reference for the importance of multi-scale processing.\n- It is unclear from the main manuscript precisely how the present work differs from previous state-of-the-art foundation models for EEG.\n\nQUESTIONS:\n### Questions regarding the number of kernels comparison\nFor better comparison, the number of parameters and/or compute should be kept constant between ablations in the number of kernels comparison.\n- Can the authors provide insight into the number of parameters and FLOPS used in this part of the study?\n- Is it just an increase in compute or parameters that gives the perceived performance when increasing the number of kernels used? How much of the performance increase is because of a stronger model (more filters, parameters etc.)?\n- Have the authors experimented with using multiple kernels but keeping the scale the same? Using three kernels of the form [3, 3, 3], for example? Here we are adding more parameters (more kernels) but forgoing the multiple scales.\n- What about [3, 3, 3] + Dense Attention? This would seem like the naïve baseline.\n\n### Questions for the cross-scale spatiotemporal tokenization procedure\nThe inter-region self attention has been proposed before (in Wang et al.), and the proposed alternating stacks of CST and SSA follows the conventional methodology of hierarchical models.\nUsually, in hierarchical transformer models, instead of a convolution, attention is applied intra-regionally and then inter-regionally, e.g. in SWIN transformers.\n- Have the authors considered a model where an intra- region/window attention module replaces the convolutions?\n\nIt is a bit unclear how inter-region attention provides inputs to the next stack in the model. The inter-window attention mechanism makes sense since the attention is just limited using a periodic mask. However, in the inter-region attention, after pooling and randomly sampling tokens from the region, I assume the signal is reduced to having only $R_r$ tokens in total over which the attention is performed.\n- How are these tokens then added back to $x_{i,j}$ in the next part?\n- The authors state that the multi-scale kernels are only applied to the relevant neighborhood. Do you then extract each of these and use zero-padding to perform the convolution (e.g. in regions where there are less than the kernel scale electrodes)?\n\nThe attention mechanism of the inter-window attention seems a bit arbitrary.\n- What is the justification for relative indices being grouped together? As I see it, these are not necessarily similar unless there is a periodicity of the size of the window in the signal.\n- Why not use the same token descriptor method as in inter-region attention? Does it perform worse?\n\n### Questions regarding the spatial assignment into brain regions\nIt would be helpful if the authors provided the brain region assignment in the 10-20 system, and justify why the chosen assignment is chosen as it is.\n- What is the chosen assignment? Is it based on prior works?\n- Why have the authors chosen this assignment?\n- Have the authors experimented with other assignments?\n\n### References\nSzegedy, Christian, et al. \"Going deeper with convolutions.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.\n\nWang, Zhe, et al. \"Transformers for EEG-based emotion recognition: A hierarchical spatial information learning model.\" IEEE Sensors Journal 22.5 (2022).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-26"}

comments: SUMMARY:
The paper argues that cross-scale modeling is lacking from LLM-inspired brain models. Introduces cross-scale spatiotemporal tokenization and structured sparse attention. Problems with current models include scale-agnostic tokenization and dense attention.

STRENGTHS AND WEAKNESSES:
### Strength
- large pre-training dataset
- very comprehensive downstream evaluation

### Weaknesses
I think the term tokenization is misleading, as these are simply convolutional layers applied between attention layers.
- Temporal tokenization is really just a stack of convolutions
- Spatial tokenization is just standard convolution, but nearby regions are correlated anyway so I don't see what's the point.
- The masked learning uses MSE which has fundamental issues, such as focusing on low-frequency features.
- Is the pre-training amount used for the other foundation models similar? Otherwise comparison might not be fair
- Lack of comparisons with other foundational models in terms of evaluating why yours is better. Not just downstream performance.
- No direct evaluation of the pre-trained model itself

QUESTIONS:
In section 2.1 you mention FFT for spectral information, why not use learned 1D convolutions, aren’t those enough?

SSA: How is this different from combining temporal and spatial attention layers?

Kernel stacking comparison: Does this mean than even more scales would increase performance?

More comparisons with other foundational models on the pre-training task.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present CSBrain, a foundation model for EEG decoding that combines cross-scale spatiotemporal tokenization (CST) and structured sparse attention (SSA) to capture multi-scale, anatomically-aware dynamics in EEG signals. They pretrained their model on ~9,000 hours of EEG and evaluated across 11 tasks on 16 public datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.\tThe paper is well-written, with a clear structure and logical flow. Most sections are supplemented by additional material in the appendix.\n2.\tThe authors perform a comprehensive evaluation, applying CSBrain to a wide range of tasks and datasets.\n\nWeaknesses:\n\n1.\tIn Section 2.1, the choice to concatenate features into a unified embedding is not justified — why not addition or another fusion strategy?\n2.\tIn Section 2.2, the formulation of SSA lacks deeper derivation or motivation. How was this design chosen? What theoretical or empirical reasoning supports it?\n3.\tThe explanation of SSA (lines 185-203) is hard to follow — more derivation or intuition would strengthen this section.\n4.\tDetails on baseline training (e.g., whether competing foundation models were re-trained with the same objective or used published weights) are sparse, making it harder to assess comparison fairness.\n5.\tAblations focus more on SSA variations but less on CST: no analysis of token granularity choices, window/region design, or their effect on generalization.\n6.\tAs GNN architectures have shown strong performance in EEG domain for purposes such as emotion recognition and seizure detection by explicitly modeling electrode connectivity, including a GNN baseline would be valuable. For instance, GGN[1] for seizure detection, and HetEmotionNet[2] for emotion classification. Adding such baselines would provide meaningful context for evaluating CSBrain’s approach to spatial dependency modeling.\n7.\tQualitative analyses of attention maps across attention types (dense, criss-cross, SSA) are missing; this would enhance interpretability.\n8.\tThe paper provides brain maps for CSBrain but not for baselines — including these would better contextualize the model’s interpretability gains.\n\n[1] Li, Zhengdao, et al. \"Graph-generative neural network for EEG-based epileptic seizure detection via discovery of dynamic brain functional connectivity.\" Scientific reports 12.1 (2022): 18998.\n\n[2] Jia, Ziyu, et al. \"HetEmotionNet: two-stream heterogeneous graph recurrent neural network for multi-modal emotion recognition.\" Proceedings of the 29th ACM international conference on multimedia. 2021.\n\nQUESTIONS:\n1.\tWhy was concatenation chosen for feature fusion in 2.1 rather than addition or another method?\n2.\tWhat guided the specific design of SSA’s formulation? Can the authors provide deeper motivation or derivation?\n3.\tCan the authors clarify whether aggregation in 155–156 refers to functional or anatomical context? If functional, how is it designed in the model to do so?\n4.\tCould SSA’s formulation (185–203) be explained in greater depth or with additional derivation?\n5.\tCould the authors provide qualitative brain map analyses for different attention variants or for baselines?\n\nMinor: \n\nThere are minor typos, such as in line 154, where the superscript appears as t but should likely be s (referring to spatial rather than temporal context).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CSBrain, a foundation model for EEG decoding that addresses a key limitation of existing approaches: their failure to capture the cross-scale spatiotemporal structure inherent in neural activity. CSBrain introduces two main components: Cross-scale Spatiotemporal Tokenization (CST) aggregates multi-scale features within temporal windows and brain regions into compact tokens, while Structured Sparse Attention (SSA) efficiently models long-range dependencies across windows and regions. These modules are alternately stacked to progressively integrate cross-scale dependencies. The authors demonstrate CSBrain's effectiveness across 11 EEG tasks and 16 datasets, showing consistent improvements over both task-specific models and foundation baselines. The work establishes cross-scale modeling as a crucial inductive bias for generalized EEG decoding.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well-written with clear logical flow and comprehensive method descriptions. The motivation for cross-scale modeling is well-articulated, and the technical components (CST and SSA) are described with sufficient detail for understanding and reproducibility.\n2. The experimental evaluation is thorough and convincing, spanning 11 diverse EEG decoding tasks across 16 datasets with consistent improvements over both task-specific models and strong foundation baselines, demonstrating the generalizability of the approach.\n3. The paper introduces a principled architectural innovation that explicitly captures the inherent multi-scale spatiotemporal structure of neural activity, moving beyond scale-agnostic dense modeling paradigms to establish cross-scale modeling as a key inductive bias for EEG decoding.\n\nWeakness:\n\n1.\tThe paper lacks detailed explanation of how CST handles differences across temporal scales and varying channel numbers, with the cross-scale adaptation mechanism not thoroughly described in the main text.\n2. Despite proposing cross-scale modeling, CSBrain uses uniform 30s temporal windows and 19 channels during pretraining, only encountering scale diversity in downstream tasks, which may limit learning of truly scale-invariant representations.\n3. The methodology for selecting 19 channels from the Temple University Hospital EEG corpus with various channel configurations is not clearly described, affecting reproducibility and understanding of generalization capabilities.\n\nQUESTIONS:\n1.\tHow does CST specifically handle varying temporal scales and channel numbers across datasets? Please provide technical details on the adaptation mechanism for different sampling rates and electrode configurations.\n2.\tWhy use uniform 30s windows and 19 channels during pretraining if cross-scale modeling is key? Please justify this design choice and provide ablation studies on scale diversity impact.\n3.\tHow are 19 channels selected from the Temple University Hospital corpus with various configurations? How does the model handle missing/additional channels in practice? \n4.\tPlease provide detailed complexity analysis and runtime comparisons for SSA versus dense attention across different temporal lengths and channel numbers.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}