Title: REVE: A Foundation Model for EEG - Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects
Venue: NeurIPS 2025
Accepted: yes
Keywords: Foundation Model

EEG

SSL

BCI
Abstract: Foundation models have transformed AI by reducing reliance on task-specific data through large-scale pretraining. While successful in language and vision, their adoption in EEG has lagged due to the heterogeneity of public datasets, which are collected under varying protocols, devices, and electrode configurations. Existing EEG foundation models struggle to generalize across these variations, often restricting pretraining to a single setup, resulting in suboptimal performance, in particular under linear probing.
We present REVE (Representation for EEG with Versatile Embeddings), a pretrained model explicitly designed to generalize across diverse EEG signals. REVE introduces a novel 4D positional encoding scheme that enables it to process signals of arbitrary length and electrode arrangement. Using a masked autoencoding objective, we pretrain REVE on over 60,000 hours of EEG data from 92 datasets spanning 25,000 subjects, representing the largest EEG pretraining effort to date.
REVE achieves state-of-the-art results on 10 downstream EEG tasks, including motor imagery classification, seizure detection, sleep staging, cognitive load estimation, and emotion recognition. With little to no fine-tuning, it demonstrates strong generalization, and nuanced spatio-temporal modeling. We release code, pretrained weights, and tutorials to support standardized EEG research and accelerate progress in clinical neuroscience.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the limited generalization of existing EEG foundation models across diverse datasets, especially under linear probing. The authors propose REVE, a new pretrained model with a novel 4D positional encoding scheme that enables flexible handling of variable-length signals and electrode configurations. Trained on a large-scale EEG corpus, REVE achieves strong performance across multiple downstream tasks with minimal fine-tuning, offering a promising step toward more robust and transferable EEG representations.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths\n\n- The scale of the pretraining dataset is truly impressive, covering over 60,000 hours of EEG data from 92 different datasets and 25,000 subjects. This kind of breadth has not been seen before in the field of EEG foundation models.\n\n- The proposed 4D positional encoding scheme introduces a fresh idea for handling EEG signals with varying electrode configurations and lengths. It’s a thoughtful design that could help future models better adapt to real-world EEG setups.\n\n- The authors evaluated their model across a wide range of downstream tasks — including motor imagery, seizure detection, sleep staging, emotion recognition, and more — showing both broad coverage and solid performance across the board.\n\n- The results are consistently strong, often outperforming existing methods by a meaningful margin. Especially under linear probing or minimal fine-tuning settings, REVE still delivers competitive accuracy, which speaks well of its generalization capability.\n\n\n#### Weaknesses\n\n- Although the paper claims strong performance under linear probing, this is mostly supported by experiments on PhysioNet-MI. That’s just one dataset, and it’s hard to draw general conclusions from such limited evidence. More cross-dataset comparisons would make this claim much stronger.\n\n- Looking at Table 17, it seems like the secondary loss might actually be the key factor behind REVE’s impressive linear probing performance. However, the authors don’t explain what this loss does or why it helps. It feels underemphasized, especially given how important it appears to be. I’d strongly suggest expanding on this part and running more ablation studies to back it up.\n\n- While the empirical results are convincing, the paper lacks deeper analysis. There’s no visualization or interpretability study showing what the model learns. Also, the advantage of the 4D positional encoding over standard approaches isn’t clearly demonstrated through ablation experiments.\n\n- Some practical details are missing — for example, how many devices were used during training, or how long the pretraining actually took. These are important for reproducibility and understanding the computational cost. There are also some citation formatting issues that should be cleaned up.\n\nQUESTIONS:\n- The paper adopts a Transformer-based architecture with both encoder and decoder, but it’s not entirely clear why the decoder is needed. From what I could tell, the model doesn’t seem to involve any autoregressive training — so what role does the decoder play in the overall design? A clearer explanation would help readers understand its purpose.\n- The authors borrow several training techniques from large language models, such as RMSNorm and GEGLU activation. While these components are well-established in NLP, it’s less clear whether they bring real benefits to EEG foundation model training compared to more traditional choices like LayerNorm or GELU. Some ablation or discussion on this would be helpful.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces REVE (Representation for EEG with Versatile Embeddings), a foundation model for electroencephalogram (EEG) designed to address the heterogeneity of EEG datasets by enabling generalization across diverse recording setups, electrode configurations, and task types. The core challenge addressed is the limited generalizability of existing EEG foundation models, which often rely on homogeneous datasets and fixed positional encodings. Key Contributions:\n\nNovel 4D Positional Encoding: REVE uses a Fourier-based 4D encoding that integrates spatial (3D electrode coordinates) and temporal (patch indices) dimensions, allowing flexibility in processing signals of arbitrary length and electrode arrangements. This addresses the rigidity of prior absolute or convolutional encodings.\n\nLarge-Scale Heterogeneous Pretraining: REVE is pretrained on 60,000+ hours of EEG data from 92 datasets spanning 25,000 subjects, the largest such effort to date. This diversity enhances generalization to unseen setups.\n\nSpatio-Temporal Block Masking: A joint masking strategy disrupts contiguous regions in both spatial and temporal dimensions during pretraining, improving representation learning for robust feature extraction.\n\nState-of-the-Art Performance: REVE achieves superior results on 10 downstream tasks, including motor imagery classification, seizure detection, and sleep staging, outperforming prior models like CBraMod and LaBraM.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nQuality:\n\nThe experimental design is rigorous, with comprehensive evaluations across diverse tasks and datasets. The use of linear probing and fine-tuning protocols ensures fairness against baselines.\nThe large pretraining corpus and clear ablation studies (e.g., masking ratio, secondary loss impact) validate the method’s robustness.\n\nClarity:\n\nThe paper is well-structured, with detailed explanations of the 4D encoding, masking strategy, and training pipeline. Technical details (e.g., hyperparameters, dataset composition) are provided in appendices.\nVisualizations (e.g., pretraining framework diagram) and tables (performance metrics) aid comprehension.\n\nSignificance:\n\nREVE advances EEG foundation modeling by addressing long-standing challenges in data heterogeneity, which could accelerate progress in brain-computer interfaces (BCIs) and clinical diagnostics.\nThe focus on generalizable representations reduces reliance on task-specific fine-tuning, making it accessible for low-data scenarios.\n\nOriginality:\n\nThe 4D positional encoding is a novel approach for EEG, enabling true cross-setup generalization. The scale of pretraining and integration of spatial-temporal masking are also innovative.\n\nWeaknesses\n\nQuality:\n\nWhile the dataset is large, the preprocessing (e.g., z-score normalization, clipping) might introduce biases. The impact of data cleaning on downstream tasks is not fully explored.\nSome ablation results (e.g., Table 18) show inconsistent performance across masking ratios, leaving questions about optimal configurations for specific tasks.\n\nClarity:\n\nThe 4D encoding’s mathematical derivation (Section 2.2) could be more transparent, especially regarding how Fourier frequencies are selected and truncated to match embedding dimensions.\nThe secondary loss function’s design (attention pooling across layers) is briefly described; more intuition on its effectiveness in preventing over-specialization is needed.\n\nSignificance:\n\nThe clinical impact is implied but not directly validated (e.g., case studies in real-world diagnostics). The model’s utility in resource-constrained settings remains unaddressed.\n\nOriginality:\n\nWhile the 4D encoding is novel, the use of masked autoencoding (MAE) follows prior work in vision. The contribution lies more in adaptation than radical architectural innovation.\n\nQUESTIONS:\nQuestion1: How does the 4D encoding perform with extremely sparse electrode setups (e.g., <4 channels) or non-standard montages not seen in pretraining? Are there limitations in capturing fine-grained spatial patterns with low channel counts?\n\n\nQuestion2: The pretraining corpus includes clinical, BCI, and cognitive data, but how is data quality ensured? Are there biases in dataset selection (e.g., overrepresentation of certain demographics)?\n\nQuestion3: How does REVE perform in zero-shot or few-shot scenarios, where minimal task-specific data is available? The current evaluation focuses on fine-tuning, but real-world applications often have limited labels.\n\n\nQuestion4: Does the 4D encoding incur higher computational costs than prior methods? How does it scale with increasing channel counts and signal length?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nREVE is a foundation model for EEG data that generalizes across diverse setups. It uses 4D positional encoding and was pretrained on 60,000+ hours of EEG data from 92 datasets (25,000 subjects). The model achieves SOTA performance on 10 tasks, including motor imagery, seizure detection, and emotion recognition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper demonstrates outstanding model performance and achieves state-of-the-art results in 10 different downstream EEG tasks. The paper provides particularly detailed experimental methodology, including thorough implementation specifics that ensure transparency. Importantly, the author ensured the repeatability of the model by publicly releasing all the code and pre-trained weights.\nWeaknesses: While the paper emphasizes REVE’s advantages in linear probing and limited fine-tuning scenarios, its core experimental design exhibits notable gaps. Specifically: \n1.\tAbstract (lines 5-7) critiques existing EEG foundation models for their poor linear probing performance, and (lines 15-16) claim REVE’s strong generalization with “little to no fine-tuning.”\n2.\tThe introduction (lines 41-42) further argues that current models require “full fine-tuning” for transfer.\n3.\tResults and Discussion (lines 319-322) highlight REVE’s “key advantage” in “producing high-quality latent spaces without heavy fine-tuning”.\nYet, the primary results (Table 2) rely on a two-stage fine-tuning strategy (Section 3.3 Downstream Tasks). Only limited validation for the above statement is provided (“w/PT” and “Frozen” settings in Table 3), tested on just one dataset, failing to generalize the “lightweight fine-tuning” advantage.\nThis inconsistency weakens the paper’s argument for REVE’s “lightweight fine-tuning benefits”, potentially undermining the conclusions’ credibility. To substantiate this argument, comparisons of all pre-trained baselines under linear probing across all downstream datasets are essential—a gap the current experiments leave unaddressed. Such validation is critical: if pre-trained models still require two full fine-tuning stages despite their pre-training, REVE’s position in the performance-simplicity tradeoff becomes significantly less competitive.\n\nQUESTIONS:\n1. The introduction (lines 37-39) criticizes existing models for issues such as being trained solely on the TUH dataset, leading to poor generalization to EEG data with different electrode layouts. However, among the pretrained models cited in lines 35-36, none actually suffer from this limitation—for example, CBraMod, despite being trained only on TUH dataset, has been successfully applied to downstream tasks with varying electrode configurations. Could the authors clarify the basis for this claim? \n2. Did the pretrained baseline models use the same two-stage fine-tuning strategy as the proposed method in this paper? If the results were directly taken from the original baseline papers without adopting the two-stage fine-tuning strategy, such a comparison might be unfair. \n3. As mentioned in the weaknesses, could the author provide more results of the linear probing experiment to prove the performance of REVE under lightweight finetuning?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents REVE a foundation model trained on over 60,000 hours of EEG data from 92 datasets and 25,000 subjects. REVE achieves SOTA performance on 10 downstream EEG tasks. REVE addresses several challenges in training an EEG foundation model such as data heterogenity, low SNR, and different electrode configurations. The main contributions of this paper are\n\n1. A novel 4D positional encoding to model EEG signals with varying temporal lengths and electrode configuration\n2. Curation of large scale EEG by combining 92 datasets.\n3. State-of-the-art performance on multiple benchmarks\n\nSTRENGTHS AND WEAKNESSES:\n1. This paper has created a large scale EEG dataset by combining 92 datasets spanning data from clinical, BCI and research domains. This will be a valuable resource for foundation model research in EEG\n2. This paper adapts MAE to spatiotemporal EEG signal and proposes a 4D positional encoding to address variation in temporal length and electrode configuration across datasets.\n3. Strong results demonstrating the importance of large scale pretraining on multiple downstream tasks.\n4. Clear writing , description of methods and results. \n5. Modeling choices are inspired by successes in different modalities \n\nI do not see any major weaknesses in this paper and believe it is a strong contribution towards foundation model development in EEG Domain. A minor weakness of this paper is that it is leveraging the ideas from other modalities but is missing an ablation study on the design choices for the architecture.\n\nQUESTIONS:\nWhat other architecture design choices did you consider? Were there any other successful or failed designs that are not mentioned in the paper. It would be great to mention failed designs as that can help guide future directions\n\nDid you have an ablation on performance vs scale to find out whether there is room for improvement with more data or we have reached a saturation in performance with this dataset\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-07-03"}