Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
Authors: Na Lee
Konstantinos Barmpas
Yannis Panagakis
Dimitrios Adamos
Nikolaos Laskaris
Stefanos Zafeiriou
Venue: ICML 2025 poster (2025)
Keywords: Foundation Models
Large Brainwave Foundation Models
Brain-Computer Interface (BCI)
Low-Rank Adaptation (LoRA)
Electroencephalogram (EEG)
Primary area: deep_learning->foundation_models

Abstract: Foundation Models have demonstrated significant success across various domains in Artificial Intelligence (AI), yet their capabilities for brainwave modeling remain unclear. In this paper, we comprehensively evaluate current Large Brainwave Foundation Models (LBMs) through systematic fine-tuning experiments across multiple Brain-Computer Interface (BCI) benchmark tasks, including memory tasks and sleep stage classification. Our extensive analysis shows that state-of-the-art LBMs achieve only marginal improvements (0.5\%) over traditional deep architectures while requiring significantly more parameters (millions vs thousands), raising important questions about their efficiency and applicability in BCI contexts. Moreover, through detailed ablation studies and Low-Rank Adaptation (LoRA), we significantly reduce trainable parameters without performance degradation, while demonstrating that architectural and training inefficiencies limit LBMs' current capabilities. Our experiments span both full model fine-tuning and parameter-efficient adaptation techniques, providing insights into optimal training strategies for BCI applications. We pioneer the application of LoRA to LBMs, revealing that performance benefits generally emerge when adapting multiple neural network components simultaneously. These findings highlight the critical need for domain-specific development strategies to advance LBMs, suggesting that current architectures may require  redesign to fully leverage the potential of foundation models in brainwave analysis.

The advent of deep learning has made the need for man- ual feature extraction redundant, new data-driven ap- proaches led to state-of-the-art performance in various BCI paradigms [(Lawhern et al., 2018), (Santamaría- Váziizz et al., 2020), (Song et al., 2023), (Barmpas et al., 2023), (Bakas et al., 2022), (Wei et al., 2022)]. Although deep learning models have demonstrated impressive results, they generally demand substantial supervision and task-specific and Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning
Na 11 Konstantinos Barmpas * 123 Yannis Panagakis 234 Dimitrios Adamos 12
12
Nikolaos Laskaris 25
Stefanos Zafeiriou 12
Abstract
1. Introduction
Foundation Models have demonstrated signifi- cant success across various domains in Artifi- cial Intelligence (AI), yet their capabilities for brainwave modeling remain unclear. In this pa- per, we comprehensively evaluate current Large Brainwave Foundation Models (LBMs) through systematic fine-tuning experiments across multi- ple Brain-Computer Interface (BCI) benchmark tasks, including memory tasks and sleep stage classification. Our extensive analysis shows that state-of-the-an LBMs achieve only marginal im- provements 1.2%% over traditional deep architectures while requiring significantly more parameters (millions vs thousands), raising im- portant questions about their efficiency and ap- plicability in BCI contexts. Moreover, through detailed ablation studies and Low-Rank Adapta- tion (LoRA), we significantly reduce trainable pa- rameters without performance degradation, while demonstrating that architectural and training inef- ficiencies limit LBMs' current capabilities. Our experiments span both full model fine-tuning and parameter-efffcient adaptation techniques, provid- ing insights into optimal training strategies for BCI applications. We pioneer the application of LoRA to LBMs, revealing that performance bene- fits generally emerge when adapting multiple neu- network components simultaneously. These findings highlight the critical need for domain- specific development strategies to advance LBMs, suggesting that current architectures may require redesign to fully leverage the potential of founda- tion models in brainwave analysis.
Brain- Coain-Computer Interface (BCI) technology promises a new way to interact with machines by creating direct commu- nication between the human brain and computers. This technology is based on the analysis of brainwaves from elec- troencephalogram (EEG) recordings using advanced signal processing and, more recently, machine learning techniques, BCI technology finds application in various areas like emo- tion recognition [(Torres et al., 2020), (Xu et al., 2018)], epileptic seizure detection [(Alkawadri, 2019), (Djoufack Nkengfack al., 2021)], robotic control (Irimia et al., 2012) and video gaming (Kerous et al., 2018). BCIs can also aug- ment human abilities and have the potential to transform how we interact with our environment and each other, offer- ing hope to people with disabilities to regain lost functions [[Chaudhary et al., 2016), (Luu et al., 2017), (Biasiucci et al., 2018), (Kumarasinghe et al., 2021), (Sharma et al., 2016)].
The early days of the BCI placed human experts the center of brainwave analysis, with manual feature extrac- tion by neuroengineers regarded as the gold standard for many years [(Bashashati et al., 2007), (Handy, 2009), (Rao, 2013), (Nam et al., 2018), (McFarland et al., 2006)]. How- ever, these hand-crafted features often fail to generalize effectively real-world data, limiting their practicality in everyday BCI applications.
data collection, making the process both resource-demanding
'Equal contribution 'Imerial College London "Cogitat 'Archimedes/ / Athena Research Unit "National and Kapodistrian University Athens A AAristotle University of Thessaloniki. Corre- spondence to: Na Lee l.w..lel2@i..cc......
Foundation Models have recently emerged as a promising approach to address these limitations, showing remarkable results in various domains, particularly in Natural Language Processing and Computer Vision [BBrwwn et al., 2020), (Tou-
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
vron al., 2023), (Mizrahi al., 2023), (Paraperas Papan-
2.1. LaBraM
toniou et al., 2024)]- Inspired by this tremendous progress, researchers have begun to develop similar Large Brainwave Models (LBMs) to the domain of BCIs [[Jiang et al., 2024), (Cui et al., 2024), (Wang et al., 2025), (Jiang et al., 2025)]- Theoretically, these large models offer several potential ad- vantages: they are capable of identifying complex patterns and relationships EEG data, thanks to their extensive wide array of unlabeled datasets. Thus, they demonstrate improved generalization, reducing the need for task-specific data collection and model training and create more robust and versatile BCI systems capable of adapting to various users, tasks and environments. In addition, their generative nature equips these models with strong adaptability to novel downstream tasks while also enabling them to generate high-quality synthetic data, thus offering promising solutions for predicting brain activity and reconstructing corrupted brain signals (Barmpas et al., 2024a).
LaBraM (Jiang et al., 2024) is a unified EEG foundation model designed to enable tross-datasel learning by segment- ing EEG signals into channel-speciffc patches. Inspired by VQ-GAN (Esser et al., 2020), it employs vector-quantized neural spectrum prediction to train a semantically rich neu- ral tokenizer, which encodes continuous raw EEG channel patches into compact neural codes, known as a neural code- book.
self-supervised pre-training
EEG Patches
Neural Codebook
Figure I. Illustration of LaBraM's Neural Codebook
In this work, we explore the current state of Large Brain- wave Models (LBMs) for EEG--aasd BCIs, adopting a struc- tured multi-step methodology:
This neural codebook serves as a strong base for pre-training the foundation model. LaBraM follows a two-step pre- training approach: first training of the neural codebook with target objective being the reconstruction of the fourier amplitude and phase of the EEG patch. Then, the core training of the foundation model, where the model learns by predicting the original neural codebook for masked EEG channel patches.
1. We begin by evaluating the performance of publicly available pre-trained state-of-the- ohrtte- LBMs, specifi- cally LaBraM (Jiang et al., 2024) and NeuroGPT (Cui et al., 2024), against traditional deep learning models. This comparison aims to highlight the advantages and limitations of LBMs in comparison well-established techniques.
LaBraM was pre-trained on approximately 2,500 hours of diverse EEG signals sourced from around 20 datasets. Its effectiveness was validated on a variety of downstream tasks, demonstrating its versatility and robustness for different bEG-based applications.
2.
We investigate the application of Low-Rank Adapta- tion (LoRA) (Hu et al., 2021), a widely used method for parameter efficient fine-tuning (PEFT) of large pre- trained models across diverse tasks. Similarly to the exploration of time-series foundation models (Gupta et al., 2024), through ablation studies and extensive ex- perimental analysis, we examine the efficacy of LoRA when applied to pre-trained LBMs.
2.2. NeuroGPT
3. We demonstrate that, by carefully selecting LoRA pa- rameters, it is possible to significantly reduce the num- ber of trainable parameters in pre-trained L.MMs. No- tably, this reduction is achieved without compromising model performance, offering a practical path towards more rescurce-efffcient applications of LBMs BCI systems.
EEG Patches
Masked Patches
GPT
2. Background
In recent years, several Large Brainwave Models (LBMs) have been introduced that promise strong generalization capabilities across various BCI paradigms. In this work, we will focus mainly on two of these LBMs, namely LaBraM (Jiang et al., 2024) and NeuroGPT (Cui et al., 2024).
Figure 2. Illustration of NeuroGPT's Auto-Regressive Training
NeuroGPT (Cui et al., 2024) is a foundation model that combines an EEG encoder with a GaT-based architecture. The EEG encoder draws inspiration from the widely recog-
2

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
nized deep learning framework EEGConformer (Song et al., 2023) that utilizes spatio-temporal convolutional feature extraction followed by a series of self-attention layers. The model leverages GPT-style self-supervised training, employ- ing an auto-regressive approach (Brown et al., 2020) where it predicts the next masked token based on preceding to- kens. This training paradigm enables NeuroGPT to capture complex temporal and spatial patterns in EEG data, making it a robust foundation model for variety of downstream EEG-based applications.
3. Analysis
3.1. Data Preprocessing
All models were evaluated in downstream classification tasks for the following five benchmark EEG datasets (Lee et al., 2025): Motor paradigm in High Gamma (Schirrmeis- ter et al., 2017), the ERP (Event-Related Potential) paradigm from Korean University (Hong-Kyung et al., 2019), Work- ing Memory dataset (Pavlov et al., 2022), Physionet's sleep staging dataset, Sleep-EID (Kemp et al., 2000) and Eyes Open vs Closed classification on the Physionet Motor dataset (Schalk et al., 2004). These tasks were selected to capture a diverse range of BCI paradigms and the datasets were specifically chosen for their minimal spurious artifacts, reducing the likelihood of specious performance during training (Lee et al., 2025).
NeuroGPT is pre-trained on recordings from the Temple University Hospital (TUH) EEG Corpus (Obeid & Picone, 2016), a comprehensive dataset that offers diverse and ex- tensive data for model training. Specifically, NeuroGPT was trained on 20,000 EEG recordings from the TUH corpus dataset with total duration of 5656 hours.
For each of the baseline Large Brainwave Models, bench- mark data was preprocessed to match the input data structure used during pre-training:
2.3. Low-Rank Adaptation (LoRA)
Low-Rank Adaptation is a PEFT technique that introduces low-rank updates to pre-trained models, significantly re- ducing the number of trainable parameters. In standard fine-tuning, the full weight matrix W E Badk is updated during training which can be computationally expensive for large-scale models. LoRA, instead, decomposes the update into the product of two low-rank matrices. Mathematically:
1. For LaBraM, a sample frequency of 200Hz was used, a bandpass filter from 0.5-45Hz was applied, as well as notch filters at 50Hz, 60Hz and 100Hz to remove powerline noise. Trials were cut into Is patches taken across channels, to give 256 patches per sample. In addition the EEG trial data, temporal and spatial embeddings for each sample were given as input to the model. Temporal embeddings include each patch's ss temporal position within the length of the trial, whereas spatial embeddings include the position of the patch's 's channel within a global list of all known electrodes. Only data from electrodes which were present in the global list provided were used.
AW = AB,
where A E Rdxr and B E Benk with r < min(d, k). During fine-tuning, the original weight matrix W remains frozen, and only the low-rank matrices A and B are trained. The effective weight becomes:
w' WWW WBBB.
2. For the NeuroGPT model, the data were resampled to 250Hz and bandpass filter of 0.05-100HH was applied. Similarly LaBraM, notch filters 50Hz, 60Hz and harmonics were also applied. NeuroGPT's input data need to include a specific set of channels in a fixed order. Therefore, for each benchmark dataset, we only use data from electrodes that are present in the pre- training data. For any expected channels which are not included in the benchmark data, the nearest available electrode's data is used (if the location is within few centimeters), otherwise the channel data are set to zero.
This approach greatly reduces the number of trainable pa- rameters from d x k to x (d + k), where r is the rank of the decomposition. The low-rank matrices capture task- specific adaptations while preserving the original model's pre-trained knowledge. For example, in transformer- based layers, LoRA is typically applied to the weight matrices of key, query or value projections in self-attention layers, significantly reducing computational and memory demands. Mathematically, during inference, the computational over- head of LoRA is negligible because the low-rank updates A W are pre-computed. This constitutes LoRA an effective method for adapting large pre-trained models to specific tasks, where tne-ttning efficiency and generalization are critical.
For all downstream datasets, Common Average Re- referencing (CAR) was applied across all channels to reduce noise.
3

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
Table Classification accuracy of finetuned foundation models and standard deep learning architectures, reported mean (std). Fach trained/finetuned for 20 epochs with 10 fold cross-vale valiidanii. Trainable parameters include the size of the classification heads. Bold values indicate best performance per task or overall.
MODEL
MOTOR
ERP
MEMORY
SLEEP
EYES
MEAN
PARAMS
EEGNET
0.657 (.087) 0.590 (.087)
0.912 (.009) 0.896 (.007)
0.660 (.022)
0.624 (.037)
0.803 (.061)
0.731 (.024)
2,394
BEG-
0.669 (.021)
0.688 (.057)
0.823 (.038)
0.733 (.021)
22,366
INCEPTION
LABRAM
0.614 (.096)
0.911 (.013)
0.643 (.040)
0.704 (.025)
0.840 (.041)
0.742 (.023)
5,854,288
NEUROGPT
0.682 (.083)
0.904 (.012)
0.610 (.052)
0.665 (.030)
0.821 (.052)
0.736 (.025)
78,536,146
(FULL
MODEL)
NEUROGPT
0.695 (.085)
0.908 (.012)
0.634 (.035)
0.647 (.024)
0.843 (.045)
0.745 (.027)
717,958
(ENCODER)
Table 2. P-values of paired-t tests between EEGInception and finetuned foundation models. Bold values indicate statistically significant
result (p < 0.05)
MODELS
MOTOR
ERP
MEMORY
SLEEP
EYES
EEGINCEPTION / LABRAM
0.5860
0.0123
0.1090
0.2995
0.4468
EEGINCEPTION NEUROGPT (FULL MODEL)
0.0314
0.0401
0.0041
0.2226
0.8979
EEGINCEPTION NEUROGPT (ENCODER)
0.0072
0.0051
0.0399
0.0495
0. 0.1056
3.2. Comparing Brainwave Foundation Models with Deep Learning Models
that no participant would be present in both the training and validation sets. To perform the downstream tasks, un- trained classification heads were added to the pre-trained transformer- Large Brainwave Foundation Models be- fore finetuning. The size and structure of the classifier depend on the latent dimension of the model and the num- ber of target classes for the given benchmark. The exact architecture of the classifiers is given in Table 4:
The performance of large foundation models varies dramat- ically between domains. In some areas, large-scale pre- training has revolutionized task performance by enabling the models to generalize across a broad range of applica- tions, often surpassing traditional methods. However, in other domains, foundation models sometimes fail to demon- strate significant advantages and are even outperformed by simpler, domain-specifie baselines. It is therefore important to critically measure the effectiveness of recent Large Brain- wave Foundation Models (LBMss). To achieve this, here systematically evaluate the performance of fine-tuned Large Brainwave Foundation Models in comparison to other deep learning architectures. By benchmarking on a variety of tasks and datasets as described in section 3.1, we aim to assess whether fine-tuned LBMs consistently provide an advantage over more specialized or traditional approaches.
For LaBraM, a simple dropout and fully connected layer were used
2. For NeuroGPT, three-layer MLP with dropout and ELU activations were used
Using the same fine-tuning setup, we performed similar training (from scratch) process for the EEGNet and EEGIn- ception models to provide a basis for comparison. As shown in Table 1, standard deep learning baselines can achieve comparable or even superior performance to some large models. However, NeuroGPT outperforms other models on average, including both standard deep learning baselines and other large foundation models. While NeuroGPT might not lead in every benchmark task, it consistently demon- strates strong average performance.
We perform finetuning on three configurations: the pre- trained LaBraM base model, the pre-trained NeuroGPT model and the encoder-only module of the pre-trained Neu- roGPT model (as discussed in (Cui et al., 2024) where the authors claim that fine-tumie teeettnnng the encoder alone pro- duces similar or improved results over the full model). Each configuration was trained for 20 epochs (to avoid overfit- ting) and evaluated using 10-fold subject-independent cross- validation, where samples were split on subject level such
Both foundation models (LaBraM and Neuro-GPT) have in their pre- training datasets paradigms that include motor-, ERP-, sleep- and eyes-related tasks. The results in Table

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
Table Classification accuracy of foundation models where all parameters except classification heads are frozen. Each trained for 20 epochs with 10 fold cross noss-validation Bold values indicate best performance per task or overall.
MODEL
MOTOR
ERP
MEMORY
SLEEP
EYES
MEAN ACCURACY
LABRAM
0.297
0.884
0.670
0.608
0.717
0.635
NEUROGPT (FULL MODEL)
0.366
0.884
0,656
0.597
0.734
0.647
NEUROGPT (ENCODER)
0.431
0.883
0.655
0.602
0.746
0.663
next sections.
Table 4. Classification heads for each model configuration where n.cls is the number of classes for each benchmark task
3.3. Low-Rank Adaptation Exploration in Brainwave Foundation Models
MODEL
CLASSIFIER
In this section, we investigate how fine-tuning strategies, such as Low-Rank Adaptation, influence the performance of these pre-trained large brainwave foundation models, shedding light on ways to maximize the utility of these models across diverse domains. Specifically, we explore the performance and parameter efficiency of LoRA when applied in different ways to finetuning these pre-trained large brainwave foundation models.
LABRAM
Linear (200, n cls), Dropout (0.5)
NEUROGPT
Linear (1024,256), ELU,, Dropout (0.5), Linear (256,32), ELU, Dropout (0.3), Linear (32, n cls)
FULL
MODEL
NEUROGPT
Linear (2160,256), ELU, Dropout (0.5), Linear (256,32), ELU, Linear (32, n.cls)
ENCODER
In our analysis, we use the original formulation of LoRA wherein each target module's weight matrix W adapted with two low-rank matrices A, B with a chosen rank, r. We choose not to adapt any bias terms and leave them frozen dur- ing finetuning. When adapting attention modules we treat queries, keys and values single combined matrix W Wakes gkuu however the output projection is left frozen. Furthermore, we also apply LoRA to fully connected and convolutional layers in our analysis to effectively evaluate the importance of these layers during the finetuning process. all experi- ments, the scaling factor a (as described in (Hu et al., 2021)) is set 8.
1 show that foundation models achieve comparable perfor- mance to baseline models in ERP, sleep and eyes tasks. NeuroGPT significantly outperforms baseline models in motor. Interestingly, in the memory wask- was not explicitly included in the pre-training models slightly outperform foundation models. Although the performance margin is 1.2% compared to the next-best bascline standard deep learning model, this indicates that, while these foundation models represent promising step to- ward advancing Large Brainwave Foundation Models, their substantial benefits over traditional approaches have yet to be fully realized. In summary:
3. LOW-RANK ADAPTATION IN ALL LAYERS
Standard deep learning baselines trained only on specific tasks can achieve comparable performance to fine-tuned pre-trained large brainwave foundation models while having only a fraction of trainable parameters.
In this subsection, we investigate the effect of LoRA when different ranks are applied to the attention and fully- connected layers of the large brainwave foundation models.
The rank of the convolutional layers Te is set to the maxi- mum power of 2 which would not lead adapters with a greater number of parameters than the original weight ma- trices. For LaBraM r. 4, and NeuroGPT r. for both configurations, the full model and encoder only. Given the fixed re, we experiment with different ranks for the attention and fully connected modules r E (1, 2,4, 4, 8, 16).
Testing the generalization capabilities of large foundation models is also crucial. Therefore, we performed the same fine-tuning process as in the above-mentioned tasks but kept the pre-trained foundation models frozen and trained only the classification heads during the fine-tuning step. From the results in Table 3, training just the classification heads yields models that lack behind traditional deep learning approaches by a large margin of almost 8-10%. This demon- strates the necessity of full-m full-model fine-tuning, and in turn makes parameter efficient fine-tuning (PEFT) techniques like LoRA extremely valuable, which we will explore in the
5

---

Are Large Brainwave Foundation Models Capable Yet Insights from Fine-Tuning
Table Performance of foundation models finetuned using IoRA with varying ranks for attention and fully connected layers. Ranks for LoRA adapters on convolutional layers are fixed to the maximum possible for the given model. Bold values indicate best performance per task or overall.
MODEL
RANK
ERP
MEMORY
SLEEP
EYES
MEAN ACCURACY
TRAINABLE PARAMETERS
0.905
0.624
0.729
0.839
0.774
34,149
2
0.902
0.643
0.725
0.836
0.777
67,749
LABRAM
4
0.902
0.638
0.715
0.822
0.770
134,949
8
0.901
0.636
0.712
0.827
0.769
269,349
0.902
0.626
0.708
0.845
0.770
538,149
0.884
0.650
0.643
0.788
0.741
373,218
NEUROGPT
0.885
0.650
0.635
0.800
0.743
467,226
FULL
4
0.884
0.656
0.643
0.796
0.745
655,242
MODEL
8
0.885
0.642
0.642
0.798
0.742
1.031,272
16
0.884
0.646
0.643
0.791
0.741
1,783,338
1
0.896
0.643
0.643
0.796
0.744
$73,866
2
0.897
0.641
0.643
0.801
0.746
577,706
NEUROGPT
0.897
0.643
0.644
0.799
0.746
585,386
ENCODER
0.897
0.643
0.643
0.806
0.747
600,746
16
0.894
0.642
0.644
0.801
0.745
631,466
0.80
0.78
0.75
Mean Accuracy
Mean Accuracy
0.76
XClassic Nets
LaBraM
0.74
NeuroGPT Model)
LaBraM
NeuroGPT (Encoder Only)
0.60
(full model)
100
1,000
10,000
100,000
1,000,000
10,000.000 100,000,000
NeuroGPT (encoder)
Trainable Parameters
16
Figure 3. Number of trainable parameters against mean accuracy across downstream tasks.
MLP and Attention Layer Rank
Figure 4. Mean accuracy vs rank of attention and fully connected layers, given fixed rank for convolutional layers
As is shown in Table 5:
Using LoRA in large brainwave foundation models significantly reduce the number of trainable parameters without compromising model performance
performance, we performed a series of ablation studies using the LoRA technique. For each of the three model config- urations, we use the rank that produces the best average classification performance across all benchmarks, denoted as r'. We then apply LoRA to all possible combinations of convolution, attention and fully-connected layers.
Theoretically, would expect performance increase with rank. For NeuroGPT, that assumption holds while for LaBraM its performance peaks at rank = 2.
As it is shown in Table 6:
3.3.2. LOW-RANK ADAPTATION ABLATION STUDIES
Performing LoRA only on specific layer, e.g. atten- tion or convolution, usually yields lower performance compared to a combination of two or three these layers.
In order to perform extensive experimental analysis to fur- ther understand the importance of each element of the large brainwave foundation model the fine-tuned downstream
6

---

Are Large Brainwave Foundation Models Capable Yet Insights from Fine-Tuning
Table Performance of foundation models finetuned using LoRA adapters different combinations of layer types. Ranks of attention and fully connected layers are fixed to the best performing rank r' for each model configuration as given in Table 5, Bold values indicate best performance per task or overall.
MODEL
LORA LAYERS
KU ERP
MEMORY
SLEEP EDF
EYES OPEN/- CLOSED
MEAN ACCURACY
TRAINABLE PARAMETERS
ATTENTION
0.902
0.644
0.722
0.852
0.780
19,401
FC
0.900
0.657
0.727
0.835
0.780
48,201
LABRAM r =
CONV
0.884
0.670
0.659
0.768
0.745
549
ATTENTION, FC
0.899
0.623
0.717
0.843
0.770
67,401
ATTENTION, CONV
0.904
0.652
0.729
0.834
0.780
19,749
FC, CONV
0.901
0.659
0.732
0.828
0.780
48,549
ATTENTION
0.883
0.656
0.599
0.726
0.716
374,754
NEUROGPT FULL MODEL r'=4
FC
0.884
0.656
0.579
0.732
0.713
542,658
CONV ATTENTION, FC
0.884
0.657
0.620
0.772
0.733
279.210
0.883
0.656
0,594
0.736
0.717
646,722
ATTENTION, CONV
0.882
0.654
0.635
0.785
0.739
383,274
FC, CONV
0.884
0.646
0.637
0.788
0.739
551,178
ATTENTION
0.883
0.652
0.612
0.750
0.724
$73,026
NEUROGPT ENCODER r' =/=8
FC
0.883
0.655
0.607
0.751
0.724
580,706
CONY
0.894
0.644
0.631
0.801
0.742
570,026
ATTENTION, FC
0.883
0.647
0,613
0.749
0.723
592,226
ATTENTION, CONV
0.896
0.639
0.643
0.800
0.745
581,546
FC. CONY
0.897
0.644
0.635
0.803
0.745
589,226
2. The combination of convolution layers with either fully connected or attention layers has the best average per- formance across all benchmarks.
Therefore, in this subsection we explore the effects of intro- ducing dropout the parameter space of LoRA's low-rank matrices. To achieve this apply LoRA to the LaBraM model, adapting attention, fully-conneeted and convolu- tional layers as in Table 5, with identical setup except this time introducing dropout for each adapter. The findings of (Dettmers et al., 2023) suggest a dropout probability 0.1 when applying LoRA to 7B and 13B parameter models, or 0.05 for 33B and 65B models. The size of the full LaBraM base model only around 5.8M parameters, therefore select relatively high dropout of 0.5.
3. Interestingly, the combinations of attention and volution and fully-connected and convolution demon- strate the same performance. This unveils that for these state-of- ofte-offthhe-rt brainwave foundation models the atten- tion layers might not capture as important information as their temporal encoding parts.
Therefore, conlude that:
As it is shown in Table 7:
LoRA in large brainwave foundation models can demonstrate performance benefits when used combination of two or three different types of layers.
Introducing a dropout LoRA adapters can match or improve classification performance.
Dropout's improvements in classification accuracy typ- ically increase with rank.
Performance is more positively affected in the memory and sleep classification tasks.
3.3.3. EFFECT DROPOUT ON LOW-RANK ADAPTATION
4. Discussion
As demonstrated, foundation models can marginally out- perform traditional deep learning models average and, when finetuned with LoRA, yield an additional performance boost. To investigate this further, we aimed to dive deeper into the LoRA training process for the model with best average performance from Table 6 (LaBraM) and explore potential strategies for further enhancing its performance.
Foundation models have revolutionized numerous fields in computer science, enabling breakthroughs in natural language processing, computer vision and other domains. While early efforts have been made to develop Large Brain- wave Foundation Models (LBMs), these models have yet to reach their full potential, In this work, we investigated

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine- Fine-Tuning
Table LaBraM finetuned using LoRA with varying ranks for attention and fully connected layer adapters. Each value the difference classification accuracy between using dropout with probability 0.5 vs no dropout.
RANK
KU ERP
MEMORY
SLEEP EDF EYES OPEN/CLOSED
MEAN
1
-0.002
+0.046
+0.007
-0.005
+0.011
2
+0.003
+0.025
+0.010
+0.005
+0.011
4
+0.004
+0.032
+0.019
+0.012
+0.017
+0.006
+0.034
+0.024
+0.012
+0,019
16
+0.006
+0.042
+0.029
-0.014
+0.016
fine-tuning techniques applied to two state-of-the-arr Large Brainwave Foundation Models.
will largely outperform all current state-of-the-ee baselines in various tasks with minimum required fine-tuning.
Our findings reveal that large pre-trained models which of- fer interpretability insights (LaBraM) outperform standard deep learning baselines and black-box models (NeuroGiT)). However, the margin of improvement is small when consid- ering the relative sizes of the models: for example, fully fine-tuned LaBraM has over 2000 times more trainable pa- rameters than EEGNet. This finding signifies the importance of developing more efficient large brainwave models and the need to develop new domain-specifie training techniques to train these large models.
5. Conclusion
In this work, we investigate fine-tuning techniques for large brainwave foundation models (LBMs), providing com- prehensive evaluation of their performance in range of downstream BCI benchmark tasks. Our experiments show that, despite their scale and pre-training, current fine-tuned LBMs underperform compared to standard deep learning models, which have significantly fewer trainable parameters. Furthermore, we demonstrate that the Low-Rank Adapta- tion (LoRA) fine-tuning technique can effectively reduce the trainable parameters of LBMs without compromising performance, but usually when applied to combination of two or three different types of layers.
Additionally, we conducted an in-depth study of the widely used Low-Rank Adaptation technique. Our results demon- strate that applying LoRA to large brainwave foundation models can substantially reduce the number of trainable parameters without sacrificing performance. However, through a series of ablation studies, we uncovered that per- formance improvements with LoRA are achieved only when is applied a combination of two or three different types of layers. This raises important questions about the pre- training processes used to develop these large models, un- veiling cross-stage dependencies (rather than being limited e.g to attention layers) and suggesting that their architec- ture and training methodologies may require refinement and domain-specific training techniques to better capture the underlying nature of brainwave signal.
To the best of our knowledge, this is the first study to objec- tively and systematically assess the fine-tuned performance of LBMs on diverse and carefully curated set of BCI down- stream tasks. Furthermore, similarly the study in time-series foundation models (Gupta et al., 2024), this work pioneers the use of LoRA in the context of brainwave foundation models, field that remains largely unexplored. This exten- sive study highlights critical considerations for the research community, emphasizing the need for more efficient and effective approaches to developing and fine-tuning Large Brainwave Foundation Models (LBMs).
Previous works in the field of causal reasoning for deep learning brainwave models (Barmpas et al., 2024b) as well as LBMs (Barmpas et al., 2024a) have showcased important training considerations that one must take into account when training LBMs. These can be used in conjunction with this work to further guide the research community in the development of efficient LBMs.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted here.
We believe the future of LBMs should go beyond merely adopting transfer techniques from other domains. Instead, they should integrate domain-specitn knowledge-as such as leveraging various EEG modalities-an employ tailored training strategics, like brain-inspired masking techniques. These are essential elements to fully capture the diverse nature of EEG and build an efficient and effective LBM that
A Acknowledgments
This work was supported by the EPSRC Turing AI Fel- lowship (Grant Ref: EP/Z534699/1); Generative Machine Learning Models for Data Arbitrary Underlying Geome- try (MAGAL).
8

---

Are Large Brainwave Foundation Models Capable Yet? ? Insights from Fine-Tuning
References
Cui, W., Jeong, W., Thölke, P., Medani, T., Jerbi, K., Joshi, A. A., and Leahy, R. M. Neuro-gpt: Towards foundation model for eeg, 2024.
Alkawadri, R. Brain-computer interface (bci) applica- tions in mapping of epileptic brain networks based on intracramial- An update. Frontiers in Neuroscience, 13:191, 2019. 2019. ISSN 1662-453X. doi: 10.3389/fnins 00191.
Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized 1lms, 2023.
Djoufack Nkengfack, L. C.. Tchiotsop, D., Atangana, R., Louis-Door, V., and Wolf, D. Classification of eeg signals for epileptic seizures detection and eye states identifi- cation using jacobi polynomial transforms-basee bassd mea- sures of complexity and least-square support vector ma- chine. Informatics in Medicine Unlocked. 23:100536, 2021. ISSN 2352-9148. doi: 10. 10.1016/j.imm 2022 100536.
Bakas, S., Ludwig. S., Barmpas, K., Bahri, M., Panagakis, Y., Laskaris, N., Adamos, D. A., and Zafeiriou, S. Team cogitat at neurips 2021: Benchmarks for eeg transfer learning competition, 2022.
Barmpas, K., Panagakis, Y., Adamos, D. A., Laskaris, N., and Zafeiriou, S. Brainwave-scateering net: a lightweight network for eeg-based motor imagery recognition. Jour- nal of Neural Engineering, 20(5):05601,, September 2023. ISSN 1741-2552.
Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis, 2020.
Gupta, D., Bhatti, A., Parmar, S., Dan, C., Liu, Y., Shen, B., and Lee, S. Low-rank adaptation of time series foun- dational models for out-of-domain modality forecasting, 2024.
Barmpas, K., Panagakis, Y., Adamos, D., Laskaris, N., and Zafeiriou, S. causal perspective in brainwave founda- tion models. In Causality and Large Models @ @NeurIPS 2024, 2024a.
Handy, T. C. Brain signal analysis advances in neuroelec- tric and neuromagnetic methods. Cambridge, Mass, MIT Press. 2009.
Barmpas, K., Panagakis, Y., Zoumpourlis, G., Adamos, D. A., Laskaris, N., and Zafeiriou, S. A causal perspec- tive on brainwave modeling for brain-computer inter- faces. Journal of Neural Engineering, 21(3)003601,, may 2024b. doi: 10. (108//1741-2552/d3ee
Hong- Kaung, K., John, W., Min-Ho, L., O-Yeon, K., Seong- Whan, L., Siamac, F., Yong-Jeong, K., and Young-Eun, Eun, L. Supporting data for "eeg dataset and openbmi toolbox for three bci paradigms: An investigation into bei illiteracy", 2019.
Bashashati, A., Fatourechi, M., Ward, R. K., and Birch, G. E. A survey of signal processing algorithms in brain-computer interfaces based on electrical brain sig- nals. Journal of Neural Engineering, 4(2):R32-R57, mar 2007. doi: 10.08/1774--550///
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models, 2021.
Biasiucci, A., Leeb, R., Iturrate, I., Perdikis, S., Al- Khodairy, A., Corbet, T., Schnider, A., Schmidlin, T., Zhang, H., Bassolino, M., Viceic, D., Vuadens, P., Gug- gisberg, A., and Milkán, J. d. R. Brain-actuated func- tional electrical stimulation elicits lasting arm motor recovery after stroke. Nat Commun, 9, 2018 doi: 10.1038/s4 068-444167--8-04667---
Irimia, D., Ortner, R., Krausz, G., Guger, C., and Poboro- niue, M. Bci application in robotics control. IFAC Pro- ceedings Volumes, 6670. doi:
1869-18644144 2012. ISSN 1474- 14th IFAC Symposium on Information Control Problems in Manufacturing.
Jiang, W., Wang, Y., liang Lu, B., and Li, D. NeuroLM: A universal multi-task foundation model for bridging the gap between language and EEG signals. In The Thirteenth International Conference on Learning Representations, 2025.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert- Vourbe A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu,
J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners, 2020.
Jiang, W.-B., Zhao, L.-M., and Lu, B.-L. Large brain model for learning generic representations with tremendous eeg data in bci, 2024.
Kemp, B., Zwinderman, A., Tuk, B., Kamphuisen, H., and Oberye, J. Analysis of a sleep-dependent neuronal feed- back loop: the slow-wave microcontinuity of the eeg. IEEE Transactions on Biomedical Engineering. 47(9): 1185-1194, 2000. doi: 0111009100.67999
Chaudhary, U., Birbaumer, N., and Ramos Gamos-Murguialdayy A. Brain-comppun interfaces for communication and rehabilitation. Nat Rev Neurol, 12, 2016. doi: 10310 mneurol.2016.11
9

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
Kerous, B., Skola, F., and Liarokapis, F. Eeg-based bci and video games: a progress report. Virtual Reality, 22, 2018. doi: 10. 10.10077/10055-00--02288
Rao, R. P. N. Brain-computer interfacing: an introduction. 2013.
Santamaria-Vázze
E., Martinez-Cagigal, V., Vaquerizo- Villar, F., and Hornero, R. Beg-inception: A novel deep convolutional neural network for assistive erp-based compuer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineeringg 28(12):2773 2782, 2020. doi: 10. 10.11099TNSREE22200334488
Kumarasinghe, K., Kasabov, N., and Taylor, D. Brain- inspired spiking neural networks for decoding and under- standing muscle activity and kinematics from electroen- cephalography signals during hand movements. Sci Rep,
11, 2021. doi: 10. 10.1038/s41590
Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gor- don, S. M., Hung, C. P., and Lance, B. J. Eegnet: compact convolutional neural network for eeg- eeg-based brain-computer interfaces. Journal of Neural Engineer- ing, 15(5):056013, July 2018. ISSN 1741-2552. doi: 10.1088/1741 -1158/2/71225//.
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. BC12000: a general-purpose brain- computer interface (BCT) system. IEEE Trans. Biomed. Eng. 51(6): June 2004.
Schirrmesster, R. T., Springenberg, J. T., Fiederer, L. D.J. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hut- ter, F., Burgard, W., and Ball, T. Deep lcarning with convolutional neural networks for eeg decoding and vi- sualization. Human brain mapping, 38(11):5391-544 2017.
Lee, N., Bakas, S., Barmpas, K., Panagakis, Y., Adamos, D., Laskaris, N., and Zafeiriou, S. Assessing the capabilities of large brainwave foundation models. In Workshop on Spurious Correlation and Shortcut Learning: Founda- fions and Solutions, 2025.
Sharma, G., Friedenberg, D., Annetta, N., Glenn, B., Bock- brader, M., Majstorovic, C., Domas, S., Mysiw, J., Rezai, A., and Bouton, C. Using an artificial neural bypass restore cortical control of rhythmic movements in human with quadriplegia. Sci Rep, 6, 2016. doi: 10. 1000////ep33807
Luu, T., Nakagome, S., He, Y., and Contreras- Vidal, Real-time eeg-based brain compuerr interface to a vir- tual avatar enhances cortical involvement in human. Sei Rep, 7, 2017. doi: 10. N0.1038/441598-0177 017-09187-0
McFarland, D., Anderson, C., Muller, K.-R., Schlogl, A., and Krusienski, D. Bei meeting 2005-workshop on bei signal processing: feature extraction and transla- tion. IEEE Transactions Neural Systems and Re- habilitation Engineering, 14(2):135-133 138, 2006. doi: 10.1 109/TNSSE..206..755667
Song, Y., Zheng, Q., Liu, B., and Gao, X. Eeg conformer: Convolutional transformer for eeg decoding and visu- alization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710-719, 2023. doi: 10. 00.11099TNREE.2222.330000
Mizrahi, D., Bachmann, R., Kar, O. F., Yeo, T., Gao, M., Dehghan, A., and Zamir, A. 4m: Massively multimodal masked modeling. In Teirth- Conference on Neu- ral Information Processing Systems, 2023.
Torres, E. P., Torres, E. A., Hernández-Alvarez, M., and Yoo, S. G. Eeg-based bei emotion recognition: A survey, Sensors, 20(18), 2020. ISSN 1424-8220. doi: 10.3390/ $20185083.
Nam, C. S., Nijholt, A., and Lotte, F. Brain-Compunt Interfaces Handbook: Technological and Theoretical Ad- vances, CRC Press. 2018
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lam- ple, G. Llama: Open and efficient foundation language models, 2023.
Obeid, I. and Picone, J. The temple university hospital eeg data corpus. Frontiers in Neuroscience, 10, 2016. ISSN 1662-453X. doi: 10.3389/fnins... wtw.iiis........00 2016.00196
Wang. J., Zhao, S., Luo, Z., Zhou, Y., Jiang, H., Li, S., Li, T.. and Pan, G. CBramod: A criss-cross brain foundation model for EEG decoding. In The Thirteenth International Conference on Learning Representations, 2025.
Paraperas Papantoniou. F. Lattas, A., Moschoglou, S., Deng. J., Kainz, B., and Zafeiriou, S. Arc2face: A founda- tion model for id-consistent human faces. In Proceedings of the European Conference on Computer Vision (ECCV), 2024.
Wei, X., Faisal, A. A., Grosse- Grosse-Wentrup, M., Gramfort, A., Chevallier, S., Jayaram, V., Camille Jeunet, S. B., Ludwig, S., Barmpas, K., Bahri, M., Panagakis, Y., Laskaris, N., Adamos, D. A., Zafeiriou, S., Duong, W. C., Gordon, S. M., Lawhern, V. J., Śliwowski, M., Rouanne, V., and Tempezyk, 2021 beetl competition: Advancing transfer
Pavlov, Y. G., Kasanov, D., Kosachenko, A. I., and Ko- tyusov, A. I. "eeg, pupillometry, ecg and photoplethys- mography, and behavioral data in the digit span task and rest", 2022.
10

---

Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine- Tine-Tuning
learning for subject independence & heterogenous eeg data sets, 2022.
Xu, T., Zhou, Y., Wang, Z., and Peng, Y. Learning emotions eeg-based recognition and brain activity: A survey study bci for intelligent tutoring system. Procedia Computer Science, 130:376-382, 2018. ISSN 1877-0509. doi: 1.procs.2018.04.00 The 9th International Conference on Ambient Systems, Networks and Technologies (ANT 2018) / The 8th International Conference on Sustainable Energy Information Technology (SEIT-2018) Affiliated Workshops.


Reviews:
[{"review_id": "wTaH0e1LzM", "reviewer": "Reviewer_QMtY", "summary": "An interesting perspective-style paper comparing state of the art EEG-focused ML models in traditional versus foundation application. The paper is well written and presents a solid comparison of two methods resulting in  a statement state-of-the-art LBMs achieve only marginal improvements (0.5%) over traditional deep architectures. The results is an important message to the community, that simple transfer of methods between domains is sometimes not necessarily improving outcomes.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their thoughtful review and valuable feedback. As the reviewer highlights we have meticulously selected various benchmark BCI paradigms datasets, reflecting the current state-of-the-art within the EEG community. We decided also to extend these results to capture one more popular BCI paradigm, namely Motor. Therefore,  we added a new movement benchmark based on the High Gamma dataset (Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(11):5391–5420, 2017). The updated results are presented below: \n\n\nTable: Classification accuracy of finetuned foundation models and standard deep learning architectures, reported as mean (std). Each trained/finetuned for 20 epochs with 10 fold cross-validation. Bold values indicate best performance per task or overall, italic values indicate next best performance.\n| Model                 | Motor          | ERP            | Memory        | Sleep         | Eyes          | Mean          |\n|-----------------------|---------------|---------------|--------------|--------------|--------------|--------------|\n| EEGNet               | 0.657 (0.087)  | **0.912** (0.009)  | _0.660_ (0.022)  | 0.624 (0.037)  | 0.803 (0.061)  | 0.731 (0.024)  |\n| EEGInception         | 0.590 (0.087)  | 0.896 (0.007)  | **0.669** (0.021)  | _0.688_ (0.057)  | 0.823 (0.038)  | 0.733 (0.021)  |\n| LaBraM               | 0.614 (0.096)  | _0.911_ (0.013)  | 0.643 (0.040)  | **0.704** (0.025)  | _0.840_ (0.041)  | _0.742_ (0.023)  |\n| NeuroGPT (full)      | _0.682_ (0.083)  | 0.904 (0.012)  | 0.610 (0.052)  | 0.665 (0.030)  | 0.821 (0.052)  | 0.736 (0.025)  |\n| NeuroGPT (encoder)   | **0.695** (0.085)  | 0.908 (0.012)  | 0.634 (0.035)  | 0.647 (0.024)  | **0.843** (0.045)  | **0.745** (0.027)  |\n\nThe performance difference between NeuroGPT (the best foundation model in the new analysis) and EEG-Inception (the best baseline network) is 1.2\\% (improvement compared to the previous 0.5\\%) but with a considerable increase in the number of trainable parameters and larger std among folds.\n\nIn terms of a more pronounced perspective on the future outlook for LBMs, we strongly believe that this extensive study highlights critical considerations for the research community. In the camera-ready version of the manuscript, we could also add that we believe the future of Large Brainwave Foundation Models (LBMs) should go beyond merely adopting transfer techniques from other domains. Instead, they should integrate domain-specific knowledge—such as leveraging various EEG modalities—and employ tailored training strategies, like brain-inspired masking techniques. These are essential elements to fully capture the diverse nature of EEG and build an efficient and effective LBM that will largely outperform all current state-of-the-art baselines in various tasks with minimum required fine-tuning. \n\nIn the camera-ready version of the manuscript, we will incorporate the aforementioned discussion points. We sincerely thank the reviewer for their valuable comments, which have helped us to improve our work.", "claims_and_evidence": "This paper delivers a thorough and insightful comparison of open-source datasets, providing a valuable perspective/review that is highly relevant to the ICML community. The analysis is well-structured and demonstrates a strong understanding of the field. This is a solid contribution that fits the ICML scope perfectly.", "methods_and_evaluation_criteria": "The authors have meticulously selected benchmark BCI and sleep study datasets, reflecting the current state-of-the-art within the EEG community. Furthermore, the rigorous evaluation of both cutting-edge LBMs and established deep learning architectures underscores the paper's contribution to the field. This is a highly valuable contribution, offering important and insightful results.", "theoretical_claims": "The selection of Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning (PEFT) of large pretrained models is a particularly insightful technical and theoretical contribution. This decision enables a clear and solid comparative analysis across diverse tasks. This is a very well-executed piece of work.", "experimental_designs_or_analyses": "The authors have made excellent choices in selecting datasets and comparing models, reflecting current state-of-the-art practices. This paper provides a comprehensive analysis, yielding significant and well-supported results.", "relation_to_broader_scientific_literature": "This is a strong contribution that effectively evaluates the current state-of-the-art in the EEG field, specifically comparing foundation models with classical methodologies. This work has the potential for considerable impact.", "essential_references_not_discussed": "The provided references are comprehensive and relevant to the scope of this work.", "other_strengths_and_weaknesses": "This is a solid contribution with the potential for significant impact within the field.", "other_comments_or_suggestions": "None.", "questions_for_authors": "The concluding statements are carefully worded and diplomatic. Adding a more pronounced perspective on the future outlook for LBMs in this application could offer valuable insight to readers.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pEicd0X8ac", "reviewer": "Reviewer_9Has", "summary": "The paper evaluates the performance of two Large Brainwave Foundation Models, LaBraM and NeuroGPT, by fine-tuning them on multiple EEG-based benchmark tasks. The authors compare these LBMs to well-known deep learning baselines (e.g. EEGNet, EEGInception) and investigate parameter-efficient fine-tuning via Low-Rank Adaptation (LoRA). Their results show that LaBraM slightly outperforms baselines but at a much higher parameter cost, while NeuroGPT lags in performance. They also find that LoRA can significantly reduce trainable parameters in LBMs without harming accuracy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their thoughtful review and valuable feedback. As mentioned in our response to Reviewer QMtY, we have decided also to extend our analysis results to capture one more popular BCI paradigm, namely Motor. Therefore, we added a new movement benchmark based on the High Gamma dataset (Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and\nTonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(11):5391–5420, 2017). Please see our response to Reviewer QMtY for the updated results Table. This extends our thorough analysis even further as highlighted by the current Reviewer 9Has. \n\nAs highlighted by the reviewer, explainability is crucial in BCI applications. Therefore, developing interpretable AI models is essential. In recent years, many research papers have developed interpretable EEG models with strong performance. As we head toward large brainwave foundation models, maintaining this requirement remains critical. While Neuro-GPT follows a black-box approach, LaBraM incorporates a neural codebook method, enhancing model interpretability, as discussed in the background section of the manuscript. \n\nAs shown in the analysis Table in our response to Reviewer QMtY, foundation models slightly outperform baseline models by an average of 1.2\\%. However, as the reviewer rightly points out, a closer examination of individual tasks and the pre-training datasets used by these models is worth mentioning. Both foundation models (LaBraM and Neuro-GPT) have in their pre-training datasets paradigms that include motor-, ERP-, sleep- and eyes-related tasks. From the results, we observe that foundation models achieve comparable performance to baseline models in ERP, sleep, and eyes tasks. NeuroGPT significantly outperforms baseline models in motor. Interestingly, in the memory task—which was not explicitly included in the pre-training datasets—baseline models slightly outperform foundation models. This evidence further strengthens our claim that these models have yet to reach their full potential - particularly in achieving significant generalization across new tasks. By integrating domain-specific knowledge, such as leveraging various EEG modalities, and employing tailored training strategies, like brain-inspired masking techniques, these models could fully capture the diverse nature of EEG and largely outperform all current state-of-the-art baselines in various tasks with minimum required fine-tuning.\n\nIn the camera-ready version of the manuscript, we will incorporate the aforementioned discussion points. We sincerely thank the reviewer for their valuable comments, which have helped us to improve our work.", "claims_and_evidence": "- The paper claims that existing LBMs do not yield substantially better performance than smaller specialized networks, and the authors provide numerical evidence that LaBraM improves average accuracy by only about 0.5% over baselines.  \n- LoRA can retain or even improve performance of LBMs while drastically cutting trainable parameters. Their experiments across rank settings and layer types support this.", "methods_and_evaluation_criteria": "- The authors use standard EEG preprocessing steps including bandpass filtering, notch filtering, resampling that are consistent with typical BCI literature.  \n- Fine-tuning is done with 10-fold subject-independent cross-validation, ensuring robust performance estimates.  \n- The evaluation focuses on classification accuracy across four EEG tasks, which is a standard metric for BCI tasks.  \n- The approach to LoRA is well-detailed, including how the low-rank adapters are integrated into each layer.", "theoretical_claims": "- The paper focuses on empirical evaluations.", "experimental_designs_or_analyses": "- The paper uses four benchmark datasets (ERP, working memory, sleep staging, and eyes open/closed). This selection strengthens the generalizability of findings.  \n- Baselines are well-chosen (EEGNet, EEGInception). These are standard architectures for EEG classification.  \n- Fine-tuning setups are described in sufficient detail, including the rank hyperparameters for LoRA, dropouts, and parameter counts.\nOverall, the experiments are well-structured.", "relation_to_broader_scientific_literature": "- The paper positions LBMs alongside established large-scale modeling trends in NLP and computer vision, citing GPT-like training (NeuroGPT) and codebook pre-training (LaBraM).  \n- The discussion references prior EEG-based deep learning approaches (EEGNet, EEGInception) and prior BCI tasks. This frames their contributions within a standard BCI pipeline and highlights the difference between specialized vs. foundation models.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:  \n  - Thorough experimental design with multiple downstream tasks.  \n  - Clear demonstration of how LoRA can be systematically applied in EEG foundation models.  \n  - Useful ablation studies clarifying which layers benefit most from adaptation.  \n\nWeaknesses:  \n  - The performance improvement of LBMs over smaller baselines is minimal, raising questions about practical benefits.", "other_comments_or_suggestions": "- It may be helpful to elaborate on interpretability aspects of LBMs vs. smaller EEG models, especially given the importance of model explainability in BCI applications.", "questions_for_authors": "- Are there specific EEG paradigms where LBMs offer bigger gains, or is the 0.5% improvement consistent across tasks?  \n- How sensitive are LBMs to the choice of pre-training datasets? Could incorporating additional diverse EEG tasks in pre-training yield larger improvements?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n0awO8K3Cp", "reviewer": "Reviewer_MQLQ", "summary": "In this paper, the authors compare state‐of‐the‐art Large Brainwave Foundation Models (LBMs) with traditional deep learning baselines on multiple EEG‐based tasks and find only marginal accuracy gains despite a massive increase in parameters. They then apply Low‐Rank Adaptation (LoRA) to substantially reduce trainable parameters without sacrificing performance and show that adapting multiple model components simultaneously (e.g., convolutional, fully connected, and attention layers) yields the greatest benefit.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their thoughtful review and valuable feedback. As mentioned in our response to Reviewer QMtY, we have decided also to extend our analysis results to capture one more popular BCI paradigm, namely Motor. Therefore, we added a new movement benchmark based on the High Gamma dataset (Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(11):5391–5420, 2017). Please see our response to Reviewer QMtY for the updated results Table. This extends our thorough analysis even further as mentioned by the current Reviewer MQLQ.\n\nThe performance difference between NeuroGPT (the best foundation model in the new analysis) and EEG-Inception (the best baseline network) is 1.2\\% (improvement compared to the previous 0.5\\%) but with a considerable increase in the number of trainable parameters and larger std among folds. As highlighted by the reviewer, although we demonstrate 10‐fold cross‐validation performance, the original manuscript does not show whether that small improvement margin of 1.2\\% is statistically significant or not. For that reason, we conducted paired-t tests between EEGInception (the best baseline network) and all examined foundation models. The results are demonstrated below, showing a clear statistical significance for NeuroGPT in many benchmark tasks: \n\nP-values of paired-t tests between EEGInception and finetuned foundation models. Bold values indicate statistically significant result $(p < 0.05)$\n| Models                                  | Motor          | ERP            | Memory        | Sleep         | Eyes          |\n|-----------------------------------------|---------------|---------------|--------------|--------------|--------------|\n| EEGInception / LaBraM                   | 0.5860        | **0.0123**     | 0.1090       | 0.2995       | 0.4468       |\n| EEGInception / NeuroGPT (full model)    | **0.0314**    | **0.0401**     | **0.0041**   | 0.2226       | 0.8979       |\n| EEGInception / NeuroGPT (Encoder)       | **0.0072**    | **0.0051**     | **0.0399**   | **0.0495**   | 0.1056       |\n\nWe agree with the reviewer \"brain foundation models\" could refer to models on fMRI as well therefore we have used the term \"Brainwave Foundation Models (LBMs)\", clearly hinting the EEG modality.\n\nIn the camera-ready version of the manuscript, we will incorporate the paired-t test analysis. We sincerely thank the reviewer for their valuable comments, which have helped us to improve our work.", "claims_and_evidence": "The authors make a strong claim that Large Brainwave Foundation Models (LBMs) offer only a marginal improvement over traditional deep nets in EEG tasks, but they never show whether that small margin is statistically significant. Although they do 10‐fold cross‐validation, there is no clear significance testing or confidence intervals for the performance numbers they report.\n\nIt’s difficult to conclude if the LBMs are truly better or if the difference is just random variation.", "methods_and_evaluation_criteria": "The comparison and evaluation in this paper are quite standard. However, significance testing was missed, which seriously undercuts the authors’ key assertions.", "theoretical_claims": "There is no theoretical claims in this paper.", "experimental_designs_or_analyses": "The comparison in the current version is not valid since there is no significance test, this is especially important when the increase is marginal like 0.5%", "relation_to_broader_scientific_literature": "This paper appears to be the first to systematically compare large‐scale, pre‐trained EEG foundation models against conventional deep learning baselines in BCI tasks. While earlier studies have applied large‐model concepts to EEG or introduced individual LBMs, no prior work has comprehensively benchmarked multiple LBMs and standard architectures side by side.", "essential_references_not_discussed": "The reviewer is not aware of any missing reference.", "other_strengths_and_weaknesses": "The key problem of this paper is that there is no significance test. With statistical test added, this paper could be a good contribution to this field.\n\nThe authors are suggested to revise the title to specify EEG modality, since there are also brain foundation models on fMRI.", "other_comments_or_suggestions": "N/A.", "questions_for_authors": "Could authors add statistical test to all the comparisons provided?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gz0X03uuFs", "reviewer": "Reviewer_bhSh", "summary": "The authors performed the analysis of the current Brainwave Foundation Models. The results indicate that current LBMs show limited improvement over traditional deep-learning models. The authors further introduced LoRA for the fine-tuning of LBMs. LoRA fine-tuning technique can sufficiently reduce the training parameters and training efficiency when applying LBMs to downstream tasks.\n\n## After rebuttal \nThank the authors for their responses. I am inclined to keep my score, considering the technical contribution aspect.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their thoughtful review and valuable feedback. As mentioned in our response to Reviewer QMtY, we have decided also to extend our analysis results to capture one more popular BCI paradigm, namely Motor. Therefore, we added a new movement benchmark based on the High Gamma dataset (R T Schirrmeister et al, Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(11):5391–5420, 2017). Please see our response to Reviewer QMtY for the updated results Table. This extends our thorough analysis even further as mentioned by the current Reviewer bhSh.\n\nThe performance difference between NeuroGPT (the best foundation model in the new analysis) and EEG-Inception (the best baseline network) is 1.2\\% (improvement compared to the previous 0.5\\%) but with a considerable increase in the number of trainable parameters and larger std among folds. As highlighted by the reviewer, although we demonstrate 10‐fold cross‐validation performance, the original manuscript does not show whether that small improvement margin of 1.2\\% is statistically significant or not. For that reason, we conducted paired-t tests between EEGInception (the best baseline network) and all examined foundation models. The results are demonstrated below, showing a clear statistical significance for NeuroGPT in many benchmark tasks: \n\nTable: P-values of paired-t tests between EEGInception and finetuned foundation models.\nBold values indicate statistically significant results $(p < 0.05)$.\n| Models                                  | Motor          | ERP            | Memory        | Sleep         | Eyes          |\n|-----------------------------------------|---------------|---------------|--------------|--------------|--------------|\n| EEGInception / LaBraM                   | 0.5860        | **0.0123**     | 0.1090       | 0.2995       | 0.4468       |\n| EEGInception / NeuroGPT (full model)    | **0.0314**    | **0.0401**     | **0.0041**   | 0.2226       | 0.8979       |\n| EEGInception / NeuroGPT (Encoder)       | **0.0072**    | **0.0051**     | **0.0399**   | **0.0495**   | 0.1056       |\n\nAs suggested by the reviewer, we conducted another study: fixed LBMs but learnable classifiers. The results are shown in the table below:\n\nTable 2: Classification accuracy of foundation models where all parameters except classification heads are frozen.\nEach model was trained for 20 epochs with 10-fold cross-validation.\n**Bold values indicate the best performance per task or overall.**\n| Model                   | Motor        | ERP          | Memory        | Sleep        | Eyes         | Mean Accuracy |\n|-------------------------|-------------|-------------|--------------|-------------|-------------|---------------|\n| LaBraM                 | 0.297        | **0.884**   | **0.670**    | **0.608**   | 0.717       | 0.635         |\n| NeuroGPT (full model)  | 0.366        | **0.884**   | 0.656        | 0.597       | 0.734       | 0.647         |\n| NeuroGPT (encoder)     | **0.431**    | 0.883       | 0.655        | 0.602       | **0.746**   | **0.663**     |\n\nFrom the results above, training just the classification yields models that lack behind baselines by a large margin almost 8-10\\% (comparing with the results in our response to reviewer QMtY). This further demonstrates that current brainwave foundation models lack essential elements to fully capture the diverse nature of EEG and largely outperform all current state-of-the-art baselines in various tasks with minimum required fine-tuning. This constitutes full-model fine-tuning necessary and in turn makes PEFT techniques like LoRA extremely valuable.\n\nThe classifiers in both foundation models were designed based on the implementations of the original foundation models papers (LaBraM and Neuro-GPT). As mentioned in our response to Reviwer 9Has, both foundation models (LaBraM and Neuro-GPT) have in their pre-training datasets paradigms that include motor-, ERP-, sleep- and eyes-related tasks (not the specific datasets chosen for the downstream tasks). Interestingly, in the memory task—which was not explicitly included in the pre-training datasets—baseline models slightly outperform foundation models.\n\nWe thank the reviewer for the additional references provided. In our studies, we included the two state-of-the-art open-source EEG foundation models of that time. The trained weights of CBraMod and NeuroLM were released close to the ICML submission deadline. Including these models in our analysis is not possible due to rebuttal's time constraints. But we will definitely acknowledge these recent works in the background section of our revised manuscript.\n\nIn the camera-ready version of the manuscript, we will incorporate the paired-t test analysis, the frozen foundation models analysis and the aforementioned discussion points. We sincerely thank the reviewer for their valuable comments, which have helped us to improve our work.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "The preprocessing step makes sense, but what's the variation of the accuracy in Table 1? Does the result show any statistical significance?\n\nThese LBMs can be applied to different tasks by only training the classifier. The parameters for the classifier would be much smaller compared to fine-tuning the whole model. It would be good to show the performance with fixed LBMs but learnable classifiers. This could be helpful to demonstrate the necessity of the LoRA finetuning.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The authors provide a detailed analysis of the number of ranks, the layer types that apply LoRA, and the dropout parameter. However, the necessity of LoRA itself is not demonstrated, especially when compared with the case; only the classifier is trainable.", "relation_to_broader_scientific_literature": "It is important to study how to apply the existing foundational models, such as LaBraM and NeuroGPT, for various downstream tasks.\nThe authors leverage LoRA. for fine-tuning which is an existing technique.", "essential_references_not_discussed": "Additional models for EEG training, such as BIOT [1] and EEGformer [2], can be included in the study. \nNewer ones, such as CBraMod [3] and NeuroLM [4], are also worth discussing. \n\n\n[1] Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. In Advances in Neural Information Processing Systems, volume 36, pp. 78240–78260, 2023.\n[2] Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. In Advances in Neural Information Processing Systems, volume 36, pp. 78240–78260, 2023.\n[3] Wang, Jiquan, et al. \"CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding.\" arXiv preprint arXiv:2412.07236 (2024).\n[4] Jiang, Wei-Bang, et al. \"NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals.\" arXiv preprint arXiv:2409.00101 (2024).", "other_strengths_and_weaknesses": "The paper is well-structured and easy to understand. \n\nHowever, the necessity of the LoRA for current LBMs should be more clearly demonstrated. Also, it would be good to have a comprehensive discussion about the different LBMs. \nAlso, a more technical contribution is expected.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "Why the classifier architectures connected with LaBraM and NeuroGPT are different?\nIs there any overlap in the dataset between the ones used in the pertaining and the downstream tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a mainly empirical contribution by systematically evaluating the capabilities of Large Brainwave Foundation Models (LBMs), in comparison to traditional deep learning models for several EEG-based BCI tasks (e.g., sleep stage classification, memory tasks), revealing only marginal improvements in accuracy despite a substantial increase in parameters. One key observation from a technical aspect follows the use of low-rank adaptation (LoRA) for parameter-efficient fine-tuning of LBMs (specifically for LaBraM and NeuroGPT), which significantly reduces the number of trainable parameters. The study highlights that simultaneously adapting multiple LBM network components provides the greatest benefit across multiple BCI tasks, suggesting that LBMs require domain-specific strategies and architectural redesign to effectively leverage their potential.

The paper went through a productive rebuttal phase. One of the main concerns was the limited methodological contribution of the paper, since LoRA is a well-known technique which is being used in this work, and whether its necessity was justified at all. The authors addressed the issue of necessity justification with additional experimental validations, which appeared to be sufficient. Another discussed limitation was about the lack of statistical significance tests on the accuracy comparisons and claims, which the authors clarified with additional results during the rebuttal. Authors also included new results on the High Gamma dataset to successfully extend their results with generalizability to another common BCI paradigm (i.e., motor imagery). In general, the AC acknowledges that the rebuttal was effective, although some reviewers maintained diverse ratings. Overall, the paper presents an interesting contribution and is of interest for the EEG-based BCI community. Thus, the AC recommends acceptance of this paper, if there is room in the program.
Decision: Accept (poster)