Is Limited Participant Diversity Impeding EEG-based Machine Learning?
Authors: Philipp Bomatter
Henry Gouk
Venue: NeurIPS 2025 poster ()
Keywords: EEG
electroencephalography
scaling behaviour
distribution shift
augmentation
self-supervised learning
Primary area: neuroscience_and_cognitive_science
TL;DR: We investigate how EEG model performance scales with participant count and total sample size, and assess the impact of data augmentations and pre-training across different data regimes.

Abstract: The application of machine learning (ML) to electroencephalography (EEG) has great potential to advance both neuroscientific research and clinical applications. However, the generalisability and robustness of EEG-based ML models often hinge on the amount and diversity of training data. It is common practice to split EEG recordings into small segments, thereby increasing the number of samples substantially compared to the number of individual recordings or participants. We conceptualise this as a multi-level data generation process and investigate the scaling behaviour of model performance with respect to the overall sample size and the participant diversity through large-scale empirical studies. We then use the same framework to investigate the effectiveness of different ML strategies designed to address limited data problems: data augmentations and self-supervised learning. Our findings show that model performance scaling can be severely constrained by participant distribution shifts and provide actionable guidance for data collection and ML research. The code for our experiments is publicly available online.

Is Limited Participant Diversity Impeding EEG-based Machine Learning?
Philipp Bomatter
Henry Gouk School of Informatics University of Edinburgh henry gookkyeg ac. uk
School of Informatics University of Edinburgh philipp.. bomatter@ed ac uk
Abstract
The application machine learning (ML) to electroencephalography (EEG) has great potential to advance both neuroscientific research and clinical applications. However, the generalisability and robustness of EEG-based ML models often hinge the amount and diversity of training data. It is common practice to split EEG recordings into small segments, thereby increasing the number of samples participants.
substantially compared to the number of individual recordings We conceptualise this as a multi-level data generation process and investigate the scaling behaviour of model performance with respect the overall sample size and the participant diversity through large-scale empirical studies. We then use the same framework to investigate the effectiveness of different ML strategies designed to address limited data problems: data augmentations and self-supervised learning. Our findings show that model performance scaling can be severely constrained by participant distribution shifts and provide actionable guidance for data collection and ML research. The code for our experiments publicly available online
Introduction
century after the first human electreencephalogramm was recorded in 1924, Electroencephalograppyy (EEG) many ways still the best available technique for the direct and non-invasive measurement of brain activity. In particular, with newer generations of dry-electrode recording hardware, it has unique of desirable properties, including its low cost and patient burden, portability, and high temporal resolution. Given the complexity and high dimensionality of EEG signals, machine learning (ML) has shown considerable promise in advancing neuroscientifi research and clinical applications 3 4. However, the successful deployment of ML in EEG-based applications is not without challenges. fundamental obstacle the limited availability of high-quality EEG data, which is often sparse, noisy, and costly to collect.
While prior works have acknowledged limited data challenge for EEG-based machine learning 6).6) systematic studies are currently lacking. In particular, an aspect that has largely been ignored is the hierarchical nature of the data generation process relevant EEG machine learning work. EEG- based ML models rarely operate on full recordings; samples usually correspond to more manageable segments on the order of few seconds, extracted from recordings through sliding window approaches [5]. As a result, the overall sample size is usually considerably larger than the number of individual participants in the dataset. Many applications, particularly
diding-wwndow
medical context, require models to generalise to new participants c.g., a diagnostic model is of little use if it can only diagnose patients the training data. Consequently, there is significant distribution shift between participants,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

low participant diversity covul could impede model performance even with a large overall sample size. Understanding the relevance of participant diversity is critical to guide data collection and future machine learning work. If models generalise well across participants, data could be collected from smaller cohorts, significantly reducing costs and logistical challenges. Conversely, if participant diversity is crucial, this motivates not only higher participant counts during data collection, but also the development of machine learning strategies that explicitly address participant distribution shifts.
We address these challenges by systematically investigating the data scaling behaviour of different EEG machine learning models (TCN, mAtt, LaBraM) across a range of large datasets (TUAB, CAUEEG, PhysioNet), each with more than 1,000 participants, and three tasks (EEG normality prediction, dementia diagnosis, sleep staging). Crucially, we control both the number of participants in the training data and the amount of data per participant, allowing us to disentangle the impact of participant diversity from the overall sample size. Furthermore, we assess the utility of existing data augmentations (AmplitudeScaling, FrequencyShift, PhaseRandomisation and a pre-trained foundation model (LaBraM) for varying amounts of (downstream) training data. Specifically, we quantify their effectiveness in data-limited regimes, whether they continue to provide value as the dataset size increases, and how they address specific limitations, such as low participant diversity versus limited data per participant.
In summary, we formalise the data generation process for EEG-based machine learning and use recent, relevant advances in statistical learning theory to inform our experiment designs and answer the following research questions:
Q1: Does a lack of participants in EEG datasets substantially limit the performance of machine learning models?
Q2: Is collecting more data per participant reliable way to compensate for lack of participants?
Q3: Can contemporary EEG data augmentations effectively increase performance across different data regimes, including limited participant counts or data per participant?
Q4: Can self sepe-supee wre-trainnn effectively increase performance across different data regimes, including limited participant counts or data per participant?
Our findings provide useful insights to guide data collection and the development of EEG-based machine learning methods.
2
Related Work
EEG Data Scaling Studies Banville et al. have studied scaling laws for image decoding from brain activity measured with EEG, MEG, or fMRI [7]. A critical difference to this work is that they trained participant-specifi models, where data from the same participant is used during training and test time and models do not have to generalise unseen participants. Our work is focused settings where generalisation to new participants is crucial, such as applications like diagnosis or prognosis. Kiessner et al. specifically investigated the scaling behaviour of convolutional neural networks for EEG normality prediction on the TUAB dataset and presented scaling laws for model and data size D.. While their experiments controlled for data size in terms of the number of recordings, we aim to disentangle the overall sample size, controlling for both the number of participants and the amount of data per participant. This allows us study the relevance of participant distribution shifts and the need for sufficient participant diversity in datasets used for model training. Our conclusions are also more broadly applicable, as we study different tasks, datasets, and more diverse model architectures.
EEG data augmentations Several works have proposed and employed EEG data augmentations (6) 11 12 13 14). Mohsenvand et al. introduced several augmentations in the context of contrastive
learning approach for EEG (12). Rommel et al. henchmarked various EEG data augmentations on sleep staging dataset with 78 participants and motor imagery task with 9 participants (10). We build on their work, including two promising augmentations from Rommel et al. and one from Mohsenvand al. in our experiments. Our work is different in that we specifically investigate
the utility of data augmentations in dealing with limited participant counts and limited data per
In this study. 'participant diversity' refers to the number of individuals contributing data to the training dataset, with higher diversity corresponding to greater number of unique participants.

---

participant. Furthermore, our experiments were conducted on much larger datasets with over 1,000 participants each, allowing us to study the impact of augmentations across a broader range of data regimes. Finally, their experiments were limited to a single model per task, whereas we henchmarked all augmentations with different model architectures for each task and observed differential effects.
Self-supervised pre-training Recently, lot of work has also been published on pre-trained EEG (foundation) models [LS 16
18 191, most of which is based on transformer models operating on patches that correspond to channel-wise EEG segments and masked token prediction. Our experiments include LaBraM as state-of-the-are art example of these approaches and provides an independent evaluation of the utility of self-supervised pre-training across different data regimes. Furthermore, our aim was again to disentangle the effect of participant counts vs overall sample size.
17
3 Methods
We model datasets used for EEG-based machine learning by the hierarchical data generation process illustrated in Figure [AA. A dataset comprises recordings from n different participants, which are in turn segmented into m distinct samples per participant. As result, the overall sample size usually substantially larger than the number of individual participants from which the samples are obtained. Formally, we can denote the sample features by Xiy where i = I...., 7 indexes the participants, and j = 1, ..... indexes the samples for participant i, and with corresponding ground-truth labels denoted by Yij. For each participant, these samples follow a participant-specife distribution P.. which is itself drawn from a population-level distribution Q. We use a loss function, E(Y, Y). to measure the quality of prediction, Y = s(X), provided by some model, f. This could be, e.g. the zero-one loss or the cross entropy loss, and need not be the loss actually used to train a model. We can define the empirical risk (measured on the training set) and population risk, respectively, as
m
r()
(((())) and R(S) = (X)X(()
7I
In the classic learning setting, where each P; is assumed to be the same, it is well established in the statistical learning theory literature that the gap between empirical risk and population risk (i.e., the extent to which a model will overfit) scales é(1/(mm)))) where p ( 1} (2)). The value of p will be unless some assumptions are satisfied relating to the suitability of the model architecture and the amount of information the features contain about the labels (2I]. The two-level data generation process we consider is comparatively less studied, but is known that with minimal assumptions about model choices and feature information content, the gap will scale as @(1/n² + +1/(mm) 1 (22). We include the second term, which is asymptotically dominated by the first, because if the distribution shift is small, or one able to select a model architecture with substantial robustness to distribution shifts, then the second term can be larger in the non-asymptotic regime (22). This analysis tells us that in the presence of a substantial distribution shift, the extent to which one should expect to overfit is governed primarily by the number of participants, rather than the amount of data available per participant, unless one is able to design a model architecture with the appropriate invariances. Under this framework, we address the four research questions outlined in the introduction. We assess the relevance and magnitude of participant distribution shifts through the scaling behaviour of model performance (Q1 & Q2) and the effectiveness of augmentations (Q3) and self-supervised pre-training (Q4) to improve performance by endowing models with useful invariances or boosting effective participant counts.
3.1 Experimental Design and Model Training
We conducted data scaling experiments to empirically assess the dependence of ML model perfor- mance on the participant count n and the number of samples per participant m across three large datasets and prediction tasks (see Section (.2). Each dataset was split into train, validation, and test splits with no participant overlap. Subsequently, we randomly subsampled training datasets with a fixed number of participants and overall sample size from the train split as visualised by the grid depicted in Figure BB. Different MI. models (see Section 3.4) were then trained on the subsampled training data (see Appendix A. for details and hyperparameters) Early stopping (on the
3

---

A
Number of Partic/pants
800
Datasets
TLaB
Dataset
subset of the Plys otter
400
4000
8030
Normal ction
Diagness
Participants
Sleep Stagiing
100
1000
2000
4000
8000
Models
40
mAHt
Segments
Segments per Participant
Figure 1: (A) Multi-level EEG data distribution. A dataset contains EEG recordings from multiple participants, which are divided into smaller segments to train machine learning models. (B) Grid visualising the subsampling of training datasets controlling for both the number of participants and the number of segments per participant. (C) The datasets, tasks, and models used in the experiments.
fixed, shared validation set) used to prevent overfitting with limited amounts of training data, while still allowing training to converge for larger training datasets. To obtain uncertainty estimates, each trial-i.e. each combination of participant count and number of samples per participant- repeated multiple times with different random seeds for the training data subsampling. Finally, all trained models were evaluated on the same test data. To further study different strategies deal with limited data, we repeated the same procedure for each studied method designed to address a lack of data (data augmentation: see Section 3.5 supe-super learning: see Section 3.6).
3.2 Data and Tasks
TUAB
The Temple University Hospital Abnormal (TUAB) EEG Corpus 23 24
TUH EEG Corpus [25], was collected a hospital environment and is provided with labels about the normality of the EEG according to standardised criteria (26). The full dataset contains more than 2300 participants with most recording durations the range 15-25 minutes. For the current study, used version 3.0.1 of the dataset. Data access can be requested from the dataset authors.
The Chung-Ang University Hospital EEG (CAUEEG) dataset 27 comprises data from 1379 participants with most recording durations in the range of 20 minutes. In the current study, we used a subset of data from 1122 participants for which diagnostic labels for the categories normal, mild cognitive impairment (MCT), or dementia were available. Diagnostic labels were assigned by neurologists based on neuropsychological examination. Data access for academic and research purposes can be requested from the dataset authors.
CAUEEG
PhysioNet The dataset referred as PhysioNet in this study corresponds the data used in the PhysioNet/Computing in Cardiology Challenge 2018 (28 29 While the official challenge was focused on the classification of arousal regions, we focused on sleep staging here. The dataset com- prises polysomnographic (PSG) recordings along with sleep stage annotations for non-overlapping 30-second epochs following AASM standards. While the PSG recordings contained additional measurements such electrooculography (EOG), electromyography (EMG), or airflow and oxygen saturation, only the six EEG channels were used this study. Furthermore, we only used the training data of version 1.0.0 of the dataset, annotations for the official test split were not made publicly available. The data available under the Open Data Commons Attribution License (ODC-By) v1.0.
3.3 Preprocessing
Preprocessing of the data comprised the following steps: Band-pass filtering to 1-50 Hz and resam- pling to a sampling frequency of 100 Hz, cropping (to 15 minutes for TUAB, omitting the first 30 seconds due to frequent strong artefacts the beginning of recordings; to 5 minutes for CAUEEG), selection a fixed channel subset (the 19 10-20 channels for TUAB and CAUEEG; the 6 EEG channels for PhysioNet), re-referencing to average reference, and segmentation into non-overlapping epochs (2 seconds for TUAB and CAUEEG; 30 seconds for PhysioNet).
For compatibility with the pre-trained LaBraM model, the following adjustments were made for the training and fine-tuning LaBraM: amplitudes scaled units of mV, sampling frequency of 200
Hz, filtering 0.1-75 Hz with notch filter at 50 Hz (see [15 We note that the notch filter was

---

not matched to the line noise of the data in the original publication (e.g. LaBraM was also evaluated on TUAB, where line noise is 60 Hz). For compatibility with the pre-trained weights, we did not correct this potentially suboptimal setting. Furthermore, since LaBraM only supports segment lengths up to seconds without modification (due to the implementation of the temporal positional encodings), we used 15 second epochs instead of 30 second epochs for PhysioNet.
3.4 Models
We selected three influential and widely used models with distinct architectures to obtain a diverse perspective EEG model behaviour and reduce the risk of model-specific effects.
TCN
The temporal convolutional network (TCN) architecture was originally proposed by Bai, Kolter, and Koltun [30] and has since been adapted and widely used for EEG data (.). The model consists of a stack of ID causal convolution layers. We used the braindecode implementation [31] with the following hyperparameters: n n_blocks=4, n filtter==64, kernel_size-t dtrp_ppro==0.
mAtt The manifold attention network (mAtt) is a geometric deep learning model developed for EEG data (32]. Briefly, the architecture starts with two convolutional layers and then encodes segments in the input EEG into sequence of covariance matrices. The diagonals these covariance matrices correspond to the signal power in each channel. The architecture then makes use of the fact that the covariance matrices are symmetric positive definite (SPD) through a manifold attention module, where instead of dot product attention, an approximation of the Riemannian distance is used. The output is then transformed back to Euclidean space and passed through fully connected layer to get classifications. We adapted the code from the official mAtt repository (https:/github..cm hwtcclhuuu.oomcCCC wwwC//C/E//AAIIIIII and used 100 kernels conv 50 kernels in conv2 with a kernel size of 11, and embedding size of 25 for the SPD transforms.
LaBraM The Large Brain Model (LaBraM) architecture was designed by Jiang, Zhao, and Lu [1]. The model operates on patches corresponding to channel-wiss time segments. Patches are first passed through a small convolutional encoder and spatial and temporal positional encodings are added. The resulting tokens are then processed through a stack of transformer encoder layers and
average pooling is used to aggregate the output tokens, followed by a classification head. We used the LaBraM Base model with the recommended default hyperparameters.
3.5 Augmentations
We selected three promising augmentations based on prior work (see Appendix A.2 for details).
Amplitude Scaling The AmplitudeScaling data augmentation scales (multiplies) each EEG channel by a random factor E [0.5, 2). This augmentation was proposed by Mohsenvand, Izadi, and Maes and used for contrastive learning (12). They consulted four neurologists to identify augmentations that would not change the interpretation of the data along with recommended ranges for the parameters (min and max scaling factor in this case; we used the same settings in our experiments).
Frequency Shift FrequencyShift was introduced in [IO] and further evaluated in (6). This
augmentation introduces small shift in the frequency domain, i.c. the power spectrum is shifted by some random frequency Af. The same shift is applied to all EEG channels. Rommel et al. tuned the hyperparameter defining the maximum shift for a sleep staging task and achieved the best results with Afmaz 0.3 Hz. We used the same parameter in our experiments.
Phase Randomisation Schwabedal et al. introduced the randomisation of Fourier transform phases for data augmentation under the name of "FT surrogates" [13]. We refer to this augmentation as PhaseRandomisation. Rommel et al. evaluated different hyperparameters for the maximum phase shift and observed the best results with high values (6). We adopt the maximum range (Aymma 2T), which corresponds to complete randomisation of the phase information. We also sample different phases for each EEG channel, the setting that Rommel et al. used for their sleep staging experiments.
3.6 Pre-training
The LaBraM model (see Section 3.4) was developed foundation model for BEG data and pre- trained to predict discrete tokens, previously learned by a separate tokeniser that was itself trained to predict the Fourier spectrum (see the original publication for details (15). The model was pre-trained
5

---

Normality Prediction
Dementia Diagnosis CAUEEG
Sleep Staging PhysioNet
TUAB
0.85
Model:
0.80
0.55
TCN
0.6
0.75
LaBraM
0
500
1000
1500
0
200
400
600
800
200 400
900
800
Number Participantt
Figure Scaling behaviour of model performance. Average accuracies of the different models for normality prediction TUAB, dementia diagnosis CAUBEG, and sleep staging PhysioNet for increasing participant counts at fixed number of segments per participant (m 40), Averages were computed across seeds used to subsample the training datasets and the shading illustrates the standard error of the mean. Across all datasets, performance increased strongly as the size was increased to several hundred participants, after which improvements started to diminish.
in unsupervised manner collection of 16 datasets with total of around 500 participants, and the weights for the smallest version (LaBraM Base) were made publicly available by the authors. We used these weights for experiments with the pre-trained LaBraM model.
The model was then fine-tuned using the labelled training data of the corresponding task. We fine- tuned the entire architecture without freezing any layers and determined the learning rate through hyperparameter optimisation. Specifically, we evaluated grid learning rates [1c-7, 5c-7, 1e-6. le-2] for each dataset (TUAB, CAUEEG, PhysioNet) using the respective training and validation splits. To ensure that performance differences between the pre-traine pre-trained and baseline (trained from scratch) models can be attributed to pre- me-ttrining rather than tuning of the learning rate, also tuned the learning rate for the baseline in the same way. T the selected learning rates.
Results
4.1 Scaling the Number of Participants
Q1: Does lack of participants in EEG datasets substantially limit the performance of machine learning models?
Figure depicts the model performance the number of participants in the training data is varied while keeping the number of segments per participant fixed. We observe strong improvements in accuracy as the participant count increased from 25 several hundred participants across all datasets and models, after which accuracy improvements diminish. While performance appears not to have fully saturated across the three tasks, meaningful further improvements through the collection of additional labelled training data from new participants is likely infeasible. For example, recruiting an additional 1, 1,600 participants to double the number of participants in the TUAB experiments would often be prohibitively expensive for an expected improvement in performance of roughly 1%. To put this into context, Roy al. reported the sizes of datasets used across 154 studies on deep learning- based EEG analysis
(5). Astonishingly, half of the datasets contained data from less than 13 participants and only six comprised more than 250 participants. Although this might have changed since the review's 's publication in 2019, according to our experimental results, substantial proportion of EEG datasets fall into the range where a scarcity of participants is likely a critical limiting factor. In such cases, substantial gains could be achieved through the collection of additional training data from more participants. Differences between model architectures are discussed in Appendix AA.5 Overall, accuracies were relatively similar across architectures and the sample size was the primary driver of performance.
A1: Yes. Across all datasets, tasks, and models, performance improved substantially as the participant count was increased to several hundred participants. The point where performance began to saturate was substantially higher than the number of parlicipants seen in many EEG datasets, indicating that collecting data from more participants could lead to non-trivial gains in performance many typical contexts.

---

Normality Prediction TUAB
Dementia Diagnosis CAUEEG
Sleep Staging PhysioNet
Samppl 5.50
3800
6.04
0.31
3.05
2,55
500
Number Participants
66 9011
+84633
404031
+34.03
+3/033
400000
400
0:81
8.81
0185
3.83
0:91
0.86
6:07
60
0.84
3.09
0.10
0.71
5.73
0.16
0.71
200
0:61
6:33
8.00
0::5
6-81
3.81
8:53
0.54
6:03
0.61
0:61
3:67
0.04
0.70
0.71
0101
+36735
0111
+1000
WORHT
300
6:73
8-21
0.51
0:31
0:81
0.87
0.23
50
6.78
6:76
9:77
E:79
1.74
8:57
3.48
0:40
0:67
0.50
0:37
3
0.53
0.94
0.55
0:01
DUAIN
22084
2MLL
mm
50000
-810TS
13080
10
20
48
80
180
520
30
20
40
5G
10
2U
40
50
100
320
640
Number Segments per Participant
Figure 3: Differential effects of participant count and overall sample size on model perfor- mance. Accuracies of the TCN model for normality prediction on TUAB, dementia diagnosis on CAUEEG, and sleep staging on PhysioNet averaged across seeds, along with the standard error of the mean. Performance on TUAB and CAUEEG was dominated by the participant count. Sleep staging performance on PhysioNet was more dependent on the overall sample size and competitive performance was achieved even with very limited participant counts.
4.2 Scaling Segments per Participant
Q2: Is collecting more data per participant a reliable way to compensate for lack of participants?
Siguurajillusrattes the performance of the TCN model when varying both m and n. Additional results using the mAtt and LaBraM models are provided in Appendix AA8 where we observe qualitatively similar trends. Focusing first on normality prediction on TUAB, we observe that increasing the amount of data for fixed participant count (moving along the horizontal axis) did not yield any substantial performance improvements. As already observed in Section 4.1 performance was highly dependent on the participant count (moving along the vertical axis). Even for fixed sample size, higher participant diversity yielded significant improvements (moving diagonally from bottom right top left). The results for the CAUBEG dementia prediction task exhibit similar pattern. Linking back to the theoretical analysis in Section 3.1 this suggests a substantial distribution shift between participants such that the participant count n dominated model generalisation.
We observe a different trend for PhysioNet sleep staging: higher participant diversity still yielded some improvements in performance (moving diagonally from bottom right top left), but substantial improvements were also achieved by increasing the sample size for a fixed participant count (moving along the horizontal axis). In fact, with the maximum number of samples per participant, training data from a small cohort of just 25 participants was sufficient to match the performance achieved with training data from 400 participants (with 5 segments per participant).
A2: No. In two of the three case studies that we investigated, scaling the amount of data per participant had minimal impact the performance of the model. The number of participants in the training datasets had a substantially larger impact on model performance.
4.3 Effectiveness of EEG Data Augmentation
Q3: Can contemporary EEG data augmentations effectively increase performance across different data regimes, including limited participant counts or data per participant?
Figure 4 illustrates the effect of different data augmentations (see Section 3.5 on the resulting model performance. With one exception, the impact of the different augmentations was small and inconsistent. No significant associations between sample size and accuracy improvements were observed (see Appendix A.9). What stood out, however, was systematically detrimental effect of the PhaseRandomisstion augmentation on PhysioNet for the TCN and mAtt models, whereas performance for the LaBraM model was improved. LaBraM also benefitted from the FrequencyShift augmentation on PhysioNet. In terms of absolute performance, LaBraM performed substantially worse PhysioNet without data augmentation (see Figure 2), such that with data augmentation,
7

---

Normality Prediction
Dementia Diagnosis CAUEEG
Sleep Staging PhysioNet
TUAB
+30M6
Model:
+10%
TCN
mAz
-10%
-15%
A3
FS PR
Figure 4: Impact of data augmentations. Each boxplot shows pairwise accuracy differences between augmented and unaugmented training (positive means augmentation improved performance) across all combinations of participant counts (n) and segments per participant (m). Results for single seed are shown for better visibilityy. AS = AmplitudeScaling. FS FrequencyShift, PR = PhaseRandomisationn On PhysioNet, the LaBraM model benefitted from FS and PR, whereas PR decreased performance for TCN and mtt. Other than that, augmentations did not lead to consistent improvements in performance.
all three model architectures perform about equally well. The observed performance improvements for LaBraM can therefore likely be attributed its greater reliance on large amounts of data, which is partially offset by augmentation. Decreased performance in the smaller and more data-efficient models suggests that PhaseRandomisation destroys useful information.
The Pandsmaanomisttioon augmentation randomises the phase of each Fourier coefficient and is applied chsnnel-wise. It therefore preserves the power distribution across different frequencies, but completely destroys wave morphologies and cross-channel correlations. Given that certain waveforms highly informative of the sleep stage (e.g. K-complexes known to occur during N2 sleep), the performance decreases should be expected in such situations.
A likely explanation for why AmplitudeScaling and FrequencyShift did not yield consistent accuracy improvements that these augmentations are not truly label preserving. For example, overall ampli- tude and differences in amplitude between EEG channels (e.g. asymmetries between corresponding channels on the left and right hemisphere) can provide clinically relevant information (26). Similarly, frequency shifts can be clinically relevant (e.g. slowing may reflect abnormalities [26) and has been
linked to MCI and AD [33 34). Alternatively, it is possible that even the smallest amount of training
data already provided sufficient diversity of amplitudes and frequency shifts for the model to learn sufficient invariances to these aspects of the data, rendering the augmentations redundant.
A3: No. The augmentations benchmarked in this study failed to consistently improve performance. Small gains were observed for the largest model on a single task, where it still underperformed compared to the smaller models trained without augmentations.
4.4 Effectiveness of Self-supervised Pre-training
Q4: Can self-super self-supervised pre-training effectively increase performance across different data regimes, including limited participant counts or data per participant?
Figure shows comparison of the LaBraM model performance with and without pre-training (see Section 3.6) on the TUAB normality prediction task. Pre-training consistently improved model performance across all data regimes, except when the amount of downstream fine-tuning data was most severely limited. Comparing the baseline trained from scratch models (left heat map) and the pre-trained models (middle heat map), the effect of pre-training looks like a boost in the amount of data per participant: with only 10-20 segments per participant, the pre-trainet model achieved the same performance as the baseline model with the maximum amount 320 segments per participant. However, pre-training also improved performance when the participant count was low and it continued to provide value even when the maximum amount of data was used. Consistent results were observed for dementia prediction the CAUEEG dataset and sleep staging PhysioNet (see Appendix Appendix(A.1)).
In Figure 2 we observed that the LaBraM model required substantially more data to achieve the same performance as the mAtt and TCN models. With pre-training, however, the much larger and more data-demanding LaBraM model was able to match or even outperform them. This can seen by comparing the heatmaps in Figure 5 to the corresponding heatmap TUAB of the TCN model

---

Baseline Accuracy
Pretrained Accuracy
Accuracy Difference
1800
D181
0.
2.111
Number of Parti cipants
1810
0.79
9.50
C-11
0.52
0.53
EAD
0.54
9.54
0.35
3.55
0.54
3.04
D
0.02
2.50
0.03
400
0.39
6:20
8.50
8:88
043
5.04
0.84
0.00
5.14
6103
0.03
5.60
2010
0.77
0.73
8.60
0.82
0.03
5.00
0.11
0.83
3.85
0,83
0.03
9.00
0.04
0.64
03
100
0:35
0:78
6.70
0.79
0.74
6:99
0.79
0.:0
0.44
600000
45 H1
6C 000
44,00
4AMH
S08112
50
0.72
0.75
0.76
0.77
0.31
6:53
0.36
1.76
6.70
0.03
0.66
6022
*6 MTT
MTO
+3:001
25
2017
48 /
800
320
Number Segments
Participant
Figure 5: Effectiveness of self-supervieed pre-training. Comparison of LaBraM performance with and without pre-training on TUAB. The left and middle heatmaps show average accuracies (+ standard eιror of the mean) for LaBraM trained from scratch (baseline) and pre-trained on collection of 16 datasets before fine-tuning on TUAB, respectively. The right heatmap visualises the average of pairwise accuracy differences (# standard error of the mean), where pairs correspond to the accuracies of pre-trained and baseline models for the same seed. Pre-training consistently improved performance across all data regimes, except when the amount of fine-tuning data was most severely limited.
in Figure 3 Alternatively, Appendix A.6 shows version of Figure 2 with additional lines for the pre-trained LaBraM model.
Pre-training exposed the model to a larger diversity of EEG data from additional participants than those seen during fine-tuning. The positive results suggest that, even without labels, this has allowed the model to learn useful representations that transfer to diverse downstream tasks.
A4: Yes. Self-supervised pre-training led to consistent improvements in model performance across all data regimes except when fine when fine-tuming data was most severely limited.
5
Discussion
The present study demonstrates that participant diversity is critical factor for training BEG-based machine learning models. In two out of three tasks studied, collecting more data per participant was not effective to compensate for a small participant count. At the same time, our findings also show that good performance can be achieved with limited participant cohorts in specific applications. Pilot studies can help identifying such cases and inform the extent to which participant diversity should be prioritised in data collection and methodological machine learning work.
The use of large datasets allowed us to study the scaling behaviour of machine learning models over a large range of data regimes. Across all datasets, tasks, and models, we observed strong improvements in performance as the participant count was increased from 25 to several hundred participants. This highlights the value of initiatives like AI-Mind [35], a collaboration of 15 academic and industry partners pioneering a large study involving 1, 1,000 participants with mild cognitive impairment. However, as participant counts approached the maximum sizes explored in our study, performance improvements began to diminish. While the majority of current EEG-based machine learning work still operates on smaller datasets, it is thus possible that further performance improvements through the collection of even larger labelled datasets will become prohibitively expensive. Complementary approaches, such as machine learning strategies explicitly designed to address participant distribution shifts and make more effective use of limited EEG data could become essential.
Based on these findings, we assessed the effectiveness of existing machine learning strategies for dealing with limited participant diversity and data per individual participant. contrast other areas where ML is used, data augmentations did not prove effective as a strategy to deal with limited participant diversity or overall sample size. It is important to note that our experimental setup considered only existing EEG data augmentation methods; we do not exclude the possibility that more effective augmentations can be designed. In fact, our results encourage the development of augmentations that specifically imitate participant differences.

---

Self-supervised pre- pre-training consistently improved model performance across all data regimes. These results are particularly promising considering the broad potential for further improvements: This study benchmarked LaBraM Base (the smallest version of the model), for which weights were made publicly available by the authors. Further improvements are thus likely achievable with increased model size as indicated by comparisons of the model versions in the original publication [15] as well as generally observed trends in self-supervised learning (36). Moreover, the data used to pre-train LaBraM is still relatively limited with total participant count of around 500-less than a third of the maximum number of participants used in our evaluations on the TUAB dataset. The existing body of literature on sulf--uue learning strongly suggests that more data would result in better performance [36]. Since pre- tre-training does not require labelled data, the creation of a larger pre- training dataset is comparatively easy. Prior work has also shown that sule--upe learning can be used to enforce approximate invariances (37), which can help circumvent the poor dependence of overfitting behaviour on the number of participants in the training data [2
6 Limitations and Future Work
While our study provides insights into the scaling behaviour of EEG models, the importance of participant distribution shifts, and the utility of data augmentations and self-supervised learning to address EEG data limitations, several limitations remain that warrant further investigation.
First, our study focuses applications of EEG that require models to generalise to unseen participants. We see this as a particularly important setting, as is essential for most clinical applications such as diagnosis prognosis. However, there are other (BCI) applications where it is common and often practical to train participant-specife models. In concurrent work, Banville al. have studied scaling laws for image decoding from brain activity and observed little gain from increasing the number of participants in the training data, although their experiments were limited to three datasets with only 8-48 participants (7]. Kong al. have observed more promising generalisation to unseen participants with larger datasets (38).
brain-computer interface
Beyond the augmentations and self-supervised learning methods studied in our work, refined prepro- cessing methods could provide another approach to dealing with limited EEG data by improving data quality instead of quantity. There is a rich literature on EEG preprocessing (39 40 41). although most methods were designed for classical EEG analysis and it is unclear how well they translate to ML-based EEG applications. Some work suggests that even with large datasets, machine learning
models still benefit in particular from the removal of high-amplitude artefacts 42 43
Finally, future research should attempt to determine the nature of participant distribution shifts. We observed large participant count to be critical for model performance in EEG normality prediction and EEG-based dementia diagnosis, and we can see several plausible underlying reasons. On the one hand, anatomical individual differences could impede the generalisability of the observed patterns of scalp activity measured with EEG. On the other hand, the distribution shift might also be explained by variability in how the target presents in different participants. For instance, in the EEG normality prediction task, there are numerous reasons why EEG can be abnormal, ranging from focal epileptiform discharges to diffuse slowing. A model that is trained on data from only a small cohort of participants will therefore not only be exposed to little demographic and anatomical variation but also to a limited number of target (or disease) phenotypes. This could be particularly relevant, e.g., in the context of neurodegenerative or psychiatric discases, where substantial individual differences in symptomatology and neural correlates have been described [44] 45).
7 Conclusion
Our study shows that participant diversity a key determinant of EBG-based machine learning perfor- mance, with task-dependent importance. While increasing data per participant cannot compensate for low participant counts most cases, pilot studies can help to identify contexts where smaller cohorts suffice. Furthermore, while performance generally scales with participant numbers, gains diminish at larger scales, suggesting limits to dataset growth as a strategy and motivating more work on machine learning methods to deal with limited EEG data. Particularly self superviedd pre- rre-training provided consistent benefits in our experiments, highlighting its promise alongside approaches designed to explicitly address participant distribution shifts.
10

---

Acknowledgements
This project was supported by the Royal Academy of Engineering under the Research Fellowship programme.
References
[1] Lukas A. W. Gemein et al. "Machine-lcarning-baeee diagnostics of EEG pathology". In: Neurolmage 220 (Oct. 2020), p. 11 117021. ISSN: 1053-8119. DO1: 10. 1016/j neuroimage
2020.. 117021
[2]
Wei Wu et al. "An electmencephaloogappicc signature predicts antidepressant response in major depression", In: Nature Biotechnology 38.4 (Apr. 2020), pp. 439-447. ISSN: 1546- 1696-1696. DOL:
10.1038/ 01110//55587709-9-100
[3] Jin Jing al. "Development of Expert- Lever Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation". In: Neurology 100.17 (Apr. 2023), e1750-e1762.
DOI: 1ww1//111...... 00000000002071227
[4] James Mitchell Crow. "Could biomarkers mean better pain treatment?" In: Nature 633.8031
(Sept. 2024), $28 S30. DOI: 10. 1000//441560002000000000000
[5] Yannick Roy et al. "Deep learning based electroenecephalography analysis: a systematic re- view". In: Journal of Neural Engineering 16.5 (Aug. 2019), p. 051001. ISSN: 1741-2552. DOL:
10. 10.1108//00 11088/1741-1 114111111111 - 2552/5522222 ab260c
[6] Cédric Rommel et al. "Data augmentation for learning predictive models on EEG: systematic comparison". In: Journal of Neural Engineering 19.6 (Nov. 2022), p. 066020. ISSN: 1741 1741-2552.
10. 1088/ 1741 25112/255222/2 aca220
[7] Hubert Banville et al. Scaling laws for decoding images from brain activity. Jan. 2025. DOI:
10. 48550//aXixi 2501. 15322
[8] Ann-Kathrin Kiessner et al. "Reaching the ceiling? Empirical scaling behaviour for deep EEG pathology classification". In: Computers in Biology and Medicine 178 (Aug. 2024), p. 108681.
ISSN: 0010-4825, DO1: 10. 1016/j0-jjj- compbi omed. 2024. 108681
[9] aan-Kathrin Kiessner al. "An extended clinical EEG dataset with 15,300 automatically la- belled recordings for pathology decoding". In: Newrolmage: Clinical 39 (Jan. 2023), 103482,
ISSN: 2213- DOT: 10. 1016/ po016/..mic.. micl. 2023. 103482
[10] Cédric Rommel et al. "CADDA: Class-wise Automatic Differentiable Data Augmentation for
EEG Signals". In: International Conference Learning Representattinss 2022.
[11] Olawunmi George al. "Data augmentation strategies for EEG-based motor imagery de-
coding". In: Heliyon 8.8 (Aug. 2022). ISSN: 2405-8440, DO1: 10. 1016/j hww.yon..nn...... 2022
e10240
[12] Mostafa Neo Mohsenvand, Mohammad Rasool Izadi, and Pattie Maes. "Contrastive Repre- sentation Learning for Electroencephalogram Classification" In: Proceedings of the Machine Learning for Health NeurIPS Workshop. PMLR, Nov. 2020, pp. 238-253.
[13] Justus T. C. Schwabedal et al. Addressing Class Imbalance in Classification Problems of Noisy Signals by using Fourier Transform Surrogates. Jan. 2019. DOI: 10. 48550/ arxiv 1806
08675
[14] Fang Wang et al. "Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks". In: MultiMedia Modeling. Ed. by Klaus Schoeffmann et al. Cham: Springer International Publishing, 2018, pp. 82-93. ISBN: 978-3-319-736006 DOI:
10. 10.1007/978-3-1 311--119977-----
[15] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. "Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCT", In: The Twelfth International Conference on Learning Representations. 2024.
[16] Chaoqi Yang. M. Brandon Westover, and Jimeng Sun. "BIOT: Biosignal Transformer for Cross- data Learning in the Wild". In: Thirth-seventh Conference on Neural Information Processing Systems. Nov. 2023.
[17] Yuqi Chen al. EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model. Jan. 2024. DOI: 10. 48550/ arXiv. 2401, 10278.

---

[18] Jiquan Wang et al. "CBraMod: A Criss- Cross Brain Foundation Model for EEG Decoding". In: The Thirteenth International Conference on Learning Representations. Oct. 2024.
[19] Guagnyu Wang et al. "EEGPT: Pretrained Transformer for Universal and Reliable Representa- tion of EEG Signals". In: Advances in Neural Information Processing Systems 37 (Dec. 2024),
pp.
[20] Steve Hanneke, Kasper Green Larsen, and Nikita Zhivotovskiy. Revisiting Agnostic PAC Learning. July 2024. DO1: 10. 448550/arXia 2407 19777
[21] Steve Hanneke. "The Optimal Sample Complexity of PAC Learning.. In: Journal of Machine Learning Research 17.38 (2016), pp. 1-15. ISSN: 1533-7928.
[22] Henry Gouk et al. On the Limitations of General Purpose Domain Generalisation Methods.
May 2024. DO1: 10. 48550/ wrw...22................ 2w..2000233333 00563
[23] S. López et al. "Automated Identification of Abnormal Adult EEGs". In: IEEE Signal Process- ing in Medicine and Biology Symposium (SPMB) 2015 (Dec. 2015). ISSN: 2372-7241. DOI:
10.1109/ 0009//PMB2222 2015. 7405423
[24] Lopez de Diego and Silvia Isabel. "Automated Interpretation of Abnormal Adult Electroen- cephalograms (2017).
[25]
Iyad Obeid and Joseph Picone. "The Temple University Hospital EEG Data Corpus". In: Frontiers in Neuroscience 10 (May 2016). ISSN: 1662-453X. DOI: 10. w389/f3innnss 2216
00196
[26]
Devon I. Rubin. Clinical Neurophysiology. 5th ed. Contemporary Neurology Ser. Oxford: Oxford University Press USA- OSO, 2021. ISBN: 978-0- 18880-19-006785-4
[27] Min-jae Kim, Young Chul Youn, and Joonki Paik. "Deep learning- EEG analysis to classify normal, mild cognitive impairment, and dementia: Algorithms and dataset".. In: Neu- rolmage 272 (May 2023), P. 120054. ISSN: 1053-8119. 120054
DOI: 10.
2023
[28) Mohammad M. Ghassemi et al. "You Snooze, You Win: the PhysioNet/Computing in Cardiol- ogy Challenge 2018". In: Computing in cardiology 45 (June 2019), 10. 200.2222/cccc.. 8.049. DOL: 2.l..2289//cia cinc. 2018. 049
[29] A. L. Goldberger et al. "PhysioBank, ToosioTo and PhysioNet: components of new research resource for complex physiologic signals". In: Circulation 101.23 (June 2000), E215-
220. ISSN: 1524-4539. DO1: 10. 1161/01. cir. 101. e215
[30] Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An Empirical Evaluation of Generic Convolu- tional and Recurrent Networks for Sequence Modeling. Apr. 2018. DOI: 10. 48550/arXiv
01271
[31] Robin Tibor Schirrmeister et al. "Deep learning with convolutional neural networks for EEG decoding and visualization". In: Human Brain Mapping 38.11 (2017), pp. 5391-5420 ISSN: 1097-0193, DOI: 10. 100./hb0.hb.mmmmm 23730
[32] Yue-Ting Pan, Jing-Lun Chou, and Chun-Shu Wei. "MAtt: A Manifold Attention Network for EEG Decoding". * In: Advances in Neural Information Processing Systems 35 (Dec. 2022),
pp. 31116-31129,
[33] Raymundo Cassani et al. "Systematic Review on Resting-State EEG for Alzheimer's Disease Diagnosis and Progression Assessment". In: Disease Markers 2018.1 (2018), p. 5174815.
ISSN: 1875-8630. DO1: 10. 1155//011/517777
[34] Aslan Modir, Sina Shamekhi, and Peyvand Ghaderyan. "A systematic review and methodolog- ical analysis of EEG-based biomarkers of Alzheimer's disease". In: Measurement 220 (Oct. 2023). P. 113274. ISSN: 0263-2241. DOI: 10. 1016/j measurement 2023. 113274
[35] Ira R. J. Hebold Haraldsen et al. "AI-Mind: Revolutionizing Personalized Neurology Through Automated Diagnostics and Advanced Data Management". In: Drug Repurposing 1 (July
2024). p. 20240005. ISSN: 2941-292222 DO1: 00./10..8877 2B6677/DRUGREOO 24. 1000
[36] Jared Kaplan et al. Scaling Laws for Neural Language Models. Jan. 2020. DOL: 48550/
arXiv 2001.0002333
[37] Linus Ericsson, Henry Gouk, and Timothy M. Hospedales. "Why Do Supervipe Models Transfer? Investigating the Impact of Invariance on Downstream Tasks". In: British Machine
Vision Conference (BMVC). 2022.
[38] Xiangtao Kong et al. "Toward Generalizing Visual Brain Decoding to Unseen Subjects". In: The Thirteenth International Conference Learning Representations. Oct. 2024.
12

---

[39]
Ranjan Debnath et al. "The Maryland analysis of developmental EEG (MADE) pipeline". In: Psychophysiology 57.6 (June 2020), e13580, ISSN: 1469-8986. DoI: 10. 1111/psyp..... 13580
[40] Mainak Jas et al. "Autoreject: Automated artifact rejection for MEG and EEG data". In: Neurolmage 159 (Oct. 2017), pp. 417-429.
ISSN: 1095-9572. DOI: 10. 1016/ www.
2017.06. 030
[41]
Luca Tonachinn,i,n, Ken Kreutz-Delgado, and Scott Makeig. "ICLabel: An automated elec- troemccphalograapiic independent component classifier, dataset, and website", In: Neurolmage 198 (May 2019), 181.
2019. 05..5.5.4 026
[42] Anders Gjølbye et al. "Speed: Scalable Preprocessing of EEG Data for Self-Supervised Learning.. In: 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP). Sept. 2024, pp. 1-6. DOl: 10. 1109 http//92SS552220.22 2024. 10734749
[43] Philipp Bomatter al. "Machine learning of brain-specific biomarkers from EEG". In: eBioMedicine 106 (Aug. 2024). ISSN: 2352-3964. DOI: 10. 1016/j jeni://eeiim....m ebiom........ 2024.022222 105259
[44] Alexandra I.. Young et al. "Uncovering the heterogeneity and temporal complexity of neurode-
generative diseases with Subtype and Stage Inference". In: Nature Communications 9.1 (Oct.
2018), 4273. ISSN: 2041-1724 DOI: 10. 1038/ 011///11167--1----000
[45] Amanda M. Buch and Conor Liston. "Dissecting diagnostic heterogeneity in depression by integrating neuroimaging and genetics". In: Neuropyychophummccolgy 46.1 (Jan. 2021),
pp. 156-175. ISSN: 1740-634X. DOI: 10. 1038/ 020-1088---00--0-9----9-
[46] H Aurlien et "EEG background activity deseribed by large computerized database". In: Clinical Neurophysiology 115.3 (Mar. 2004), pp. 665 673. ISSN: 1388-2457. DOI: 10. 1016/
J J.clinph. 2003. 10. 019)
[47] A. K. I. Chiang et al. "Age trends and sex differences of alpha rhythms including split alpha peaks". In: Clinical Neurophysiology 122.8 (Aug. 2011), pp. 1505-1517. ISSN: 1388-2457.
DO1: 1016// j. clinph 20110 01.0400...00
[48] Dillan Cellier et al. "The development of theta and alpha neural oscillations from ages 24 years". In: Developmental Cognitive Neuroscience 50 (Aug. 2021), p. 100969. ISSN:
1878-9293, DO1: 10. 1ew.1011..... dcn. 2021 100969
[49] Adam Paszke et al. PyTorch: An Imperaative Style, Hligh-Performanc Deep Learning Library. Dec. 2019, DOI: 4.0.48850//50 arxiv. 01703
[50] Alexandre Gramfort et al. "MEG and EEG data analysis with MNE-Python". In: Frontiers in Neuroscience 7 (Dec. 2013). ISSN: 1662-453X. DOI: 3...3399/// $389/fnins. 2013. 00267
[51] Stefan Appelhoff et al. "MNE-BIDS: Organizing clectrophysiological data into the BIDS format and facillitating their analysis". In: Journal of open source software 4.44 (2019), p. 1896.
ISSN: 2475-9066. DOI: 10. 201211100eesee 01896
[52] Zhi Zhang, Sheng-hur hua Zhong, and Yan Liu. "TorchEEGEMO: A deep learning toolbox towards BEG-based emotion recognition". In: Expert Syst. Appl. 249. (Sept. 2024). ISSN: 0957-4174
DOL: 10. 1016/j j. eswa. 2024. 123550
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The formalisation and theoretical analysis of the data generation process is pre- sented in Section Empirical results for scaling behaviours and the utility of augmentations and self-supervised pre-training are presented in Section 4 and discussed in Section 5
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations and future work are discussed Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification:
The theoretical analysis in Section 3 iss based on prior work, which is cited
accordingly.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All experiments and the used data and models are carefully described in Section 3.1 and Appendix A.1 Furthermore, the code is publicly released.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The research code is publicly available, including code and instructions to prepare the data and notebook to reproduce all results. The PhysioNet dataset is publicly available and access to TUAB and CAUEEG can be requested from the dataset authors directly.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://niis. cc/
public/guide httpei/ggiiieee/eeeuubmimseee (CodeSubmissiooooll11
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
btciss.cc/publle http://uwlccgguuddeeeeeeeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details are specified the paper (see Section 3.1 and Appendix A.1 and the code is publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Results are reported with the standard error of the mean across resamplings of the training datasets detailed in Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Exact training times were not tracked during the experiments and there was substantial variability between runs, given the use of early stopping, different dataset sizes, and model architectures. However, we describe the compute resources used and provide ballpark runtime estimates to reproduce the results in Appendix A.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:::wworrri...cc/aaabbt/ttta
Answer: [Yes]
Justification: We have reviewed the code of ethics and confirm that this research complies with it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
17

---

The findings of this study may contribute to the development of better EEG models, and therefore have a positive downstream societal impact, given the applications of EEG in healthcare as motivated in the introduction. No negative societal impacts are anticipated.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
No new datasets pretrained models are released with this study.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets properly attributed (Section 3.2 Section 3.4 Appendix A.4).
License information for the PhysioNet dataset is reported. Access to CAUEEG (for academic and research purposes) and TUAB can be requested from the dataset authors.
Guidelines:
18

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, papprssithcaseeo thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The research code released with this paper is documented to enable the reproduction of all results and facilitate the conduction of similar experiments, e.g. to study the impact of participant distribution shifts in other tasks and datasets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/modell as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: No data was collected as part of this study. We refer to the reference publica- tions for the datasets for details.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer: [No].
Justification: The use of data in this study constitutes reanalysis of anonymised data shared via institutionally controlled or public access. This secondary data analysis does meet the definition of research requiring IRB oversight. We refer to the ethics information shared by the reference publications for the datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA].
Justification:
The methods presented here do involve LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Training Details and Hyperparameters
The official train-test splits were used for TUAB and CAUEEG, and in both cases, part of the train split was set aside for validation. For PhysioNet, where labels for the official test split are not publicly available, an age and sex stratified split into train, val, and test was created on the participant level (i.e. without overlapping participants). The exact splits can be reproduced with the shared research code.
Models were trained using Adam (betas= (0.9, O. 0999,), -eiightdecaa=O.... 01), global gradient norm clipping imax_nomı = 1. 1). and a cross- cross-entropy loss. The learning rate was set to 1e-3 for mAtt and TCN and tuned for LaBraM (see Section 3.6 and Table [ for details and the used learning rates respectively). For the CAUEEG dataset, we used class weights in the loss function to account for class imbalance. Since the amount and diversity of the training data differed substantially between trials, early stopping (on the fixed validation set) was used to prevent overfitting, while allowing sufficient training time if the amount of data was larger. Specifically, the validation loss was evaluated every 500 batches and patience of 5 was used to trigger early stopping.
Table 1: Learning rates for the LaBraM model.
Dataset
From Scratch
Fine-tuning
TUAB CAUEEG
le-4
1e-6
1e-3
5e-6
PHYSIONET
1e-3
1e-5
All models were trained on an internal cluster with RTX 2080 Ti (IIGB) and A40 (48GB) GPUs. Training times varied given the different models, dataset sizes, and the use of early stopping. We estimate the total runtime reproduce the baseline results at around a week on 20 RTX 2080 Ti GPUs and at around two weeks for all results, including the augmentation, pre-training, and ablation experiments.
A.2 Selection of EEG Data Augmentations
The selection of EEG data augmentations investigated in our work was informed by prior work. PhaseRandomisation (termed FTSurrogate) consistently emerged the most effective augmentation in the systematic comparison on two tasks (sleep staging and BCI motor imagery) by Rommel et al. (6). FrequencyShift showed more modest gains, but particularly interesting to study in the context of participant diversity, because differences in peak frequencies are associated with demographic variables like age and sex [46] 47 48). Importantly, other augmentations like GaussianNoise. ChannelSymmetry. ChannelShuffle TammeMasking, and BandstopFilter did not show consistent and significant improvements in their comparison, which is why we did not include these augmentations in the present work. The two augmentations SignFlip and TimeReverse showed some improvements in limited settings, but were not included our study because they clearly violate the structural integrity of EEG data. Furthermore, while ChannelDropout seemed to provide limited value in the BCI task, closer look revealed that the dropout rate was tuned to (i.e. dropout of all channels), hinting an issue with the evaluation task or the training, rather than genuine augmentation benefit. Finally, we also studied AmplitudeScaling [12). which was originally proposed in the context of a contrastive learning approach and was not part of the benchmarks by Rommel et al.
A.3 Metrics
We assessed model performance through prediction accuracy. For TUAB and CAUEEG, segment predictions were first aggregated into participant-level predictions via majority voting, such that the average was then computed across participants rather than across all pooled segments. We chose this approach because the labels were originally assigned on the participant level rather than to individual segments within the recordings, and because better captures model's 's ability to diagnose new participants.
21

---

As outlined in Section 3.1 each trial (i.e. each combination of given number of participants and number of segments per participant) was repeated multiple times with different random seeds for the subsampling of the training data. This allowed us to report average accuracies along with the standard error of the mean across seeds. The baseline experiments were repeated with 25 seeds and the pre-training experiments were repeated with 25 seeds on TUAB and CAUEEG and with 5 seeds on PhysioNet. For the augmentations, we report pairwise comparisons for a single seed. On the PhysioNet dataset, where considerably longer recordings were available for some participants, assessed conditions with up to 640 segments per participant. However, since this number segments is not available for all participants in the dataset, experiments failed for some of the seeds, which is also the reason for the missing tiles in the top right corner of the PhysioNet results heatmaps. Rather than only reporting the results up 40 segments per participant, up to which point we have complete results for all seeds, we included trials with failed seeds as they provide interesting and unbiased additional information.
Accuracy differences were computed as pairwise differences between corresponding results (same number of participants, number of segments per participant, and seed) across conditions (pre-trained vs baseline or augmented vs unaugmented). For average accuracy differences, these pairwise differences were averaged over seeds.
A.4 Software
The research code for our experiments publicly available lttttt::////t thtbb...ttuu... com/bomatter/ www.iipptnntiiessstseeeee
Dependencies notably include Py'Torch [49]. MNE-Python (50]
and WNE-MIIII BIDS [1] for data harmonisation and preprocessing. TorchEEG [52] for the implemen-
tation of the data loader, Braindecode (31] for the implementation of the TCN model and data
augmentations, and the repositories of the original LaBraM [15] and mAtt 32 models.
A.5 Model Comparison
One aspect determining the sample efficiency of model is its architecture. Comparing model architectures in Figure 2
we observe that the TCN and mÄtt architectures performed slightly better than LaBraM on TUAB and CAUEEG and markedly better on PhysioNet. On PhysioNet, LaBraM's substantially lower performance can partially be attributed to differences in preprocessing. As explained in Section 3.3 15 second epochs had to be used for LaBraM instead of the 30 second epochs used for TCN and mAtt. We performed an ablation study where the TCN model was trained using the same preprocessing, which showed that differences in preprocessing explained part of the performance gap, but that TCN still outperformed the LaBraM model (see Appendix [A. We hypothesise that LaBraM's 's lower performance stems from the weaker inductive bias of the transformer architecture and the fact that the LaBraM model is considerably larger in terms the number of model parameters compared to the other models. The LaBraM model would thus be expected to be less sample efficient.
The differences between mAtt and TCN could reflect the usefulness of different priors for different tasks. TCN, as a convolutional neural network, likely better at picking up on specific wave morphologies (e.g. sharp waves or K-complexes), which may be useful for normality prediction and sleep staging. The mAtt model, on the other hand, embeds the EEG into sequence of covariance matrices that reflect signal power and cross-channel correlations, and might excel in tasks where these are the most relevant features.
On the whole, and with the exception of LaBraM's performance on PhysioNet, the achieved accuracies were relatively similar, despite substantial differences in the architectures and model sizes. The sample size was the primary driver of performance.
22

---

A.6 Ablation: LaBraM--ompatible Preprocessing
Data for the training and fine-tuning of the LaBraM model was preprocessed separately ensure compatibility with the official pre-trained weights. On the PhysioNet dataset, this also meant that the segment length had be reduced from 30s to 15s. Since LaBraM's performance on PhysioNet was particularly low compared to the other models, we performed an ablation study to see how much of the performance gap should be attributed to the different preprocessing rather than to architectural differences. Specifically, we trained the TCN model using the same preprocessing that was used for the LaBraM model. Figure G that performance decreased (light blue line vs dark blue line) but was still higher than the LaBraM bascline performance (light green line). With pre-training, LaBraM achieved comparable performance to the TCN model when the same preprocessing was used (dark blue and dark green lines). Overall, this suggests that the different preprocessing explains part of the performance gap, but not all it. When models were trained from scratch, TCN still achieved better performance.
Sleep Staging
PhysioNet
0.7
Model
Accuracy
TCN
TCN (aBrarM-rreproc
0.6
LaBraM
LaBraM (pretrained)
Number of Participants
Figure 6: Ablation: LaBraM-compatible preprocessing. The light blue and dark blue lines represent TCN performance with standard and LaBraM-compatible preprocessing, respectively, while the light green and dark green lines show LaBraM performance without and with pre-training. The performance drop observed for TCN with LaBraM-comne caBraM-compatible preprocessing suggests that preprocessing differences contributed to LaBraM's lower baseline performance. However, the fact that TCN still outperformed LaBraM when both used the same preprocessing indicates that architectural differences also played role. Pre-training LaBraM mitigated the performance gap, allowing it to achieve results comparable to the TCN model when both used the same preprocessing.
23

---

A.7 Control Experiment: Contiguous Segment Subsampling
In our main experiments, segments were subsampled uniformly random. However, in practice. collecting fewer segments would usually correspond to shorter recordings, which are less likely to capture wide range of long-term -on-stationssiiiss such as drifts, participant fatigue, or changes in mental state. Uniform subsampling may therefore artificially increase segment diversity compared to realistic data collection.
To examine whether this affects our conclusions, we performed a control experiment using contiguous subsampling, where training segments are drawn from consecutive blocks rather than uniformly. Specifically, for an epoched EEG with N non-overlapping fixed-length segments and a desired number of segments m. we uniformly sample random starting index [0, N m]. We then
extract contiguous block of m segments: segments(s:
m].
Results shown in Figure 7 which compares performance with uniform versus contiguous subsam- pling for normality prediction on TUAB with the TCN model. Patterns are consistent across both conditions, indicating that the conclusions not affected by the choice of the subsampling strategy.
Uniform Segments seeds)
Contiguous Segments seeds)
1800
0.84
0.65
0.54
0.64
0.56
22.117
0995
531645
9920
18.094
18.0114
503114
0920
18.01
500
0.83
0.84
0.85
0.65
0.85
0.85
0.85
0.83
0.83
0.55
0.84
0.85
0.85
0.84
Number of Partioparts
5870
20.022
#00037
200075
20,0155
40.0
40,0037
40,040
20,103
400
60,90
600016
60.176
200
0.64
0.83
0.83
0.58
0.60
0.82
0.82
0.80
0.81
0.58
0.61
0.87
0.82
0.32
6873
6.062
a3 6-12
600112
440079
60.042
+2
60.000
100
0.77
0.78
60
0.78
0.78
0.78
0.76
0.7g
0.74
0.74
0.74
0.76
0.74
0.74
0.74
0.74
0.75
20.0122
281
55.0kg
133
203153
50,07 14
18.0192
$0,0000
22.022
50.1127
22 54
10.100
50.211
25
0.75
0.78
P4
0.24
0.75
0.27
0.75
0.28
80
320
40
160
Number of Segments per Participant
Figure 7: Uniform versus contiguous segment subsampling. Normality prediction accuracy on TUAB with the TCN model, averaged over random seeds along with the standard error of the mean.
24

---

A.8 Accuracy Heatmaps for mAtt and LaBraM
Normality Prediction
Dementa Diagnosis CAUEEG
Sleep Staging PhysioNet
TUAB
1600
0.82
2.04 JARRY
C.D2 XAMOOY
300011
LAGOS
000
0.40
0.80
0.60
8.89
FISE
D/00
0199
0199
0.91
0.68
0154
0.09
2.64
35
40880
A20012
60.000
+0/000
40001
Number of Part-ipatt
400
0:62
E43
6.11
0.50
D46
6.70
200
100
JASSA
LAMGG
LEGOND
LMAGO
JUCD
100
A1004
4U85
AURVY
JANL
50
3.37
2.72
17
2:19
5:75
0.75
D:CK
D10
0:39
0.39
0:01
0.99
0:41
0.64
385
063
283
D167
JASOL
LACOR
AZCOM
LECETT
JEOLTT
10.00
4:000
0.72
072
5:13
212
0:72
5/72
1.72
0-05
C
RAT
0V4
0:43
0.54
0.53
0:83
2.1
8.5
282
0.55
Number of Segments per Parscipant
Figure 8: Baseline performance heatmap for the mAtt model. Accuracies for normality prediction on TUAB, dementia diagnosis CAUREG, and sleep staging PhysioNet averaged across seeds, along with the standard error of the mean.
Normality Prediction TUAB
Dementia Diagnosis CAUEEG
Sleep Staging
PhysioNet
38001
+1000
-1/02L
+1
-2:000
Numbar Puricipatt
4181
JASC
40000
403010
LCCOM
45
JUSCE
AECOI
JUSEIO
2003
3.77
0.78
0:76
0.50
282
0.52
0.50
0.42
0.51
2.52
0.54
0.43
0.53
0.52
0:50
0.57
C.00
1000
+1
0007
1001
3.74
5.75
0.38
5:76
0:28
5:75
0.79
0.45
0.44
0.89
2.41
0190
0.49
0.97
0.99
5.57
30
3.22
0.75
D17
075
2.77
0.77
0:44
CAR
0A47
5.4
0-43
2.44
DAB
0:43
2:50
0.23
2.54
0.5T
2.50
28
3.99
6:71
0.73
6/14
0.22
7:75
0.23
0.81
2:43
0.43
244
0.25
0:41
D:41
0.45
5:48
0.53
8.63
0.54
Number of Seements per Portcipont
Figure 9: Baseline performance heatmap for the LaBraM model. Accuracies for normality prediction on TUAB, dementia diagnosis on CAUEEG, and sleep staging on PhysioNet averaged across seeds, along with the standard error of the mean.
25

---

A.9 No Association between Sample Size and Accuracy Improvements with Augmentations
Table 2: Spearman correlations between sample size and accuracy difference to baseline performance with different augmentations. No associations between accuracy improvements and sample size were observed. p-values 0.05 are marked with an asterisk. Note that these are uncorrected and none would remain significant under Bonferroni correction.
Dataset
Augmentation
TCN
mAtt
LaBraM
AmplitueeScaling
0.047 (p=0..44)
0.079 (p=0.5590
-0.141 (p=0.335)
TUAB
FrequencyShift
0.053 (p=0.717)
0.174 (p=0.232)
0.049 (p=0.738)
Phademaadoiisttionn
0.083 (p=0.571)
-0,076 (p=0.609)
0.045 (p=0.761)
AmpliudeScallin
-0.311 (p=0.095)
-0.195 (p=0.302)
0.128 (p=0.499)
CAUBEG
FrequencyShift
-0,430 (p=0.018)
0.055 (p=0.772)
0.027 (p=0.885)
PhaseRandomisation
-0.407 (p=0.026)
-0.324 (p=0.081)
-0.215 (p=0.253)
Amplitudeccalin
0.245 pp=0.155)
+0339* (p=0.046)
-0.024 (p=0.886)
PhysioNet
FrequencyShift
-0.076 (p=0.663)
-0,037 (p=0.833)
0.289 (p=0.075)
PlaseRandomisation
-0.322 (p=0.059)
-0.144 (p=0.409)
0.117 (p=0.503)
26

---

A.10
Pre-training Results on CAUEEG and PhysioNet
Baseline Accuracy
Pretrained Accuracy
Accuracy Difference
800
0.53
0.54
0.54
0.56
0.56
0.56
0.57
0.88
0.59
0.80
0.03
0.00
0.34
0.02
0.04
s3.0081
s0,009
=0,0077
+3,0088
s0,0046
#3.0041
s0.0098
=0.0000
50.0333
000000
Number Participants
400
0.61
0.56
0.53
0.57
0.03
0.01
60.0003
#00296
=0,0059
20,0057
40.0374
20.067
80.0002
20,0020
#00103
20.0113
50.0085
230
0.50
0.49
0.51
0.52
0.54
0.52
0.50
0.52
9.51
0.54
0.01
0.01
0.01
-0.00
-3.00
10.0072
20 0106
10,0274
10,0344
10 0082
10.0033
10,0077
100057
10.0770
10.0102
10.0114
10.0144
10,0375
100116
10.0103
100
0.45
0.48
0.48
0.40
0.50
0.45
0.51
0.51
0.50
0.50
0.30
0.08
0.32
0.01
3.01
50
0:44
0:49
0:46
0.00
I0:0155
10.0148
10/0355
10.0044
10.0124
10/0109
10,0157
10.009
25
0.40
0.41
0.44
0.44
0.45
0.41
0.63
0.44
0.04
0.45
0.01
0.02
-0.00
-0.00
-3.00
+2. +0,0093
0125
=5.0141
=0. 01.01122
0126
=0.0140
+7,0140
=0.0137
20
0140
+3.0187
50.0104
=0.0199
=0.0109
=00011
s0.0125
5
10
20
40
80
10
20
40
80
5
10
20
40
80
Number of Segments per Participant
Figure 10: Effect of pre-training on the CAUEEG dataset. Comparison LaBraM performance with and without pre-training on CAUEEG. The left and middle heatmaps show average accuracies (+ standard error the mean) for LaBraM trained from scratch (baseline) and pre-trained on collection of 16 datasets before fine-tuning on CAUBEG, respectively. The right heatmap visualises the average of pairwise accuracy differences (# standard error of the mean), where pairs correspond to the accuracies of pre-trained and baseline models for the same seed.
Baseline Accuracy
Pretrained Accuracy
Accuracy Difference
25 50 100 200 400 800
Number Participants
13000 mm
c30000
30000
1998
c300es
0.55
0.54
0.59
0.82
0.03
0:04 www.
9.04
0.07
.a 22
40580
43 mast
0.48
0.51
0.52
0.55
0.57
0.50
0.63
6666
0.51
0.54
0.55
0.50
0.81
0.88
068
0.72
0.04
0.04
0.05
0.05
0.06
0.06
0 08
0.06
0.50
encce
40
01040 100
H5
0.44
0.45
0.48 050 0.59
0.54
0.57
59
048
0.51
0.54
0.58
0.57
0.57
0.81 0.64
0.
0.08
9.67
0.05
0.06
0.04
08
0.05
21
12
-8.00
-3 cm
H1
0.41
0.41
0.46
0.48
0.50
0.53
0.64
0.55
0.43
0.47
0.52
0.62
0.55
0.56
0.58 0.68
0.03
0.38
0.06
0.05
0.05
0.03
0.04
0.05
20
160
320 640
5
20
320
640
10
40
80
160
320
640
Number of Segments per Participant
Effect of pre-training on the PhysioNet dataset. Comparison of LaBraM performance with and without pre-training on PhysioNet. The left and middle heatmaps show average accuracies (+: standard error of the mean) for LaBraM trained from scratch (baseline) and pre-trained on collection of 16 datasets before fine-tuning on PhysioNet, respectively. The right heatmap visualises the average of pairwise accuracy differences (# standard error of the mean), where pairs correspond to the accuracies of pre-trained and bascline models for the same seed.
Figure 11:
27

---

A.11
Scaling Behaviour Plots with Pre-trained LaBraM Model
Normality Prediction
Dementia Diagnosis CAUEEG
Sleep Staging PhysioNet
TUAB
0.85
Model
Accuracy
0.80
0.55
TCN
LaBraM
0.75
Laftradd (peatriined)
0.45
500
400
Number Participants
Number Partcipantss
Participants
Figure 12: Scaling behaviour with pre-trained LaBraM model. Lines show the average accuracy across seeds with the standard error of the mean. Pre-training consistently improved LaBraM's performance and allowed it achieve comparable even superior results to the other models. The lower performance on the PhysioNet dataset is due to differences in preprocessing (see Appendix A.6)).
28


Reviews:
[{"review_id": "UjJkybnxFk", "reviewer": "Reviewer_2Wvo", "summary": "This paper systematically investigates the impact of participant diversity versus overall sample size on the performance of EEG-based machine learning models. The findings highlight that participant diversity can be a critical factor, often more so than simply increasing the total number of segments, and demonstrate the potential of self-supervised pre-training to mitigate data scarcity.", "questions": "1. Why were TCN, mAttn and LaBraM chosen? Can these three models cover the commonly used method categories in EEG-based tasks? It is suggested to explain it in the paper.\n2. In each task domain (normality prediction, dementia diagnosis, sleep staging), what augmentations are commonly adopted in prior literature? Are amplitude scaling, frequency shifting and phase randomization considered best-practice baselines? If not, how can the universality of your Q3 conclusion be claimed?", "limitations": "YES", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\n1. Good writing and organization\n2. The paper provides a valuable and systematic empirical analysis of the relative importance of \"participant diversity\" and \"overall sample size\" in EEG machine learning, a crucial aspect often overlooked in prior work.\n# Weakness\n1. The experiments in this paper are not sufficient the number of tasks, models, and datasets used is too small, which reduces the persuasiveness of the results.\n2. The manuscript lacks a conclusion section.\n3. While the manuscript provides a detailed analysis of the experimental phenomena, it offers limited discussion on the underlying causes behind these observations.", "was_revised": true, "final_justification": "During the rebuttal session, the authors have addressed most of my concerns, and I appreciate the clarification regarding the motivation for the choice of augmentation methods, which will be rephrased in the camera-ready version. However, I think the methodological novelty is somewhat limited. Therefore, I have increased my global score from 3 to 4.", "rebuttal": "Thank you for your review and constructive feedback. We are glad you valued our systematic empirical analysis.\n\n\n\n**Limited number of tasks, models, and datasets**\n\nWe aimed to design a diverse and sufficiently broad set of conditions to support meaningful and generalisable conclusions. Our experimental setup provides a solid basis to answer our research questions, and experiments on additional datasets would not change our conclusions.\n\n- We deliberately included three models with fundamentally different architectures: TCN (CNN-based), mAtt (geometric deep learning using covariance representations), and LaBraM (Transformer-based). These span a range of methodological paradigms explored in recent EEG literature and reduce the likelihood of model-specific conclusions.\n- We focused on large-scale datasets (1000+ participants), which are rare in EEG research [1]. Large datasets are essential for our experimental design, as they allow us to systematically vary the number of participants and segments over several orders of magnitude. Our work could not have been done with a large number of small datasets. Additionally, evaluations on small datasets, while easier to scale, are often noisy and less reliable.\n- Compared to prior work, we are not aware of any EEG-based ML study with evaluations on a higher number of large (1000+ participants) datasets, diverse model architectures, and different tasks.\n- Additional datasets or tasks would not change the answers to our research questions. Normality prediction on TUAB and dementia diagnosis on CAUEEG already provide striking examples of severe participant distribution shifts, whereas sleep staging experiments on PhysioNet showed that there are specific applications where it is possible to achieve good performance even with limited participant counts. So while additional tasks and datasets could have some value to inform application-specific future work, we do not require more experiments to answer the research questions in this paper.\n\nFinally, we would like to emphasise the computational scale of our current experiments. For example, for the baseline experiments on TUAB alone, we evaluated 49 different data regimes (combinations of participant and segment counts) for each of the three models and repeated each configuration with 25 random seeds, resulting in 3675 training runs.\n\n\n\n[1] Roy, Yannick et al. ‘Deep Learning-Based Electroencephalography Analysis: A Systematic Review’. Journal of Neural Engineering, 2019.\n\n\n\n**Missing conclusion section**\n\nWe aimed to use the Discussion section as an extended Conclusion section. We will include a short summary of the main points of this Discussion section in a new Conclusion section for the camera ready version.\n\n\n\n**More discussion of underlying causes behind observations**\n\n- Observations for Q1 and Q2 are addressed through the analysis in Section 3, which describes a theoretical framework under which the importance of participant diversity can be explained. We link back to this analysis in the results section, e.g. line 264.\n- Observations for Q3 are discussed in Section 4.3. For instance, we discuss underlying reasons why the PhaseRandomization augmentation failed to improve performance in line 291. We will extend the camera-ready version of the manuscript with further discussion about AmplitudeScaling and FrequencyShift (see below).\n- Observations for Q4 are discussed in Section 4.4. We will extend the camera-ready version of the manuscript with further discussion (see below). \n- In addition, model differences are discussed in a dedicated section in A.5 with references and additional discussion in the results section, e.g. lines 248 and 287.\n\n\n\nExtended discussion of observations about AmplitudeScaling and FrequencyShift:\n\n> A likely explanation for why AmplitudeScaling and FrequencyShift did not yield consistent accuracy improvements is that these augmentations are not truly label preserving. For example, overall amplitude and differences in amplitude between EEG channels (e.g. asymmetries between corresponding channels on the left and right hemisphere) can provide clinically relevant information [1]. Similarly, frequency shifts can be clinically relevant (e.g. slowing may reflect abnormalities [1] and has been linked to MCI and AD [2, 3]). Alternatively, it is possible that even the smallest amount of training data already provided sufficient diversity of amplitudes and frequency shifts for the model to learn sufficient invariances to these aspects of the data, rendering the augmentations redundant.\n>\n> [1] Rubin, Devon I. et al. ‘Clinical Neurophysiology’. Contemporary Neurology Ser., 2021.\n>\n> [2] Cassani, R. et al. ‘Systematic Review on Resting-State EEG for Alzheimer’s Disease Diagnosis and Progression Assessment’. Disease Markers, 2018.\n>\n> [3] Modir, Aslan et al. ‘A Systematic Review and Methodological Analysis of EEG-Based Biomarkers of Alzheimer’s Disease’. Measurement, 2023. \n\n\n\nExtended discussion of observations for self-supervised pre-training:\n\n> Pre-training exposed the model to a larger diversity of EEG data from additional participants than those seen during fine-tuning. The positive results suggest that, even without labels, this has allowed the model to learn useful representations that transfer to diverse downstream tasks.\n\n\n\nIf there are other observations that the reviewer believes would benefit from further discussion, could they let us know more specifically what they are?\n\n**Motivation for selected models**\n\nWe selected three influential and widely used models with distinct architectures to obtain a diverse perspective on EEG model behavior and reduce the risk of model-specific effects.\n\n- **TCN** is a convolutional neural network (CNN), which was developed for generic sequence modeling and has since been widely adopted in EEG research. CNNs remain a strong baseline in EEG-based machine learning.\n- **mAtt** is a geometric deep learning model specifically developed for EEG. It encodes signals as sequences of covariance matrices and exemplifies a line of work that incorporates Riemannian geometry to leverage domain-specific structure.\n- **LaBraM** is a transformer architecture. As in other domains, transformers have become a common architecture choice in recent works, particularly for the development of foundation models.\n\nThese models provide a good picture of the landscape of the current state-of-the-art in EEG-based machine learning.\n\n\n\n**Motivation for selected augmentations**\n\nAs discussed in the related works section, our selection was informed by the systematic evaluation of EEG augmentations conducted by Rommel et al. In their benchmarks on two tasks (sleep staging and BCI motor imagery), PhaseRandomization (termed FTSurrogate) consistently emerged as the most effective augmentation. FrequencyShift showed small improvements when data was more limited. Inclusion of the FrequencyShift augmentation was further motivated, because differences in peak frequencies have been shown to be associated with demographic variables like age and sex [1, 2, 3], making it an interesting augmentation to study in the context of participant diversity. In addition, we also studied AmplitudeScaling, an augmentation introduced by Mohsenvand et al. in the context of a contrastive learning approach, but not evaluated in the review by Rommel et al.\n\nImportantly, Rommel et al. also reported that other augmentations (Gaussian noise, channel symmetry, channel shuffle, time masking, bandstop filtering) did not provide consistent and significant improvements in performance, which is why we did not include them in our work. The two augmentations SignFlip and TimeReverse showed some improvements in limited settings, but were not included in our study because they do not preserve the structure of EEG data. Finally, while ChannelDropout seemed to provide limited value in the BCI task, a closer look revealed that the dropout rate was tuned to 1 (i.e. dropout of all channels), hinting at an issue with the evaluation task or the training, rather than genuine augmentation benefit.\n\nWe will rephrase the relevant part of the related work section in the camera-ready version of the manuscript to make the motivation for our choice of augmentation methods more clear.\n\nTaken together, and given the more comprehensive evaluation of a wide range of augmentations in prior work that informed our study, we believe the three selected augmentations provide an informative picture of contemporary EEG data augmentations. We would also like to add that we do not exclude the possibility that more effective augmentations could be developed as discussed and encouraged in the discussion section of our manuscript.\n\n[1] Aurlien, H et al. ‘EEG Background Activity Described by a Large Computerized Database’. Clinical Neurophysiology, 2004.\n\n[2] Chiang, A. K. I. et al. ‘Age Trends and Sex Differences of Alpha Rhythms Including Split Alpha Peaks’. Clinical Neurophysiology, 2011.\n\n[3] Cellier, Dillan et al. ‘The Development of Theta and Alpha Neural Oscillations from Ages 3 to 24 Years’. Developmental Cognitive Neuroscience, 2021."}, {"review_id": "saJl8l24NC", "reviewer": "Reviewer_77HZ", "summary": "This paper explores the impact of both sample size and participant diversity for model generalisability and robustness when applying machine learning to EEG data. By viewing EEG generation as a multi-level process, the authors empirically analyse how model performance scales with data quantity and participant diversity. They further evaluate the impact of data augmentation and self-supervised learning in scenarios with limited data. This study reveals that distributional shifts among participants can significantly limit performance gains and offers practical recommendations for data collection and machine learning methodologies. The experimental code is also made publicly accessible to advance research within the community.", "questions": "1. The main answers to the proposed four questions in Section 4 seem quite intuitive and might not offer much novelty to readers already knowledgeable in this area. Could the authors clarify what new insights or conceptual advancements their work brings beyond what is already expected?\n\n2. The purpose and significance of the analysis presented in Section 3 are somewhat ambiguous. How does this section specifically relate to the core experimental results or support the main conclusions of the paper? Providing a clearer explanation of its role would help improve the coherence and impact of this study.", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**: \n\n1.\tThe paper presents what appears to be the first systematic investigation into how the number of data samples and subjects influences the performance of EEG-based machine learning models.\n2.\tThe experimental design is thorough and methodologically sound, incorporating both supervised and self-supervised learning approaches across datasets from diverse domains.\n\n**Weaknesses**:\n\n1.\tThe main conclusions—particularly the answers to the four questions posed in Section 4—are largely intuitive and may be considered unsurprising by readers familiar with the field. As a result, the paper offers limited new insights or conceptual advancement.\n2.\tThe role of the analysis in Section 3 is unclear. It is not evident how this section connects with the core experiments or contributes directly to the study’s central conclusions. A more explicit explanation of its relevance would strengthen the narrative.", "was_revised": true, "final_justification": "The authors have provided a thoughtful rebuttal addressing the main concerns.\n\nRegarding the concern about the intuitive nature of the conclusions, the authors clarified that some of their findings—such as the failure of data augmentation on large-scale EEG datasets and the dominant role of participant diversity—challenge common assumptions and offer actionable insights. Their recommendation for pilot studies to assess participant diversity is well-argued.\n\nThe concern regarding the role of Section 3 was also addressed with a clearer explanation of how the theoretical framework supports the experimental results and research questions. \n\nHowever, I believe the paper would benefit further from developing a more explicit or trade-off framework that quantifies the optimal balance between the number of participants and the number of samples per participant. Such a formulation could serve as a practical tool for dataset construction and selection in EEG and ML research. Given that data scale, diversity, and domain alignment between pretraining and downstream datasets are all critical to model training and generalization, formalizing this interplay would significantly enhance the utility and impact of the findings.\n\nMoreover, while the paper frames itself as an empirical study of scaling behaviour and participant diversity in EEG modeling, the experimental coverage remains somewhat limited. Only three datasets (TUAB, CAUEEG, PhysioNet), three models (TCN, mAtt, LaBraM), and three task types (normality prediction, dementia diagnosis, sleep staging) were included. For a study aiming to establish broadly applicable empirical trends, this setup may be too narrow to support strong general conclusions. Including a wider range of EEG tasks, datasets with varying recording protocols, and more diverse model architectures would substantially strengthen the work’s external validity and practical relevance.\n\nOverall, while the paper is methodologically sound and the rebuttal addresses the raised concerns, I still find the broader impact to be somewhat limited.", "rebuttal": "Thank you for your helpful review and comments. We appreciate that you found our experiments thorough and methodologically sound.\n\n\n\n**Limited new insights due to intuitive conclusions**\n\nFirst, we would like to emphasise that our results provide quantitative, more nuanced, surprising, and non-trivial insights.\n\n- We evaluated EEG data augmentations on much larger datasets and across a more diverse set of models and tasks than prior work. Our results show that data augmentations largely failed to improve performance—a conclusion that stands in contrast to their reported success in smaller-scale studies in the literature.\n- Our experiments provide *quantitative* evaluations of the scaling behaviour of different models and the impact of participant diversity across multiple tasks. We would argue that it was not expected that even a more than 50-fold increase in sample size would fail to improve performance in the absence of sufficient participant diversity on TUAB. This result demonstrates how severely model performance can be bottlenecked by the participant count.\n- It is currently not common practice to inform data collection with pilot studies that evaluate the importance of participant diversity. As mentioned in the discussion section, our results motivate such pilot studies given the severely participant-count-dominated scaling behaviour on TUAB and CAUEEG on the one hand, and the PhysioNet results on the other hand, where participant diversity was less critical. We are not aware of prior work that provides such actionable guidance using empirical scaling results.\n\nFinally, while some of our results might be well aligned with the intuition of experts in the field, we argue that it is important to test intuitions and substantiate them with empirical results. When two researchers have conflicting intuitions about the existence or importance of participant distribution shifts, our work provides empirical evidence that can inform the discussion.\n\n\n\n**Role of the analysis in Section 3**\n\nIn Section 3, we lay out the theoretical foundation that motivates our experiments and provides a framework in which the results can be interpreted.\n\nSection 3 currently concludes with the following summary:\n\n> “This analysis tells us that in the presence of a substantial distribution shift, the extent to which one should expect to overfit is governed primarily by the number of participants, rather than the amount of data available per participant, unless one is able to design a model architecture with the appropriate invariances.”\n\nTo make the connection to our research questions clear, we will extend this paragraph with the following explanation:\n\n> “Under this framework, we address the four research questions outlined in the introduction. We assess the relevance and magnitude of participant distribution shifts through the scaling behaviour of model performance (Q1 & Q2) and the effectiveness of augmentations (Q3) and self-supervised pre-training (Q4) to improve performance by endowing models with useful invariances or boosting effective participant counts.”"}, {"review_id": "ukgKa9xHxc", "reviewer": "Reviewer_LnQb", "summary": "This paper examines the impact of participant diversity and data volume on the performance of machine learning models used for EEG analysis. The authors use various EEG datasets and tasks and analyze the effectiveness of data augmentation and self-supervised learning as strategies to address data limitations. Their findings suggest that participant diversity (i.e., in 2/3 tasks including more data per participant was not effective to compensate for small participant numbers) often limits model generalization, highlighting the need for methods that specifically tackle inter-individual distribution shifts.", "questions": "1. What was the train/test/validation split used for each dataset?\n2. The paper shows that common EEG data augmentation methods failed to improve performance and, in some cases, even decreased the performance. While the authors mention that PhaseRandomization can destroy wave morphology and cross-channel correlation, which are critical for sleep stages, could the authors elaborate why they believe that AmplitudeScaling and FrequencyShift did not show an improvement in accuracy? Additionally,  what type of augmentations would mitigate participant differences?\n\n\nSmaller improvement:\n- Figure 5 caption: Capitalise pre-training. The sentence should be: \"...baseline models for the same seed. Pre-training consistently improves...\"", "limitations": "The paper includes a dedicated section for \"Limitations and Future Work\" in Appendix A.1, however for transparency, this should be moved to the main text.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The authors distinguish between overall sample size and participant diversity (i.e., number of participants and segments per participant). This level of experimental control enables a careful disentanglement of their respective effects.\n- The paper is clearly written and well-structured.\n- The study presents a systematic comparison across models and datasets. The use of multiple datasets and model types strengthens the generalizability of the findings. It is particularly valuable that the authors directly compare the effects of self-supervised pre-training and data augmentation.\n\nWeaknesses\n- The discussion of limitations is confined to the appendix. These should be addressed in the main text to provide a more balanced and transparent assessment of the work.", "was_revised": true, "final_justification": "In the rebuttal, the authors cleared up my concerns, and I appreciate the effort. They also responded thoughtfully to all the reviewers and covered most of the concerns that were raised. After reading the discussions and answers provided to the other reviewers, I agree with some of the concerns that the paper is not methodologically novel, but I think the paper provides novel insights, so I've bumped my score up from a 4 to a 5.", "rebuttal": "Thank you for your constructive review and insights. We appreciate your recognition of the generalisability of our findings and the value of directly comparing the effects of data augmentations and self-supervised pre-training.\n\n\n\n**Limitations section in the appendix**\n\nThe camera ready copy allows for an additional page in the main paper, in which we will include this limitations section.\n\n\n\n**Details on train/val/test splits**\n\nThe official train-test splits were used for TUAB and CAUEEG, and in both cases part of the train split was set aside for validation. For PhysioNet, where labels for the official test split are not publicly available, we created an age and sex stratified split into train, val, and test on the participant level (i.e. without overlapping participants). We will clarify this in the manuscript. For all datasets, the exact splits we used can be reproduced with the shared code.\n\n\n\n**Discussion of AmplitudeScaling and FrequencyShift augmentations**\n\nWe will add the following discussion to address why AmplitudeScaling and FrequencyShift may not have improved model performance:\n\n> A likely explanation for why AmplitudeScaling and FrequencyShift did not yield consistent accuracy improvements is that these augmentations are not truly label preserving. For example, overall amplitude and differences in amplitude between EEG channels (e.g. asymmetries between corresponding channels on the left and right hemisphere) can provide clinically relevant information [1]. Similarly, frequency shifts can be clinically relevant (e.g. slowing may reflect abnormalities [1] and has been linked to MCI and AD [2, 3]). Alternatively, it is possible that even the smallest amount of training data already provided sufficient diversity of amplitudes and frequency shifts for the model to learn sufficient invariances to these aspects of the data, rendering the augmentations redundant.\n>\n> [1] Rubin, Devon I. et al. ‘Clinical Neurophysiology’. Contemporary Neurology Ser., 2021.\n>\n> [2] Cassani, R. et al. ‘Systematic Review on Resting-State EEG for Alzheimer’s Disease Diagnosis and Progression Assessment’. Disease Markers, 2018.\n>\n> [3] Modir, Aslan et al. ‘A Systematic Review and Methodological Analysis of EEG-Based Biomarkers of Alzheimer’s Disease’. Measurement, 2023. \n\n\n\n**Discussion of potential augmentations to mitigate participant differences**\n\nWhat augmentations would effectively simulate participant differences is an open research question for which we encourage future work in our discussion section. An important consideration will be the nature of participant distribution shifts as we outline in the Future Work section: for example, if we hypothesise that anatomical differences are the main driver of participant diversity, future work could attempt to develop augmentations to imitate anatomical differences such as skull thickness or head circumference. If participant distribution shifts are instead explained by variability in disease phenotypes, domain knowledge of the specific disease could inform the design of augmentations for it."}, {"review_id": "pOU6EnF5Gg", "reviewer": "Reviewer_cReP", "summary": "In this article, the authors investigate scaling laws for EEG-based decoding of normality prediction, dementia diagnosis, and sleep staging. In particular, they investigate how decoding performance scales with the number of training subjects and with the number of examples per subject. Additionally, they investigate if data augmentation techniques and SSL pre-training improve performance.", "questions": "Can you confirm that the examples were sampled using a uniform distribution over the available samples?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The figures and diagrams are clear and of good quality (vectorized).\nThe research questions (RQs) are relevant. In particular, answers to RQ 2 (Is collecting more data per participant a reliable way to compensate for a lack of participants?) could be used to guide data collection, known to be costly, in the future.\n\nHowever, **the experiments conducted by the authors do not allow to answer RQ2**. Indeed, to simulate a *scarce data regime*, the authors randomly sample examples from the train sets (lines 126 and 127). Because they do not provide more details, I assume that they are sampled using a uniform distribution over the available samples. This method is often used when benchmarking decoding algorithms. However, it can not be used to answer RQ 2 because it omits the long-term non-stationarities inherent to EEG recordings. These non-stationarities include slow drifts in the signal, loss of contact with the electrode leading to increased noise, subject fatigue, change in the mental state of the subject, eventual drug intake, etc. When the authors randomly sample n examples among all, they have a large diversity of examples regarding the different non-stationarities aforementioned, larger than if the n examples were consecutive, which will lead to overestimated generalization abilities.\n\nAdditionally, four RQs is probably too much content for one conference article. This article could easily be split into three separate articles: RQs 1+2, RQ3, and RQ4.\nFinally, this article might out of the scope of NeurIPS as it does not introduce any novel method. However, with a corrected evaluation procedure to properly answer RQ 2, ths article would be of great interest for the BCI comunity.", "was_revised": true, "final_justification": "I expressed doubts regarding the methodology in my original review.\nThe authors provided us with an additional experiment with corrected methodology. It showed that this apparent gap did not impact the results, and the study conclusions remain unchanged. \nTherefore, I increased my significance score (1 to 3) and my global score (2 to 4).", "rebuttal": "Thank you for your thoughtful review and feedback. We appreciate your acknowledgement of the relevance of our research questions and the practical value of our work for guiding data collection.\n\n\n\n**Uniform subsampling of segments**\n\nWe confirm that segments were sampled uniformly in our experiments and we agree that contiguous sampling would more accurately reflect data collection in practice. Indeed, random subsampling of segments could lead to an artificially increased segment diversity and overestimation of model performance.\n\nTo address this issue, we conducted additional experiments where segments are subsampled contiguously. Specifically, we repeated the experiments for the leftmost heatmap in Figure 3 (varying the number of participants and segments for normality prediction on TUAB with the TCN model) with segments sampled contiguously. We repeated these experiments for 5 different seeds. The results show that our conclusions still hold. In particular, even when segments are sampled contiguously instead of uniformly, collecting additional segments is not a reliable way to compensate for a lack of participants.\n\nWe will include the results of this control experiment in the manuscript, showing accuracy heatmaps (with the standard error of the mean across seeds) for uniformly sampled vs contiguously sampled segments side-by-side. Since we are unable to share the plots via openreview, we provide the results in the form of a table here.\n\n\n\nUniform sampling (average accuracies along with the standard error of the mean across 5 seeds):\n\n| n_participants \\ n_segments | 5             | 10            | 20            | 40            | 80            | 160           | 320           |\n| --------------------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |\n| 25                          | 0.76 (0.0252) | 0.75 (0.0184) | 0.75 (0.0218) | 0.75 (0.0200) | 0.76 (0.0150) | 0.74 (0.0256) | 0.74 (0.0262) |\n| 50                          | 0.78 (0.0125) | 0.78 (0.0151) | 0.78 (0.0142) | 0.76 (0.0119) | 0.76 (0.0153) | 0.74 (0.0114) | 0.74 (0.0100) |\n| 100                         | 0.79 (0.0057) | 0.80 (0.0018) | 0.78 (0.0194) | 0.80 (0.0109) | 0.77 (0.0155) | 0.79 (0.0101) | 0.78 (0.0119) |\n| 200                         | 0.84 (0.0054) | 0.83 (0.0073) | 0.83 (0.0099) | 0.83 (0.0145) | 0.83 (0.0092) | 0.82 (0.0162) | 0.82 (0.0112) |\n| 400                         | 0.85 (0.0060) | 0.84 (0.0011) | 0.81 (0.0165) | 0.84 (0.0050) | 0.84 (0.0091) | 0.84 (0.0082) | 0.85 (0.0043) |\n| 800                         | 0.83 (0.0128) | 0.84 (0.0078) | 0.85 (0.0022) | 0.85 (0.0037) | 0.85 (0.0075) | 0.85 (0.0067) | 0.83 (0.0075) |\n| 1600                        | 0.83 (0.0117) | 0.84 (0.0056) | 0.84 (0.0058) | 0.85 (0.0046) | 0.85 (0.0030) | 0.85 (0.0029) | 0.84 (0.0054) |\n\n\n\nContiguous sampling (average accuracies along with the standard error of the mean across 5 seeds):\n\n| n_participants \\ n_segments | 5             | 10            | 20            | 40            | 80            | 160           | 320           |\n| --------------------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |\n| 25                          | 0.75 (0.0200) | 0.76 (0.0082) | 0.77 (0.0118) | 0.75 (0.0169) | 0.74 (0.0196) | 0.76 (0.0094) | 0.71 (0.0186) |\n| 50                          | 0.74 (0.0149) | 0.76 (0.0098) | 0.74 (0.0128) | 0.74 (0.0167) | 0.74 (0.0154) | 0.74 (0.0126) | 0.75 (0.0210) |\n| 100                         | 0.77 (0.0192) | 0.77 (0.0129) | 0.78 (0.0121) | 0.77 (0.0126) | 0.75 (0.0266) | 0.76 (0.0206) | 0.77 (0.0102) |\n| 200                         | 0.80 (0.0082) | 0.81 (0.0129) | 0.83 (0.0078) | 0.81 (0.0062) | 0.80 (0.0134) | 0.82 (0.0086) | 0.82 (0.0087) |\n| 400                         | 0.80 (0.0154) | 0.82 (0.0116) | 0.83 (0.0059) | 0.80 (0.0139) | 0.82 (0.0080) | 0.83 (0.0031) | 0.84 (0.0060) |\n| 800                         | 0.83 (0.0087) | 0.83 (0.0164) | 0.85 (0.0037) | 0.84 (0.0049) | 0.85 (0.0049) | 0.83 (0.0120) | 0.84 (0.0062) |\n| 1600                        | 0.84 (0.0039) | 0.83 (0.0134) | 0.85 (0.0014) | 0.84 (0.0104) | 0.85 (0.0029) | 0.85 (0.0018) | 0.85 (0.0025) |\n\n\n\n**Too much content**\n\nWe have considered splitting the research questions into separate pieces of work. However, we believe that it is valuable to present them together under the theoretical framework introduced in Section 3. By including research questions 3 and 4, investigating the effectiveness of common machine learning methods to address data scarcity, we are able to present a more complete picture of current data limitations in EEG-based machine learning.\n\n\n\n**No novel methods**\n\nThe NeurIPS2025 Reviewer Guidelines state that: “[...] originality does not necessarily require introducing an entirely new method. Rather, a work that provides novel insights by evaluating existing methods, or demonstrates improved efficiency, fairness, etc. is also equally valuable.”\n\nWe believe it is fair to claim this for our work."}, {"review_id": "zsAwydBOXh", "reviewer": "Reviewer_dTAg", "summary": "The paper investigates how participant diversity impacts the performance and generalisation of machine learning models trained on EEG data. It frames EEG datasets as multi-level data structures, where large numbers of samples are derived from a small number of participants via windowing. The authors conduct large-scale empirical studies to disentangle the effects of total sample size versus number of participants. They demonstrate that increased participant diversity consistently leads to more robust generalisation, even when total sample size is held constant. The paper also evaluates common ML strategies like pretraining, domain adaptation and model architecture choices under varying diversity conditions to show that some methods are disproportionately sensitive to participant homogeneity.", "questions": "How is your analysis of augmentations under limited participant counts fundamentally different from standard augmentation studies aimed at data scarcity?\n\nOn what basis do you consider LaBraM indicative of broader self-supervised learning approaches in EEG?\n\nProvide the derivation or source for the \"~1% improvement\" figure in line 239; is this drawn from your own results, external literature or an estimation?\n\nExplain the rationale behind the choice of augmentation techniques. Why these four and what criteria were used to select them over others?\n\nClarify the scope of your conclusions on self-supervised learning. Are your findings intended to generalise beyond LaBraM and if so, how is that generalisation justified?", "limitations": "The authors do include a limitations section but it is in the appendix and not integrated into the main discussion. The listed limitations are reasonable. They acknowledge the limited number of datasets and models used and the difficulty of decoupling all confounding factors in EEG data. However, these caveats are not reflected in the framing of their main conclusions, which sometimes overstate generality (e.g., claims about augmentation or self-supervised learning). These limitations should be surfaced earlier and more clearly tied to the scope of their findings.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- The paper systematically varies total data volume and number of unique participants to allow for controlled experiments that isolate the effects of diversity on model generalisation. This is very useful and not common in EEG literature.\n\n- The paper assesses pretraining, domain adaptation, self-supervised learning and architecture variants across multiple datasets with a well-rounded perspective on how methods scale with diversity.\n\n- By explicitly thinking of EEG datasets as nested hierarchies (participants -> sessions -> segments), the paper gives a helpful abstraction that clarifies why naïve sample count comparisons can be misleading.\n\nWeaknesses:\n\n- The authors claim in line 83 that their work is distinct because it investigates the utility of data augmentations under limited participant counts and limited data per participant. However, this distinction is unclear and weakly justified. Most augmentation methods in EEG, and in machine learning in general, are motivated precisely by data scarcity and generalisation issues. Without a clearer articulation of what makes their analysis novel in this space, the claim reads more as a restatement of standard goals rather than a meaningful point of differentiation.\n\n- The claim that LaBraM is a \"representative example\" of self-supervised learning approaches (line 93) is non-trivial and unsubstantiated. LaBraM is just one specific method, and the field of self-supervised learning in EEG is diverse and covers paradigms like contrastive learning, masked prediction and generative approaches. The authors provide no justification for why LaBraM adequately captures the range of behaviors or limitations of these different methods. This weakens the argument that the findings generalise to self-supervised learning as a whole. A claim of representativeness must be supported by either a taxonomy of the space or comparative results, neither of which are provided.\n\n- The claim in line 239 (doubling participants would yield “roughly 1%” improvement) is presented without any accompanying justification, reference to specific experiments or quantitative derivation. The number appears arbitrary. Given the paper’s central argument about the value of participant diversity, unsupported numerical claims like this undermine the credibility of the analysis. \n\n- The question \"Can contemporary EEG data augmentations effectively increase performance across different data regimes?\" is framed too broadly given the narrow scope of the study. The authors test augmentation methods that are all relatively standard (e.g., Gaussian noise, cropping). No principled selection is made and there is no attempt to explore the space of more recent or task-specific augmentations, nor any ablation or combination strategies. As such, the findings are limited to a small subset of possible techniques and cannot support general conclusions about the effectiveness of augmentations across EEG tasks or regimes. The question overstates what the study can validly answer.\n\n- The paper poses the broad question of whether self-supervised pre-training can improve performance across different data regimes, but only evaluates this using a single pre-trained model, LaBraM. This is insufficient to support a general conclusion. Evaluating just one model, trained on one dataset and applying it to a few target conditions does not constitute a rigorous or comprehensive assessment. The conclusion lacks external validity and is not grounded in enough empirical breadth to answer the question as posed.", "was_revised": true, "final_justification": "The authors have addressed a lot of the concerns addressed by the reviewers in their rebuttals.", "rebuttal": "Thank you for your review and comments. We appreciate your recognition of the value of our theoretical analysis and the usefulness of our experiments.\n\n\n\n**Clarification of differentiation to prior work on data augmentations**\n\nThe point we were trying to make is not that others have not looked at augmentations in the context of limited data. Our work looks at two different dimensions of data scarcity: the number of participants, and the amount of data available for each participant. In contrast, the only previous piece of work systematically investigating the impact of augmentations in EEG has only considered a single dimension encapsulating the total amount of data available.\n\nMoreover, we would like to emphasise that we provide additional reasons why our work is different in lines 85-89: “Furthermore, our experiments were conducted on much larger datasets with over 1,000 participants each, allowing us to study the impact of augmentations across a broader range of data regimes. Finally, their experiments were limited to a single model per task, whereas we benchmarked all augmentations with different model architectures for each task and observed differential effects.”\n\n\n\n**Representativeness of selected model for self-supervised learning**\n\nWe will rephrase this claim in the manuscript to make our intentions clearer. LaBraM provides an example of a state-of-the-art method for self-supervised pre-training in EEG-based machine learning.\n\nWe would like to remark that the use of only one SSL method does not impact our analysis of research question Q4. Our research question is stated as follows: “Q4: Can self-supervised pre-training effectively increase performance across different data regimes, including limited participant counts or data per participant?” This does not require us to show that *all* self-supervised learning methods increase performance. We can answer this question by showing that at least one self-supervised learning approach works.\n\n\n\n**Evidence for claim in line 239 (doubling participants would yield “roughly 1%” improvement)**\n\nThis claim refers to the observations stated earlier in the paragraph. Specifically, we refer to normality prediction on the TUAB dataset in Figure 2, where extrapolating the curve after 1600 participants, we would expect roughly a 1% improvement if the participant count was doubled for the TCN or mAtt model. The same is already observable (so we don’t have to extrapolate) when moving from 800 to 1600 participants. Finally, this can also be seen by comparing the top 2 rows in any of the plots in Figure 3. We will clarify this in the manuscript.\n\n\n\n**Motivation for selected augmentations**\n\nFirst we would like to clarify that Gaussian noise and cropping are not augmentations studied in our work. We studied AmplitudeScaling, FrequencyShift, and PhaseRandomization.\n\nAs discussed in the related works section, our selection was informed by the systematic evaluation of EEG augmentations conducted by Rommel et al. In their benchmarks on two tasks (sleep staging and BCI motor imagery), PhaseRandomization (termed FTSurrogate) consistently emerged as the most effective augmentation. FrequencyShift showed small improvements when data was more limited. Inclusion of the FrequencyShift augmentation was further motivated, because differences in peak frequencies have been shown to be associated with demographic variables like age and sex [1, 2, 3], making it an interesting augmentation to study in the context of participant diversity. In addition, we also studied AmplitudeScaling, an augmentation introduced by Mohsenvand et al. in the context of a contrastive learning approach, but not evaluated in the review by Rommel et al.\n\nImportantly, Rommel et al. also reported that other augmentations (Gaussian noise, channel symmetry, channel shuffle, time masking, bandstop filtering) did not provide consistent and significant improvements in performance, which is why we did not include them in our work. The two augmentations SignFlip and TimeReverse showed some improvements in limited settings, but were not included in our study because they do not preserve the structure of EEG data. Finally, while ChannelDropout seemed to provide limited value in the BCI task, a closer look revealed that the dropout rate was tuned to 1 (i.e. dropout of all channels), hinting at an issue with the evaluation task or the training, rather than genuine augmentation benefit.\n\nWe will rephrase the relevant part of the related work section in the camera-ready version of the manuscript to make the motivation for our choice of augmentation methods more clear.\n\nTaken together, and given the more comprehensive evaluation of a wide range of augmentations in prior work that informed our study, we believe the three selected augmentations provide an informative picture of contemporary EEG data augmentations. We would also like to add that we do not exclude the possibility that more effective augmentations could be developed as discussed and encouraged in the discussion section of our manuscript.\n\n\n[1] Aurlien, H et al. ‘EEG Background Activity Described by a Large Computerized Database’. Clinical Neurophysiology, 2004.\n\n[2] Chiang, A. K. I. et al. ‘Age Trends and Sex Differences of Alpha Rhythms Including Split Alpha Peaks’. Clinical Neurophysiology, 2011.\n\n[3] Cellier, Dillan et al. ‘The Development of Theta and Alpha Neural Oscillations from Ages 3 to 24 Years’. Developmental Cognitive Neuroscience, 2021."}]

Author rebuttal:

Meta-review: This work proposes an in depth and well executed study of the impact of evaluating ML model on neural EEG data from novel subjects (can be seen as out of domain generalisation where new subjects are new domains). Reviewers have appreciated the detailed rebuttal and consider their concerns addressed. While the ML contribution can be judged limited for NeurIPS, the detailed discussion and new insights are considered interesting to move the field forward, justifying an endorsement for publication.
Decision: Accept (poster)