NeuroRVQ: Multi-Scale EEG Tokenization for Generative Large Brainwave Models

## Abstract

Abstract Electroencephalography (EEG) captures neural activity across multiple temporal and spectral scales, yielding signals that are rich but complex for representation learning.
Recently, EEG foundation models trained to predict masked signal-tokens have shown promise
for learning generalizable representations.
However, their performance is hindered by their signal tokenization modules. Existing neural tokenizers fail to preserve high-frequency dynamics, limiting their ability
to reconstruct EEG signals with high fidelity. We introduce NeuroRVQ , a scalable Large Brainwave Model (LBM) centered on a codebook-based tokenizer. Our tokenizer integrates: ( i ) multi-scale feature extraction modules that capture the full frequency neural spectrum; ( ii ) hierarchical residual vector quantization (RVQ) codebooks
for high-resolution encoding; and,
( iii ) an EEG signal phase- and amplitude-aware loss function for efficient training. This design enables efficient EEG compression while supporting accurate reconstruction across all frequency bands, leading to robust generative masked modeling. Our empirical results demonstrate that NeuroRVQ achieves lower reconstruction error and outperforms existing LBMs on a variety of downstream tasks. More broadly, the NeuroRVQ tokenizer establishes a strong prior for codebook-based general-purpose brainwave models, enabling advances in neural decoding, generative modeling and multimodal biosignal integration.

## 1 Introduction

Brain-Computer Interface (BCI) systems
enable direct communication between the brain and the external world
by analyzing brainwaves recorded by Electroencephalography (EEG) devices. EEG signals can represent the full spectrum of human experience, from sleep and emotions to movement. Decoding EEG signals is a fundamental component of BCIs that find application in various areas such as emotion recognition  (Torres et al., 2020; Xu et al., 2018), epileptic seizure detection  (Alkawadri, 2019; Djoufack Nkengfack et al., 2021) and robotic control  (Irimia et al., 2012).

Brain activity unfolds across multiple frequency scales.
Brainwave signals can be categorized into distinct frequency bands—namely, delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz) and gamma (>30 Hz)—each linked to specific cognitive and physiological states. For example, delta waves dominate during deep sleep, while
alpha waves emerge during relaxing states (especially with closed eyes) but tend to decrease in amplitude during movement.
Together, these bands provide a dynamic window into the brain’s activity and internal states. EEG analysis must capture the multiscale structure of brainwaves, both slow and fast dynamics, to fully represent neural information.

A range of techniques, from advanced signal processing to machine learning, have been employed for brainwave decoding. More recently, Large Brainwave Models (LBMs) offer strong generalization capabilities without relying on BCI
task-specific
data collection and
model training.
These models are typically trained using self-supervised masked learning: EEG signals are discretized into *tokens*; and, during training,
tokens are intentionally masked so
that models learn to reconstruct them. However, reconstructing raw EEG signals remains a significant challenge due to their inherently noisy and complex nature.
Tokenizers
must effectively capture spatio-temporal patterns (e.g., location of recording electrodes, time and frequency) to convert them into meaningful token representations.
Current tokenizers
struggle to
preserve the complete
structural
information (especially high frequency components) necessary for robust generative masked modeling.
The key to unlocking foundation-scale masked modeling for EEG lies in the tokenizer:

In this work, we introduce NeuroRVQ, an efficient and scalable LBM architecture. The main contribution of this work is its state-of-the-art tokenizer that:

- (1)
captures multi-scale frequency features by applying temporal convolutions with varying kernel sizes;
- (2)
encodes hierarchical representations in Residual Vector Quantization (RVQ) codebooks, one per frequency scale,
enabling us to capture the complex patterns necessary
for high-fidelity signal reconstruction in
32 codebooks ($2^{32}$ parameters);
- (3)
employs a training procedure grounded in strong signal processing principles that captures both amplitude and wrapped phase information of the EEG signal via sine and cosine representations.

We evaluate the efficacy of our tokenizer
by comparing NeuroRVQ with state-of-the-art LBMs.
NeuroRVQ achieves up to 15% higher accuracy
on four
BCI classification tasks,
demonstrating the effectiveness
of codebook-based modeling
when codebooks faithfully reconstruct brain signals.

## 2 Background & Related Work

The
roots of EEG analysis
can be traced back to
classical neuroscience methods,
when
handcrafted features
such as
Power Spectral Density (PSD) bands
and Independent Component Analysis (ICA)
were
regarded as
standard (Bashashati et al., 2007; Handy, 2009; Rao, 2013; Nam et al., 2018; McFarland et al., 2006).
While these methods
provided
valuable insights into neural dynamics,
they often failed to generalize
due to
high
inter-subject variability, the noisy nature of EEG signal
and their limited
capacity to capture
the full complexity
of brain activity.

The deep learning era marked a shift towards
data-driven feature extraction:
brainwave decoders such as
EEGNet (Lawhern et al., 2018),
EEGInception (Santamaría-Vázquez et al., 2020),
EEGConformer (Song et al., 2023),
and Brainware-Scattering Net (Barmpas et al., 2023)
reduced reliance on handcrafted features
by automatically learning representations of brain activity.
To
address
domain-specific challenges
such as subject variability,
researchers
incorporated domain-specific knowledge into these architectures, such as subject-alignment to adjust feature representations (Bakas et al., 2022; Wei et al., 2022; Barmpas et al., 2024b).
Despite these advances, most deep learning models remain limited:
they require carefully labeled data and extensive task-specific training,
hindering
their ability to scale
and
generalize
across diverse BCI tasks.

More recently,
the success of large foundation models
in natural language processing (Brown et al., 2020; Touvron et al., 2023)
and computer vision (Mizrahi et al., 2023) has motivated their adoption in EEG decoding.
LaBraM (Jiang et al., 2024),
NeuroGPT (Cui et al., 2024),
CBraMod (Wang et al., 2025), EEGPT (Wang et al., 2024) and
BIOT (Yang et al., 2023)
are examples of
foundation models for
brain signals,
pretrained on massive unlabeled datasets
using self-supervised
signal reconstruction objectives.
These models
represent a promising direction
for capturing complex
signal patterns
beyond task-specific methods. However,
they face a fundamental bottleneck—signal tokenization.

Tokenization is critical for
robust generative masked modeling, yet
current tokenizers
struggle to capture
the full structure
of brain signals.
NeuroGPT relies on a deep backbone-based tokenizer while other models like BIOT and EEGPT on transformer-based tokenizers,
which often learn
entangled and redundant
representations
not optimized for reconstruction.
As a result,
the generated tokens
lack the fidelity required for masked prediction.
In contrast, LaBraM
adopts a codebook-based approach
to learn reconstructable
neural tokens.
However, while discrete codebook-based
tokenizers such as VQ-VAE (Esser et al., 2020)
work well in computer vision,
directly adopting them
for brain signals have
not yielded faithful signal reconstruction—until NeuroRVQ.

## 3 NeuroRVQ Tokenizer

### 3.1 Model Architecture

NeuroRVQ tokenizer is a specialized network designed to convert raw EEG signals into a sequence of compact and informative neural tokens. This transformation reduces the inherently high-dimensional and noisy nature of EEG into a structured lower-dimensional representation that preserves essential temporal–spectral patterns. In doing so, the tokenizer provides a kind of "neural grammar" for brain activity.

Figure [1](https://arxiv.org/html/2510.13068v1#S3.F1) illustrates the tokenizer architecture and processing flow.
The input multi-variate timeseries is first segmented into EEG patches. These patches are encoded by the multi-scale temporal encoder, that captures features in multiple resolutions and are then combined via the transfromer encoder. For each scale, RVQ codebooks discretize the embeddings into a sequence of neural tokens. Finally, these tokens are combined and passed through the tokenizer decoder to reconstruct the input EEG patches using the Fourier spectrum.

Figure: Figure 1: a) The NeuroRVQ tokenizer model architecture; b) the multi-scale temporal encoder architecture; and c) the temporal branch architecture. The multi-scale temporal encoder extracts multi-scale temporal features that are passed to the transformer encoder. RVQ codebooks per temporal branch discretize the EEG patches into neural tokens. The tokenizer decoder reconstructs the EEG signal by using the Fourier spectrum.
Refer to caption: images/tokenizer_new_new_new_na_2.png

Generating Patches: Let $X\in\mathbb{R}^{C\times T}$ denote the input EEG signal, where $T$ is the number of time points and $C$ is the number of electrodes. The signal is first segmented into temporal patches. To ensure that the tokenizer can accommodate EEG signals with arbitrary numbers of channels and variable durations, we adopt the following approach: during model pre-training, each input multi-variate EEG signal is represented by $P$ patches of length $w$ (corresponding to a 1-second time window). This results in a segmented input sample
$\mathbf{x}\in\mathbb{R}^{P\times w}$ which provides flexibility to handle heterogeneous EEG recordings and facilitates robust pre-training across diverse datasets.

Multi-Scale Temporal Encoder: The first step is to extract multi-scale temporal features from each input patch using an inception-style module with $S$ distinct temporal scales. To achieve this, we apply 1-D temporal convolutions with varying kernel sizes $K_{\text{temporal}_{1}},K_{\text{temporal}_{2}},\dots,K_{\text{temporal}_{S}}$. Each temporal branch consists of the following sequence of operations: a 1-D convolution, followed by group normalization, GELU activation and pooling (this sequence of operations is repeated twice). This design enables the network to capture temporal features at multiple scales resulting in $S$ outputs $F_{1},F_{2},\dots,F_{S}$ where $F_{i}\in\mathbb{R}^{w}$ where $w$ is the patch time window length.

Transformer Encoder: Since the tokenization of EEG signal takes place in
a per-patch level, the vital temporal and spatial information is incorporated through the use of trainable temporal $TE$ and spatial $SE$ embeddings, each with embedding dimension $w$. $SE=\{se_{1},...,se_{|\mathcal{C}|}\}$ is indexed by the positions of the patches’ electrodes within a list of all electrodes in the entire pre-training database $\mathcal{C}$ and $TE=\{te_{1},...,te_{P}\}$ since each input consists of $P$ patches of length $w$.
These embeddings are added to the extracted multi-scale features $F_{1},F_{2},\dots,F_{S}$ and are shared across all $S$ temporal branches. The combined embeddings of each temporal branch are then passed through a series of shared transformer layers (Vaswani et al., 2023) resulting in the multi-scale patch representations $\mathbf{p}_{1},\mathbf{p}_{2},\dots,\mathbf{p}_{S}\in\mathbb{R}^{D}$. The shared transformer layers have incorporated the modification of Dehghani et al. (2023).

RVQ Codebook: To efficiently discretize the multi-scale patch representations $\mathbf{p}_{1},\mathbf{p}_{2},\dots,\mathbf{p}_{S}$ while preserving fine-grained information of the EEG signal, we use $S$ Residual Vector Quantization (RVQ) codebooks. For each temporal branch, the RVQ codebook $\mathcal{R}$ is defined as:

$$ $\mathcal{R}=\{\mathcal{V}_{i}|i=1,...,N\}$ (1) $$

where $\mathcal{V}_{i}$ is a single codebook of neural tokens defined as:

$$ $\mathcal{V}_{i}=\{\mathbf{v}_{j}|j=1,...,K\}\in\mathbb{R}^{K\times D}$ (2) $$

where $K$ is the number of discrete neural tokens and $D$ is the dimensionality of each token. The discretization of the patch representations is performed iteratively. For each $\mathcal{R}$, at step 1, a quantizer maps each patch representation $\mathbf{p}^{1}$ to its corresponding neural token $\mathbf{v}^{1}\in\mathcal{V}_{1}$. Specifically, for each patch, the quantizer selects the nearest neighbor $\mathbf{z_{1}}$ from the codebook $\mathcal{V}_{1}$:

$$ $\mathbf{z_{1}}=\arg\min_{\mathbf{v}\in\mathcal{V}_{1}}\left\|l_{2}(\mathbf{p}^{1})-l_{2}(\mathbf{v})\right\|$ (3) $$

At each next step $i+1$, the residual patch representation $\mathbf{p}^{i+1}$ for the next codebook $\mathcal{V}_{i+1}$ is updated by subtracting the selected token $\mathbf{z}_{i}$:

$$ $\mathbf{p}^{i+1}=\mathbf{p}^{i}-\mathbf{z}_{i}$ (4) $$

This process is repeated for all $N$ codebooks, producing a sequence of tokens $\mathbf{z}_{1},\mathbf{z}_{2},\dots,\mathbf{z}_{N}$ for each temporal branch separately. The final reconstructed patch representations $\hat{\mathbf{p}}$ is obtained by summing the contributions of all selected tokens $\mathbf{z}_{1},\mathbf{z}_{2},\dots,\mathbf{z}_{N}$:

$$ $\hat{\mathbf{p}}=\sum_{i=1}^{N}\mathbf{z}_{i}$ (5) $$

At the end of this module, we have $\hat{\mathbf{p}}_{1},\hat{\mathbf{p}}_{2},\dots,\hat{\mathbf{p}}_{S}$ corresponding to the $S$ temporal branches.

Tokenizer Decoder: The purpose of the decoder module (a series shared transformer layers followed by prediction heads) is to reconstruct the original signal information based on the learned codebook tokens. Directly reconstructing raw EEG signals has been shown to yield unstable training and poor performance, due to the inherently noisy and high-dimensional nature of EEG. Following the strategy of LaBraM (Jiang et al., 2024), we instead leverage the Fourier spectrum (more specifically the phase $\phi$ and spectral amplitude $A$) as a more structured target for reconstruction.

### 3.2 Tokenizer Training

Reconstructing the phase $\phi$ directly with mean squared error (MSE) is suboptimal due to the periodic nature of the Fourier phase (see Appendix [A](https://arxiv.org/html/2510.13068v1#A1)). To address this, our architecture replaces raw phase reconstruction with a sine/cosine phase representation and we predict the $\sin\hat{\phi}$ and $\cos\hat{\phi}$, which are smooth, continuous and better suited as loss terms. For the amplitude component, rather than reconstructing raw spectral magnitudes $A$, we operate in the log-amplitude domain $\log(1+A)$. This logarithmic scaling compresses the large dynamic range of EEG spectra and places relatively greater emphasis on reconstructing high-frequency components, a key weakness in existing tokenizers. Therefore, the tokenizer decoder has three prediction heads responsible for reconstructing $\log(1+\hat{A})$, $\sin\hat{\phi}$, and $\cos\hat{\phi}$, respectively.

Naively treating the sine and cosine components independently by applying mean squared error to each dimension separately during training fails to explicitly capture the circular nature of phase: minimizing the MSE of sine and cosine individually does not guarantee that the resulting vector lies on the unit circle and it may allow predictions that deviate from valid phase angles. To address these limitations, we introduce a unit-circle-aware phase loss $\mathcal{L}_{\text{unit-loss}}$ that directly optimizes the directional alignment of the predicted phase vector with the target phase while enforcing unit-norm constraints. Specifically, we represent each phase as a 2D vector ($\cos\phi,\sin\phi$) and compute the cosine similarity between predicted and target vectors, combined with a small regularization term that penalizes deviations from unit length:

$$ ${\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{L}_{\text{unit-loss}}}=1-\underbrace{\sum_{i}\frac{\cos{\hat{\phi_{i}}}\cos{\phi_{i}}+\sin{\hat{\phi_{i}}}\sin{\phi_{i}}}{\sqrt{\cos{\hat{\phi_{i}}}^{2}+\sin{\hat{\phi_{i}}}^{2}}\,\sqrt{\cos{\hat{\phi_{i}}}^{2}+\sin{\hat{\phi_{i}}}^{2}}}}_{{\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\text{Cosine Similarity}}}\;+\;\underbrace{\lambda_{circle}\cdot\sum_{i}\big(\cos{\hat{\phi_{i}}}^{2}+\sin{\hat{\phi_{i}}}^{2}-1\big)^{2}}_{{\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\text{Unit-Circle Penalization Term}}}$ (6) $$

Figure: (a) Prediction not constrained to unit circle.
Refer to caption: images/phase_loss_mse_clear.png

This formulation has several advantages: it naturally respects the circular topology of phase, avoids discontinuities near angular boundaries and ensures that all predicted vectors correspond to valid phase angles. The phases are essentially represented as unit vectors in 2D space, making the errors between them more geometrically meaningful.

Finally, although reconstructing raw EEG waveforms is unstable, we found that incorporating a temporal-domain reconstruction loss provides additional guidance. Specifically, we transform the predicted log-amplitude and phase back to the time domain (the predicted log-amplitudes are exponentiated and rescaled to recover the original amplitude spectrum before inverse Fourier transformation) and compute a temporal MSE term. The NeuroRVQ tokenizer is thus trained end-to-end with the following objective:

$$ $\mathcal{L}_{T}=\sum_{i}\underbrace{\left\|\log(1+\hat{A_{i}})-\log(1+A_{i})\right\|_{2}^{2}}_{{\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\text{Log-Amplitude Loss}}}+{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{L}_{\text{unit-loss}}}+\underbrace{\left\|\hat{X_{i}}-X_{i}\right\|_{2}^{2}}_{{\color[rgb]{1,.5,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,.5,0}\text{Temporal Loss}}}+\mathcal{L}_{Q},$ (7) $$

where $\hat{A}$, $\sin\hat{\phi}$, and $\cos\hat{\phi}$ denote reconstructed Fourier components, $\hat{X}$ denotes reconstructed EEG in the time domain and $\mathcal{L}_{Q}$ is the quantization loss as defined in Esser et al. (2020).

## 4 NeuroRVQ Foundation Model

Figure: Figure 3: NeuroRVQ Foundation Model. During pre-training, random EEG patches are masked, and the model is trained to reconstruct the missing tokens from the RVQ codebook using the surrounding visible patches as context.
Refer to caption: images/second_stage_na_new_2.png

NeuroRVQ is a scalable
foundation model
that operates
on the tokenized representation.
By working at the token level
rather than raw signals,
this model can better
capture long-range dependencies,
learn abstract neural dynamics and enable efficient pretraining across diverse EEG datasets.
The
model leverages
the learned codebooks
$\mathcal{R}$
from the tokenizer stage
and is trained
using a masked-token prediction strategy,
where a subset of input patches
is randomly masked.
This objective encourages
the network to infer missing
tokens from their surrounding context.

Figure [3](https://arxiv.org/html/2510.13068v1#S4.F3) shows the architecture of NeuroRVQ foundation model.
The architecture consists of a multi-scale temporal encoder (Section [3.1](https://arxiv.org/html/2510.13068v1#S3.SS1)) that extracts hierarchical temporal features across multiple resolutions, followed by a series of transformer layers that model long-range complex dependencies. A series of token prediction heads reconstructs the masked tokens from the contextualized embeddings using the trained $\mathcal{R}$ codebooks, which are used only during the pre-training of the backbone model. In line with LaBraM, a symmetric masking strategy is employed, generating the spatially inverse of each original mask during training to enhance efficiency and data diversity. By combining hierarchical temporal encoding, transformer-based context modeling and symmetric masking, the foundation model effectively captures rich spatio-temporal patterns and produces high-fidelity token predictions. For downstream tasks, the pre-trained backbone is extended with task-specific classifications heads.

## 5 Experiments

We evaluate NeuroRVQ along two dimensions (*i*) the quality of its tokenizer,
compared with the tokenizer of LaBraM,
the current state-of-the-art codebook-based method
for signal reconstruction;
and (*ii*)
the performance of the NeuroRVQ foundation model,
compared with other
pre-trained models, fine-tuned for downstream classification tasks.

### 5.1 Experimental Setup

Hyperparameters / Training Process: NeuroRVQ was trained using the settings described in Appendix [B](https://arxiv.org/html/2510.13068v1#A2). The tokenizer was trained for 100 epochs with $S=4$ temporal branches and 4 $\mathcal{R}$ RVQ codebooks each consisting of 8 single codebooks $\mathcal{V}_{i}\in\mathbb{R}^{8192\times 128}$ and $\lambda_{circle}=0.4$. The foundation model was trained for 50 epochs in the same datasets. The experiments were run on NVIDIA DGX with 4 NVIDIA Tesla V100 GPUs.

Datasets: We have used 13 large-scale EEG datasets, consisting of 12 public and 1 self-collected ($\sim$235 hours of our own self-collected motor data) datasets (we refer the reader to Appendix [D](https://arxiv.org/html/2510.13068v1#A4)). All datasets were resampled to the same sampling frequency of 200Hz.

### 5.2 Tokenizer Evaluation

The first step involves comparing the tokenization capabilities LaBraM (Jiang et al., 2024) with NeuroRVQ. We performed both (*i*) in-distribution and (*ii*) out-of-distribution signal reconstruction comparisons at a per-patch level.

#### 5.2.1 In-Distribution Evaluation

To ensure a fair comparison and to properly evaluate the in-distribution reconstruction capabilities, both architectures (LaBraM and NeuroRVQ) were trained on the same datasets,
with the same train-validation split,
and identical training regime, as described in Section [5.1](https://arxiv.org/html/2510.13068v1#S5.SS1)
(see Appendix [E](https://arxiv.org/html/2510.13068v1#A5) for the learning curves). Figure [4](https://arxiv.org/html/2510.13068v1#S5.F4) illustrates the reconstruction of a single EEG signal/patch using the LaBraM and NeuroRVQ tokenizers.

Figure: Figure 4: Per-Band Analysis of Reconstructed EEG signal from LaBraM and NeuroRVQ codebook-based tokenizers. Green lines denote the input EEG signal, orange the reconstructed EEG signal for LaBraM (ours) and red the reconstructed EEG signal for NeuroRVQ.
Refer to caption: images/reconstructions_new_new.png

Table [1](https://arxiv.org/html/2510.13068v1#S5.T1) reports
the overall
reconstruction performance in the validation set
across frequency bands. Across all bands, NeuroRVQ achieves
orders-of-magnitude lower reconstruction error
compared to LaBraM, demonstrating
that NeuroRVQ can produce faithful and robust representation of EEG signals across both broadband and band-specific components.

**Table 1: In-distribution validation mean squared error (MSE) reconstruction performance across frequency bands for NeuroRVQ and LaBraM tokenizers. Both tokenizers were trained for 100 epochs.**
|  | Raw Signal | Delta | Theta | Alpha | Beta | Gamma |
| --- | --- | --- | --- | --- | --- | --- |
| LaBraM (ours) | 1.071 | 1.561 | 0.184 | 0.099 | 0.122 | 0.020 |
| NeuroRVQ | 0.016 | 0.006 | 0.002 | 0.002 | 0.005 | 0.002 |

#### 5.2.2 Out-of-Distribution Evaluation

We further evaluated the generalization capabilities of both NeuroRVQ and LaBraM in reconstructing EEG signals in two datasets unseen during training: a motor dataset High Gamma (Schirrmeister et al., 2017) and a working memory dataset (Pavlov et al., 2022). For LaBraM, we used two variations: (*i*) the version trained with the same datasets as NeuroRVQ, highlighted as *ours*; and (*ii*) the authors’ pre-trained model, highlighted as *original*. For each dataset, input patches were extracted using the class-related parts of each trial in the dataset.

Tables [2](https://arxiv.org/html/2510.13068v1#S5.T2) and [3](https://arxiv.org/html/2510.13068v1#S5.T3) show that
NeuroRVQ consistently outperforms both variations of
LaBraM across all frequency bands,
demonstrating the robustness and superior generalization of NeuroRVQ,
even when evaluated on EEG signals from entirely unseen datasets.

**Table 2: Out-of-distribution mean squared error (MSE) reconstruction performance on memory task across frequency bands for NeuroRVQ and LaBraM tokenizers (original pre-trained and ours).**
|  | Raw Signal | Delta | Theta | Alpha | Beta | Gamma |
| --- | --- | --- | --- | --- | --- | --- |
| LaBraM (orig.) | 1.985 | 1.093 | 0.216 | 0.191 | 0.211 | 0.054 |
| LaBraM (ours) | 1.880 | 2.757 | 0.361 | 0.352 | 0.161 | 0.031 |
| NeuroRVQ | 0.084 | 0.046 | 0.006 | 0.009 | 0.022 | 0.010 |

**Table 3: Out-of-distribution mean squared error (MSE) reconstruction performance on motor task across frequency bands for NeuroRVQ and LaBraM tokenizers (original pre-trained and ours).**
|  | Raw Signal | Delta | Theta | Alpha | Beta | Gamma |
| --- | --- | --- | --- | --- | --- | --- |
| LaBraM (orig.) | 2.004 | 0.863 | 0.326 | 0.400 | 0.350 | 0.064 |
| LaBraM (ours) | 1.893 | 3.313 | 0.669 | 0.721 | 0.339 | 0.049 |
| NeuroRVQ | 0.090 | 0.032 | 0.010 | 0.017 | 0.038 | 0.009 |

### 5.3 Downstream Performance

We evaluated the downstream task performance of fine-tuned NeuroRVQ in comparison to other fine-tuned LBMs,
namely
NeuroGPT (Cui et al., 2024),
CBraMod (Wang et al., 2025),
LaBraM (Jiang et al., 2024),
EEGPT and
BIOT (Yang et al., 2023).(^1^11BIOT could not be tested on Sleep since the benchmark electrodes are missing from the pre-trained model.) All models were evaluated in four downstream classification tasks (Lee et al., 2025): (*a*) Motor, a four-class movement classification on High Gamma (Schirrmeister et al., 2017) dataset; (*b*) Memory, a binary working memory classification on Pavlov et al. (2022); (*c*) Sleep, a 6-class sleep stage classification on Physionet’s Sleep-EDF (Kemp et al., 2000); and (*d*) Eyes, eyes open vs. closed classification on Physionet’s Motor (Schalk et al., 2004).These tasks
capture a diverse range of BCI
paradigms and the datasets were specifically chosen for their minimal spurious artifacts, reducing the likelihood of spurious performance during
training (Lee et al., 2025).

We applied the same full fine-tuning procedure to all models, training for 20 epochs since performance plateaued beyond that point. For the sleep task, however, we employed early stopping, as the models began to overfit before reaching 20 epochs. For evaluation, we used a consistent 10-fold subject-independent cross-validation across all models and reported balanced accuracy for each task. (For downstream performance comparison between codebook-based LBMs pre-trained on the same datasets, see Appendix [F](https://arxiv.org/html/2510.13068v1#A6)).

**Table 4: Comparison of classification balanced accuracy between NeuroRVQ and other LBMs, finetuned on the same datasets using identical training settings (^∗ denotes early stopping). Bold and underlined values indicate best performance and next-best performance respectively (per task or overall). Size denotes the number of parameters for the backbone foundation models excluding the classification heads (see Appendix [G](https://arxiv.org/html/2510.13068v1#A7)).**
| Model | Motor | Memory | Sleep^∗ | Eyes | Mean | Size |
| --- | --- | --- | --- | --- | --- | --- |
| NeuroGPT | 0.682$\pm$0.083 | 0.597$\pm$0.029 | 0.674$\pm$0.033 | 0.827$\pm$0.036 | 0.695$\pm$0.045 | 79.5M |
| CBraMod | 0.614$\pm$0.104 | 0.574$\pm$0.038 | 0.635$\pm$0.041 | 0.839$\pm$0.041 | 0.666$\pm$0.056 | 4.9M |
| BIOT | 0.443$\pm$0.079 | 0.510$\pm$0.018 | – | 0.763$\pm$0.049 | – | 3.2M |
| LaBraM | 0.630$\pm$0.076 | 0.526$\pm$0.026 | 0.652$\pm$0.037 | 0.799$\pm$0.047 | 0.652$\pm$0.047 | 5.8M |
| EEGPT | 0.313$\pm$0.035 | 0.520$\pm$0.017 | 0.634$\pm$0.044 | 0.797$\pm$0.037 | 0.566$\pm$0.033 | 25.7M |
| NeuroRVQ | 0.700$\pm$0.073 | 0.574$\pm$0.027 | 0.728$\pm$0.028 | 0.869$\pm$0.026 | 0.717$\pm$0.038 | 5.9M |

Table [4](https://arxiv.org/html/2510.13068v1#S5.T4) shows that, when full-finetuned,
NeuroRVQ achieves the best or next-best performance in each task,
and the best overall performance,
measured as the mean accuracy across all tasks.
NeuroRVQ has 5.9M parameters in the backbone model
and approximately 3,000 parameters in the classification heads.
Yet, it achieves higher mean accuracy
than NeuroGPT, which has 79.5M parameters in its backbone;
and it also outperforms CBraMod, which has
similar backbone parameters to NeuroRVQ but
much larger classification heads (approximately 50M parameters; see Appendix [G](https://arxiv.org/html/2510.13068v1#A7) for a detailed comparison of classification head sizes).
This combination of strong generalization
and compact size highlights
the effectiveness of our NeuroRVQ tokenizer.

## 6 Discussion

The NeuroRVQ tokenizer demonstrates state-of-the-art reconstruction capabilities on both in-distribution and out-of-distribution signals, as shown in Section [5.2](https://arxiv.org/html/2510.13068v1#S5.SS2). Importantly, these results cannot be attributed solely to the size of the codebook but also reflect the inductive biases embedded in the tokenizer design together with the specifically designed training process, which are particularly well suited to the characteristics and complex nature of EEG signals.
The main contribution of this work is the NeuroRVQ codebook, which can be integrated into larger LBM architectures and may enable further performance gains. Further improvements could also come from incorporating training principles inspired by causal reasoning, such as more targeted temporal and spatial masking strategies (Barmpas et al., 2024a). All these directions are beyond the scope of this work and could be promising avenues for future research.

## 7 Conclusion

In this work, we introduce NeuroRVQ, an LBM that leverages a Residual Vector Quantization (RVQ) codebook-based tokenizer. Our main contribution is the design and proper training of the NeuroRVQ tokenizer, which captures low- and high-frequency components of EEG signals, achieves state-of-the-art reconstruction performance and facilitates the development of more effective LBMs. We further showed that NeuroRVQ outperforms existing pre-trained LBMs on various downstream tasks. In summary, NeuroRVQ advances the development of efficient LBMs by providing a high-fidelity tokenizer. The principles underlying NeuroRVQ’s tokenizer can be extended to other biosignals, paving the way for broader applications of foundation models in the biosignal space.

#### LLM Declaration

Portions of this manuscript were edited for clarity and grammar with the assistance of a large language model. All scientific content, results and conclusions are the authors’ own.

## References

- Alkawadri (2019)
Rafeed Alkawadri.
Brain–Computer Interface (BCI) Applications in Mapping of Epileptic Brain Networks Based on Intracranial-EEG: An Update.
*Frontiers in Neuroscience*, 13:191, 2019.
ISSN 1662-453X.
doi: 10.3389/fnins.2019.00191.
- Bakas et al. (2022)
Stylianos Bakas, Siegfried Ludwig, Konstantinos Barmpas, Mehdi Bahri, Yannis Panagakis, Nikolaos Laskaris, Dimitrios A. Adamos, and Stefanos Zafeiriou.
Team Cogitat at NeurIPS 2021: Benchmarks for EEG Transfer Learning Competition, 2022.
doi: 10.48550/arXiv.2202.03267.
- Barmpas et al. (2023)
Konstantinos Barmpas, Yannis Panagakis, Dimitrios A Adamos, Nikolaos Laskaris, and Stefanos Zafeiriou.
BrainWave-Scattering Net: a lightweight network for EEG-based motor imagery recognition.
*Journal of Neural Engineering*, 20(5):056014, September 2023.
ISSN 1741-2552.
doi: 10.1088/1741-2552/acf78a.
- Barmpas et al. (2024a)
Konstantinos Barmpas, Yannis Panagakis, Dimitrios Adamos, Nikolaos Laskaris, and Stefanos Zafeiriou.
A Causal Perspective in Brainwave Foundation Models.
In *Causality and Large Models @NeurIPS 2024*, 2024a.
doi: 10.1088/1741-2552/ad3eb5.
- Barmpas et al. (2024b)
Konstantinos Barmpas, Yannis Panagakis, Georgios Zoumpourlis, Dimitrios A Adamos, Nikolaos Laskaris, and Stefanos Zafeiriou.
A causal perspective on brainwave modeling for brain–computer interfaces.
*Journal of Neural Engineering*, 21(3):036001, may 2024b.
doi: 10.1088/1741-2552/ad3eb5.
- Bashashati et al. (2007)
Ali Bashashati, Mehrdad Fatourechi, Rabab K Ward, and Gary E Birch.
A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals.
*Journal of Neural Engineering*, 4(2):R32–R57, mar 2007.
doi: 10.1088/1741-2560/4/2/r03.
- Blankertz et al. (2007)
Benjamin Blankertz, Guido Dornhege, Matthias Krauledat, Klaus-Robert Müller, and Gabriel Curio.
The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects.
*NeuroImage*, 37(2):539–550, 2007.
ISSN 1053-8119.
doi: https://doi.org/10.1016/j.neuroimage.2007.01.051.
- Brown et al. (2020)
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.
Language Models are Few-Shot Learners, 2020.
doi: 10.48550/arXiv.2005.14165.
- Buckwalter et al. (2021)
Grace Buckwalter, Sidney Chhin, Sabariah Rahman, Iyad Obeid, and Joseph Picone.
Recent Advances in the TUH EEG Corpus: Improving the Interrater Agreement for Artifacts and Epileptiform Events.
In *2021 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)*, pp. 1–3, 2021.
doi: 10.1109/SPMB52430.2021.9672302.
- Cui et al. (2024)
Wenhui Cui, Woojae Jeong, Philipp Thölke, Takfarinas Medani, Karim Jerbi, Anand A. Joshi, and Richard M. Leahy.
Neuro-GPT: Towards A Foundation Model for EEG, 2024.
doi: 10.48550/arXiv.2311.03764.
- Dehghani et al. (2023)
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, and Neil Houlsby.
Scaling Vision Transformers to 22 Billion Parameters, 2023.
doi: 10.48550/arXiv.2302.05442.
- Detti et al. (2020)
Paolo Detti, Giampaolo Vatti, and Garazi Zabalo Manrqiue de Lara.
EEG Synchronization Analysis for Seizure Prediction: A Study on Data of Noninvasive Recordings.
*Processes*, 8:846, 07 2020.
doi: 10.3390/pr8070846.
- Djoufack Nkengfack et al. (2021)
Laurent Chanel Djoufack Nkengfack, Daniel Tchiotsop, Romain Atangana, Valérie Louis-Door, and Didier Wolf.
Classification of EEG signals for epileptic seizures detection and eye states identification using Jacobi polynomial transforms-based measures of complexity and least-square support vector machine.
*Informatics in Medicine Unlocked*, 23:100536, 2021.
ISSN 2352-9148.
doi: 10.1016/j.imu.2021.100536.
- Esser et al. (2020)
Patrick Esser, Robin Rombach, and Björn Ommer.
Taming Transformers for High-Resolution Image Synthesis, 2020.
doi: 10.48550/arXiv.2012.09841.
- Handy (2009)
Todd C. Handy.
*Brain Signal Analysis: Advances in Neuroelectric and Neuromagnetic Methods*.
The MIT Press, 07 2009.
ISBN 9780262013086.
doi: 10.7551/mitpress/9780262013086.001.0001.
- Irimia et al. (2012)
D.C. Irimia, R. Ortner, G. Krausz, C. Guger, and M.S. Poboroniuc.
BCI Application in Robotics Control.
*IFAC Proceedings Volumes*, 45(6):1869–1874, 2012.
ISSN 1474-6670.
doi: 10.3182/20120523-3-RO-2023.00432.
14th IFAC Symposium on Information Control Problems in Manufacturing.
- Jiang et al. (2024)
Weibang Jiang, Liming Zhao, and Bao liang Lu.
Large brain model for learning generic representations with tremendous EEG data in BCI.
In *The Twelfth International Conference on Learning Representations*, 2024.
- Kemp et al. (2000)
Bastiaan Kemp, Aeilko H. Zwinderman, Bert Tuk, Hilbert A. C. Kamphuisen, and Josefien J. L. Oberyé.
Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG.
*IEEE Transactions on Biomedical Engineering*, 47(9):1185–1194, 2000.
doi: 10.1109/10.867928.
- Korczowski et al. (2019)
Louis Korczowski, Martine Cederhout, Anton Andreev, Grégoire Cattan, Pedro Rodrigues, Violette Gautheret, and Marco Congedo.
Brain Invaders calibration-less P300-based BCI with modulation of flash duration Dataset (bi2015a), 07 2019.
doi: 10.5281/zenodo.3266930.
- Lawhern et al. (2018)
Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, and Brent J. Lance.
EEGNet: A Compact Convolutional Neural Network for EEG-based Brain–Computer Interfaces.
*Journal of Neural Engineering*, 15(5):056013, July 2018.
ISSN 1741-2552.
doi: 10.1088/1741-2552/aace8c.
- Lee et al. (2025)
Na Lee, Stylianos Bakas, Konstantinos Barmpas, Yannis Panagakis, Dimitrios Adamos, Nikolaos Laskaris, and Stefanos Zafeiriou.
Assessing the Capabilities of Large Brainwave Foundation Models.
In *Workshop on Spurious Correlation and Shortcut Learning: Foundations and Solutions*, 2025.
- Luciw et al. (2014)
Matthew Luciw, Ewa Jarocka, and Benoni Edin.
Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and friction.
*Scientific data*, 1:140047, 11 2014.
doi: 10.1038/sdata.2014.47.
- Margaux et al. (2012)
Perrin Margaux, Emmanuel Maby, Sébastien Daligault, Olivier Bertrand, and Jérémie Mattout.
Objective and Subjective Evaluation of Online Error Correction during P300-Based Spelling.
*Advances in Human-Computer Interaction*, 2012, 01 2012.
doi: 10.1155/2012/578295.
- McFarland et al. (2006)
Dennis J. McFarland, Charles W. Anderson, Klaus-Robert Müller, Alois Schlögl, and Dean J. Krusienski.
BCI meeting 2005-workshop on BCI signal processing: feature extraction and translation.
*IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 14(2):135–138, 2006.
doi: 10.1109/TNSRE.2006.875637.
- Mizrahi et al. (2023)
David Mizrahi, Roman Bachmann, Oguzhan Fatih Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir.
4M: Massively Multimodal Masked Modeling.
In *Thirty-seventh Conference on Neural Information Processing Systems*, 2023.
doi: 10.48550/arXiv.2312.06647.
- Nam et al. (2018)
Chang S. Nam, Anton Nijholt, and Fabian Lotte (eds.).
*Brain-Computer Interfaces Handbook: Technological and Theoretical Advances*.
CRC Press (Taylor & Francis), January 2018.
ISBN 9781498773430.
- Pavlov et al. (2022)
Yuri G. Pavlov, Dauren Kasanov, Alexandra I. Kosachenko, and Alexander I. Kotyusov.
"EEG, pupillometry, ECG and photoplethysmography, and behavioral data in the digit span task and rest", 2022.
doi: 10.18112/openneuro.ds003838.v1.0.2.
- Rao (2013)
Rajesh P.N. Rao.
*Brain-Computer Interfacing: An Introduction*.
Cambridge University Press, USA, 2013.
ISBN 0521769418.
- Santamaría-Vázquez et al. (2020)
Eduardo Santamaría-Vázquez, Víctor Martínez-Cagigal, Fernando Vaquerizo-Villar, and Roberto Hornero.
EEG-Inception: A Novel Deep Convolutional Neural Network for Assistive ERP-Based Brain-Computer Interfaces.
*IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 28(12):2773–2782, 2020.
doi: 10.1109/TNSRE.2020.3048106.
- Schalk et al. (2004)
Gerwin Schalk, Dennis J McFarland, Thilo Hinterberger, Niels Birbaumer, and Jonathan R Wolpaw.
BCI2000: a general-purpose brain-computer interface (BCI) system.
*IEEE Trans. Biomed. Eng.*, 51(6):1034–1043, June 2004.
doi: 10.1109/TBME.2004.827072.
- Schirrmeister et al. (2017)
Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball.
Deep learning with convolutional neural networks for EEG decoding and visualization.
*Human brain mapping*, 38(11):5391–5420, 2017.
doi: 10.48550/arXiv.1703.05051.
- Shah et al. (2018)
Vinit Shah, Eva von Weltin, Silvia López, James Riley McHugh, Lily Veloso, Meysam Golmohammadi, Iyad Obeid, and Joseph Picone.
The Temple University Hospital Seizure Detection Corpus, 2018.
doi: 10.48550/arXiv.1801.08085.
- Song et al. (2023)
Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao.
EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization.
*IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 31:710–719, 2023.
doi: 10.1109/TNSRE.2022.3230250.
- Torkamani-Azar et al. (2019)
Mastaneh Torkamani-Azar, Sumeyra Demir Kanik, Serap Aydin, and Mujdat Cetin.
Prediction of Reaction Time and Vigilance Variability from Spatiospectral Features of Resting-State EEG in a Long Sustained Attention Task, 2019.
doi: 10.48550/arXiv.1910.10076.
- Torres et al. (2020)
Edgar P. Torres, Edgar A. Torres, Myriam Hernández-Álvarez, and Sang Guun Yoo.
EEG-Based BCI Emotion Recognition: A Survey.
*Sensors*, 20(18), 2020.
ISSN 1424-8220.
doi: 10.3390/s20185083.
- Touvron et al. (2023)
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.
LLaMA: Open and Efficient Foundation Language Models, 2023.
doi: 10.48550/arXiv.2302.13971.
- Trujillo (2020)
Logan Trujillo.
Raw EEG Data, 2020.
doi: 10.18738/T8/SS2NHB.
- Trujillo et al. (2017)
Logan T. Trujillo, Candice T. Stanfield, and Ruben D. Vela.
The Effect of Electroencephalogram (EEG) Reference Choice on Information-Theoretic Measures of the Complexity and Integration of EEG Signals.
*Frontiers in Neuroscience*, Volume 11 - 2017, 2017.
ISSN 1662-453X.
doi: 10.3389/fnins.2017.00425.
- Vaswani et al. (2023)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention Is All You Need, 2023.
doi: https://doi.org/10.48550/arXiv.1706.03762.
- Veloso et al. (2017)
Lily Veloso, James R. McHugh, Eva von Weltin, Silvia López, Iyad Obeid, and Joseph Picone.
Big data resources for EEGs: Enabling deep learning research.
In *2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)*, pp. 1–3, 2017.
doi: 10.1109/SPMB.2017.8257044.
- Wang et al. (2024)
Guangyu Wang, Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li.
EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals.
In *The Thirty-eighth Annual Conference on Neural Information Processing Systems*, 2024.
- Wang et al. (2025)
Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan.
CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding.
In *The Thirteenth International Conference on Learning Representations*, 2025.
doi: 10.48550/arXiv.2412.07236.
- Wei et al. (2022)
Xiaoxi Wei, A Aldo Faisal, Moritz Grosse-Wentrup, Alexandre Gramfort, Sylvain Chevallier, Vinay Jayaram, Stylianos Bakas Camille Jeunet, Siegfried Ludwig, Konstantinos Barmpas, Mehdi Bahri, Yannis Panagakis, Nikolaos Laskaris, Dimitrios A Adamos, Stefanos Zafeiriou, William C Duong, Stephen M Gordon, Vernon J Lawhern, Maciej Śliwowski, Vincent Rouanne, and Piotr Tempczyk.
2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets, 2022.
doi: 10.48550/arXiv.2202.12950.
- Xu et al. (2018)
Tao Xu, Yun Zhou, Zi Wang, and Yixin Peng.
Learning Emotions EEG-based Recognition and Brain Activity: A Survey Study on BCI for Intelligent Tutoring System.
*Procedia Computer Science*, 130:376–382, 2018.
ISSN 1877-0509.
doi: j.procs.2018.04.056.
The 9th International Conference on Ambient Systems, Networks and Technologies (ANT 2018) / The 8th International Conference on Sustainable Energy Information Technology (SEIT-2018) / Affiliated Workshops.
- Yang et al. (2023)
Chaoqi Yang, M Westover, and Jimeng Sun.
BIOT: Biosignal Transformer for Cross-data Learning in the Wild.
In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), *Advances in Neural Information Processing Systems*, volume 36, pp. 78240–78260. Curran Associates, Inc., 2023.
doi: 10.48550/arXiv.2305.10351.

## Appendix A Sine-Cosine Phase Representation

We describe the benefits of using a
sine-cosine representation of the
Fourier phase $\phi$ in the loss function,
instead of the raw phase values.

LaBraM (Jiang et al., 2024) computes
phase loss
directly
as the squared error between
predicted $\hat{\phi}$ and true $\phi$:

$$ $\mathcal{L}_{\phi}=||\hat{\phi}-\phi||^{2}.$ (8) $$

This formulation, however,
is discontinuous
at the boundaries
of the phase domain $[-\pi,\pi]$.
For instance, with
$\hat{\phi}=\pi-\epsilon$ and
$\phi=-\pi+\epsilon$
for small $\epsilon>0$,

$$ $\mathcal{L}_{\phi}=||\hat{\phi}-\phi||^{2}=(2\pi-2\epsilon)^{2}.$ (9) $$

The loss approaches $4\pi^{2}$,
even though the two angles are nearly identical.
Thus, small phase shifts
near the $\pm\pi$ boundary
(e.g., $2^{\circ}$ between $179^{\circ}$ and $-179^{\circ}$)
result in large discontinuous
jumps in the loss,
leading to unstable gradients
and poor convergence.

To address this issue,
we map phase $\phi$
to its sine and cosine components,
a point on the unit circle:

$$ $r(\phi)=(\sin(\phi),\cos(\phi))\in\mathbb{R}^{2},$ (10) $$

The corresponding
loss is:(^2^22Although the NeuroRVQ architecture uses the $\mathcal{L}_{\text{unit-loss}}$ as described in the manuscript, we use $\mathcal{L}_{r(\phi)}$ here to illustrate the effectiveness of the sine-cosine representation.)

$$ $\mathcal{L}_{r(\phi)}=||\sin(\hat{\phi})-\sin(\phi)||^{2}+||\cos(\hat{\phi})-\cos(\phi)||^{2}=2-2\cos(\hat{\phi}-\phi)$ (11) $$

Our formulation
is continuous across the $\pm\pi$ boundary.
Revisiting the earlier example with $\hat{\phi}=\pi-\epsilon$ and $\phi=-\pi+\epsilon$,

$$ $\mathcal{L}_{r(\phi)}=2-2\cos(2\epsilon)\approx 4\epsilon^{2}$ (12) $$

which smoothly approaches 0 as $\epsilon\rightarrow 0$.

More generally, for any two phases $\phi_{1}$ and $\phi_{2}$:

$$ $\mathcal{L}_{r(\phi)}=2-2\cos(\phi_{1}-\phi_{2})=4\sin^{2}\left(\frac{\phi_{1}-\phi_{2}}{2}\right).$ (13) $$

## Appendix B Model Configuration & Hyperparameter Settings

This section details:
(*i*) the configuration of NeuroRVQ’s
multi-scale temporal encoder (§[B.1](https://arxiv.org/html/2510.13068v1#A2.SS1)),
and
(*ii*) the hyperparameter settings used for
training the tokenizer and foundation model,
as well as fine-tuning the latter on
downstream tasks (§[B.2](https://arxiv.org/html/2510.13068v1#A2.SS2)).

### B.1 Multi-Scale Temporal Encoder Architecture

**Table 5: Configuration of the multi-scale temporal encoder, used identically in both the tokenizer and the foundation model. Here, $x\rightarrow y$ indicates values for the first ($x$) and second ($y$) sequence.**
| Branch № | Filters | Kernel | Padding | Pooling |
| --- | --- | --- | --- | --- |
| 1 | 8 $\rightarrow$ 8 | (1, 21) $\rightarrow$ (1, 9) | (0, 10) $\rightarrow$ (0, 4) | (1, 2) $\rightarrow$ (1, 4) |
| 2 | 8 $\rightarrow$ 8 | (1, 15) $\rightarrow$ (1, 7) | (0,   7) $\rightarrow$ (0, 3) | (1, 2) $\rightarrow$ (1, 4) |
| 3 | 8 $\rightarrow$ 8 | (1,   9) $\rightarrow$ (1, 5) | (0,   4) $\rightarrow$ (0, 2) | (1, 2) $\rightarrow$ (1, 4) |
| 4 | 8 $\rightarrow$ 8 | (1,   5) $\rightarrow$ (1, 3) | (0,   2) $\rightarrow$ (0, 1) | (1, 2) $\rightarrow$ (1, 4) |

Our temporal encoder comprises four temporal branches.
Each branch applies the following sequence of operations twice in succession (Figure [1](https://arxiv.org/html/2510.13068v1#S3.F1)c):
an 1-D convolution,
followed by group normalization
with
$N=4$ groups and $C=8$ channels,
GELU activation, and pooling.
Table [5](https://arxiv.org/html/2510.13068v1#A2.T5)
lists the
filter counts,
kernel sizes,
padding,
and pooling parameters
for the two sequences,
where $x\rightarrow y$
denotes
the configuration
value(s)
used
in the first and second sequence,
respectively.

### B.2 Hyperparameter Settings

**Table 6: Hyperparameters**
| Hyperparameter | Tokenizer | Pre-training FM | Finetuning |
| --- | --- | --- | --- |
| Temporal Inception Branches $S$ | 4 | 4 | 4 |
| RVQ Codebook $\mathcal{R}$ | 4 | 4 | 4 |
| Single VQ Codebook $\mathcal{V}_{i}$ | 8 | 8 | 8 |
| Batch size | 256 | 64 | 64 |
| Learning rate scheduler | Cosine | Cosine | Linear |
| Base learning rate | 5e-5 | 5e-4 | 5e-4 |
| Min learning rate | 1e-5 | 1e-5 | - |
| Warmup lr start-end factors | - | - | (0.1,1) |
| Lr start-end factors | - | - | (1,0.1) |
| Total epochs | 100 | 50 | 20 |
| Warmup epochs | 10 | 5 | 4 |
| Optimizer | AdamW | AdamW | AdamW |
| Weight decay | 1e-4 | 0.05 | 0.01 |
| Adam $\beta$ | (0.9, 0.999) | (0.9, 0.999) | (0.9, 0.999) |
| Layer lr decay | - | - | 0.975 |
| Gradient clipping | 3 | - | - |
| Layer scale init | 0.001 | 0.001 | - |
| Encoder depth | 12 | 12 | 12 |
| Decoder depth | 3 | - | - |
| Hidden dimension | 200 | 200 | 200 |
| No. Attention heads | 10 | 10 | 10 |
| MLP hidden dimension | 800 | 800 | 800 |

Table [6](https://arxiv.org/html/2510.13068v1#A2.T6)
summarises the hyperparameters used for
training the tokenizer and for
pre-training and fine-tuning
the foundation model.
Certain
choices (e.g., batch size)
were constrained by
hardware,
while others
were guided by
ablation studies (e.g., using
8 codebooks per RVQ yielded
the lowest reconstruction error;
see §[C](https://arxiv.org/html/2510.13068v1#A3)).

## Appendix C Number of Codebooks per RVQ

Figure: Figure 5: Validation MSE reconstruction loss for the NeuroRVQ tokenizer with different numbers of codebooks (layers) per RVQ.
Refer to caption: images/ablation.png

We conducted an ablation
study to determine the best
number of codebooks
(layers) per RVQ.
The tokenizer was trained
using our self-collected dataset
(see §[D](https://arxiv.org/html/2510.13068v1#A4)),
varying the number of layers from 2 to 8.
Figure [5](https://arxiv.org/html/2510.13068v1#A3.F5)
shows that using eight layers
$\mathcal{V}_{i}\in\mathbb{R}^{8192\times 128}$
achieve the best
reconstruction performance on the validation set.

## Appendix D Datasets

**Table 7: Datasets used for pre-training NeuroRVQ, with BCI paradigm and number of EEG recording channels.**
| Datasets | BCI paradigm | # channels |
| --- | --- | --- |
| BCI Competition IV-1 (Blankertz et al., 2007) | Motor | 64 |
| Grasp and Lift (Luciw et al., 2014) | Motor | 32 |
| Physionet MI (Schalk et al., 2004) | Motor | 64 |
| bi2015a (Korczowski et al., 2019) | ERP | 32 |
| Inria BCI Challenge (Margaux et al., 2012) | ERP | 56 |
| Trujillo (2020) | II-RB | 64 |
| Trujillo et al. (2017) | Resting | 64 |
| SPIS Resting (Torkamani-Azar et al., 2019) | Resting | 64 |
| TUAR (Buckwalter et al., 2021) | Artifacts (e.g., eye movement) | 31 |
| Siena Scalp (Detti et al., 2020) | Epilepsy | 31 |
| TUEP (Veloso et al., 2017) | Epilepsy | 31 |
| TUSZ (Shah et al., 2018) | Seizure | 31 |
| Self-Collected Dataset | Motor | 29 |

We use a collection of
publicly available EEG datasets
to pre-train
NeuroRVQ.
Table [7](https://arxiv.org/html/2510.13068v1#A4.T7) summarizes the datasets,
their associated BCI paradigms,
and the number of EEG recording channels.
The collection spans
multiple experimental settings,
including
motor imagery,
event-related potentials (ERP),
resting-state activity,
information-integration and
rule-based cognitive categorization (II-RB),
and clinical data for
epilepsy and
seizures.

We additionally include
a proprietary dataset
comprising
approximately 235 hours
of motor imagery data
recorded with 29 channels.
Participants in this
data collection
were instructed as follows:

> “We will record your brain waves while you engage in simple tasks. These include real and imagined movements of the extremities, and a game which uses movements as controls. The collected data will be used to train algorithms that can be used for many things, such as facilitating navigation in virtual environments, seamless user interaction with objects in computer gaming or even rehabilitation training of post-stroke patients towards regaining the ability to control their limbs. The project received approval from the Research Governance and Integrity Team. If you want to help out our research team you can register to be a participant in our experiment! There is a small gift of £30 to every participant.”

## Appendix E NeuroRVQ vs. LaBraM: Reconstruction Performance

We train NeuroRVQ and LaBraM tokenizers
on the same datasets
with the same train–validation split
and an identical training regime
(including preprocessing and training steps),
as described in §[5.1](https://arxiv.org/html/2510.13068v1#S5.SS1).
Figure [6](https://arxiv.org/html/2510.13068v1#A5.F6) shows the validation learning curves
for both models.
NeuroRVQ achieves nearly two orders of magnitude
lower reconstruction MSE compared to LaBraM.

Figure: Figure 6: Validation learning curves showing reconstruction MSE for NeuroRVQ and LaBraM tokenizers, trained under identical regimes.
Refer to caption: images/train_plots_new.png

## Appendix F NeuroRVQ vs. LaBraM: Downstream Task Performance

We
compare
the downstream task performance
of
NeuroRVQ with LaBraM
when both models
are pre-trained
and fine-tuned on
the
same datasets
and with
an identical training regime.
We use 10-fold subject-independent
cross-validation to evaluate models
on the four classification
benchmarks described in §[5.3](https://arxiv.org/html/2510.13068v1#S5.SS3).
Table [8](https://arxiv.org/html/2510.13068v1#A6.T8)
shows that, when all else is equal,
NeuroRVQ achieves 1%–13% higher balanced accuracy than LaBraM
on all tasks,
demonstrating
the effectiveness
of our
tokenizer in training codebook-based models.

**Table 8: Comparison of classification balanced accuracy between NeuroRVQ and LaBraM, both pre-trained and finetuned on the same datasets using identical settings. (^∗ denotes early stopping)**
| Model | Motor | Memory | Sleep^∗ | Eyes |
| --- | --- | --- | --- | --- |
| LaBraM (ours) | 0.570 | 0.565 | 0.715 | 0.805 |
| NeuroRVQ | 0.700 | 0.573 | 0.728 | 0.869 |

## Appendix G Classification Heads: Architecture & Size

Earlier,
in Table [4](https://arxiv.org/html/2510.13068v1#S5.T4),
we compared
the downstream classification performance
of NeuroRVQ against five state-of-the-art models:
CBraMod, BIOT, EEGPT, NeuroGPT and LaBraM.
Here, we report the number of parameters
in the task-specific classification head
for each model.
CBraMod and NeuroGPT employ
a 3-layer MLP as their
classification heads, according to their original implementations,
whereas
the others use a single layer.
The head sizes vary with the number of target
classes (Motor has 4, Memory 2, Sleep 6 and Eyes 2);
and for CBraMod, also
with the number of EEG channels.
Table [9](https://arxiv.org/html/2510.13068v1#A7.T9)
show that NeuroRVQ has relatively low number of parameters compared to other models,
both in the backbone
and in the classification heads.

**Table 9: Parameter counts for model backbones and task-specific classification heads.**
| Model | Backbone | Motor Head | Memory Head | Sleep Head | Eyes Head |
| --- | --- | --- | --- | --- | --- |
| CBraMod | 4.9M | 50,081,804 | 40,481,201 | 73,207,406 | 41,121,201 |
| NeuroGPT | 79.5M | 270,756 | 270,657 | 270,822 | 270,657 |
| BIOT | 3.2M | 1,028 | 257 | - | 257 |
| LaBraM | 5.8M | 804 | 201 | 1,206 | 201 |
| EEGPT | 25.7M | 260 | 65 | 390 | 65 |
| NeuroRVQ | 5.9M | 3,204 | 801 | 4,806 | 801 |

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 2
presentation: 3
questions: 1. The RVQ uses one codebook per frequency scale—how was the number of scales (e.g., delta-gamma) chosen, and what happens if bands overlap or vary by task? 

2. Reconstruction error is lower than baselines, but does this translate to better perceptual quality? 

3. Downstream gains are up to 15%—are these consistent across subjects/datasets, or driven by specific ones? Details on variance and p-values would help assess robustness.

4. The model is for generative masked modeling, but how does it compare to non-codebook tokenizers (e.g., BIOT's transformer) in masking ratios or pretraining efficiency? 

5. For broader impact, test on non-EEG biosignals (e.g., ECG)—does the multi-scale approach generalize, or is it EEG-specific? 

I will consider raising my score if all questions and concerns are solved.
soundness: 2
strengths: The paper addresses a timely challenge in EEG foundation models: improving tokenization to capture multi-scale frequency dynamics, which is crucial for noisy, complex brain signals. The multi-scale convolution and RVQ design incorporate domain knowledge (e.g., EEG frequency bands), potentially aiding reconstruction fidelity. The phase/amplitude loss is a thoughtful addition grounded in signal processing. Experiments cover reconstruction metrics and downstream tasks, with claims of superior performance, and the work highlights scalability for LBMs.
summary: This paper presents NeuroRVQ, a large brainwave model (LBM) for EEG signals that focuses on a multi-scale tokenizer using temporal convolutions with varying kernel sizes for frequency band extraction, hierarchical residual vector quantization (RVQ) codebooks (one per scale), and a phase- and amplitude-aware loss for training. The tokenizer aims to enable high-fidelity reconstruction across frequency bands (delta to gamma) for generative masked modeling. Evaluated on reconstruction error and four downstream BCI tasks (e.g., emotion recognition, seizure detection), it claims up to 15% accuracy gains over existing LBMs like LaBraM, BIOT, and NeuroGPT, positioning it as a foundation for neural decoding and multimodal integration.
weaknesses: Novelty is marginal: The tokenizer builds directly on RVQ from VQ-VAE (Esser et al., 2020) and multi-scale convolutions common in EEG models (e.g., EEGNet's depthwise convolutions for bands; Lawhern et al., 2018), without introducing new architectural or theoretical elements—e.g., the hierarchical codebooks per scale are a straightforward extension, and the loss is similar to phase-aware objectives in audio (e.g., EnCodec).

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: See weaknesses.
soundness: 2
strengths: The manuscript is clearly written and well-structured. The extensive use of tables and figures effectively presents detailed results, ablation studies, and interpretability analyses, which greatly clarify the authors' model choices. The supplemental materials are thorough and provide valuable additional support.
summary: This paper introduces NeuroRVQ, a scalable Large Brainwave Model (LBM) architecture for EEG data, which features multi-scale temporal convolutions, hierarchical residual vector quantization (RVQ) codebooks, and a training loss that jointly reconstructs phase and amplitude information using a sine-cosine representation. Extensive experiments demonstrate superior reconstruction error and downstream classification performance compared to prior codebook and transformer-based LBMs.
weaknesses: 1. Applying RVQ and phase-aware loss to EEG modeling is not novel; see more details in [1,2]. For example, BrainOmni replaces the original VQ with RVQ to separate different source components, and combines both time and frequency domain loss to guide pretraining.

2. The datasets evaluated in this work (Table 4) appear to have little overlap with the datasets evaluated by the baseline models [3-7], which poses a significant challenge to head-to-head model comparisons.

3. The multi-scale feature extraction module does not appear to integrate information from different scales into a single token (Line 740). While this operation does not affect the model size, it significantly increases computation time. Furthermore, this operation constitutes an implicit model ensemble, leading to unfair comparisons between models.

**References**:

[1] Xiao Q, Cui Z, Zhang C, et al. BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals[J]. arXiv preprint arXiv:2505.18185, 2025.

[2] Carzaniga F S, Hoppeler G T, Hersche M, et al. The Case for Cleaner Biosignals: High-fidelity Neural Compressor Enables Transfer from Cleaner iEEG to Noisier EEG[J]. arXiv preprint arXiv:2502.17462, 2025.

[3] Wang J, Zhao S, Luo Z, et al. Cbramod: A criss-cross brain foundation model for eeg decoding[J]. arXiv preprint arXiv:2412.07236, 2024.

[4] Jiang W B, Zhao L M, Lu B L. Large brain model for learning generic representations with tremendous EEG data in BCI[J]. arXiv preprint arXiv:2405.18765, 2024.

[5] Cui W, Jeong W, Thölke P, et al. Neuro-gpt: Towards a foundation model for eeg[C]//2024 IEEE International Symposium on Biomedical Imaging (ISBI). IEEE, 2024: 1-5.

[6] Yang C, Westover M, Sun J. Biot: Biosignal transformer for cross-data learning in the wild[J]. Advances in Neural Information Processing Systems, 2023, 36: 78240-78260.

[7] Wang G, Liu W, He Y, et al. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals[J]. Advances in Neural Information Processing Systems, 2024, 37: 39249-39280.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. The authors mention training the tokenizer for 100 epochs and the foundation model for 50 epochs. Was this choice due to limited data quantity or diversity, or could it risk overfitting? This point should be more thoroughly discussed.

2. How were the hyperparameters (e.g., S = 4, λ = 0.4) selected? Were hyperparameter sensitivity analyses conducted to justify these values?

3. Please report the number of subjects in the training and evaluation datasets, as this is essential for assessing cross-subject generalization.

4. In the in-distribution and out-of-distribution experiments, the tokenizer is compared only with LaBraM. Could the limited number of baselines undermine the strength of the conclusions?

5. The authors are encouraged to include missing baselines (such as the series work of Brant [1] and Brant-2 [2]), or at least explain why these works are not suitable for comparison, and discuss them in the related work section to reflect completeness and fairness.

6. As this is a large-scale EEG model, have the authors considered testing the model and baselines under a zero-shot setting? This would better demonstrate the generalization capabilities of the proposed approach.

7. It is recommended that the authors release their model code to promote reproducibility and community engagement.

> [1] Brant: foundation model for intracranial neural signal. (https://papers.neurips.cc/paper_files/paper/2023/file/535915d26859036410b0533804cee788-Paper-Conference.pdf)
> 
> [2] Brant-2: Foundation Model for Brain Signals. (https://www.arxiv.org/abs/2402.10251v4)
soundness: 2
strengths: - Although using frequency-domain information across EEG bands is common in existing brainwave foundation models, this paper’s approach effectively leverages such domain knowledge grounded in neuroscience principles, leading to more biologically informed EEG modeling.

- The paper offers clear and insightful analysis on the role of the EEG tokenizer, and proposes a well-designed hierarchical improvement strategy that enhances existing tokenization methods. The methodology is technically sound and clearly presented.

- The case analysis in Figure 4 vividly and intuitively demonstrates the performance advantages of the proposed codebook-based tokenizer, helping readers grasp the model’s effectiveness.
summary: The paper introduces NeuroRVQ, a scalable Large Brainwave Model (LBM) for EEG signal modeling that addresses limitations in existing signal tokenization methods. It employs a codebook-based tokenizer integrating (i) multi-scale feature extraction, (ii) hierarchical residual vector quantization, and (iii) a phase- and amplitude-aware loss for efficient training. This design enables accurate reconstruction across all frequency bands and efficient EEG compression, outperforming existing approaches in reconstruction quality and downstream tasks.
weaknesses: - The authors report training the tokenizer for 100 epochs and the foundation model for 50 epochs. For large models, this is unusually high — typical large-scale training often involves vast data with fewer epochs (e.g., 1–2). Such a high number may suggest insufficient training data or diversity, or risk overfitting.

- Although the paper lists the datasets used, the number of subjects in the training and evaluation sets is not clearly specified. This metric is crucial for assessing generalization, as limited subject diversity can restrict the universality of conclusions.

- The baselines in the four classification tasks are not comprehensive enough. For example, Table 4 includes models up to 79.5M parameters, but omits larger baselines such as Brant (≈500M), which could weaken the strength of the empirical comparisons.

- The absence of released code is a reproducibility concern, especially for a work proposing a new large-scale EEG foundation model.


Decision: Reject