Tokenizing Single-Channel EEG with Time-Frequency Motif Learning
Authors: Jathurshan Pradeepkumar
Xihao Piao
Zheng Chen
Jimeng Sun
Venue: ICLR 2026 Poster (2026)
Keywords: EEG
Tokenization
Representation learning
Biosignal Foundation Models
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose TFM-Tokenizer, a EEG tokenization framework that encodes single-channel EEG into discrete tokens, improving downstream performance, enhancing existing FMs as a plug-in component, and scaling to other brain signal modalities.

Abstract: Foundation models are reshaping EEG analysis, yet an important problem of EEG tokenization remains a challenge. 
This paper presents TFM-Tokenizer, a novel tokenization framework that learns a vocabulary of time-frequency motifs from *single-channel* EEG signals and encodes them into discrete tokens. 
We propose a dual-path architecture with time–frequency masking to capture robust motif representations, and it is model-agnostic, supporting both lightweight transformers and existing foundation models for downstream tasks. 
Our study demonstrates three key benefits:
*Accuracy:* Experiments on four diverse EEG benchmarks demonstrate consistent performance gains across both single- and multi-dataset pretraining settings, achieving up to $11\%$ improvement in Cohen’s Kappa over strong baselines.
*Generalization:* Moreover, as a plug-and-play component, it consistently boosts the performance of diverse foundation models, including BIOT and LaBraM. 
*Scalability:* By operating at the single-channel level rather than relying on the strict 10–20 EEG system, our method has the potential to be device-agnostic.
Experiments on ear-EEG sleep staging, which differs from the pretraining data in signal format, channel configuration, recording device, and task, show that our tokenizer outperforms baselines by $14\%$.
A comprehensive token analysis reveals strong class-discriminative, frequency-aware, and consistent structure, enabling improved representation quality and interpretability.
Code is available at https://github.com/Jathurshan0330/TFM-Tokenizer.

Published as a conference paper at ICLR 2026
TOKENIZING SINGLE-CHANNEL EEG WITH TIME- FREQUENCY MOTIF LEARNING
Jathurshan Pradeepkumar" Xihao Piao", Zheng Chen² Jimeng Sun' University Illinois Urbana-Champaign SANKEN, Osaka University (ip65, jimeng) Seaenngg@iiiiioooenoeeneell iillinois. edu, (park88, chenz @aanken. osw.a-u.akk.ooooo
ABSTRACT
Foundation models are reshaping EEG analysis, yet an important problem EEG tokenization remains a challenge. This paper presents TFM-Toker zer, novel tokenization framework that learns vocabulary of time-frequency motifs from single-channel EEG signals and encodes them into discrete tokens. We propose dual-path architecture with time-frequency masking to capture robust motif repre- sentations, and it is model-agnostic, supporting both lightweight transformers and existing foundation models for downstream tasks. Our study demonstrates three key benefits: Accuracy: Experiments four diverse EEG benchmarks demon- strate consistent performance gains across both single- and multi-dataset pretrain- ing settings, achieving up to 11% improvement in Cohen's Kappa over strong baselines. Generalization: Moreover, as plug-and-ple component, it consis- tently boosts the performance of diverse foundation models, including BIOT and LaBraM. Scalability: By operating at the single-channel level rather than rely- ing on the strict 10-20 EEG system, our method has the potential to be device- agnostic. Experiments on ear-EBG sleep staging, which differs from the pretrain- ing data signal format, channel configuration, recording device, and task, show that our tokenizer outperforms baselines by 14%. A comprehensive token analy- sis reveals strong cuuss-dscriminative frequency-awarg and consistent structure, enabling improved representation quality and interpretability. Code is available at https: / / thub. com Jathu Jathurshann3333 wwtpshhh0030000000eeeeeee zer.
INTRODUCTION
Foundation models have revolutionized how machines understand human language, leading to major breakthroughs natural language processing (NLP) (OpenAI et al., 2024; DeepSeek-AI al., 2025) and cross-modality tasks such as text-to-image generation (Bordes et al., 2024). Inspired by this success, researchers are now advancing a paradigm shift in electroencephalogram (EEG) analysis toward task-agnosti foundation models (Mohammadi Foumani et al., 2024; Yang et al., 2024; Jiang et al., 2024b; Wang et al., 2024a). By pretraining on massive, diverse EEG data corpora, these models learn universal representations that generalize well across various downstream tasks.
Despite substantial recent progress, an important open problem remains: how to design an effective tokenization method for EEG signals. Tokenization, into meaningful tokens, which reduces data complexity and introduces a helpful inductive bias in foundation models (Gastaldi et al., 2025). Typically, tokenization performed by learnable func- tion that trains a vocabulary of tokens and statistics from a given corpus. However, existing EEG foundation models tokenize signals by directly segmenting continuous EEGs into short-duration tokens, without learning a vocabulary. They merely discretize EEG signals, failing to capture sta- tistically grounded representations in a data-driven manner. LaBraM (Jiang et al., 2024b) proposes a neural tokenizer to learn data-driven tokens before pretraining. However, these tokens primarily serve as training objectives rather than as actual inputs for subsequent model training and are dis- carded during downstream inference, limiting their reusability. As result, the foundation model is still trained on continuous segment- embeddings. failing to fully leverage the benefits of tok- enization, such improving the quality of input representations. In this paper, study novel and critical problem of developing principled EEG tokenization that seamlessly integrates with various foundation models and enhance downstream performance and generalization.
component in NLP, transforms text

---

Published as a conference paper at ICLR 2026
(a) Single Channel EEG Tokenization
(b) Multi-Channel Flexible
(c) Plug-and-Plas with FMs
(d) Cross-Device Scalability
TFM- TPM-Tokenizer
EEG Tokens
Downstcemm
BIOT
100%
P -
57
LaBraM
Ear-EEG based Skoop Monitoring
Tokenn may capture
dlincallly usetul pattenss
Multi-Channel EEG
Labust TrM
Labed muran
Figure I: (a) Our TFM-Tokenizer converts single-channel EEG into discrete tokens by capturing time- freuufryy motifs. (b) It is adaptable to any different multi-channel settings, (c) can be inte- grated with existing foundation models to enhance their performance, and (d) enables cross-device scalability.
Various studies have shown that developing effective tokenization is a non-trivial task in general, as it is influenced by multiple factors (Schmidt et al., 2024). In this paper, we recognize and focus on three key challenges of EEG tokenization. 1) Tokenization target: real-world EEG recordings exhibit diverse formats due to varying devices, channel configurations, and recording lengths ( (Yang al., 2024). We argue that tokenizers should be trained and operated at the single-channel level to learn channel-agnostic discrete tokens. This design enables flexible adaptation to multi-channel tasks and can generalize non-standard EEG devices. In Section 4.4, we provide scalability ex- periments on ear-EEG settings. 2) Token resolution: in NLP, tokenization can be defined at different resolutions (characters, subwords, words), each reflecting different assumptions about se- mantic granularity. However, EEG signals are characterized by diverse oscillatory (e.g., alpha, beta) (Pradeepkumar et al., 2024) and transient patterns (e.g., spikes) (Chen et al., 2022). Thus, effective tokens must represent such underlying motifs (Xu et al., 2023) that reflect distinct neural or phys- iological events. Motifs can understood as short, recurring patterns in a time series that exhibit limited variability and often carry discriminative significance (Xu et al., 2023). However, these motifs are often distorted by noise, amplitude scaling, and temporal warping, making it challenging to design robust EEG tokenization methods. 3) Tokenization learning objective: EEGs exhibit various temporal variations, manifested as a mixture of low- and high-frequency components that co-occur and are intermixed in complex ways. Relying solely capturing mased motifs into discrete tokens and expecting the model to implicitly infer spectral structure from raw signals risks overlooking important frequency information. We therefore argue that the tokenization learning objective should explicitly incorporate time fime-freyenny representations, enabling the tokenizer to capture wand-specifie and cross- frequefyy patterns and to encode more meaningful neural motifs
To tackle these challenges, we propose JFM-Tokenize:: zer. a novel EEG tokenization framework that captures rone-frequency motifs from single-channel EEG signals and encodes them into dis- tinct tokens. Specifically. 1) Tokenizing FEGs at single-channel: We tokenize single-channel EEG signals into discrete token sequences akin to NLP models, which are then paired with a generic transformer to perform multi-channel modeling using these single-channel tokens. Our tokenizer is model-agnostic and can be paired with any downstream model. Our experiments confirmed that TFM-Tokenizer can seamlessly integrate with existing foundation models, and further improve their performance (see Figure 1). 2) Learning motif' features as tokens: We introduce a motif learn- ing architecture that encodes time-frequency motifs into tokens through dual-path encoding de- sign. Capturing frequency-band characteristics or compositions is crucial for EEG analysis, and to model such dynamics, we designed Localized Spectral Window Encoder, which isolates and aggre- gates information across frequency bands prior to fusion with temporal features. 3) Explicit time- frequency masking prediction: this learning objective disentangles the entangled time-frequency representations, enabling the model to explicitly learn distinct time. By forcing the model to predict masked regions in both domains, encourages the tokenizer to discover and encode meaningful neural motifs that are localized in time and frequency. Overall, our contributions are summarized as follows:
patterns across
Formulating Single-Channel EEG Tokenization. To our knowledge, we are the first inves- tigate the problem of learning a discrete token vocabulary that captures time frequency motifs in single-channel EEG signals from given corpus and directly utilizes them as inputs for down- stream modeling.
Proposing Novel TFM-Token Framework. We introduce a single-channel EEG tokenization framework that transforms EEG into discrete token sequence via TFM--Tokenizer which is then
2

---

Published as a conference paper at ICLR 2026
used by lightweight transformer model for cross-channel and downstream modeling. As shown in Figure 1c, JFM-Tokenizer integrates smoothly with existing models and consistently boosts performance, improving BIOT and LaBraM by approximately 4% on TUEV dataset.
Broad Evaluation across Foundation Models and Devices. Extensive experiments across four datasets show that our method outperforms strong baselines, achieving up to a 11% gain over the baseline model on TUEV dataset. We also evaluate cross-device scalability on an car-EEG sleep staging task, using electrodes outside the standard 10-20 EEG system, where our tokenizer outperforms baselines by 14%. Beyond performance, we comprehensively analyze token quality, including token consistency, class-specific uniqueness, and frequency learning analysis, validating that our learned tokens are informative and interpretable.
2 RELATED WORK
EEG Foundation Models and Tokenization Methods. Existing EEG foundation models can be categorized into decoding and encoder-based methods. recoding-based methods focus on genera- tive tasks like cross-modal translation (Duan et al., 2023; Liu et al., 2024; Wang et al., 2024c). In contrast, encoder-based methods focus on classification tasks and representation learning. Notable models include LaBraM (Jiang et al., 2024b), BIOT (Yang et al., 2024), BRANT (Zhang et al., 2024), and MMM (Yi al., 2024). Our work aligns with this latter category, aiming enhance input representations to improve classification performance and generalization across diverse foun- dation models. A parallel question is how to tokenize EEG signals. Existing methods primarily adopt segment-based continuous tokenization (Yang et al., 2024; Wang et al., 2024b; Zhang et al., 2024). Vector Quantized (VQ) tokenizers (Van Den Oord et al., 2017), which have been success- ful in tokenizing continuous images (Esser et al., 2020), have recently been adapted for EEG by LaBraM (Jiang et al., 2024b). However, in LaBraMM, the tokenizer is not designed to represent EEG data and replace raw signals inputs to foundation models; instead, it mainly serves training objective. In this paper, we propose a new tokenization framework for EEG signals that encodes inputs into discrete representations and provide reusable interface for foundation models.
EEG Motif Learning. Motifs are short, recurring patterns with small variability in a time series and may hold predictive or discriminative value (Xu al., 2023). In the EEG domain, motif learning has been studied in several prior works, including (Mueen et al., 2009; Barthélemy et al., 2013; Gips al., 2017; Jas et al., 2017: 7; Schäfer & Leser, 2022), though these approaches focus solely on the temporal domain. EEG motifs correspond to neurophy neurophysiological events such as oscillatory bursts or transient spikes, which are best characterized by joint temporal-specttee structure. Frequency- domain modeling is therefore essential, yet raw time-domain signals often entangle multiple spectral components. This can cause models to overemphasize dominant frequency rhythms while over- looking informative high-frequency details (Zhi-Qin John Xu et al., 2020; Piao et al., 2024). Such bias limits the ability to capture diverse EEG waveforms and degrades representation quality (Park & Kim, 2022). To the best of our knowledge, we are the first to propose methods to encode diverse, informative time-frequene motifs as discrete tokens.
METHODOLOGY
3.1 FRAMEWORK OVERVIEW AND FORWARD PROCESS
Our TEM-Tokenizen zer framework consists of two major phase, as shown in Figure 2:
TFM-Tokenizer with Motif Learning. The tokenizer is trained in a single-channel, unsuper- vised setting, capturing key motif features. We regard motifs as various waveforms that encode characteristic fime-frequency patterns in EEGs. To represent these motif's, the tokenizer is com- posed of four components: (i) a Localized Spectral Window Encoder that extracts frequency patterns within short spectral windows, (ii) Temporal Encoder that incorporates raw EEG con- text, (iii) a Temporal Transformer that models dependencies across windows, and (iv) codebook quantizer that maps embeddings into a discrete vocabulary. Therefore, we train a motif-based vo- cabulary that transforms continuous EEGs into interpretable discrete tokens (Sec. 3.2).
2. Downstream Transformer Model. This phase serves as an example to illustrate how a foun- dation model processes tokenized sequences for downstream lasks such as classification. Raw
3

---

Published as a conference paper at ICLR 2026
n. TV TTM--Tkkeie Pretraining
Time-Faauaccy
Vocabulary
d. Downstream Enceder Pretraining
Siggl Clannel
Puthing with Overlass
Tempoeal Encoder
TTM Souator
(x))) 1
Recornmmacion
STPIC1
Tesppral Patching
TrasSmop
Manking
Croo-Patory
Quantzationn
Masking
Localized Spectral
Window Encode:
Masking
Localized Spectral Windew Encoder
Token Predictin Hoal
Gared Patchnise
Agumggation
Maack
Single Speciall Window S/
Owd ain Goodde
auggo sareooannng fead evedialle
8
MG
g:
Madked Prodiction
Tesporal
Figure 2: Overview our framework. (a) TFM-Tokenizer Pretraining: Through dual-path en- coding and masked prediction, learns to capture time-frequency motifs into discrete tokens, (b) Masking Strategy: A combination of frequency band masking and temporal masking used for SEM-Tokenizer pretraining. (c) Localized Spectral Window Encoder: Processes individual spec- tral windows from S, extracts frequency band information, and aggregates features across all bands into single compact embedding per window. (d) Downstream Transformer Encoder Pretraining: Trains on learned EEG tokens using masked token prediction.
EEGs are first passed through our pretrained tokenizer, where they are converted into discrete tokens that serve as inputs to foundation models. Since the tokenizer is model-agnostic, it can be paired with different backbone models. In our implementation, we adopt a lightweight Trans- former (Vaswani et al., 2017) with linear attention (Katharopoulos al., 2020), demonstrating that the tokenizer (~0.7M parameters) enables strong performance even with a compact model (Sec. 3.3).
Overall, we first pretrain the tokenizer to learn a discrete vocabulary of EEG motifs. The tokenizer is then frozen, and the downstream Transformer is pretrained with masked token prediction objective. Finally, the downstream Transformer is tneet on target EEG tasks such as classification.
3.2
FFM-TOKENIZER WITH MOTIF LEARNING
TFM- TTM-Tooenizer encodes EEGs into discrete motifs tokens through dual-path frequency-time paradigm (Figure 2a). Given multi-channel EEG X E CCT, segment each channel signal into overlapping patches of length and hop size H, yielding N = L(T + patches aligned with spectral windows (S. { To define the pretraining task, masking is applied in both temporal and frequency domains (Figure 2b), where unmasked patches provide context and masked ones are reconstructed. Feature learning performed as follows: each spectral window S, is encoded by the Localized Spectral Window Encoder (Figure 2c) and fused with raw EEG patch features through a Temporal Encoder. A Temporal Transformer then integrates the time-frequency features, and the output embeddings are mapped into a learnable VQ vocabulary, producing motif tokens.
Localized Spectral Window Encoder. Capturing frequency-baid characteristics is essential for EEG analysis, as the signals often exhibit oscillatory components (e.g., alpha, beta) with varying am- plitudes and temporal dynamics. Unlike prior work that projects an entire spectral window through single linear layer (Yang et al., 2024), we divide the window into patches along the frequency axis, allowing effective modeling of cross-frequency dependencies. This process consists of three steps.
Frequency Patch Encoder: Given set of spectral windows {S.) we isolate and divide each spectral window S; into P non-overlapping patches (S ) p=1- each spanning A frequency bins such that P.Af F. We then project each frequency patch into a latent space: ((j,)) GroupNorm (GeLU W (((,)))))) where W, E DxAf the parameter matrix that maps each patch into D-dimensional embedding.
4

---

Published as a conference paper at ICLR 2026
Frequency Transformer. We then apply a frequency transformer that operates along the frequency axis of S,, to model intra-spectral window cross-frequency band dependencies.
Gated Patchwise Aggregation. In many EEG scenarios, large portions of the frequency spectrum can be irrelevant. For instance, tasks related to sleep primarily focus on frequency bands up to ap- proximately 32 Hz (Chen et al., 2023). Also, the frequencies of interest vary across conditions and tasks. To emphasize important frequency patches and suppress the rest, we adopt gated aggre- gation mechanism to obtain embedding for each S₁ E = Concat (ar (W g()())))))))))))))))))))))) W (1p(() where W Wg1, W g2 are trainable parameters and (-)) is the element-wise sigmoid function.
Temporal Encoder and Temporal Transformer. To capture temporal dynamics from raw EEG patches {ar)) each patch is projected linearly, followed by GELU activation and group normal- ization, producing temporal embeddings [E]] Each aggregated frequency embedding Ef" is then concatenated with its corresponding temporal embedding ET and the resulting sequence is processed by temporal Transformer. This module integrates time and frequency features across N EEG patches, enabling the modeling of long-range dependencies. Finally, the outputs Z; are quan- tized into discrete tokens using a learnable vocabulary V Notably, we omit positional encoding because EEG signals are inherently non-stationary and often exhibit chaotic dynamics; our objective to capture distinctive features without enforcing positional constraints (see Appendix C.6).
VQ Tokenizer Vocabulary. Our vocabulary is based on the discrete codebook of Vector-Quanticn Variational Autoencoders (VQVAE)) We perform vector quantization to fused embedding Z, that enables the vocabulary to capture time-frequency motifs as discrete tokens, supporting timestamp- level retrieval and improving EEG interpretability. Formally, given Z = {z: } each 2; is mapped to the closest code in the codebook {V1.-... VK) } by nearest-neighhor search.
(Z,) = arg min I. vall.. VYEV
where K denotes the number of latent vectors in the codebook and defines K-way discrete cat- egorical distribution. Each patch 2; is mapped to its nearest code entry U/. As result, given a single-channel EEG X Toke-ioeenzer generates a sequence of N tokens {c; [im]--
Frequency Masking Prediction for Tokenizer Learning
We employ joint requency-temporay masking strategy for gram S is partitioned along the frequency axis into N₂ [F/5y] groups of size oy, and random frequency-band masks Mr and temporal masks Mr are applied obtain the masked input SM Following (Jiang et al., 2024b), we further adopt symmetric masking for data augmentation and training stability. The overall objective combines masked reconstruction and vocabulary loss:
training. The spectro-
Lroken
S(s.(tt) (icT)))-$(ff))))) S(f.))11 + El[sg(E] ex 1 3 Σ E. sglu:]] (f.)
where $ is the reconstruction, sg[.] is the moopggradeent operator, and a, B are hyperparameters. We also apply exponential moving average updates for stable codebook training.
3.3 DOWNSTREAM TRANSFORMER TRAINING
We employ a lightweight transformer model to aggregate tokenized representations across chan- nels, learn cross-channel dependencies and perform downstream tasks. It consists of token- embedding lookup table (initialized from the VQ codebook) followed by linear attention transformer layers. Given a multi-channel recording X E RCT,, the pretrained TFM-Tokenizer produces
ken sequences { [1
for each channel e independently. We flatten the token embeddings across channels and incorporate channel and position embeddings. An addtional class token prepended (Devlin, 2018), and the sequence is processed by transformer layers.
c=1
In order to pretrain the model and enable the model to learn intra and cross-channel dependencies of tokens, we adopt strategy akin to masked language modeling. We first randomly mask tokens across multiple channels and time steps and then train the model to predict these masked tokens via a cross-entropy loss. Along with representation learning, this approach enhances robustness to missing or corrupted data, common world EEG systems where channels or time segments may be dropped noisy. Finally, the transformer model is finetuned for downstream tasks.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS AND RESULTS
4.1 EXPERIMENT SETUP
Datasets: We evaluated our method on four EEG datasets. (1) TUEV (Harati et al., 2015): A subset of the TUH EEG Corpus (Obeid & Picone, 2016), containing clinical EEG recordings notated for six event types: spike and sharp wave (SPSW), generalized periodic epileptiform dis- charges (GPED), periodic lateralized epileptiform discharges (PLED), eye movement (EYEM), ar- tifact (ARTF), and background (BCKG). (2) TUAB (Lopez et al., 2015): Also from Temple Uni- versity Hospital, labeled for normal and abnormal EEG activity. (3) CHB-MIT (Shoeb, 2009): A widely used benchmark for epilepsy seizure detection, comprising EEG recordings from 23 pediatric subjects with intractable seizures.
(4) IICC Seizure (Jing et al., 2023; Ge et al., 2021): I): Designed for detecting six ictal-interictal-iniii continuum (IIIC patterns, including others (OTH), elec- trographic seizures (ESZ), lateralized periodic discharges (LPD), generalized periodic discharges (GPD), lateralized rhythmic delta activity (LRDA), and generalized rhythmic delta activity (GRDA). Scalability Validation. In this paper, we provided a scalability experiment to evalute the usability of our tokenizer across different EEG devices. Since tokenizer is trained in kinle-channel setting, it can naturally be applied to recordings from non-standard devices. Therefore, we evaluated on the Ear-EEG Sleep Monitoring (EESM23) (Bjarke Mikkelsen et al., 2025; Tabar et al., 2024) dataset, which contains ear-EEG sleep recordings from 10 subjects. Detailed dataset statistics, splits, and preprocessing procedures are provided in Appendix B. B.2, and B.3.
Baselines: We evaluated our approach against the baselines from Yang et al. (2024) and recent -fate-of-the-art methods, including BIOT, LaBraM, NeuroLM, and EEGPT. We adopted the best re- sults reported in BIOT, except for the IIIC Seizure dataset, where we re-evaluated the methods due to sample size mismatch. Experiments were conducted under two settings: (1) Single-dataset setting: pretraining and finetuning on the same single dataset, and (2) Multiple dataset setting: pretraining on four EEG datasets. For BIOT, we reproduced their unsupervised pretraining and finetuning pipeline in the single-dataset setting (denoted BIOT*) to enable fair comparison, as their vanilla BIOT vari- ant does not include pretraining. Similarly, we reproduced LaBraM by training its neural tokenizer, performing masked EEG modeling, and finetuning within the same dataset (LaBraM") )) Since focus is on EEG tokenization rather than full foundation modeling, we reproduced LaBraM under the multiple dataset setting using the previously mentioned four EEG datasets (denoted LaBraM"), ) This was necessary to ensure a fair comparison because the original LaBraM used a substantially larger pretraining corpus. Additional experiment details are provided in Appendix B.4 and B.5.
4.2 How DOES
COMPARE TO EXISTING BASELINES?
Table reports results on TUEV (event classification) and TUAB (abnormal detection), while Ta- ble 2 summarizes performance on IIIC-Seizure (seizure type classification) and CHB-MIT (seizure detection). Our T'FM-Tokenng in both experiment settings. On the challenging six-class event-type classification task in TUEV, it achieves a 5% gain in Cohen's 's Kappa in the single-dataset setting and a notable ~ 1 11% improve- ment (0.5588 + 0.6189) in the multi-dataset setting over the next best baseline. On III-Seizere, which is another six-class classification task, IFM-Tokenize improves Cohen's Kappa by 36% over the LaBraM (0.3658 0.4979) and 3% improvement over CBraMod (0.4792 - 0.4979) in multiple dataset settings, demonstrating the strong capability of our tokenizer in modeling class- discriminative features for complex clinical EEG tasks. Additionally, is worth noting that TFM- Tokenizer achieves better performance with fewer parameters, being 3 times smaller than LaBraM and 1.5 times smaller than BIOT. The ability to achieve best performance with low model size can be attributed to our tokenization approach, which compresses the EEG into token sequence, thereby reducing data complexity. Notably, the ToM-Tikenizer is paired with a lightweight transformer comprising only ~0.7M parameters.
paired with a downstream transformer outperforms the baselines
4.3 CAN T TOKENIZER IMPROVE EXISTING FOUNDATION MODELS?
To evaluate the generalizability of TFM-Tokenizer, we integrated it into two representative EEG foundation models, BIOT and LaBraM, under both single- and multi-dataset settings. For BIOT, we replaced raw EEG inputs with token embeddings while following the original training protocol. For

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison on TUEV and TUAB datasets.
Models
Model Size
TUEV (event type clacsification)
TUAB (ahnormal detection)
Balanced Ace.
Cohen's 's Kappa
Weighted F1
Balanced Acc.
AUC-PR
AUROC
Single Dalasel Setting
SPaRCNet Cling 2023)
0.79M
0.4161
0,0262
0,4253 0.0181
0,2024 = 0.0104
0.2896 0.0018
0.8414 = 0.0018
0.8676 +: 0.0012
CoarnWR (Yang et. 2023)
1.6
0.434
0.0549
0,3912 + 0.0257
0.6893 = 0.0136
0.7746
+ 0.0041
0.8421 = 0.0104
0.8456 + 0.0074
CNN-TransfomerrcC (Poh et 2022)
0,4087
0,3815
0.0344
0.655
0.0293
0.7777
0.0022
0.8433
0.5461 +: 0.0013
FFCL (L. ol,, 2022)
2.4M
0.3979
0.0104
0,3732 + 0.0188
0.678= = 0.0120
0,7848 + 0:0038
0.8448 = 0.0066
0,850 + 0.0051
ST-Transfomer (Song ot al., 2021)
3.5M
0.3984
0.0228
0,3765
0.0306
0.0190
0.7966
0.0023
0.8521
0.0066
0.8707 + 0.9019
Vanilla BIOT OYing et ail,, 2024)
3.2M
0.4682
0.0125
0,4482
0.0285
0.7085
0.0184
0,7925
0.0035
0,8707
1 0.0087
0,3691 1. 0.0033
BIOT" (Yang et al,, 2024)
3.2M
0.4679
0,0854
0,4890 + 0.0407
0.7352
0.0236
0.7955
+ 0:0047
0.8819
+ 0.0046
0,8344 + 0.0041
LaBraM- Base" (Jang al,, 20246)
5.8M
0.0856
0,5067
0.0413
0.7466
0.0202
0.7720
0.0046
0.8498
0.0036
0.8534
0.0027
TFM- Tokenizee (Ours)
1.9M
0,4944
1: 0,0516
0,5337
0.0306 0.7570 0.0163
0,8152 1 0.0014
0,8946
1 0.0008
0.5897 1: 0.0008
Multiple Dataset Pretraining
BIOT (Yang al., 202-1)
3.2M
0.5281
0.0225
0,5273 1 0.0249 0.7492 0.0082
0,7459 1 0.0057
0.8792 L 0.0023
0,8815 1 0.0043
REGPT (Wang et al., 2024a)
4.7M
0.5670 4: 0.0066
0,5085 4: 0.0173
0.7535 = 0.0097
0,7959 1: 0.0021
0.8716 1: 0.0041
NouroL (liang ol,, 2024a)
254M
1.0048
0.0028
0.7826 0.0065
0.6975 1: 0.0081
0,7816 0.0079
LaBraM Base' Chang et al., 202
5.8M
0.5550
0,0403
0,5175
0.0339
0,7450 -: 0.0194
0,7735 1: 0.0030
0.853] : 0.0028
0.8557 1. 0.0027
CBraMldd (Wang et al., 20244)
0.5696
0.0221
0,5588
0.0273
0.7702
0.0137
0,5000
0.0000
0,4938 = 0.0443
0.5281 + 0:0409
TFM- Tokenizer (Ours)
1.9M
0.5974 0.0079
0,6189
0.0302 0,8010 0.0161 0,8032 0.0035
0,8886 1 0.0022
0.8570 T 0.0022
Table 2: Performance comparison on IIIC Seizure and CHB-MIT datasets.
Models
Model
HIC Seizure (seizure ty clactification)
CHB-MIT (seizure detection)
Size
Halanced
Cohen's Kappa
Weighted
Ralanced Acc.
AUC-PR
AUROC
Single Daluse Setting
SPaRCNet (Jing et 2023)
0.79M
0,5011
0.0297
0.1996
0.0262 0.5876 0.011
0.1247
0.019
0.8143 1 0.0148
Co (Yang etal, 2023)
1.6M
0.5421
0,0123
0,4549 0.0166
0,5387 - 0.0138
0.6344 + 0.0012
0.2264
+ 0.0144
0,5007 + 0.0114
CNN- SoN--Trmsoomerr (Peh et al., 2022)
3.2M
0.5395
0.0144
0,4500
0.0165
0,5113 1. 0.0176
0.6389 1 0.0067
0.2479 L 0.0227
0.3662 1. 0.00K2
FFCL (Li et 2022)
2,4M
0.5309
0.0217
0,4412
0.0253
0.5315
0.0277
0.6262 1: 0,0104
0.2049 1 0.0466
0.8271 1 0.0051
ST- TT-Transfommrr (Song et al., 2021)
3.5M
0,5093
0,0122
0,4217
1 0.0151
0.5217
:: 0.0110
0,5915 1 0.0195
0,1422 : 0.0094
0.8237 1: 0.0491
Vanilla BIOT (Yang
2024)
3.2M
0.5762
0.4932
0.0066
0,5773
0.0031
0,6640 0.0037
0.2573
0.0088
0.3646
0.0030
BIOT" (Yang et al,, 20144
3.2M
0.4458
0,0183
0,3418
0.0228
0,4511
0.0207
0.6582 + 0.0896
0.3122 = 0.0890
0.8456 # 0.0333
LaBraM- BBaee Oling et al,, 2024b)
5.8M
0.1736
1 0,0101
0,3716 0.0128
0,4765
L. 0.0097
0.5035 1 0.0078
0.0959 1. 0.0712
0.6621 1 0,1050
ToMenoken (Ours)
1.M
0.5775
0,0042
0,4985 0.009
0.5847 0.0050
0.6750 0,0392
0.3379
0.0515
0.8839 : 0.0173
With Muttiple Dataset Pretraining
BIOT (Yang al., 2024))
3.2M
0,0000
0.0033
0.7068
0.0457
0.3277 1 0.0160
0.5761 1 0.0284
EEGPT (Wang al,, 2024a)
4.7M
0.4545
0,0193
0,3502 0.0255
0.4559 = 0.0311
0.6644
+ 0.0227
0.3373
= 0.0064
0.8185 #: 0.0252
LaBoaM-Base' (Binng et ol., 2024))
5.8
0.4736
0.0037
0,3658
0.0033
0,4708
0.0015
0.5260
0.0309
0.2138
0.0223
0.7750 + 0.0540
CBraMod' (Wang et al., 20244)
4M
0.5566
0.0126
0,4292
0.0167
0.5743 = 0.0138
0,6646
0.0598
0.3469
= 0.0281
0.9071 4: 0.0199
ToM--okenizer (Ours)
1.9
0.5747
0.0022
0.4979 + 0.0038 0.5797 = 0.0017 0.6471 + 0.0145 0.3554 = 0.0264
0,8188 + 0.0117
1. The best and second-bess best results for each dataset setting are bolded and undertined, respectively. The number of parameters for LaBraM is only considering their classifier model. The size of their neural tokenizer was 8.6M. 3. indicates reproduced in a single datasel setting and i indicates pretraining EFG datasets.
LaBraM, we substituted its neural tokenizer with ours during masked EEG modeling. As shown in Figure 3, our method consistently improves performance on TUEV, IIC, and CHB-MIT, achieving gains of at least 3% in most cases. LaBraM notably underperforms dataset setting, yet integrating our tokenizer yields a 147% improvement in AUC-PR, demonstrating its effectiveness in capturing class-discrmminatie features in data-scarce scenarios. These results highlight the broad applicability of TFM-Tokenizer across architectures and its capacity to enhance diverse EEG foundation models.
CHB-MIT in the single-
4.4 DOES TFM- IFM-TOKENIZER SCALE TO OTHER BRAIN-SIGNAL TYPES / DEVICES?
In order to assess the scalability of TFM-Tokenizer beyond the modal- ities and tasks seen during pre- training, we evaluate its perfor- mance on the EESM23 cBr-EBG dataset (Bjarke Mikkelsen et al., 2025) for sleep staging, a task, brain signal modality, acquisition system, number of channels and channel con- figuration entirely distinct from those in the pretraining set. Specifically, we only finetune pretrained
Table 3: Scalability experiments results on EESM23.
Ear-EEG (Sleep Staging)
Models
Balanced Acc.
Cohen's Kappa
Weighted F1
BIOT
0.3858 1 0.0085
0.3406 0.0096
0.4888 1 0.0124
BIOT-TEM
0,3952 + 0.0170 1
0,3603 : 0.0252
0.5033 = 0.0155 1
LaBraM-Baa Base
0.3890 + 0.0182
0,3322 + 0.0232
0.4827 + 0.0157
L.BMrMMTTMM
0.4004
0.0086
0.3475 0.0128
0.4864 0.0118
WYYM-Tkeeiier
0.4148 + 0.0209
0,3883 = 0.0233
0,5174 + 0.0141
7

---

Published as a conference paper at ICLR 2026
Multiple Dataset Pretraining
BIOT
LaBraM
w/ TFM- IFMM-ooeeizerr
TFM-Token
0.54
0.41
Single Dutaset
6.84%
6.32%
Cohen's Kappa
0.6
Pretruining
Cohen's Kappa
6.99%
3626
21,89N
AUC-PR CRC
147,7654
36%
0.4
2.13%
0.2
0.5
0.1
0.4
0.3
0.0
TUEV
IIIC-Seizure
CHB-MIT
Figure 3: Performance comparison of existing foundation models with and without integration of TFM-Token zer the TUEV, IIIC, and CHB-MIT datasets. For each dataset, the first three bars show single-datasee pretraining and the latter three show multi-dataset pretraining. Percentage values above each bar indicate the relative performance gain achieved by incorporating TFM-Tokenizer.
models (our method, BIOT, and LaBraM) the EESM23 dataset using only ~K labeled training samples. EEGPT was not scalable in this setting due to its reliance on a fixed EEG channel layout for spatial embeddings (Wang et al., 2024a). As shown in Table TFM-Tokenizer demonstrates strong generalization, outperforming both baselines (p = 0.02) in this out-of-domain setting.
4.5 How IMPORTANT ARE FREQUENCY AND TEMPORAL MODELING FOR EEG TOKENIZATION?
To evaluate the importance of joint frequency-temporal modeling, we conducted an ablation study with three tokenization variants: (1) "FMM-Tokenizer- which uses only raw EEG patches predict the masked spectrogram; (2) S.M-Tokenizer-S, which uses only the spectrogram as input; and (3) TFM-Tokenizer, which jointly models both domains. Masked modeling was applied for token learn- ing in the latter two. On TUEV (Figure 4a), TFM-Tokenizer-S achieves higher Cohen's Kappa than TFM-Tokenizer-RR R, while TFM- Tokenizer-R yields better AUC-PR in abnormal detection (Appendix Figure 6). These results show that different EEG tasks rely different feature domains, underscor- ing the need for joint modeling, where TFM-Tokenizer consistently outperforms both variants.
4.6 HOW EFFECTIVE ARE TEM-TOKENIZER TOKENS?
We evaluate the quality of BEG tokens learned by our tokenizer across four aspects: (I) class-specific distinctiveness, (2) token consistency, (3) frequency learning capability, and (4) token utilization (results in Appendix C.I). For this analysis, we compare all three TFM-Tokenizer variants with the neural tokenizer from LaBraM, using the test splits of TUEV and III,, which both contain multiple classes. To ensure fairness, tokenizers employ a fixed vocabulary size 8192. Results on TUEV shown in Figure 4b-c, with additional results for other datasets provided in the Appendix.
Class- Token uniqueness. To assess whether tokenizers capture class-specific motifs, we define the # Unique Tokens Clas
Class-Tokee Token Uniqueness Score as # Tokens Utilized by Class x 100%. This metric quantifies how well a tokenizer assigns distinctive tokens to each class. Figure 4b shows the scores for TUEV, where a ro- bust tokenizer should yield high distinctiveness across all classes through unsupervised pretraining. TFM-Tokenizer consistently achieves higher scores than its variants and LaBraM's 's neural tokenizer, indicating that it produces more compact and informative token representations and validating the benefit of joint frequency-temporal modeling in EEG analysis.
Class-wise Token Consistency Analysis. We conduct a retrieval- based EEG signal mining exper- iment to evaluate token consistency within the same class, using similar-class sample retrieval (see Figure 4c). Given multi-channel EEG sample, we first obtain discrete token representation. Using the Jaccard similarity score, we then retrieve the top K most similar samples from the dataset and compute the precision score for correctly retrieving samples of the same class. For this study, we constructed balanced subset from the IIIC and TUEV datasets and tested all four tokenization methods. Results show that all TFM- Tokenizer variants significantly outperform the neural tok- enizer. Among all variants, method yields the best retrieval performance, reflecting better token consistency. Notably, TFM-Tokenizers and IFM-Tokenizer achieve nearly 60% precision on the TUEV for K 1. While the Jaccard similarity measure demonstrates initial feasibility, further
8

---

Published as a conference paper at ICLR 2026
TFM-Tokenizer
TFM-Tokenize--S
TFM-Tokenizer-R
LaBram-Baa Base
Cohen's Kappa
0.55
Unnniqu Tokens
0,7
Precision@eK
100
0.6
0:45
0.5-
0.40
K
SPSW GPED PLED EYEM ARTF BCKG
10
50
(a) Token Encoder Ablation
(b) Token Uniqueness
(c) Token Consistency
Figure 4: (a) Frequency and temporal token encoder ablation TUEV. (b) Comparison of class- token uniqueness scores across all classes and (c) Class-wise token consistency analysis.
work is needed identify optimal metrics. Nonetheless, the results suggest that EEG tokens can support the identification of similar pairs, with potential applications in contrastive learning.
4.7 Do THE LEARNED TOKENS CAPTURE MEANINGFUL EEG MOTIFS?
We perform small-scale qualitative analysis to examine whether TFM-Tokenizer captures meaningful time-frequency motifs in EEG sig- nals. Figure 5 shows some representative to- kens learned by our method on the TUEV dataset. Each token represents spectral win- dow and its corresponding raw EEG patch (1s window with 0.5s overlap). For clarity, highlight the most frequent tokens per class us- ing distinct colors. Periodic Lateralized Epilep- tiform Discharges (PLEDs) are periodic pat- terns consisting of sharp waves or spikes fol- lowed by slow wave, occurring every 1- 2s (Pohlmann-Eden consistently captures this characteristic wave- form across different samples in the PLED class, despite variations in noise, amplitude, and minor temporal shifts. This confirms that TFM-Tokenizer can capture class-specific physiologically meaningful EEG motif's into discrete tokens. Similarly, tokens such as 5096 and 3751 in the GPED class highlight the benefit of joint time-frequency modeling, as they remain bust to minor temporal shifts and warping within a window due to emphasizing spectral patterns. However, we found limitations associated with using fixed windowing for tokenization, as large patterns or shifts may cause splits across windows, leading to separate token assignments and mis- interpretation as distinct events.
4882
6634
3751
5096
1097
PLED
b) GPED
(1)
al., 1996). Token 4035
Figure 5: Overview of motifs captured by TFM- Tokenizer on TUEV: (a) three samples from the PLED class and (b) three samples from the GPED.
CONCLUSION
In this paper, presented TTW-Tokenizer. model-agnostic tokenization framework that en- codes single-channel EEG into discrete tokens by capturing time-frequency motifs. Our study demonstrated three key benefits: Accuracy: By accurately extracting single-channel features, our tokenizer enabled stronger representations and surpassed competitive baselines across four EEG benchmarks. (ii) Generalization: As plug-and-plas component, our method consistently boosted the performance of existing foundation models, showing its broad applicability. (iii) Scalability: Be- cause operates at the single-channel level rather than depending on the strict 10-20 EEG system, our method readily extended to ear-EEG sleep staging tasks, validating its cross-device scalability. Furthermore, analyses confirmed the class distinetiveness, consistency, and interpretability of the learned tokens, providing deeper insights into EEG tokenization. We hope this work will inspire the development of more robust tokenization frameworks and advance scalable, generalizable EEG foundation models across diverse modalities, devices, and tasks.

---

zer The repos- Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work was supported by NSF awards SCH-2205289, SCH-2014438, IIS-2034479, and JSPS KAKENHI Grant-in-Aid for Scientific Research Number JP24K20778. This project has been funded by the Jump ARCHES endowment through the Health Care Engineering Systems Center.
REPRODUCIBILITY STATEMENT
To support the reproducibility of our work, we provide our complete source code and pretrained model weights at https: //g thub. com/ Ja wttturrsaa0330///a0a
itory includes scripts for data preprocessing. loading, and model training to reproduce our results presented in this paper. In the main text, Section 4.1 outlines our experimental setup, including descriptions of the dataset and baselines, Additional implementation details, such as dataset statis- tics, preprocessing steps, ear- car-EfG-specific processing, evaluation metrics, and baseline configura- tions, are provided in Appendix B.1, B.2, B.3, B.4, and B.5. The Appendix also includes extended experiments across multiple datasets, including frequency learning analysis (Appendix C.I), cross- dataset generalization studies (Appendix C.3), additional results on improving foundation models (Appendix C.4), and further ablation studies. We have made every effort to ensure that our work can be easily reproduced by the community.
REFERENCES
Quentin Barthélemy, Cedric Gouy-Pailler, Yoann Isaac, Antoine Souloumiac, Anthony Larue, and Jérôme Mars. Multivariate temporal dictionary learning for eeg. Journal of neuroscience meth- ods, 215(():19-28, 2013.
Kaare Bjarke Mikkelsen, Yousef Rezai Tabar, Laura Revsbæk Birch, Simon Lind Kappel, Chris- tian Bech Christensen, Lars Dalskov Mosgaard. Marit Otto, Martin Christian Hemmsen, Mike Lind Rank, and Preben Kidmose. Ear-eeg sleep monitoring data sets. Scientific Data, 12(1):301, 2025.
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, et al. An introduction to vision-language modeling. arXiv preprint arXiv:2405.17244 2024.
Zheng Chen, Lingwei Zhu, Ziwei Yang, and Renyuan Zhang. Multi-tier platform for cognizing massive electroenceehallgram. In IJCAI-22, pp. 2464-2470, 2022.
Zheng Chen, Ziwei Yang, Lingwei Zhu, Wei Chen, Toshiyo Tamura, Naoaki Ono, Md Altaf-Ul- Amin, Shigehiko Kanaya, and Ming Huang. Automated sleep staging via parallel frequency-cut attention. IEEE Transactions on Neural Systems and Rehabilitation Engineering, pp. 1974-1985, 2023.
DeepSeck-AL, Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, al. Deepseck-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1xXiv:8810004805: 2018
Yiqun Duan, Charles Zhou, Zhen Wang, Yu-Kai Wang, and Chin-teng Lin. Dewave: Discrete encod- ing of eeg waves for eeg to text translation. In Thirty-seventt Conference on Neural Information Processing Systems, pp. 9907 9999,991... 8, 2023.
Filip Elvander and Andreas Jakobsson. Defining fundamental frequency for almost harmonic sig- nals. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2020.
Patrick Esser, Robin Rombach, and Bjöm Ommer. Taming transformers for high-resolution image synthesis, 2020.
10

---

Published as a conference paper at ICLR 2026
Juan Luis Gastaldi, John Terilla, Luca Malagutti, Brian DuSell, Tim Vieira, and Ryan Cotterell. The foundations of tokenization: Statistical and computational concerns. In The Thirteenth Interna- tional Conference on Learning Representations, 2025.
Wendong Ge, Jin Jing, Sungtae An, Aline Herlopian, Marcus Ng, Aaron Struck, Brian Appavu, Emily Johnson, Gamaleldin Osman, Hiba Haider, et al. Deep active learning for interictal ictal injury continuum eeg patterns. Journal of neuroscience methods, 351:108966, 2021.
Bart Gips, Ali Bahramisharif, Eric Lowet, Mark J Roberts, Peter de Weerd, Ole Jensen, and Jan van der Eerden. Discovering recurring patterns in clectrophysiological recordings. Journal of Neuroscience Methods, 275:66-79, 2017.
Amir Harati, Meysam Golmohammadi, Silvia Lopez, lyad Obeid, and Joseph Picone. Improved eeg event classification using differential energy. In 2015 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), pp. 1-4. IEEE, 2015.
Long Steven R Wu Manli C Shih Hsing H Zheng Quanan Yen Nai-Chyuan Tung Chi Chao Huang Norden Shen Zheng and Liu Henry H. The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London, Series mathematical, physicul, and engineering sciences, pp. 903-995, 1998.
Mainak Jas, Tom Dupré la Tour, Umut Simsekli, and Alexandre Gramfort. Learning the morphology of brain signals using alpha-stable convolutional sparse coding, Advances in Neural Information Processing Systems, 30, 2017.
Wei-Bang Jiang, Yansen Wang, Bao-Liang Lu, and Dongsheng Li. Neurolm: A universal multi- task foundation model for bridging the gap between language and eeg signals. arXiv preprint wwXii:2409.00/01, 2024a.
Weibang Jiang, Liming Zhao, and Bao liang Lu. Large brain model for learning generic represen- tations with tremendous EEG data in BCI. In The Twelfth International Conference Learning Representations, 2024b.
Jin Jing, Wendong Ge, Shenda Hong, Marta Bento Fernandes, Zhen Lin, Chaoqi Yang, Sungtae An, Aaron Struck, Aline Herlopian, Ioannis Karakis, et al. Development of expert-level classifica- tion of seizures and rhythmic and periodic patterns during eeg interpretation. Neurology, 100(17); el e1750-e1762, 2023.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are mns: Fast autoregressive transformers with linear attention. In International conference ma- chine learning, pp. 5156-5165. PMLR, 2020.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. pp. 95-104, 2018.
Hongli Li, Man Ding, Ronghua Zhang, and Chunbo Xiu. Motor imagery eeg classification algorithm based on enn-lstm feature fusion network. Biomedical signal processing and control, 72: 103342, 2022.
Hanwen Liu, Daniel Hajialigol, Benny Antony, Aiguo Han, and Xuan Wang. Eeg2text: Open vocabulary eeg-to-text decoding with eeg pre-training and multi-view transformer. arXiv preprint 0rXiv:2405.02165, 02650. 2024.
Sebas Lopez, G Suarez, D Jungreis, I Obeid, and Joseph Picone. Automated identification of abnor- mal adult eegs. In 2015 IEEE signal processing in medicine and biology symposium (SPMB), pp- 1-5. IEEE, 2015.
Navid Mohammadi Foumani, Geoffrey Mackellar, Soheila Ghane, Saad Irtza, Nam Nguyen, and Mahsa Salehi. Eeg2rep: enhancing sepe-supervised eeg representation through informative masked inputs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discov- ery and Data Mining, pp. $544-5555 2024.
11

---

Published as a conference paper at ICLR 2026
Abdullah Mueen, Eamonn Keogh, Qiang Zhu, Sydney Cash, and Brandon Westover. Exact dis- covery time series motifs. In Proceedings of the 2009 SIAM international conference on data mining, pp. 473 184.484.20 SIAM, 2009.
Iyad Obeid and Joseph Picone. The temple university hospital eeg data corpus. Frontiers in neuro- science, 10:196, 2016.
OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, and othres. Gpt-4o system card. arXiv preprint arXiv: 2410.21276, 2024.
Namuk Park and Songkuk Kim. How do vision transformers work? 2022.
Wei Yan Pch, Yuanyuan Yao, and Justin Dauwels. Transformer convolutional neural networks for automated artifact detection in scalp eeg. In 2022 44th Annual International Conference of the IEEE Engineering Medicine Biology Society (EMBC), pp. 3599-3602. IEEE, 2022.
Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer: Frequency debiased transformer for time series forecasting, In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, 2024.
Bernd Pohlmann-Eden, Daniel B Hoch, Jeffrey I Cochius, and Keith H Chiappa. Periodic lateralized epileptiform discharges-a critical review. Journal of clinical neurophysiology, 13(6):519-533) 1996.
Jathurshan Pradeepkumar, Mithunjha Anandakumar, Vinith Kugathasan, Dhinesh Suntharalingham, Simon L Kappel, Anjula C De Silva, and Chamira US Edussooriya. Towards interpretable sleep stage classification using cross-modal transformers. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024.
Patrick Schäfer and Ulf Leser. Motiflets-simpa and accurate detection of motifs in time series.
arXiv preprint wwXiv:226.003335 2022.
Craig W Schmidt, Varshini Reddy, Haoran Zhang. Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner. Tokenization more than compression. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 678-702, November 2024.
Ali Hossam Shoeb. Application of machine learning to epileptic seizure onset detection and treat- ment. PhD thesis, Massachusetts Institute Technology, 2009.
Yonghao Song, Xueyu Jia, Lie Yang, and Longhan Xie. Transformer-basee opatial-temporal feature learning for eeg decoding. arXiv preprint wrXiv:2106.111777 2021.
Yousef Rezaci Tabar, Kaare B Mikkelsen, Mike Lind Rank, Martin Christian Hemmsen, Marit Otto, and Preben Kidmose, Ear-eeg for sleep assessment: a comparison with actigraphy and psg. Sleep and Breathing, 1633)) 2021.
Yousef Rezaci Tabar, Kaare Mikkelsen, Laura Birch, Nelly Shenton, Simon L Kappel, Astrid R Bertelsen, Reza Nikbakht, Hans Toft, Chris Henriksen, Martin Hemmsen, Mike L Rank, Marit Otto, and Preben Kidmose. "ear-eeg sleep monitoring 2023 (eesm23)", 2024.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Guagnyu Wang, Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals. In Advances in Neural Infor- mation Processing Systems, pp. 39249-39280, 2024a.
Guangyu Wang. Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals. Advances in Neural Informa- fion Processing Systems, 37:39249-392888 2024b.
12

---

Published as a conference paper at ICLR 2026
Jiaqi Wang, Zhenxi Song, Zhengyu Ma. Xipeng Qiu, Min Zhang, and Zhiguo Zhang. Enhancing eeg-to-text decoding through transferable representations from pre-trained contrastive eeg-text masked autoencoder. arXiv preprint arX1v:2402.17433, 2024c.
Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. Cbramod: A criss-cross brain foundation model for eeg decoding. arXiv preprint
arXiv:2412.07236, 2024d.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven C. H. Hoi. Etsformer: Expo- nential smoothing transformers for time-series forecasting. 2022.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet:
Temporal 2d-variation modeling for general time series analysis. 2023.
Maxwell A Xu, Alexander Moreno, Hui Wei, Benjamin Marlin, and James Rehg. Rebar: Retrieval-based reconstruction for time-series contrastive learning. arXiv preprint arXiv:2311.000 1.311,005/9 2023.
Chaoqi Yang. Danica Xiao, M Brandon Westover, and Jimeng Sun. superviie eeg representa- tion learning for automatic sleep staging. JMIR Al, pp. e46769, 2023.
Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild. Advances in Neural Information Processing Systems, 36, 2024.
Ke Yi, Yansen Wang, Kan Ren, and Dongsheng Li. Learning aypology-annostie eeg representations with geometry-aware modeling. Advances in Neural Information Processing Systems, 36, 2024.
Daoze Zhang, Zhizhang Yuan, Yang Yang, Junru Chen, Jingjing Wang, and Yafeng Li. Brant: Foundation model for intracranial neural signal. Advances in Neural Information Processing Systems, 2024.
Zhi-Qin John Xu Zhi-Qin John Xu, Yaoyu Zhang Yaoyu Zhang, Tao Luo Tao Luo, Yanyang Xiao Yanyang Xiao, and Zheng Ma Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. Communications in Computational Physics, (3(5):774-1166) 2020.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. pp. 1-12, 2022.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
Contents
Problem Formulation
14
B Additional Experiment Details
15
B.1 Dataset Statistics and Splits
15
B.2 Preprocessing
16
B.3 Ear-EEG Preprocessing
16
B.4 Evaluation Metrics
16
B.5 Additional details on baselines
16
B.6 STFT parameters
16
Extended Experiment Results
17
C.I Additional Results Token Quality Analysis and Frequency Learning
17
C.2 Additional results on Frequency and Temporal Modeling for EEG Tokenization
18
C.3 Token Generalization Assessment through Cross-Dataset Experiments
18
C.4 Additional Results on FFM-Tokeni- zer Improving Existing Foundation Models
19
C.5 Effect of Masked Token Prediction in EEG Tokenization
19
C.6 Removing Position Embedding in TFM-Token zer Improves Token Learning
20
C.7 Downstream Model Ablation
20
C.8 Ablation on Token Vocabulary Size
C.9 Ablation Masking
21
C.10 Masking Ratio Ablation
22
C.11 Window Length (L) and Hop Size (H) Ablation
22
C.12 Token Embedding Size Ablation
23
TFM-Tokenizer Implementation and Hyperparameter Tuning
23
D.I Hyperparameter Tuning of TFM-Tokenizer and Downstream Transformer
23
D.2 TFM-Tokenizer Hyperparameters
24
D.3 Downstream Transformer Encoder Hyperparameters
25
BIOT-TFM and LaBraM-TFM Implementation Details
26
More Related Works
26
G LLM Usage Statement
27
A PROBLEM FORMULATION
EEG Data. Let X E RCXT denote a multi-channel EEG recording with C channels and time samples. Each channel of RT is decomposed into (1) raw patches {ar J and (2) corresponding
14

---

Published as a conference paper at ICLR 2026
time--frequee free-freyy representation windows {S.) where N is the number of time windows. For simplicity, we omit the channel index and refer to z as a single-channel EEG signal unless stated otherwise. To obtain the time-frequency representation. i..., spectrogram, S, we apply the short-time Fourier transform (STFT) to x using a windowing function we(-) of length I. and a hop size H.
Short-Time Fourier Transform (STFT). To obtain the time-frequeney representation, i.e.g, spec- trogram, S, we apply a STFT to x using a windowing function wi.) of length L and hop size H:
L-1
S(W, r) = Σ x(TH + l)w(()e
-22m!!
(1)
where w indexes the discrete frequencies and T indexes the time segments (i.e., time windows shifted by H). We retain only the magnitude |.| form S RRN,, where F is the number of frequency bins and N the number of time windows.
Problem Statement (EEG Tokenization); Given a single channel EEG x, we aim to learn a tokenization function fukkenizrr TR -> VNxD that maps x (or transformations) discrete tokens {ur) ji= where each token is from learnable EEG token vocabulary V of size k and embedding size of D. These tokens should represent various time-frequency "motifs" derived from both and S, Therefore, V is learnable from S and the temporal patches {z; H Remark. We here hold several expectations for the learned motif tokens. First, these tokens are expected to reduce redundancy, noise, and complexity, providing a compact, sparse, and informative representation of EEGs. Second, these motifs should capture key neurophysiological patterns from both temporal and frequency domains. Third, the tokens should generalize well across different EEG tasks.
sequence of
Problem Statement (Multi-Channel EEG Classification): Given EEGs X and a fixed, learned single-channel tokenizer frokenizers apply fiokenizer independently to each channel obtain
tokenization representation { [19)/11 e=i" These tokens are aggregated and mapped to output labels be:sssusiffir (DD) - Y where Y is the target labels (e.g.. EEG events, seizure types). Notably, felassitier can be any downstream model, and its training performed separately from the EEG tokenizer flokenizer-
B ADDITIONAL EXPERIMENT DETAILSS
B. DATASET STATISTICS AND SPLITS
Table 4: Evaluation Dataset Summary
Dataset
Recordings
# of Samples
Duration (s)
Task
TUEV
11,914 914
112, 491
EEG Event Classification
IIC Seizure
2, 689
135, 096
10
Seizure Type Classification
CHB-MIT
686
326, 993
10
Seizure Detection
TUAB
2, 339
409, 455
10
Abnormal EEG Detection Far-EEG based Sleep Staging
HESM23
120
14,509 509
30
This section provides detailed information on the datasets used in our experiments and their respec- tive splits. Table 4 summarizes key statistics, including the number of recordings, the total number of samples after preprocessing, their duration, and the corresponding downstream tasks. For TUEV and TUAB, we utilized the official training and test splits provided by the dataset and further di- vided the training splits into 80% training and 20% validation sets. We performed subject-wise split into 60% training. 20% validation, and 20% test the IIIC Seizure dataset. In the CHB-MIT dataset, we used 1-19 subjects for training, 20-21 for validation, and 22-23 for testing. For the distrii-istiinnoonn evaluation the Ear-EEG EESM23 (Bjarke Mikkelsen et al.. 2025) dataset, followed a subject-wiss split, where subjects 1-6 were used for fine-tuning, 7-8 for validation, and 9-10 for testing.
15

---

Published as a conference paper at ICLR 2026
B.2 PREPROCESSING
We follow the preprocessing setup of BIOT (Yang et al., 2024). We adhere to the 16-channel bipolar montage from the international 10-20 system, as used in (Yang et al., 2024). All EEG recordings are resampled to 200 Hz. For TUEV and TUAB, we apply a bandpass filter (0.1-75 Hz) and a notch filter (50 Hz), following the preprocessing pipeline of LaBraM (Jiang et al., 2024b). We then segment the recordings according to the provided annotations and preprocessing guidelines. STFT computation of the signals is performed using PyTorch, with detailed parameters provided in Appendix B.6. For training, validation, and test splits, we follow the recommendations from (Yang al., 2024). We adopt window length of 1s with 0.5s overlap segment BEG signals during training and inference, following prior work for consistency (Yang et al., 2024).
B.3 EAR-EEG PREPROCESSING
We follow the preprocessing guidelines of Tabar et al. (2021) for the EESM-23 ear-EEG dataset, which includes four channels (RB, RT, LB, LT). A bandpass filter (0.1-100 Hz) and a 50Hz notch filter applied. Each patients perform certain tasks before sleep. To isolate sleep segments, crop each session from the onset of annotated sleep scoring, segment the signal into 30-second epochs, and discard corrupted segments.
B.4 EVALUATION METRICS
For evaluation, we used balanced accuracy, Cohen's kappa coefficient, and weighted F1 for multi- class classification, and balanced accuracy, AUROC, and AUC-PR for binary classification. During finetuning, we employed binary cross-entropy loss for TUAB, cross-entropy loss for TUEV and IIIC, and focal loss for CHB-MIT due to class imbalance. All experiments were conducted using five different random seeds, and we report the mean and standard deviation.
B.5 ADDITIONAL DETAILS ON BASELINES
All baselines were reproduced using their official open-source repositories. LaBraM's primary con- tribution lies in large-scale EEG pretraining using over 2,500 hours of data (Jiang et al., 2024b), whereas our focus is on developing an effective EEG tokenizer. To ensure fair comparison, we reproduced LaBraM using its official repository under our dataset and experimental settings. For EEGPT, we report the published results for the 4.7M model on TUEV and TUAB (Wang et al., 2024a). Since results on CHB-MIT and IIIC-Seizure were not available, we used the official pre- trained weights and fine-tuned the model on these tasks.
B.6 STFT PARAMETERS
Table 5: STFT parameters
Parameter
Value
Description
FFT size (ni, L)
200
Number of frequency bins (equal resampling rate)
Hop length H
100
Step size for sliding window (50% overlap) smoothing window function to reduce spectral leakage
Window type
Hann
Output representation
Magnitude
Only the absolute values of the STFT are retained
Centering
False
The STFT is computed without implicit zero-padding
One-sided output
True
Only the positive frequency components are kept
To extract frequency-domalle representations of the EEG, we utilized the STFT function from Py- Torch. The recommendations of Yang et al. (2024) guided our parameter selection and empirical analysis of different configurations to optimize the trade-off between time-frequency resolution. The final parameters follows:
16

---

Published as a conference paper at ICLR 2026
TTF--Tkeenzerr
TFM-Tokenizer-S
TTM-Tokenizer-R
LaBram-Bal Base
Cohen's Kappa
0.85
%Unique Tokens
Precision
0.40-
0.80
10
0.35
0.30
0.75
022
0.25
10
0.70
OTH
ESZ
LPD
GPD
LRDA
GRDA
0.20
10
20
30
K
(a) Token Encoder Ablation (TUAB)
(b) Token Uniqueness
(c) Token Consistency (IIIC-eeizur)
(IIIC-Seizure)
Figure 6: (a) Frequency and temporal token encoder ablation on TUAB. (b) (c) presents Anal- ysis of token quality across three TFM-Token keni zer variants and the neural tokenizer on IIIC. (b) Comparison of class-token uniqueness scores across all classes and (c) Class- token consistency analysis
Table 6: Token Utilization and class-token uniqueness comparison
Tokenization Method
# Params
Utilization
Class-Token Uniqueness (GM) %
%
TUEV
IIC
TUEV
IIC
Neural Tokenizer (LaBraM)
8.6M
21.13
15.25
0.034
0.000
TFM-Tokenizer-R
1.1M
5.29
7.87
0.000
0.000
T
1.1M
13.93
11.04
0.004
0.619
TFM-Tokenizer
1.2M
9.78
8.26
2.14
1.429
C EXTENDED EXPERIMENT RESULTS
C.I
ADDITIONAL. RESULTS ON TOKEN QUALITY ANALYSIS AND FREQUENCY LEARNING
In this section, we present more results on token quality analysis, specifically focusing on token utilization and frequency learning capability of tokenizer. Additional token uniqueness and consistency experiments on IIIC dataset is presented in Figure 6b and
Token utilization (%) score calculated the percentage of unique tokens activated from the total available vocabulary size. Additionally, we computed the geometric mean (GM) of class-token uniqueness scores along with the utilization score, and the results presented
Token utilization:
Frequency Encoder
Temporal Encoder
TUEV
TUAB
CHBMIT
0.20
0.25
0.25
Spectral Entropy
0.20
0.15
0.20
0.35
0.10
0.05
0.05
0.05
0.00
0.00
0.00
#samples
Figure 7: An analysis of how the proposed frequency and temporal-domain encoders capture fre- quency features, by using the spectral entropy the learned token sequences from randomly se- lected samples. Higher values indicate that the tokens contain richer frequency information.
17

---

Published as a conference paper at ICLR 2026
in Table 6. Our TFM-Tokenizer reduces token utilization by more than two-fold compared to the neural tokenizer on TUEV (21.13% + 9.78%) and nearly two-fold on IIIC (15.25% + 8.26%) It also significantly improves learning of class-unique tokens compared to the neural tokenizer (0.034% - 2.14% on TUEV, 0.0% + 1.429% on IIIC).
Evaluating the Frequency Learning of TEM-Tokenize Tokens: In this experiment, we com- pare the frequency and temporal-domain encoders of the TFM-Tokenizer to evaluate their ability to capture diverse frequency features in EEG signals. Specifically, we arrange all tokens in temporal order and perform a discrete Fourier transform on the token sequence. This process decomposes the tokens into frequencies. where each frequency reflects the degree of change between tokens at various scales. Larger changes indicate more diverse token representations. Then, we compute spec- tral entropy, defined as the normalized Shannon entropy of the amplitude values, to quantify how energy is distributed across the spectrum. Higher spectral entropy means that the model has learned a broader range of frequency features, capturing differences from both large-scale trends and fine details. Figure 7 shows that on the TUEV, TUAB, and CHBMIT datasets, the frequency encoder produces tokens with significantly higher spectral entropy than the temporal encoder. For example, on the TUEV dataset, the frequency encoder achieved an average spectral entropy of 0.26, while the temporal encoder reached only 0.14. This multi-scale sensitivity benefits downstream tasks such as classification, where learning detailed differences in EEG tokens can improve performance.
C.2 ADDITIONAL RESULTS ON FREQUENCY AND TEMPORAL MODELING FOR EEG TOKENIZATION
Table 7: Ablation study on input representation to TTM-Tokenizer
Models
TUEV (event type classification
TUAB (abuormal delection)
Balanced
Cohen's Kappa
Weighted
Balanced
AUC-PR
AUROC
TFM- Tokenizer-R
D. 1898 + 0.0103
0.5194 = 0.0193
0.7518 = 0,0085
0.8033 0.0221
0.80U + 0.0127
0.8819 = 0,0124
TFM- Tokenizer-S
0.1708 0.0339
0.5275 0.0314
0,7538 0.0152
0.792T
0.004
0.811 0.0095
0,8836
0.0052
TFM-Tokenizer
0.4943 0.0516
0.5337 0.0306
0,7570 0.0163
0.8152 0.0014
0.8946 0.0008
0.8897
0.0008
1. The best results are bolded, while the second-best are underlined.
In Table 7 we provide detailed results of our ablation study discussed under Section 4.5.
C.3 TOKEN GENERALIZATION ASSESSMENT THROUGH CROSS- DASSSEDT EXPERIMENTS
Table 8: Cross dataset generalizability experiments under single dataset settings
Testing
Tokenizer
MTP
Performance Metries
Dataset
Dataset
Dataset
Balanced Acc.
Cohen's Kappa
Weighted F1
TUEV
TUEV
0.4943 + 0.0516
0.5337 + 0.0306
0.7570 0.0163
TUEV
0.4722 + 0.0578
0.4990 = 0.0237
0.7380 + 0.0137
IIC
IIC
0.4291 + 0.0235
0.5195 = 0.0200
0.7534 + 0.0100
TUEV
TUEV
0.4651 + 0.0449
0.5925 = 0.0249
0.7847 0.0136
TUAB
TUAB
0,5252 + 0.0431
0.6187 = 0.0285
0,8018 + 0.0138
TUEV
0.4979 + 0.0444
0.5995 + 0.0225
0.7885 + 0.0122 0.8196 + 0.0045
CHB-MIT
CHB-MIT
0.5898 + 0.0192
0.6591 + 0.0106
To evaluate the robustness of our tokenizer, we conducted cross-dataset experiments under two set- tings: (1) fixing the tokenizer and performing masked token prediction (MTP) & finetuning on a different target dataset and (2) fixing the tokenizer and MTP, followed by finetuning downstream transformer only on the target dataset. Results are presented in Table 8, which demonstrates strong generalizability, with our TFM-TToennize achieving the best performance TUEV when pretrained on CHB outperfrrnngg the best-reported result in four dataset settings. These findings high- light the potential of our tokenizer foundation for a scalable, universal EEG tokenizer.
18

---

Published as a conference paper at ICLR 2026
C.4 ADDITIONAL RESULTS ON 1FM-TOXEXIZER IMPROVING EXISTING FOUNDATION MODELS
Table Performance comparison of LaBraM and BIOT with and w/o our IFM-Tokenizer.
Dataset
Exp.
Method
Performance Metrics
Setting
Balanced Acc.
Cohen's Kappa
Weighted F1
BIOT
0.4679 0.0354
0.4890 + 0.0407
0.7352 0.0236
BIOT-TFM
0.4228 0.0162
0,5230 = 0.0226
0.7490 0.0114
Single
LaBraM
0.4682 0.0856
0.5067 + 0.0413
0.7466 + 0.0202
TUEV
LaBraM-TFM
0.5147 0.0174
0,5220 + 0.0153
0.7533 + 0.0094
BIOT
0.5281 0.0225
0.5273 1 0.0249
0.7492 0.0082
BIOT-TFM
0.0089 0.5530
0.5464 1: 0.0137
0.7625 1 0.0069
Multiple
LaBraM
0.5550 0.0403
0.5175 + 0.0339
0.7450 0.0194
LaBraM-TFM
0.5541 0.0316
0.5367 + 0.0281
0.7567 0.0165
BIOT
0.4458 0.0183
0.3418 + 0.0228
0,4511 + 0.0207
BIOT-TFM
0.4633 + 0.0078
0.3663 + 0.0103
0.4689 + 0.0090
Single
LaBraM
0.4736 1: 0.0101
0.3716 1 0.0128
0,4765 1 0.0097
InC
LaBraM-TFM
0.4814 1 0.0075
0.3795 + 0.0091
0.4841 1 0.0062
BIOT
0.4414 + 0.0035
0.3362 + 0.0040
0,4483 + 0.0033
BIOT-TFM
0.5050 + 0.0037
0.4098 = 0.0052
0.5139 = 0.0025
Multiple
LaBraM
0.4736 0.0037
0.3658 + 0.0033
0.4708 + 0.0015
LaBraM-TFM
0.4782 + 0.0065
0.3737 = 0.0076
0,4790 + 0.0082 1
Balanced Acc.
AUC-PR
AUROC
BIOT
0.6582 0.0896
0.3127 + 0.0890
0.8456 0.0333
BIOT-TFM
0.5893 0.0197
0.3341 = 0.0349
0.8752 + 0.0123
Single
LaBraM
0.5035 0.0078
0.0959 1. 0.0742
0.6624 11 0.1050
CHB-MIT
LaBraM-TFM
0.5473 1. 0.047 1
0.2376 1 0.0461
0.7863 0.0438
BIOT
0.7068 0.0457
0.3277 + 0.0460
0,8761 = 0.0284
BIOT-TFM
0.6197 0.0085
0.3484 = 0.0078
0.8726 0.0098
Multiple
LaBraM
0.5260 + 0.0369
0.2138 + 0.0523
0.7750 + 0.0540
LaBraM-TFM
0.5579 1: 0.0394
0.2445 + 0.0351
0.7887 1 0.0423
Table presents detailed results integrating TFM-Tokenizer with BIOT and LaBraM. Across all metrics and settings, TFM- Tokenizer improves performance in 93% of cases, demonstrating its effectiveness in enhancing existing EEG foundation models.
C.5 EFFECT OF MASKED TOKEN PREDICTION IN EEG TOKENIZATION
wo MTP
MTP w MTP
wo MTP
MTP
MTP
0.8
kappa
F1 weightted
0.7
0.4
Balanced
0.6
0.5
0.2
0.2
6.4
0.1
01
0.1
Raw Signal.Oll Only
STET
FaW Signal
STF" Only
Rew Signal Ony
STFT Only
Figure 8: Masked Token Prediction Ablation
19

---

Published as a conference paper at ICLR 2026
We conducted ablation study downstream transformer to assess the impact of masked token prediction pretraining in a fully discretized framework. Using a pretrained JFM-Tokenizer. we compared two approaches: (I) masked token prediction pretraining followed by fine-tuning and (2) direct fine-tuning without pretraining. This experiment was performed on the TUEV dataset across all three TFM-Tokenizer variants, with results summarized in Figure 8. While Cohen's Kappa and Weighted Fl showed no significant differences between the two approaches, masked token pre- diction pretraining significantly improved balanced accuracy across all TFM-Tokenizer variants. This suggests that pretraining enhances class- wise prediction consistency by capturing token depen- dencies and making downstream transformer more robust missing channels or time segments, common challenge in EEG analysis.
C.6 REMOVING POSITION EMBEDDING IN TFM- TOKENIZER IMPROVES TOKEN LEARNING
Table 10: TFM-Tokenizer Comparison with and w/o Position Embedding (PE) TUEV Dataset
Method
Utilization
Uniqueness (GM)
Balanced Acc.
Cohen's Kappa
Weighted F1
%
TFM-Token' zer + PE
12.87 9.78
1.94
0.4765 + 0.038
0.5119 + 0.022
0.7457 + 0.012 0.7570 10.016
TEW-Tokenizer w/o PE
2.14
0.4943 +0.052
0.5337 10.031
Through our empirical analysis, we found that the performance significantly improved when position embedding was applied to the TFM-Tokenizer.. EEG patterns are inherently chaotic and non-stationary, meaning similar motif's can occur at any position within the signal. An ideal tok- enizer should be capable of capturing and representing such EEG motifs as distinct tokens without relying on positional information.
We conducted an ablation study comparing the -okeiizzei'er's performance with and without position embeddings to critically analyze this phenomenon. The results of this analysis, presented in Table 10, clearly show that the TFMM-Tokenizer without position embedding achieves significantly better performance, with an increase of 4% in Cohen's 's Kappa (0.51 19 - 0.5337).
We further studied the quality of the learned tokens in terms of token utilization and class-uniqueness scores, Token utilization decreased (12.87% + 9.78%) when position embeddings were removed, while the class-token uniqueness score increased (1.94% + 2. 1.44.. This suggests that the TFM- Tokenizer, when using positional encoding, learns different tokens for the same motifs depending on their location in the signal, leading redundancy. Removing the position embedding allows the TFM--Tokenizer to learn more compact and meaningful tokens without introducing unnecessary data complexities. This improvement is further illustrated in the motifs captured by the TFMe-iikeiizer's tokens in Figure 5 in Section 4.7.
C.7 DOWNSTREAM MODEL ABLATION
We ablated the number of trans- former layers in the down- stream model on the TUEV dataset, with results presented in Table 11. Notably, even with significantly
Table 11: Ablation on number of transformer layers in the downstream model
Number of
Performance Metries
Layers
Params.
Balanced Acc.
Cohen's Kappa
Weighted F1
0.58M
0.4486 + 0.0297
0.5404 + 0.0168
0.7603 + 0.0096
0.63M
0.4920 + 0.0595
0.5758 + 0.0169
0.7780 + 0.0089
0.72M
0.4943 + 0.0516
0.5337 + 0.0306
0.7570 + 0.0163
6
0.82M
0.5025 + 0.0592
0.4996 = 0.0208
0.7410 = 0.0104
1.12M
0.5016 0.0730
0.5088 = 0.0272
0.7456 + 0.0139
fewer parameters (two layers), the model maintains competitive and, in some cases, better performance across key metrics. This highlights the potential for developing lightweight and efficient models for EEG analysis without substantial performance trade-offs.
20

---

Published as a conference paper at ICLR 2026
C.8 ABLATION ON TOKEN VOCABULARY SIZE
To evaluate the impact of token vocabulary size on performance and token learning, we conducted an ablation study by varying the vocabulary size from 256 8192 in powers of two. As shown in Figure 9, no monotonic trend was observed for Cohen's 's Kappa and Weighted FI scores. However, balanced accuracy increased with larger vocabulary sizes. Further analysis of token utilization and class-token uniqueness scores is presented in Figure 10. Notably, Figure 10b shows that class- token uniqueness scores increase with vocabulary size, contributing to the improvement in balanced accuracy by enabling learning more unique class-specific tokens.
0.000
0.50
0.575
0.70
Cohan' Kappe
Weighted
0.76
0.43
0350
0.775
0.35
0.72
0.30
532
204n
4246
m142
0.003
252
112
3524
2043
4008
1102
0.70
517
1004
2010
4946
mi4
Token Vocabulary Stze
Taken Vocabulary Size
Figure 9: Token vocabulary size ablation with performance metries
a.
b.
Token Utilization Score (%)
2.0
Mean]
Unique (%)
1.5
1.0
256
512
1024
2048
4096
8192
256
512
1024
2048
4096
8192
Token Vocabulary Size
Token Vocabulary Size
Figure 10: Token vocabulary size ablation with token utilization and uniqueness
C.9
ABLATION ON MASKING
Table 12: Ablation on masking used for the pretraining TFM-Tokenizer on TUEV Dataset
Masking Strategy
Balanced Ace.
Cohen's Kappa
Weighted F1
Random Masking
0.4351 + 0.0462
0.4772 + 0.0140
0.7296 + 0.0076
Frequency Band Masking
0.4673 + 0.0540
0.5193 + 0.0243
0.7536 = 0.0125
Frequency Band
0.4946 + 0.0392
0.5045 + 0.0221
0.7462 + 0.0116
Temporal Masking
Frequency Band
Temporal Masking
0.4943 = 0.0516 0.5337 0.0306 0.7570 0.0163
Symmetric Masking
We conducted an ablation study on masking strategies during Tokennier pretraining to as- sess their impact performance. Results shown in Table 12 indicate that random masking the spectrogram S performs poorly compared to other strategies, underscoring the need for ef- fective masking capture frequency and temporal features from EEG. Frequency band masking
21

---

Published as a conference paper at ICLR 2026
significantly improves performance random masking, with 8% increase in Cohen's Kappa (0.4772 - 0.5193) and a 7% increase in balanced accuracy (0.4351 - 0.4673), highlighting the importance of modeling frequency band dynamics. The addition of temporal masking fur- ther boosts balanced accuracy by 5% (0.4673 + 0.4946), underscoring the importance of joint semporal-frquuenc modeling. However, temporal masking results in decline in Cohen's Kappa and Weighted FI, which is then resolved by introducing symmetric masking, achieving the overall best performance.
C.10 MASKING RATIO ABLATION
Ablation on frequency band masking ratio used for the pretraining of TFM-Tokenizer on TUEV, IIC Seizure and CHB-MIT Datasets.
Table 13:
Dataset
Frequeney Mask Ratio
Balanced Ace.
Cohen's Kappa
Weighted F1
0.5
0.4946 0.0392
0.5045 + 0.0221
0.7462 0.0116
TUEV
0.3
0,4306 + 0.0187
0,5025 = 0.0193
0.7432 + 0.0090
0.1
0.3859 = 0.0580
0.4308 0.0755
0.7057 = 0.0376
0.5
0.5315 0.0102
0.4427 0.0143
0.5369 + 0.0114
IIC
0.3
0.5148 0.0158
0.4250 0.0193
0.5222 + 0.0167
0.1
0.4381 + 0.0032
0.3286 + 0.0046
0.4420 + 0.0047
Balanced Acc.
AUC-PR
AUROC
0.5
0.6809 + 0.0380
0.3335 + 0.0182
0.8859 + 0.0137
CHB-MIT
0.3
0.6313 + 0.0599
0.3233 + 0.0337
0,8708 0.0187
0.1
0.6530 + 0.0486
0.3502 + 0.0441
0.8742 + 0.0116
We conducted ablation study examine how varying the frequency band masking ratio affects model performance and generalization across datasets. All experiments were performed under the single-channel setting. with the temporal masking ratio fixed 0.5 without symmetric masking, and the results are summarized in Table 13. For the TUEV and III Seizure datasets, a frequency mask ratio of 0.5 yielded the best overall performance. MIT dataset, except for Cohen's 's Kappa, which showed a slightly higher score at masking ratio of 0.1. Considering these results along with the added benefit that 0.5 masking ratio enables more effective use of symmetric masking data-augmentation strategy, we selected frequency mask ratio 0.5 for our final approach.
similar trend was observed in the CHB-
C.11 WINDOW LENGTH (L) AND HOP SIZE (H) ABLATION
Table 14: Ablation on window length (I.) and stride or hop size (H) used to segment raw signals and compute STFT for the pretraining of TFM-Token zer TUEV Dataset.
Window
Hop Size (s)
Balanced Acc.
Cohen's Kappa
Weighted F1
Length (s)
0.5
0.25
0.5038 + 0.0561
0.6059 = 0.0170
0.7935 + 0.01
1.0
0.25
0.4796 + 0.0598
0.5761 0.0171
0.7780 + 0.0098
1.0
0.5
0.4943 0.0516
0.5337 = 0.0306
0.7570+ 0.0163
1.0
0.75
0.4068 + 0.0182
0.4868
0.0210
0.7327 + 0.0085
2.0
0.5
0.1726
0.0093
0.0168 0.0137
0.5202 0.0074
2.0
1.0
0.2123 + 0.0143
0.1504 = 0.0146
0.5748 + 0.0087
2.0
1.5
0.3948 0.0287
0.4042 = 0.0282
0.6878 0.0167
22

---

Published as a conference paper at ICLR 2026
To investigate how window length and stride affect the tokenizer's ability to capture time- frequency motifs and performance, we conducted an ablation varying both parameters and adjusted the STFT configuration to preserve one-to-one alignment between time and frequency windows. The results, summarized in Table 14, indicate that smaller windows with greater overlap yield the strongest per- formance. This suggests that shorter segments allow the tokenizer capture finer-grained motifs that may be lost when using larger windows. For consistency with baselines and prior work, how- ever, we adopt I-second window length with 0.5-second hop size in all reported experiments.
C.12
TOKEN EMBEDDING SIZE ABLATION
Table 15: Token embedding size ablation TUEV Dataset.
Embedding Dimension
Balanced Acc.
Cohen's Kappa
Weighted F1
32
0.4213 = 0.0529
0.4974 + 0.0165
0.7417 + 0.0081
64
0.4943 0.0516
0.5337 + 0.0306
0.7570 0.0163
128
0.3199 = 0.0193
0.1909 + 0.0245
0,5700 + 0.0276
256
0.3864 0.0082
0.3575 + 0.0157
0.6682 0.0091
Table 15 summarizes the results of the token embedding size ablation. Performance improves up to embedding dimension of 64, after which it begins to decline. We do not observe a consistent trend as the embedding size increases, which may be attributed to training instability when using larger embedding dimensions.
D TFM-TOKENIZER IMPLEMENTATION AND HYPERPARAMETER TUNING
Mult-Channel Sample
Channel-xfae EEG
Tollnn Emacdding
Charnal Festtoo
Add (CLS)) Token
Dlarsiiizztion
Embocting
[CLS]
Flatten
Blocks
Transo mar
X
Single Channel EEG Tokerizer Module
TFo-nikeeeeer
Downstream Transformer Model
Figure 11: TFM FFM-Tokenizer framework Overview
Figure 11 presents an overview of the framework during inference. This section provides additional details on the implementation and training of the framework.
D.1 HYPERPARAMETER TUNING OF FFM-TOKENIZER AND DOWNSTREAM TRANSFORMER
We employed a systematic approach to optimize the hyperparameters of both the Tokenizee and downstream transformer models using Ray Tune with the Optuna² search algorithm. Our optimization process followed a three-phase strategy.
In the first phase, optimized the TFM-Tokenizer architecture by tuning the depth and number of attention heads in the frequency transformer, temporal transformer, and transformer decoder mod- ules to minimize the masked reconstruction loss Lrecon- This followed by tuning the training optimizer's 's parameters, including learning rate and weight decay. The second phase focused on
https://bee.raaylll/m//ttttssste hhtps://optuma.org/
23

---

Published as a conference paper at ICLR 2026
the downstream transformer optimization for the classification task, where we first tuned its archi- tectural parameters (depth and number of heads), followed by training the optimizer's 's parameters while keeping the tokenizer frozen. The third phase focused on tuning optimizer parameters for the masked token prediction pretraining of the downstream transformer.
To ensure fair comparison with LaBraM's neural tokenizer, we main- tained vocabulary size of 8, 192 and an embedding dimension of 64. For our ablation studies involving raw signal-only and STFT-only variants, we doubled the embedding dimen- sions of the temporal encoder and fre- quency patch encoder match the codebook dimension while maintain- ing all other parameters same. De- tailed hyperparameter configurations for both TFM-Tokenize and down- stream transformer provided in Appendices D.2 and D.3, respectively.
Training
Transformer Training
Figure 12: Training loss curves for (a) the TFM-Tokenizer learning and (b) the masked--okenn-rrcdiiioon pretraining of the downstream transformer
In Figure 12a and present the training loss curves for both the TFM-Tokenizer training stage and the masked-token-rrediete curves demonstrate stable training behavior, even with large codebook and a relatively small dataset. We kept the codebook size at 8192 to ensure a fair comparison with LaBraM's neural tokenizer.
pretraining of the downstream transformer, respectively. The
D.2 TOM-TIKENIZER HYPERPARAMETERS
Table 16: Hyperparameters for ToM-Tikenizer unsupervised pretraining on single-channel setting
Hyperparameter
Values
Batch size
256
Optimizer
Adam W
Weight decay
0.00001
31
0.9
82
0.99
Learning rate scheduler
Cosine
Minimal Learning rate
0.001
Peak Learning rate
0.005
# of Warmup Epochs
10
# of Pretraining Epochs
100
24

---

Published as a conference paper at ICLR 2026
Table 17: Hyperparameters for TFM-Tokenizer
Hyperparameter
Values
Input Channels
1
Output Dimension
64
Convolution layer
Kernel Size
200
Stride
100
Output Dimension
64
Temporal Encoder
Convolution layer 2
Kernel Size
Stride
1
Output Dimension
32
Convolution layer 3
Kernel Size
Stride
Input Channels
Output Dimension
64
Convolution layer
Kernel Size
5
Stride
5
Output Dimension
64
Frequency Patch Encoder
Convolution layer
Kernel Size
Stride
I
Output Dimension
64
Convolution layer 3
Kernel Size
Stride
Transformer Encoder Layers
2
Embedding Dimension
64
Prequency Transtormer
Number of Heads
8
Output Dimension
52
Gated Patchwise Aggregation
Kernel Size
Stride
Transformer Encoder Layers
Embedding Dimension
64
Temporal Transformer
Number of Heads
8
Token vocabulary (Codebook size)
8192
Transformer Facoder Layers
8
Embedding Dimension
64
Transformer Decoder
Number of Heads
8
Linear Decoder
100
D.3 DOWNSTREAM TRANSFORMER ENCODER HYPERPARAMETERS
Table 18: Hyperparameters for downstream transformer, its masked token prediction pretraining and downstream finetuning
Hyperparameter
Values
Transformer Encoder Layers
4
Embedding Dimension
64
Number of Heads
8
Masked Token Prediction Pretraining
Batch size
512
Optimizer
Adam W
Weight decay
0,00001
31
0.9
32
0.99
Learning rate scheduler
Cosine
Minimal Learning rate
0.001
Peak Learning rate
0.005
# of Warmup Epochs
5
of training Epochs
50
Finetuning
Other parameters are the same as above except:
32
0.999
label smoothing (multi-class)
0.1
25

---

Published as a conference paper at ICLR 2026
E BIOT-TFM AND LABRAM-TFN IMPLEMENTATION DETAILS
LaBraM
BIOT
Paaching
Tolias
Time
Noad Foot
Paching with Overlag
Fregens Lave Leser Proudlar
Cherne BBG
Packins
Predicee Musded
Nidiin
Tnkeens
Enozdre
Masked EDG Madding
12011b
BEG Tokeninaize
Chassel
LaBraM-TFM
Tokenizatin
Data
BIOT-TEM
Suppinn 2007
Inkcetter
Packing
Poodiced Misketi
Maddee
eppraaly
Sugnakk
Marked EEG Madding
Figure 13: Schematics integrating the proposed TFM-Tokenizer with BIOT and LaBraM founda- tion models.
Figure 13 illustrates how the proposed ToM-Tikenizer is seamlessly integrated into existing EEG foundation models such BIOT and LaBraM.
BIOT-TFM. For BIOT, replace the initial patch-based linear projection layers with our tok- enizer, which converts multi-channel EEG signals into discrete token sequences. Similar to BIOT, add channel embeddings and positional embeddings before feeding the tokens into the linear transformer layers. We follow BIOT's original training pipeline, which includes unsupervised pre- training, supervised pretraining, and downstream finetuning, allowing direct comparison while isolating the effect of our tokenizer.
LaBraM-TFM. LaBraM employs a neural tokenizer only during its masked EEG modeling (MEM) pretraining stage, where multi-channel EEG signals are patched, subset of patches is masked, and the model is trained to predict the tokenizer-generennn patches. In LaBMaM-TFM, and conduct MEM pretraining as in the original workflow. After pretraining, the tokenizer dis- carded and only the LaBraM transformer is finetuned for downstream tasks.
codebook indices for masked simply replace LaBraM's 's neural tokenizer with our Tokenizee
F MORE RELATED WORKS
Frequency Representation Collapse. Frequency domain analysis is crucial in EEG and general time series analysis (Elvander & Jakobsson, 2020; Wu et al., 2021; 2023; Woo et al., 2022). In real-world signals, time-domain observations inherently mix multiple frequency components, and high-energy, cow-irequency signals often dominate the spectrum (Huang Norden E Shen Zheng & H. 1998; Lai et al., 2018). As a result, these entangled frequency features makes difficult for models to distinguish between them (Zhou al., 2022; Piao et al., 2024). Recent studies have shown that these entangled signals can lead to collapse the learned frequency representations (Zhi-Qin John Xu al., 2020; Piao et al., 2024). Models tend to overemphasize the dominant low-frequency
26

---

Published as a conference paper at ICLR 2026
features while neglecting the high-frequency details. This issue lead lack of capturing various EEG waveforms and degenerating data representation (Park & Kim, 2022). Motivated by these works, paper focuses developing methods to learn diverse, informative frequency features. In Section C.I. we provide an analysis of our proposed frequency-domain tokenizer and its impact model performance.
G LLM USAGE STATEMENT
We used large language models (LLMs) solely for writing support, including grammar correction, sentence sentencerefinemeee refinement, and clarity improvements. All conceptual contributions, algorithm design, code development, experiments, and analyses were conducted entirely by the authors.
27


Reviews:
[{"review_id": "SlRDyW3IoH", "reviewer": "Reviewer_7X8j", "summary": "This paper introduces TFM-Tokenizer, a novel framework that learns discrete tokens from single-channel EEG signals by capturing time-frequency motifs. Unlike existing methods that use raw or continuous embeddings, TFM-Tokenizer builds a learnable vocabulary of meaningful EEG patterns, enabling plug-and-play integration with any foundation model. It uses a dual-path encoder to jointly model time and frequency domains and is trained with a mask-and-reconstruct strategy. Evaluated on four EEG datasets and an ear-EEG sleep staging task, TFM-Tokenizer consistently outperforms strong baselines, improves existing models like BIOT and LaBraM, and offers cross-device generalization with fewer parameters and better interpretability.", "questions": "How long does it take to train the tokenizer?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. First to introduce single-channel EEG tokenization using time-frequency motifs, filling a critical gap.\n\n2. Outperforms SOTA by up to 17% with fewer parameters.\n\n3.  Plug-and-play enhancement for existing foundation models like BIOT and LaBraM.\n\n4.  Cross-device generalization (e.g., ear-EEG) demonstrates robust transferability.\n\n5. Learned tokens show clear class discriminability and frequency awareness, aiding clinical understanding.", "weaknesses": "Downstream tasks focus on seizure and hospital datasets. The diversity of task types is relatively weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mpruw2aRnU", "reviewer": "Reviewer_n2sv", "summary": "This paper proposes TFM-Tokenizer, a framework for tokenizing single-channel EEG signals by learning a vocabulary of time-frequency motifs using a dual-path architecture with frequency and temporal masking. The tokenizer is designed to be model-agnostic, integrating with existing EEG foundation models like BIOT and LaBraM, and claims improvements in accuracy (up to 17% in Cohen’s Kappa), generalization, and scalability (e.g., to ear-EEG). Experiments are conducted on four EEG datasets under single- and multi-dataset pretraining settings, with additional analysis on token quality.", "questions": "1. Could the authors provide more ablation studies on the vocabulary size K and masking strategies (e.g., frequency band size δf)? How sensitive is performance to these, and do they generalize across datasets?\n\n2. The paper claims the tokenizer is \"model-agnostic,\" but integration details with BIOT/LaBraM are brief—e.g., how exactly are token embeddings fused, and what modifications were needed? A response with pseudocode or specifics could clarify.\n\n3. For the ear-EEG scalability experiment, what adaptations (if any) were made for differences in sampling rate or noise profiles? Baseline comparisons here seem weak; adding results from non-tokenized transfers could strengthen the claim.\n\nI will consider raising my score if all my concerns are solved or clarified.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper addresses an interesting problem in EEG foundation models: effective tokenization of signals to improve representation learning, which is underexplored compared to NLP or vision domains. The single-channel approach is a reasonable design choice for device-agnostic scalability, and the integration with existing models (e.g., BIOT and LaBraM) as a plug-and-play component shows practical potential. The experiments include a range of datasets (TUEV, TUAB, CHB-MIT, IIIC-Seizure) and settings (single- vs. multi-dataset pretraining), with some ablation studies on token quality (e.g., class-discriminative analysis). The inclusion of a scalability test on ear-EEG is a nice touch for real-world applicability. Overall, the work is clearly motivated by challenges like motif capturing and frequency entanglement in EEG signals.", "weaknesses": "The core contribution lacks sufficient novelty: the proposed TFM-Tokenizer heavily builds on existing VQ-based tokenization (e.g., from LaBraM) and time-frequency representations common in EEG analysis (e.g., spectrograms with masking, as in BIOT or related works like Yang et al., 2024). The \"time-frequency motif learning\" is essentially a combination of spectral patching, transformers, and VQ quantization, but it doesn't introduce fundamentally new mechanisms—e.g., the localized spectral window encoder is similar to patch-based processing in vision transformers, and the masking strategy mirrors BERT-like objectives without EEG-specific innovations. Claims of up to 17% improvement in Cohen’s Kappa are overstated, as they are relative to baselines on specific datasets (e.g., TUEV), but absolute gains are modest (e.g., 0.5273 to 0.6189), and statistical significance is only reported sporadically (e.g., p=1.5e-4 on IIIC-Seizure). Experiments are limited: no comparisons to more recent EEG models (e.g., BRANT or MMM beyond superficial mentions), insufficient ablation on key hyperparameters (e.g., codebook size K, masking ratios), and the multi-dataset setting uses only four datasets, which may not capture broader diversity in EEG corpora. Scalability to ear-EEG is promising but under-evaluated—only a 14% gain is claimed without details on transfer learning adaptations or failure cases. Interpretability analysis (e.g., token consistency) is superficial and lacks quantitative metrics like mutual information or visualization of failure modes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4y4vWJvZbK", "reviewer": "Reviewer_EGJo", "summary": "This submission proposes TFM-Tokenizer, a single-channel EEG tokenization framework that learns a discrete vocabulary of time-frequency motifs via a dual-path encoder. The tokenizer is used to produce per-channel token sequences which are then fed to a lightweight transformer (or plugged into existing foundation models) for  downstream tasks. Experiments over four dataset report performance gains; an ear-EEG sleep-staging dataset is used to argue scalability.", "questions": "1/ Please include more recent baselines such as CBraMod (ICLR’25) and evaluate the proposed single-channel design on unseen EEG datasets or channel configurations that were not used during pretraining. This would help verify the claimed channel-invariant generalization and rule out potential data leakage.\n\n2/ Could you clarify your fine-tuning strategy?\n\n3/ This submission fixes the patch size at 1 s with 0.5 s overlap, but the reason for this choice is unclear. How does this fixed window align with the paper’s motivation of token resolution?\n\n4/ The paper claims interpretability and perform motif case studies. Could you offer a quantitative measure of token-motif correspondence, for instance, the proportion of tokens aligning with known EEG events?\n\n5/ As the model is based on VQ-VAE, which can suffer from instability or codebook collapse, please provide evidence that training remains stable, e.g., by reporting gradient norms, especially given the small datasets and large codebook size (8192).", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1/ This paper is related to a very timely topic. EEG tokenization for heterogeneous devices and non-stationary signals is an important and current problem.\n\n2/ This paper provides rich token analyses with motif visualizations. It contains multiple analyses (such as classtoken uniqueness and class-wise token consistency) and visual examples of learned motifs in section 4.7, which enhance the insight.", "weaknesses": "1/ The novelty may be overclaimed, especially the third, tokenization learning objective, arguing that “Relying solely on capturing time-based motifs into discrete tokens risks losing important spectral structure” in L81-82. However, most existing EEG tokenization methods are already frequency or time-frequency-oriented: e.g., LaBraM reconstructs the frequency domain (with a TimeConv module on raw signals), and NeuroLM includes both time and frequency domain reconstruction. The proposed change mainly shifts from FFT to STFT, which feels incremental and weakens the claim of being “the first to encode diverse time-frequency motifs” in L142-143.\n\n2/ The single-channel design fully discarding inter-channel topology, which seems questionable. Many EEG tasks, especially localization or differential montages in epilepsy or sleep staging, depend critically on spatial relationships and cross-channel synchrony. The relatively lower performance on CHB-MIT, compared to BIOT, may partly reflect this limitation. Moreover, such a setup implicitly assumes that the downstream backbone needs reintroduce spatial structure (as EEGPT or LaBraM do with hard-coded topographic embeddings), so the claim of being model-agnostic is also overstated.\n\n3/ The baselines are outdated and inconsistent. Recent models such as CBraMod (ICLR’25)[1], EEG2Rep(KDD’24)[2] are not compared, and NeuroLM is included for only two of four datasets. This selective evaluation raises fairness concerns and weakens empirical credibility.\n\n4/ This paper shows partial data leakage in its experimental setup, which weakens the claim of dataset generalization. Both the single- and multi-dataset experiments pretrain and evaluate on the same set of datasets. This design still leads to partial data leakage, as the tokenizer indirectly sees the target data distribution.\n\n5/The writing could be more structured. The term motifs, central to the paper and appearing in the title, is not defined until the Related Work section. As this term is uncommon in EEG representation learning, it should be briefly introduced earlier to avoid confusion.\n\n[1] Wang J, Zhao S, Luo Z, et al. CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding[C]//The Thirteenth International Conference on Learning Representations.\n\n[2] Mohammadi Foumani N, Mackellar G, Ghane S, et al. Eeg2rep: enhancing self-supervised eeg representation through informative masked inputs[C]//Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024: 5544-5555.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yMElF0N1ev", "reviewer": "Reviewer_RxB5", "summary": "This paper proposes TFM-Tokenizer, a model-agnostic tokenization framework that learns discrete time–frequency motifs from single-channel EEG signals. The tokenizer produces interpretable and compact tokens via dual-path time–frequency masking, enabling integration with various EEG foundation models such as BIOT and LaBraM. Extensive experiments on four EEG datasets show consistent improvements in both single- and multi-dataset pretraining settings, as well as strong scalability to ear-EEG data. The authors also provide comprehensive analyses of token quality, distinctiveness, and interpretability.", "questions": "How sensitive is the model to the choice of STFT parameters (e.g., window length, hop size)? Is the tokenizer robust to different preprocessing pipelines?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed TFM-Tokenizer is a model-agnostic and reusable component that can enhance a wide range of EEG foundation models.\n2. The authors conduct detailed analyses of token quality (e.g., class-specificity, frequency awareness, consistency, and utilization), lending strong support to the claim that the tokens are both informative and interpretable.\n3. The paper is technically sound and presents a well-motivated formulation of single-channel EEG tokenization, which addresses an underexplored yet important problem in EEG representation learning.", "weaknesses": "1. In Section 4.3, the authors only test replacing the neural tokenizer in LaBraM with TFM-Tokenizer. It would strengthen the claim of generalizability if the authors also tested using TFM-Tokenizer's token embeddings as direct inputs for masked EEG modeling.\n2. The token utilization score decreases with larger vocabulary size (Appendix C.8). Could the authors explore some ways to improve utilization?\n3. The embedding dimension of tokens is fixed in experiments. A discussion or ablation on how this dimension affects performance would improve clarity.\n4. The paper could better articulate computational costs — for example, how much training overhead or inference latency is introduced by TFM-Tokenizer compared to standard segment-based tokenization.\n5. There is a typo \"Abnoral\" in Table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)