LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis
Authors: Berkay Döner
Thorir Mar Ingolfsson
Luca Benini
Yawei Li
Venue: NeurIPS 2025 poster ()
Keywords: Electroencephalography (EEG)
Biosignal Processing
Transformers
Self-Supervised Learning
Foundation Models
Representation Learning
Primary area: neuroscience_and_cognitive_science
TL;DR: This paper introduces a computationally efficient foundation model that achieves topology-invariant EEG analysis by projecting signals from diverse electrode configurations onto a unified latent space.

Abstract: Electroencephalography (EEG) offers a non-invasive lens into human brain activity, but building large‐scale models is hampered by $\textit{topological heterogeneity}$: each public corpus defines its own electrode layout, limiting generalization. We introduce $\textbf{LUNA}$ ($\textbf{L}$atent $\textbf{U}$nified $\textbf{N}$etwork $\textbf{A}$rchitecture), a self-supervised foundation model that reconciles disparate electrode geometries while scaling linearly---not quadratically---with channel count. LUNA compresses multi-channel EEG into a fixed-size, topology-agnostic latent space via learned queries and cross-attention. Downstream transformer blocks then operate exclusively on this latent representation using patch-wise temporal self-attention, decoupling computation from electrode count. Pre-trained on TUEG and Siena ($\>$21,000 h raw EEG across diverse montages) using a masked-patch reconstruction objective, LUNA transfers effectively to four downstream tasks: abnormality detection, artifact rejection, slowing classification, and emotion recognition. It demonstrates highly competitive performance across several benchmarks, achieving state-of-the-art results on TUAR and TUSL, e.g., $\textbf{0.921 AUROC}$ on TUAR, while reducing FLOPs by $\textbf{300}$$\times$ and trimming GPU memory use by up to $\textbf{10}$$\times$. Critically,  these gains are consistent across all evaluated electrode configurations. Code is available at https://github.com/pulp-bio/biofoundation

LUNA:
Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis
Berkay Döner! Thorir Mar Ingolfsson"' Luca Benini'
Yawei Li!
'Integrated Systems Laboratory, ETH Zürich, Switzerland
Abstract
Fleetroencephalograppyy (EEG) offers a non-invasive lens into human brain activity, but building large-scale models is hampered by topological heterogeneity: each public EEG data defines its own electrode layout, limiting generalization. intro- duce LUNA (Latent Unified Network Architecture), a self-supervised foundation model that reconciles disparate electrode geometries while scaling linearly-not quadratically-wiit channel count. LUNA compresses multi-channel EEG into fixed-size, topology topology-agnosti agmostic latent space via learned queries and cross-attention. Downstream transformer blocks then operate exclusively on this latent represen- tation using patch-wise temporal self-attention, decoupling computation from electrode count. Pre-trained TUEG and Siena (> 21,000 hours of raw EEG across diverse montages) using a masked-patee reconstruction objective, LUNA transfers effectively to four downstream tasks: abnormality detection, artifact jection, slowing classification, and emotion recognition. It demonstrates highly competitive performance across several benchmarks, achieving state-of-the-art results on TUAR and TUSL, e.g., 0.921 AUROC on TUAR, while reducing FLOPs by 300x and trimming GPU memory use by up to 10x. Critically, these gains are consistent across all evaluated electrode configurations. Code is available at https://gitthu. com/pulp bio/ offfouniiiin
Introduction
Electroencephalogrrahyy (EEG) provides deep insight into brain activity without requiring invasive procedures, and plays crucial role in clinical diagnostics, cognitive neuroscience, and human- computer interaction. In recent years, deep neural networks have significantly advanced EEG analysis, shifting from handerafted pipelines to end-to-end learning systems [I]. Transformer- models rival traditional signal processing techniques by jointly modelling long-range temporal dynamics and cross-channel correlations 223
Despite this progress, fundamental bottleneck remains: EEG corpora exhibit significant topological heterogeneity. Electrode count and placement vary widely across public and private datasets, making it difficult to transfer models across montages. This limitation manifests in pronounced performance degradation during cross-dataset evaluation. For example, motor- inttery decoders lose up to 14 percentage points (pp) in accuracy when transferring from PhysioNet KU datasets [4) while state- of-the-art emotion reatiionrrcognitioon models such as BIOT and MMM exhibit 13-15 pp drops between SEED and DEAP montages [5] 6). Similarly, patient-to teseentttt-pace to-patient transfer in stereotactic EEG (sEEG) remains an unsolved challenge, with naive models performing near chance without explicit spatial encoding [
Existing approaches offer limited solutions to this problem. Some train bespoke models for each montage, while others retain only shared electrodes-discrddiee discarding up to 80% of available data [8].
"Correspondence to (thoriri, yawoiirr,,ywli))iis ee. etw.eth..... ch.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

More general approaches that flatten channels and time into long sequences incur quadratic self- attention complexity,
where S is the number of time segments and C' is the number of electrodes (channels), rapidly exhausting memory on dense caps [5]. These challenges underscore the need for single, montage-agnostic architecture that scales efficiently with electrode count.
LUNA (Latent Unified Network Architecture) directly addresses this gap. Our key innovation is topology-invarian encoder that maps arbitrary electrode layouts into fixed latent space via learned queries and cross-attention. Temporal self-attention layers then operate exclusively on this latent space, decoupling computational cost from the number of electrodes. We pre-train LUNA using masked- -atechhpphhhhhhh reconstruction objective on TUEG [9] and SIENA [10] (over 21,000 hours of raw EEG data), and fine-tune on four downstream benchmarks spanning abnormality and artifact detection, slowing classification, and emotion recognition.
The key contributions of this work the following:
-nvaaig-nn encoder. A learnt query / cross-attention module that projects arbitrary-sizee channel sets into fixed latent space.
-ieaar-n-chhanelss complexity. Patch-wise temporal attention that decouples FLOPs and memory from electrode count.
State-of-the-ard
accuracy-efficieng trade-off. LUNA achieves strong results across a range of EEG benchmarks, demonstrating significant capabilities with balanced accuracies of 81.57% % on
TUAB and 39.18% on SEED-V [II] and AUROC scores of 0.921 on TUAR and 0.802 on TUSL, while reducing FLOPs by x and GPU memory footprint by up to 10x high-density EEG recordings. Crucially, these gains hold across diverse electrode configurations, confirming LUNA's generalization capability.
2
Related Work
To contextualize our contributions, this section discusses relevant state-of-the-arr methodologies that will compare against. We focus on advancements self-supervised learning for time series, the emergence of foundation models for physiological signals, and existing approaches to managing variable input structures, especially concerning topological heterogeneity in the EEG domain and computational efficiency.
2.1 Self-Supervised Learning Strategies in EEG
Foundation models for EEG primarily rely on self-supervised learning (SSL) leverage large unlabeled datasets. Masked signal modeling is a dominant paradigm. BENDR 112 pioneered this for EEG by adapting masked prediction concepts from speech, applying contrastive objective predict masked convolutional features. Subsequent models refined this: BrainBERT 13) performs masked prediction on Shannel-independen spectrograms for intracranial eloctroencephallgraphh (iEEG); EEGFormer [14) and LaBraM (15) predict rector-quantized (VQ) representations of masked patches, learning discrete codebooks: CBraMod [16] directly reconstructs masked raw signal patches. LUNA employs similar masked reconstruction objective but applies it after projecting channel information into unified latent space, requiring the decoder to reconstruct channel channel-specific details from this compressed representation.
2.2 Modeling Spatial Structure and Topology Variation in EEG
Capturing the spatial relationships between EEG channels is vital but complicated by varying elec- trode counts and layouts across datasets. Several strategies have been explored in the literature: Channel Independence: Early approaches and models like BrainBERT [33 and EEGFormer [14 process each channel's data independently before potentially combining them later. While inherently handling varying channel numbers, this neglects early modeling of cross-channel interactions.
Tixe-TTopology Spatial Modeling: Models like Brant [17] use dedicated spatial encoders alongside temporal ones but assume a consistent channel configuration, limiting cross-dataset generalization. Graph Neural Networks (GNNs) [18] explicitly model spatial relationships using predefined ad- jacency graph, but require mechanisms to handle dynamically changing graph structures when topologies vary. LUNA avoids pre-defined graphs or fixed structures.

---

and compute- efficient transfer Joint Spatio-Temporas Attention: LaBraM [5] flattens channel and patch dimensions into one long sequence, allowing a standard Transformer to learn
dependencies simultancously. However, this incurs ((SS . 2) complexity, scaling quadratically with both sequence length/patches (S) and channels (C). CBraMod [66 and CHReBrO [19] use alternating or parallel spatial and tempo- ral attention mechanisms, reducing complexity to (maxx(S² C2)) but still scaling quadratically with the dominant dimension. BIOT [5] uses linear attention after flattening, improving efficiency but potentially limiting modeling capacity. LUNA differs significantly by performing channel unification first before applying temporal attention with quadratic complexity only on the patch dimension and the much smaller latent dimension Q.
Explicit Topology Mapping: Some methods explicitly map varying topologies to a canonical repre- sentation. MMM [6] maps channels to predefined anatomical regions but relies hand- -nginneedde features ( (Differential Entropy) rather than raw signals. Pop'T' [20] aggregates pre-computed channel- independent temporal features using 3ID electrode coordinates. While achieving topology invariance, these methods are not fully end-to-end or rely on external information (regions). LUNA learns an end-to-end mapping from raw signals using learned queries without requiring pre-defined structures. Differentiable Channel Reordering, Saeed et al. [21] also use learned attention to reorder/project heterogeneous channels into fixed space, but under supervised training and without explicit 3D channel encodings, In contrast, LUNA is self-supervised foundation model trained at scale, inte- grates explicit spatial (3D) information, and targets across datasets and tasks.
2.3 Learned Queries and Efficient Attention for Set Abstraction
LUNA's core mechanism for topology unification draws inspiration from architectures designed for permutation -ivvaaiinnnn processing of set-structured data. Set Transformer [22] introduced the concept of using a small set of learnable inducing points (queries) and an Induced Set Attention Block to summarize information from a larger input set via cross-attention, reducing the complexity from O(N2) to O(M .). PerceiverIO (23] further developed this mechanism, demonstrating its power in creating a fixed-size latent bottleneck capable of handling diverse, variable-sized inputs across different modalities (images, text) and enabling flexible decoding via task-specific output queries.
LUNA adapts this principle specifically for EEG topology invariance. We treat the set of EEG channel features at a given time interval (patch) as the input set. By applying cross-attention between the channel features (as keys/values) and a small number (Q) of learned queries, LUNA projects the variable-channel input onto a fixed-size latent space Qx) This projection is permutation-invarfe with respect to the input channels, thus achieving topology agnosticism. Furthermore, it improves computational efficiency, as the complexity of this step scales linearly with the number of channels. Where MMM relies on predefined regions (and hand-crafted features) and LaBraM/CBraMon rely on quadratic spatial attention after flattening space-time, LUNA first unifies variable channel sets with learned queries (with explicit 3D encodings) and then applies temporal attention fixed-size latent, yielding linear-in-C unification and reduced temporal sequence length. This design specifically targets applloyyagnostee scaling and inference efficiency within a self-supervised foundation -odell framework, rather than combining prior components unchanged.
3 Methodology
Developing generalizable foundation models for EEG is hindered by two primary obstacles: the topo- logical heterogeneity of EEG montages (varying channel counts and layouts) and the computational complexity of attention mechanisms. Standard models struggle with diverse input channel configu- rations, limiting data aggregation and generalizability. Furthermore, transformer-bast -based approaches often face prohibitive O((C- 2)²) or Ommax(C. 5²)). as discussed in the section 2.2 complexity when processing C' channels and S temporal patches. This limits their applicability to high-density EEG or long recordings.
LUNA addresses these challenges using smaller latent space. Firstly, Channel-Unification Module (Sec 3.1 employs learned queries and cross-attention to project variable-channel features into a fixed- dimension latent space, achieving topology invariance. Secondly, by unifying channel information into a compact set of Q queries (Q < C) before temporal processing. LUNA significantly reduces computational demands. This design enables efficient and scalable processing of heterogencous EEG
3

---

data, paving the way for more robust foundation models. LUNA adopts an mncoder-decodee architec- ture that transforms EEG signals from heterogeneous montages into unified latent representation. enabling togology-agnostic modeling and efficient downstream decoding (Figure
Time Features
Network
1-D Corrs.
Input Data
Patching Masking
Raw Signnl
Prequency Features
Fourler
Penceitoo
Milteyyerr
Embeddings
K x
Tokens
Queries
Q.
Reconstruction
Decoder
Temporal Attention Layer
Attention
Cross Attention
XM
Channel Unification Module
Figure 1: Overview of LUNA. EEG signals (B T) are segmented into patches and embedded. Channel-Unificatie Module maps channel-wise features into fixed-size latent space using learned queries (Q). Patch-wise Temporal Attention processes this latent sequence. The decoder generates task-specific outputs.
3.1 Encoder
The encoder comprises three key modules that transform the input EEG into a topology-agnostic latent representation: patch feature extraction, channel unification, and patch- patch-wise temporal modeling.
Patch Feature Extraction Given raw EEG x E BxCxT (Batch B, Channels C, Time T), we segment each channel into S = T/P non-overlapping temporal patches of size P. These patches are embedded two parallel pathways:
Temporal Embedding: A ID convolutional network (with GroupNorm 24 GELU 25 encodes
local temporal features similar to state-of-the-att methods such as LaBraM and CBraMod [16].
Frequency Embedding: The magnitude and phase from each patch's 's Fourier transform are projected through an MLP. These representations are summed obtain patch features Features.
Channel Positional Encoding To encode electrode locations, we apply NeRF-inspired sinusoidal encoding (26) normalized 3D electrode coordinates, followed by an MLP projection. This yields Epos E R BxCxE which is added to Tfeatures
During pre-training, random subset of tokens masked using learnable embedding.
Module To handle varying channel counts (C) across recordings, we intro- duce cross-attention module that maps patch-wise features into fixed latent space. Specifically, Q learned queries Qteam E RQXE, (lcarnable parameters without batch dimension, initialized orthogonally) cross-aues aots-- patch features. For attention, these queries are repeated across the B patch instances as Q E IR (B-3)xxxx
Let the input to this module be the tensor Xioken E Bx(-S)) representing the spatially-aware features for B samples, S patches per channel, and feature dimension We first reshape this tensor treat each patch instance across the batch independently while isolating the
X' (B-S))xCXE channel dimension for attention.
The cross attentioon mechanism then computes the output representation Aou (BBS))xQXE,
Aqut MultiHeadAttentionn Q Q,X'x')
(I)

---

A feed-forward network (FFN) with residual connection refines the outputs, followed by L Trans- former encoder layers operating on the query dimension Q.
X unified =
TransfomerFncoder(AA.. FFN (FN(Aou:)
(2)
The result uniiteed R. xxx decouples further processing from the original electrode layout. For clarity: learnable tensors (e.g.. Qleara) omit the batch dimension; repetition occurs only at attention time (e.g., Q).
Patch-wise Temporal Encoder The unified representations are reshaped into temporal sequences X Xunified RBxxx(Q-E), B) These are processed by a stack of Transformer encoder blocks with Rotary Positional Embeddings (RoPE) [27] to capture temporal dependencies efficiently. A key advantage of this encoding approach is that each of the S temporal tokens in X now encapsulates richer, aggregated information from multiple input channels, rather than representing single channel's segment. Furthermore, by not tokenizing each channel independently for temporal processing, the effective sequence length for the temporal Transformers is reduced from S C to just S, leading to significant reductions in computational complexity and memory requirements.
Euu CamporalEncoder(XMaaann
3.2 Decoder
LUNA supports two decoding strategies depending the task: reconstruction (pre-training) and classification (fine-tuning).
Reconstruction Head (Pre-training) For masked patch reconstruction, C' learned decoder queries Elear E C (learnable parameters without batch dimension) repeated across the batch and patches as Edec R. (B-)))) and attend reshaped Faut via cross cross-attentions producing channel-specific representations Zex (BB-S)xCXE Bout Bx is reshaped be used as keys/values K, V (R-SS))xxE for attention. A linear projection R - R applied Zee recovers the patch values. Decoder queries are indexed by channel labels and can be reused across datasets when electrodes overlap; this reconstruction head is used only during pre-training.
Classification Head (Fine-tuning) For downstream tasks, single aggregation query Eleam [lx(Q-E) (no batch) is repeated across the batch as Eage E [xxIx(Q-E)) and attends to Eour to produce pooled representation, which is passed an MLP for classification.
3.3 Training Objectives
LUNA is pre-trained with masked reconstruction loss and an auxiliary query specialization loss.
Reconstruction Loss
A Smooth L1 loss is applied both masked and visible patches:
Lieec
SmoothL. (Forig,) Frecuns,) ) + a Nvisible
SmoothL (Farig, Frecuns,)
Nmwsked
iEM
ig
and ()) 8)2 if 2) clse B|x - - with 1.
Affinity matrix. Let A E BBxHHxXXC denote the cross-attention weights (queries + channels) from channel-unification module, for instances and heads. We define
> Aumll l,, i. E xQxx
Aaffinity
H
1=1
i.e., attention weights averaged over heads, so that (A g.c. measures the affinity between query g and channel for instance 6'.
5

---

To promote diversity among queries, we penalize similarity in query- channel affinity matrices by minimizing the mean value of off-diagonal elements:
Query Specialization Loss
B²
Q
Upec
Aspoc
Σ
((Aaffinity A A
altiniy ) i..))
B¹ QQ
W=1 i=1 j=l,
4 Results
4.1
Experimental Setup
Datasets We pre-train LUNA on a combined corpus of Temple University Hospital EEG Corpus
(TUEG) [9] and the Siena Scalp EEG Database [10], spanning recordings with 20, 22, and 29 channels amounting to over 21, 1,900 hours of EEG data (see Table [T].. Downstream evaluations cover four diverse benchmarks: TUAB (2: Abnormal EEG detection (binary classification), TUAR
[9] We follow prior work [28] and treat artifact detection as a multiclass (single-label) problem with five classes (one label per segment). Artifact detection (multi-class classification) TUSL 9 Slowing event classification (4-class classification). SEED-V [I]]: Emotion recognition (5-class classification), with unseen 62-channel topology. All subjects and recordings from the downstream evaluation datasets (TUAB, TUAR, TUSL, SEED-V) were strictly excluded from this pre-training set to ensure fair evaluation of generalization. For LUNA, the input EEG is segmented into patches, consisting of 40 timestamps. For most datasets, EEG recordings are sliced into overrlaapgpnn 5-second segments to form individual training/evaluation samples. SEED-V -- dataset uses its default 1-second sample duration.
Fine-tuning and Data Splits For the TUAB dataset, we use the official train-test split. As TUSL and TUAR lack official subject-wise test splits, we follow recent leading work (e.g.. BEGFormer [14] and adopt an 80%/10% 1096/10%/109 110% randomized sample-level split for train/val/test to allow direct, like-for- like comparison. We acknowledge that clinical generalization and recommend them for future benchmark comparisons. For SEED-V V, fifteen trials are divided equally into train, validation, and test sets for each session. For the TUAR dataset, we adopt a multiclass classification approach, restricting to 5 distinct artifact types in a single-label setting, similar EEGFormer [14. We optimize binary cross-entropy loss for TUAB and cross- entropy loss for other datasets. We report the mean and standard deviation of results obtained across three different random seeds.
splits the gold standard for assessing
Preprocessing We apply minimal, standardized preprocessing pipeline to all EEG data. Signals are first bandpass filtered between 0.1 Hz and 75 Hz. A notch filter (50Hz or 60Hz) applied to remove power-line interference. All signals are then resampled to 256 Hz. For TUEG, TUAB, TUAR, and TUSL datasets we construct bipolar longitudinal electrode pairs provided in the dataset documentation; the full list of channel pairs used given in Appendix A.7 Siena and SEED-V are processed in unipolar format. Finally, each channel within each sample is normalized using z-score normalization.
)oouublebbanaaa) montage by differencing predefined
Computational Environment All experiments were conducted on a cluster of eight NVIDIA A GPUs, using Python 3.11.6 and PyTorch 2.4.1 with CUDA 12.1. Training utilizes "bf16' mixed- precision. Detailed hyperparameters for pre-training and fine-tuning are provided in Appendix A.3
Baselines and Variants We compare against state-of-the-art art supervised and superviee meth- ods, including transformer-beg based architectures such as LaBraM [15], CBraMod [16]. EEGFormer [14], and BIOT [5). LUNA evaluated three configurations: Base (7M), Large (43M), and Huge (31 (311M 1M parameters). Model size is increased by expanding the depth of the Patch-wise Temporal Encoder, the hidden embedding dimension E, and the number/size of queries Q in the Channel-Unification Module. Key architectural settings are detailed in Appendix A.1
4.2 Downstream Task Performance
Abnormal EEG Detection (TUAB) LUNA delivers competitive performance on TUAB (Table D. LUNA-Huge achieves AUROC of 0.8957 and AUPR of 0.9029, surpassing most self-supervised

---

baselines and approaching large-scale models like LaBraM and CBraMod. Notably, LUNA maintains strong performance while offering substantial efficiency and opplogy-agnostio benefits relative to strong self-superviser and large-scale baselines.
Table 1: Performance comparison on TUAB abnormal EEG detection.
Model
Size
Bal. Acc. (%) 1
AUC-PR 1
AUROC 1
Supervised Models
SPaRCNet 29
0.8M
78.96 + 0.18
0.8414 + 0.0018
0.8676 + 0.0012
ContraWR 30
1.6M
77.46
0.41
0.8421 + 0.0140
0.8456 0.0074
CNN-Transformer 31
3.2M
77.77
0.22
0.8433 + 0.0039
0.8461 + 0.0013
FFCL 32
2.4M
78.48
0.38
0.8448 = 0.0065
0.8569 + 0.0051
ST-Transformer
33
3.2M
79.66 + 0.23
0.8521 + 0.0026
0.8707 + 0.0019
Self-supervised Models
BENDR 12
0.39M
76.96 + 3.98
0.8397 + 0.0344
BrainBERT [B]
43.2M
0.8460 1 0.0030
0.8530 0.0020
EEGFormer-
2.3M
0.8670 1 0.0020
0.8670 0.0030
BIOT S
3.2M
79.59 0.57
0.8692 1 0.0023
0.8815 1 0.0043
EEG2Rep
80,52 + 2.22
0.8843 + 0.0309
FEMBA-HUge Huge
35
386M
81.82 = 0.16
0.9005 = 0.0017
0.8921 + 0.0042
CEReBrO
19
85.15M
81.67
0,9049 = 0.0026
0.8916 0.0038
LaBraM-Ban -Bse
5.9M
81.40 = 0.19
0.8965 = 0.0016
0.9022 + 0.0009
LaBraM-Huge
15
369.8M
82.58 0.11
0.9204 + 0.0011
0.9162 0.0016
CBraMod
16
69.3M
82.49 0.25
0.9221 + 0.0015
0.9156 + 0.0017
LUNA-Base
7M
80.63 + 0.08
0.8953 + 0.0016
0.8868 + 0.0015
LUNA-Large
43M
80.96 0.10
0.8986 1 0.0005
0.8924 0.0010
LUNA-Huge
311.4M
81.57 0.11
0.9029 + 0.0014
0.8957 0.0011
Artifact and Slowing Detection (TUAR and TUSL) LUNA delivers state-of-the-ard results TUAR and TUSL (Table(2). LUNA-Huge achieves AUROC 0.921 on TUAR, outperforming FEMBA- Large and other methods. On TUSL, LUNA-Huge reaches AUROC 0.802, the highest among all compared models.
Table 2: Performance comparison on TUAR (artifact detection) and TUSL (slowing event classifica- tion).
TUAR
TUSL
Model
Size
AUROC 1
AUC-PR 1
AUROC
AUC-PR 1
Supervised Models
ERGNett 36
0.752 + 0.006
0.433 + 0.025
0.635 + 0.015
0.351 + 0.006
EEG-GNN
0.837 1 0.022
0.488 1 0.015
0.721 1 0.009
0.381 1 0.004
GraphS4mer 37
0,833 1 0.006
0.461
1 0.024
0.632 1 0.017
0.359 1 0.001
Self-supervised Models
BrainBERT 13
43.2M
0.753 + 0.012
0.350 = 0.014
0.588 = 0.013
0.352 + 0.003
EBGForme Basc
14
2.3M
0.847 = 0.014
0.483 = 0.026
0.713 0.010
0.393 = 0.003
EEGForer--Lagee
14
3.2M
0.852 1 0.004
0.014 0.483 1 0.014
0.679 1 0.013
0.389 1 0.003
FEMBA-Base
B5
47.7M
0.900 + 0.010
0.559 + 0.002
0.731 + 0.012
0.289 1 0.009
FEMBA-Large
35
77.8M
0.915 : 0.003
0.521 + 0.001
0.714 + 0.007
0.282 + 0.010
LUNA-Base
7M
0.902 + 0.011
0.495 = 0.010
0.767 = 0.023
0.301 + 0.003
LUNA-TLarge
43M
0.918 + 0.003
0.505
+ 0.010
0.771 + 0.006
0.293 +00021
LUNA-Huge
311.4M
0.921 + 0.011
0.528 + 0.012
0.802 + 0.005
0.289 +0.008
Emotion Recognition on Unseen Montage (SEED-V) The SEED-V benchmark tests generaliza- tion to a novel 62-channel montage, distinct from pre-training data. Results Table show that while LUNA effectively operates this unseen topology, its performance (e.g.. Bal. Acc.) lags behind leading methods like CBraMod by 2-3 pp. This suggests trade-off inherent in LUNA's design: while its query-basse based unification enables efficient, appology-annoste processing across common montage
7

---

variations (as demonstrated on TUAB/TUAR/USLL) generalizing zero-shot to vastly different, high-density layouts remains challenging, possibly due to positional encoding constraints. Despite this gap, LUNA shows positive scaling from Base to Large models, underscoring its potential.
Table 3: Performance comparison on SEED-V emotion recognition (5-classes).
Model
Size
Bal. Acc. (%)
Cohen's Kappa
Weighted F1
Supervised Models
SPaRCNet 22
0.79M
0.2949 0.0078
0.1121 + 0.0139
0.2979 + 0.0083
ContraWR 330
1.6M
0.3546 + 0.0105
0.1905 + 0.0188
0.3544 = 0.0121
ENN-Transforee
31
3.2M
0.3678 1 0.0078
0.2072 + 0.0183
0.3642 + 0.0088
FFCL 32
2.4M
0.3641 0.0092
0.2078 + 0.0201
0.3645 1 0.0132
ST-Transformer
3.5M
0.3052 0.0072
0.1083 0.0121
0,2833 + 0.0105
Self-supervised Models
BIOT [5]
3,2M
0.3837 + 0.0187
0.2261 + 0.0262
0,3856 + 0.0203
LaBraM-Base 15
5.8M
0.3976 + 0.0138
0.2386 + 0.0209
0.3974 + 0.0111
CBraMod [6
14M
0.4091 + 0.0097
0.2569 + 0.0151
0.4101 + 0.0108
LUNA-Base
7M
0.3730 + O. 0.0098
0.1831 + 0.0103
0.3389 = 0.0091
LUNA-Large
43M
0.3918 0.0066
0.2073 + 0.0045
0.3586 + 0.0013
LUNA--uge
311.4M
0.3900 0.0096
0.2037 + 0.0103
0.3506 + 0.0047
Unless noted, we report mean + s.d. over matched seeds and focus effect sizes and confidence intervals. Formal significance tests are summarized Appendix A.9
4.3 Computational Efficiency
LUNA achieves substantially better calling efficiency compared to full and alternating attention models. As shown in Figure 2a LUNA's patch-wise attention enables thousands of temporal patches without the quadratic cost faced by LaBraM. Likewise, Figure 2b shows that LUNA maintains near- constant compute cost when channel count increases, outperforming CBraModd's (C?²) scaling for dense EEG recordings. These results confirm that LUNA decouples inference cost from input mon- tage, making well-suited for long recordings or high-density EEG scenarios. We also consider BIOT (linear attention) as an efficiency-oriented baseline; detailed FLOPs/memory scaling versus LUNA is provided in Appendix A.. Although many public datasets use 20-30 channels, research/clinical systems often employ 64-256 channels and longer windows; LUNA's OCC) unification and reduced temporal sequence length enable such regimes where quadratic spatial attention becomes impractical.
4.4 Ablation Studies
Choice of Q. We explore the trade-off between the number of queries Q and embedding size E under fixed Q- E budget; see Appendix A.6 We validate the impact of LUNA's key design choices on TUAB and TUAR 4
Learned Queries vs. Fixed Regions Replacing learned queries with predefined spatial regions
(similar MMM (6) yields small AUROC changes (-0.004 to -0.006,, within seed variation. We therefore emphasize the practical advantages of learned sieries-data-drrvvee flexibility without anatomy-specific priors-rather than statistically significant metric gain (Appendix A.9).
Query Specialization Loss Removing the specialization loss results in modest AUROC changes (-0.003 to -0.006), again on the order of the reported variation, with small mixed effects AUC-PR. We retain this loss for its regularizing role: it encourages diverse, non-redundant set of spatial filters (see Fig. 4.. which is desirable for robustness in challenging artifact conditions.
Frequency Features Ablating frequency embeddings leads to the largest drop (up to -0.012 AUROC), indicating more consistent contribution complementary to temporal features.
4.5 Latent Space Analysis
Pre-trained Representations t-SNE visualizations (Figure 3 reveal that even before fine-tuning, LUNA'S encoder captures task-relevant structure. Normal and abnormal EEGs form separate clusters in TUAB, while artifact classes are partially separated in TUAR, demonstrating effective pre-training.
8

---

Ont Of Memory
Nambbe of FLOPs (CFLOPs)
104
101
107
Menocy (M.8)
10a
Oul OF Mienory
10
107
10²
10P
102
10P
10
30F
102
10*
Number patches
Number of patches
(a) Scaling with number patches (Channels fixed at 22).
Out Of Menury
Stunber of FLOPs (GFLOP))
107
Monony (MiB)
10%
10'
109
10a
Mennary
10
10
10
Number chomals
Number of chomnel
CBraMod
LaBra
LaBesM(Lanp)
LaBraM(Hupp)
LUNA(Bare)
LUXA(Large)
LUNA(Huge)
(b) Scaling with number of channels (Patches fixed 20).
Figure 2: Computational cost scaling of LUNA and baseline models. (a) FLOPs and Memory usage vs. number of patches. (b) FLOPs and Memory usage vs. number of channels. LUNA demonstrates significantly better efficiency and scalability, especially compared to full attention (LaBraM), and favorable scaling compared alternating attention (CBraMod) due the fixed latent query space. CBraMod has a variable sized decoder based on the number of patches and channels; therefore, its model size as well as its resource usage grows rapidly. FLOPs are measured with fucore's CopCountaan tortttaalysis over 50 random inputs, including encoder+decoder, with window length T, patch size P, and reported as GFLOPs per forward pass.
Ncana Sgnal Alesmral Strvel
No Actflctt
Chesing
30-
Sheentag
Electedd
21-
L-NN Componet
10
1-NNE
-314
20-
-30
-30
-10
-30
-20
-10
10
+SN Compowert
S5E Component
(a) TUAB dataset (Normal vs. Abnormal Signal).
(b) TUAR dataset (Artifact Types).
Figure 3: t-SNE of LUNA-Bace -aae embeddings downstream datasets before fine-tuning.

---

4.6 Learned Query Specialization Visualization
Query Specialization Visual analysis of the learned queries (Figure ! highlights their role in aoollgy-agnossie representation. Queries exhibit distinct spatial profiles: some are localized (e.g.. frontal regions), while others aggregate broader signals. This emergent specialization confirms that cross-attention learns flexible, data-driven basis functions for spatial unification.
Queeyy
Qooryy
Queer
Query
Figure 4: Visualization of the attention patterns of queries in LUNA-Base on Siena 10 topology.
Table 4: Ablation study results (LUNA-Base) on TUAB and TUAR datasets.
Model Configuration
TUAB AUROC
TUAB AUC-PR
TUAR AUROC
TUAR AUC-PR
Model)
0.887 + 0.002
0.902 0.011
0.010
Unification Module: Region based Attention
0.883 0.001 0.000 0,892 + 0.002 0.003) 0.896 + 0.001 (4 0.006) 0.509 + 0.006 0.014)
Other Components:
w/o Query Specialization Loss
0.884 0.003 (L 0.003)
0.892 + 0.002 (L 0.003) 0.895 + 0.005 (1 0.007) 0.498 + 0.010 0.003)
Frequency Features
0.876 0.012 (1 0.011) 0.883 + 0.005 ( 0.012) 0.011 (4 0.009) 0,490 0.011 (4 0.005)
5
Conclusion
We introduced LUNA, a sulf--sup foundation model designed to address the challenge of topological heterogeneity in EEG analysis. By leveraging learned queries and cross-attention, LUNA unifies recordings with diverse electrode layouts into a fixed latent space, enabling modeling. Through extensive experiments across abnormality detection, artifact recognition, slowing classification, and emotion recognition, we demonstrate that LUNA matches or surpasses state-of-the- performance while offering substantial efficiency gains in FLOPs and memory usage. Critically, these benefits hold across all evaluated electrode configurations.
While LUNA achieves strong results, especially on heterogeneous montages, our analysis also reveals limitations. Performance on SEED-V suggests sensitivity to unseen channel topologies, likely stemming from reliance on positional encodings learned during pre-training. Addressing this limitation, through enhanced spatial generalization strategics or hybrid learned/geometric embeddings, is an important direction for future work.
More broadly, this work highlights the promise of topology-agnostic latent representations for scalable EEG modeling. Future extensions include exploring unified models across EEG and invasive modalities (e.g., sEEG, ECoG), integrating domain -specfi priors (e.g., neurophysiological constraints), and adapting LUNA for real-time inference scenarios. Beyond technical advancements, the development efficient, topology-invariant EEG models like LUNA could enhance neurological diagnostics and research accessibility. However, careful attention must be paid mitigating risks such as algorithmic bias and ensuring patient data privacy for deployment. Future work should integrate ethical concerns alongside technical improvements. Pre-training montage diversity is limited (three dominant layouts); future work will incorporate multi-dataset pre- training and randomized channel dropout improve generalization to unseen, dense and sparse montages.
Acknowledgments and Disclosure of Funding
This project supported by the Swiss National Science Foundation under the grant number 193813 (PEDESITE project). This work was supported by the ETH Future Computing Laboratory (EFCL), financed by donation from Huawei Technologies. This work was supported by a grant from the Swiss National Supercomputing Centre (CSCS) under project ID Ip12 on Alps.
10

---

References
[1] Alexander Craik, Yongtian He, and Jose L Contreras- Vidal. Deep learning for electroencephalo- gram (EEG) classification tasks: review. Journal of Neural Engineering, 16(3):031001 April 2019.
[2] Yonghao Song. Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. EEG conformer: Convo- lutional transformer for EEG decoding and visualization. IEEE Transactions Neural Systems and Rehabilitation Engineering, 31:710-719, December 2023.
[3] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-2023, pages 6778-6786. International Joint Conferences on Artificial Intelligence Organization, August 2023.
[4] Lichao Xu, Minpeng Xu, Yufeng Ke, Xingwei An, Shuang Liu, and Dong Ming. Cross-dataset variability problem in EEG decoding with deep learning. Frontiers in Human Neuroxcience, 14:103, April 2020.
[5] Chaoqi Yang, M Westover, and Jimeng Sun. BIOT: Biosignal transformer for cross-data learning in the wild. In A. Oh, Т. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 78240-78266 Curran Associates, Inc., September 2023.
[6] Ke Yi, Yansen Wang, Kan Ren, and Dongsheng Li. Learning topology-an -agoossic EEG represen- tations with geometry-awant modeling. In Thirty-seventh Conference on Neural Information Processing Systems, September 2023.
[7] Georgios Mentzelopoulos, Evangelos Chatzipantaziss Ashwin G Ramayya, Michelle Hedlund, Vivek Buch, Kostas Danillidis, Konrad Kording, and Flavia Vitale. Neural decoding from stereotactic BEG: accounting for electrode variability across subjects. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, September 2024.
[8] Xuefen Lin, Jiclin Chen, Weifeng Ma, Wei Tang, and Yuchen Wang. EEG emotion recognition using improved graph neural network with channel selection. Computer Methods and Programs in Biomedicine, 23 1003800, April 2023.
[9] Iyad Obeid and Joseph Picone. The temple university hospital EEG data corpus. Frontiers in Neuroscience, 10:196, May 2016.
[10] Paolo Detti, Giampaolo Vatti, and Garazi Zabalo Manrique de Lara. Siena scalp EEG database, 2020. This work licensed under the Creative Commons Attribution 4.0 International License. To view copy of this license, visit ttsitttttp:///rraiiiiee
[11] Wei Liu, Jie- Lin Qiu, Wei-Long Zheng, and Bao-Liang Lu. Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition. IEEE Transactions on Cognitive and Developmental Systems, 14(2):715-729, June 2022.
[12] Demetres Kostas, Stephane Aroca-Ouellette and Frank Rudzicz. BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Frontiers in Human Neuroscience, 15:653659, June 2021.
[13] Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. BRainBERT: Self-supervised representation learning for intracranial recordings. In The Eleventh International Conference on Learning Representations, February 2023.
[14] Yugi Chen, Kan Ren, Kaitao Song, Yansen Wang, Yifan Wang, Dongsheng Li,, and Lili Qiu. EFGFormer: Towards transferable and interpretable large-scale EEG foundation model. In AAAI 2024 Spring Symposium on Clinical Foundation Models, February 2024.
[15] Weibang Jiang, Liming Zhao, and Bao liang Lu. Large brain model for learning generic representations with tremendous EEG data in BCI. The Twelfth International Conference on Learning Representations, January 2024.

---

[16] Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. CBraMod: A criss-cross brain foundation model for EEG decoding. In The Thirteenth International Conference on Learning Representations, January 2025.
[17] Daoze Zhang, Zhizhang Yuan, Yang Yang, Junru Chen, Jingjing Wang, and Yafeng Li. Brant: Foundation model for intracramial neural signal. In Thirty-seventh Conference on Neural Information Processing Systems, September 2023.
[18] Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang. Florian Dubost, Daniel Rubin, and Christopher Lee-Messer. Self-supervised graph neural networks for improved electroencephalograahic seizure analysis. In International Conference on Learning Representations, January 2022.
[19] Alexandru Dimofte, Glenn Anta Bucagu, Thorir Mar Ingolfsson, Xiaying Wang, Andrea Cossettini, Luca Benini, and Yawei Li. CEReBrO: Compact encoder for representations of brain oscillations using efficient alternating attention, January 2025.
[20] Geeling Chau, Christopher Wang, Sabera J Talukder, Vighnesh Subramaniam, Saraswati Soedar- madji, Yisong Yue, Boris Katz, and Andrei Barbu. Population transformer: Learning population- level representations of neural activity. In The Thirleenth International Conference Learning Representations, January 2025.
[21] Aaqib Saeed, David Grangier, Olivier Pietquin, and Neil Zeghidour. Learning from heteroge- neous eeg signals with differentiable channel reordering. In ICASSP 202/-2021 IEEE inter- national conference on acoustics, speech and signal processing (ICASSP), pages 1255-1259, IEEE, 2021.
[22] Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention- permutation-invarian neural networks. In International conference on machine learning, pages 3744-3753. PMLR, June 2019.
[23] Andrew Jaegle, Sehastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier J Henaff, Matthew Botvinick, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. Perceiver IO: A general architecture for structured inputs & outputs. In International Conference on Learning Representations, January 2022.
[24] Yuxin Wu and Kaiming He. Group normalization. International Journal of Computer Vision, 128(3):742-755, July 2019.
[25] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), June 2016.
[26] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: representing scenes as neural radiance fields for view synthesis. Commun. ACM, 65(1):99-100 December 2021.
[27] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063,, February 2024.
[28] Thorir Mar Ingolfsson, Andrea Cossettini, Simone Benatti, and Luca Benini. Energy-efficient tree-based eeg artifact detection. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 3723-3728. IEBE, 2022.
[29] Jin Jing, Wendong Ge. Shenda Hong. Marta Bento Fernandes, Zhen Lin, Chaoqi Yang, Sungtac An, Aaron F. Struck, Aline Herlopian, Ioannis Karakis, Jonathan J. Halford, Mareus C. Ng. Emily L. Johnson, Brian L. Appavu, Rani A. Sarkis, Gamaleldin Osman, Peter W. Kaplan, Monica B. Dhakar, Lakshman Arcot Jayagopal, Zubeda Sheikh, Olga Taraschenko, Sarah Schmitt, Hiba A. Haider, Jennifer A. Kim, Christa B. Swisher, Nicolas Gaspard, Mackenzie C. Cervenka, Andres A. Rodriguez Ruiz, Jong Woo Lee, Mohammad Tabaeizadeh, Emily J. Gilmore, Kristy Nordstrom, Ji Yeoun Yoo, Manisha G. Holmes, Susan T. Herman, Jennifer A. Williams, Jay Pathmanathan, Fábio A. Nascimento, Ziwei Fan, Samaneh Nasiri, Mouhsin M. Shafi, Sydney S. Cash, Daniel B. Hoch, Andrew J. Cole, Eric S. Rosenthal, Sahar F. Zafar,
12

---

Jimeng Sun, and M. Brandon Westover. Development of expert-level classification of seizures and rhythmic and periodic patterns during EEG interpretation. Neurology, 100(17)ee1500<e1722 April 2023.
[30] Chaoqi Yang, Cao Xiao, M. Brandon Westover, and Jimeng Sun. Self-supervised electroen- cephalogram representation learning for automatic sleep staging: Model development and evaluation study. JMIR AI, 2:e46769, July 2021.
[31] Wei Yan Peh, Yuanyuan Yao, and Justin Dauwels. Transformer convolutional neural networks for automated artifact detection in scalp EEG, July 2022.
[32] Hongli Li, Man Ding, Ronghua Zhang, and Chunbo Xiu. Motor imagery EEG classification algorithm based on enn-lstm feature fusion network. Biomedical Signal Processing and Control, 72:103342, February 2022.
[33] Yonghao Song, Xueyu Jia, Lie Yang, and Longhan Xie. Transformer-based spatial-temporal feature learning for EEG decoding, June 2021.
[34] Navid Mohammadi Foumani. Geoffrey Mackellar, Soheila Ghane, Saad Irtza, Nam Nguyen, and Mahsa Salchi. Eeg2rep: Enhancing self- supervuped EEG representation through informative masked inputs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, pages
ACM, August 2024.
[35] Anna Tegon, Thorir Mar Ingolfsson, Xiaying Wang, Luca Benini, and Yawei Li. FEMBA: Efficient and scalable EEG analysis with a bidirectional mamba foundation model, February 2025.
[36] Vernon Lawhern, Amelia Solon, Nicholas R Waytowich, Stephen M Gordon, Chou Hung, and Brent Lance. Eegnet: compact convolutional neural network for eeg-based brain-computer interfaces. Journal of Neural Engineering, 15(5)0056013, July 2018.
[37] Siyi Tang, Jared A Dunnmon, Qu Liangqiong, Khaled Saab, Tina Baykaner, Christopher Lee-Messer, and Daniel L Rubin. Modeling multivariate biosignals with graph neural networks and structured state space models. In Bobak Mortazavi, Tasmie Sarker, Andrew Beam, and Joyce C. Ho, editors, Proceedings the Conference on Health. Inference. and Learning, volume 209 of Proceedings of Machine Learning Research, pages 50-71. PMLR, June 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose topology-agne foundation model with focus computa- tional efficiency. We tediously compare our model based on downstream task performance, including datasets with different electrode topologies, and computational efficiency with respect to prior art in Section 4
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
These limitations include lower performance the SEED-V dataset, which is an unseen and denser topology compared to pre-training datasets, and the reliance on positional encodings learned during pre-training.
Justification: We discuss the limitations in Section 4.2 and 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specificattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
14

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper doesn't include theoretical results. We discuss the computational complexity of different methods theoretically, but report the empirical computational cost in
Figure 2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We thoroughly explain our model architecture and experimental details in Section 3 4.1 We also use public datasets and describe the preprocessing details. We will also release the code and the pre-trained model weights upon publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We will publish the code and the pre-trained model weights upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.cc/put wwtpcccccppccccceee coroolicgguieeee www.deeeeeeeebmmeeeeeeee
ssionnolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The details are discussed in detail in Scction as well as in A.1 in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the mean and the standard deviation of results obtained from initializing the models with different seeds, as described in Section 4.1
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative етror rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the GPU types and the training duration in Section 4.1 and A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwrrpps.ccccolll/////eea
Answer: [Yes]
Justification: We use public EEG datasets that conform to ethical guidelines and are com- monly used in literature.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
17

---

Justification:
The societal impacts are discussed in Section S
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper does not introduce any such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We correctly cite and conform the licenses of the papers that produced the public datasets we used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
18

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssippsiiiie thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pa se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We do not release any assets yet. The code and the released model will be released open source with the appropriate license and documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not include research with human subjects. We use existing public datasets in the area.
Guidelines:
The answer NA means that the paper doess not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not include research with human subjects. We use existing public datasets in the area.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were only used for writing, editing, amd formatting the content.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

A
Appendix
This appendix provides supplementary details that complement the main paper: additional effi- ciency analyses, reporting notes for statistical testing, small implementation clarifications, and extra qualitative comparisons.
A.1 Model Architecture Details
The following tables show the hyperparameter setup for the pre-training and the downstream fine- tuning for LUNA.
A.1.1
Hyperparameters for pre-training
Table 5: Hyperparameters for EEG pre-training.
Hyperparameters
LUNA-Base
LUNA-Large
LUNA-Huge
Input channels Output channels
(1,8,8)
(1,16,16)
(1,32,32)
(16,16,16)
[24.24,24)
[32,32,32]
Temporal Encoder
Kernel size
(20,3,3)
Stride Padding
(10,1,1)
(9,1,1)
Patch size
40
Transformer encoder layers
10
Number of queries
6
Query size
96
128
Hidden size
256
576
1024
MLP size
1024
2304
4096
Attention head number
12
16
Batch size per GPU
2040
2040
720
Total batch size
8160
8160
11520
Peak learning rate
1.25e-4
Minimal learning rate
2.5e-7
Learning rate scheduler
Cosine
Optimizer
Adam W (0.9.0.98)
Adam
Weight decay
0.05
Total epochs
60
Warmup epochs
10
Loss type
Smooth-L!
Non-me Non-masked region loss coefficient
0.05
Query specialization loss coefficient
0.8
Gradient clipping
Mask ratio
0.5
Precision
bf16-mixed
21

---

A.1.2 Hyperparameters for downstream fine-tuning
Table 6: Hyperparameters for downstream fine-tuning.
Hyperparameters
Values
Batch size per GPU
512
Peak learning rate
le-4
Minimal learning rate Learning rate scheduler
5e-6
Cosine
Optimizer
AdamW
Adam / Weight decay
(0.9,0.999)
0.05
Total epochs
50
Early stopping patience
10
Warmup epochs
5 0.1 (B/L.) 0.2 (H)
Drop path
Layer-wise learning rate decay
0.5 (B) 0.8 (L/H) 0.1
Label smoothing (multi-class classification)
A.1.3
Complexity Analysis
The computational complexity of key attention stages and a comparison with alternatives are shown
in 7 and g
Table 7: Complexity Breakdown of LUNA Encoder Stages.
Stage
Input Shape
Complexity
Module (Cross-Attn)
(B x E E S) x S.B.S.Q2.B x
U.B.SQQ.... E)
Query leff-Attention
Patch-wise Attention Encoder (Self-Attn)
S2. Q.FF) F)
Table 8: Attention Complexity Comparison.
Method
Bottleneck Complexity
LUNA (Latent Space Attention)
O(B S2
Q F) or O(B 8 C.Q.--
Full-Attention (e.g.. LaBraM)
O(B 52 C2. E)
Alternating Attention (Patches, e.g., CBraMod)
O(B C..
Alternating Attention (Channels, e.g., CBraMod)
O(B S.CC. C E)
BIOT vs. LUNA scaling. patch and channel counts.
Tables report GFLOPs and peak activation memory across varying
A.2 Dataset and Preprocessing Details
Datasets Used
We use publicly available EEG datasets, provided in 11
A.3 Experimental Settings
Pre-training LUNA is pre-trained using masked patch reconstruction task. Key hyperparameters
are listed in 5
Computational Resources Experiments were conducted using NVIDIA A 100 GPUs. Pre-training took approximately 1 day on GPUs for the base and large models and 16 GPUs for the huge model.
22

---

Table 9: Scaling with patch count (GFLOPs and MiB per forward).
Model
#Patches
FLOPs (G)
Memory (MiB)
BIOT
2000
1143
3534
LUNA-Bas
2000
253
1835
LUNA-Larrge
2000
1073
2969
LUNA-Huge
2000
6570
5549
BIOT
3000
1714
5231
LUNA-Base
3000
478
3873
LUNA-Large
3000
1886
6041
LUNA-Hlue
3000
11035
9685
BIOT
4000
2286
6931
LUNA-Bast
4000
768
6678
LUNA-Lard -arre
4000
2884
10265
LUNA-Huge
4000
16286
15344
Table 10: Scaling with channel count (GFLOPs and MiB per forward).
Model
#Channels
FL.OPs (G)
Memory (MiB)
BIOT
6000
3117
9270
LUNA-Bas
6000
18
1571
LUNA-Large
6000
43
2410
LUNA-Huge
6000
112
4198
BIOT
7000
3637
10791
LUNA-Base
7000
20
1826
LUNA-Large
7000
49
2777
LUNA- Huge
7000
122
4680
BIOT
8000
4156
12307
LUNA-Base
8000
23
2069
LUNA-Large
8000
55
3139
LUNA-Huge
8000
133
5163
Table 11: Summary of Datasets Used.
Dataset
# Subjects
Samples (Train/Val/Test Total)
Hours of Recordings
Channels
Montage Used
TUEG (Pre-train)
14,987
15. 25,686,874 (Toral)
21,787.32
20 22
Bipolur
Siena (Pre-train)
101,520 (Tatal)
141.0
Unipolar
TUAB
2,329
591,357 154.938 74,010
1,139.31
Bipolar
TUAR
213
49,241 / 5,870 5.179
83,74
22
Bipolar
TUSL.
16.088 1, 2,3 2,540
27.54
Bipolar
SEED-V
43,328 /3,360 31,056
32.70
Unipolar
A.4
Additional Quantitative Results
Training Curves The pre-training loss curves for LUNA-Base are shown in The reconstruction loss drops shows and initial plateau then drops slowly over the epochs, while the query specialization shows a jump and then a slow decrease, indicating more orthogonal query usage over time. The initial drop of the query specialization might be due trivial case where query attends only one channel. The queries learn to attend to their own specialized areas afterwards while covering all the channels in the input.
A.5 Additional Visualizations
Reconstruction Examples Figures 6 g show examples of the model reconstructing masked patches (gray regions) for inputs with 20. 22, and 29 channels, respectively. The reconstructions capture the underlying signal trend and demonstrate robustness across different topologies seen during pre-training
23

---

Reconstruction Loss Epochs
Train Reconstruction Loss
Reconstruction Loss
0.45
Validation Reconstruction Loss
0.40
0.35
0.30
0
5
10
15
20
25
30
35 Epochs
40
45
50
55
60
65
Query Specialization Loss Epochs
Entropy Loss
0.0015
0.0010
0.0005
Train Query Sperialization
Validation Query Speciolization Loes
0
10
15
20
25
30
35 Epochs
40
45
50
55
60
65
Figure 5: Loss curves during pre-training for LUNA-Base (Reconstruction and Query Specialization Loss).
Secordtrucion randem batch with 20 channels
Channel2
0.0
-3.25
1.00
Decomstructed
0
200
600
500
1000
1200
Figure Example reconstruction on input with 20 channels (masked regions in gray).

---

1.0
0.79
0.50
-1.0
-1.7
-1.00
200
-00
600
903
1000
1200
Figure 7: Example reconstruction on input with 22 channels (masked regions in gray).
Reconstruction random batch with 29 chanmels
Charnel 18
Ding focomiiaatooo
6.2
0.50
079
-1.00
Figure 8: Example reconstruction on input with 29 channels (masked regions in gray).
A.6 Trade-off between Q and E
Observation. Increasing Q the expense of degrades performance; too few queries can also bottleneck capacity. A balanced Qx (e.g., 4x64) works well under the same latent budget.
A.7 Bipolar montage electrode pairs
We use the following longitudinal pairs to construct the bipolar montage for TUEG, TUAR, TUSL, and TUAB (left/right symmetric sets):
Fp1-F7, T3-T5, T5 01, Fp2-F8, F8-T4, T6. T6 O2 T3 C3, C3-CZ . FP1-F3, F3-C3, C3-P3, P3-O1, Fp2-F4, C4-C4, C4 P4-02 CZ-C4, C4 C4-T4
A.8 Edge deployment reference
Typical low-power edge SoCs used in wearables/IoT offer single-digit to few dozen MB of RAM and on-chip/SiP compute in the 10-100 GOPS range. Under these constraints, LUNA-Base (~7M params; 14 MB at 16-bit) and its measured GFLOPs per window 2b) fit comfortably within real-time budgets, whereas quadratic-in-6 spatial attention and larger activation footprints some baselines make deployment more challenging at higher channel counts.
A.9 Significance Testing
25

---

Table 12: LUNA- Base variants with fixed Q E-256
Variant (Q x A))
Q
E
TUAB AUROC
TUAB AUPRC
TUAR AUROC
TUAR AUPRC
TUSL AUROC
x 64
64
0.887
0.895
0,902
0,495
0.767
2 128
128
0.885
0.890
0.885
0,501
0.759
82
0.884
0.892
0.899
0,505
0.766
16 16
16 16
0.874
0.881
0.866
0.487
0.757
Unless noted, we report mean + s.d. over matched seeds. For ablations ran two-sided paired (-tests across seeds for specific comparisons requested by reviewers. For the TUAR AUROC comparison of the model with without the query specialization loss, the paired t-test yielded p=0.2136, i.e., not statistically significant at a=0.05. Observed AUROC deltas across ablations were small (absolute s 0.01).
Table 13: Paired t-test on TUAR AUROC for the specialization-los ablation.
Comparison w/o specialization full
Mean (w/o full)
-0.007
0.2136
We adopt a pragmatic tolerance of =0.01 AUROC for considering two variants practically equivalent these datasets; all reported ablations fall within this band.
Practical tolerance.
A.10 Effect of Specialization Loss on Query Maps
Figure 9 depicts spatial attention maps of the Q queries when the specialization loss is removed. In Figure@depictts
this setting, two queries converge to coarse lateralized patterns (left and right longitudinal chains), while the remaining queries display broad, overlapping support over conto-ceetton and midline sites with weaker focal peaks, indicating partial redundancy and gaps in complementary coverage. Overall, the maps exhibit higher overlap and reduced distinctiveness across queries compared to the model trained with the loss, where query maps are more complementary and less overlapping Fig. 4
Query
Queey
Query 3
Quasy
Figure 9: Query attention visualization without specialization loss.
A.11 t-SNE of Raw Frequency Features vs. LUNA Features
We compute per-segment frequency features (magnitude/phase statistics per band, averaged across channels) and compare 2D t-SNE embeddings to those of LUNA's latent features. Raw features exhibit less separation beyond clear artifacts on TUAR; LUNA features show tighter clustering aligned with labels.
26

---

30
Xanal Sgaal
30-
Alounzal Steak
20-
I-SNE Componet
U-NN Cotppventt
-10
-00
Shiesit
-30
Blestodl Artilats
20
10
20
-30
-10
10
50
40
+SSE Component
6-508 Compotete:
(a) TUAB dataset (Normal vs. Abnormal Signal).
(b) TUAR dataset (Artifact Types).
Figure 10: t-SNE of raw features on downstream datasets before fine-tuning.
27


Reviews:
[{"review_id": "IDMbQNIq1J", "reviewer": "Reviewer_GV5R", "summary": "This paper introduces a self-supervised foundation model designed to tackle the topological heterogeneity of EEG recordings (variations in channel configurations) during large-scale training. The method first segments each input sample into non-overlapping, single-channel patches and applies both temporal and frequency embeddings to these patches. A channel-unification module, built on a cross-attention mechanism, then compresses the variable channel sets into a fixed channel dimension. The unified representation is reshaped along the temporal axis and fed into a patch-wise temporal encoder. Self-supervised pre-training is performed through masked-patch reconstruction. The model is pre-trained on the TUEG and Siena Scalp EEG databases, and then fine-tuned on four downstream datasets, achieving state-of-the-art performance.", "questions": "See weakness.", "limitations": "See the conclusion part of the paper.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**: 1) The manuscript is clearly written and easy to follow. The authors articulate a compelling motivation: topological heterogeneity across EEG datasets hampers the training of large-scale foundation models. 2) The proposed embedding module jointly encodes temporal, spatial, and spectral information—three pillars of effective EEG representation learning. 3) The channel-unification module built on cross-attention is intuitive and, in principle, broadly applicable to heterogeneous-channel scenarios. 4) The cross-attention mechanism is again utilized during reconstruction, and the downstream task is simplified and effective, thereby avoiding the need for different projection heads for various downstream datasets. 4) The paper presents a thoughtful analysis of complexity. By introducing a two-stage attention scheme—first unifying channels, then applying temporal self-attention—the computational cost is reduced from $S \\times C$ to $S$, where $S$ is the number of patches per channel and $C$ the number of channels.\n\n**Weaknesses**: 1) For a fair evaluation, the experiments on TUSL and TUAR should adopt subject-independent splits rather than random sample-level splits. 2) In Algorithm 1 (line 148), should the tensor $Q_{\\text{learn}}$ have shape $\\mathbb{R}^{B \\times Q \\times E}$ to remain symmetric with $E_{\\text{dec}}$ on line 163 ($\\mathbb{R}^{B \\times C \\times E}$)? Please clarify if my understanding is incorrect.", "was_revised": true, "final_justification": "I kept my score for 5.", "rebuttal": "We thank the reviewer for their constructive and helpful feedback! You can find our question-wise responses below.\n### Weakness-1: Fairness of Evaluation Splits for TUAR/TUSL\n\n> For a fair evaluation, the experiments on TUSL and TUAR should adopt subject-independent splits rather than random sample-level splits.\n\nWe agree that subject-independent evaluation is the gold standard for assessing clinical generalization, and we appreciate the feedback.\n\nOur decision to use a sample-level split for the TUAR and TUSL benchmarks was a deliberate one, to ensure a direct and fair comparison with the established previous literature. The evaluation protocols in recent leading works that we compare against, such as EEGFormer, have consistently used this random splitting mechanism. Adopting a different, subject-independent split would have made our results fundamentally incomparable to theirs, making it impossible to fairly assess the performance gains of our model relative to prior art. We would like to run all the baselines on a subject-independent split and hope to include this in future work.\n\n### Weakness-2: Clarification on Tensor Shapes\n\n> In Algorithm 1 (line 148), should the tensor $Q_{learn}$ have shape $B \\times Q \\times E$ to remain symmetric with $E_{dec}$ on line 163 ($B \\times C \\times E$)? \n\nThe query tensor $Q_{learn}$ is a set of $Q$ vectors of shape $E$, so the learnable parameter shape size is indeed $Q \\times E$. Similarly, $E_{dec}$ is a collection of $C$ vectors of size $E$. These tensors are repeated in the batch dimension as needed. In the channel-unification module, the queries are repeated in the batch dimension to have the shape $(B.S) \\times Q \\times E$, to be able to apply cross-attention with the input features of size $(B·S) \\times C \\times E$. We will clarify this distinction of the initial learnable tensors and their reshaped version in the final manuscript better to avoid any confusions."}, {"review_id": "vEUvqNkwmu", "reviewer": "Reviewer_44Ri", "summary": "The paper presents a foundation model architecture for EEG brain data, which can ingest various “topologies”, or sensor layouts, and as a result, make the memory complexity of the attention layers of the subsequent transformer layers independent of the number of input sensors. This is achieved through a “channel unification” module, which uses learned query cross-attention to mix information from different sensors to a fixed size latent space. The model is pretrained on ~21k hours of clinical EEG data using a masked patch reconstruction task. The model is then finetuned on 4 downstream tasks and performance is compared to various baselines. While reported performance is sometimes lower than existing baselines, computational complexity can be significantly lower than baselines depending on the selected query size.", "questions": "1. How was $Q$ (the number of latent queries) selected? Intuitively, 4-8 seems like a very small number given that each query can be interpreted as a sort of (dynamic) spatial filter, and that there are potentially many more brain sources that may be relevant to the downstream tasks. I understand that this is in fact an important tradeoff, as the larger $Q$ becomes, the higher the computational complexity becomes (if this gets on the same order as the channel dimension, the computational complexity benefits may actually disappear). Exploring, or at least discussing, the tradeoff between $Q$ vs.$E$ would be useful to understand this point.\n2. The layouts seen during pretraining are in fact very consistent: only three montages were seen (20, 22, 29 channels). This, along with the lower performance on the SEED-V dataset with 62 channels, limits the impact of the study, as the appeal of such an approach is that very different montages can be handled in a unified manner. Exposing the model to more diverse channel configurations, e.g. by adding datasets with different montages to the pretraining set, or by randomly dropping out channels during training, could help generalization on new unseen montages. Also, testing on more varied electrode layouts would be important to support the conclusion (for instance, very sparse layouts for e.g. sleep data, and dense layouts from research-grade EEG systems).\n3. If I understand correctly, the decoder of Section 3.2 requires knowing the number of channels of the input. Does this mean a separate decoder is required for each montage/dataset, or can these be reused between datasets? Of note, $E_{dec}$ is used to refer to both the decoder queries and the output of the cross-attention at line 163.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n* Quality: The experimental setup, as in other EEG foundation model papers, relies on relevant downstream tasks to evaluate the usefulness of the representations learned during pretraining. This overall setup is sound, as is the comparison of theoretical computational complexity of the different existing models.\n* Clarity: The paper is overall clearly written. The problem that is being solved is clearly exposed (varying electrode layouts and quadratic computational complexity), and the channel unification module is clearly explained. Most details necessary to reproduce the paper are described.\n* Significance: The results are likely to have some impact on the community, as EEG foundation models are still being actively developed, and solutions to make them sensor layout-agnostic and reduce their computational complexity may facilitate scaling and usability in downstream scenarios. \n* Originality: This work builds upon previous EEG foundation model work that are well cited and contextualized in the manuscript. However a relevant reference appears to be missing (see Weaknesses).\n\nWeaknesses\n\n* Quality: \n    * The ablation studies of Section 4.4 (learned queries vs. fixed regions, query specialization loss) suggest that the change in performance is very small (3e-3 to 7e-3). These changes are on the same order as the reported standard deviation over a few seeds, and seem to actually yield improved performance on one of the metrics (TUAR AUC-PR). Drawing conclusions from these experiments is therefore difficult, and I would suggest increasing downstream task coverage before confirming these observations.\n    * The layouts seen during pretraining are in fact very consistent, which limits the impact of the results (see Q2).\n* Clarity: a few points remained unclear from the manuscript:\n    * $A_{affinity}$ in the query specialization loss (line 172) is not defined.\n    * It is not clear whether “multiclass classification” (line 189) refers to a multiclass classification problem, or actually to a “multilabel classification” problem, i.e. where each label is treated as a separate classification problem.\n    * Preprocessing: it is not clear how the data can be converted to a bipolar montage (line 196).\n    * How are the FLOPs of Figure 2 computed?\n* Significance: Despite downstream performance being on a similar scale as existing EEG foundation models, the proposed approach underperforms similar-scale or smaller baseline models on three out of four downstream tasks. This limits the impact of the claims as it is not yet clear how important computational complexity constraints should be a limiting factor in EEG foundation model use cases (vs. performance).\n* Originality: \n    * The work of Saeed et al. (2020) is similar to the proposed approach, i.e. using learned query cross-attention to project input EEG channels to a fixed space (however without channel position information, and using a purely supervised learning task) and is tested on similar datasets. The differences with this work should be highlighted in the manuscript.\n    * I would have expected BIOT to be included in the analysis of computational complexity, as it uses a linear attention mechanism, and still outperforms a larger LUNA model on at least one downstream task. \n\nSaeed et al., 2020. Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering, http://arxiv.org/abs/2010.13694.", "was_revised": true, "final_justification": "The submission presents a relevant EEG foundation model architecture with higher computational efficiency than existing models. The model is below the SOTA on three out of four tasks, but downstream performance remains generally competitive. As my concerns were appropriately addressed during the discussion period, notably on the importance of computational efficiency considerations, I raise my original score.", "rebuttal": "We thank the reviewer for their constructive and helpful feedback! You can find our question-wise responses below.\n\n### Weakness-1: Conclusiveness of Ablation Studies\n\nWe believe that these design choices have both quantitative and practical benefits:\n\n* Learned Queries vs. Fixed Regions: The primary advantage of using learned queries is not just the performance boost, but their practicality and flexibility. Our approach is fully data-driven and does not require any prior anatomical knowledge or mappings of electrodes to brain regions. This is a significant advantage over methods that rely on such priors, which can be overly simplistic, as brain regions are highly interconnected, or unavailable for non-standard montages. \n* Query Specialization Loss: The goal of this loss is to foster a more structured and interpretable latent space. By encouraging diversity, the loss pushes each query to learn a distinct spatial profile, effectively acting as an emergent, data-driven basis function for spatial unification. This specialization is visualized in Figure 4, where queries focus on different spatial regions.\n\nWhile we acknowledge the performance gains are on the same order as the standard deviation, it is important to note that the full model demonstrates a consistent positive trend. It achieves the highest AUROC on both TUAB and TUAR compared to its ablated counterparts. We believe that this consistent improvement in metrics, when combined with the significant practical benefits described above, provides a strong justification for our design choices.\n\n***\n\n### Weakness-2: Clarity of Technical Details\n\n1.  **$A_{affinity}$ Definition**: We apologize for this omission. The term **$A_{affinity}$** refers to the **attention score matrix** from the Channel-Unification Module's cross-attention layer. This matrix has a shape of $(B \\cdot S) \\times Q \\times C$ and represents the relationship between each of the $Q$ queries and $C$ input channels. We will add this precise definition in the revised manuscript.\n\n2.  **Multiclass vs. Multilabel**:  This is indeed a \"multiclass\" problem. Each segment is assigned a single artifact label from a set of five distinct classes, similar to the EEGFormer setup. We will clarify this wording. \n\n3.  **Bipolar Montage Conversion**: This is a standard EEG preprocessing technique where new virtual channels are created by taking the voltage difference between pairs of adjacent electrodes. We used the \"double banana\" method described in the [dataset description](https://isip.piconepress.com/publications/reports/2020/tuh_eeg/electrodes/electrodes_and_channels_v30.docx). We will clarify this.\n\n4.  **FLOPs Calculation**: The FLOPs reported in Figure 2 were computed empirically using the `fvcore` library. We calculated the total GFLOPs for a single forward pass using both the encoder and decoder parts of these models.\n\n***\n\n### Weakness-3: Performance vs. Baselines and Significance\n\nWe appreciate the reviewer's careful analysis of our results. While LUNA does not universally outperform all baselines on every metric, its primary contribution is providing a favorable accuracy-efficiency trade-off and a scalable architecture that is inherently topology-agnostic.\n\nWe achieve state-of-the-art results on both the TUAR and TUSL benchmarks. On the tasks where performance is slightly lower (TUAB, SEED-V), LUNA offers substantial computational advantages. We want to highlight that models with less number of parameters might end up requiring more resources than LUNA, e.g., CBraMod. We believe computational efficiency to be an important aspect of EEG model development, as these models can potentially be deployed on edge devices for real-time monitoring.\n\n***\n\n### Weakness-4: Missing References and Baseline Comparisons\n\nWe are very grateful to the reviewer for bringing the work of **Saeed et al. (2020)** to our attention and apologize for this oversight. It is indeed a relevant piece of work. We will add it to our Related Work section and highlight the key differences, such as LUNA being a self-supervised foundation model that focuses on efficiency.\n\nRegarding BIOT, this is an excellent suggestion. It can serve as an important benchmark for efficiency. We also added BIOT to our computational complexity analysis, and you can see a snippet of the resource usages below:\n\n#### Scaling with Patch Count\n\n| Model | Number of Patches | FLOPs (G) | Memory (MiB) |\n|:---|:---:|:---:|:---:|\n| **BIOT** | 2000 | 1143 | 3534 |\n| LUNA-Base | 2000 | 253 | 1835 |\n| LUNA-Large | 2000 | 1073 | 2969 |\n| LUNA-Huge | 2000 | 6570 | 5549 |\n| | | | |\n| **BIOT** | 3000 | 1714 | 5231 |\n| LUNA-Base | 3000 | 478 | 3873 |\n| LUNA-Large | 3000 | 1886 | 6041 |\n| LUNA-Huge | 3000 | 11035 | 9685 |\n| | | | |\n| **BIOT** | 4000 | 2286 | 6931 |\n| LUNA-Base | 4000 | 768 | 6678 |\n| LUNA-Large | 4000 | 2884 | 10265 |\n| LUNA-Huge | 4000 | 16286 | 15344 |\n\n#### Scaling with Channel Count \n\n| Model | Number of Channels | FLOPs (G) | Memory (MiB) |\n|:---|:---:|:---:|:---:|\n| **BIOT** | 6000 | 3117 | 9270 |\n| LUNA-Base | 6000 | 18 | 1571 |\n| LUNA-Large | 6000 | 43 | 2410 |\n| LUNA-Huge | 6000 | 112 | 4198 |\n| | | | |\n| **BIOT** | 7000 | 3637 | 10791 |\n| LUNA-Base | 7000 | 20 | 1826 |\n| LUNA-Large | 7000 | 49 | 2777 |\n| LUNA-Huge | 7000 | 122 | 4680 |\n| | | | |\n| **BIOT** | 8000 | 4156 | 12307 |\n| LUNA-Base | 8000 | 23 | 2069 |\n| LUNA-Large | 8000 | 55 | 3139 |\n| LUNA-Huge | 8000 | 133 | 5163 |\n| | | | |\n\nAs can be seen from the table, BIOT is a more efficient model compared to LaBraM and CBraMod, but it still fails to scale better than LUNA when the number of patches/channels is increased. Moreover, BIOT uses fixed positional encodings for the time points and also for channels. Therefore, it is not topology-agnostic in the same way as LUNA is, and it is more limited.\n***\n\n### Question-1: Selection of Q\nThe number of latent queries, `Q`, was determined empirically, guided by the goal of finding an optimal balance between model performance and computational efficiency. Each query can be interpreted as a learned, dynamic spatial filter, and the trade-off between the number of these filters (`Q`) and their representational capacity (the embedding dimension, `E`) is crucial.\n\nTo provide a concrete answer and explore this trade-off, we conducted an ablation study on the LUNA-Base architecture. We kept the total latent feature dimension constant (`Q` × `E` = 256) while varying the two factors. The results are presented below.\n| Model Variant (`Q` x `E`) | #Queries(`Q`) | Embed Dim(`E`) | TUAB AUROC  | TUAB AUPRC  | TUAR AUROC  | TUAR AUPRC  | TUSL AUROC  | TUSL AUPRC  |\n|:-----------------------------:|:------------------:|:------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|:--------------------:|\n| **LUNA-Base (4 x 64)**        | **4**              | **64**             | **0.887**      | **0.895**     | **0.902**      | 0.495    | **0.767**      | **0.301**    |\n| LUNA-Variant (2 x 128)        | 2                  | 128                | 0.885     | 0.890     | 0.885     | 0.501      | 0.759      | 0.288      |\n| LUNA-Variant (8 x 32)         | 8                  | 32                 | 0.884     | 0.892      | 0.899      | **0.505**     | 0.766      | 0.284      |\n| LUNA-Variant (16 x 16)        | 16                 | 16                 | 0.874      | 0.881      | 0.866      | 0.487      | 0.757      | 0.272      |\n\n***\nThe analysis reveals two key insights into the trade-off:\n\n1.  A larger number of queries is not always better. The performance of the `16 x 16` variant is the lowest. This suggests that increasing `Q` at the expense of `E` is detrimental. While having more spatial filters is appealing, they are ineffective if they lack the individual embedding capacity (`E`) to learn rich representations. The queries might become too simplistic to capture meaningful patterns.\n\n2.  A minimum number of diverse queries is necessary. The `2 x 128` variant, which uses only two highly expressive queries, performs slightly worse than the balanced `4 x 64` configuration on most tasks. This suggests that having too few queries can create an information bottleneck, even if each query is powerful. The model may struggle to simultaneously represent the diverse spatial patterns needed for complex downstream tasks.\n\n### Question-2: Diversity of Pre-Training Montages\nThis is a key limitation of our current study, which we acknowledge in the paper. The available large-scale public datasets are not topologically very diverse. This lack of diversity during pre-training is the likely reason for the performance gap on the unseen 62-channel SEED-V montage.\n\nWe thank the reviewer for the excellent suggestions on how to mitigate this. Exposing the model to more diverse configurations during pre-training, like a random channel dropout strategy, is a promising direction for future work. \n\n***\n\n### Question-3: Decoder Design and Reusability\n1. We initialize a learnable decoder query for each distinct electrode in the pre-training dataset. The reconstruction head used for pre-training retrieves the corresponding decoder queries from the learnable set for the input channels.\n2. This reconstruction head is only used during pre-training to capture channel-specific features required to reconstruct each channel. If different datasets include shared electrodes, they can indeed be reused. For all downstream fine-tuning tasks, this head is discarded and replaced with a task-specific classification head. This classification head uses a single aggregation query and is completely independent of the original channel count, $C$. Therefore, the pre-trained LUNA encoder is fully reusable and montage-agnostic.\n3. We apologize for the confusing reuse of the variable $E_{dec}$. We will rename a variable in the manuscript to clarify."}, {"review_id": "qOo5tac83n", "reviewer": "Reviewer_YX5p", "summary": "This paper focuses on advancing the EEG foundation model by constructing the single, montage-agnostic architecture, and compared to the baselines, the expected advantage is to achieve a balance between efficiency and performance. The authors proposed the model named LUNA, which sets a bunch of learnable queries and uses the cross-attention module to map the patch-wise features into the fixed latent space, reducing the time complexity significantly compared to some state-of-the-art foundation model such as LaBraM and CBraMod while achieving reasonable performances on several benchmarks.", "questions": "1. Figure 3, what if I directly apply t-SNE to the original dataset (or some statistics of the original dataset)? I suspect that the original data would demonstrate some kinds of clustering phenomenon without undergoes the encoding process of LUNA. \n\n2. Is there practical benefit to enlarge the input size? For most of the dataset, the number of channels is fixed and relatively small, and there is no evidence that enlarging the time window would bring in better results for prediction. In a certain sense, the proposed trade-off between performance and complexity may not worth it.", "limitations": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n1. This proposed model mitigates the problem of scaling for the EEG foundation model, and is topology-agnostic, which means its format-compatibility.\n\n2. The proposed methods are straightforward and possibly helpful for the general EEG foundation training task. \n\n3. The writing is well structured and easy to follow. \n\n**Weakness**\n1. The model looks like a direct fusion of MMM and LaBraM / CBraMod, somehow limiting its own novelty. But I would admit that it's not a trivial attempt given the efforts to train the large EEG foundation model.\n\n2. My main concern lies in the performance. As is shown in the Table 1, the LaBraM and CBraMod achieved better performance even with less number of parameters. I wonder whether shrinking the large models can compensate their extra computational efforts. And Also, I wonder whether there is room for the proposed LUNA model to achieve SOTA after scaling up.\n\nTo add, in the first paragraph of Section 4.2, the authors claims that LUNA \"surpasses most self-supervised baselines and approaching large-scale models like LaBraM and CBraMod\", but in fact almost half of compared self-supervised baselines achieved comparable results. I suggest the authors rephrasing the statement and being more conservative. \n\n3. I'm also worried about the effectiveness of the Query Specialization Loss. While in Figure 4, the queries demonstrate different attended brain areas, losses shown in Figure 5 barely drop after the increase. I wonder whether removing the loss can yeild similar results.", "was_revised": true, "final_justification": "I have read the comments from other reviewers and the authors. The authors provided more information to enhance the paper. While it requires revision of the current manuscript, I believe the authors can handle them well. Thus I decide to keep my score as a borderline acceptance.", "rebuttal": "We thank the reviewer for their constructive and helpful feedback! You can find our question-wise responses below.\n### Weakness-1: Novelty\n\n> The model looks like a direct fusion of MMM and LaBraM / CBraMod, somehow limiting its own novelty. But I would admit that it's not a trivial attempt given the efforts to train the large EEG foundation model.*\n\nWhile we build upon established concepts like learned queries and large-scale transformers, the architectural innovation of LUNA lies in *how* these components are integrated to solve a different and critical problem in EEG analysis: the trade-off between performance, topological heterogeneity, and computational efficiency.\n\nOur key distinctions are:\n1. Unlike MMM, which applies mapping to hand-engineered features and predefined anatomical regions, LUNA learns an end-to-end mapping directly from raw EEG signals.\n2. Unlike LaBraM or CBraMod, which flatten spatial and temporal dimensions and incur quadratic complexity with respect to channels (O(C²)), LUNA's primary innovation is the Channel-Unification Module. This module first projects variable-channel data into a fixed-size latent space (with linear complexity O(C)) and *then* applies temporal attention. This architectural choice fundamentally decouples the computational cost from the electrode count, which is a novel and significant contribution designed specifically for efficiency and scalability.\n\nWe believe, LUNA to be a novel architecture designed to create a topology-agnostic and computationally efficient foundation model, a problem not directly addressed by the architectural designs of prior work.\n\n---\n\n### Weakness-2: Performance and Scaling\n\n> My main concern lies in the performance. As is shown in the Table 1, the LaBraM and CBraMod achieved better performance even with less number of parameters. I wonder whether shrinking the large models can compensate their extra computational efforts. And Also, I wonder whether there is room for the proposed LUNA model to achieve SOTA after scaling up.\n\nWe appreciate the reviewer raising this important point. Our main goal is to strike a new balance between accuracy and efficiency, and we believe the results demonstrate this successfully. \n1. While LaBraM-Huge and CBraMod show a ~1-2% higher AUROC on the TUAB benchmark (Table 1). This demonstrates that LUNA's architecture is highly effective and does not universally underperform. Another important point is that the smaller baseline models might need more resources depending on number of channels/time patches analyzed. For example, all LUNA model variants require less memory than even the smallest LaBraM model variant when the number of channels/time patches were increased. The trade-off is not just about the number of parameteres, but enabling analyses that are infeasible with quadratically scaling models, such as on high-density EEG or very long recordings. \n2.  **On Scaling Up:** We absolutely agree that there is room for LUNA to improve with scale. Our results already show a positive scaling trend from LUNA-Base (7M) to LUNA-Huge (311M) across different benchmarks. Further scaling, potentially with more pre-training data and compute, is a very promising direction for pushing performance even higher. We would also want to highlight that CBraMod was trained using a highly-preprocessed version of TUEG which only had 16 channels out of original 22 and 9k hours of EEG data compared to 22k hours of data we used. They showed that the preprocessing itself was a significant factor in improving the performance. Therefore, we believe that such preprocessing on the pre-training data and also increasing the amount of data are also promising directions for LUNA to achieve SOTA performance.\n---\n\n### Weakness-3: Phrasing of Claims\n\n> To add, in the first paragraph of Section 4.2, the authors claims that LUNA \"surpasses most self-supervised baselines and approaching large-scale models like LaBraM and CBraMod\", but in fact almost half of compared self-supervised baselines achieved comparable results. I suggest the authors rephrasing the statement and being more conservative.\n\nThis is an excellent suggestion. We agree that the original phrasing could be misinterpreted. We will revise this statement in the final manuscript to be more precise and conservative, ensuring it accurately reflects the results in Table 1. A better version might be \"LUNA demonstrated competitive performance on TUAB (Table 1). LUNA-Huge achieves an AUROC of 0.8957, which, while not surpassing the largest specialized models like LaBraM and CBraMod on this specific task, is competitive with several other self-supervised baselines while offering significant efficiency advantages.\"\n\n---\n\n### Weakness-4: Query Specialization Loss\n\n> I'm also worried about the effectiveness of the Query Specialization Loss. While in Figure 4, the queries demonstrate different attended brain areas, losses shown in Figure 5 barely drop after the increase. I wonder whether removing the loss can yeild similar results.\n\nWe thank the reviewer for this sharp observation. \n1. We found the loss to be effective the most initially, indicated by the initial drop in the loss, as it signifies the model moving away from a trivial state (e.g., all queries attending to the same location) to a state of diverse, specialized roles. The increase afterwards might mean that the model is putting a bigger emphasis on decreasing the reconstruction loss but in the end the slow decay shows that the model finds a balance between these two loss components.\n2.  The effectiveness of this loss is confirmed by our ablation study in **Table 4**. Removing the Query Specialization Loss (`- w/o Query Specialization Loss`) leads to a clear performance degradation on both TUAB (AUROC drop of 0.003) and TUAR (AUROC drop of 0.007). In addition to the metrics, this loss results in a clear specialization of the queries which makes it much easier to interpret what they learn and which electrodes they focus on.\n\n---\n\n### **Question-1: t-SNE on Raw Data**\n\n> Figure 3, what if I directly apply t-SNE to the original dataset (or some statistics of the original dataset)? I suspect that the original data would demonstrate some kinds of clustering phenomenon without undergoes the encoding process of LUNA.\n\nRaw EEG signals are high-dimensional, noisy, and contain significant subject-specific variance. Applying t-SNE directly to raw signal segments would likely show much less separation and structure compared to what is seen in Figure 3. \n\nTo make a fair comparison and to answer to your question, we designed an experiment where we calculated frequency domain features using Fourier transform, which are highly important for EEG signals, and visualized the t-SNE embeddings on these features (averaged over channels). These visualizations show that there are some clusters captured also by the raw frequency features such as electrode artifacts in TUAR, which might be very easy to detect with abnormal frequency features. However, raw frequency features don't capture other types of artifacts or the normal vs abnormal signal clusters in the TUAB dataset as well as LUNA produced features. We couldn't provide this visualization in the rebuttal now due to the no image rule in the rebuttal format but we will provide the comparison of raw frequency features vs LUNA features in the paper content. Please note that a similar analysis was done by Wang, Jiquan, et al. in the [CBraMod paper](https://arxiv.org/abs/2412.07236), in Figure 11 in the Appendix, which also shows that the raw features for EEG are not informative enough to show clear clusters in the t-SNE visualization.\n\n### **Question-2: Practical Benefit of Scalability**\n\n> Is there practical benefit to enlarge the input size? For most of the dataset, the number of channels is fixed and relatively small, and there is no evidence that enlarging the time window would bring in better results for prediction. In a certain sense, the proposed trade-off between performance and complexity may not worth it.\n\nThe trade-off is indeed highly valuable for real-world applications beyond the constraints of some public datasets.\n\nWhile many public benchmarks use 20-30 channels, high-density EEG systems (with 64, 128, or 256 channels) are standard in modern neuroscience research and advanced clinical settings. Models with quadratic channel complexity, like LaBraM, become computationally infeasible in these scenarios.\n\nFor the time length, the ability to efficiently process longer time windows is critical for detecting neurological phenomena that lasts over many minutes even hours, such as sleep stage transitions. In this work, we didn't use these long-duration recordings as a downstream task but we showed that LUNA has the ability to scale to much longer recordings than other SOTA methods.\n\n### References:\nWang, Jiquan, et al. \"CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding.\" The Thirteenth International Conference on Learning Representations."}, {"review_id": "8koFVroeP9", "reviewer": "Reviewer_jKJ9", "summary": "This paper aim to address the performance degradation during cross-dataset evaluation caused by significant topological heterogeneity of EEG data. The author propose LUNA (Latent Unified Network Architecture) directly addresses this gap. Their key innovation is a topology-invariant encoder that maps arbitrary electrode layouts into a fixed latent space via learned queries and cross-attention. They pre-train LUNA using a masked-patch reconstruction objective on two datasets and fine-tune on four downstream benchmarks.", "questions": "1. The conceptual and technical issues listed in the *Weaknesses* section require clarification—particularly the definition and justification of topology invariance, and the motivation behind LUNA’s design.\n2. What is the overall loss function used in pre-training LUNA? While the paper introduces the masked reconstruction loss and the auxiliary query specialization loss separately, the formulation of the total loss function is missing.\n3. The selection criteria and rationale for the patch length parameter $P$ should be discussed, as it likely plays a critical role in the model’s performance.", "limitations": "The *Methodology* section lacks a principled explanation of why LUNA was designed in its current form. This omission weakens the theoretical grounding of the proposed method and limits the paper's overall persuasiveness.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The topic addressed in this paper is important and timely for the brain-machine domain.\n\nWeakness:\n\n1. The writing is often difficult to follow. A clearer explanation of the motivation and methodological framework would significantly improve readability and understanding.\n2. The authors state that the Channel-Unification Module (Section 3.1) uses learned queries and cross-attention to project variable-channel features into a fixed-dimensional latent space, thereby achieving topology invariance. This appears to be a central contribution, yet the concept of topology invariance is neither clearly defined nor sufficiently explained. It remains unclear what topology invariance entails in this context, how the module achieves it, and why it is beneficial for addressing the stated challenges.\n3. The *Methodology* section focuses on how LUNA is constructed but lacks discussion on the underlying design motivations. Without such explanation, the architecture appears ad hoc, which undermines the theoretical credibility of the approach.\n4. The claim of achieving a state-of-the-art accuracy-efficiency trade-off is questionable. This metric is only meaningful when the accuracy itself reaches an acceptable benchmark. The paper does not provide sufficient justification that this condition is met.", "was_revised": true, "final_justification": "Based on previous reviews and discussions, we maintain our original score.\n\nThe primary reason is that the technical contribution of this work is insufficient. Simply combining established techniques without providing the rationale behind this combination is fatal to a paper.", "rebuttal": "We thank the reviewer for their constructive and helpful feedback! You can find our question-wise responses below.\n\n\n### Weakness-1: Clarity\n\n> The writing is often difficult to follow. A clearer explanation of the motivation and methodological framework would significantly improve readability and understanding.\n\nWe sincerely apologize that our explanations were not clear enough. This is a critical point, and we will thoroughly revise Sections 1 and 3 to provide the clarity the reviewer rightly expects. Our motivation in this work relies on solving a practical challenge in the field of EEG:\n\n* EEG is a non-invasive technique that records the brain's electrical activity using a set of electrodes placed on the scalp. The specific number and 3D positions of these electrodes are called an EEG montage or topology. The challenge i that there is no universal standard for this topology.\n* A clinical EEG for seizure monitoring might use a sparse montage of ~20 channels (e.g., the 10-20 system).\n* A cognitive neuroscience research study might use a high-density cap with 64, 128, or even 256 channels for higher spatial resolution.\n* This variation across datasets is what we call topological heterogeneity. It poses a massive barrier to building large-scale AI models. A standard neural network designed for a fixed input shape will fail to analyze this variable-sized input.\n\nOur goal in this work is to design a model that can tackle this topological heterogeneity problem.\n\n### Weakness-2: Topology Invariance Definition\n> The authors state that the Channel-Unification Module (Section 3.1) uses learned queries and cross-attention to project variable-channel features into a fixed-dimensional latent space, thereby achieving topology invariance. This appears to be a central contribution, yet the concept of topology invariance is neither clearly defined nor sufficiently explained. It remains unclear what topology invariance entails in this context, how the module achieves it, and why it is beneficial for addressing the stated challenges. \n\nWe thank the reviewer and apologize for the oversight. We will add a clear and explicit definition of topological invariance in the text. For reference, we define topology invariance as an architectural property of a model that allows it to process EEG signals from any montage, regardless of the number or specific layout of electrodes, without requiring any changes to the model's core structure. This means the core computational blocks of the model do not change, regardless of whether the input comes from a 20-channel or a 62-channel montage. The Channel-Unification Module achieves this by mapping variable-sized input to a fixed size, to be further processed by transformer blocks. \n\n### Weakness-3: Architecture Design\n> The Methodology section focuses on how LUNA is constructed but lacks discussion on the underlying design motivations. Without such explanation, the architecture appears ad hoc, which undermines the theoretical credibility of the approach.\n\nThe design for LUNA is not ad hoc but is motivated by principles from previous work about set processing to solve the topological heterogeneity problem. An EEG montage can be viewed as an unordered *set* of channels. Architectures like Set Transformer and PerceiverIO have shown that learned queries with cross-attention are a principled and effective way to summarize information from variable-sized sets into a fixed-size latent representation. We adapt this principle to EEG:\n\n*   **Channel-Unification Module:** At each time step (or patch), our module treats the `C` input channels as a set of `C` feature vectors.\n*   **Cross-Attention Mechanism:** We use a small, *fixed number (`Q`)* of learnable vectors, which we call \"queries.\" These `Q` queries \"attend\" to the `C` channel vectors via cross-attention.\n*   **Fixed-Size Output:** The output of this cross-attention operation is always a set of `Q` vectors, regardless of the initial number of channels `C`.\n*   **Decoupling:** All subsequent temporal processing operates *only* on this fixed-size (`Q`) representation. This effectively decouples the main computational body of the model from the input topology, thus achieving topology invariance.\n\nThis design choice directly solves the problem of heterogeneity and is the primary reason LUNA can be pre-trained on datasets with different montages (TUEG and Siena) and generalize to others. We will explicitly add this detailed motivation to the Methodology section.\n\n---\n\n### Weakness-4: State-of-the-Art Accuracy-Efficiency Trade-off\n\n> The claim of achieving a state-of-the-art accuracy-efficiency trade-off is questionable. This metric is only meaningful when the accuracy itself reaches an acceptable benchmark. The paper does not provide sufficient justification that this condition is met.\n\nWe appreciate the reviewer's skepticism and agree that efficiency gains are meaningless without strong performance. We believe our results show that LUNA's accuracy is not just \"acceptable\" but highly competitive and even state-of-the-art on several benchmarks. As shown in Table 2, LUNA-Huge achieves state-of-the-art results on both TUAR and TUSL. This outperforms all other compared methods, including larger and more complex models. On the TUAB benchmark (Table 1), LUNA's performance is highly competitive with models like LaBraM and CBraMod, falling short by only a small margin (~1-2% AUROC) while being vastly more efficient. \n\nThe efficiency gain is not marginal; it enables applications on high-density EEG (64-256 channels) or long recordings where quadratically scaling models are computationally prohibitive.\n\nTherefore, because LUNA achieves both SOTA results on some tasks and highly competitive results on others, we believe the claim of a state-of-the-art accuracy-efficiency trade-off is well-justified. We will re-emphasize this in our Results section.\n\n---\n\n### **Question-1: Overall Loss Function**\n\n> What is the overall loss function used in pre-training LUNA? While the paper introduces the masked reconstruction loss and the auxiliary query specialization loss separately, the formulation of the total loss function is missing.*\n\nThank you for pointing this out. We provided the query specialization loss coefficient in Table 5 in the Appendix, which is set to 0.8. We will move this to the main paper content to clarify any confusion. We will add the explicit formula to Section 3.3. The formula is:\n\n$L_{total} = L_{rec} + λ_{spec} * L_{spec}$\n\nHere, $L_{rec}$ is the Smooth L1 reconstruction loss, $L_{spec}$ is the query specialization loss, and `λ_spec` is the query specialization loss coefficient.\n\n---\n\n### **Question-2: Rationale for Patch Length**\n\n> The selection criteria and rationale for the patch length parameter should be discussed, as it likely plays a critical role in the model’s performance.*\n\nThis is an excellent point. The patch length is a critical hyperparameter to balance temporal resolution and the ability to capture relevant signal characteristics.\n\nOur methodology is built on a two-stage temporal hierarchy. First, we process local features within each patch, and second, we model the global context across the sequence of patches.\n*  Our input samples are 5 seconds long, sampled at 256 Hz. The choice of 40 timestamps per patch corresponds to a 156 ms window. This duration is neurophysiologically meaningful, as it is long enough to fully capture brain rhythms (e.g., a full alpha wave cycle at ~10 Hz is 100 ms) and characterize artifacts. This ensures each patch embedding is a rich, informative feature vector.\n* This patch length transforms each 5-second sample into a sequence of 32 tokens (1280 timestamps / 40). A sequence of this length is ideal for our temporal attention mechanism. It is long enough for the model to learn complex temporal dynamics across the full sample, yet short enough to remain computationally efficient and avoid the quadratic scaling costs that would come from a much longer sequence.\n\n### References:\n\nLee, Juho, et al. \"Set transformer: A framework for attention-based permutation-invariant neural networks.\" International conference on machine learning. PMLR, 2019.\n\nJaegle, Andrew, et al. \"Perceiver io: A general architecture for structured inputs & outputs.\" arXiv preprint arXiv:2107.14795 (2021)."}]

Author rebuttal:

Meta-review: This paper introduces a self-supervised foundation model for EEG data. The proposed model provides a direct architectural solution to a critical bottleneck in EEG analysis: the dual challenge of topological heterogeneity and computational complexity.

The reviewers acknowledged several strengths, including clear writing, a well-motivated solution, and a novel, intuitive architectural design. During the rebuttal, the authors addressed most of the concerns raised and committed to revising the paper accordingly. Although Reviewer jKJ9 did not actively participate in the discussion and only responded after the rebuttal, I believe the issues they raised were satisfactorily addressed in the author response. The other three reviewers expressed positive inclinations toward acceptance.

I encourage the authors to incorporate the reviewers’ suggestions in the final version to further strengthen the work.

Recommendation: Accept as poster.
Decision: Accept (poster)