Title: LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: Electroencephalography (EEG)

Biosignal Processing

Transformers

Self-Supervised Learning

Foundation Models

Representation Learning
Abstract: Electroencephalography (EEG) offers a non-invasive lens into human brain activity, but building large‐scale models is hampered by $\textit{topological heterogeneity}$: each public corpus defines its own electrode layout, limiting generalization. We introduce $\textbf{LUNA}$ ($\textbf{L}$atent $\textbf{U}$nified $\textbf{N}$etwork $\textbf{A}$rchitecture), a self-supervised foundation model that reconciles disparate electrode geometries while scaling linearly---not quadratically---with channel count. LUNA compresses multi-channel EEG into a fixed-size, topology-agnostic latent space via learned queries and cross-attention. Downstream transformer blocks then operate exclusively on this latent representation using patch-wise temporal self-attention, decoupling computation from electrode count. Pre-trained on TUEG and Siena ($\>$21,000 h raw EEG across diverse montages) using a masked-patch reconstruction objective, LUNA transfers effectively to four downstream tasks: abnormality detection, artifact rejection, slowing classification, and emotion recognition. It demonstrates highly competitive performance across several benchmarks, achieving state-of-the-art results on TUAR and TUSL, e.g., $\textbf{0.921 AUROC}$ on TUAR, while reducing FLOPs by $\textbf{300}$$\times$ and trimming GPU memory use by up to $\textbf{10}$$\times$. Critically,  these gains are consistent across all evaluated electrode configurations. Code is available at https://github.com/pulp-bio/biofoundation


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aim to address the performance degradation during cross-dataset evaluation caused by significant topological heterogeneity of EEG data. The author propose LUNA (Latent Unified Network Architecture) directly addresses this gap. Their key innovation is a topology-invariant encoder that maps arbitrary electrode layouts into a fixed latent space via learned queries and cross-attention. They pre-train LUNA using a masked-patch reconstruction objective on two datasets and fine-tune on four downstream benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The topic addressed in this paper is important and timely for the brain-machine domain.\n\nWeakness:\n\n1. The writing is often difficult to follow. A clearer explanation of the motivation and methodological framework would significantly improve readability and understanding.\n2. The authors state that the Channel-Unification Module (Section 3.1) uses learned queries and cross-attention to project variable-channel features into a fixed-dimensional latent space, thereby achieving topology invariance. This appears to be a central contribution, yet the concept of topology invariance is neither clearly defined nor sufficiently explained. It remains unclear what topology invariance entails in this context, how the module achieves it, and why it is beneficial for addressing the stated challenges.\n3. The *Methodology* section focuses on how LUNA is constructed but lacks discussion on the underlying design motivations. Without such explanation, the architecture appears ad hoc, which undermines the theoretical credibility of the approach.\n4. The claim of achieving a state-of-the-art accuracy-efficiency trade-off is questionable. This metric is only meaningful when the accuracy itself reaches an acceptable benchmark. The paper does not provide sufficient justification that this condition is met.\n\nQUESTIONS:\n1. The conceptual and technical issues listed in the *Weaknesses* section require clarification—particularly the definition and justification of topology invariance, and the motivation behind LUNA’s design.\n2. What is the overall loss function used in pre-training LUNA? While the paper introduces the masked reconstruction loss and the auxiliary query specialization loss separately, the formulation of the total loss function is missing.\n3. The selection criteria and rationale for the patch length parameter $P$ should be discussed, as it likely plays a critical role in the model’s performance.\n\nLIMITATIONS:\nThe *Methodology* section lacks a principled explanation of why LUNA was designed in its current form. This omission weakens the theoretical grounding of the proposed method and limits the paper's overall persuasiveness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on advancing the EEG foundation model by constructing the single, montage-agnostic architecture, and compared to the baselines, the expected advantage is to achieve a balance between efficiency and performance. The authors proposed the model named LUNA, which sets a bunch of learnable queries and uses the cross-attention module to map the patch-wise features into the fixed latent space, reducing the time complexity significantly compared to some state-of-the-art foundation model such as LaBraM and CBraMod while achieving reasonable performances on several benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n1. This proposed model mitigates the problem of scaling for the EEG foundation model, and is topology-agnostic, which means its format-compatibility.\n\n2. The proposed methods are straightforward and possibly helpful for the general EEG foundation training task. \n\n3. The writing is well structured and easy to follow. \n\n**Weakness**\n1. The model looks like a direct fusion of MMM and LaBraM / CBraMod, somehow limiting its own novelty. But I would admit that it's not a trivial attempt given the efforts to train the large EEG foundation model.\n\n2. My main concern lies in the performance. As is shown in the Table 1, the LaBraM and CBraMod achieved better performance even with less number of parameters. I wonder whether shrinking the large models can compensate their extra computational efforts. And Also, I wonder whether there is room for the proposed LUNA model to achieve SOTA after scaling up.\n\nTo add, in the first paragraph of Section 4.2, the authors claims that LUNA \"surpasses most self-supervised baselines and approaching large-scale models like LaBraM and CBraMod\", but in fact almost half of compared self-supervised baselines achieved comparable results. I suggest the authors rephrasing the statement and being more conservative. \n\n3. I'm also worried about the effectiveness of the Query Specialization Loss. While in Figure 4, the queries demonstrate different attended brain areas, losses shown in Figure 5 barely drop after the increase. I wonder whether removing the loss can yeild similar results.\n\nQUESTIONS:\n1. Figure 3, what if I directly apply t-SNE to the original dataset (or some statistics of the original dataset)? I suspect that the original data would demonstrate some kinds of clustering phenomenon without undergoes the encoding process of LUNA. \n\n2. Is there practical benefit to enlarge the input size? For most of the dataset, the number of channels is fixed and relatively small, and there is no evidence that enlarging the time window would bring in better results for prediction. In a certain sense, the proposed trade-off between performance and complexity may not worth it.\n\nLIMITATIONS:\nYes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a foundation model architecture for EEG brain data, which can ingest various “topologies”, or sensor layouts, and as a result, make the memory complexity of the attention layers of the subsequent transformer layers independent of the number of input sensors. This is achieved through a “channel unification” module, which uses learned query cross-attention to mix information from different sensors to a fixed size latent space. The model is pretrained on ~21k hours of clinical EEG data using a masked patch reconstruction task. The model is then finetuned on 4 downstream tasks and performance is compared to various baselines. While reported performance is sometimes lower than existing baselines, computational complexity can be significantly lower than baselines depending on the selected query size.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n* Quality: The experimental setup, as in other EEG foundation model papers, relies on relevant downstream tasks to evaluate the usefulness of the representations learned during pretraining. This overall setup is sound, as is the comparison of theoretical computational complexity of the different existing models.\n* Clarity: The paper is overall clearly written. The problem that is being solved is clearly exposed (varying electrode layouts and quadratic computational complexity), and the channel unification module is clearly explained. Most details necessary to reproduce the paper are described.\n* Significance: The results are likely to have some impact on the community, as EEG foundation models are still being actively developed, and solutions to make them sensor layout-agnostic and reduce their computational complexity may facilitate scaling and usability in downstream scenarios. \n* Originality: This work builds upon previous EEG foundation model work that are well cited and contextualized in the manuscript. However a relevant reference appears to be missing (see Weaknesses).\n\nWeaknesses\n\n* Quality: \n    * The ablation studies of Section 4.4 (learned queries vs. fixed regions, query specialization loss) suggest that the change in performance is very small (3e-3 to 7e-3). These changes are on the same order as the reported standard deviation over a few seeds, and seem to actually yield improved performance on one of the metrics (TUAR AUC-PR). Drawing conclusions from these experiments is therefore difficult, and I would suggest increasing downstream task coverage before confirming these observations.\n    * The layouts seen during pretraining are in fact very consistent, which limits the impact of the results (see Q2).\n* Clarity: a few points remained unclear from the manuscript:\n    * $A_{affinity}$ in the query specialization loss (line 172) is not defined.\n    * It is not clear whether “multiclass classification” (line 189) refers to a multiclass classification problem, or actually to a “multilabel classification” problem, i.e. where each label is treated as a separate classification problem.\n    * Preprocessing: it is not clear how the data can be converted to a bipolar montage (line 196).\n    * How are the FLOPs of Figure 2 computed?\n* Significance: Despite downstream performance being on a similar scale as existing EEG foundation models, the proposed approach underperforms similar-scale or smaller baseline models on three out of four downstream tasks. This limits the impact of the claims as it is not yet clear how important computational complexity constraints should be a limiting factor in EEG foundation model use cases (vs. performance).\n* Originality: \n    * The work of Saeed et al. (2020) is similar to the proposed approach, i.e. using learned query cross-attention to project input EEG channels to a fixed space (however without channel position information, and using a purely supervised learning task) and is tested on similar datasets. The differences with this work should be highlighted in the manuscript.\n    * I would have expected BIOT to be included in the analysis of computational complexity, as it uses a linear attention mechanism, and still outperforms a larger LUNA model on at least one downstream task. \n\nSaeed et al., 2020. Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering, http://arxiv.org/abs/2010.13694.\n\nQUESTIONS:\n1. How was $Q$ (the number of latent queries) selected? Intuitively, 4-8 seems like a very small number given that each query can be interpreted as a sort of (dynamic) spatial filter, and that there are potentially many more brain sources that may be relevant to the downstream tasks. I understand that this is in fact an important tradeoff, as the larger $Q$ becomes, the higher the computational complexity becomes (if this gets on the same order as the channel dimension, the computational complexity benefits may actually disappear). Exploring, or at least discussing, the tradeoff between $Q$ vs.$E$ would be useful to understand this point.\n2. The layouts seen during pretraining are in fact very consistent: only three montages were seen (20, 22, 29 channels). This, along with the lower performance on the SEED-V dataset with 62 channels, limits the impact of the study, as the appeal of such an approach is that very different montages can be handled in a unified manner. Exposing the model to more diverse channel configurations, e.g. by adding datasets with different montages to the pretraining set, or by randomly dropping out channels during training, could help generalization on new unseen montages. Also, testing on more varied electrode layouts would be important to support the conclusion (for instance, very sparse layouts for e.g. sleep data, and dense layouts from research-grade EEG systems).\n3. If I understand correctly, the decoder of Section 3.2 requires knowing the number of channels of the input. Does this mean a separate decoder is required for each montage/dataset, or can these be reused between datasets? Of note, $E_{dec}$ is used to refer to both the decoder queries and the output of the cross-attention at line 163.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a self-supervised foundation model designed to tackle the topological heterogeneity of EEG recordings (variations in channel configurations) during large-scale training. The method first segments each input sample into non-overlapping, single-channel patches and applies both temporal and frequency embeddings to these patches. A channel-unification module, built on a cross-attention mechanism, then compresses the variable channel sets into a fixed channel dimension. The unified representation is reshaped along the temporal axis and fed into a patch-wise temporal encoder. Self-supervised pre-training is performed through masked-patch reconstruction. The model is pre-trained on the TUEG and Siena Scalp EEG databases, and then fine-tuned on four downstream datasets, achieving state-of-the-art performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**: 1) The manuscript is clearly written and easy to follow. The authors articulate a compelling motivation: topological heterogeneity across EEG datasets hampers the training of large-scale foundation models. 2) The proposed embedding module jointly encodes temporal, spatial, and spectral information—three pillars of effective EEG representation learning. 3) The channel-unification module built on cross-attention is intuitive and, in principle, broadly applicable to heterogeneous-channel scenarios. 4) The cross-attention mechanism is again utilized during reconstruction, and the downstream task is simplified and effective, thereby avoiding the need for different projection heads for various downstream datasets. 4) The paper presents a thoughtful analysis of complexity. By introducing a two-stage attention scheme—first unifying channels, then applying temporal self-attention—the computational cost is reduced from $S \\times C$ to $S$, where $S$ is the number of patches per channel and $C$ the number of channels.\n\n**Weaknesses**: 1) For a fair evaluation, the experiments on TUSL and TUAR should adopt subject-independent splits rather than random sample-level splits. 2) In Algorithm 1 (line 148), should the tensor $Q_{\\text{learn}}$ have shape $\\mathbb{R}^{B \\times Q \\times E}$ to remain symmetric with $E_{\\text{dec}}$ on line 163 ($\\mathbb{R}^{B \\times C \\times E}$)? Please clarify if my understanding is incorrect.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nSee the conclusion part of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}