UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding

## Abstract

Abstract Decoding human brain activity from electroencephalography (EEG) signals is a central challenge at the intersection of neuroscience and artificial intelligence, enabling diverse applications in mental state assessment, clinical monitoring, and human–machine interaction. Recent efforts have extensively explored EEG-based brain foundation models for generalized brain decoding, employing large-scale training on multiple datasets. However, most of these attempts struggle with generalizability and fail to achieve satisfactory performance without task-specific tuning due to pronounced inherent heterogeneity among decoding tasks.
To address these challenges, we present UniMind , a general-purpose EEG foundation model for unified multi-task brain decoding by uniquely unleashing the power of LLMs to comprehend complex neural patterns.
UniMind enjoys several merits. First, we design a Neuro-Language Connector to bridge the modality gap between neural signals and LLMs, distilling and transforming the spatiotemporal neural patterns of EEG data into LLM-understandable representations.
Second, a Task-aware Query Selection module is proposed to inject task-awareness into the cross-modal alignment by dynamically generating task-adaptive query tokens, enabling the learning of task-relevant neural patterns across diverse tasks. Extensive experiments across 10 datasets demonstrate that UniMind substantially outperforms state-of-the-art multi-task decoding models ( 12% gain on average), while also offering valuable neuroscientific insights into neural functional correlations across tasks. The code will be made publicly available.

## 1 Introduction

Electroencephalography (EEG) is a widely adopted technique used to measure and record electrical activity in the brain, where electrodes are placed on the scalp to detect and amplify the brain’s electrical signals. EEG plays a crucial role in Brain-Computer Interfaces (BCI) by providing a non-invasive, real-time measure of brain activity, and also serves as a powerful tool for investigating brain’s perceptual mechanisms. By analyzing neural patterns in EEG signals, many studies have been conducted to decode specific brain states, demonstrating strong potential across diverse applications such as seizure classification intro_seizure, sleep stage classification intro_sleep, motor imagery recognition intro_motorimage, abnormality detection intro_abnormal, emotion analysis intro_emotion, and acute
stress detection intro_workload.

Figure: Figure 1: UniMind leverages LLMs to interpret brain signals, enabling multi-task EEG decoding without fine-tuning and demonstrating strong adaptability to task variability.
Refer to caption: x1.png

Despite the advances in EEG signal decoding, numerous deep learning models jing2023SPaRCNet; 1d-cnnforeegsignals; dar2020cnnPhysiological; yang2022atd; kim2020ecg; jing2020automated; ObstructiveSleepApnoea; yang2023Electroencephalogram; CNNTransformer; li2022FFCL; song2021transformer; liu2022spatialtemporaltransformerseegemotion have been confined to task-specific paradigms, due to the variations in EEG signal formats across different tasks. While effective for intended tasks, these models struggle to generalize to new tasks. Moreover, collecting task-specific EEG data is costly, making it impractical to build sufficient datasets for each task. These limitations highlight an urgent need for general-purpose EEG models that can learn transferable representations across tasks. In response, foundation models such as BIOT yang2023biot and LaBraM jiang2024Labram have been proposed to acquire universal perceptual capabilities of EEG signals through large-scale self-supervised pre-training. However, these models require additional fine-tuning for each downstream task to outperform task-specific models, thus still struggling with generalizability and causing extra computational overhead.

To tackle the aforementioned challenges, recently proposed NeuroLM jiang2025neurolm developed a multi-task EEG foundation model that integrates Large Language Models (LLMs) using adversarial domain-level alignment and multi-channel autoregression to bridge EEG and language modalities.
While NeuroLM demonstrates the potential of LLMs in multi-task EEG understanding, its performance remains unsatisfactory compared to single-task methods, underperforming
by more than 10% on datasets such as SEED zheng2015seed, TUEV harati2015tuab, and SleepEDF intro_sleep.
To unlock LLMs’ potential for decoding neural patterns and unifying diverse EEG tasks, two critical challenges need to be addressed:

(1) Huge modality gap between neural signals and language.
Unlike the rich, structured features of text tokens, EEG signals are distinguished by high noise,
sparse semantic information, and complex neural patterns that are difficult for LLMs to directly perceive. Alignment between EEG and language modalities is a prerequisite for LLMs to comprehend EEG signals and reason across modalities.
Existing methods use adversarial training to align EEG and text at the domain level, but the modality gap remains unresolved due to neglected semantic-level relations.
*The core issue remains how to conduct effective cross-modality bridging for alleviating the modality gap between EEG and LLMs.*

(2) Heterogeneity across EEG decoding tasks.
EEG tasks show significant heterogeneity nonstationary_nature due to varied configurations and cognitive mechanisms, resulting in diverse signal characteristics across datasets, such as electrode channels, trial durations, and spatiotemporal brain activity.
Given this, purely task-agnostic mixed-task tuning may lead to degraded decoding performance on specific tasks.
Therefore, a task-aware mechanism is urgently required for generalizable brain decoding in the multi-task setting.
Moreover, task-agnostic methods overlook neural correlations across tasks, limiting insights into brain cognition.
*How to develop a task-aware mechanism for achieving unified multi-task learning robust to EEG task heterogeneity without sacrificing individual task performance,* remains an open challenge.

To address these challenges, we introduce UniMind, a general-purpose brain foundation model for unified multi-task brain decoding by unlocking the potential of LLMs in understanding brain patterns, as shown in Figure [1](https://arxiv.org/html/2506.18962v1#S1.F1). To the best of our knowledge, it is the first multi-task EEG decoding model to match the performance of single-task approaches in one unified model.
To narrow the modality gap, we propose a Neuro-Language Connector (NLC) which acts as a compact, trainable bridge between the EEG encoder and the LLM, condensing essential brain patterns from sparse EEG data in a semantically meaningful way for the LLM to interpret.
Specifically, to leverage the spatio-temporal nature of EEG signals liu2022spatialtemporaltransformerseegemotion; spatial_temporal_featurefusion,
the neuro-language connector adopts a dual-branch architecture, with learnable query tokens that separately aggregate temporal dynamics and spatial dependencies from neural signals via cross-attention. The aggregated features are then aligned and mapped to the semantic space of a frozen LLM. By doing this, NLC transforms spatio-temporal neural patterns into interpretable features for LLMs, thereby enabling seamless neuro-language integration.
To facilitate effective multi-task learning across heterogeneous EEG tasks, we propose a Task-aware Query Selection Module (TQS) to generate task-adaptive query tokens. TQS maintains spatial and temporal query pools containing multiple query tokens. A router mechanism is used to dynamically look up task-relevant queries from query pools based on input features.
By allowing each task to adaptively choose its own queries, TQS promotes knowledge sharing and mutual enhancement across related tasks while mitigating interference from conflicting ones.
Moreover, through the task selection mechanism, we uncover how diverse tasks are functionally organized across the brain by examining inter-task correlations, shedding light on how the brain regulates different cognitive functions.
Our contribution can be summarized as follows:

- •
We propose UniMind, a general-purpose brain foundation model for unified multi-task brain decoding by integrating a spatio-temporal cross-modality bridging strategy between EEG and language, along with a task-aware mechanism.
- •
We propose a dual-branch neuro-language connector that encodes the temporal and spatial patterns of EEG signals into LLM-interpretable representations, along with a task-aware module that generates adaptive queries for task-relevant representations.
- •
Experiments demonstrate that UniMind outperforms the existing best multi-task decoding model by 12% on average and is the first to achieve comparable or even superior performance to single-task decoding models across various tasks.
- •
Insights from a neuroscientific perspective are provided via query visualizations, revealing shared neural mechanisms and offering empirical support for knowledge sharing in multi-task brain decoding.

## 2 Method

As illustrated in Figure [2](https://arxiv.org/html/2506.18962v1#S2.F2), UniMind empowers large language models to understand brain signals from heterogeneous tasks by integrating two modules. (1) The Neuro-Language Connector (NLC) (Section [2.1](https://arxiv.org/html/2506.18962v1#S2.SS1)) aims to bridge the modality gap between neural signals and language models by learning query tokens to interpret brain patterns.
(2) The Task-aware Query Selection Module (TQS)
(Section [2.2](https://arxiv.org/html/2506.18962v1#S2.SS2)) aims to enhance task adaptability by dynamically learning task-adaptive query tokens for feature selection across heterogeneous EEG tasks.

### 2.1 Neuro-Language Connector

Bridging the modality gap between EEG and language is the core issue in leveraging LLMs for multi-task brain decoding.
Unlike images and text, which have dense and well-structured data patterns, EEG signals differ significantly due to their sparse nature, low signal-to-noise ratio, and complex spatio-temporal neural patterns.
EEG signals are characterized by temporal dynamics and multichannel structures spatial_temporal_featurefusion; song2021transformer.
To leverage these neural properties, the neuro-language connector is designed to condense and interpret spatiotemporal neural characteristics from noisy EEG signals, aligning them with the LLM’s semantic space.
Next, we present the EEG encoder for extracting neural representations, followed by the design of our neuro-language connector.

EEG Encoder.
Given an EEG signal sample $X\in\mathbb{R}^{C\times S}$, where $C$ and $S$ respectively denote the number of electrode channels and temporal sampling points, a pre-trained EEG encoder is borrowed from LaBraM jiang2024Labram to transform EEG signals with varying channels and lengths into standard token sequences, which consists of a temporal encoder for patch embedding and a Transformer encoder. Specifically, each EEG channel is divided into non-overlapping patches using a sliding window of length $t$, resulting in $T=\left\lfloor\frac{S}{t}\right\rfloor$ patches per channel. Finally, the signal is encoded as a sequence of patch token embeddings, with a prepended class token, resulting in $\bm{E}\in\mathbb{R}^{(C\times T+1)\times D_{\text{e}}}$.

Neuro-Language Connector.
To bridge the gap between LLM and EEG signals with sparsity and low signal-to-noise ratio,
the neuro-language connector aggregates spatio-temporal features in a decoupled way via cross-attention.
Specifically, given the EEG embeddings $\bm{E}$ from the EEG encoder, the neuro-language connector designs two sets of learnable queries: temporal queries $Q_{t}\in\mathbb{R}^{n_{qt}\times D_{\text{e}}}$ and spatial queries $Q_{s}\in\mathbb{R}^{n_{qs}\times D_{\text{e}}}$, where $n_{qt}$ is the number of temporal queries and $n_{qs}$ is the number of spatial queries. These queries interact with EEG embeddings through separate cross-attention layers.
$Q_{t}$ captures temporal neural dynamics by attending to time-varying patterns within each channel, while $Q_{s}$ explores spatial dependencies by analyzing spatial-wise channel activation patterns at each time step.
We employ a cross-attention mechanism to discover and aggregate key brain patterns from noisy EEG signals, with the EEG token sequence serving as Keys and Values and learnable query tokens are Queries. Formally:

$$ $\displaystyle\bm{F}_{t}$ $\displaystyle=\text{CrossAttn}(Q_{t},\bm{E}_{t})\in\mathbb{R}^{n_{qt}\times C \times D_{\text{e}}},\quad\bm{F}_{s}=\text{CrossAttn}(Q_{s},\bm{E}_{s})\in \mathbb{R}^{n_{qs}\times T\times D_{\text{e}}}.$ (1) $$

Here, $\bm{E}_{t}\in\mathbb{R}^{C\times T\times D_{\text{e}}}$ and $\bm{E}_{s}\in\mathbb{R}^{T\times C\times D_{\text{e}}}$ are the reshaped EEG embeddings for temporal and spatial attention, respectively.
$\bm{F}_{t}$ and $\bm{F}_{s}$ are the temporally and spatially condensed EEG embeddings after attention-based interaction, which are then concatenated with the initial class token $\bm{F}_{\text{cls}}\in\mathbb{R}^{1\times D_{\text{e}}}$, resulting in the neuro-semantic embeddings $\bm{F}$:

$$ $\displaystyle\bm{F}=\text{Concat}(\bm{F}_{t},\bm{F}_{s},\bm{F}_{\text{cls}}) \in\mathbb{R}^{(n_{qt}\times C+n_{qs}\times T+1)\times D_{\text{e}}}.$ (2) $$

Finally, the neuro-semantic embeddings are projected to the embedding space of LLM to make them linguistically understandable to the LLM,
resulting in $\bm{F}^{\prime}\in\mathbb{R}^{(n_{qt}\times C+n_{qs}\times T+1)\times D_{\text
{L}}}$,
where $D_{\text{L}}$ denotes the LLM hidden size.
By transforming spatiotemporal neural patterns into semantically structured representations, neuro-language connector can enhance neuro-LLM alignment, empowering LLMs to interpret neural signals based to their underlying neurophysiological characteristics.

Figure: Figure 2: Overview of the UniMind architecture. Raw EEG signals are encoded into EEG embeddings $\bm{E}$, which are processed by the Task-aware Query Selector to extract task-adaptive queries. These queries are combined with static queries and jointly processed with $\bm{E}$ by the Neuro-Language Connector, which aligns spatio-temporal neural features with the LLM’s semantic space. The resulting embeddings, together with task prompts, guide the LLM to generate text output.
Refer to caption: x2.png

### 2.2 Task-aware Query Selection Module

Integrating multiple EEG tasks into one model is challenging due to heterogeneous signal characteristics across tasks.
The shared architecture struggles to manage task heterogeneity, leading to degraded performance on specific tasks and restricting mutual enhancement across related tasks in multi-task training.
To address this limitation, we introduce a Task-aware Query Selection Module (TQS) that integrates a task-aware mechanism to the cross-modality bridging process, dynamically learning query tokens for task-adaptive neural pattern mining and decoding.

To achieve dynamic query learning, TQS introduces a query pool $\bm{P}$ that contains a series of learnable query tokens $\bm{P}=[P_{1},P_{2},\cdots,P_{N_{q}}]$, and a query router $\bm{R}$ for query selection. Specifically, TQS contains two branches for respective temporal and spatial query learning, denoted as $\bm{P}_{t},\bm{R}_{t}$ and $\bm{P}_{s},\bm{R}_{s}$.
The query router $\bm{R}$ generates routing scores $S$ over the query tokens in $\bm{P}$ conditioned on the input EEG embedding $\bm{E}$.
The routing scores $S$ indicate how well each query fits to the input EEG features, allowing the model to generate task-adaptive queries tailored for the sample from the current task.
Subsequently, the task-adaptive queries $q_{a}$ are selected from $\bm{P}$ based on routing scores:

$$ $\displaystyle q_{a}^{i}=\text{TopK}(\bm{P}_{i},S=R_{i}(\bm{E})),\quad i\in\{t, s\},\quad q_{a}^{i}\in\mathbb{R}^{n_{i}\times D_{e}},$ (3) $$

where $n_{t}$ and $n_{s}$ denote the number of task-adaptive queries for the temporal ($t$) and spatial ($s$) modalities, respectively.
Notably,
we incorporate an additional static query token $q_{st}\in\mathbb{R}^{1\times D_{e}}$ besides $q_{a}$, which captures the stable characteristics of each task.
The final query token sequence can be formulated as: $Q_{i}=[q_{a}^{i};q_{st}^{i}]\in\mathbb{R}^{(n_{i}+1)\times D_{e}},i\in\{t,s\}$. The resulting queries $Q_{t}$ and $Q_{s}$ are then fed into the NLC to extract task-relevant temporal and spatial features from EEG signals.
TQS enables each task to select its own queries adaptively, fostering knowledge sharing and mutual enhancement across related tasks while reducing the interference risk from conflicting ones.

### 2.3 Multi-Task Decoding with LLMs

Multi-task instruction tuning dataset.
We adopt instruction tuning for training the multi-task foundation model. To this end, we construct an instruction tuning dataset specifically tailored for brain decoding,
comprising five diverse task types and containing a total of 929k samples.
To further enrich the instruction diversity, we design ten distinct prompts for each task type. For every sample, one of these prompts is randomly selected and paired as a task-specific instruction. Further details regarding the instruction templates are provided in the Appendices.

LLM Training.
We formulate the EEG decoding process as a conditional language modeling task using the instruction tuning dataset, where the model is expected to generate the correct label or description conditioned on EEG signals and the instruction. The LLM input sequence is formulated as the concatenation of neural-semantic representation $F^{\prime}$, a special separator token [SEP] and the instruction prompt $P$ describing the task:
$X=[F^{\prime},\text{[SEP]},P]$.
The model is trained to generate the ground truth label $Y$ by predicting the corresponding target token sequence $t_{a}$, and is optimized using a causal cross-entropy loss for LLM training:

$$ $\mathcal{L}=-\sum_{i=1}^{L}\log p(t_{a,i}\mid X,t_{a,<i}).$ (4) $$

Where $t_{a,i}$ is the predicted token and $t_{a,<i}$ are the preceding tokens.

## 3 Experiments

### 3.1 Experimental Setup

Datasets & Metrics.
To comprehensively evaluate UniMind, we adopt ten publicly available EEG datasets across five task domains: sleep stage classification (HMC alvarez2021hmc, SleepEDF kemp2000sleepedf, SHHS quan1997shhs), emotion recognition (SEED zheng2015seed, SEED-IV zheng2019seediv), clinical anomaly detection (TUAB harati2015tuab, TUEV harati2015tuab, TUSL von2017tusl), cognitive workload classification (Workload zyma2019workload), and motor imagery classification (SHU ma2022shu). All experiments follow the training, validation, and test splits to ensure fair and consistent evaluation. Detailed information for each dataset, additional preprocessing, and data split strategies are provided in the Appendices. Given the class imbalance in EEG datasets, we evaluate model performance using three primary metrics: Balanced Accuracy, which averages recall across classes and is robust to imbalance; Cohen’s Kappa, which measures agreement between predicted and true labels while accounting for chance; and Weighted F1, which balances precision and recall, weighted by class frequency.

Implementation Details. We utilize the EEG encoder from LaBraM jiang2024Labram and adopt InternLM2.5-7B cai2024internlm2 as the language model.
In NLC, each branch employs an 8-head cross-attention layer, followed by a two-layer MLP.
The query router $\bm{R}$ is implemented as a two-layer MLP, and the query pool size is set to 16. Static queries use 1 token for both spatial and temporal dimensions, while task-adaptive queries use 2 spatial tokens and 1 temporal token.
We discuss the choice of pool size and the number of queries in detail in  [3.3](https://arxiv.org/html/2506.18962v1#S3.SS3). During training, the parameters of the LLM are frozen to preserve its pre-trained language understanding, while the other components are trainable.
All experiments are conducted on a machine with eight NVIDIA A800 GPUs (80GB). Additional training details and computational resource analysis are provided in the Appendices.

Compared Methods. To ensure a comprehensive evaluation, we compare UniMind against both non-foundation and foundation model baselines. (1) Non-foundation models. We include state-of-the-art deep learning models such as SPaRCNet jing2023SPaRCNet, ContraWR yang2023Electroencephalogram, CNN-Transformer (CNN-Trans) CNNTransformer, FFCL li2022FFCL, and ST-Transformer (ST-Trans) song2021transformer. These models typically adopt supervised learning strategies that are specifically designed for individual tasks or datasets. (2) Foundation models. We consider NeuroLM jiang2025neurolm, BIOT yang2023biot, and LaBraM jiang2024Labram. NeuroLM is available in three parameter scales (B, L, XL), and we report the best-performing version for each dataset. For fair comparison, we follow NeuroLM’s data splits and evaluation metrics on SEED, HMC, TUEV, and TUSL, and directly report their published results as provided in NeuroLM. For the remaining datasets, we re-implement the baseline models using our own data splits and evaluation metrics.

**Table 1: Performance comparison of various models across different EEG datasets. We underline the best performance among multi-task models and bold the best overall.**
|  | Single-task | Multi-task |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Metrics | SPaRCNet | ContraWR | CNN-Trans | FFCL | ST-Trans | BIOT | LaBraM | NeuroLM | UniMind |
| SEED | B-Acc | 55.96 | 61.06 | 61.61 | 58.08 | 54.79 | 70.97 | 73.18 | 60.34 | 70.55 |
| F1-W | 55.85 | 61.37 | 61.50 | 57.43 | 55.05 | 71.34 | 73.54 | 60.63 | 70.98 |  |
| HMC | B-Acc | 47.56 | 42.42 | 65.73 | 44.27 | 25.59 | 68.62 | 72.86 | 67.37 | 75.27 |
| F1-W | 41.08 | 29.87 | 68.96 | 29.02 | 14.28 | 70.91 | 75.54 | 71.26 | 77.40 |  |
| TUAB | B-Acc | 78.69 | 80.17 | 79.53 | 78.19 | 79.66 | 79.59 | 81.40 | 79.69 | 81.76 |
| F1-W | 75.13 | 80.65 | 78.76 | 77.83 | 80.90 | 78.82 | 81.47 | 78.93 | 82.03 |  |
| TUEV | B-Acc | 41.61 | 43.84 | 40.87 | 39.79 | 39.84 | 52.81 | 64.09 | 46.79 | 63.19 |
| F1-W | 70.24 | 68.93 | 68.54 | 67.83 | 68.23 | 74.92 | 83.12 | 73.59 | 78.04 |  |
| Workload | B-Acc | 59.77 | 69.66 | 57.93 | 70.69 | 61.03 | 66.55 | 66.09 | 63.45 | 78.67 |
| F1-W | 54.60 | 69.33 | 58.63 | 72.54 | 60.67 | 51.66 | 64.72 | 66.57 | 78.65 |  |
| TUSL | B-Acc | 41.85 | 58.57 | 35.75 | 39.19 | 40.00 | 57.58 | 76.25 | 68.45 | 78.95 |
| F1-W | 35.00 | 54.58 | 22.35 | 21.20 | 37.93 | 23.94 | 76.14 | 68.39 | 75.40 |  |
| SEED-IV | B-Acc | 29.88 | 38.38 | 35.21 | 37.81 | 36.93 | 36.19 | 47.63 | 32.30 | 45.56 |
| F1-W | 32.05 | 40.21 | 36.57 | 39.76 | 36.95 | 42.76 | 49.14 | 34.65 | 43.58 |  |
| SleepEDF | B-Acc | 60.16 | 67.05 | 60.29 | 65.66 | 55.17 | 64.95 | 68.96 | 56.40 | 72.98 |
| F1-W | 58.61 | 66.92 | 58.96 | 64.79 | 53.18 | 60.91 | 87.30 | 54.02 | 88.23 |  |
| SHHS | B-Acc | 63.93 | 67.01 | 64.63 | 67.59 | 64.63 | 72.22 | 71.69 | 59.15 | 74.00 |
| F1-W | 61.82 | 56.80 | 63.78 | 67.07 | 63.30 | 83.96 | 82.90 | 63.54 | 84.20 |  |
| SHU | B-Acc | 62.15 | 62.13 | 56.25 | 62.82 | 63.39 | 59.16 | 67.90 | 59.36 | 65.77 |
| F1-W | 62.15 | 57.51 | 55.86 | 62.78 | 63.25 | 55.51 | 67.84 | 56.03 | 65.73 |  |
|  | Avarage | 47.71 | 52.95 | 49.94 | 50.28 | 46.33 | 57.29 | 66.56 | 55.20 | 67.50 |

### 3.2 Comparison with SOTA Methods

Table [1](https://arxiv.org/html/2506.18962v1#S3.T1) compares UniMind with state-of-the-art methods on ten brain decoding datasets. UniMind achieves the highest average performance, leading multi-task brain decoding.
Next we provide a comparative analysis between UniMind and both multi-task and single-task models. We present the results for Balanced Accuracy and Weighted F1 here, while the complete results, including full metrics and error analysis, are provided in the Appendices.

Comparison with multi-task models. We first compare our method with NeuroLM in the same multi-task setting. UniMind consistently outperforms NeuroLM across all datasets by a large margin, achieving an average performance gain of 12%.
This improvement stems from UniMind’s deep semantic alignment through the neuro-language connector, which enhances the ability of the LLM to interpret neural signals for multi-task decoding, unlocking its neural inference ability. Besides, the task-aware query selection module boosts the model’s task adaptability to heterogeneous decoding tasks.
Notably, our method achieves remarkable performance
on the challenging SEED-IV and TUEV datasets, substantially improving their initially low balanced accuracies (i.e., $\uparrow$ 13.26% and $\uparrow$16.4%). Another interesting phenomenon is that UniMind demonstrates a substantial improvement of 15.22% in balanced accuracy on the small-scale Workload dataset with only 1k samples.
These improvements stem from TQS’s task-adaptive query selection from a shared pool, which promotes knowledge transfer and mutual enhancement among tasks. This mechanism facilitates effective multi-task training, boosting performance on both complex and data-limited tasks.

Comparison with single-task models. We further compare UniMind with several state-of-the-art single-task models. It should be noted that multi-task learning is inherently more challenging. Therefore, this comparison is not entirely fair and is intended primarily as a reference for performance evaluation. UniMind achieves the best balanced accuracy among all single-task models on HMC, TUAB, SleepEDF, SHHS, Workload, and TUSL, and also obtains the best average score across all datasets. On the remaining datasets SEED, SEED-IV, TUEV, and SHU, it ranks second only to LaBraM, with a marginal performance gap. These results demonstrate that UniMind is capable of matching or even surpassing leading single-task baselines in a unified model. The effective brain-language alignment reduces the performance gap compared to task-specific tuning. Besides, the knowledge sharing facilitated by the task-aware query selector allows each dataset to benefit from semantically similar datasets, a capability that single-task models do not possess.

### 3.3 Ablation Study

Table: Table 2: Ablation of Model Components (Balanced Accuracy).

Table: Table 3: Ablation of Using Static and Task-adaptive Queries (Balanced Accuracy).

Table: Table 4: The Impact of Number of Task-adaptive Queries (Balanced Accuracy).

Analysis of Key Model Components.
We conduct an ablation study to evaluate the contributions of key model components, as shown in Table [2](https://arxiv.org/html/2506.18962v1#S3.T2). The baseline model employs a fixed number of learnable queries to extract features from the entire EEG sequence. Introducing the NLC module results in substantial performance gains across all datasets, with improvements around 5% on more challenging tasks such as SEED and TUEV. This is due to the NLC’s ability to capture temporal and spatial patterns in EEG embeddings and convert them into semantic representations that improve neuro-LLM alignment.
The addition of the TQS further enhances performance, yielding gains of 3.40% on TUEV, 2.82% on SHHS, and 8.01% on Workload. These improvements stem from the task-aware mechanism that enables adaptive query learning across tasks, promoting knowledge sharing among related tasks while reducing inter-task interference in cross-modality bridging.

Comparison of Different Types of Queries. We compare different query types used in the NLC module to evaluate the impact of static and task-adaptive query selection, as shown in Table [3](https://arxiv.org/html/2506.18962v1#S3.T3).
Compared to using only static queries, employing task-adaptive queries selected by the TQS module yields notable improvements on most datasets, with gains of 3.04% on TUEV and 3.34% on Workload. This improvement is primarily due to the ability of each task to adaptively select queries that better capture task-relevant temporal and spatial features.
Furthermore, combining static and task-adaptive queries yields the best overall performance across all datasets.

Analysis of the Number of Task-adaptive Queries. We analyze the impact of the number of dynamic queries in Table [4](https://arxiv.org/html/2506.18962v1#S3.T4), where $n_{s}$ and $n_{t}$ denote the numbers of task-adaptive queries $q_{a}^{s}$ and $q_{a}^{t}$. The results show that increasing the number of queries does not always improve performance. Too few queries limit the model’s capacity to capture diverse EEG patterns, while too many queries may introduce noise because some queries may not align well with the EEG embeddings.
While the optimal setup varies by dataset, the best overall results occur with $n_{s}=2$ and $n_{t}=1$.

Analysis of Query Pool Size.
We study the effect of the query pool sizes $\bm{P}_{t}$ and $\bm{P}_{s}$ (of equal size, denoted as $n_{q}$) in Figure [3](https://arxiv.org/html/2506.18962v1#S3.F3).
When the pool is too small, task-specific selections tend to overlap, which can lead to interference between tasks. Conversely, an overly large pool reduces query overlap among similar tasks, weakening the mutual enhancement that could occur between related tasks. Setting the pool size to 16 yields the best overall performance by balancing diversity and shared representation.

Figure: Figure 3: Configuration of Query Pool Size $n_{q}$.
Refer to caption: x3.png

Figure: Figure 4: Ablation Study on Model Parameter Size.
Refer to caption: x4.png

Analysis of Model Parameter Size. Figure [4](https://arxiv.org/html/2506.18962v1#S3.F4) investigates the impact of different model sizes on performance. We compare InternLM models with approximately 0.5B, 1.8B, and 7B parameters. We find that models with more parameters tend to bring consistent improvements on more challenging datasets such as SEED and TUEV, likely due to their stronger capacity to model complex patterns. In contrast, the performance of TUSL degrades as model size increases. This may be attributed to the limited amount of training data (245 samples), which makes larger models more prone to overfitting.

### 3.4 Task-level Query Analysis

In this section, we explore task-level query selection patterns and inter-task correlations
by query visualization, shedding light on underlying neural functional mechanisms. We further investigate the mutual enhancement effect, examining whether functionally similar tasks benefit from joint training.

Visualization of Task-adaptive Query Selection. To investigate the working mechanism of the Task-adaptive Query Selection module, we visualize the spatial and channel routing scores $S$ from query routers $\bm{R}$ across EEG datasets using t-SNE. The query scores reflect the query selection pattern in generating task-adaptive queries.
As shown in Figure [5](https://arxiv.org/html/2506.18962v1#S3.F5) (a), the neural routing distribution presents a clear task-aware pattern for both temporal and spatial routers, with samples from different datasets forming clearly separated clusters. The well-separated distribution suggests that TQS is capable of dynamically selecting queries for task-adaptive neuro-LLM alignment across heterogeneous tasks.

Furthermore, we analyze the selection frequencies of task-adaptive spatial queries across datasets to reveal each task’s preferences, as illustrated in Figure [5](https://arxiv.org/html/2506.18962v1#S3.F5) (b). The results indicate that query selection exhibits clear task-dependent trends. For example, emotion recognition and clinical event detection tasks (SEED, SEED-IV, TUAB, TUEV) tend to select queries $P_{4}$ and $P_{10}$, while sleep stage classification tasks (SHHS, HMC) consistently favor $P_{5}$. These findings suggest that the queries in $\bm{P}$ effectively capture diverse underlying neural characteristics across brain regions.

Neural Correlations across Tasks.
To capture underlying neural correlations across tasks, we measure similarities in query selection frequencies among EEG tasks to reflect neural patterns. Here we analyze the spatial query tokens.
As illustrated in Figure [6](https://arxiv.org/html/2506.18962v1#S3.F6), we highlight two key observations. (1) Datasets/tasks belonging to the same task domain
tend to exhibit much higher similarity scores. For example, TUAB and TUEV (clinical), SEED and SEED-IV (emotion), as well as SHHS and HMC (sleep), show remarkably consistent selection patterns. These findings indicate that TQS can help capture consistent neural activation patterns underlying neural signals from similar tasks.
(2) Certain tasks from different domains exhibit strong similarities, suggesting shared neural mechanisms across cognitive functions. For example, clinical anomaly detection tasks (TUAB, TUEV) and emotion recognition tasks (SEED, SEED-IV) show similar spatial patterns, due to overlapping activity in the bilateral temporal regions zheng2015seed; SeizureChannel, particularly around electrodes T7 and T8.
Similarly, sleep staging and cognitive workload tasks demonstrate over 75% similarity in spatial query usage, due to converging neural activity in central and frontal regions moctezuma2024sleepchannel; cognitiveload_channel.
These findings reveal underlying neural correlations between tasks, which is consistent with existing neuroscientific research, implying the potential of shared representations for multi-task learning.

Mutual Enhancement in Multi-task Training.
We further validate the mutual enhancement effect in multi-task training by
selecting six pairs of datasets with high similarity scores.
As shown in Figure [7](https://arxiv.org/html/2506.18962v1#S3.F7),
joint training consistently leads to better performance than training each task separately. For epilepsy-related tasks, TUEV gains 5.64% with TUAB, SEED-IV improves 5.54% with SEED, workload task benefits 2.66% from sleep data, and TUEV improves 5.74% when paired with SEED.
The results demonstrate that UniMind facilitates effective multi-task training, enhancing the performance across diverse decoding tasks by sharing neural representations. The improvement highlights the potential of multi-task foundation models in advancing generalized brain decoding.

Figure: Figure 5: (a): t-SNE-based visualization of neural routing distributions across datasets; (b): task-adaptive spatial query distributions across datasets.
Refer to caption: x5.png

Figure: Figure 6: (a): Similarity of task-adaptive query distributions across tasks; (b): The attention values of task-adaptive queries across different channels; (c): Topography visualization on different tasks.
Refer to caption: x6.png

Figure: Figure 7: Comparison of individual and joint training balanced accuracy across multiple tasks.
Refer to caption: x7.png

## 4 Conclusion

In this paper, we present UniMind, a general-purpose EEG foundation model for multi-task brain decoding. It leverages a Neuro-Language Connector to align spatiotemporal EEG features with LLMs and a Task-aware Query Selection Module to adapt to diverse tasks. Experiments on ten datasets show UniMind outperforms prior models. We believe this work lays a solid foundation for future research on LLM–brain interaction and multi-task learning in the EEG domain.

Limitations and Future Work. This work does not investigate the model’s generalization ability to completely unseen subjects and novel tasks, which may limit its ability to wider applications. Future efforts will focus on conducting cross-subject experiments in zero-shot or few-shot settings and integrating a broader range of EEG tasks to enhance the model’s robustness and generalizability.

## References

- [1]
Poomipat Boonyakitanont, Apiwat Lek-Uthai, Krisnachai Chomtho, and Jitkomut Songsiri.
A review of feature extraction and performance evaluation in epileptic seizure detection using eeg.
Biomedical Signal Processing and Control, 57:101702, 2020.
- [2]
Khald Ali I Aboalayon, Miad Faezipour, Wafaa S Almuhammadi, and Saeid Moslehpour.
Sleep stage classification using eeg signal analysis: A comprehensive survey and new investigation.
Entropy, 18(9):272, 2016.
- [3]
Syed Umar Amin, Mansour Alsulaiman, Ghulam Muhammad, Mohamed Amine Mekhtiche, and M Shamim Hossain.
Deep learning for eeg motor imagery classification based on multi-layer cnns feature fusion.
Future Generation Computer Systems, 101:542–554, 2019.
- [4]
Subhrajit Roy, Isabell Kiral-Kornek, and Stefan Harrer.
Chrononet: A deep recurrent neural network for abnormal eeg identification.
In International Conference on Artificial Intelligence in Medicine, volume 11526 of Lecture Notes in Computer Science, pages 47–56. Springer, 2019.
- [5]
Nazmi Sofian Suhaimi, James Mountstephens, and Jason Teo.
Eeg-based emotion recognition: A state-of-the-art review of current trends and opportunities.
Computational Intelligence and Neuroscience, 2020.
- [6]
Lakhan Dev Sharma, Vijay Kumar Bohat, Maria Habib, Al-Zoubi Ala’M, Hossam Faris, and Ibrahim Aljarah.
Evolutionary inspired approach for mental stress detection using eeg signal.
Expert Systems with Applications, 197:116634, 2022.
- [7]
Jin Jing, Wendong Ge, Shenda Hong, Marta Bento Fernandes, Zhen Lin, et al.
Development of expert-level classification of seizures and rhythmic and periodic patterns during eeg interpretation.
Neurology, 100(17):e1750–e1762, 2023.
- [8]
P. Nagabushanam, S. Thomas George, Praharsha Davu, P. Bincy, Meghana Naidu, and S. Radha.
Artifact removal using elliptic filter and classification using 1d-cnn for eeg signals.
In International Conference on Advanced Computing and Communication Systems (ICACCS), pages 551–556, 2020.
- [9]
Muhammad Najam Dar, Muhammad Usman Akram, Sajid Gul Khawaja, and Amit N.
Cnn and lstm-based emotion charting using physiological signals.
Sensors, 20(16):4551, 2020.
- [10]
Chaoqi Yang, Cheng Qian, Navjot Singh, Cao Xiao, M Brandon Westover, Edgar Solomonik, and Jimeng Sun.
Atd: Augmenting cp tensor decomposition by self supervision.
Advances in Neural Information Processing Systems, 2022.
- [11]
Min-Gu Kim, Hoon Ko, and Sung Bum Pan.
A study on user recognition using 2d ecg based on ensemble of deep convolutional neural networks.
Journal of Ambient Intelligence and Humanized Computing, 11:1859–1867, 2020.
- [12]
Jin Jing, Haoqi Sun, Jennifer A Kim, et al.
Development of expert-level automated detection of epileptiform discharges during electroencephalogram interpretation.
JAMA Neurology, 77(1):103–108, 2020.
- [13]
Haifa Almutairi, Ghulam Mubashar Hassan, and Amitava Datta.
Detection of obstructive sleep apnoea by ecg signals using deep learning architectures.
In European Signal Processing Conference, pages 1382–1386, 2021.
- [14]
Chaoqi Yang, Cao Xiao, M. Brandon Westover, Jimeng Sun, et al.
Self-supervised electroencephalogram representation learning for automatic sleep staging: Model development and evaluation study.
JMIR AI, 2(1):e46769, 2023.
- [15]
Wei Yan Peh, Yuanyuan Yao, and Justin Dauwels.
Transformer convolutional neural networks for automated artifact detection in scalp eeg.
In International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 3599–3602, 2022.
- [16]
Hongli Li, Man Ding, Ronghua Zhang, and Chunbo Xiu.
Motor imagery eeg classification algorithm based on cnn-lstm feature fusion network.
Biomedical Signal Processing and Control, 72:103342, 2022.
- [17]
Yonghao Song, Xueyu Jia, Lie Yang, and Longhan Xie.
Transformer-based spatial-temporal feature learning for eeg decoding.
arXiv preprint arXiv:2106.11170, 2021.
- [18]
Jiyao Liu, Hao Wu, Li Zhang, and Yanxi Zhao.
Spatial-temporal transformers for eeg emotion recognition.
In Proceedings of the 6th International Conference on Advances in Artificial Intelligence, pages 116–120, 2022.
- [19]
Chaoqi Yang, M Brandon Westover, and Jimeng Sun.
Biot: Biosignal transformer for cross-data learning in the wild.
In Advances in Neural Information Processing Systems, 2023.
- [20]
Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu.
Large brain model for learning generic representations with tremendous EEG data in BCI.
In International Conference on Learning Representations, 2024.
- [21]
Wei-Bang Jiang, Yansen Wang, Bao-Liang Lu, and Dongsheng Li.
NeuroLM: A universal multi-task foundation model for bridging the gap between language and EEG signals.
In International Conference on Learning Representations, 2025.
- [22]
Wei-Long Zheng and Bao-Liang Lu.
Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks.
IEEE Transactions on Autonomous Mental Development, 7(3):162–175, 2015.
- [23]
A. Harati, M. Golmohammadi, S. Lopez, I. Obeid, and J. Picone.
Improved eeg event classification using differential energy.
In IEEE Signal Processing in Medicine and Biology Symposium (SPMB), pages 1–3. IEEE, 2015.
- [24]
Alexander Ya. Kaplan, Andrew A. Fingelkurts, Alexander A. Fingelkurts, Sergei V. Borisov, and Boris S. Darkhovsky.
Nonstationary nature of the brain activity as revealed by eeg/meg: Methodological, practical and conceptual challenges.
Signal Processing, 85(11):2190–2212, 2005.
Neuronal Coordination in the Brain: A Signal Processing Perspective.
- [25]
Zhe Wang, Yongxiong Wang, Jiapeng Zhang, Chuanfei Hu, Zhong Yin, and Yu Song.
Spatial-temporal feature fusion neural network for eeg-based emotion recognition.
IEEE Transactions on Instrumentation and Measurement, 71:1–12, 2022.
- [26]
Diego Alvarez-Estevez and Rene Rijsman.
The helsinki university sleep corpus (hmc): A benchmark dataset for sleep staging algorithms.
PhysioNet, 2021.
- [27]
Bob Kemp, Aeilko H Zwinderman, Bert Tuk, H A C Kamphuisen, and J J L Oberyé.
Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg.
IEEE Transactions on Biomedical Engineering, 47(9):1185–1194, 2000.
- [28]
Stuart F Quan, Barbara V Howard, Conrad Iber, James P Kiley, F Javier Nieto, George T O’Connor, David M Rapoport, Susan Redline, John Robbins, Jonathan M Samet, and Peter W Wahl.
The sleep heart health study: design, rationale, and methods.
Sleep, 20(12):1077–1085, 1997.
- [29]
Wei-Long Zheng, Wei Liu, Yifei Lu, Bao-Liang Lu, and Andrzej Cichocki.
Emotionmeter: A multimodal framework for recognizing human emotions.
IEEE Transactions on Cybernetics, 49(3):1110–1122, 2019.
- [30]
Emily von Weltin, Thomas Voorhis, Yijun Cui, Vinay Shah, Xiaoxiao Jiang, Yanshan Li, Yuxuan Yang, Mohammad Golmohammadi, Iyad Obeid, and Joseph Picone.
A public eeg database for the evaluation of eeg abnormality detection algorithms.
Clinical Neurophysiology, 128(8):1524–1532, 2017.
- [31]
Iryna Zyma, Serhii Tukaev, Andrii Seleznov, Andrii Karpov, Oleksandr Tkachenko, and Radek Martinek.
Eeg-based mental workload estimation with data fusion and transfer learning.
Frontiers in Neuroscience, 13:702, 2019.
- [32]
Jianqun Ma, Banghua Yang, Wenhua Qiu, Fenqi Rong, Xueyuan Zhang, Yijun Liu, and Haibo Lu.
A large eeg dataset for studying cross-session variability in motor imagery brain–computer interface.
Scientific Data, 9(1):531, 2022.
- [33]
Zheng Cai, Maosong Cao, Haojiong Chen, and et al.
Internlm2 technical report, 2024.
- [34]
Rihat Rahman, Shiva Maleki Varnosfaderani, Omar Makke, Nabil J. Sarhan, Eishi Asano, Aimee Luat, and Mohammad Alhawari.
Comprehensive analysis of eeg datasets for epileptic seizure prediction.
In IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–5, 2021.
- [35]
L.A. Moctezuma, Y. Suzuki, J. Furuki, et al.
Gru-powered sleep stage classification with permutation-based eeg channel selection.
Scientific Reports, 14:17952, 2024.
- [36]
Faisal M. Alessa, Mohammed H. Alhaag, Ibrahim M. Al-harkan, Mohamed Z. Ramadan, and Fahad M. Alqahtani.
A neurophysiological evaluation of cognitive load during augmented reality interactions in various industrial maintenance and assembly tasks.
Sensors, 23(18), 2023.
- [37]
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al.
Flamingo: a visual language model for few-shot learning.
Advances in Neural Information Processing Systems, 35:23716–23736, 2022.
- [38]
Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al.
Openflamingo: An open-source framework for training large autoregressive vision-language models.
arXiv preprint arXiv:2308.01390, 2023.
- [39]
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu.
Mimic-it: Multi-modal in-context instruction tuning.
arXiv preprint arXiv:2306.05425, 2023.
- [40]
Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao.
Llama-adapter: Efficient fine-tuning of language models with zero-init attention.
arXiv preprint arXiv:2303.16199, 2023.
- [41]
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi.
Instructblip: Towards general-purpose vision-language models with instruction tuning.
arXiv preprint arXiv: 2305.06500, 2023.
- [42]
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.
Visual instruction tuning.
arXiv preprint arXiv: 2304.08485, 2023.
- [43]
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny.
Minigpt-4: Enhancing vision-language understanding with advanced large language models.
arXiv preprint arXiv:2304.10592, 2023.
- [44]
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao.
Videochat: Chat-centric video understanding.
arXiv preprint arXiv:2305.06355, 2024.
- [45]
Hang Zhang, Xin Li, and Lidong Bing.
Video-llama: An instruction-tuned audio-visual language model for video understanding.
arXiv preprint arXiv:2306.02858, 2023.
- [46]
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan.
Video-chatgpt: Towards detailed video understanding via large vision and language models.
arXiv preprint arXiv:2306.05424, 2024.
- [47]
Ruipu Luo, Ziwang Zhao, Min Yang, Junwei Dong, Da Li, Pengcheng Lu, Tao Wang, Linmei Hu, Minghui Qiu, and Zhongyu Wei.
Valley: Video assistant with large language model enhanced ability.
arXiv preprint arXiv:2306.07207, 2023.
- [48]
Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, et al.
Wavchat: A survey of spoken dialogue models.
arXiv preprint arXiv:2411.13577, 2024.
- [49]
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing.
Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.
arXiv preprint arXiv:2406.07476, 2024.
- [50]
Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai.
Pandagpt: One model to instruction-follow them all.
arXiv preprint arXiv:2305.16355, 2023.
- [51]
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua.
NExT-GPT: Any-to-any multimodal LLM.
In International Conference on Machine Learning, pages 53366–53397, 2024.

## Appendix A Related Works

Task-Specific EEG Decoding Models.
Due to the variations in EEG signal formats across different datasets, numerous deep learning models have been proposed to tackle the aforementioned tasks within their respective domains. These models primarily focus on designing tailored feature extraction networks to accommodate EEG samples for specific tasks. Some works jing2023SPaRCNet; 1d-cnnforeegsignals; dar2020cnnPhysiological apply 1D convolutional neural networks (CNNs) directly to raw signals, while others yang2022atd; kim2020ecg; kim2020ecg first preprocess the data using short-time Fourier transform (STFT) and then adopt 2D CNNs on the resulting spectrograms. However, these methods offer a relatively narrow scope of perception, focusing mainly on local information. To address this, some methods jing2020automated; ObstructiveSleepApnoea; yang2023Electroencephalogram; li2022FFCL build on CNN-based encoders by incorporating Transformers, LSTMs, or recurrent neural networks (RNNs) to model temporal dynamics and capture global dependencies.
To further enhance EEG feature representation, some approaches employ multiple encoders for ensemble learning li2022FFCL, or leverage multi-level Transformers to encode spatial and temporal features both across and within channelssong2021transformer; liu2022spatialtemporaltransformerseegemotion. However, these task-specific models often suffer from limited generalization and poor transferability, restricting their applicability to broader EEG analysis tasks.

EEG Foundation Model.
Because task-specific models can only handle a single task and lack cross-task learning capabilities, their broader applicability is limited.
As a result, EEG foundation models have attracted increasing attention. These models aim to learn universal EEG representations by training on diverse datasets, enabling generalization across various downstream tasks.
To address the challenges posed by heterogeneous EEG formats, Yang et al. yang2023biot proposed BIOT, which tokenizes EEG channels into fixed-length segments and incorporates channel and relative position embeddings to preserve spatio-temporal features.
Inspired by masked language modeling in LLMs, Jiang et al. jiang2024Labram extended this idea with LaBraM, introducing a neural tokenizer and pre-training it via masked neural code prediction on large-scale EEG data, achieving SOTA results across multiple tasks.
While these models effectively address many challenges, they still require separate fine-tuning for each downstream task. NeuroLM jiang2025neurolm was introduced as the first multi-task foundation model for EEG, leveraging the capabilities of LLMs to support multi-task learning and inference. However, its performance is limited by the modality gap between EEG and language, as well as interference among tasks. In contrast, UniMind bridges this gap through a spatio-temporal cross-modality alignment strategy and reduces task interference via task-aware query selection. This design allows UniMind to achieve stronger performance across diverse tasks while supporting unified multi-task decoding within a single model.

Multimodal Large Language Models.
Building on the success of Large Language Models (LLMs), the Multimodal Large Language Models (MLLMs) are developed to enhance cross-modal understanding by combining visual, auditory, and textual data. Some apporaches alayrac2022flamingo; awadalla2023OpenFlamingo; li2023MIMIC-IT; zhang2024LLaMA-Adapter enhance LLMs by incorporating components like gated cross-attention layers or adapter layers to handle multimodal inputs. Others dai2023InstructBLIP; liu2023LLaVA; zhu2023MiniGPT-4 use projection layers or Q-Formers to map visual encoder outputs into LLM input space. Video LLMs li2024VideoChat; zhang2023Video-LLaMA; maaz2024Video-ChatGPT; luo2023Valley; ji2024wavchat extend MLLMs for video tasks, primarily using projection layers or Q-Formers to process visual tokens. Besides, recent MLLMs have successfully integrated modalities such as audio cheng2024VideoLLaMA2; su2023pandagpt; wu2024NextGPT, applying them to EEG signals remains particularly challenging. While both audio and EEG exhibit continuous temporal patterns, EEG is inherently more complex due to its non-stationary characteristics across both temporal and spatial dimensions. It captures fine-grained neural activity that varies over time and across different brain regions, introducing significant variability and noise.
Unlike text, which is composed of discrete and structured tokens, EEG signals are fluid, high-dimensional, and constantly changing.
This makes it difficult for LLMs to directly process and align with EEG representations, posing a major challenge for effective cross-modal integration.
Building on these insights, UniMind proposes the Neuro-Language Connector, a crucial module that bridges the substantial modality gap between continuous, complex EEG signals and the discrete token representations of LLMs. By encoding spatiotemporal neural patterns into interpretable features, it enables more effective cross-modal alignment, paving the way for improved integration of EEG within multimodal language models.

## Appendix B Dataset Details

Dataset information.
To comprehensively evaluate UniMind, we utilize ten publicly available EEG datasets covering five representative task domains.
(1) For sleep stage classification, we employ HMC alvarez2021hmc, SleepEDF kemp2000sleepedf, and SHHS quan1997shhs. These datasets provide long-term polysomnographic EEG recordings annotated by experts, with each EEG segment labeled into one of five standard sleep stages: Wake, NREM-1, NREM-2, NREM-3, and REM.
(2) For emotion recognition, we adopt SEED zheng2015seed and SEED-IV zheng2019seediv, both containing multi-session EEG data collected while subjects viewed emotional video stimuli. SEED includes three emotion categories (positive, negative, neutral), whereas SEED-IV expands this to four categories: neutral, sad, fear, and happy.
(3) For Clinical Abnormaly Detection, we use TUAB harati2015tuab, TUEV harati2015tuab, and TUSL von2017tusl. TUAB focuses on detecting abnormal EEG activity, TUEV classifies six types of clinical events such as epileptiform discharges and eye movements, and TUSL categorizes seizure, slowing, and complex background activity.
(4) For cognitive workload classification, we utilize the Workload dataset zyma2019workload, which contains EEG recordings collected during mental arithmetic tasks designed to distinguish between high and low cognitive load.
(5) Finally, for motor imagery classification, we incorporate SHU ma2022shu, a large-scale dataset featuring EEG signals corresponding to imagined left- and right-hand movements, enabling binary decoding of motor intentions in brain–computer interface applications.

**Table 5: Overview of Downstream EEG datasets and tasks used for evaluation.**
| Task | Dataset | Rate (Hz) | #Channels | Duration | #Samples | #Subjects | Label |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Motor Imagery | SHU ma2022shu | 250 | 32 | 4s | 11,988 | 25 | Binary-class |
| Emotion Recognition | SEED zheng2015seed | 1000 | 62 | 1s | 144,851 | 15 | 3-class |
| SEED-IV zheng2019seediv | 1000 | 62 | 1s | 143,610 | 16 | 5-class |  |
| Clinical Abnormaly Detection | TUAB harati2015tuab | 250/256/512 | 23 | 10s | 409,083 | 2,383 | Binary-class |
| TUEV harati2015tuab | 250 | 23 | 5s | 112,237 | 370 | 6-class |  |
| TUSL von2017tusl | 250 | 23 | 10s | 245 | 28 | 3-class |  |
| Sleep Classification | SHHS quan1997shhs | 125 | 1 | 30s | 324,854 | 329 | 5-class |
| SleepEDF kemp2000sleepedf | 100 | 1 | 30s | 414,961 | 78 | 5-class |  |
|  | HMC alvarez2021hmc | 256 | 4 | 30s | 137,243 | 151 | 4-class |
| Workload Classification | Workload zyma2019workload | 500 | 19 | 4s | 1080 | 36 | Binary-class |

Data Splits. Each dataset is divided into training, validation, and test sets according to task-specific strategies. (1) TUAB and TUEV: we follow the official training and test split, and further divide the training subjects into 80% for training and 20% for validation. (2) SEED and SEED-IV: trials are split chronologically. SEED contains 15 trials, divided into 9 for training, 3 for validation, and 3 for testing; SEED-IV contains 24 trials, divided into 16, 4, and 4 respectively. All sessions within each split are merged. (3) HMC: subjects 1 to 100 are assigned to the training set, 101 to 126 to the validation set, and 127 to 154 to the test set. (4) Workload: subjects 0 to 25 are used for training, 26 to 30 for validation, and 31 to 35 for testing. (5) TUSL: the dataset is randomly divided into 60% for training, 20% for validation, and 20% for testing. (6) SleepEDF and SHHS: subjects are randomly assigned to training, validation, and test sets in an 8:1:1 ratio. (7) SHU: sessions are split into training, validation, and test sets in a 3:1:1 ratio.

Data Preprocessing.
We develop a standardized data pre-processing pipeline comprising band-pass filtering, notch filtering, resampling, and normalization, designed to be universally applicable across various BCI tasks and decoding models. Following established practice, we apply a 0.1–75 Hz band-pass filter to retain task-relevant frequency components while suppressing low-frequency drifts and high-frequency noise.To remove power-line interference, we first perform a Fast Fourier Transform (FFT) on the raw EEG data to identify the specific interference frequency, and then apply a notch filter at 50 Hz or 60 Hz based on the local power-line frequency. Finally, all EEG signals are resampled to 200 Hz. To mitigate the effects of low-amplitude signals and enhance numerical stability during optimization, we apply z-score normalization to most datasets. Specifically, for the SHU dataset, we instead use 95% normalization due to its signal distribution characteristics.

Metrics.
Due to the inherent class imbalance in EEG datasets, we evaluate model performance using three primary metrics. (1) Balanced Accuracy is defined as the average recall obtained on each class. By equally weighting the recall of all classes, it effectively mitigates the bias caused by imbalanced class distributions, ensuring that minority classes are not overlooked during evaluation.
(2) Cohen’s Kappa measures the level of agreement between the predicted labels and the true labels, while correcting for the agreement that could occur by random chance. This statistic provides a more robust assessment of classification performance than simple accuracy, especially in datasets where class distribution is uneven.
(3) Weighted F1 Score is the harmonic mean of precision and recall calculated for each class individually and then averaged by weighting each class’s score according to its support (i.e., the number of true instances). This metric balances false positives and false negatives while accounting for class imbalance, providing a comprehensive measure of overall model effectiveness.

Since our model produces discrete class labels directly from LLM outputs without associated probability scores, metrics that require confidence values such as AUROC (Area Under the Receiver Operating Characteristic Curve) and AUC-PR (Area Under the Precision-Recall Curve) are not applicable. Therefore, Weighted F1 and Cohen’s Kappa are prioritized for both binary and multi-class classification tasks to ensure reliable evaluation under these constraints.

## Appendix C Training Details

Experimental Setup. All experiments are conducted on a single machine equipped with eight NVIDIA A800 GPUs, each with 80GB of memory. We adopt a batch-wise alternating training strategy, where each mini-batch is drawn from a single dataset. The software environment includes Python 3.9 and PyTorch 2.0.1 with CUDA 11.8 support. We train our model using 8-GPU data parallelism with a total effective batch size of 128. Training is performed for 10 epochs using the AdamW optimizer with a cosine learning rate schedule, a base learning rate of 4e-5, and a warmup ratio of 3%.
Please refer to Table [6](https://arxiv.org/html/2506.18962v1#A3.T6) for additional training details.

**Table 6: Key training and model configuration parameters**
| Parameter | Value |
| --- | --- |
| Training Epochs | 10 |
| Batch Size | 128 |
| Learning Rate | $4\times 10^{-5}$ |
| Weight Decay | 0.01 |
| Warmup Ratio | 0.03 |
| LR Scheduler | Cosine decay |
| Max Sequence Length | 4096 |
| Dataloader Workers | 4 |
| BFloat16 Precision | True |
| Use Thumbnail | True |
| EEG Hidden Size | 1152 |
| Number of EEG Encoder Layers | 12 |
| Number of EEG Attention Heads | 10 |
| LLM Hidden Size | 4096 |
| Number of LLM Layers | 32 |
| Number of LLM Attention Heads | 32 |
| RoPE Scaling | Dynamic, factor = 2.0 |

Computational Requirements. Our language models adopt three different sizes, corresponding to InternLM-0.5B, InternLM-1.8B, and InternLM-7B as shown in Table [7](https://arxiv.org/html/2506.18962v1#A3.T7). During training, we freeze the LLM parameters and only fine-tune other components, with a total of 87,592,512 trainable parameters. The table reports the training time and maximum GPU memory consumption on 8 NVIDIA A800 GPUs.

**Table 7: Training resource comparison of UniMind**
| Language Model Size | Time (8-GPU, hours) | Max GPU Memory (GB) |
| --- | --- | --- |
| InternLM-0.5B | 7.34 | 31.2 |
| InternLM-1.8B | 8.37 | 39.4 |
| InternLM-7B | 21.78 | 61.7 |

## Appendix D More Results.

We present the complete results for Balanced Accuracy, Cohen’s Kappa, and Weighted F1 across all ten datasets. By conducting multiple experiments with different random seeds, we calculate the mean values along with their standard deviations. The outcomes on these three metrics are generally consistent with the analysis provided in the main text. Additionally, UniMind demonstrates relatively smaller standard deviations compared to other models, especially on the smaller datasets such as TUSL and Workload, which indicates more stable and reliable performance.

**Table 8: Performance comparison on SEED zheng2015seed and HMC alvarez2021hmc datasets. “MT” denotes multi-task learning usage. Underlining shows best multi-task results; bold shows best overall.**
| Model | MT | SEED | HMC |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | B-Acc | Kappa | F1-W | B-Acc | Kappa | F1-W |
| SPaRCNet | ✕ | 55.96 ($\pm$2.44) | 34.64 ($\pm$3.72) | 55.85 ($\pm$2.97) | 47.56 ($\pm$11.09) | 31.47 ($\pm$13.15) | 41.08 ($\pm$13.10) |
| ContraWR | ✕ | 61.06 ($\pm$0.78) | 42.20 ($\pm$1.29) | 61.37 ($\pm$0.85) | 42.42 ($\pm$5.41) | 23.40 ($\pm$5.54) | 29.87 ($\pm$2.88) |
| CNN-Trans. | ✕ | 61.61 ($\pm$3.84) | 42.62 ($\pm$6.01) | 61.50 ($\pm$4.63) | 65.73 ($\pm$1.41) | 59.61 ($\pm$1.05) | 68.96 ($\pm$0.65) |
| FFCL | ✕ | 58.08 ($\pm$3.22) | 37.32 ($\pm$4.62) | 57.43 ($\pm$4.02) | 44.27 ($\pm$7.02) | 25.42 ($\pm$6.54) | 29.02 ($\pm$4.85) |
| ST-Trans. | ✕ | 54.79 ($\pm$0.91) | 32.61 ($\pm$1.69) | 55.05 ($\pm$0.91) | 25.59 ($\pm$1.41) | 5.03 ($\pm$1.83) | 14.28 ($\pm$1.22) |
| BIOT | ✕ | 70.97 ($\pm$0.24) | 56.82 ($\pm$0.51) | 71.34 ($\pm$0.27) | 68.62 ($\pm$0.41) | 62.95 ($\pm$1.13) | 70.91 ($\pm$1.47) |
| LaBraM | ✕ | 73.18 ($\pm$0.19) | 59.94 ($\pm$0.31) | 73.54 ($\pm$0.21) | 72.86 ($\pm$1.01) | 68.12 ($\pm$0.73) | 75.54 ($\pm$0.24) |
| NeuroLM | ✓ | 60.34 ($\pm$0.10) | 40.82 ($\pm$0.36) | 60.63 ($\pm$0.30) | 67.37 ($\pm$0.50) | 61.88 ($\pm$0.57) | 71.26 ($\pm$0.34) |
| UniMind | ✓ | 70.55 ($\pm$0.61) | 56.28 ($\pm$0.73) | 70.98 ($\pm$0.96) | 75.27 ($\pm$0.56) | 70.58 ($\pm$0.48) | 77.40 ($\pm$0.73) |

**Table 9: Performance comparison on TUAB harati2015tuab and TUEV harati2015tuab datasets. “MT” denotes multi-task learning usage. Underlining shows best multi-task results; bold shows best overall.**
| Model | MT | TUAB | TUEV |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | B-Acc | Kappa | F1-W | B-Acc | Kappa | F1-W |
| SPaRCNet | ✕ | 78.69 ($\pm$0.47) | 50.67 ($\pm$0.29) | 75.13 ($\pm$0.80) | 41.61 ($\pm$2.62) | 42.33 ($\pm$1.81) | 70.24 ($\pm$1.04) |
| ContraWR | ✕ | 80.17 ($\pm$0.58) | 61.02 ($\pm$0.38) | 80.65 ($\pm$0.42) | 43.84 ($\pm$3.49) | 39.12 ($\pm$2.37) | 68.93 ($\pm$1.36) |
| CNN-Trans. | ✕ | 79.53 ($\pm$0.94) | 57.33 ($\pm$0.63) | 78.76 ($\pm$0.31) | 40.87 ($\pm$1.61) | 38.15 ($\pm$1.34) | 68.54 ($\pm$2.93) |
| FFCL | ✕ | 78.19 ($\pm$0.22) | 55.81 ($\pm$0.84) | 77.83 ($\pm$0.75) | 39.79 ($\pm$1.04) | 37.32 ($\pm$1.88) | 67.83 ($\pm$1.20) |
| ST-Trans. | ✕ | 79.66 ($\pm$0.31) | 61.69 ($\pm$0.45) | 80.90 ($\pm$0.93) | 39.84 ($\pm$2.28) | 37.65 ($\pm$3.06) | 68.23 ($\pm$1.90) |
| BIOT | ✕ | 79.59 ($\pm$0.89) | 59.42 ($\pm$0.13) | 78.82 ($\pm$0.90) | 52.81 ($\pm$2.25) | 52.73 ($\pm$2.49) | 74.92 ($\pm$0.82) |
| LaBraM | ✕ | 81.40 ($\pm$0.20) | 62.52 ($\pm$0.12) | 81.47 ($\pm$0.57) | 64.09 ($\pm$0.65) | 66.37 ($\pm$0.93) | 83.12 ($\pm$0.52) |
| NeuroLM | ✓ | 79.69 ($\pm$0.53) | 54.61 ($\pm$0.98) | 78.93 ($\pm$0.91) | 46.79 ($\pm$3.56) | 45.70 ($\pm$4.98) | 73.59 ($\pm$2.19) |
| UniMind | ✓ | 81.76 ($\pm$0.33) | 63.80 ($\pm$0.77) | 82.03 ($\pm$0.60) | 63.19 ($\pm$1.93) | 56.29 ($\pm$1.11) | 78.04 ($\pm$0.84) |

**Table 10: Performance comparison on TUSL zheng2019seediv and Workload kemp2000sleepedf datasets. “MT” denotes multi-task learning usage. Underlining shows best multi-task results; bold shows best overall.**
| Model | MT | TUSL | Workload |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | B-Acc | Kappa | F1-W | B-Acc | Kappa | F1-W |
| SPaRCNet | ✕ | 41.85 ($\pm$4.52) | 13.99 ($\pm$7.99) | 35.00 ($\pm$9.68) | 59.77 ($\pm$3.67) | 10.67 ($\pm$2.42) | 54.60 ($\pm$4.11) |
| ContraWR | ✕ | 58.57 ($\pm$6.62) | 35.67 ($\pm$9.68) | 54.58 ($\pm$7.98) | 69.66 ($\pm$4.33) | 31.08 ($\pm$3.81) | 69.33 ($\pm$3.78) |
| CNN-Trans. | ✕ | 35.75 ($\pm$3.51) | 3.06 ($\pm$4.79) | 22.35 ($\pm$2.51) | 57.93 ($\pm$5.14) | 8.37 ($\pm$4.35) | 58.63 ($\pm$4.62) |
| FFCL | ✕ | 39.19 ($\pm$6.88) | 6.28 ($\pm$8.88) | 21.20 ($\pm$7.86) | 70.69 ($\pm$2.46) | 41.36 ($\pm$3.12) | 72.54 ($\pm$2.93) |
| ST-Trans. | ✕ | 40.00 ($\pm$3.29) | 8.60 ($\pm$4.49) | 37.93 ($\pm$4.59) | 61.03 ($\pm$1.87) | 12.48 ($\pm$2.33) | 60.67 ($\pm$1.52) |
| BIOT | ✕ | 57.58 ($\pm$3.03) | 20.12 ($\pm$2.12) | 23.94 ($\pm$0.40) | 66.55 ($\pm$2.74) | 30.67 ($\pm$3.30) | 51.66 ($\pm$2.61) |
| LaBraM | ✕ | 76.25 ($\pm$2.31) | 64.07 ($\pm$3.04) | 76.14 ($\pm$2.12) | 66.09 ($\pm$1.94) | 29.81 ($\pm$1.35) | 64.72 ($\pm$2.43) |
| NeuroLM | ✓ | 68.45 ($\pm$3.04) | 51.94 ($\pm$4.61) | 68.39 ($\pm$2.97) | 63.45 ($\pm$2.25) | 23.19 ($\pm$1.74) | 66.57 ($\pm$1.92) |
| UniMind | ✓ | 78.95 ($\pm$1.34) | 61.53 ($\pm$1.58) | 75.40 ($\pm$1.77) | 78.67 ($\pm$1.43) | 57.33 ($\pm$1.12) | 78.65 ($\pm$1.67) |

**Table 11: Performance comparison on SEED-IV zheng2019seediv and SleepEDF kemp2000sleepedf datasets. “MT” denotes multi-task learning usage. Underlining shows best multi-task results; bold shows best overall.**
| Model | MT | SEED-IV | SleepEDF |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | B-Acc | Kappa | F1-W | B-Acc | Kappa | F1-W |
| SPaRCNet | ✕ | 29.88 ($\pm$1.37) | 6.45 ($\pm$0.92) | 32.05 ($\pm$1.48) | 60.16 ($\pm$1.21) | 65.31 ($\pm$1.36) | 58.61 ($\pm$1.44) |
| ContraWR | ✕ | 38.38 ($\pm$1.61) | 14.83 ($\pm$1.26) | 40.21 ($\pm$1.43) | 67.05 ($\pm$1.67) | 70.46 ($\pm$1.52) | 66.92 ($\pm$1.58) |
| CNN-Trans. | ✕ | 35.21 ($\pm$1.42) | 10.92 ($\pm$1.17) | 36.57 ($\pm$1.39) | 60.29 ($\pm$1.35) | 66.12 ($\pm$1.21) | 58.96 ($\pm$1.30) |
| FFCL | ✕ | 37.81 ($\pm$1.55) | 15.35 ($\pm$1.44) | 39.76 ($\pm$1.61) | 65.66 ($\pm$1.33) | 70.13 ($\pm$1.27) | 64.79 ($\pm$1.45) |
| ST-Trans. | ✕ | 36.93 ($\pm$1.46) | 15.72 ($\pm$1.30) | 36.95 ($\pm$1.50) | 55.17 ($\pm$1.49) | 67.32 ($\pm$1.38) | 53.18 ($\pm$1.42) |
| BIOT | ✕ | 36.19 ($\pm$1.38) | 23.21 ($\pm$1.47) | 42.76 ($\pm$1.52) | 64.95 ($\pm$0.26) | 71.32 ($\pm$0.32) | 60.91 ($\pm$0.40) |
| LaBraM | ✕ | 47.63 ($\pm$0.73) | 28.77 ($\pm$0.60) | 49.14 ($\pm$0.82) | 68.96 ($\pm$0.75) | 75.49 ($\pm$0.68) | 87.30 ($\pm$0.71) |
| NeuroLM | ✓ | 32.30 ($\pm$1.69) | 9.46 ($\pm$1.53) | 34.65 ($\pm$1.66) | 56.40 ($\pm$0.60) | 68.76 ($\pm$0.59) | 54.02 ($\pm$0.63) |
| UniMind | ✓ | 45.56 ($\pm$0.71) | 24.06 ($\pm$0.58) | 43.58 ($\pm$0.77) | 72.98 ($\pm$0.66) | 76.95 ($\pm$0.63) | 88.23 ($\pm$0.69) |

**Table 12: Performance comparison on SHHS quan1997shhs and SHU ma2022shu datasets. “MT” denotes multi-task learning usage. Underlining shows best multi-task results; bold shows best overall.**
| Model | MT | SHHS | SHU |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | B-Acc | Kappa | F1-W | B-Acc | Kappa | F1-W |
| SPaRCNet | ✕ | 63.93 ($\pm$0.97) | 63.47 ($\pm$0.91) | 61.82 ($\pm$0.95) | 62.15 ($\pm$0.78) | 25.32 ($\pm$0.81) | 62.15 ($\pm$0.74) |
| ContraWR | ✕ | 67.01 ($\pm$1.02) | 69.44 ($\pm$1.09) | 56.80 ($\pm$0.96) | 62.13 ($\pm$0.83) | 24.15 ($\pm$0.88) | 57.51 ($\pm$0.86) |
| CNN-Trans. | ✕ | 64.63 ($\pm$0.92) | 67.83 ($\pm$0.87) | 63.78 ($\pm$0.90) | 56.25 ($\pm$0.76) | 12.48 ($\pm$0.80) | 55.86 ($\pm$0.73) |
| FFCL | ✕ | 67.59 ($\pm$0.89) | 69.48 ($\pm$0.93) | 67.07 ($\pm$0.90) | 62.82 ($\pm$0.74) | 25.67 ($\pm$0.78) | 62.78 ($\pm$0.76) |
| ST-Trans. | ✕ | 64.63 ($\pm$0.95) | 67.18 ($\pm$0.91) | 63.30 ($\pm$0.94) | 63.39 ($\pm$0.70) | 26.76 ($\pm$0.73) | 63.25 ($\pm$0.75) |
| BIOT | ✕ | 72.22 ($\pm$0.73) | 78.35 ($\pm$0.78) | 83.96 ($\pm$0.76) | 59.16 ($\pm$0.61) | 16.68 ($\pm$0.66) | 55.51 ($\pm$0.64) |
| LaBraM | ✕ | 71.69 ($\pm$0.68) | 77.07 ($\pm$0.72) | 82.90 ($\pm$0.69) | 67.90 ($\pm$0.64) | 32.85 ($\pm$0.66) | 67.84 ($\pm$0.63) |
| NeuroLM | ✓ | 59.15 ($\pm$1.23) | 61.68 ($\pm$1.15) | 63.54 ($\pm$1.10) | 59.36 ($\pm$0.78) | 16.91 ($\pm$0.75) | 56.03 ($\pm$0.73) |
| UniMind | ✓ | 74.00 ($\pm$0.62) | 76.63 ($\pm$0.66) | 84.20 ($\pm$0.69) | 65.77 ($\pm$0.60) | 31.54 ($\pm$0.63) | 65.73 ($\pm$0.59) |

## Appendix E Instruction Construction

We adopt instruction tuning to train UniMind. To enhance instruction diversity, we design ten distinct prompts for each dataset, with each sample paired with one randomly selected task-specific instruction. Each sample contains a unique identifier, a label, the file path to the corresponding EEG signal, and a simulated human-AI dialogue. The conversation begins with the human posing a question about the category represented by the EEG signal, using one of ten predefined templates selected at random. The GPT model then generates a response based on the associated label. This dataset is organized as natural language dialogues, making it well-suited for instruction tuning and improving the model’s ability to interpret the semantics of EEG signals.

## Appendix F Discussion

In this work, UniMind establishes a promising approach for unified multi-task EEG decoding by leveraging a Neuro-Language Connector to align complex spatiotemporal EEG features with large language models, and a Task-aware Query Selection Module to dynamically adapt to heterogeneous decoding tasks. Our extensive experiments across ten datasets demonstrate that UniMind consistently outperforms previous multi-task models and achieves comparable or superior results to single-task approaches, highlighting the effectiveness of integrating LLMs in brain signal decoding fOr multi-task learning.

Despite these encouraging results, several limitations remain. First, the current evaluation focuses on datasets and tasks seen during training, without fully exploring the model’s ability to generalize to entirely unseen subjects or novel EEG tasks.
Second, due to the limited amount of available data for visual and textual decoding tasks (fewer than 10k samples), these tasks were not included in the training stage. Unlike simpler tasks such as sleep stage classification, visual and textual decoding require richer semantic understanding and finer neural representation learning, which demands larger datasets for effective training.
Third, our work focuses exclusively on EEG data and does not consider other brain-computer interface modalities such as fMRI or MEG. Integrating these complementary neural signals may provide richer information but also introduces additional challenges that remain to be addressed in future work.

Future Work. Future research will focus on enhancing UniMind’s ability to generalize to unseen subjects and novel EEG tasks, including cross-subject zero-shot and few-shot learning scenarios. Furthermore, we plan to incorporate visual and textual decoding tasks into the training process. Compared to simpler classification tasks, these decoding tasks require capturing richer semantic information and more complex neural representations, which are crucial for advancing the model’s understanding of high-level cognitive processes. Integrating these tasks will significantly enhance UniMind’s semantic comprehension and extend its applicability to more sophisticated brain decoding scenarios. Finally, we aim to progressively extend the framework to include other neural modalities such as fMRI and MEG, building a more comprehensive and robust foundation model for multi-modal brain decoding.

Overall, UniMind provides a solid foundation and new insights for future exploration of large language models in multi-task brain decoding, marking an important step toward more generalizable and interpretable neuro-AI systems.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. Could the authors provide deeper justification for the design choices of the NLC and TQS? Specifically, what motivates the use of a router-based mechanism and the decoupled spatial–temporal aggregation, and how do these contribute to improved EEG–LLM alignment beyond serving as architectural heuristics?

2. How does the proposed router-style conditioning differ in substance from prior multi-task or mixture-of-experts approaches?
soundness: 2
strengths: 1. This paper proposes Neuro-Language Connector to bridge the representational gap between neural and linguistic modalities. It uses a dual-branch spatiotemporal cross-attention mechanism, projecting EEG signals into the semantic embedding space of LLMs.

2. The Task-Aware Query Selection module achieves a balanced integration of shared and specialized representations. This design enhances cross-task generalization while maintaining task discriminability

3. The authors conduct experiments on ten EEG datasets spanning five cognitive domains.
summary: This paper introduces UniMind, a general-purpose EEG foundation model designed to perform unified multi-task brain decoding across heterogeneous EEG datasets without task-specific fine-tuning. The core idea is to leverage large language models (LLMs) to interpret neural signals by transforming EEG data into representations compatible with LLM input spaces. The authors propose 1. Neuro-Language Connector: A dual-branch spatiotemporal cross-attention module that projects EEG embeddings into the LLM’s semantic space, effectively bridging the modality gap between neural signals and textual representations. Additionally, a Task-Aware Query Selection routing mechanism is proposed to promot knowledge sharing across related tasks while reducing interference among dissimilar ones.
weaknesses: 1. The novelty and necessity of using NLC and TQS are not sufficiently established. The rationale for using a router-based mechanism and for decoupling spatial and temporal aggregation is only superficially discussed. It remains unclear why these specific architectural designs lead to improved EEG–LLM alignment or mitigate data sparsity. 

2. The router-style task conditioning has been explored extensively in multi-task and mixture-of-experts literature, so the contribution risk feels incremental without a clearer theoretical or empirical justification.

3. The performance gains over prior models are relatively modest and often within expected statistical variance. In several datasets, the improvements are smaller than 1–2%, which makes it difficult to attribute success to the proposed modules rather than training noise or dataset differences.

---

confidence: 5
contribution: 1
initial_rating: 4
presentation: 3
questions: 1a. Can you justify the generative 7B LLM architecture over a simpler, cheaper contrastive alignment (e.g., EEG-CLIP, ELM-MIL)? 

1b. Have you tried a contrastive baseline with your synthetic instruction data?

2. The TUEV benchmark is a 6-class event classification task (e.g., spike, eye movement, artifact). However, the instruction templates provided in Appendix A.6 for TUEV are exclusively focused on "epileptic events" and "epileptic states". How can the model make significant gains on TUEV and correctly classify non-epileptic events (artifacts, eye movements) when the prompt only asks for "epileptic states"?

3.  How can you be sure your "task synergy" is not just a data-level artifact of using multiple datasets from the same TUH and SEED corpuses? A baseline (e.g., LaBraM, but ideally another model like Cbramod) fine-tuned on the combined datasets would be needed to disentangle this. Currently, the baselines are inadequate in my opinion.

4. Given that the multi-task model is much, much larger and sometimes less accurate than the single-task model, what is the practical use-case for UniMind over a set of smaller, more accurate specialists?
soundness: 2
strengths: - The paper is clearly written, and the proposed architecture is well-explained and is well-reasoned for the purpose of multi-task decoding.
- The research is high-effort and the presentation is polished, with clear figures and tables.
- The model achieves a substantial performance improvement over the primary multi-task baseline (NeuroLM), with particularly large gains on difficult datasets like SEED-IV and TUEV.
summary: The paper presents UniMind, a general-purpose foundation model for multi-task EEG decoding. The core innovation is to use a large language model to unify diverse brain decoding tasks within a single framework. The authors propose two key components: first, a Neuro-Language Connector (NLC) to map EEG into representations the LLM can process, and second, a Task-aware Query Selection (TQS) module to select relevant query tokens for the various tasks. The model is trained on a large 'instruction tuning' dataset spanning 10 public EEG benchmarks. The authors report state-of-the-art results for multi-task decoding, with an 11% average gain over the main baseline, NeuroLM.
weaknesses: - **The paper's core motivation appears contradictory**. The authors rightly state that multi-task learning is challenging and that their model is theoretically at a disadvantage compared to specialized single-task models. This is confirmed by some of the results, where the unified model can be less performant than the finetuned Labram baseline. To me, this begs the main question: what is the use-case? Why would someone adopt a 7B-parameter model that is computationally demanding and less accurate, simply for the benefit of it also performing unrelated tasks? The paper fails to articulate a compelling answer.

- **Missing baselines and architectural justification** 1/2. I'm finding it difficult to assess the work's contribution due to the way it appears to be benchmarked. The work is a direct followup to NeuroLM and integrates parts from LaBraM, two works which share the authors. In case the current work is performed by the same research group, then the primary baselines, LaBraM and NeuroLM, come from the same lab, making the comparisons feel incremental. In this case, the authors do not compare against any external EEG work published in the last ~2 years (since BIOT if I see correctly). A model like e.g. CBraMod would be a valuable addition. 

- 2/2 But perhaps more importantly, the paper omits a class of computationally cheaper EEG-language alignment methods, such as those using contrastive learning (e.g., EEG-CLIP https://doi.org/10.48550/arXiv.2503.16531;  ELM-MIL; https://doi.org/10.48550/arXiv.2409.07480). The authors never justify why their complex, generative 7B LLM approach is necessary for what are ultimately classification tasks. A simpler contrastive model, which aligns EEG and text in a latent space, could be a far more efficient and practical solution, but this alternative is not explored or even discussed.

- **Confounding data overlap with task synergy**: The model is trained on datasets from shared corpuses (e.g., TUAB, TUEV, TUSL are all from the TUH EEG corpus; SEED and SEED-IV are from the same lab). The authors interpret query similarity between TUAB and TUEV (Fig 7a) as capturing "consistent neural activation patterns". A far simpler and more likely explanation is that the model is just seeing data from the same source, subjects (e.g. TUAB and TUEV may share subjects), and recording setups. This interpretation is further weakened by Figure 6a, where the t-SNE plots show clear separation between datasets, suggesting the TQS module is learning to isolate tasks, not find shared representations.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: From table 1, it seems that the model size (e.g., UniMind-1.7B and UniMind-7B) doesn't matter much. Any comment on this? 
Also, what is the model size of NeuroLM-B, L, and XL? 
What are the language models in Uni-Mind and NeuroLM? Are they the same? Could the performance difference be also due to the language models?
soundness: 4
strengths: 1. The general EEG foundation model topic itself is emerging and attractive, with many potential applications; 
2. The proposed UniMind model is novel;
3. Results are promising.
summary: The proposed UniMind, a general-purpose EEG foundation model for unified multi-task brain decoding, is quite interesting and attractive. Its  novel idea of Neuro-Language Connector and task-adaptive query scheme directly interface EEG signals with LLMs is new and interesting.
Results across 10 datasets demonstrate that UniMind outperforms other EEG foundation models.
weaknesses: I noted an earlier version of this paper on arXiv (back to June 2025: https://arxiv.org/abs/2506.18962
The content significantly overlaps with the current version. (I would say pretty much the same.)
I am curious what happened to the earlier version. 

For the performance comparison, only Balanced Accuracy results were reported; while other performance metrics (e.g., F1, Kappa etc) were also reported in related refs. E.g., even the earlier arXiv version reported F1 results. Any reason for this?

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. How sensitive is UniMind's performance to the choice of the LLM? Could a smaller or instruction-tuned LLM (e.g., Qwen3-0.6B) achieve similar results?
2. Have you explored whether UniMind can zero-shot transfer to unseen EEG tasks or datasets without retraining, given its multi-task instruction-tuned nature?
soundness: 3
strengths: 1. The manuscript is logically structured, with intuitive figures that clearly illustrate the model pipeline and experimental design.
2. Evaluation on 10 datasets, along with ablations on NLC, TQS, and query configurations, provides solid empirical support. The visualizations of task-adaptive queries and inter-task correlations are particularly insightful.
3. UniMind achieves comparable or even superior results to the best single-task model (LaBraM) in a unified multi-task setting, which is a clear technical and conceptual advancement.
summary: This paper proposes UniMind, a general-purpose EEG foundation model for unified multi-task brain decoding, leveraging large language models to interpret complex neural patterns. The key idea is to bridge EEG signals and LLMs through a Neuro-Language Connector and a Task-aware Query Selection module. Extensive experiments on 10 EEG datasets spanning 5 domains demonstrate that UniMind achieves state-of-the-art performance in multi-task settings and even matches or surpasses leading single-task models like LaBraM in several cases.
weaknesses: 1. The current experiments use only InternLM2.5 as the LLM. Since the LLM plays a central role in cross-modal reasoning, it would be valuable to test the generality of UniMind with other open-source backbones such as Qwen3 or LLaMA 4. This would help assess how much performance depends on the underlying language model.
2. Figure 4 examines query pool sizes on only five datasets. It would strengthen the claim if this analysis were extended to all 10 datasets or at least one representative dataset per task domain, since optimal query size might vary with task heterogeneity.
3. In Figure 3, UniMind surpasses LaBraM, and Figure 5 shows that joint training benefits each single task. It remains unclear whether UniMind's improvement mainly stems from joint training synergy or from the TQS/NLC architecture itself. A controlled conparison where UniMind trained independently on each task would clarify this.
4. The paper mentions constructing 929k instruction–EEG pairs, but does not discuss the diversity or linguistic quality of these prompts. Since task instructions may influence LLM behavior, providing examples and ablations on prompt formulation could strengthen the claims.