Assembling the Mind's Mosaic: Towards EEG Semantic Intent Decoding
Authors: Jiahe Li
Junru Chen
Fanqi Shen
Jialan Yang
Jada Li
Zhizhang Yuan
Baowen Cheng
Meng Li
Yang Yang
Venue: ICLR 2026 Poster (2026)
Keywords: Electroencephalography (EEG)
Brain-computer interface (BCI)
Semantic Intent
Neural decoding
Primary area: applications to neuroscience & cognitive science

Abstract: Enabling natural communication through brain–computer interfaces (BCIs) remains one of the most profound challenges in neuroscience and neurotechnology. While existing frameworks offer partial solutions, they are constrained by oversimplified semantic representations and a lack of interpretability. To overcome these limitations, we introduce **Semantic Intent Decoding(SID)**, a novel framework that translates neural activity into natural language by modeling meaning as a flexible set of compositional semantic units.
SID is built on three core principles: semantic compositionality, continuity and expandability of semantic space, and fidelity in reconstruction.
We present **BrainMosaic**, a deep learning architecture implementing SID. BrainMosaic decodes multiple semantic units from EEG/SEEG signals using set matching and then reconstructs coherent sentences through semantic-guided reconstruction. 
This approach moves beyond traditional pipelines that rely on fixed-class classification or unconstrained generation, enabling a more interpretable and expressive communication paradigm. Extensive experiments on multilingual EEG and clinical SEEG datasets demonstrate that SID and BrainMosaic offer substantial advantages over existing frameworks, paving the way for natural and effective BCI-mediated communication.

Published as a conference paper at ICLR 2026
ASSEMBLING THE MIND'S MOSAIC: TOWARDS EEG SEMANTIC INTENT DECODING
Jiahe Li"; Junru Chen'; Fangi Shen', Jialan Yang', Jada Li', Zhizhang Yuan', Baowen Cheng', Meng Li Yang Yang't
'Zhejiang University, Uniieesitty of Texas at Austin, SShnghai Institute of Microsystem and Information Technology. Chinese Academy of Sciences
(jiaheli, jrchen_caallie shenfanqi, jlyang, zhi ywwggagaaaaanannaaaaaaaaa. @zi en..e...ccc....cc jada. 1 @utexas edu, chengbaowen2 3ma 1s. ucas. ac.cr on limeng. b r com
ABSTRACT
Enabling natural communication through brain- interfaces (BCIs) re- mains one of the most profound challenges in neuroscience and neurotechnology. While existing frameworks offer partial solutions, they are constrained by over- simplified semantic representations and a lack of interpretability. To overcome these limitations, work that translates neural activity into natural language by modeling meaning as flexible set of compositional semantic units. SID is built on three core princi- ples; semantic compositionality, continuity and expandability of semantic space, and fidelity reconstruction. We present MRSINM deep learning archi- tecture implementing SID. BRAINMOSAIC decodes multiple semantic units from EEG/SEEG signals using set matching and then reconstructs coherent sentences through semantic-guided reconstruction. This approach moves beyond traditional pipelines that rely on fixed-class classification or unconstrained generation, en- abling more interpretable and expressive communication paradigm. Extensive experiments on multilingual EEG and clinical SEEG datasets demonstrate that SID and BRAINMOSAIC offer substantial advantages over existing frameworks, paving the way for natural and effective BCI-mediated communication.
introduce Semantic Intent Decoding (SID), novel frame-
INTRODUCTION
Despite advances neuroscience and neurotechnology. understanding and restoring communication remains one of the most profound challenges in brain-computer interface (BCI) research. Condi- tions such as aphasia and locked-in syndrome can sever an individual's ability to speak or write, isolating them from even the simplest forms of interaction. BCIs, recording neural activity via scalp or intracranial electroenceephalograpyy (EEG), offer promising pathway to bypass these physical barriers by translating brain signals directly into language. Most existing approaches fall into one of two primary paradigms: Speech Decoding and Concept Decoding (see Table [ for a compar- ison). Speech Decoding aims to reconstruct overt or imagined speech from motor-related cortical areas. While recent advances have improved performance, this paradigm remains fundamentally constrained by its reliance on motor regions, which represent only a limited subset of the brain's expansive language network. This focus overlooks the distributed nature of semantic processing and limits its cross-linguistic generalizability by depending on phoneme-level reconstruction (Dronkers) et al. 2017). In contrast, Concept Decoding seeks directly extract the intended meaning of an utterance from neural activity (Zhang et al. 2024). Leveraging distributed neural activity, it sup- ports communication for individuals with impairments and advances understanding of language and thought mechanisms. This paradigm thus extends BCI research towards decoding abstract cognition and paves the way for developing gext-generation neurotechnologies.
"Equal contribution. i Corresponding author
Additional Affiliations: GGunggdng--Howe Kong-Macao Greater Bay Area Center for Brain Science and Brain- Intelligence, Southern Medical University, "INSIDE Institute for NeuroAT

---

Published as a conference paper at ICLR 2026
ward
aNTRAiic and rqud)
sle
ona offii
daily
day
year
shun!
Semantic Decomposer
apple
frait
drisk
orange
(Semmnndcc Units)
apples
food
frail
Text Sumantic Space
Text Encoder
Semantic Decoder
apples siogle dry. reallyy upply jaice
EEG Encoder
spples ove food ross. Aike syyler with Arualfiins Do they not speitss
EEC Sumantic Space
Stage Semantic Unit Decomposition
Stage 2 Semantic Space Alignment
Stage Semantic-Guided Reconstruction
Figure 1 Overview of the Semantic Intent Decoding (SID) framework.
However, current implementations of Concept Decoding follow two contrasting frameworks, both presenting notable limitations. The first formulates concept decoding as a classification task on a fixed, predefined set of concepts or topics (Liu et al. 2009: Zhang et al. 2024). Though simple to implement, this approach is fundamentally restrictive. Discrete labels inherently struggle to capture the continuous and overlapping nature of meaning, often leading to poor alignment with real-world communicative intent. Alternatively, a more recent direction seeks to enhance expressive capacity by mapping neural signals directly into the latent representation space of large language models (LLMs) (Shams et al. 2025 Lu et al. 20255: Duan et al. (023). However, these methods typically require vast amounts of paired neural- linguusic data and function as black boxes. Consequently, they lack interpretability and scientific transparency, making their output difficult to verify and control.
To address the limitations of existing frameworks in Concept Decoding, particularly the trade-off' between interpretability and expressiveness, we introduce a novel framework: Semantic Intent Decoding (SID). its core is Semantic Intent, defined articulated in natural language. This notion spans a broad spectrum of communicative content, from casual, everyday expressions to formal and literary language. Table 2 datasets commonly used in Concept Decoding research and presents examples from each, illustrating the stylistic diversity encompassed by the definition Semantic Intent. At representational level, Semantic Intent is not a single point in space but flexible set of core semantic units. For example, the intent behind "I eat apples every day" can be represented as the set {I, eat, apple, daily).
coherent unit of meaning that can be
summarizes representative
To translate this abstract representation into practical decoding system, design three- stage pipeline (Figure D]. The SID framework comprises: (1) Semantic Unit Decomposition: The Seman- tic Decomposer transforms neural signals into set of core semantic components; (2) Semantic Space Aligument: The Semantic Retriever matches neural representations within an open space of conceptual representations; and (3) Semantic-Guide Guided Reconstruction: The Semantic Decoder as- sembles the retrieved elements into a coherent natural language utterance. Each stage is designed based on converging evidence from linguistic theory and cognitive neuroscience, Specifically, these three processing steps correspond to the three foundational principles outlined below, which reflect how meaning is internally represented, externally expressed, and inferentially reconstructed.
1. Compositionality: A Semantic Intent can be naturally represented as variable set of mantic units. Human communicative intent inherently multidimensional, comprising a finite, variable set of distinct yet interrelated units drawn from vast conceptual universe. Consider the intent of "I eat apples every day", which decomposes into [I], [eat], [apple], and [daily]. Conversely,
Table 1: Comparison of brain wwigaalltnnnaninunnee decoding paradigms and frameworks.
Paradigm
Framework
Compositionality
Continuity Expandability Fidelity
SID (proposed)
Semantic Units
V
High
Concept Decoding
Label Classification
Single Category
X
X
Low
End-to- End Generation
Full Sentence
V
Low
Speech
Phonetic Reconstruction
Phoneme Sequence
N/A
V
High
Decoding
Spectrum Mapping
Spectrogram
N/A
Low
2

---

Published as a conference paper at ICLR 2026
Table 2: Overview of datasets in multiple languages for EEG semantic intent decoding, with illus- trative examples of sentence styles.
Dataset
Language
Style
Example
Chisco Zhang et al.
2024
Chinese
Daily
(Today is my birthday.)
(RS R 991778 E
ChineseBEG2 Chen et al. 2025
Chinese
Literary
(The bristled with jagged stones.)
ZuCo (Hollenstein et al.
2018
English
Formal
As remake, it's a pale imitation.
#8. T 18 IE (I'm sleepy and ready for bed.)
Clinical
Chinese
Daily
the sentence "I eat apples every day, usually the sweet ones my grandma picks during harvest" integrates attributes ([sweet)) source ([grandma picks]), and temporal context (lduring harvest]). This complexity highlights why single-label classification cannot capture the richness of commu- nicative intent (Zhang al., 2024).. This compositional perspective aligns with evidence that the brain constructs meaning through dynamic, context-sensitee processes rather than fixed, localized representations (Kutas & Federmeier, 2000; /Zhang & Pylkkänen 2018).
2. Continuity and Expandability: Semantic units should be decoded within open and con- tinuous space. Linguistically. semantic space is inherently open-ended and continuous: when new terms (e.g., Apple's "Vision Pro") emerge, speakers rapidly infer meaning from context and inte- grate them with related concepts (AR headsets, spatial computing). Thus, decoding should operate within an open, continuous space that supports similarity-basee retrieval and seamless generaliza- tion. Neuroscientificllly, the brain encodes concepts smooth, low-dimensional maps across the cortex (Huth et al.) 2012), paralleling behavioral similarity judgments and vector spaces in language models (Hebart et al.) 2020 Doerig et al. 2025 Grand et al. 2022). Decoding into this continuous space enables generalization to novel concepts beyond fixed label sets.
Fidelity: Reconstructed sentences must reflect decoded meaning and natural language structure. From linguistic perspective, fidelity has two dimensions. Semantic fidelity ensures that decoded units constrain generation, keeping outputs grounded in intended concepts (e.g.. "ap- ple," "harvest," "cat"") rather than drifting unrelated ones like "rabbit." Linguistic coherence en- sures these units form grammatical, plausible sentences, avoiding violations like "apples cat the harvest." Together, these constraints guide contextually appropriate inferences such as "orchard" over "classroom," yielding faithful and intelligible outputs. From a neuroscience perspective, com- prehension similarly depends on semantic congruity and structural control, supported by the brain's semantic control network for flexible, context-sensitive retrieval (Jefferies) 2013). Enforcing these constraints yields simultaneously interpretable and intelligible interpretations. circumventing the idiosyncratic errors common in unconstrained end-to-end decoding (Duan et al. 2023).
Guided by these principles, we introduce BRAINMOSAIC, a novel deep learning method for open- vocabulary communication within the Semantic Intent Decoding (SID) framework. BRAINMOSAIC decodes Semantic Intent through three stages: it first decomposes neural meaning by modeling EEG signals at the level of multiple semantic units; second, it aligns the decoded semantic space with large-scale, open, and continuous linguistic space through set matching, aligning brain-derived units with linguistic concepts; and finally, it guides sentence reconstruction, leveraging the rich semantic representations and reasoning capabilities of large language models (LLMs) to generate coherent, faithful outputs. BRAINMOSAIE fully realizes the advantages of the SID framework, overcoming the limitations of existing mainstream approaches. The key contributions of this work are as follows:
1. We propose Semantic Intent Decoding (SID), framework for EEG-based concept decoding that maps neural signals into a continuous semantic space, providing an interpretable link between concept-level representations and natural language generation.
2. To implement SID, we introduce BRAINMOSAIC, concrete implementation of SID that pre- dicts a variable-sized set of semantic representations via set matching and uses semantic-guide language modeling for constrained sentence generation.
3. Extensive experiments both public multilingual EEG datasets and a private clinical Stereo- EEG (SEEG) dataset demonstrate that SID and BRAINMOSAIC deliver substantial improve- ments over existing frameworks.
3

---

Published as a conference paper at ICLR 2026
2
METHOD
This section presents BRAINMOSAIC within the context of the SID framework, describing its ar- chitecture and the way each component implements the core principles of SID. We first outline the three Foundational principles and key components of SID, explaining how BRAINMOSAIC is de- signed to address them. We then describe how these components are integrated into unified system for semantic intent decoding.
2.1 SEMANTIC DECOMPOSER: DECOMPOSING NEURAL SIGNALS INTO SEMANTIC UNITS
Principle 1 (Compositionallty: Set-based Representation of Intent). A Semantic Intent I can be represented variable set of semantic units S = {u,, : 122.... ). Semantic unit sets follow the fundamental properties of sets: they are unordered, contain no duplicates, and can vary in size.
Semantic Intent corresponds to a single, coherent meaning expressed as finite set of semantic units. While fixed word order is required for well-formed sentences, the core meaning of short, simple utterances can often be preserved despite changes word order. Psycholinguistic evidence supports this approximation: eye-tracking studies show that reading is not strictly serial, with readers prioritizing semantic gist over exact token order (Schotter & Dillon 2025). and transposed-won experiments in both English and Chinese demonstrate that moderate word swaps do not necessarily disrupt comprehension (Mirault et al.] 2018 Liu et al.) (022). Moreover, the size of a Semantic Intent is bounded by cognitive constraints on working memory, which can only actively maintain limited number of informational "chunks" at once (Cowan) 2001). This means simple intent may contain only a few units (c.g.. "I eat apples every day."). while more detailed one may involve more (e.g., "I eat apples every day, usually the sweet ones my grandma picks during harvest."). Longer and more complex sentences can thus be naturally decomposed into multiple semantic intents, each forming manageable, approximately ionnnutationiivvrrian unit for modeling.
BRAINMOSAIC Semantic Decomposer. According to Principle I BRAINMOSAIC decodes EEG features into fixed-size set of K query slots, each representing potential semantic unit. The value of is chosen based on the maximum expected number of semantic units for a single intent in the dataset, with the actual number of active units varying freely (1 S n S K). This design balances flexibility for sentence complexity and a clear upper bound. Inspired by set-based object detection frameworks such as DETR (Carion et al. 2020p, we adopt bipartite matching formulation to handle the variable and unordered nature of semantic units. During training, each ground-truth unit is uniquely matched to at most one predicted slot, while unmatched slots are assigned "no-object" class. This converts the variable target set into well-defined training signal, explicitly guiding the network to decompose EEG features into semantically meaningful components. The optimal matching and final training loss are defined as:
special
ā arg min dESK
Cmatch (yte, (o)s (1)
Liuungriian (v.
Σ matcc (yi: (e());; (2)
where Lmatch combines semantic alignment loss for matched slots and a binary classification loss for unmatched slots labeled as no-object. This ensures one-to-one matching and provides stable supervision regardless of the order and number of semantic units. In this way. BRAINMOSAIC achieves both permutation invariance and bounded cardinality as required by Principle [ laying the groundwork for the next stage, where predicted units are aligned with continuous semantic space for retrieval and reconstruction.
2.2 SEMANTIC RETRIEVER: ALIGNING SEMANTIC UNITS WITH CONTINUOUS SPACE
Principle 2 (Continuity and Expandability: Continuous Representation of Semantic Space). Se- mantic units should be decoded within an open and continuous space V C Id, where each unit is mapped to an embedding E(u) E V. In this space, similar concepts lie closer together, with semantic similarity inversely proportional to their embedding distance.
Semantic relationships are inherently continuous rather than discrete, as evidenced by findings from linguistics and psycholinguistics. From human cognition, the distributional hypothesis links contex-
4

---

Published as a conference paper at ICLR 2026
tual co-occurrence to semantic relatedness (Harris 1954), and phenomena such as semantic priming and graded synonymy show that similarity is gradual and quantifiable (Meyer Schvaneveldt 1971 Miller & Charles I99I). The continuity of semantic space implies that human cogmition open and compositional. New concepts emerge from the organic integration of existing ones into the global associative network. Consequently, semantic space does not expand indefinitely but instead becomes increasingly dense (Fodor) 1975 De Deyne et al., 2019). EEG studies further show that neural ac- tivity captures continuous semantic representations, with patterns brain signals predicting graded word vector relationships beyond discrete category boundaries (Foster et al., (021). Modern pre- trained LLM embedding spaces instantiate the same principle computationally (Zhang et al. 2025b] ByteDance| 2024). Trained with multi-stage contrastive objectives on massive corpora, they yield a stable manifold V in which vector proximity reliably tracks semantic affinity. Thus, the structure observed in human language and cognition can be mirrored in these learned spaces, providing well-validated target for aligning EEG-derived semantic units to precise coordinates within V.
BRAINMOSAIC Semantic Retriever. Building on Principle 2 the BRAINMOSAIC Semantic Re- triever uses bipartite matching procedure to align each predicted slot with a ground -tuuth semantic unit in the continuous space V. The per-slot matching loss, Lmatch, consists of two terms: a se- mantic alignment term that encourages the predicted embedding ý to close to its corresponding target word embedding E(u) based on cosine similarity, and a slot activity classification term that distinguishes active semantic units from "no-object" slots. This joint objective enables the model to learn token-level semantic alignment.
Cmatch Cmateh(E(w))B) 1. () = sim( 9) + Acta (- (1 () ())(1
(3)
Here, sim((:,) denotes a cosine-based similarity, / E {0, 1} the slot's match indicator (1 the slot is assigned to ground-truth unit; 0 for "no-object"), and ý is the predicted activity probability. This ensures that only matched slots are pulled toward their targets in V, while all slots learn a calibrated decision boundary between active and "no-object". In parallel, the retriever predicts global embedding & E V. This embedding is aligned with the ground-truth sentence embedding E(s) capture holistic semantic intent, and is further processed through multiple classification heads. Each head predicts distinct global attribute of the intent (e.g., tone, subjectivity). This dual supervision enables the model to integrate token-level information with high-level global properties.
Lglobbll (F((), 2, [1 sim [))) +aattr Σ CF(2
global alignment
global attribute classification
where (0)) and g(c) are the ground-truff and predicted outputs for the e-th global attribute, Aattr balances the alignment term and the classification term. The overall retriever loss combines the Hungarian objective from the toven-level matching with the global-level supervision.
Cretriever Hungaria + Aglobal Eglobal:
(5)
2.3 SEMANTIC DECODER: RECONSTRUCTING LANGUAGE FROM SEMANTIC UNITS
Principle 3 (Fidelity: Faithful and Coherent Reconstruction). Given a predicted set of semantic units Stetrieved {m)'...., un " the reconstruction process seeks sentence T ((Setriieva)) that is both semantically grounded in these units and linguistically well-formed.
Modern LLMs, trained on massive and diverse text corpora, exhibit exceptional capabilities in con- textual reasoning, compositionality, and natural language generation (Achiam al.) 2023 Team et al. 2023 Liu et al. 2024). These properties make them particularly well-suited for reconstruct- ing fluent, coherent sentences from a discrete set of semantic units.
BRAINMOSAIC Semantic Decoder. In line with Principle 3 the Semantic Decoder in BRAIN- MOSAIC is built around a LLM. To interface with it, Semantic Retriever outputs are first converted into a structured prompt. For each predicted slot, the retriever searches the entire semantic space and returns returns ranked candidate units with probabilities (reflecting importance). Collectively, this
5

---

Published as a conference paper at ICLR 2026
matching process yields the final retrieved set Stetrieved {u1' , ..... I with corresponding prob- abilities {PI..... ... pm These token-level candidates are then integrated with global sentence-level attributes predicted by the model, such sentence type or tone, to form the final prompt:
Prompt P(Sretrieved: Z) T= G(Prompt),
(6)
where Z denotes the set of global contextual signals. Through this, LLM G reverses semantic de- composition to integrate retrieved units into a fluent sentence. Output T remains faithful to decoded units and robust to retrieval noise.
2.4 FORMAL DEFINITION AND ASSEMBLY OF BRAINMOSAIC
Figure I provides an overview of the complete workflow, reflecting the overall structure of the Semantic Intent Decoding framework which BRAINMOSAIC is grounded. Other components and the detailed algorithm description are provided in Appendix
During training, raw EEG recordings first processed by the EEG Encoder, which converts multi- channel temporal signals into set of neural feature tokens. These tokens are passed to the Semantic Decomposer, where the Transformer and its learnable queries produce K candidate slots represent- ing potential semantic units. The Text Encoder provides the reference semantic space, including unit-level embeddings and sentence-level targets. The Semantic Retriever aligns each predicted slot with this space through bipartite matching, simultaneously predicting token-level assignments and global intent attributes. The entire network is optimized end-to-end using the combined losses intro- duced above, ensuring that neural features, semantic slots, and linguistic targets are jointly shaped through a unified objective. The full training procedure summarized Algorithm | which out- lines the data flow and optimization steps within each training epoch.
During inference, the EEG Encoder and Semantic Decomposer generate slot embeddings and activ- ity scores from unscen EEG input. The Semantic Retriever then searches the text embedding space for the nearest semantic units, filtering by confidence thresholds to produce retrieved set Sretrieved and global attributes. These passed the Semantic Decoder, which constructs structured prompt and uses a LLM to generate the final natural- sentence. In this way, information flows seamlessly across all five components, transforming raw brain signals into coherent and mantically faithful sentences. The complete inference pipeline is detailed in Algorithm 2
3 EXPERIMENTS
We evaluate BRAINMOSAIC on neural-to-semantic decoding tasks and address three questions:
Q1: Is communicative intent adequately modeled as set of semantic units?
Q2: Does decoding in a continuous semantic space enable better generalization and scalability?
Q3: Does semantic-constrained generation improve fidelity and interpretability?
3.1 EXPERIMENT SETUP
Datasets. To rigorously evaluate SID and BRAINMOSAIC, we curated three public datasets based on two criteria: (1) neural signals must be segmentable into semantically complete units (i.e., sen- tences); (2) recordings must be collected during cognitive tasks. Their basic characteristies are summarized in Table 2 with dataset-specific details and preprocessing procedures in Appendix A Notably, the Chisco dataset (Zhang et al.] 2024) was recorded directly at the sentence level, with each trial corresponding with character-leve timestamps, which are later segmented into sentences during preprocessing. As the stimuli are presented as full sentences with self-contained semantics and focus on everyday ac- tivities, Chisco most closely aligns with our definition of Semantic Intent. In addition, we collected a private clinical SEEG dataset at partner hospital on invasive recordings. It includes one participant performing sentence-level imagined speech task involving 515 everyday Chinese sentences with colloquial semantics covering daily activities and interpersonal interactions. During recording, the participant completed memory-judgment task to decide if each sentence related to target topic. Detailed information is provided in Appendix B Additional experiments on the clinical dataset, including brain-region contribution analyses, are provided in Appendices H Haadd]]
complete utterance. Other public datasets contain continuous passages

---

Published as a conference paper at ICLR 2026
Baselines. We adapt open-source methods and design four main baselines: (1) Cls-Align trains an EEG classifier using classification labels (Zhang et al.) 2024). Its frozen backbone is then aligned with pre-trained sentence embeddings for cross-modal mapping. (2) Multi-Cl, uses the same back- bone as Cls-Align for multi-label classification, predicting top-k candidate semantic units to form a complete sentence. (3) NeuroSSeaatic aligns EEG signals to a pre-trained text embedding space, and then employs a language model to generate sentences from the aligned embeddings, It enables unconstrained text generation but is limited to English corpora (Shams al., (222). (4) Seq-Decode replaces the set-matching stage with an LSTM-basee sequential decoder, while keeping the same ModemnTNN encoder and LLM. This baseline is designed assess the contribution of set match- ing. Implementation details are in Appendix C Appendix C.5 further reports baselines using fully randomized labels and random/corpus-basee predictions.
Evaluation Metrics. SID operates in a continuous, open-vocabulary space where traditional dis- crete metries are inadequate. At the concept level, existing measures fail capture whether decoded units fall into the correct semantic neighborhood. At the sentence level, we prioritize semantic fi- delity rather than surface overlap: n-gram metrics (e.g., BLEU, ROUGE) emphasize exact token matches while underestimating synonymy and paraphrasing (detailed analysis in Appendix D).
To address these gaps, we design three embedding- metries. Unit Matching Accuracy (UMA) reflects hard correctness at the concept level: predicted unit 2, is counted as correct only when its similarity to the gold unit zł exceeds a predefined threshold т. Mean Unit Similarity (MUS) complements this with a soft measure of alignment by averaging unit-wise similarities, capturing graded improvements even when predictions fall close to the threshold. We further define MUSeup to quantify the semantic density within a corpus space. It is computed by uniformly and indepen- dently sampling semantic units from the corpus without frequency weighting and then averaging the embedding similarities over all sampled pairs. This value serves theoretical lower bound, repre- senting the expected MUS inherent to the corpus itself. Sentence Reconstruction Similarity (SRS) evaluates sentence-level semantic fidelity by comparing the embedding of the generated sentence with that of the reference s", emphasizing meaning rather than exact wording. For BRAINMOSAIC, we generate five candidate sentences using the GPT-4o-mini LLM and report the average SRS.
(1)UMA 1 Σ (sim (2,, z;) > r). (2)MUS
1 Σ sim zf), (3)SRS- sim(s. s*)
where N denotes the total number of ground-truth semantic units, sim(..-) denotes baseee-based similarity and I(-) is an indicator function. We also report the BERTScore-FI (Zhang* et al. 2020
as sentence-level reference metric. In Appendix D we provide qualitative examples for the above metrics to more intuitively illustrate the semantic relevance reflected by their values.
All experiments are conducted in-subject. For datasets with multiple subjects, we report mean across subjects; for the clinical dataset, the reported variance comes from multiple runs. Unless noted otherwise, all experiments use the doubao-embeeddin-llrgee (ByteDance) (024) text space. Appendix D reports additional results with alternative and random text encoders. Regarding base- line comparisons, metric applicability varies by architecture: for Baseline 1. which yields only representation-level outputs without explicit sentence generation, only SRS is calculated; for Base- line 3, evaluation is restricted to sentence-level metries; and for Baselines 2 and 4, we explicitly map their predictions into the unified text semantic space to ensure a fair comparison on MUS.
3.2 Q1: IS COMMUNICATIVE INTENT ADEQUATELY MODELED AS A SET OF SEMANTIC UNITS
For Question 1, we compare the set-based BRAINMOSAIC with two alternative paradigms, detailed in Table Baseline (/) Cls-Align assigns a single, predefined topic category to each sentence, mak- ing overly reductive for diverse expressions. For fair comparison, we first train the classification model using its original categorical framework. We then freeze its parameters and train an addi- tional mapping module to project its outputs into the same sentence-level embedding space used by BRAINMOSAIC for similarity evaluation. (1) Cls-Align shows limited performance, highlighting the difficulty of capturing nuanced, compositional meaning within the constrained label sets of existing cabbl-classification frameworks and datasets.
The second comparison examines whether modeling intent as an unordered set offers advantages over sequential decoding. Baseline (4) Seq-Decode replace the proposed set-matching module with
7

---

Published as a conference paper at ICLR 2026
Table 3: Comparison of baselines and BRAINMOSAIC with error bars; best results in bold.
Cases with 0.001 are marked with *o* based on two-sided t-test against the best baseline.
Clinical
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-FI
Cls-Align
0.5976 0.0030
Multi-Cls
0.0359 0.0006
0.6739-10.0061 0.6503+10.0115
0.4400 +
0.6173
Seq eec-Decode
0.1786 = 0.0126
0,5104 +
0.6126 +
MRAANC
0.6596 + 0.0102
0.8124 + 0.0108
0.6651 + 0.0045
0.6629 + 0.0137***
Chisco
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-F1
Cls-Align Multi-Cls
0.5040 + 0.0117
0.0143 0.0008
0.6332 0.0051
0.3585 0.0063
0.5692 0.0060
eeq-Decode
0.0301 = 0.0016
0.5503-+0.0042
0.5439 0.0055
0.5706 0.0034
BRSAIC
0.5617 1 0.0085
0.8009 0.0024***
0.6206 0.0034
0.6195 + 0.0019
ChineseEEG-2
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-F1
Cls-Align Multi-Cls
0.5475 + 0.0192
0.0099 = 0.0002
0.6058110.0047
0.3992 0.0084
0.5573
0.0058
Seq- Beq-Decode
0.0420 0.0377
0.53731000016
0.5357 0.0073
0.5586 0.0053
BRAINMOSAIC
0.3707 + 0.0144***
0.7687 0.0035 9*
0.6163 0.0025$**
0.6002 0.0031 *.*
ZuCoSR
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-F1
Cls-Align
0,4630 + 0.0390
Multi-Cls
0.0003 + 0.0001
0.6469-10.0038
0.3385 + 0.0053
0.4008 + 0.0036
Neuro2Semantic
0.5211 0.0085
0.4296 0.0230
Seq-Decode
0.0451 + 0.0101
0.6321
0.6171 0.0051
0.4274 0.0114
MOSAIC
0.7506 1 0.
0.8586 0.0044
0.6982
0.0026
0.4640 + 0.0116
ZuCoNR
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-F1
Multi-Cls
0.0013 = 0,0012
0.6393 #0.0116
0.2349 + 0.0150
0,3828 = 0.0107
Neuro2Semantic Seq- Decode
0.4445 0.0057
0.3865 0.0199
0.0669 0.0141
0.610410.0103
0.5161 0.0044
0.3926 0.0112
BRSANMOSAIC
0.7144 + 0.0371
0.8453 0.0043 *9
0.6094 0.0102
0.4423 + 0.0139***
ZuCoTSR
Concept Level
Sentence Level
UMA
MUS
SRS
BERT-FI
Cls-Align Multi-Cls
0.3797 + 0.0179
0.0101 1 0.0010
0.6030.10.0126
0.2245 1 0.0099
0.3753 +. 0.0105
Neuro2Semantic
0.4591 0.0068
0.4158
0.0190
Seq-Decode
0.0654 1: 0.0114
0.6076.10.0047
0.5181 0.0079
0.3860 0.0091
BRAINMOSAIC
0.5520 + 0.
0.8198 0.0096 *..
0.5956 0.0039
0.4431 + 0.0100*+*
an LSTM decoder that predicts an ordered sequence of semantic units. Both models share the same EEG encoder and LLM module. Results show that the sequential baseline underperforms, yielding lower concept-level UMA and MUS and producing less coherent outputs, which underscores that overly rigid sequential modeling constrains the semantic intent expressed in EEG signals.
8

---

Published as a conference paper at ICLR 2026
Table 4: BRAINMOSAIC performance under (A) vocabulary expansion and (B) training data expan- sion settings.
Clinical
Chisco
Training Ratio
MUS
MUS
Corpus Size
UMA
MUS
SRS
BS-F1
UMA
MUS
SRS
BS-FI
(A) Vocabulary Expansion @ Training Ratio
1.0
Base
.6596
.8124
.5151
.6651
.6629
.5617
.8009
.4811
.6206
.6195
Base + 500
.6539
.088
.5351
.6454
.6580
.4603
.8022
.4834
.6184
.6184
Base 1000
.6439
8103
.5372
.6453
.6597
.4595
.8018
.4849
.6152
.6170
1.0
Base 3000
.6263
80633
.5399
.6320
.6579
.4597
.8019
.4948
.6154
.6143
Base 10000
.6276
.8002
.5519
.6295
.6464
.4555
.8001
.5279
.6232
.6145
Base + 30000
.5944
.8042
.5689
.6316
.6393
.4533
.8016
.5600
.6222
.6137
(B) Training Expansion
Word Size
Base + 30000
0.5
.6672
.7624
.5689
.6004
.5741
.4337
.8010
.5660
.6220
.6013
Basc 30000
0.3
.4944
.7171
.5689
.5697
.5672
.4292
.7935
.5660
.6083
.5810
0.1
.3399
.6963
.5689
.5888
.5479
.4186
.7867
.5600
.5760
.5665
3.3
Q2: DOES DECODING IN CONTINUOUS SEMANTIC SPACE ENABLE BETTER GENERALIZATION AND SCALABILITY
Continuity. To answer Question 2, we first compare BRAINMOSAIC with baseline (2) Multi-Cls The baseline directly classifies each input into one of the discrete semantic units from the tokenized vocabulary, which contains thousands of possible classes. For evaluation, the predicted labels are projected into the same embedding space used by BRAINMOSAIC via the shared Text Encoder, after which we apply the identical procedure to compute UMA and MUS. The same L.M is used to reconstruct sentences from predicted units for sentence-level evaluation. Results in Table 3 show that the discrete approach performs poorly, with U U/MA remaining extremely low and MUS close to the random expectation (see Table When semantic categories grow into the hundreds thousands, the limitations of discrete classification methods become apparent. natural language, however, the number of semantic categories reaches the tens of thousands. In contrast, BRAINMOSAIC decodes directly into a continuous semantic space, where vector proximity encodes conceptual relatedness, enabling far greater flexibility and scalability.
Expandability. We conducted two experiments to evaluate the generalization and scalability of continuous semantic decoding. First, we test open-set decoding by expanding retrieval vocabulary with unseen high-frequency words, specifically the top 500, 1k. 3k, 10k, 30k from the List of Commonly Used Words in Modern Chinese (Li & Sul 2021). on Clinical and Chisco datasets (see Table A). This setting simulates realistic language use and tests whether the continuous semantic space enables effeettve handling of out-of-vocabulaat exceeds everyday vocabulary and includes domain specific terms that primarily appear in academic fields. Second, examine scalability by starting with the largest vocabulary and gradually raising training data proportion from 10% up to 100% (see Table B). This tests whether the semantic space learnable and extensible without degrading performance on previously learned labels.
(OOV) words. Notably, the 30k condition far
Results confirm that continuous semantic decoding enables strong generalization and scalability. In the open-vocabulary inference scenario (A), U decreases only modestly the vocabulary expands, showing robustness to a larger search space. MUS and SRS remain relatively stable or even improve slightly as the vocabulary expands, showing that when an exact match is missing, the model retrieves semantically related substitute. The model's MUS remains stable even as the expected random similarity (MUSesp) increases with vocabulary density, indicating that retrieval driven by meaningful semantic structure, not random proximity. In the training expansion scenario (B), performance scales consistently with training data's volume. While performance is limited with only 10% of the data, both U 7 A and SRS improve substantially as the training set grows. Critically, the model not only expands its active vocabulary but also learns to generalize, predicting words that never appeared in the smaller subsets. This expansion does not degrade performance previously learned units, demonstrating that the continuous semantic space can be learned without a closed vocabulary constraint.

---

Published as a conference paper at ICLR 2026
Table 5; Ablation Results of BRAINMOSAIC on Clinical and Chisco.
Clinical
Chisco
UMA
MUS
SRS
UMA
MUS
SRS
w/o Set
0.0792
0.7052
0.5721
0.0260
0.6690
0.5506
w/o Space
0.0137
0.6393
0,4604
0.0044
0.6009
0.3727
w/o LLM
0.6596
0.8124
0,5456
0,5617
0.8009
0,5133
Full Model
0.6596
0.8124
0.6651
0.5617
0.8009
0.6206
3.4 Q3: DOES CEMAANIC--ONSTRAANI GENERATION IMPROVE FIDELITY AND INTERPRETABILITY
To address Question 3, we compare BRAINMOSAIC with the end-to-end generation- based baseline Neuro2Semantic assess differences in fidelity and interpretability. Neuro2Semantic employs an LSTM-basee Mb--based architecture to directly translate neural signals into continuous text by mapping EEG data into a pre-trained text embedding space, enabling unconstrained sentence generation. However, key limitation of Neuro2Semante and similar methods is their reliance on the language of the pre- trained generator. As the released model English-only, our comparison confined to the English ZuCo datasets. BRAINMOSAIC consistently outperforms the baseline, confirming that constraining generation with decoded semantic units significantly improves fidelity to the intended meaning. Fur- thermore, the transparent intermediate semantic units. Reliability tests using various L.Mss and prompt variations confirmed consistent performance advantages, validating semantie-constrained generation robustly improves fidelity and maintains stable interpretable decoding. Details are in Appendix F
strategy in BRAINMOSAIC enhances interpretability by providing
3.5 ABLATION ANALYSIS OF BRAINMOSAIC
To assess the contribution of each component in BRAINMOSAIC, we conduct a series of ablation experiments. (a) Ablation of the Semantic Decomposer (w/o Set). The EEG encoder directly aligned with sentence embeddings without the set-baset decomposition stage. During inference, the model selects the top-K semantic units most similar to the predicted sentence embedding, where K equals to the number of slots in BRAINMOSAIC. (b) Ablation of the Continuous Semantic Space (w/o ContSpace). The semantic retrieval module is replaced by multi-label classifier. The model directly predicts K category labels, which serve the semantic units without leveraging the continuous embedding space. (c) Ablation the LLM Semantic Decoder (w/o LLM). Given the semantic unit produced by BRAINMOSAIC,
language model is used for sentence recon- struction. Instead, the units are concatenated in random orders to form 50 candidate sentences per instance, and the reported metrics use the highest SRS for each instance.
As shown in Table 5 performance decreases in all three ablation settings, demonstrating that each component is essential BRAINMOSAIC. Removing the set-based semantie decomposer forces the model to align EEG signals only with sentence-level embeddings. However, sentence-level semantics are highly entangled, and their relationships lack the structured organization as semantic units. Ablating the continuous semantic space yields behavior similar to baseline (2) Multi-Cls, as the model degenerates into a multi-label classifier. the LLM ablation, the predicted semantic units remain unchanged and only the sentence reconstruction stage is removed. The results indicate that the decoded semantic units are informative but fragmented. The LLM performs this post-processing automatically, organizing the units into fluent and interpretable outputs. This highlights the LLM's role transforming conceptual predictions into natural language.
4 DISCUSSION
We introduced Semantic Intent Decoding (SID) and its implementation BRAINMOSAIC, transpar- ent compositional framework for translating neural signals into natural language. Moving beyond fixed-label classification and opaque end-to-end models, it enables more natural BCI communication and advances study of neural meaning representations. Future work can improve semantic decom- position, explore intent representation across brain regions, and extend SID to new modalities and real-time systems. These efforts will advance BCIs to more accurate flexible systems, fostering new Al-neuroscience intersection discoveries.
10

---

kaqq// Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work supported by NSFC (62322606, 62441605).
ETHICS STATEMENT
All experiments were conducted in strict compliance with relevant ethical guidelines and institu- tional review processes. The private clinical SEEG dataset was collected under approved IRB pro- tocols at partner hospital, with informed consent obtained from the participant. No personally identifiable information is included. To protect privacy and security, the private dataset will not be publicly released, and only aggregate statistics and example stimuli are provided in the Appendix B
REPRODUCIBILITY STATEMENT
To ensure reproducibility, comprehensive descriptions of model architectures. training configura- tions, and evaluation protocols are provided in the main text and appendix. Our source code and experiment scripts are included in GitHub repository Brai PrraiMMosacco ( TCLR26) to facilitate replication. Due to privacy restrictions, the private clini- cal dataset cannot be fully released; however, detailed documentation and representative examples included in the Appendix B to aid understanding and replication with comparable data.
/tttts://gitien
USE OF LARGE LANGUAGE MODELS
Large Language Models were used solely to aid in the polishing of language and writing clarity during manuscript preparation.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.088 08774, 2023.
Jeffrey R Binder, Rutvik H Desai, William W Graves, and Lisa L. Conant. Where is the semantic system? critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral cortex, 19(12):767-2796. 2009.
Michael P Broderick, Andrew J Anderson, Giovanni M Di Liberto, Michael J Crosse, and Edmund C Lalor. Filecrophyysiolggiaa correlates semantic dissimilarity reflect the comprehension of nat- ural, narrative speech. Current Biology, 28(5):803-809, 2018.
ByteDance.
2024.
1o/
2025-09-18.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference computer vision, pp. 213-229. Springer, 2020.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijic Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1-45, 2024.
Sitong Chen, Beiqianyi Li, Cuilin He, Dongyang Li, Mingyang Wu. Xinke Shen, Song Wang, Xue- tao Wei, Xindi Wang, Haiyan Wu, et al. Chineseceg-2: An eeg dataset for multimodal semantic alignment and neural decoding during reading and listening. arXiv preprint 20Nx:2500..042400 2025.
João M Correia, Bernadette Jansma, Lars Hausfeld, Sanne Kikkert, and Milene Bonte. Eeg decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations. Frontiers in psychology, 6:71, 2015.

---

Published as a conference paper at ICLR 2026
Nelson Cowan. The magical number 4 in short-term memory: A reconsideration of men- tal storage capacity. Behavioral and Brain Sciences, 24(1):87-114 2001. doi: 10.1017/ s0140525x01003922.
Xiyang Dai, Yinpeng Chen, Jianwei Yang, Pengchuan Zhang, Lu Yuan, and Lei Zhang. Dynamic detr: End-to-end object detection with dynamic attention. In Proceedings of the IEEE/CVF inter- national conference computer vision, pp.
2021.
Simon De Deyne, Danielle Navarro, Amy Perfors, Mare Brysbaert, and Gert Storms. The "small world of words" english word association norms for over 12,000 cue words. Behavior research methods, 51(3):987-1066 2019.
Adrien Doerig. Tim Kietzmann, Emily Allen, Yihan Wu, Thomas Naselaris, Kendrick Kay, and Ian Charest. High-level visual representations in the human brain are aligned with large language models. Nature Machine Intelligence, 10. 10.10088/4225--22--010777
August 2025. ISSN 2522-5839, doi:
Nina F Dronkers. A new brain region for coordinating speech articulation. Nature, 384(6605):
159-161, 1996.
Nina Dronkers, Maria V Ivanova, and Juliana V Baldo. What do language disorders reveal about brain-language relationships? from classic models network approaches. Journal of the Inter- national Neuropsychological Society, 23(9-10):441-544 2017.
Yiqun Duan, Jinzhao Zhou, Zhen Wang, Yu-Kai Wang, and Chin-teng Lin. Dewave: Discrete encoding of eeg waves for eeg text translation. Advances Neural Information Processing Systems, 36:9907-9918, 2023.
Rongyao Fang, Peng Gao, Aojun Zhou, Yingjie Cai, Si Liu, Jifeng Dai, and Hongsheng Li. Feataug- detr: Enriching one-to-many matching for detrs with feature augmentation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 46(9):6402-6415, 2024.
Jerry Fodor. The language of thought. Harvard University Press, 1975.
Chris Foster, Chad C Williams, Olave E Krigolson, and Alona Fyshe. Using eeg to decode semantics during an artificial language learning task. Brain and Behavior, 118)),2223, 2021.
Angela D Friederici. The brain basis of language processing: from structure to function. Physiolog- ical reviews, 93(4) 2011.
Gabriel Grand, Idan Asher Blank, Francisco Pereira, and Evelina Fedorenko. Semantic projection recovers rich human knowledge of multiple object features from word embeddings. Nature human behaviour, 6(7):975-987, July 2022. ISSN 2397-3374. doi: 10.
Zellig S Harris. Distributional structure. Word, 10(2-3):146-16) 1954.
Martin N. Hebart, Charles Y. Zheng, Francisco Pereira, and Chris I. Baker. Revealing the multidimensional mental representations of natural objects underlying human similarity judge- ments. Nature Human Behaviour, 4(11):1173-118;; November 2020. ISSN 2397-3374. doi:
10.1038/441562-02000999999 02011-----
Gregory Hickok and David Poeppel. The cortical organization of speech processing. Nature reviews neuroscience, 8(5):393 402, 2007.
Nora Hollenstein, Jonathan Rotsztejn, Marius Troendle, Andreas Pedroni, Ce Zhang, and Nicolas Langer. Zuco, simultaneous eeg and eye-tracking resource for natural sentence reading. Scien- lific data, 5(1):1-13, 2018
Nora Hollenstein, Marius Troendle, Ce Zhang, and Nicolas Langer. Zuco 2.0: A dataset of phys- iological recordings during natural reading and annotation. arXiv preprint 19Xiv:1912.0090. 2019.
12

---

Published as a conference paper at ICLR 2026
Alexander G. Huth, Shinji Nishimoto, An T. Vu, and Jack L. Gallant. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6):1210-1224 December 2012. ISSN 0896-6273. doi: 10.1016/j.ncuron20122. 014.
Elizabeth Jefferies. The neural basis of semantic cognition: Converging evidence from neuropsy- chology, neuroimaging and TMS. Cortex; a journal devoted to the study of the nervous system and behavior, 49(3):61 1-625, 2013. ISSN 0010-9452. doi: 00.1016f.cortex.211..1..
Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, and Han Hu. Detrs with hybrid matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1971221971,, 2023.
Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.
Marta Kutas and Kara D. Federmeier. Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4(12):463-44) 470, December 2000. ISSN 13646613.
doi: 10. 10.1011/S1 (EV/S1364-6611 3000)0)65666
Vernon J Lawhern, Amelia Solon, Nicholas Waytowich, Stephen M Gordon, Chou Hung, and Brent Lance. Eegnet: compact convolutional neural network for eeg-based brain-computer interfaces. Journal of neural engineering, 15(5)0056013, 2018.
Xingjian Li and Xinchun Su (eds.). List of Commonly Used Words in Modern Chinese (2nd Edition). The Commercial Press, 8 2021. ISBN 978-7-100-20011-0
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Hesheng Liu, Yigal Agam, Joseph Madsen, and Gabriel Kreiman. Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron, 62(2):281-290, 2009.
Yan Liu, Zehao Zhao, Minpeng Xu, Haiqing Yu, Yanming Zhu, Jie Zhang, Linghao Bu, Xiaoluo Zhang, Junfeng Lu, Yuanning Li, et al. Decoding and synthesizing tonal language speech from brain activity. Science Advances, 9(23):cadh0478, 2023.
Zhiwei Liu, Yan Li, Michael G. Cutter, Kevin B. Paterson, and Jingxin Wang. A transposed effect across space and time: Evidence from Chinese. Cognition, 218:104922, January 2022. ISSN 00100277. doi: 10. motiiicoonitionn22....... 104922.
Diego Lopez-Berne Bernal, David Balderas, Pedro Ponce, and Arturo Molina. state-of-the-ard review of eeg-based imagined speech decoding. Frontiers in human neuroscience, 16:867281, 2022.
Jacky Tai- Yu Lu, Jung Chiang, Chi-Sheng Chen, Anna Nai- Yun Tung, Hsiang Wei Hu, and Yuan Chiao Cheng. Ecg2text-cn: An exploratory study of open-vocabulay, chinese text-eeg alignment via large language model and contrastive learning on chineseeeg. arXiv preprint arXiv:2506.000 00854, 2025.
Dongaao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The twelfth international conference on learning representations, pp. 1-43, 2024.
David E Meyer and Roger W Schvaneveldt. Facilitation in recognizing pairs of words: Evidence of dependence between retrieval operations. Journal of experimental psychology, 90(2):227, 1971.
George A. Miller and Walter G. Charles. Contextual correlates of semantic similarity. Language and Cognitive Processes, January 1991. doi: 10. 108406936.
Kai Miller, Gerwin Schalk, Dora Hermes, Jeffrey G Ojemann, and Rajesh PN Rao. Spontaneous decoding of the timing and content of human object perception from cortical surface recordings reveals complementary information the event-related potential and broadband spectral change.
PLoS computational biology, 12(1)ee100466) 2016.
13

---

Published as a conference paper at ICLR 2026
Jonathan Mirault, Joshua Snell, and Jonathan Grainger. You that read wrong again! A transposed- word effect in grammaticality judgments. Psychological Science, 29(12) 1929, 2018. doi: 10.117709557996118869969
David A Moses, Sean L. Metzger, Jessie R Liu, Gopala Anumanchipalli Joseph G Makin, Pengfei Sun, Josh Chartier, Maximilian E Dougherty. Patricia M Liu, Gary M Abrams, et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. New England Journal of Medicine, 385(3):217-227, 2021.
Elliot Murphy, Kiefer J Forseth, Cristian Donos, Kathryn M Snyder, Patrick S Rollo, and Nitin Tandon. The spatiotemporal dynamics of semantic integration the human brain. Nature com- munications, 14(1):6336, 2023.
Nicolás Nieto, Victoria Peterson, Hugo Leonardo Rufiner, Juan Esteban Kamienkowski, and Ruben Spies. Thinking out loud, open-access eeg-based bei dataset for inner speech recognition. Scientific data, 2022.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 779-788, 2016.
Zahraa Sabra, Leonardo Bonilha, and Thomas Naselaris. Spectral encoding of seen and attended object categories in the human brain. Journal of Neuroscience, 40(2):327-342, 2020.
Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38(1 1 11:5)1):-459939 2017.
Elizabeth R. Schotter and Brian Dillon. A beginner's guide to eye tracking for psycholinguistic studies of reading. Behavior Research Methods, 57(2):68, January 2025. ISSN 1554-3528. doi: 10.3758/s13428 -42-143-----22225--2-444
Siavash Shams, Richard Antonello, Gavin Mischler, Stephan Bickel, Ashesh Mehta, and Nima Mes- garani. Neuro2semantic: A transfer learning framework for semantic reconstruction of continu- ous language from human intracranial eeg, 2025. URL httpp:://aaxi...rrrrrtr/// http://aaxxi..oor/a///2/0500
00381
Alexander B Silva, Kaylo T Littlejohn, Jessie R Liu, David A Moses, and Edward Chang. The speech neuroprosthesis. Nature Reviews Neuroscience, 25(7):473 492,,-9922111 2024.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:/312.603: 2013.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint 23I.v /805, 2023.
Juan R Vidal, Tomás Ossandón, Karim Jerbi, Sarang Dalal, Lorella Minotti, Philippe Ryvlin, Philippe Kahane, and Jean-Philippe Lachaux. Category-specifie visual responses: intracranial study comparing gamma, beta, alpha, and erp response selectivity. Frontiers in human neuro- science, 4:195, 2010.
Wei Wang, Alan D Degenhart, Gustavo P Sudre, Dean A Pomerleau, and Elizabeth CTylle-Kabara, Decoding semantic information from human electrocortioorrahicc (ecog) signals. In 20/ Anmual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6294 6298. IEEE, 2011.
Holly Wilson, Mohammad Golbabace, Michael J Proulx, Stephen Charles, and Eamonn O'Neill. Eeg- based bei dataset semantic concepts for imagination and perception tasks. Scientific Data, 10(1 ( 33:6,, 2023.
14

---

Published as a conference paper at ICLR 2026
Linmin Zhang and Liina Pylkkänen. Semantic composition of sentences word by word: MEG evidence for shared processing of conceptual and logical elements. Neuropsychologia, 119:392- 404, 2018. ISSN 0028-3932. doi:
2018.08.016
Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9759-9768, 2020.
Sibo Zhang. Bihui Yu, Lili Zhou, Gang Wang, Jingxuan Wei, Linzhuang Sun, and Liping Bu. Eegtet: Eleettoenceephalogra---o based chinese text decoding. In International Conference Intelligent Computing, pp. 174-185. Springer, 2025a.
Tianyi Zhang*, Varsha Kishore", Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference Learning Representations, 2020. [tRLtttss///ppeeeeiiii http://www..ee.ooooooooeoeeeea//
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embed- ding: Advancing text embedding and reranking through foundation models, 2025b. URL
https://arriw.org//bbbb 200. 05176
Zihan Zhang, Xiao Ding, Yu Bao, Yi Zhao, Xia Liang, Bing Qin, and Ting Liu. Chisco: An ceg- based bci dataset for decoding of imagined speech. Scientific Data, 11(1):1265, 2024.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2), 2023.
Hui Zheng, Haiteng Wang, Weibang Jiang, Zhongtao Chen, Li He, Peiyang Lin, Penghu Wei, Guoguang Zhao, and Yunzhe Liu. Du-in: Discrete units-guided mask modeling for decoding speech from intracranial neural signals. Advances in Neural Information Processing Systems, 37: 79996-80033, 2024.
15

---

Published as a conference paper at ICLR 2026
A
PUBLIC DATASET
Table Summary datasets used in our experiments. Sent. represents Sentence. 00V denotes the proportion of concepts that appear in the test set but not in the training set.
Task
MUSexp (Doubao)
MUSesp (Qwen)
Datasct
#Sents
#Words
OOV
Signal
#Channels
(#Subjects)
Read (5),
Chisco
6,681
4,595
30266
.4855
.0550
EEG
122
Imagine (5)
Read (4),
ChineseEEG2
Listen 3,621
3,910
.3663
.4715
.4144
EEG
128
ZuCo
Read (12)
1,039
5,398
.6668
.6062
.4354
EEG
ZuCo 2.0
Read (18)
663
3,461
.6431
.6083
.4316
EEG
105
Private
Imagine (1)
515
350
.1215
.5244
.4567
SEEG
132
Table provides overview of the datasets, covering both Chinese and English and spanning tasks from silent reading to imagined speech, as well as different signal modalities. Following standard practice in neural decoding, all experiments were conducted under in-subject setting, where a separate model trained and evaluated for each individual subject. For each subject, all trials were randomly split into an 8:2 train-test ratio and repeated multiple times. For subjects who participated multiple experimental paradigms within the same project (e.g., both listening and reading tasks), treat each paradigm as independent subject. Across all datasets, each subject completed the experiment only once. To further analyze the linguistic characteristics of these corpora, we computed the OOV ratio, defined the proportion of words appearing in the test set but absent in the training set relative to the testing vocabulary size. This measure reflects the lexical generalization challenge across datasets. In addition, we incorporated and calculated the
widely used
two embedding models, Doubao-embeddae large and similarity. These statistics quantify corpus-level redundancy and semantic diversity from the perspective of modern sentence embeddings.
expected
A.1 CIIScO
Chisco (Zhang et al. 2024) is a large-scale EEG corpus for imagined speech decoding, containing recordings from 5 subjects with over 20,000 samples, and about 900 minutes per subject. The stimuli cover over 6,000 phrases across 39 semantic categories. Each trial consists of a Reading Phase (sentence memorization) and Recall Phase (imagined speech). We used the preprocessed data, where the continuous recordings were further segmented into a 5s reading segment and 3.3s imagined speech segment based on event markers.
A.2 CHINESBEEG
ChineseBEE-20 (Chen et al.) 2025) is a naturalistic EEG dataset collected while participants either read aloud or passive listen to two full-length novels, the Little Prince and Garnett Dream, both in Chinese version. In general, the text materials consisted of 46,591 Chinese characters. To ensure linguistic coherence, the data acquisition preserved the line-level integrity of the texts. Following the dataset protocol, segment the continuous EEG recordings according to the line-level text labels, aligning each trial with the corresponding sentence fragment.
A.3 ZUCO
ZuCo is a multimodal corpus combining high-density EEG and eye-tracking during natural read- ing. It includes two versions. ZuCo 1.0 (Hollenstein et al.) 2018). with data from 12 English native speakers reading about 4 hours text, and ZuCo 2.0 (Hollenstein et al.) 2019), which expands the corpus with 739 sentences from 18 participants. The experimental design covers three tasks: (1) normal reading with sentiment analysis on movie reviews (SR), (2) normal reading with com- prehension questions Wikipedia sentences (NR), and (3) task specific reading (TSR) requiring relation type identification (c.g., award, education). In our work, we used the preprocessed EEG data, aligned with the corresponding text stimuli based event markers.
16

---

Published as a conference paper at ICLR 2026
B THE CLINICAL DATASET
The intracranial EEG dataset was recorded using 12 electrode arrays comprising 132 channels, pro- viding broad coverage across cortical and subcortical regions. This comprehensive electrophys- iological mapping was designed to advance our understanding of neural mechanisms underlying conceptual judgment during perceptual processing.
Participants. A male individual diagnosed with epilepsy participated in our study. The partici- pant underwent implantation of iEEG electrodes as part of his clinical treatment for epilepsy. The participant voluntarily enrolled in the study and provided written informed consent. The experimen- tal protocol and data collection procedures were reviewed and approved by the institutional review board of Huashan Hospital, Fudan University.
Tasks. Five conceptual categories were selected, with 200 corresponding sentences assigned to each. Examples of participants judged whether the presented concept matched the sentence and indicated their response keypress.
sentences across these categories are provided Table During the experiment,
Electrode Localization and Visualization. lected based on clinical requirements. Anatomical designations were obtained through Freesurfer's cortical parcellation.
The number and positioning of the electrodes were
Neural signal recording. Neural activity was recorded using a multi-channel electrophyssologiiaal recording system (EEG-1200C, Nihon Kohden, Tokyo, Japan) with sampling rate of 2000 Hz.
Data preprocessing. The neural signals were extracted based on the data labels recorded during experiment, and then downsampled from 2000Hz to 1000 Hz and notch filtered 50 Hz, 100 Hz, and 150 Hz. All neural signals underwent bipolar referencing and normalization.
Table 7: Representative Chinese sentences and their English translations in the Clinical dataset.
Category
Chinese Sentence
English Translation
The food smells so good!
Dining
m4 12
My stomach is growling, let's eat something.
#L
A #
The things the phone really interesting,
Phone
want to play with my phone to relax.
T
AENE T
I'm sleepy and want to go bed.
Sleeping
#
Go to bed early, and you'll Il be energetic tomorrow.
The toilet is clogged, what should do? My stomach feels uncomfortable, I need go the toilet.
Toilet
#
Please help me get the medicine.
Health Care
have an allergic reaction to this medicine.
17

---

Published as a conference paper at ICLR 2026
C BASELINE
C.I CLS-ALIGN
This baseline follows the approach of Chisco, employing EEGNet (Lawhern al. 2018 as the backbone to extract low-level EEG features, which are subsequently processed by transformer encoder with self-attention and a fully connected layer to capture higher-level representations. The training process consists of two stages. First, classification task is performed, in which neural signals are categorized based on dataset-specific labels. These labels include 39 classes in Chisco, classes for book source in ChineseEEG-2. 2, 3 classes for sentiment in ZuCo-SR, 7 classes for rela- tion type identification in CSCO, and classes for Clinical. It worth noting that for ZuCo-NR dataset, patients required complete multiple-choice questions rather than producing categori- cal labels, making such classification-basse training infeasible. After the classification training, the backbone is frozen, and projection head is trained to align neural embeddings with corresponding sentence embeddings. The final similarity is evaluated between the generated neural embeddings and the textual embeddings.
C.2 MULTI-CLS
This baseline adopts the same backbone Cls-Align (EEGNet + transformer encoder) to extract neural representations, Instead of aligning with text embeddings, the model directly performs multi- class classification, where the predicted top-k class labels treated concept-level cluster tags. These predicted tags are then used to query a large language model (LLM), allowing the system to generate natural language sentences from neural signals.
C.3 NEURO2SEMANTIC
This model uses an LSTM adapter as the backbone to encode neural signals into fixed-dimensional embeddings aligned with pre-trained text embeddings sexx-embeeding--da---000 (Shams et al., 2025). Training proceeds in two phases: (1) in the alignment phase, the LSTM adapter is trained with a combination of contrastive loss and triplet margin loss to enforce semantic consistency be- tween neural signals and text embeddings; (2) in the decoding phase, the adapter is frozen, and Vec2Text inversion module (a transformer- beaddd corrector) is fine-tuned to reconstruct coherent text sequences from the aligned neural embeddings. Through this two-step process, the model is able to directly generate natural language sentences from neural signals.
C.4 SEQ-DECODE
This baseline employs ModernTCN (Luo & Wang 2024) as the neural signal encoder, which mod- ernizes ID convolution blocks by decoupling temporal, feature, and variable dimensions and stack- ing them in a residual architecture to capture both cross-time and cross-variable dependencies. On top of this encoder, an LSTM-based sequential decoder predicts sequence of concept labels step by step, with an internal binary classifier to determine whether to stop prediction. Finally, the predicted concept label sequence is passed LLM, which generates coherent sentences by filling in the semantic content in the predicted order.
C.5 RANDOM BASELINES
To more clearly demonstrate the contribution of the EEG semantic decoding module, we compare the proposed BRAINMOSAIC against. label-level and prediction-level random baselines under the Doubao embedding setting.
Label-level Random baseline preserves the EEG semantic-unie decoding module. For each sample, we randomly sample semantic units from the corpus, where equals the number true units in that sample's golden sentence, and compare the decoded predictions with these randomly sampled pseudo-labels using UMA and MUS. For the sentence- metric, SRS is computed between the mooell-geneatee sentence and randomly selected sentence from the corpus. This baseline evaluates whether the model's 's predictions truly reflect the semantic intent contained the input EEG, rather than matching by chance.
18

---

Published as a conference paper at ICLR 2026
Table 8: Random and text-prior baselines compared with BRAINMOSAIC across datasets.
Clinical
Chisco
ChineseEEG-2
UMA
MUS
SRS
UMA
MUS
SRS
UMA
MUS
SRS
Label Random
0.2472
0.6878
0.5394
0.1390
0.6987
0.4997
0.1308
0.6955
0.4683
Random-prion 0.0563
0.6392
0,5056
0.0152
0.6185
0.4924
0.0112
0,5815
0,5129
Topk-prior
0.1510
0.6884
0.5418
0.1215
0.6793
0.5465
0.1159
0.6606
0.5282
Freq-prior 0.1008
0.6540
0.5321
0.0289
0.6365
0.5124
0.0332
0.6046
0.5170
BRAINMOSAIC
0.6596
0.8124
0.6651
0.5617
0.8009
0.6206
0.3707
0.7687
0.6163
ZuCoSR
ZuCoNR
ZuCoTSR
UMA
MUS
SRS
UMA
MUS
SRS
UMA
MUS
SRS
Label Random
0.2557
0.7591
0.5499
0.2640
0.7339
0.5297
0.2588
0.7084
0.5009
Random-prior 0.0624
0.7198
0.5526
0.0788
0.7163
0.4879
0.0641
0.7049
0.4999
Topk-prior 0.1559
0.7407
0.5681
0.1467
0.7554
0.5227
0.1266
0.7137
0.5033
Freq-prior 0.1078
0.7311
0.5612
0.1003
0.7351
0.5033
0.0951
0.7190
0.4916
BRAINMOSAIC 0.7506
0.8586
0.6982
0.7144
0.8453
0.6094
0.5520
0.8198
0.5956
Prediction-level Random baselines bypass the EEG decoding module and directly sample the pre- diction set from the corpus. For each sample, select & semantic units (corresponding to the number of model slots), compare the resulting UMA and MUS with the golden labels, and then L.M to synthesize sentences from this randomly sampled set compute SRS against the golden sentence. We provide three text-prior strategies; (a) Random-prior randomly select lo se- mantic units from the corpus; (b) TopK-prior: deterministically take the top-k most frequent units in the corpus; (c) Freq-prior: randomly select A semantic units according to frequency distribu- tion. These baselines estimate the performance that can be achieved solely from corpus statistics or language priors, without using EEG information at all.
The results Table show that all random and text-prior baselines perform poorly within the SID framework. Although baselines that exploit corpus priors may occasionally obtain deceptively high concept-level scores, the resulting semantic sets are inherently meaningless and collapse to chance level when evaluated at sentence level. These findings further indicate that the model's outputs genuinely reflect the semantic information decoded from EEG, rather than artifacts of corpus statistics or LLM priors.
19

---

Published as a conference paper at ICLR 2026
D ANALYSIS AND QUALITATIVE EXAMPLES OF EVALUATION METRICS
_US__ex00.55
UMA UMA_thrcshee threshold=0.88
7 A from merge
10 B bro male
(a) Concept (word) Level
Western food
eat R
T -
A 300 beaver
ME #2 bathe cut
ft T fasting swallow
38
RE
solar term
have
apple
38 grandfather
30 EEL sesign
#t TE
क #3
loqual
orange juice
daily
क*
T pain
X
X
8 days
make money
nowadays
0.4
0.5
0.6
0.7
0.8
word similarity
Polly vollyaary
Sentence Level
t 89.
SRS BERT-FI BLEU-1 BLEU-2 WER
GOLDEN
Synonymy
1R 00)
0.76
0.70
0.27
0.16
0.82
BrainMosaic
ptone
great
Fraelly namt (M0 tantroi sra..
(i)
f H E
TLL relyy vant phone
0.67
0.66
0.22
0.14
0.91
Text-Prior
(b) Intent (sentence) Level
(ii)
random
#HL
R
$
BR T.
0.57
0.63
0.45
0.21
0.82
example-1
bod
Bnginnewiica You iowing
(ii1)
t FL म... #il
th
*
0.56
0.61
0.15
0.11
1.09
w/o
phore BE
wey be
(iv)
poymoot
random example 2emppee-2
FLL
RE
RI t
0.53
0.58
0.36
0.19
0.82
We Wiay Viy.
Seq-Decode
(v)
T.
0.50
0.63
0.00
0.00
1.00
www.s/altooee//ooooooooooo play
(vi)
w/o Spnc
0.47
0.62
0.16
0.04
0.82
Rodingoa's Frigeed tody
(vii) w/o Set
z 9 37.
0.37
0.54
0.08
0.03
0.91
resting 20dy g953
Figure 2: Qualitative examples of two-level evaluation metrics. Underlines mark meaningful
Chinese or English word groups. (Parentheses indicate additional elements inserted in English trans- lations for syntactic or semantic clarity.) (a) Concept (word) level examples are sampled from the union of the Clinical Chisco the 30k Chinese dictionary. Each word falls into of the embed- ding similarity intervals [0, 0.5), [0.5, 0.6), [0.6, 0.7), [0.7, 0.8), [0.8, 1.0) relative to the target units in the Doubao embedding space. (b) Intent (sentence) level examples including the gold reference. one prediction from BR MOSAOI, five baseline results and two manually constructed sentences. illustrating typical linguistic phenomena (sorted by SRS score in descending order).
20

---

Published as a conference paper at ICLR 2026
D.1 EVALUATION METRIC DESIGN
To comprehensively evaluate the SID framework, we assess model outputs at two levels: the concept (word) level and the intent (sentence) level. This ensures an objective and accurate assessment of both the decoding accuracy of individual semantic units and the extent to which the reconstructed sentences restore the actual semantic intent behind the brain signals. Traditional Natural Language Processing (NLP) token-level metrics aim for strict, precise comparison of surface characters, which are more suitable for tasks like Speech Decoding (e.g., decoding Chinese initials/finals or English syllables). In contrast, the SID framework explores human cognition, where the understanding and expression of semantics are inherently abstract. Consequently, SID and the broader concept decoding paradigm require metrics that evaluate relative semantic relationships rather than character- level exact matches. For this reason, we introduced the Unit Matching Accuracy (UMA), Mean Unit Similarity (MUS), and Sentence Reconstruction Similarity (SRS) to capture semantic intent beyond surface forms.
D.2 QUALITATIVE ANALYSIS OF CONCEPT-LEVEL METRICS
Figure aa illustrates the applicability of concept-level metrics using the sentence "I eat apples everyday" as a reference. We analyzed the semantic similarity between the four semantic units in this sentence and all words from large Chinese corpus. The results demonstrate that when UMS exceeds 0.8, the semantic correlation between words is significant. For instance, the word "eat" (FZ) matches strongly with "eat" ( ( and "have a meal" (1E which represent the same action in formal and casual contexts, respectively. Conversely, "apple" (3p lacks any synonym with similarity greater than 0.8, reflecting that "apple" this Chinese corpus. Based on these observations, we set the cluster threshold and UMA threshold at 0.8 in our experiments. This effectively clusters semantically identical or highly related terms (e.g., connecting "myself" ( A) and "we" (!E) with "T" (IR) while avoiding an overly bloated semantic space. Furthermore, the MUS eep (the expected embedding similarity of two randomly drawn semantic units) is approximately 0.57. As shown in the interval [0.5, 0.6), word pairs near this value share almost no meaningful semantic or contextual connection, confirming that our metrics effectively distinguish between random noise and true semantic relatedness.
specific entity without close synonyms in
D.3 QUALITATIVE ANALYSIS OF INTENT-LEVEL METRICS
Figure b) provides a qualitative comparison of sentence-level metrics, highlighting the limitations of traditional NLP metrics (like BLEU and WER) for SID tasks. We compared the ground truth (Golden) sentence "One can see many interesting things when scrolling on the phone" FL # & म W A'I) with the prediction from BRAINMOSAIC and several baselines. The BRAIN- (f captures the core
MOSAIC output, "Playing on my phone is a great fun" intent using phone and experiencing enjoyment despite surface-level differences. Traditional metrics fail to capture this semantic alignment due to four primary linguistic phenomena common in languages:
1. Polysemy (One Word, Multiple Meanings): Standard metrics cannot distinguish context- dependent meanings. For example, the character "#f" in the Golden sentence appears in "If " meaning "many." However, in sentence (vi), "The result was pretty good this time" ( KD), and sentence (vii). "Resting early is good for the body" (t & (TT)), the same character "IF" means "good." Similarly, "øl"" in the Golden sentence means "scrolling." whereas in sentence (iv), "Phone payment can replace swiping a card" (L AE card. BLEU-based metrics reward these character matches despite the different meanings.
ft tel F), means "swiping" a
2. Synonymy (One Meaning, Multiple Words): BRAINMOSAIC uses "5i" (play) instead of "ā"" (scroll) to describe using phone. While "Bj" is more vivid, both verbs functionally describe the same activity in this context. Additionally, the model uses * (very big/great) to express high degree, paralleling "IT- (many) in the Golden sentence. Since Chinese does not strictly distin- guish countability for degree adverbs, these are semantically equivalent, yet rigid token matching penalizes this variation.
21

---

Published as a conference paper at ICLR 2026
Table 9: Supplementary Metrics across datasets.
Clinical
Chisco
BLEU1
BLEU2
BLEU4
WER
BLEU1
BLEU2
BLEU4
WER
Multi-Cls
0.1291
0.0550
0.0257
0.9344
0.0530
0.0195
0.0112
0.9949
Seq-Decode
0.1231
0.0464
0.0231
0,9445
0.0453
0.0159
0.0091
1.0438
BRAINMOSAIC
0.1787
0.0841
0.0393
0.9635
0.0947
0.0391
0.0190
1.0361
ChineseEEG-2
ZuCoSR
BLEU1
BLEU2
BLEU4
WER
BLEU1
BLEU2
BLEU4
WER
Multi-Cls
0.0420
0.0156
0.0089
1.0896
0.0046
0.0016
0.0013
0.8900
Neuro2Semantic
0.0689
0.0175
0.0081
1,2158
Seq-Decode
0.0496
0.0197
0.0106
1.1675
0.0049
0.0018
0.0014
0.8543
BRAINMOSAIC
0.0813
0.0299
0.0158
1.0747
0.0754
0.0163
0.0081
1.0618
ZuCoNR
ZuCoTSR
BLEU1
BLEU2
BLEU4
WER
BLEU1
BLEU2
BLEU4
WER
Multi-Cls
0.0045
0.0017
0.0013
0.9197
0.0043
0.0017
0.0013
0.8954
Neuro2Semantic
0.0721
0.0199
0.0086
1.0940
0.0901
0.0247
0.0096
1.0971
Seq-Decode
0.0114
0.0043
0.0031
0.8646
0.0083
0.0035
0.0023
0.8842
BRAINMOSAIC
0.0556
0.0167
0.0082
1.0594
0.0543
0.0170
0.0081
1.0520
Keyword Proximity without Semantic Relevance: Sentence (ii), (iv) contain high keyword overlap ("FF,",, "Bl"" with the Golden sentence. However, their intents unrelated to the Golden intent. Traditional metrics often overscore these sentences due to n-gram overlap.
4. Broken Logic and Syntax: Sentence (iii) represents the result of ablation study without LLM post-procees prosesrocss::::: "You having fun phone exists fun is very big" (KAR F A). This output is concatenation of keywords without grammatical structure. While it hits several key terms, lacks coherent logic. Our SRS metric correctly penalizes such scenario, whereas n-gram precision metrics might assign it high score.
D.4 METRIC SELECTION
While we observed that BERT-FI generally aligns with SID evaluation requirements, it lacks suffi- cient discrimination ability. For instance, sentence (vi), which contains abstract and vague references ('this time". ""ssult"), achieves BERT-FI score of 0.62, which is deceptively high for sentence with low semantic relevance to the specific Golden intent. In contrast, our SRS metric better reflect- ing the semantic divergence. Therefore, we adopt SRS our primary sentence-level metric, while providing B BLEU and WER scores for reference Table
22

---

Published as a conference paper at ICLR 2026
Table 10: Performance of different text encoder configurations on Clinical and Chisco datasets.
Clinical
Chisco
UMA
MUS
SRS
BERT-F1
UMA
MUS
SRS
BERT-F1
Random
0.0644
0.6962
0.5067
0.5962
0.0417
0.6882
0,4932
0.5719
Qwen3 (Word)
0.1994
0.7194
0.6014
0.6166
0.1601
0.7073
0.6044
0,5873
Qwen3 (Definition)
0.2082
0.7389
0.6271
0.6249
0.1685
0.7108
0.6121
0,5904
Doubao (Word)
0.6373
0.8014
0.6578
0.6480
0.5502
0.7986
0.6098
0.6022
Doubao (Definition)
0.6596
0.8124
0.6651
0.6629
0.5617
0.8009
0.6206
0.6195
E ANALYSIS OF TEXT ENCODER VARIANTS
The construction of continuous semantic space is fundamental the performance of the SID framework. To systematically evaluate this impact, we conducted ablation studies comparing dif- ferent embedding models, construction strategies, and randomized baseline. For the Random Embedding condition, we generated fixed 256-dimensional embedding library where vector simi- larities were distributed within the (0, 1) interval, with a mean of approximately 0.5 and a standard deviation of 0.1. Each semantic unit and sentence in the corpus was assigned a fixed random vector from this library. This setup served as a control to measure model performance in the complete absence of semantic structure. We evaluated two multi-language embedding models, Qwen (Zhang ett al,, (2255)) and Doubao (Byte (024). We used two embedding generation strategies to assess the quality of the resulting semantic spaces:
1. Word-based Generation: The embedding is generated directly from the semantic unit itself
(e.g., inputting "orchard" + ("tembbedding""rchtttt
2. Definition-based Generation: The embedding generated from the dictionary definition or semantic explanation of the unit (e-g., inputting "orchard" - gelemtedding ("orchard: intentional plantation of trees or shrubs that is maintained for food production")).
("orhhard
The experimental results (see Table TOp demonstrate that well-structured text semantic space is critical for SID. The Random Embedding condition resulted in catastrophic degradation of perfor- mance, confirming that the random initialization completely destroys the semantic structure between concepts. This validates that our model's 's performance relies on the intrinsic geometric relationships within the semantic space rather than fitting arbitrary sigh--imensional vectors. Comparisons be- tween the generation strategies reveal that Definition-based Generation significantly outperforms the direct Word-based approach. Current LLMS are optimized for processing and understanding longer contexts. Therefore, providing a descriptive definition allows the embedding model to cap- ture richer semantic features than a single token can provide. Furthermore, this implics a promising scaling property for our framework: as LLM embedding technologies continue to advance in expres- siveness and granularity, our method will naturally inherit these improvements, potentially leading to further gains in decoding accuracy without architectural changes.
Furthermore, we compare the performance BRAINMOSAIC against baselines under both the Doubao (definition-basedl and Qwen3 (definition-based) embedding settings. Since the choice of embedding space primarily affects the MUS metric while other metrics remain largely unchanged, we report only the two baselines that support MUS evaluation. The results in Table TT show that, re- gardless of the embedding model used, BRAINMOSAIC consistently and substantially outperforms all baselines.
23

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of MUS results under Doubao and Qwen3 embedding spaces.
Clinical
Chisco
ChineseEEG-2
MUS-D
MUS-Q
MUS-D
MUS-Q
MUS-D
MUS-Q
Multi-Cls
0.6739
0.5148
0.6332
0,4508
0.6058
0,4818
Seq-Decode
0.6503
0.6149
0.5503
0.4728
0,5373
0,5077
BRAINMOSAIC
0.8124
0.7389
0.8009
0.7108
0.7687
0.7025
ZuCoSR
ZuCoNR
ZuCoTSR
MUS-D
MUS-Q
MUS-D
MUS-Q
MUS-D
MUS-Q
Multi-Cls
0.6469
0.3796
0.6393
0.3571
0,6330
0.3505
Seq-Decode
0.6321
0.4948
0.6104
0,4924
0.6076
0.4872
BRAINMOSAIC
0.8586
0.7052
0.8453
0.6868
0.8198
0.6897
Table 12: Effect of LLM, number of input tokens (k), and prompt design on SRS. Here, K i the average number of tokens per sentence in the Clinical dataset.
LLM
Prompt
2K
5K
all
GPT--4omini
with-score
0.6876
0.6696
0.6413
0.6288
no-score
0.6836
0.6685
0.6464
0.6305
Doubao-Seed
with-score 0.6840
0.6616
0.6329
0.6197
no-score 0.6954
0.6823
0.6436
0.6266
DeepSeek
with-score 0.6988
0.6810
0.6444
0.6286
no-score
0.6873
0.6726
0.6430
0.6324
F ANALYSIS OF SEMANTIC DECODER VARIATIONS
We further evaluated the impact of prompt design and parameter choices on sentence reconstruction using clinical dataset, in which each sentence contains average of K 4.365 tokens. Our analysis considered three factors: the language model backend, the number of predicted words (k) provided the model, and whether the predicted probability scores were explicitly included in the prompt. We compared three representative LLMs, GPT-4o-mini, Doubao-Seed Swwd--eeel-------- 1iae-1.--Fash,, and DeepSeek-Chan and tested different values of k K, 2K, 5K, and all predicted words with confidence scores above 0.8.
As shown Table T2] SRS remained consistently high across all conditions, indicating that each LLM could faithfully reconstruct sentences by preserving key predicted tokens and generating co- herent fillers. However, performance declined slightly when too many tokens were provided, sug- gesting that redundant noisy inputs reduce fidelity. Including probability scores in the prompt produced only marginal improvements, implying that LLMs already infer token importance inter- nally. Overall, concise and accurate token sets proved more important than prompt complexity the choice of backend model, with all three LLMS exhibiting strong and comparable reconstruction performance.
24

---

Published as a conference paper at ICLR 2026
Table 13: Effect the semantic slot number on performance. K denotes the average number of semantic units per sentence in each dataset.
Clinical (K 4.365)
Chisco (K
4.656)
K
NUS
UMA
MUS
SRS
NUS
UMA
MUS
SRS
1
0.00
0.1493
0.8023
0.5123
0.00
0.0907
0.7503
0.4988
2
0.00
0.3433
0.8208
0.5891
0.00
0.2706
0.7921
0.5772
5
0.61
0.6290
0.8063
0.6499
1.59
0.5299
0.7977
0.5937
10 (default)
5.36
0.6596
0.8124
0.6651
5.75
0.5617
0.8009
0.6206
15
10.35
0.6559
0.8162
0.6624
11.38
0.5590
0.7989
0.6172
25
20.64
0.6418
0.8104
0.6683
21.23
0.5577
0.8012
0.6208
50
45.56
0.6482
0.8138
0.6636
45.76
0,5601
0.7954
0.6160
G ANALYSIS OF THE SEMANTIC SLOT NUMBER K
To examine how the number of semantic slots K influences the model performance, we con- duct sensitivity analysis on the Clinical and Chisco datasets. The average number of semantic units per sentence, denoted as K, is 4.365 and 4.656 respectively. We evaluate slot numbers K E {1, 5, 10 (default), 15, 25, 503, and report the three main metries along with the number of unused slots (NUS) during testing.
As shown Table 13 when K smaller than the actual semantic content K, the model's s perfor- mance is clearly constrained by the insufficient number of slots. In contrast, when K exceeds the required semantic capacity, the performance remains stable, demonstrating that the robustness the model the choice of slot number. This stability suggests that the designed "no-object" mechanism the semantic retriever can effectively determine how many semantic units are actually needed.
Table 14: Five-way topic classification on the clinical dataset.
Method
Accuracy
BRAINMOSAIC LLM selection
61%
5-class classifier
44%
Chance level
27%
H TOPIC CLASSIFICATION ON THE CLINICAL DATASET
During topic-guided data collection, our private Clinical corpus was annotated into five semantic categories: Healthcare, Sleep, Phone Use. Toilet, and Diet. To compare classification strategies, we conducted two experiments: (i) using BRAINMOSAIC to generate five candidate sentences for each sample and prompting LLM select the most likely topic (five-way selection), and (ii) directly training five-class classifier using the same Modern TCN backbone. As shown Table 14 the LLM selection based BRAINMOSAIC'S outputs achieved an accuracy of 61 %. significantly higher than the 44 % obtained by the direct classifier. Since the five topics are not evenly distributed. the chance level is approximately 27%. These results indicate that BRAINMOSAIC" intermediate semantic representations capture topic-relevant meaning more effectively than direct label supervi- sion. This further indicates that decomposing sentence into a set of semantic units enhances the model's 's ability to capture and express the underlying complex meaning.
25

---

Published as a conference paper at ICLR 2026
1
REGIONAL CONTRIBUTION ANALYSIS ON THE CLINICAL DATASET
D K
E
L
W
F
M
H
B
(a) view: Top
(b) view: Left
MUS UMM
0.5
04-
0.29
03
04
05
022
Contributions
(c)
(d)
Figure 3: Regional contribution analysis. (a,b) Spatial distribution of electrodes and SEEG chan- nels on the MNI template brain (top and left views). (c) Gradient-based channel saliency, with red indicating higher contribution. (d) Positive correlations (P < 0.01) between electrode saliency and single-clectrode decoding performance (UMA/MUS).
The spatial distribution of all implanted electrodes and recording channels used for decoding is shown on the standard Montreal Neurological Institute (MNT) template brain in Fig. 3 (a.b), from superior and left-lateral views. The coverage spans key temporal, insular, and limbic structures that are well established in the literature as supporting speech and semantic processing.
To quantify the contribution of different brain regions to semantic decoding, we employed two com- plementary analyses. First, we assessed how much semantic information can be extracted from individual recording sites in isolation. For each electrode, we trained and evaluated the decoder using only the corresponding SEEG channels as inputs, and measured decoding UMA and MUS. content available locally each site. Sec-
This provided an estimate of the ond, we examined how strongly each channel contributes to the model's decisions. We trained the model on the full multichannel SEEG input and then computed gradient-based channel saliency measure (Simonyan et al.) 2013 Schirmeisstrr et al. (2017). These saliency scores reflect the degree to which perturbing a given channel would affect the decoding loss, thereby quantifying its func- tional contribution within the learned decision function. The spatial distribution of saliency values shown in Fig. 3 (c).
Each channel was subsequently mapped its anatomical label using clinical annotations, enabling aggregation at both the channel and electrode levels. At the channel level, we observed particularly high saliency in the superior temporal gyrus and sulcus (STG/STS), which core hubs of the
26

---

Published as a conference paper at ICLR 2026
sadditory-lagguagee pathway (Hickok & Poeppel 2007 Friederici 2011 Dronkers 1996). At the
electrode level, averaging saliency across channels from the same electrode identified a small set of electrodes with markedly elevated contributions, predominantly located in the superior and middle temporal cortices. Most high-contributton electrodes (A, B, C) were located in the left hemisphere, with several informative sites in the right temporal lobe. This is consistent with the well-established
left-hemisphere dominance in semantic and speech processing (Hickok & Poeppel 2007 Binder al. 2009).
Finally, we examined the relationship between regional contribution and decoding performance. For each electrode, we computed the Pearson correlation between its average saliency value from the full-model analysis and its UMA and MUS scores from the electrode selection experiments. As shown in Fig. 3(d), saliency scores were significantly positively correlated with decoding accuracy for both metrics (P I 0.01), indicating that the anatomical regions most relied upon by the model during decoding are also those from which semantic information can be most effectively read out.
27

---

Published as a conference paper at ICLR 2026
J BRAINMOSAIC IMPLEMENTATION
EEG Encoder. The EEG Encoder processes raw EEG recordings into compact, context-rich neural representations. It consists of MCdemTTNN (Luo & Wang 2024) for low-level temporal modeling and a Transformer for global context integration and query-based representation. Mod- emTCN uses large and small convolutional kemnels in a multi-stage architecture capture both local patterns and Long-range temporal dependencies, producing robust and noise-tolerany features. Given an imput EEG sequence 2 E RxCX7 it outputs temporal tokens E I. RxNxx where B the batch size, C is the number of channels, T' is the temporal length, and N is the number of temporal patches. The Transformer combines these tokens with fixed set of learnable queries to produce K slot representations, each serving candidate semantic unit for the subsequent stages.
Text Encoder. The text encoder in BRAINMOSAIC defines the open, continuous semantic space V using pre-trained embedding models and truncated to 256 dimensions. To build interpretable and stable semantic units, we expand each word into a short explanatory phrase via an LLM and embed it to reduce ambiguity and improve generalization. Sentences are embedded directly for global alignment, while high-level attributes are inferred by the L.M.. For datasets without explicit word-level annotations,
apply simple tokenization, remove non-semantic function words, and embed the remaining content words. Similar embeddings are then clustered, merging variants such "apple" and "apples" into single unit. This process yields stable, open-vocabulary semantic unit bank U = u with embeddings E(u), forming the foundation for alignment and retrieval.
Algorithm Training of BRAINMOSAIC
Require: Dataset D
{ >))) epochs E; weights Acte, Aattry Àglobal, Arep; margin
Ensure: Final parameters 0.
1: for = 1 to E do
2: Lroral 5-0
3: for cach (Tb, 8b) E do
4: So, {E(us,) M/ E(ss), Zo, U TextEnnoderrss.) // text units, embeddings, global labels
X, + EEGGncoder((z))
{g po. &b, ] SemauticRetrrivee(X... Q) JC=
7:
6, + HungarianMatcher ((E(u,))), (io.))
(())
Hongarian + ΣØ= [to. sim sim((((n,,G)) ())a + ACr BCs Po.ee.t)
9:
Global (1 - sim simo)) 5o)) + dattr Ee CB(2) 2/10))
10:
I. {j: :5:tbn tb. 03: c.0) Tha
- Ene max sim E(u)) m)
11:
C retriever Humgorion Axlobal A Sloball
(1))
12:
Crotal + Liotal + (E/itriover + >ropp (c(())) (C(
13:
Backprop (retriieerr + Arep
(1) update partial 0
14:
end for
15:
EpochSummary: optional logging with Lroral
16: end for
28

---

Published as a conference paper at ICLR 2026
Algorithm 2 Inference and Evaluation of BRAINMOSAIC
Require: Dataset { su bo) unit bank U with embeddings {b(u)):
thresholds: existence cosine r; retricval k; generation count ==
Ensure: Per-sample predictions S.. So. Tb, and metries UMAs, MUS,, SRS..
I: for each (3e, 86) D do
2: Sb, ((( (ss,,, Zo +- TextEncoder(ss)
3: Xa EEGEncoder(zs)
4: (dus. po.5 Sb, {2 J + SemanticRetriever(X. Q)
5: ō, 4- HungarianMatchhrr (E((u,,) ) (06,) 1a = M3:: oo(1) = j]
2o.1 f (1) and %6,1 + Elu..))
UMA Teb
[sim(2o.) T) x 100%,
MUS,
1
Σ
sim (20,5: 26,5)
716
7: S: = [(u, (()) ] 3j. po.s > 8, u Top-k Non,:N
8: Prompt, + P(Sb, (26 });
G(Prompt,)
9: SRS, R Σ /
sim (im(B((T E(s)))
10: end for
K RELATED WORK
K.1
GENERIC LANGUAGE DECODING
K.1.1
CONCEPT DECODING
Early research intended decoding focused primarily on decoding short semantic units, such as individual words or simple phrases, from neural activity. Most approaches adopted closed-set clas- sification paradigms, in which participants either imagined specific words or phonetics (Nieto al.] 2022 Lopez-Bernal al. 2022), and processed externally presented stimuli in visual, auditory, or textual form (Wang et al. 2011 Liu et al.) 2009. Vidal et al.) 2010 Wilson et al. 2023). In some complex linguistic tasks, such as picture naming
cases, these paradigms were followed by synonym matching in monolingual and bilingual conditions (Correia et al.) 2013). Studies have re- vealed that the electrical potentials originating from the ventral temporal cortical surface in humans contain sufficient information to enable the spontaneous and near-instantaneous identification of subject's perceptual state (Miller al. 2016
Despite recent advances, short-concept decoding still encounters significant hurdles. Existing stud- ies generally focus on limited number of highly distinct categories. Decoding tasks often revolve around isolated nouns instead of context-integrated expressions, and visual stimuli (Wang et al., 2011 Liu et al. 2009) Vidal et al., 2010, Sabra et al. 2020) are predominant in these research paradigms. These limitations severely restrict the real-worke applicability of short-concept decod- ing. Since isolated concepts are difficult to combine into coherent expressions, the current approach struggles to meet the demands of actual language use.
K.1.2 SENTENCE TOPIC DECODING
With the emergence of large language models (Zhao al. 2023 Chang et al. 2024 and pre-trainne trained models, the scope of research has broadened. Initially centered on emotion classification and mo- tor imagery (MI) tasks, has now extended to word and sentence decoding tasks (Zhang et al., 2025a). Moving from single concepts to sentence-level representation introduces new complexities in both neural modeling and linguistic reconstruction. Evidence from intracranial recordings and neuroimaging studies indicates that sentence processing relies on distributed and interactive corti- cal architectures rather than being confined to traditional language areas. Key regions such as the inferior frontal cortex and posterior temporal cortex play complementary roles in integrating seman- tie information, while the ventromedial prefrontal cortex is engaged during later stages to support high-level contextual interpretation and decision-making (Murphy al. 2023
29

---

Published as a conference paper at ICLR 2026
In recent years, sentence-level decoding has attracted considerable attention. The DRYAD dataset [Broderick et al., 2018) records EEG data from participants listening to narrative speech, while ZuCo (Hollenstein et al. 2018) combines EEG and eye-tracking signals during reading. ZuCo 2.0 (Hollenstein et al. 2019) further expands the scale and diversity of participants. In the Chinese domain, ChineseEFG2 (Chen al. 2025) collects EEG data during aloud reading and story lis- lening, and Chisco (Zhang et al. (204)) focuses on both reading aloud and recall tasks. Although sentence decoding offers a more complete representation of language compared to single-word ap- proaches, it also presents persistent challenges. Most methods are unable to reconstruct coherent sentences, relying instead on forced classification into predefined categories or extracting phrase- level embeddings from spontaneous speech, which fails to capture the diversity of natural linguistic contexts. The scareity of large-scale, publicly available datasets further limits progress, while the integration of LLM- models remains unstable and inconsistent in practice (Shams et al. 2025 Lu et al. 2025).
K.1.3 SPEECH-BASED LANGUAGE DECODING
parallel line of research has explored speech- based decoding strategies, which link the coordi- nated activity of vocal-tract articulatory and motor-planning cortical representations (Silva et al.) 2024]. These approaches typically require neural recordings obtained during overt speech produc- tion, which severely restricts their applicability to individuals with impaired articulation. Although recent work has demonstrated the feasibility decoding during covert articulation, where partici- pants silently attempt speech production (Moses et al., (202)) the resulting decoding speed remains suboptimal and performance falls short of real-time requirements. This limitation is particularly evident in non-alphabetic languages such Chinese, which present additional challenges due the complexity of logographic characters and tonal variations. Moreover, existing saeech-motor-basee models are largely confined closed-set classification of syllables or tonal patterns during reading tasks Liu et al. 2023 Zheng et al. 2024), offering limited generalization and low efficiency.
By contrast, semantic concepts in the brain constitute modality- and language-independen entities that underpin human cognition. Their distributed neural representation remains relatively consistent across linguistic systems, suggesting that decoding approaches centered on conceptual semantics, rather than phonological articulation, hold greater potential for restoring communication in individ- uals with speech and motor deficits.
K.2
SET MATCHING
Set matching has emerged as key paradigm in object detection and related recognition tasks. Early approaches were built on predefined anchors or centers, where models predicted targets relative to initial guesses (Zhang et al.] 2020). However, their performance was highly sensitive to the design of these priors. Subsequent efforts introduced set-based losses (Redmon et al.) (0166), most notably through the use of the Hungarian algorithm (Kuhn, 1955), which establishes one-to-one assignment between predictions and ground-truth objects. However, such methods still relied on contextual cues, such bounding box coordinates, and embedded prior knowledge into the pipeline. More recent research explored recurrent detectors that directly predict object sets through encoder- decoder architectures combined with bipartite matching losses, paving the way for the fully end- to-end Detection Transformer (DETR) (Carion et al.) 2020). The development of DETR further advanced the set matching framework by demonstrating the feasibility of casting detection as a direct set prediction problem. More broadly, recent work has addressed the lack of positive samples and accelerate convergence (Dai et al. 2021), through strategies such as data augmentation (Fang eet al., 2024h and one-to- one-to-many training (Jia al. 2023). which enhance matching accuracy and improve model robustness. Inspired by DERT, we adopt set matching to treat EEG decoding as a structured prediction problem.
30


Reviews:
[{"review_id": "Qc6Hz8fPBz", "reviewer": "Reviewer_MrKn", "summary": "The paper proposes to view EEG/SEEG-to-language decoding as predicting a variable-size, order-invariant set of semantic units (SID), rather than fixed-class decoding or fully free-form generation. It instantiates this with BRAINMOSAIC, including i) an EEG encoder + query slots, ii) a semantic retriever that aligns slots to an open-vocabulary unit bank, and iii) an LLM that does semantics-constrained generation from those units. Experiments on three public EEG datasets (Chinese + English) and one private SEEG dataset, using UMA, MUS, and sentence-level similarity, aim to show: (i) set-style decoding is reasonable, (ii) continuous semantic space helps scalability, (iii) constrained generation improves fidelity.", "questions": "1.Is the 8:2 split done by subject or by trial on each dataset? Is there possibility of data leakage?\n2.Please add complete random baselins for UMA/MUS/SRS. Please describe the MUS_exp more clearly.\n3.Please add ablations where only one of the three design choices (set, continuous space, LLM-constrained decoding) is changed at a time, so we can see which part actually drives the gains?\n4.Please distinguish clearly between ChineseEEG and ChineseEEG-2 as they are two distinct datasets.\nSome references are missing. For example, there should be references for this statement: “Alternatively, a more recent direction seeks to enhance expressive capacity by mapping neural signals directly into the latent representation space of large language models (LLMs).”", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Interesting formulation: treating an utterance as a set of semantic units is a neat and fairly novel way to handle variable-length EEG semantics.\n\n2. Well-structured method: the three design principles (compositionality, continuity/expandability, fidelity) map cleanly to three modules.\n\n3. Reasonable experiment designs: the same idea is run on Chinese imagined speech, Chinese naturalistic reading, English reading, and a clinical SEEG case, which supports the claim that the approach is not tied to a single dataset.", "weaknesses": "1. **Train/validation split is underspecified:** the paper only mentions a unified 8:2 train–test split, but does not say whether this is by subject or by trial. Can the model guess which sentence it is by the sample length?\n\n2. **No true random / text-prior baselines**: the main metrics (UMA, MUS, sentence similarity) are not compared against (i) picking the same number of units at random or (ii) a text-only/corpus-frequency prior. This makes it hard to see how much of the score actually comes from EEG, especially since the method later calls an LLM. The paper reports `MUS_exp` but doesn’t clearly define it.\n\n3. Experiments don’t cleanly isolate the three research questions: the section is organized around “set is better,” “continuous space scales,” and “constrained generation is more faithful”. Several experiments change multiple components at once (set + LLM + thresholds), so it’s hard to attribute the improvements to the claimed factor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EVZzCFqkfd", "reviewer": "Reviewer_xT72", "summary": "The paper introduces Semantic Intent Decoding (SID), a novel framework for brain-to-language translation that models communicative intent as a set of compositional semantic units rather than relying on fixed labels or unconstrained generation.", "questions": "1.How was K (number of semantic slots) chosen per dataset? Was it tuned? How does performance vary with K?\n\n2.Would the model scale to larger vocabularies? What are the computational bottlenecks?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The paper proposes an intriguing motivation and introduces a novel perspective for brain-to-text decoding. By modeling intent as an unordered and variable set of semantic units, it moves beyond traditional fixed-label or sequential decoding paradigms, potentially offering a more brain-plausible representation of semantic processing.", "weaknesses": "1.Lack of robustness evaluation under input noise: the paper does not assess model performance under noisy or corrupted EEG inputs, which weakens the significance of the results reported in Tables 3, 4, and 5. Given the inherent noisiness of EEG signals, such evaluations are essential to validate the practical utility of the proposed method.\n\n2.Insufficient justification for LLM-based sentence generation: the use of LLMs for sentence reconstruction raises concerns about data contamination, especially if the test set sentences or similar phrasings were present in the LLM’s pretraining corpus. The authors provide no strong evidence (e.g., n-gram overlap analysis or controlled LLM ablation) to rule out this possibility, which undermines the credibility of the generation results.\n\n3.Missing comparison with standard generation metrics and SOTA methods: the paper does not compare with mainstream brain-to-text decoding methods using standard generation metrics such as WER, CER, BLEU, ROUGE, METEOR, or BERTScore. This omission makes it difficult to benchmark the proposed method against existing literature and assess its true advancement in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "as2N6WAbY6", "reviewer": "Reviewer_y3DJ", "summary": "The paper proposes Semantic Intent Decoding (SID) and a concrete model, BRAINMOSAIC, for translating EEG or SEEG into natural language by first decoding a variable set of semantic units, aligning them in a continuous text embedding space, and then reconstructing a sentence with an LLM constrained by those units. The pipeline comprises a set-matching Semantic Decomposer trained with a Hungarian loss, a Semantic Retriever that aligns slot embeddings to word and sentence embeddings and predicts global attributes, and a Semantic Decoder that prompts an LLM to generate text from retrieved units. The work targets semantic decoding during reading, listening, or imagined speech across Chinese and English datasets, plus a private SEEG dataset, and claims improvements over classification and unconstrained generation using embedding based metrics UMA, MUS, and SRS", "questions": "1. For continuous corpora, how did you prevent leakage between train and test when the same long passage is segmented into sentences?\n2. Can you report error bars and conduct significance tests comparing BrainMosaic to baselines?\n3. How sensitive are results to the choice of embedding model? What happens with random embeddings as an ablation?\n4. For multi-subject datasets, what is cross subject variance? Can models trained on one subject decode another subject’s neural activity? How good is the generalizability\n5. Did you perform any electrode selection or analyze which channels/regions contribute most to semantic unit prediction?\n6. Table 5A shows declining UMA with vocabulary expansion. At what vocabulary size does performance approach random? Can you characterize this scaling law?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The three principles (compositionality, continuity, fidelity) are well-motivated by linguistic and neuroscience evidence.\n2. Interpretable pipeline: slots -> ranked retrieval -> prompted gen beats blackbox E2E.\n3. Consistent empirical gains across multiple datasets and baselines with both concept level and sentence level metrics.\n4. Comprehensive comparison with other relevant baselines.\n5. Extensive supplementary material includes dataset details, baseline descriptions, and sensitivity analyses.", "weaknesses": "1. Lack of qualitative examples of reconstruction quality. Without concrete examples, readers cannot verify whether predicted semantic unit sets are genuinely interpretable or noisy/scattered, (b) how well do the quantitative metrics match real semantic correctness. This falls below the standards in neuro-decoding literature where it's common to show a few samples of the proposed model's input-output vs baseline\n2. Lack of comparability to standard text metrics. While semantics-first metrics are appropriate here, it's also important to have surface level metrics common in the literature such as BLEU score, WER for surface overlap, and BERTScore for semantic similarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)