NEED: Cross-Subject and Cross-Task Generalization for Video and Image Reconstruction from EEG Signals
Authors: Shuai Huang
Huan Luo
Haodong Jing
Qixian Zhang
Litao Chang
Yating Feng
Xiao Lin
Chendong Qin
Han Chen
Shuwen Jia
Siyi Sun
Yongxiong Wang
Venue: NeurIPS 2025 poster ()
Keywords: EEG
Cross-Subject
Cross-Task
Video Reconstruction
Image Reconstruction
Primary area: neuroscience_and_cognitive_science

Abstract: Translating brain activity into meaningful visual content has long been recognized as a fundamental challenge in neuroscience and brain-computer interface research. Recent advances in EEG-based neural decoding have shown promise, yet two critical limitations remain in this area: poor generalization across subjects and constraints to specific visual tasks. We introduce NEED, the first unified framework achieving zero-shot cross-subject and cross-task generalization for EEG-based visual reconstruction. Our approach addresses three fundamental challenges: (1) cross-subject variability through an Individual Adaptation Module pretrained on multiple EEG datasets to normalize subject-specific patterns, (2) limited spatial resolution and complex temporal dynamics via a dual-pathway architecture capturing both low-level visual dynamics and high-level semantics, and (3) task specificity constraints through a unified inference mechanism adaptable to different visual domains. For video reconstruction, NEED achieves better performance than existing methods. Importantly, Our model maintains 93.7% of within-subject classification performance and 92.4% of visual reconstruction quality when generalizing to unseen subjects, while achieving an SSIM of 0.352 when transferring directly to static image reconstruction without fine-tuning, demonstrating how neural decoding can move beyond subject and task boundaries toward truly generalizable brain-computer interfaces.

NEED; Cross-Subject and Cross-Task Generalization for Video and Image Reconstruction from EEG Signals
Shuai Huang"
Huan Luo School of Electrical and Computer Engineering University of Shanghai for Science and Technology
School of Optical- A and Computer Engineering University of Shanghai for Science and Technology Shanghai, China hhsavethxorrdd@@aail com
Shanghai, China luchuan2016e163. com
Haodong Jing Institute of Artificial Intelligence and Robotics Xi' an Jiaotong University Xi an, China
Qixian Zhang
Litao Chang
School of Computer
School of Optical-Electrrcall and Computer Engincering University of Shanghai for Science and Technology Shanghai, China 1ilit07148outloon com
Science and Technology Tongji University
Shanghai, China
Jinghd@stu.25 tu. edu.
zhanggx&tongji edu.
Yating Feng School of Optical-Electricel and Computer Engineering University of Shanghai for Science and Technology Shanghai, China 243370889 @st. usst.uuu....... edu. cn
Xiao Lin
Chendong Qin School of Biomedical Engineering
School of Optical-Er Electrical Optical-Electrical and Computer Engineering University Shanghai for Science and Technology Shanghai, China 242260497 @st usst. edu, cn
Shanghai Jiao Tong University
Shanghai, China gcd25154273058163.
Han Chen
Shuwen Jia School of Optical-Fllettrcal and Computer Engineering University of Shanghai for Science and Technology Shanghai, China 231 231260077 @st. uast. edu.
Siyi Sun Computer Science University of Oxford Oxford, United Kingdom siyi. siyi. sunccs. ox. ac uk
School of Opticcl-Electrical and Computer Engineering University of Shanghai for Science and Technology Shanghai, China 221 221240068 @st. edu. cn
Yongxiong Wang School of Optical-Eleetrical and Computer Engineering University of Shanghai for Science and Technology Shanghai, China syxiongousst.co edu, en
'https://hharr son github.io/ "Correspondig author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Abstract
Translating brain activity into meaningful visual content has long been recognized fundamental challenge in neuroscience and brain-computer interface research. Recent advances EEG-based neural decoding have shown promise, yet two critical limitations remain in this area: poor generalization across subjects and con- straints to specific visual tasks. We introduce NEED, the first unified framework achieving zero-shot cross-subject and cross-task generalization for EEG-based visual reconstruction. Our approach addresses three fundamental challenges: (1) cross-subject variability through Individual Adaptation Module pretrained on multiple EEG datasets to normalize subject--ppcifice patterns, (2) limited spatial res- olution and complex temporal dynamics via dual-pat Qual-pathway architecture capturing both low-level visual dynamics and high-level semantics, and (3) task specificity constraints through a unified inference mechanism adaptable to different visual do- mains. For video reconstruction, NEED achieves better performance than existing methods. Importantly, Our model maintains 93.7% of within- vithin-subbect classifica- tion performance and 92.4% of visual reconstruction quality when generalizing unseen subjects, while achieving SSIM of 0.352 when transferring directly to static image reconstruction without fine-tuning, demonstrating how neural de- coding can move beyond subject and task boundaries toward truly generalizable brain-computer interfaces.
Introduction
X Cross-Subject
X Cross Task (Unified Model)
Unowen Subo-1 Ulowen
New Trakk Sell-Stb->
nesudmation
Truining Plase frem Subjehh)
Tuilig Phose Ulizame Seljede)
Truining Plone (Vibe Rocandracioo)
Tediy Ploe O'mage Beaestracdee)
Cross Pross-Subject & Cross Task
Sub1i-Sub20
ALL Sub
NEED
NEED
SEED-DV
THINGS-EEG
The dual challenge in neural decoding. Current models fail both to generalize to unseen subjects (left) and transfer between video and image reconstruction tasks (right). Our NEED framework addresses both challenges simultaneously.
Figure 1:
Neural decoding for visual reconstruction from electroencephalography (EEG) signals represents critical frontier at the intersection of neuroscience and artificial intelligence 2 This emerging technology offers profound implications across multiple domains: enabling new communication pathways for paralyzed individuals [3] 4) providing unprecedented insights into human visual
perception [5] (), advancing brain-computer interfaces 7 g and revealing fundamental cognitive
mechanisms
EEG signals offer distinct advantages for neural decoding despite their limited spatial resolution com- pared to other neuroimaging techniques [T]. Their non-invasive nature, high temporal resolution, portability, and relatively low cost make EEG particularly suitable for real-world applications [3] The field of EEG-based visual decoding has advanced significantly in recent years, progressing from
simple text reconstruction [4 [5] 16 17] to increasingly complex visual content [88
19
Early work
focused on classifying basic visual categories and reconstructing simple shapes 220 21 while later
research enabled reconstruction of static images with recognizable semantic content 22
23
24 25
Most recently, pioneering studies have demonstrated the possibility of reconstructing dynamic videos from EEG signals (26 27 representing a crucial step toward decoding the brain's perception of coherent scenes and motion 29 30). This progression toward decoding continuous visual stimuli

---

holds profound implications for understanding how the brain processes dynamic visual information
and potentially recreating subjective perceptual experiences 31 32].
Despite recent advances in reconstructing visual content from EEG signals, significant limitations
persist that hinder practical applications, illustrated in Fig. | (1)Current neural decoders
struggle with cross-subject generalization [33] 34 35 36], demonstrating dramatic performance deterioration when models trained on specific individuals are applied new subjects [37 38 This challenge arises from the inherently individualized nature of neural activity patterns, which vary due anatomical differences, electrode positioning inconsistencies, and unique functional organizations of visual processing regions [39 40). The resulting models often capture subject -abjecc-specific EEG patterns rather than generalizable neural representations of visual content [41] (2)Siuultan,ously
existing approaches remain constrained by task-specificityy [42]. requiring dedicated frameworks
45 Models designed for static image reconstruction prove ineffective for dynamic video decoding without substantial architectural modifications and retraining. and the reverse limitation equally applies [46] 47 This rigid task-depyndyney necessitates developing separate optimized frameworks for each visual decoding scenario, severely restricting scalability and real-world implementation. Cross-subject variability and task-specificity represent major challenges among several important bottlenecks in advancing neural decoding toward practical applications.
for different stimulus types 43
To address these limitations, we introduce NEED (Neural Decoding with Enhanced Extensibility and Diversity), a unified framework for cross-subject and cross task EEG-based visual reconstruction.
NEED tackles those challenges in EEG decoding 48
through three key innovations: (1) an Individual Adaptation Module that normalizes subject-specife patterns, (2) dual-pathway architecture capturing complementary visual information, and (3) a unified inference mechanism adaptable across tasks. Our experiments demonstrate NEED's ability to generalize to unseen subjects [52 53 36 37 and transfer between video and image reconstruction tasks 42 44 45)
10
49
27
50
ST 35]
without
neural decoding toward practical brain brain-computer interfaces that work across diverse individuals and visual experiences.
In conclusion, our contributions can be summarized as:
We propose the first unified framework achieving zero-shot generalization across both EEG decoding tasks (image/video) and subjects.
We develop DSGNet, dual-stream EEG encoder capturing both spatial patterns and temporal dynamics of neural signals.
We introduce novel subject-adaptive module enabling robust multi-dataset knowledge transfer.
We design unified inference mechanism that flexibly integrates multiple guidance signals for adaptive reconstruction across visual domains.
2
Method
For more related work, see the Appendix
A
2.1
Framework Overview
We introduce NEED, unified neural decoding framework with zero-shot cross-subject and cross-tast generalization capabilities for visual reconstruction from EEG signals. As illustrated in Fig. 2 NEED comprises three principal components: (1) Cross-Subject Adaptation with pretrained Individual Adaptation Module (IA.) that addresses subject variability, (2) Dual-Pathway Understanding with parallel Perception and Semantic modules that overcome limited spatial resolution and complex temporal dynamics, and (3) a Unified Inference Mechanism that adaptively generates reconstructions across different tasks. The complete training and inference algorithm of our NEED framework is detailed in Algorithm in Appendix
2.2 Cross-Subject Adaptation
Individual Adaptation Module (IAM) To align heterogeneous EEG patterns across subjects, we propose the Individual Adaptation Module (IAM), a specialized architecture that transforms subject- specific signals into normalized representation space. The core of IAM consists of two complemen-
3

---

Multi-datase Pretraining
Inference
IAM
?
DEAP
Sub 05
Sab 11
PU Perception Understanding
VAE Deceder
Subd1 SubID
IAM
SEED DV
Net
Decader
2019
Tas WekkAdeeeer
L DDC
VAE
DFE
MLP
Dynamic
ConboiNett
SU: Semantic Understanding
dog with big eyes and black and white face
CLIPA VIT
L HBC
Prorpe Depicttae
DSG Net
SAED-DV
eyea
LLR
LesA
BLIP-2
A dog with big and a black and white face
CLI-TTETT Encader
Frozen
Keyframe
Predicted Keyframe
Pine-tuning
mmbeddin
Figure 2: Overview of the NEED framework architecture. (1) Cross-Subject Adaptation with the Individual Adaptation Module (IAM) pretrained on multiple datasets to normalize subject-specific patterns; (2) Dual-Pathway Understanding with parallel Perception (PU) and Semantic (SU) modules that capture complementary visual information; and (3) a Unified Inference Mechanism that adaptively generates reconstructions across different tasks.
tary components: (1) subject-aware attention mechanism that captures individual characteristics, and (2) subject-invariant transformation that extracts shared neural patterns. Detailed architecture specifications and implementation details are provided in Appendix C.I
For given subject their EEG representation E; E ggxx is processed through multi-head attention mechanism augmented by a learnable subject embedding s; E IR:::
Znorm Σ wh (a
H
(E, + s,U s²U,) (E, + +²UU)²
(EWW))
+ (E, Rs,
(1)
dd
Style injoction
Attention output Zaeen.
where I's applies feature modulation guided by subject-agnostic knowledge Qs and dataset-specific embedding d, to accommodate different experimental paradigms, To enhance cross-subject general- ization, IAM incorporates a hierarchical adaptation strategy that progressively normalizes signals from low-level characteristics to high-level patterns with Z finat = aj Ziggil + Qu Z sppectral 0. Zteporrraal + 02 ZnZnorm where coefficients are learned adaptively.
Multi-Dataset Knowledge Transfer IAM is pretrained on multiple diverse EEG datasets- DEAP, and EEGEYENET capture generalized neural response patterns across subjects with varying cognitive states. During pretraining, we optimize a multi-objective loss CuntioonCrretracc = afrecom + &Badd (0,, (p,0c) 0c) + DD kL Zormm (S) p( p(Zno)))) where Lyecon ensures reconstruction quality, Lado implements adversarial subject disentanglement, and the KL divergence term minimizes subject-specific information in the representations.
2.3 Dual-Stream EEG Encoder
Our framework's 's foundation a specialized dual-stream EEG encoder (DSGNet) that addresses limited spatial resolution and complex temporal dynamics in EEG signals through parallel processing

---

streams. This architecture efficiently extracts complementary information from both the topographical distribution and temporal evolution of neural activity. Detailed architecture specifications and implementation details are provided in Appendix C.2
Spatial Stream Processing The spatial stream leverages the inherent topographical structure of EEG recordings by representing electrode positions in a spherical coordinate system. We employ
Em=--m=-10 -t.==1.,mp/ 4 "mn,y (0,, øi) where Y represent spherical harmonic functions and ct,m are learnable coefficients. These spatial embeddings are further refined through a graph convolutional network that models inter-electrode relationships via F o( A Fi!!) "spatiat W(!)), where = A + IN incorporates self-
spherical harmonic embeddings to capture spatial patterss E Eppatial
connections into the adjacency matrix.
Temporal Stream Processing The temporal stream captures dynamic neural responses across multiple time scales using generalized Riemann-Liouvill fractional transformation:
/ X(s) RsT (X), T(a) (1-8) ds,
{at,.... am
This multi-scale approach with parameters E {0.2, 0.4, 0.8) enables capturing both rapid neural responses and slower contextual dynamics. The transformed features are processed through dilated temporal convolutions with receptive field size d()) 2 model dependencies across various temporal ranges.
Cross-Stream Integration We implement a cross-stream attention mechanism that enables dynamic interaction between spatial and temporal representations Z = o [E spatiool Etemporal; Zeroze] bg) O tanh ( [EEpattial Etemporal; Zeross] + by) where Zeross represents cross-attention features computed via Zeroes MultiHead(Q
Espetion Q K Eteeporai W V Exemporal WV). To enhance robustness against common EEG artifacts, we apply multi-dimensional masking E.
E @ M spatiaa @ Mtempprra C M frequeney.
2.4 Dutl-wathway Understanding
Perception Understanding Module The Perception Understanding (PU) module extracts low-level visual dynamics essential for accurate reconstruction by integrating dual-stream encoder outputs with advanced temporal modeling. The module processes EEG signals through a cascaded architecture where Zp = V ru Pene (IAM(E) E). O Spaaio--teny patterns from normalized EEG features, and V PU providing task-adaptive processing. Our key inno- vation is the Dynamic Feature Extractor (DFE) with integrated Spatiotemporal
Oeask), with &enc extracting spatial-tempora
Recurrent Network (SMARN) that decomposes visual dynamics through multi-level processing:
Fm F.
Fmotion (TaD (V)). Om))
Fspatiol (V)). O.)
SMARN (V)). H1-11
where Fm encodes motion patterns, F. represents scene structures, and F. models temporal coherence through our novel SMARN architecture. H. = a (W,, [v, H M7--1,M]]-b, Mb)- with M, = =>; ag Mi 3. V, formulates hierarchical memory representations with attentional weights ag 3g focusing relevant temporal features. To establish effective correspondence between neural signals and visual dynamics, we employ a Bidirectional Dynamic Contrastive (BDC) alignment mechanism. Given EEG embeddings Zo and enhanced visual features V1, we compute alignment using:
exp(sim Zi. v/)/T) Σ. exp(sim(Z, V/T/T)
LBDC
Wiy log
+ 2 -.VII
(4)
Here, Wiy represents learnable attention weights for feature importance, sim(..) .) is cosine similarity, controls distribution sharpness, and V, denotes temporal gradients. This formulation ensures aligmment in both feature space and temporal dynamics, enabling the model to identify neural patterns corresponding to specific visual elements while maintaining consistent temporal evolution.
Semantic Understanding Module The Semantic Understanding (SU) module operates parallel with PU to extract high-level conceptual content. The module transforms EEG signals Z,
5

---

Asu ( Eadapt (IAM(E), E), LLM)), PLaseeCcottezt) conteztt) where Eadapt bridges neural signals and semantic concepts via CLIP- ViT-G/14 keyframe features. Semantic alignment uses Hierarchical Semantic Contrastive loss across abstraction levels:
on (-108 exp(Z'- (2/TT) Σn exp(Z, Cl/T)) +
Lusc
+.-1(((Z) [[((Z)) -((C)
(5)
1=1
where L represents semantic levels from concrete to abstract, and G(.) computes the Gram matrix capturing semantic structure. We integrate language model capabilities via Z3 with Cross-modal Semantic Aligmment LOSA = (Zan + Cxu(23, T" d Lalign (Z3, K Keyframe) where Leont is attention- Waj contrastive learn- ing between Z3 and text embeddings T, LK is KL divergence between their distributions, and Lalign ensures alignment with BLIP2( f frame) embeddings. The enhanced embeddings Z. = Fdif Za) condition SDXL unCLIP to generate predicted keyframes, enabling high- quality reconstruction across diverse visual tasks without task-specifie training.
Fatar(Fatpperr(4
2.5 Unified Inference Mechanism
Adaptive Task Conditioning We implement an adaptive conditioning mechanism that modulates the reconstruction process based on the target task Ziosk = TaskAdapter((Z.:: Ze], T(), where Te indicates either video or image reconstruction. This mechanism automatically adjusts the processing pipeline based on the reconstruction target while maintaining a shared parameter space.
Multi-Guidance Integration The integrated conditioning is computed (eerfolaeeKKy + 3g CeexCEmbedCaatiinnnnnn + 7/t Dympatem + 5; Zeey- where Caption Fusion fused textual description combining BLIP-2 generated captions from predicted keyframes and complementary semantic descriptions from EEG embeddings processed through a language model adapter. This dual-source caption fusion enhances semantic richness by in- corporating both visual and neural information streams. The adaptive weights [ar, 3,, Yo, 6,] softmax (z)) dynamically adjust each guidance signal's is contribution based on tasks.
C₁ Oz
Unified Cross-Tass Generation Our framework achieves zero-shot cross-task generalization by leveraging shared neural patterns across visual modalities through a task-adaptive diffusion archi- tecture. The diffusion process is defined as 24 ~ >N I), IT-1 ( f, C22 T₁) + or where task encoding T. E (Tusdeoy Tamage explicitly differentiates between video and image processing, enabling single model to adapt without structural modifications. For cross-task adaptation, we introduce conditional layer modulation by (2e,,, 4, Ct Ct, Tt) = 7(T)) @ Norm(h--1) + (T) where (TT)) and (Tz) dynamically transform feature spaces according to task requirements. This design creates a shared semantic representation space where (Evieo)) RO Femage (Eimage) ). When processing image-related EEG signals, the model automatically configures the diffusion pathway by adjusting conditional signals while keeping all parameters fixed. Video reconstruction activates temporal attention modules with weight Oltemp a a(W, Trideo). while image reconstruction shifts emphasis to spatial detail enhancement with appat = o(W Temage) while effectively zeroing temporal components Ctemp O. The model adapts feature representation distributions pT.) enabling conditional transformation P( (at, Twiden) # P( (0,, Timage) without architectural changes.
3 Experiments
3.1 Datasets
Our NEED framework leverages multiple EEG datasets to enable zero-shot cross-subject and cross-task generalization. We use SEED-DV (26] as our primary training dataset, containing EEG recordings from 20 subjects watching 1,400 video clips across 40 visual categories, recorded with a 62-channel cap at 1000 Hz. For IAM pretraining, we utilize SEED [54,, DEAP (55], and EEGEYENET [56), providing diverse data for robust subject-invariant representations. For cross-task generalization evaluation, we test on THINGS-EEG [57]. which contains recordings from 10 subjects viewing 22,000 static images across 1,854 object categories. Full dataset details and preprocessing provided in Appendix D

---

3.2 Experimental Settings
We preprocess EEG data with standard filtering (0.5-40 Hz) and normalize sampling rates to 200 Hz across datasets. Our framework handles channel configuration differences across datasets through the IAM module, using a cross-dataset adaptation strategy detailed in Appendix C.3 Training proceeds in three stages: IAM pretraining on multiple datasets for subject-invariant representations. dual-pathway architecture training on SEED-DV, and unified inference mechanism fine-tuning, We evaluate using perceptual metrics (SSIM, PSNR), semantic metrics (CLiP-based classification), and temporal coherence measures (CLIP--ecc FVD). Performance retention rate quantifies generalization performance. Detailed implementation specifics are
capability relative authin-subject/witinn
provided in Appendix E
4 Results
4.1
Visual Perception Benchmark
We evaluate DSGN EEG encoder used in our NEED framework) on visual perception EEG data collected during subjects viewing various stimuli. Table compares DSGNet against established EEG classifiers and recent deep learning approaches. DSGNet consistently outperforms all baselines across all tasks with statistical significance (p- < 0.05). The improvement particularly pronounced in challenging multi-class scenarios, with relative accuracy gains of 70.9% and 23.9% for 40-class and 9-class top-1 classification respectively. Even in binary tasks where conventional methods approach higher accuracy, DSGNet maintains 7-10% relative improvements.
Table Average classification accuracy (%) and standard deviation across all subjects with different EEG classifiers on visual perception tasks. The star symbol (*) represents results above chance level with statistical significance (two-sample t-test: p < 0.05).
Multi-clus Classification
Bimary Clussifictionn
Methods
40-c lop-1
40-c tup-5
9:c top-1
9-0 3
Color 20.57
Fas/Slow 50.00
Numbers 65.64
Euman Face
Human
Chance level
2,50
12:50
11.11
33.35
62.25
71.43
SVM (PSD) SC
5.19/2.81*
19. 19.02/3.74
21,31/2.97
53.56/1111
64.15/1.22
58.94/221 21
70.91/1 84
MLP (PSL)
6.20/3.02
1891/5.94
21.59/3.00"
49.86/3.78*
22.0233277
55. 15/1.222
64.480092
63.94/1.33
71.74/1.66
GLANNe (PSD) R75
(NIPS 24]
6,23/2.91*
189985.62
21.69/2220
50.034,11 10
26.40299*
55 1142//1332 324
64.688092
64,22/1.43
72,27/1 57
SVM (DE)
1K/22.80 80*
19.05/3.394
21/07/888
53.34/1.25
63.62/1.73
57.82/3.50 50
0.25/1
MLP (DE)
6.12/3.08*
19.00/5.11
21.17/326
49.1044 94
25.91/3.274
54.7611.2**
64.100.70
63.41/1.57
71.74/1766
GLMNNet (1N))) 20
INIPS 24]
6.183.18
19.12/6.7
21.34/3.34"
49.55/4.57
26.15/3:249
55.06/1.20*
64,25/0.44
63.63/1.00
12,2W1 28
Raw EEG Signals
Neel
[HBM 171
5,59/221
169934.06*
21.40/1.66
49.62/2.34
27.00209*
56.62/1.77
66.15/089
61,77/154
73.21/1 52
DeepNelG
[RRM
4561152"
14.30/3.2"
20:27/1.25
48.96/1.99"
26.3711955
55.420.59*
65.7180.24
61,58/398
72,860 40
BBGiNe [0
[INE19]
4.64/0.6"
14.25/187
19.63/0.81
47.04/1.45*
25.46/1.31
$1.99/200
64.6780000
61.37/131
72.380098
[SMC'20)
684/2.15
18.363.95
18/16/676
50.62/2.38
27.561.63"
56.8/1.54
66.5480.77
65-42/135
73.51/1 28
Confoemer 45
49V1.5T
153644.44 44
30.92/0.96*
49.25/1 APr
27.5N1.37
55 82/08
65.7500.6
64.9611 14
73.000 85
BraVL [
7.052.21
19.0513,75
22.53/1.86
51.07/211
28.12/1.52
57.88/1.64*
66.92/0.3
65.76/1338
73.65/1
EBGP 13
[HIP5'24]
6.95/63-
18851L111
22.10/1.96*
50.55/2.33*
27.95/1.73*
57.10/1.85*
66.75/094
65.35/1.42
73.25/1
TSCom 19
[ICER 24]
4.9201999
15.09/2.11
20.00/1.011
47.76/1.51
26.89183
55.320.99
65.390.41
64.391144 1447
62.680067
GLMNee22 2b
[MIP5/24]
6.2033.0
17.35/424
21.93/1.87
50.01/2.52*
27.33/1.45*
57,35/1.96*
66.21/0.91
65.10/1.45
73.34/1
DSGNet (Ours)
14.233.12
25.75/4.44
29.93/2.07
58.01/2,72
35.33166*
65.35/2.18
74,21/1.11
73.16/1.5
81.34/1.11
Relative Improrvennn DSSEN mee best bussllie
+101,8%
+35.2%
+32.8%
+13,6%
+25.6%
+12.9%
+10.9%
+11.2%
+10,4%
4.2 Video Reconstruction Results
Table[[present our framework's 's performance across video and image reconstruction tasks compared to existing methods. For SEED-DV video reconstruction, our approach achieves superior semantic accuracy (0.898 in 2-way classification) while maintaining competitive visual fidelity (SSIM: 0.356). Ablation studies reveal the Individual Adaptation Module as most critical, with performance dropping dramatically without it (0.898 0.6922 in 2-way accuracy). Both PU and SU pathways contribute complementarily to overall reconstruction quality. Fig. shows cases representative reconstructions. Successful cases demonstrate our framework's 's ability to capture both semantic content and temporal dynamics across diverse scenes. Failed cases (red) reveal limitations with complex motion patterns and lighting conditions.
7

---

Table 2: Comparison of visual reconstruction performance across EEG, fMRI, and MEG-base methods for video and image reconstruction tasks. More detailed ablation experiments are shown in the Appendix F.I.I and Appendix F.1.2
Video Reconstraction
Image Reconstruction
Model Variant
Videe-bared
Frame based
Lowelevel
High-
2-way1
40-wayt
2-way?
4ll-aay
PicCur?
SSIM
lasepion?
Av
EDO2sideo (EBG) 26
0.80210000
00000000
0.09
12202.00002
0.3001:0.33
Wing (M
0.773+0.01
0,713-0.
0. 0.118+000
Kupershmide (MR)
(50 way?
Mint-Vidra (MRR) W
0.809-10000
0. (50-wag))
0.706-0.00
0 174
00.03 (50-way)
0. 0:171+0008
NeuroClips (MR)
way?
(50 way?
ATM-S
0.155
013300
0.786
0.770
0.5K2
CuputicnCatuuer
0.500
Mind's Elye (M
0.298
0.317
0.91
0.929
0.382
META MBG EM
0,683
0.615
Model (Ours) (ERG)
0.88
0.839 0.83
1.20
0.355 02
0.162
0.795
0.742
0.755
0.642-0001
0:00++002
0.587-0.04
0.85-10.02
0.054
0750
udo PU
0,723=000
0:203+0002
0.678=0008
011440003
0.011:003
0L022
0.116
0.563
0.526
0.694
SU
0.804:0:0
0.1660005
0.756-0.0
0.00200.02
0:28::002
0.127
0.153
0.654
0.613
0.621
0.845
udo Di
0.836=0000
0.327+0002
0.748-0.04
0227+00.03
0,284-003
0.109
0.124
0.623
0.535
0.690
ConteolNer
0.005
0.31210005
0,067:0002
0.19319.02
0,26310003
0.123
0.642
0.992
0.655
Cupcion
0,03
0.29610.02
0.775
0.208
0.2711:002
0.115
0.607
0.584
0.638
GT
Ours
GT
Ours
Ours
Ours
GT
Ours
Figure 3: Sample video reconstruction results from our NEED framework on the SEED-DV dataset. Red: Failed cases.
4.3 Cross-Subject Generalization and Temporal Analysis
Fig 4 visualizes NEED's cross-subject generalization capabilities as training subject count increases. Classification accuracy (Figure 6a) shows consistent improvement across all metrics, with 2-way accuracy reaching 84.1% 1% using 19 subjects. Quality metrics (Figure 6b) follow similar trends, with CLIP-pce improving most substantially (0.128 +0.682) while FVD decreases (584.7-117.55) Notable performance jumps occur between 5-10 and 10-15 subjects, suggesting threshold effects in capturing generalizable neural patterns. Fig displays temporal analysis across subjects, revealing consistent information accumulation patterns despite individual variations. Further supporting analyses are available in Appendix F2 and Appendix [:5) These results demonstrate our framework's 's
robust ability generalize across unseen subjects while preserving temporal dynamics.
4.4 Cross-Tase Generalization
Table 2 presents our framework's zero-shot transfer capabilities on the THINGS-EEG static image reconstruction task. Without fine-tuning, NEED maintains competitive performance across all metrics compared to task-specific models, achieving the highest SSIM (0.352) among EEG--based methods. Task conditioning proves critical for enabling cross-task transfer, with performance dropping

---

a
b
100
Video 2-way
Video 40-way Acc
0.7
SSIM
600
80
Framme 2-way Ape
0.8
Frame 60-way Acc
500
CLIP-pod SSSM
Accuracy
0.4
400
0.3
300
0.1
0.0
17
19
11
19
Number of Training Subjects
d
Number of Training Subjects
Accuracy
2 SEM
(2.5%)
top-1 accuracy
40-c accuracy
0
200
400
600
800 1000 1200 1400 1600 1800 2000
200 400
600
800
1000 1200 1400 1600 1800 2000 End time (ms)
End time (ms)
Figure 4: Cross-subject analysis on the SEED-DY dataset. (a) Classification accuracy vs. training subject count. (b) Reconstruction quality metrics vs. training subject count. (c) Individual 40-class top-1 accuracy curves for all subjects. (d) Average accuracy with standard error.
dramatically without this component (SSIM from 0.352 to 0.063). The PU and SU modules contribute complementarily visual details and semantic accuracy, demonstrating our architecture effective handling of both dynamic videos and static images through task task-adaptive mechanisms.
GT
Ours
GT
Ours
Figure 5: Zero-shot cross-task generalization results for static image reconstruction on the THINGS- ERG dataset. Red: Failed cases.
Fig- shows representative examples of zero-shot image reconstruction. Successful cases demonstrate our framework's is ability to capture key visual features and semantic content despite training solely on video data. Failed cases (red) reveal current limitations in reconstructing fine details and complex object arrangements when transferring to static image tasks.

---

4.5 Brain Region Contribution Analysis
Different brain regions exhibit distinct contributions to neural decoding. Fig 6a shows classification accuracy when ablating specifie regions, with occipital areas showing lowest performance individually (8.1%) while full-brain integration yields highest accuracy (14.3%). For video reconstruction (Fig 6p), occipital-only signals capture basic scene structure but miss object details, while adding temporal regions improves semantic content. Dynamic scenes require parietal involvement for effective motion representation. Fig 66 illustrates the electrode distribution across five brain regions used our analysis. Similarly, static image reconstruction (Fig (dd) demonstrates quality improvement with combined occipital and temporal signals, while adding parietal regions further enhances structural fidelity. These findings align with established visual processing pathways, supporting our dual- pathway architecture design.
b
GT
Tempord Brai LED
d
Frontal
Temporal
Central
Paristal
Occipial
Figure 6: Brain region contribution analysis. (a) Classification accuracy with different brain regions ablated(SEEDD-VV) (b) Video reconstruction using different electrode combinations for static and dynamic scenes (SEnes(SEEDDDV) (c) EEG electrode distribution across five brain regions. (d) Static image (cconttruction((HINGGS
Conclusion
In this paper, introduce NEED, unified framework addressing cross-subject variability and task-specificity EEG-based visual reconstruction. Our approach maintains substantial performance when generalizing
subjects and enables zero-shot transfer between video and image tasks. This work demonstrates how neural decoding can move beyond subject and task boundaries toward truly generalizable brain-computer interfaces. We provide further discussions on broader impacts and future work in Appendix G and respectively.
Limitations
Despite NEED's advances, certain challenges remain in EEG-based visual reconstruction. Reconstrue- tion quality, while improved, still falls short photorealistic fidelity for scenes with complex lighting and fine details. The framework currently performs optimally with higher-density EEG recordings, presenting challenges for deployment with ultra-portable consumer devices. These represent broader challenges in neural decoding rather than specific weaknesses of our approach.
10

---

References
[1] Simone Palazzo, Concetto Spampinato, Isaak Kavasidis, Daniela Giordano, Joseph Schmidt, and Mubarak Shah. Decoding brain representations by multimodal learning of neural activity and visual features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1 1))3333- 3849, 2020.
[2] Changde Du, Kaicheng Fu, Jinpeng Li, and Huiguang He. Decoding visual neural representa- tions by multimodal learning of Analysis and Machine Intelligence, 15(9):1770-11777 2023.
features. IEEE Transactions Pattern
[3] Francis R Willett, Erin M Kunz, Chaofei Fan, Donald T Avansino, Guy H Wilson, Eun Young Choi, Foram Kamdar, Matthew F Glasser, Leigh Hochberg, Shaul Druckmann, et al. A high-performance speech neuroprosthesis. Nature, 62017976):1031-10036 2023.
[4] Sean L Metzger, Kaylo T Littlejohn, Alexander B Silva, David A Moses, Margaret P Seaton, Ran Wang, Maximilian E Dougherty, Jessie Liu, Peter Wu, Michael Berger, et al. A high- performance neuroprosthesis for speech decoding and avatar control. Nature, 620(7976):1037- 1046, 2023.
[5] Guy Gaziv, Roman Beliy, Niv Granot, Assaf Hoogi, Francesca Strappini, Tal Golan, and Michal Irani. Self-super natural image reconstruction and large-scale semantic classification from brain activity. Neurolmage, 254: 1 19121, 2022.
[6] Robin Tibor Schirrmeister, Jost Tobias Springenberg. Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human brain mapping, 38( 1):5911-5420, 2017.
[7] Ji-Hoon Jeong, Mhong-Hyun Cho, Byeong-Hoo Hoo Lee, and Seong-Whan Lee. Real-time deep neurolinguistic learning enhances noninvasive neural language decoding for brain-machine interaction. IEEE Transactions on Cybernetics, 53(12):7469-7488 2022.
[8] Thorir Mar Ingolfsson, Michael Hersche, Xiaying Wang, Nobuaki Kobayashi, Lukas Cavigelli, and Luca Benini. Eeg-tcnet: An accurate temporal convolutional network for embedded motor- imagery
interfaces. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages
IEEE, 2020.
[9] Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. Deep language algorithms predict semantic comprehension from brain activity. Scientific reports, 12(1):16327, 2022.
[10) Verno J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent Lance. Eegnet: compact convolutional neural network for eeg-based brain-computer interfaces. Journal of neural engineering, 15(5):056013, 2018.
[11] Yue-Ting Pan, Jin Chou, and Chun-Shu Wei. Matt: A manifold attention network for eeg decoding. Advances in Neural Information Processing Systems, 35:31116-3112,, 2022.
[12] Reinmar Kobler, Jun-ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. Spd domain- specific batch normalization to crack interpretable unsupervised domain adaptation in ceg. Advances in Neural Information Processing Systems, 35:6219-6233 2022.
[13] Guangyu Wang, Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals. Advances in Neural Information Processing Systems, 37:39249-39977 392800, 2024.
[14] Zhenhailong Wang and Heng Ji. Open vocabulary elettroncephabgrrphyytoo.o to-text decoding and zero-shot sentiment classification. In Proceedings the AAAI Conference Artificial Intelligence, volume 36, pages 5350-5358, 2022.
[15] Alex Murphy, Bernd Bohnet, Ryan McDonald, and Uta Noppeney. Decoding part-of-spp part-of-speech from human eeg signals. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2201-22 2022.
11

---

[16] Nicolas Affolter, Beni Egressy, Damian Pascual, and Roger Wattenhofer. Brain2word: decoding brain activity for language generation. arXiv preprint htth::20090047765.. 2020.
[17] Yiqun Duan, Jinzhao Zhou, Zhen Wang, Yu-Kai Wang, and Chin-teng Lin. Dewave: Discrete encoding of eeg waves for eeg text translation. Advances in Neural Information Processing Systems, 36:9907-9918, 2023.
[18] Prajwal Singh, Pankaj Pandey, Krishma Miyapuram, and Shanmuganathan Raman, Eeg2image: image reconstruction from eeg brain signals. In ICASSP 2023-2023 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[19] Yonghao Song, Bingchuan Liu, Xiang Li, Nanlin Shi, Yijun Wang, and Xiaorong Gao. Decoding natural images from EEG for object recognition. In The Twelfth International Conference on Learning Representations, 2024.
[20] Alessandro T Gifford, Kshitij Dwivedi, Gemma Roig, and Radoslaw Cichy. A large and rich eeg dataset for modeling human visual object recognition. Neurolmage, 264:119754, 2022.
[21] Tao Fang, Yu Qi, and Gang Pan. Reconstructing perceptive images from brain activity by shape-semantic gan. Advances in Neural Information Processing Systems, 33:13038- 2020.
[22] Sikun Lin, Thomas Sprague, and Ambuj K Singh. Mind reader: Reconstructing complex images from brain activities. Advances in Neural Information Processing Systems, 35:29624-29636, 2022.
[23] Ziqi Ren, Jie Li, Xuetong Xue, Xin Li, Fan Yang, Zhicheng Jiao, and Xinbo Gao. Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning. Neurolmage, 228: 1 17602, 2021.
[24] Tao Fang, Qian Zheng, and Gang Pan. Alleviating the semantic gap for generalized fmri-to- image reconstruction. Advances in Neural Information Processing Systems, 16:15096-15107, 2023.
[25] Paul Scotti, Atmadeep Banerjee, Jimmic Goode, Stepan Shabalin, Alex Nguyen, Aidan Demp- ster, Nathalic Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. Reconstructing the mind's eye: fmriito-image with contrastive learning and diffusion priors. Advances in Neural Information Processing Systems, 00124705-24722 2023.
[26] Xuan-Hac Liu, Yan-Kai Liu, Yansen Wang. Kan Ren, Hanwen Shi, Zilong Wang, Dongsheng Li, Bao-Liang Lu, and Wei-Long Zheng. Ecg2video: Towards decoding dynamic visual perception from eeg signals. Advances Neural Information Processing Systems, 01-72245-72275 2024.
[27] Yuanyuan Yao, Axel Stebner, Tinne Tuytelaars, Simon Geirnaert, and Alexander Bertrand. Identifying temporal correlations between natural single-shot videos and ceg signals. Journal of Neural Engineering. (111001661, 2024.
[28] Xiyu Song, Bin Yan, Li Tong, Jun Shu, and Ying Zeng. Asynchronous video target detection based single-trial eeg signals. IEEE Transactions on Neural Systems and Rehabilitation 2020.
Engineering.
[29] Soobeom Jang, Seong- Bon Moon, and Jong-Seok Lee. Eeg- base emotional video classification via learning connectivity structure. IEEE Transactions on Affective Computing, 14(2): 1586 1597,, 2021.
[30] Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic mindscapes: High-quality video re- construction from brain activity. Advances in Newral Information Processing Systems, 36:24841- 24858, 2023.
[31] Sean Metzger, Jessie Liu, David A Moses, Maximilian Dougherty, Margaret Seaton, Kaylo T Littlejohn, Josh Chartier, Gopala K Anumanchipalli, Adelyn Tu-Chan, Karunesh Ganguly, al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Nature communications, 13(1):6510, 2022.
12

---

[32] David A Moses, Sean L Metzger, Jessie R Liu, Gopala K Anumanchipalli, Joseph G Makin, Pengfei F Sun, Josh Chartier, Maximilian Dougherly, Patricia M Liu, Gary M Abrams, et al. Neuroprosthesis for decoding speech in paralyzed person with anarthria. New England Journal of Medicine, 385(3)217-227, 2021.
[33] Yuliang Ma, Weicheng Zhao, Ming Meng, Qizhong Zhang, Qingshan She, and Jianhai Zhang. Cross-subject emotion recognition based on domain similarity of eeg signal transfer learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 11:936-9445 2023.
[34] Jian Cui, Zirui Lan, Olga Sourina, and Wolfgang Müller- Wittig. Eeg-based cross-subject driver drowsiness recognition with an interpretable convolutional neural network. IEEE Transactions on Neural Networks and Learning Systems, 34(10):7921-75 -7933, 2022.
[35] Qingshan She, Chenqi Zhang, Feng Fang, Yuliang Ma, and Yingchun Zhang. Multisource associate domain adaptation for cross-subject and cross-session eeg emotion recognition. IEEE Transactions on Instrumentation and Measurement, 72:1-12, 2023.
[36] Yong Peng, Honggang Liu, Wanzeng Kong, Feiping Nie, Bao-Liang Lu, and Andrzej Ci- chocki. Joint eeg feature transfer and semisupervised cross-subject emotion recognition. IEEE Transactions on Industrial Informatics, (0(7)8104-8111 2022.
[37] Huayu Chen, Shuting Sun, Jianxiu Li, Ruilan Yu, Nan Li, Xiaowei Li, and Bin Hu. Personal- zscore: Eliminating individual difference for eeg- based cross-subject emotion recognition. IEEE Transactions on Affective Computing, 14(3):2077-2088 2021.
[38] Zixuan Gong, Qi Zhang, Guangyin Bao, Lei Zhu, Rongtao Xu, Ke Liu, Liang Hu, and Duoqian Miao. Mindtuner: Cross-subject visual decoding with visual fingerprint and semantic correction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14247- 14255, 2025.
[39] Seyedeh-Rezvan Farahibozorg, Janine D Bijsterbosch, Weikang Gong, Saad Jbabdi, Stephen M Smith, Samuel J Harrison, and Mark W Woolrich. Hierarchical modelling of functional brain networks in population and individuals from big fmri data. Neurolmage, 243:1 18513, 2021.
[40] Yohann Benchetrit, Hubert Banville, and Jean-Rémi King. Brain decoding: toward real-time reconstruction of visual perception. arXiv preprint 10. 2023.
[41] Arti Anuragi, Dilip Singh Sisodia, and Ram Bilas Pachori. Eeg-based cross-subject emotion recognition using fourier-bessel series expansion based empirical wavelet transform and nca feature selection method. Information Sciences, 610:508-533 0240, 2022.
[42] Vishal Pandey, Nikhil Panwar, Atharva Kumbhar, Partha Pratim Roy, and Masakazu Iwamura. Enhanced cross-task eeg classification: Domain adaptation with eegnet. In International Conference Pattern Recognition, pages
Springer, 2025.
[43] Xuesong Liu, Shanshan Qu, Gang Luo, Chang Yan, Dixin Wang, Na Chu, Fuze Tian, Jing Zhu, Xiaowei Li, Shuting Sun, et al. A generalized depression recognition framework based on cross-center and cross-task eeg signals. Biomedical Signal Processing and Control, 104:107492, 2025.
[44] Kaifan Zhang, Lihuo He, Xin Jiang, Wen Lu, Di Wang, and Xinbo Gao. Cognitioncapturer: Decoding visual stimuli from human eeg signal with multimodal information. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14486-14499 2025.
[45] Zixuan Gong, Guangyin Bao, Qi Zhang, Zhongwei Wan, Duoqian Miao, Shoujin Wang, Lei Zhu, Changwei Wang, Rongtao Xu, Liang Hu, et al. Neuroclips: Towards high-fidelity and smooth fmri-to-video reconstruction. Advances in Neural Information Processing Systems, 37:51655-51633 2024.
[46] Ganit Kupershmidr, Roman Beliy, Guy Gaziv, and Michal Irani. A penny for your (visual) thoughts: Self-supervised reconstruction of natural movies from brain activity. arXiv preprint vrXiv:2266003544 2022.
13

---

[47] Chong Wang, Hongmei Yan, Wei Huang, Jiyi Li, Yuting Wang, Yun- Sun-Shuang Fan, Wei Sheng, Tao Liu, Rong Li, and Huafu Chen. Reconstructing rapid natural vision with amriccondiiionl video generative adversarial network. Cerebral Cortex, 42(20):500-44511, 2022.
[48] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolu- tional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710-719, 2022.
[49] Benjamin Lahner, Kshitij Dwivedi, Polina lamshchinina, Monika Graumann, Alex Lascelles, Gemma Roig, Alessandro Thomas Gifford, Bowen Pan, Young Jin, N Apurva Ratan Murty, al. Modeling short visual events through the bold moments video fmri dataset and metadata. Nature communications, 15(1):6241, 2024.
[50] Chong Li, Xuelin Qian, Yun Wang, Jingyang Huo, Xiangyang Xue, Yanwei Fu, and Jianfeng Feng. Enhancing cuoss-subject fmri-to-video decoding with global-local functional alignment. In European Conference on Computer Vision, pages 353-369. Springer, 2024.
[51] Li-Ming Zhao, Xu Yan, and Bao-Liang Lu. Plug-and-play domain adaptation for cross-subject eeg- eeg-based emotion recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 863-870, 2021.
[52] Xinke Shen, Xianggen Liu, Xin Hu, Dan Zhang, and Sen Song. Contrastive learning of subject- invariant eeg representations for cross-subject emotion recognition. IEEE Transactions on Affective Computing, 14(3):299-2511,, 2022.
[53] Zhunan Li, Enwei Zhu, Ming Jin, Cunhang Fan, Huiguang He, Ting Cai, and Jinpeng Li. Dynamic domain adaptation for class- subjett and cross-session eeg emotion recog- nition. IEEE Journal of Biomedical and Health Informatics, 26(12):9944-5973, 2022.
[54] Wei-Long Zheng and Bao-Liang Lu, Investigating critical frequency bands and channels for eeg- based emotion recognition with deep neural networks. IEEE Transactions on autonomous mental development, 7(3):122-175, 2015.
[55] Sander Koelstra, Christian Muhl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and loannis Patras. Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing, 3(1):18-31, 2011.
[56] Ard Kastrati, Martyna B Plomecka, Damian Pascual Ortiz, Lukas Wolf, Victor Gillioz, Roger Wattenhofer, and Nicolas Langer. Fegeyenet: a simultaneous electreencephalograpyy and eye-tracking dataset and benchmark for eye movement prediction. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks / (NeurIPS Datasets and Benchmarks 2021), 2021.
[57] Alessandro T Gifford, Kshitij Dwivedi, Gemma Roig, and Radoslaw M Cichy. A large and rich eeg dataset for modeling human visual object recognition. Neurolmage, 264: 2022.
[58] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20:273-297, 1995.
[59] Francis Willett, Donald Avansino, Leigh R Hochberg, Jaimie M Henderson, and Kr- ishna V Shenoy. High-peeformance brain-to-text communication via handwriting. Nature, 593(7858):249-2555 254, 2021.
[60] Shuxian Zou, Shaonan Wang, Jiajun Zhang, and Chengqing Zong. Cross-modal cloze task: A new task to brain-to-word decoding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 648 657, 2022.
[61] Shuxian Zou, Shaonan Wang, Jiajun Zhang, and Chengqing Zong. Towards brain-to-text generation: Neural decoding with pre-trained encoder-decoder models. In NewrIPS 202 Al for Science Workshop, 2021.
14

---

[62] Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26(5):858-88) 866, 2023.
[63] Joseph G Makin, David A Moses, and Edward F Chang. Machine translation of cortical activity to text with an encoder decoder framework. Nature neuroscience, 23(4):575-582, 2020.
[64] Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King. Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence, 5(10):1097-1107, 2023.
[65] Saskia L Frisby, Ajay D Halai, Christopher R Cox, Matthew A Lambon Ralph, and Timothy T Rogers, Decoding semantic representations in mind and brain. Trends in cognitive sciences, 27(3):258-285 2023.
[66] Juliette Millet, Charlotte Caucheteux, Yves Boubenec, Alexandre Gramfort, Ewan Dunbar, Christophe Pallier, Jean-Remi King. et al. Toward realistic model of speech processing in the brain with self-supervised learning. Advances in Neural Information Processing Systems, 35:33428-33443, 2022.
[67] Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 44453114463 2023.
[68] Jun Kai Ho, Tomoyasu Horikawa, Kei Majima, Fan Cheng, and Yukiyasu Kamitani. Inter- individual deep image reconstruction via hierarchical neural code conversion. Neurolmage. 271: 120007, 2023.
[69] Yijin Zhou, Fu Li, Yang Li, Youshuo Ji, Lijian Zhang, Yuanfang Chen, and Huaning Wang. Enhancing cross-dataset eeg emotion recognition: A novel approach with emotional eeg style transfer network. IEFE Transactions on Affective Computing, 2025.
[70] Shuaiqi Liu, Zeyao Wang, Yanling An, Bing Li, Xinrui Wang, and Yudong Zhang. Da-capsnet: A multi-branch capsule network based on adversarial domain adaption for cross-subject eeg emotion recognition. Knowledge-Bacd Based Systems, 283: 11 1117, 2024.
[71] Zhangyong Xu, Ning Chen, Guangqiang Li, Jing Li. Hongqing Zhu, and Zhiying Zhu. The mitigation of heterogeneity in temporal scale among different cortical regions for eeg emotion recognition. Knowledge-Baser Systems, 309: 12826, 2025.
[72] Cunbo Li, Shuhan Zhang, Yufeng Mu, Lei Yang, Yueheng Peng, Fali Li, Yangsong Zhang, Zhen Liang, Zehong Cao, Feng Wan, et al. Emotion recognition by learning the manifold of fused multiscale information of eeg signals. IEEE Transactions on Affective Computing, 2025.
[73] Yueying Zhou, Ziming Xu, Yifan Niu, Pengpai Wang, Xuyun Wen, Xia Wu, and Daoqiang Zhang. Cross-task cognitive workload recognition based eeg and domain adaptation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30:50-60, 2022.
[74] Kai Guan, Zhimin Zhang, Tao Liu, and Haijun Niu. Cross-task mental workload recognition based on eeg tensor representation and transfer learning. IEEE Transactions Neural Systems and Rehabilitation Engineering, 31:2632-222 2239,, 2023.
[75] Mariana RF Mota, Pedro HL Silva, Eduardo JS Luz, Gladston JP Moreira, Thiago Schons, Lauro AG Moraes, and David Menotti. A deep descriptor for cross-tasking eeg-based recogni- tion. PeerJ Computer Science, 7:e549, 2021.
[76] Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11 11333-11342, 2024.
[77] Yulong Liu, Yongqiang Ma, Guibo Zhu, Haodong Jing. and Nanning Zheng. See through their minds: Learning transferable brain decoding models from cross-subject fmri. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5730-5738, 2025.
15

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We made clear claims of our contributions in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of our research.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
Our paper dose not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes, we present all the information necessary to reproduce the main experi- mental results. In addition, we will make the code and trained models publicly available to ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Yes, we will release our code as soon as possible, all datasets are publicly available, and we will also make all dataset access methods public our website.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
public/guides/ Staalees//ooeSSuusmsisiiiii
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
Cooee//bdeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We demonstrated our method and the experiment settings clearly in Section 3. Also, we will public our code of training and testing process
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, we present the error bar and all statistical significances of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we demonstrated Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, we discuss the broader impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Our models and data do not have high risks for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite all original papers of EEG/MMI/MEG models used in our experiments.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes, all our results are fully documented. The documentation will be provided along with the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
We used publicly available datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The datasets we use are all publicly available.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is not the core of our research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: http:s/s/eeaerrsssec////ooeerere for what should or should not be described.
22

---

A Related Work
A.1 EEG-based Text Decoding
Brain-to-text decoding has evolved significantly researchers explore non-invasive EEG-based methods. Willett et al. [59] established landmark performance in invasive systems using handwriting- based paradigms, while parallel non-invasive EEG approaches have been pursued for their practical advantages (7). Murphy et al. [15] pioneered part-of-speech decoding from EEG, demonstrating that grammatical information could be extracted from non-invasive recordings.
Affolter et al. introduced Brain2Word [16. a foundational framework for decoding brain activity into natural language, while Wang and Ji [14] developed open vocabulary EEG-to-text decoders with zero-shot sentiment classification capabilities. Zou et al. [60] proposed cross-modal cloze tasks for brain-to-word decoding and later integrated pre-trained encoder-decoder models to improve performance [61.
Tang et al. [62 demonstrated semantic reconstruction of continuous language from non- non-invasivv recordings, while Duan et al. [17 developed DEWave, a discrete encoding approach leveraging EEG wave properties for text translation. Makin et al. [3 established machine translation frameworks for cortical activity, and Défossez et al.
[44] presented methods for decoding speech perception from non- invasive brain recordings. Recent advances include Jeong et al.'s 7 real-time deep neurolinguistic learning system and Moses et al.'s
neuroprosthesis for decoding speech in paralyzed individuals. These developments leverage innovations in EEG signal processing [65] [22 [T] and self-supervised approach 666] that have proven effective across neural decoding domains.
32
A.2 EEG-based Image Decoding
EEG-based image reconstruction has advanced significantly through innovations in neural architec- tures and generative modeling. Early work by Palazzo et al. [ established multimodal learning for decoding brain representations by integrating neural activity and visual features, providing founda- tion for subsequent research. Song et al. [48] developed EEG Conformer, combining convolutional and transformer architectures to enhance spatial-temporal modeling of EEG signals, while Pan et al. [1I introduced MAtt, manifold attention network designed to address the dimensinnsi, non- non-linear nature of neural data.
Recent approaches have leveraged sophisticated generative models for improved reconstruction quality. Takagi and Nishimoto (67 demonstrated high-resolution image reconstruction using latent diffusion models from brain activity, while Lin et al. [22] presented Mind Reader for reconstructing complex images from EEG signals. These advances have been complemented by crose-subject generalization techniques, with Ho et al. (68) introducing hierarchical neural code conversion for inter-individual image reconstruction and Fang et al. (24) developing methods to alleviate the semantic gap in generalized brain-to-image reconstruction. Collectively, these innovations in architectures, generative modeling, and transfer learning have significantly enhanced the capabilities of EEG-based
image reconstruction systems [21 23 18
A.3 EEG-based Emotion Recognition
EEG-based emotion recognition has evolved significantly with advancements in cross-subject gen- eralization techniques. Zhou et al. [69] developed an Emotional EEG Style Transfer Network that enhances cross-dataset emotion recognition by normalizing subject-specific patterns, while Shen et al. [52] introduced contrastive learning of subject-invarine cross-subject performance. Li et al. [53] proposed dynamic domain adaptation for both cross-subject and cross-session emotion recognition, addressing the inherent variability in neural responses across individuals. Zhao et al. [51] created a plug-and-pla, domain adaptation framework that enables efficient adaptation to new subjects without extensive retraining, while Chen et al. [37] introduced Personal to normalize individual differences in signal characteristics.
EEG representations for improved
These approaches have been evaluated across diverse applications, with Peng et al. 36 implementing joint EEG feature transfer with semisupervised learning, and Liu et al. (70) developing DA-Caps Nepss,,,,
a multi-branch capsule network leveraging adversarial domain adaptation. She al. (35] focused on multisource associate domain adaptation for both cross-subject and cross-session scenarios, while Xu
23

---

et al. 77 addressed the challenge of heterogeneity in temporal scale among different cortical regions. Collectively, these innovations represent significant progress toward robust emotion recognition systems that can generalize effectively across individuals, sessions, and experimental conditions
72 70
A.4 EEG-based Cross-Task Learning
Cross-task learning has emerged as a critical approach for maximizing knowledge transfer in BEG- based applications. Zhou et al. 73 pioneered cognitive workload recognition across different tasks using domain adaptation techniques, while Guan al. (74] developed tensor representation methods with transfer learning to improve tross-task mental workload detection. Mota et al. [75] introduced a deep descriptor specifically designed for cross-tasking EEG-based recognition that enables knowledge sharing between different cognitive paradigms. The effectiveness of these approaches has been demonstrated in various applications, with Pandey al. adaptation with EEGNet architecture, and Liu et al. 43 establishing a generalized depression recognition framework based on cross-center and cross-task EEG signals.
[42 enhancing classification through domain
A.5 Cross-Subject Neural Decoding
Cross-subject generalization represents a fundamental challenge in neural decoding, with significant advances in both EEG and fMRI modalities. For BEG-basee applications, Chen et al. [37] introduced Personal-zscore climinate individual differences through effective normalization of subject-specific signal characteristics, while Zhao et al. [5]] developed plug-and- -nnd-pay domain adaptation framework that enables efficient transfer to new subjects. Li al. [53] proposed dynamic domain adaptation for both cross-subject and cross-session scenarios, addressing the critical challenge of temporal variability across recording sessions. These approaches have substantially improved generalization and video reconstruction
performance in various domains, including emotion recognition [52 35 36
27 28
In parallel, iMRI-based cross-subject decoding has seen remarkable progress through specialized alignment techniques. Wang al. [76] developed MindBridge. a comprehensive framework that establishes common neural representations across individuals, while Li et al. [50] introduced global- local functional alignment for enhanced cross-subject MMRI-to-vider decoding. Liu al. [77] focused on learning transferable brain decoding models from cross-subject fMRI data, and Gong al. [3 proposed Mind'unne, which leverages visual fingerprinting and semantic correction for improved generalization. These innovations, supported by hierarchical modeling of functional brain networks [39], collectively demonstrate significant advancement toward overcoming the substantial individual variability that has traditionally limited the generalizability of neural decoding systems.
A.6
Video-based Neural Decoding
Recent advances in video decoding from neural signals have demonstrated remarkable capabilities in reconstructing dynamic visual experiences. Liu et (26 developed EEG2video, a pioneering frame- work for decoding dynamic visual perception from EEG signals that effectively captures temporal coherence in video reconstruction. Yao al. 27 identified temporal correlations between natural single-shot videos and EEG signals, while Song et al. (28) demonstrated asynchronous video target detection based on single-trial EEG recordings. These BEG-based approaches are complemented by Jang et al.'s 229] work on emotional video classification through learning connectivity structures, establishing the relationship between neural activity patterns and affective video content.
In parallel, IMRI-based video decoding has achieved high-fidelity reconstruction through specialized architectures. Chen et al. [00] introduced Cinematic Mindscapes for high-quality video reconstruction from brain activity, while Wang et al. (30] developed an CoRI-condiiional video generative adversarial network for reconstructing rapid natural vision. et al. [50] enhanced cross-subject fMRI-to tol--tooviedo decoding through global-local functional alignment, addressing the critical challenge of individual variability. These advances have been supported by comprehensive datasets like BOLD Moments [49]. which provides rich metadata for modeling short visual events, enabling researchers to establish increasingly sophisticated mappings between neural activity and dynamic visual experiences across both EEG and fMRI modalities.
24

---

B Algorithm
Algorithm 1 Training and Inference of NEED Framework
Require:
1: Training: EEG dataset Dtrain = (xz; vi, di), auxiliary datasets Doun. pretrained models T21, B,C
2: Inference: Unseen subject EEG Etest, task type Tr {video, image)
Ensure: Trained model components: LAM, DSGNet, PU, SU, Inference Mechanism
3: Training Stage:
4: Pretrain IAM on multiple datasets: Cpretrain = O.Crecon + BLade + DDKL
5: for each (I,, 1/)) E D Draan do
6: Process EEG: Z; = IAM (x;), ZDSG (SGNNet(Z.)
7: end for
8: Train PU with Bidirectional Dynamic Contrastive loss:
exp(simi 2g (//1// + A Zo V.Vi
9: CBDC = - Eis Wiy log Σ, exp(sim Z..V+//T)
10: Train SU with Hierarchical Semantic Contrastive loss:
11: Lusc Lusc= =i= on llog Σ. exp(2i-Ci, //)T)
expl (2) (O,(/(CL/))
+ || G(C+))lF
12: Train unified inference with Cross-modal Alignment and Diffusion:
LESA Ccont (Zz. T) + 7 LKI T) + Lulign (Z3, Keyframe)
14: Learn task-specifi parameters (T(T)) and (T())) for conditional layer modulation
14( t, Cs. Tv) = (T)) Norm(hy-1) (T,)
16: Inference Stage:
Process test EEG: Ztest TAM (A ZDSa DSGNet (Ztest)
18: Extract perception features: Zp PU (DsG)
Extract semantic features: Z, SU (sG)
20: Apply task adaptation: Zrask TaskAdapter Z.),
21: Configure conditional weights based on task: atemp = oW, Tt), O'spat = oWW, (W,+T.)
22: Generate integrated conditioning:
23: C = 0q ControlNet (Keyframepred + 3e Tex(Embed(Caption) + DynPattem + 6, Zeeg
24: Initialize: IT (N( 1)
25: for t = T to do
26: Apply adaptide denoising: 2:-1 = po t, Ct, Tτ) + 00.
27: Apply conditional layer modulation: h, C,, (T()) C Norm(h-11) + ((()))
28: end for
29: Return: Reconstructed visual content zip (video image based on T7))
C Extended Method Details
C.1 Hierarchical Individual Adaptation Module
The Individual Adaptation Module (IAM) is central to our framework's cross-subject generalization capability. Here we provide a detailed description of its multi-level adaptation architecture and working principles.
C.1.1 Multi-Level Adaptation Architecture
As shown in Fig IAM implements hierarchical adaptation approach that processes EEG signals at four progressively abstract levels: signal level, spectral level, temporal level, and semantic level. This design allows the module to address different aspects of inter-subject variability simultaneously.
Signal-Level Adaptation. At the lowest level, IAM normalizes physical characteristics of EEG signals that vary across subjects due to differences in skull thickness, electrode placement, and skin conductivity. For each subject 8, we learn transformation parameters:
Esignal Wsignal (s) Eoriginal (s)gnal(s)
(6)
where W (s) CC is subject-specift transformation matrix that recalibrates channel relationships. and (s) E I bias term that adjusts baseline activity. For subjects with
25

---

different numbers of channels than our reference configuration, the transformation includes an additional projection step that maps between different dimensionalities.
Spectral-Leeee Adaptation, Individual differences in oscillatory patterns are well-documented in EEG literature, with substantial variability in the power and frequency characteristics of canonical brain rhythms. Our spectral adaptation addresses this by learning subject-specific weights for different frequency bands:
Espectral
Σ
Whand (6) band Esignat)
bandE (8.0.a.P.n)
where BPF band represents bandpass filtering for canonical frequency bands, and (band(s) are subject- specific importance weights. The filtered signals are further processed through a subject-conditioned spectral attention mechanism:
Aspectral (f) softmax(u, tanh(W,- PSSDPSDDSDA PSD Espectral) ba))
(8)
where PSD computes the power spectral density, and Us. W.. and bg are subject-specific parameters.
Temporal-Level Adaptation. Different subjects exhibit varying temporal dynamics in their neural responses, including latency differences and distinct temporal patterns. To address this, IAM employs subject-conditioned recurrent module:
Etemporal = l4 GRU ( Espectrat, ht-1; Otempoorat (8)
where Otemporal (s) represents subject-specific GRU parameters. To reduce parameter overhead, implement this through hypernetworks where a small subject embedding generates the modification base GRU parameters:
Oremporat (8) Obase + MLP( (embea)
(10)
Semantic-Level Adaptation. At the highest level, IAM processes cognitive patterns that vary across subjects through our subject-aware attention mechanism:
Zattn Σ w'b (a
H
(0((())(() (E.W3s + S,UR) + ST (LEEWW
(E,
(11)
Vdz
This mechanism allows the model to attend differently to EEG features based subject-specific characteristics, capturing individual differences neural encoding of visual information.
While the multi-level adaptation captures subject-specific patterns, the subject-invariant transforma- tion a extracts common neural patterns across subjects:
I (E,, 2s, dd) MHA (LN (FILM (E,, W d2 b3d,,) Its)
(12)
where MHA denotes multi-head attention, LN is layer normalization, and FiLM represents feature- wise linear modulation with learned weight matrix W, and bias vector bg. The dataset embedding d, allows the transformation to adapt to different experimental paradigms, while Ωg represents subject-agnostic knowledge learned during pretraining.
C.1.2 Adaptive Integration Mechanism
The final normalized representation integrates information from all adaptation levels through dynamic gating mechanism:
[gsignal 9epectral 9epectrai--temporak Jattn grl sofftmax softmax(MLP((Esignnd) Espectral; Etemporat: Zattn Io)))
(13)
fitmal Isignal Esignal + Ispectral Expectral + 9temporal Etemporal + Juttn Zottn + gr T
(14)
This adaptive weighting allows IAM to emphasize different adaptation levels based on input charac- teristies, providing flcxibility across diverse EEG patterns.
26

---

Orignal EEG Signal
E Borigna
Signal-Level Adaptation
E Exigal
BPF
E Exectrrai
BPF
E, B.W. + A,U
E.WA a,
E.W
MatMul
Ousee
Scale
Santed
Softmax
MatMul
Softmax
Z
E
Basoporol
A Apectrol
Figure 7: Multi-Level Adaptation Architecture
C.1.3 Adversarial Training Strategy
To further enhance subject invariance in the normalized representations, we employ adversarial training strategy. During pretraining, subject discriminator D attempts to identify subject identity from the normalized representations, while IAM is trained to generate representations that maximize reconstruction quality while minimizing the discriminator's performance:
Ladu (0p. 8G) = Ez ( PP2), [log D(Znorm)) ) + Eznarm <pp [log(1 ((Zorrm) )
(15)
This adversarial objective is balanced with reconstruction quality and KL divergence terms:
Epretrain a.Crecon + BLado (On, 0a) + (DDii(((Zoorn)) (ll ((norm)
(16)
C.1.4 Implementation Details
The IAM architecture 6 attention heads with embedding dimension 256 for the subject-aware attention mechanism. Subject embeddings are initialized randomly and learned during training, with dimension d, 64. For the spectral processing. we use standard frequency bands: delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma (30-100 Hz).
The module is implemented with layer normalization and residual connections throughout to stabilize training. Dropout (rate 0.2) is applied after each major component to prevent overfitting to subject- specific patterns. The adaptive integration mechanism a 2-layer MLP with hidden dimension 128 and LeakyReLU activation.
During pretraining, use Adam W optimizer with learning rate le-4 and weight decay 0.01. The loss function weights are set to a 1.0, 0.5, and = to balance reconstruction quality with subject disentanglement.
27

---

C.2 Dual-Stream EEG Encoder Details
Here we provide detailed mathematical formulations of Dual- Stream BEG Encoder (DSGNet). which forms the foundation of our framework.
C.2.1 Spatial Stream Details
The spatial stream begins with spherical harmonic representation of electrode positions. For electrodes located at spherical coordinates (0;, 04) on the scalp, the initial spatial embedding is:
E E Eppatial (i)
Σ
Σ
alm Y, (0,, 0.)
(17)
where Y spherical harmonics of degree and order and Ct. learnable coefficients. The geodesic-based adjacency matrix Aij exp( (Pp /202) captures inter-electrode relationships, which are processed through graph convolutions:
E E((+1) spatial
= σ - E spatial w() BatchNorm
(Eepatial
(18)
where Ā = A AIx incorporates self-connections and D. -E Aij is the degree matrix.
C.2.2 Temporal Stream Details
The temporal stream implements a discrete approximation of the Riemann-Liouviln fractional transformation:
WA/L Σ
Ra Rx(X),
(0) w
X(t xo)
kAt)
(19)
I(a +1)
where (a) r(k r(k+a)
[a fractional binomial coefficients. This multi-scale approach with E
(0.2, 0.4, 0.6, 0.8} captures temporal dependencies at various scales. Dilated temporal convolutions further expand the receptive field:
K-1 FO) = Σ (k) F(l (0)
(20)
with kernel size = 5 and dilation factor d 2/ for layer yielding an effective receptive field of 31 time steps with 4 layers.
C.2.3 Cross-Stream Integration Details
The cross-stream attention implements multi-head attention with heads:
Zoross Concat [head],.... head,) W
(21)
where head, Attention E Espatiat W Etemporat W.KK Etemporat WV) and Attention(Q, V) softmax ( V.
The gated fusion mechanism combines features through:
Z o (W, (Espatiill Etemporal; Zoross] (croos]+bbr tttn)ttnnt/WW,.... (
[Espatial; Etemporati Zerosel+by) (22)
Our mulen-ddimenional masking strategy applies three types of masks: Mspatial (simulating electrode disconnections), Mtemporul (temporal signal loss), and M (requency (frequency- band noise), sampled
28

---

from Bernoulli distributions with parameters Pspotial = 0.8, Ptemporal 0.9, and Pfrequency 0.85 respectively. The final representation is computed as E = Z @ Mapatial O Mtemporal ① M frequency
C.3 Cross-Dataset Channel Adaptation
A fundamental challenge in EEG-based neural decoding is addressing the heterogeneity of acquisition systems. Our NEED framework implements comprehensive channel adaptation strategy that enables seamless generalization across varied EEG configurations. This approach is critical for practical applications where data might come from different hardware setups.
Our unified coordinate system aligns all electrode locations to a standardized spherical model based on the international 10- system. For similar channel setups, establish direct correspondence between electrodes based on spatial proximity. FigBBillustrates our mapping approach across different datasets.
SEED-DV
SEED
DEAP
THINGS-EEG
EYEEEGNET
Spherical spline interpolation
Figure 8: Electrode mapping visualization across different EEG datasets. The figure shows the spatial distribution of electrodes in SEED-DV. SEED, DEAP, THINGS-EEG and EEGEYENET datasets. The rightmost visualization demonstrates spherical spline interpolation used to normalize channel configurations.
When transferring to systems with fewer channels (e.g., from 62 to 14 channels), we prioritize electrodes covering key visual processing regions. Conversely, for adaptation to higher-density configurations, we employ spherical spline interpolation to reconstruct signals at missing locations:
E
Wiy WiyEe E.
(23)
SEC
where F. is the interpolated value at channel location j in the reference configuration, E, is the measured value at existing channel i, and wi are weights based on the spherical distance between locations:
29

---

exp(- (dkiy//222
Wij = Σ exp(-d2,/20""
(24)
The parameter controls the influence radius of each electrode and is set to 0.15 radians (approxi- mately 8.6 degrees on the scalp surface) based on empirical validation across datasets.
Beyond spatial mapping, address spectral differences through unified preprocessing: resampling all data to 200Hz, applying a consistent 0.5-40Hz bandpass filter, and implementing frequency band-specific gain adjustments. This approach normalizes signal characteristics across acquisition systems:
Fnnormalizea G())
Foriginas (f)
(25)
where G(f) is frequency-dependen gain function derived from comparing power spectral density distributions across datasets. The gain functions defined per frequency band to account for hardware-specific filtering effects:
G8, if E [0.5, 4 Hz]
Go, if E [4, 8 Hz]
G(f)
Gas if f E [8, 13 Hz]
(26)
G3: if f E [13, 30 Hz]
Gy,, if f E [30, 40 Hz]
The IAM module integrates channel importance learning, automatically adapting to available elec- trodes by assigning importance weights (at;) during training:
Zehannel Σ
0g E,
(27)
These weights are initialized based neurophysiological priors (higher weights for occipital and temporal regions for visual tasks) but evolve during training to optimize cross-dataset generalization. The importance weights are learned through attention mechanism:
exp(v" tanh ( + b.))
(28)
2j=1 == exp(e TT tanh E, + bc))
where v, We, and be learned parameters. This approach enables framework to process EEG from diverse recording systems within a unified computational architecture, enabling true cross-dataset compatibility.
Datasets and Preprocessing
Our NEED framework leverages multiple EEG datasets in a strategic manner enable zero-shot cross-subject and cross-task generalization. Table 3 provides an overview of these datasets and their roles in our framework.
Table 3: Overview of EEG datasets and their roles in the NEED frame work.
Dataset
Subjects
EEG Channels
Stimuli Type
Role in NEED
SEED-DV
20
62
Video clips (40 classes)
Primary training video reconstruction
SEED
15
62
Emotional film clips
IAM pretraining (dynamic stimuli)
DEAP
32
32
Music vidcos
1AM pretraining (affective responses)
THINGS-FEG
64
Static images (1,854 concepts)
Evaluation cross-task generalization
BEGEYENET
356
Various visual tasks
Evaluation cross-subject generalization
30

---

D.1 SEED-DV Dataset
SEED-DV (SJTU Emotion EEG Dataset Dynamic Video) serves as our primary training dataset for video reconstruction. It contains high-density EEG recordings from 20 subjects watching 1,400 video clips across 40 visual categories, as shown table 4 Each 2-second clip is presented at 24 frames per second with resolution of x920x 1080 pixels. EEG data is recorded using 62-channel cap according to the international 10-10 system, sampled at 1000 Hz using ESI NeuroScan System.
Table 4: Visual Categories in the SEED-DV Dataset
SEED-DV Dataset: 40 Visual Categories
Food Items
Animals
Human Activities
Natural Scenes
Dynamic Events
Watermelon
Turtle
Skiing
Mountain
Fireworks
Pizza
Shark-like
Running
Buildings
Water
Drink
Jellyfish
Dancing
Beach
Car
Cake
Fish/Coral
Boxing
Tree
Hot balloon
Banana
Bird
Face
Mushroom
Airplane
Rabbit
Crowd
Flower
Ship
Panda
Couple
Road
Bike
Horse
Piano
Elephant
Guitar
Dog
Drum
Cat
The visual stimuli in SEED-DV include a diverse range of content categories: natural scenes (c.g.. mountains, oceans, forests), urban environments (e.g., city streets, buildings, traffic), human activities (e.g., sports, dancing, cooking), animal behaviors, and various object interactions. This diversity is crucial for training a robust visual decoder capable of reconstructing wide spectrum of visual experiences.
For preprocessing, we apply a bandpass filter (0.5-40 Hz) to remove artifacts, downsample to 200 Hz, and segment the continuous EEG into 2-second epochs corresponding individual video clips. Independent Component Analysis (ICA) is used to remove eye movement and musele artifacts. The cleaned EEG signals are then normalized per channel using z-score normalization.
D.2
SEED Dataset
The SEED dataset contains EBG recordings from subjects watching emotional film clips designed to elicit specific emotional states (positive, neutral, and negative). EEG is recorded using the same 62-channel system as SEED-DV. While originally developed for emotion recognition, we leverage this dataset during IAM pretraining to capture neural responses dynamic visual stimuli with varying emotional content.
The film clips in SEED range from 4-6 minutes in length and include excerpts from popular movies and documentaries that have been validated to reliably induce the target emotions. This dataset helps our model generalize across different types of dynamic visual content with varied emotional significance.
D.3 DEAP Dataset
DEAP (Database for Emotion Analysis using Physiological Signals) contains EEG recordings from 32 subjects watching 40 one-minute music videos. EEG is recorded using 32 channels, providing complementary information about neural responses to audiovisual stimuli. We use this dataset during IAM pretraining to enhance cross-subject generalization capabilities.
Although DEAP uses fewer EEG channels (32) compared to our primary datasets, larger subject pool provides valuable information about inter-subject variability. The IAM module learns to normalize these diverse patterns into common representation space through our multi-dataset knowledge transfer approach.
31

---

D.4 THINGS-EEG Dataset
The THINGS-EECS dataset contains EEG recordings from 10 subjects viewing 22,000 static images across 1,854 object categories. EEG is recorded using 64-channel system. Importantly, this dataset uses static image stimuli rather than videos, making it ideal for evaluating cross-task generalization.
The THINGS image collection contains diverse everyday objects photographed under various condi- tions. Each stimulus is presented for 500ms followed by a 1000ms inter-stimulus interval. We extract epochs corresponding to the stimulus presentation for our cross-task evaluation. The high diversity of object categories in this dataset provides a rigorous test for semantic reconstruction capabilities.
D.5 EEGEYENET Dataset
EEGEYENET the largest publicly available EEG dataset for visual tasks, containing recordings from 356 subjects performing various visual tasks including object recogmition, scene categorization, and visual search. EEG recorded using portable 14-channel system (EMOTIV EPOC+), making substantially different from our training datasets in terms of channel configuration and signal quality,
This dataset provides extreme test of framework's s ability generalize across subjects, particularly with significantly different EEG acquisition systems. The 14-channel configuration poses a particular challenge for spatial pattern extraction, requiring our IAM to effectively normalize these sparse recordings into the common representation space learned from denser EEG montages.
D.6 Data Preprocessing
We preprocess EEG data using standard filtering (0.5-40 Hz), downsampling to 200 Hz, and segment- ing the continuous EEG into 2-second epochs corresponding to individual visual stimuli. Independent Component Analysis (ICA) applied to remove eye movement and muscle artifacts. The cleaned EEG signals are then normalized per channel using Z-score normalization. For SEED-DV specifically, we apply additional preprocessing steps to account for the high temporal resolution of video stimuli, including temporal alignment of EEG with visual frame transitions and removal of inter-trial noise. Details of our cross datsset channel adaptation approach, which addresses the varying number of EEG channels across datasets, are provided in Appendix C33
E
Implementation Details
We implement our framework using PyTorch and conduct experiments on NVIDIA A100 GPUs. The Individual Adaptation Module (IAM) employs multi-head attention mechanism (6 heads, dimension 256) with subject embeddings (dimension 64) and dataset-conditional modulation. Our Dual-Stream EEG Encoder (DSGNet) integrates spatial processing using spherical harmonic representations and 4- layer graph neural networks, with temporal processing through multi-scale fractional transformations at alpha parameters (0.2, 0.4, 0.6, 0.8).
The Perception Understanding (PU) module implements thrnc-branch architecture extracting complementary visual features: motion patterns through 3D networks, spatial structures via 2D networks, and temporal dynamics using SMARN architecture (2 layers, hidden size 256). The Semantic Understanding (SU) module adapts CLIP-ViT-G/1K 14 and LLaMA-2-132 1-3B-BB- with LoRA fine- tuning (rank 16) for EEG feature translation. Training employs the AdamW optimizer (learning rates le-4 to 2e-5) with weight decay (0.01) and cosine scheduling.
For video reconstruction, we extract 6 keyframes per 2-second clip and employ temporal attention mechanisms to ensure coherence, For image reconstruction, we adapt the same framework but modify the conditioning signals with different weighting coefficients (I0.3, 0.4, 0. 0.2] versus [0.4, 0.3, 0.2, 0.1] for videos). The unified inference mechanism uses classifier- free guidance (scale 7.5) with 50 denoising steps.
We follow different evaluation protocols for different generalization scenarios: 5-fold cross-validation for within-subject assessment, leave-one-out testing for cross-subject generalization, and zero-shot transfer (without fine-tuning) for cross-task evaluation. This enables comprehensive assessment of framework's generalization capabilities across different dimensions.
32

---

F
Supplementary Results
F1
Ablation results
F.1.1
PU Ablation results
Table 5: Detailed ablation study on Perception Understanding (PU) module components, This analysis examines how different components of the PU module affect temporal coherence and visual fidelity in reconstruction tasks.
Video Reconstruction (SKED-DV)
Image Reconstruction (THINGS-EGG)
Module Configuration
Temporal Metrics
Visual Quality
Structural Metries
Detail
CLIP-pecT
FVDJ
2-way Aoc.f
SSIM
Edge-FIf
Stuctssimf
PIPS)
PixCorr?
Full PÚ Module
0.738+00.04
175.348.6
0.399:0003
0.356=0.02
0.42
0.493-0.02
0.387=0.03
0.162:00.0
Compunent Anutysi
wlo Dynamic Feature Extractor (DFF)
0,485+0.05
283,7+1588
0.713+0.04
0.231+0.03
0,315+004
0,356-003
0,528+0.04
0.103+00.02
wlo Spatistemporal Momorr-Augmented RNY
0.54240.04
254.8=12.77
0.752:0.04
0.267
0.359
0.402=0.03
0466:20.04
0.125
Bidirectional Dynomic Contrastive (BDC)
0.61710.05
218661113
0.787110.0
0,29810:03
0,38510.01
0:138:0.03
0/42.10.03
0.M4310.02
2. DFE Component Andlyis
wdo Motion Branch (Fm)
0.56310.05
237.221335
0.776.10004
0.27410.03
0.362.0.01
0.413.0.03
0.49910.01
0.12910.02
Spanial Brunch (Fs)
0.685:0004
202.6::108
0.705:00.03
0.312:003
0,372:003
0:426:0033
0.429:0003
0.138:0.02
weo Temporal Brunch (Fi)
0,59510.00
231.811244
0.783.10.01
0,28910:03
0,36810011
0.11910.03
0.44..0.01
0.135
Monion Spotial only (no Temposol)
0.678+0005
222.3+11.5
0.72++0:3
0.301:0033
0.376+003
0.432=003
0443:00.03
0.141+0.02
Motion: Temporal only (no Spatiol)
0.694#0.04
195.849.3
0.802:00.03
0.328=0.08
0.382=004
0.448=0.02
0.412=0.03
0.148#0.02
Spatial+ + Tomporal only (no Motion)
0,65210.05
214.7210.6
0.798.10004
0.31510:03
0,37910:03
0.441.0.03
0.41810.03
0.14510.02
SMARN Architeszure Analysis
LSTM instend of SMARN
0.64310.05
216.9=11.2
0.79510.03
0.30910:08
0.375.0004
0,436=0.03
0.421:0.04
0,14240.02
GRU instead of SMARN
0.632.10001
214.5:10.8
0.79111.04
0,30510:03
0,372.0003
0.430_0003
0.42510.03
0.14010.02
Tranfformer instead of SMARN
0.68110.055
203.819.77
0.81310.03
0.3271.002
0.392.0:03
0.452.0.03
0,45510.03
0.15310.01
wle Hierorchical Memory
0.686:10.04
201.2:19.5
0.81510003
0.3291:002
0,39510003
0,460:0.02
0,402:0.03
0,154:0.01
ulo Atteationaal Wriglos
0.708+0004
193.4+9.11
0
0.337+0.02
0.403-003
0.409-0.2
0.398-0.03
0.156+0.01
4. Bidirectional Dynamic Contustivv Anulysiz
woo Temporal Grudient Temm
0.689+0.04
195.3+94
0,877+H103
0.333+0.22
0,3990003
0.46400022
0440-0003
0,155+0.01
wlo Learablee Attention Weights
0.70240.04
192.74922
0.823+00.03
0.339+0.02
0.406=0003
0.471=0.02
0.366:0.03
0.157+0.01
Cosine vs. Euclideun Distance
0.726100001
183.418.8
0.83110.03
0.31710:02
0.1110003
0.186.0.02
0.39210.03
0.15910.01
Different Temperature (=0,05)
0,723::004
185.7:90
0:829:4333
0,345:0.02
0,415:003
0.48300022
0.394:00.3
0,158:0.01
Different Temperature (=0.5)
0.71710005
188.219.1
0.82510.03
0.31110.02
0:11110.03
0.178.0.03
0.39710.03
0.156100.01
Temporal Modeling Appesaches AD ComoNeess in instead SMARN
0.698+0.04
194,2+933
0.81++003
0.334-0.02
0.401-003
0.46-00.2
0,400-0.03
0.156+0.01
Temporal Attention instead of SMARN
0.706:1004
191.8192
0.82310:03
0.338:002
0,4051003
0.47:00022
0.396:0.03
0.157:10.11
Stou Network instend SMARN
0,71240004
189.5+9.0
0.826+00.03
0.342+002
0.499=003
0.476-0.22
0.394-0.33
0.158+0.D1
TimeSforner ustead of SMARN
0.72120.04
186.318.9
0.830::0:03
0.347:0:02
0.415
0.484=0.02
03911:0.03
0.160
Vimaal Feature Extractors
Reshien-5 50 issted of T3D/D2D/TD
0.687#0.05
198.5
0.81240.03
0.327=0:08
0.39620.03
0.461=0.02
0.40==0.03
0.153+0.01
VIT B/16 instead of T3D//2D/TTD
0.70210.01
192.61922
0.82310.03
0,33910:02
0,40510003
0.472.0.02
0.39610003
0.15710.01
Swin-T inttend T3D/T20/TID
0.714::004
1889299
0.88::003
0.343:002
0,40=0005
0.477:0022
0333::0:33
0.158:0.01
ComNeXt- instead of T3D/T2//TD
0.719#0.04
186.7+89
0.829+0.03
0.345:0002
0.413=0003
0.481=0.02
03322=0.03
0.159#0.01
7. Integration with Semantic Underssanding
Late Fusion (PU+SU az vnd)
0.721+0004
187.2+89
0
0.346+0.22
0414=003
0.82-0002
0332--0.33
0.159+0.01
Feature level Fasion
0.72610004
183.81888
0.833:0.03
0.34910.02
0.41820003
0.488=0.02
0390-0.03
0.16010.01
Hierorchical Fusion (ours)
0.738+-0.04
175.3+-8.
0.839+0.03
0.356=0.02
0.427=0.03 0.493-0.02 0 0.162+0.00
As shown in T5 we conducted an importance evaluation of the submodules of the PU module on two different datasets: SEED-DV for video reconstruction and THINGS-EEG for image recon- struction. The table is divided into seven modules, each demonstrating the significance of different branches within the seven submodules of the PU module. The first section focuses on the three core components of the PU module: the Dynamic Feature Extractor (DFE), the Spatiotemporal Memory- Enhanced Recurrent Network (SMARN), and the Bidirectional Dynamic Contrastive Alignment Module (BDC).
When the DFE branch is removed, the Structural Similarity Index (SSIM) for the reconstruction task decreases by 0.125, which is twice the decrease observed when the BDC branch is removed. The quality of the model's reconstruction output significantly degrades, highlighting the crucial role of the DFE module in output similarity for reconstruction tasks. Further analysis reveals that the DFE module consists of three main branches: motion, spatial, and temporal. The motion branch has the most significant impact, with an SSIM drop of 0.082 after its removal, accounting for 66% of the overall decrease. This validates the core contribution of the motion branch in dynamic EEG decoding.
33

---

In terms of spatiotemporal sequence processing, we replaced the SMARN module with standard sequence models such as LSTM. GRU, and Transformer. Although the Transformer replacement outperforms the models using LSTM and GRU in terms of temporal alignment, the model with the SMARN module still achieves better performance across all metrics compared to the Transformer- based model. This demonstrates the importance of the SMARN module in spatiotemporal sequence processing. Notably, the hierarchical memory structure within SMARN is one of the key reasons for its superior performance over the Transformer module.
F.1.2 SU Ablation results
Table 6: Detailed ablation study on Semantic Understanding (SU) module components. This analysis examines the effect of different component combinations and model variants on reconstruction performance.
Video Reconstruction (SEED-DV)
Image Reconstruction (THINGS-EEG)
SU Module Configuration
Semantic Accuracy
Visual Quality
Semantic Accuracy
Visual Quality
40-way Acc.f
CLIP-sim
SSIM
FVD]
CLIP
Inceptiont
SSIM
PixCortt
Full Module
0.00510.02
0.503-0003
0.356L0.02
175.38.66
0.79510.02
0.742-10033
0.352 0.02
0.162:0001
1. Core Component Analysis
w/o CLIP-WIT-/11
0.146 0,14++0.03
0.349=0.04
0,278=0.02
267.8+12.3
0.654+0.33
0.613=0.03
0.153=0.03
0.127=0.02
w/o BLIP-2
0.283=0.00
0.462=0.03
0.298=0.05
214.6+9.7
0.712=0003
0.663=0.02
0.285=0.02
0.143=0.02
w/o LIM Adapter
0.328+00.02
0.487=0.03
0.321+0.02
193.549.3
0.742+0.02
0.693++0.3
0.312- 0.151=0.01
2. Modality Integration Analysis
Vision-only (CLIP+BLIP-2)
0.352=0.03
0:495=0.22
0,325=0.03
194.728.9
0.758#0.03
0.705=0.02
0.324=0.02
0.154=0.02
Language-only (LLM) CLIP-only
0.24510.03
0,412=0:03
0,28710.02
236.8111.4
0.6832003
0.63520003
0,257=0:03
0.131=0.02
0.296+0.03
0,456=0003
0,30==002
10.
0,706+0.02
0.652=003
0,2670002 0.135=0.02
0.312:0.02
0.468
0.30910.03
217.19.8
0.7151003
0.665.0022
0.278-0.03
0.13810.02
CLIP (no BLIP-2)
0.365++0.03
0.496-0.03
0.332+0.02
1928+8.7
0.763+0.02
0.711-003
0.326
0.155-0.01
BLIP-2+LLM (no CLIP)
0.37910.02
0.501
0.33910.03
186.21..5
0.712:1003
0.71810002
0.335=0:02
0.157-0.01
3. Losz Function Analysis
wlo HSC Loss
0.376+0.02
0.501+000
0.343+0.03
185.6+9.1
0.76340.02
0.719+0002
0.338-0.02
0.18-0001
odo CSA Loss
0.384+0.03
0,507=0033
0,347=0.02
182.1
0.775+0.03
0.726=0033
0.34=0002
0.159=0.01
Divergence
0.39
0,515:0002
0.35310.02
178448.6
0.788
0.02
0.347-0.02
0.160:0.01
Contrastive Term
0.372:0.03
0.49820:03
0.33920.03
187.219.3
0.76122003
0.715=0003
0.335=0.03
0.157=0.02
4. Open-Source LLM Comparison
LLaMA-2-7B
0.39220.03
0.512=0.02
0.34920.03
181.2
0.78220.02
0.731=0003
0.45=00.2
0,158=0.11
Pho-2
0.386+0.02
0,508-0.03
0,345+0.02
185.7+9.1
0,777+0.03
0.727+0002
0.342-0.33
0.1770001
Vicuna
0.380.1.03
0.501
0.34010.03
189.319.2
0.768
0.71910:03
0.336-0.02
0.15410.01
1LMMA-2-13B
0.412::.02
0.529-0.03
0.361:0.02
1736:1833
0.802:0.02
0.747=0.03
0.356
0.165:0.01
Mistral-BB
0.39820.03
0,517=0.02
0,353:0.03
178.928.4
0.79010.03
0.736::.02
0.348=0.02
0.160=0.01
5. Vision Encoder Compurison
CLIP-VVT-1/14 (smaller
0.387+0002
0,507=0003
0.342+0.02
187.6+9.2
0.776+0.03
0.725+0003
0.340-0.2
0.156-0.01
CLIP-V VIF-8/3 (smollest)
0.362+0.03
0,493=0:02
0,331+0.03
196.7
0.756+0.02
0.702=003
0.329-0:03
0.149=0.02
OpenCLIP- DINOx2-VT-1/2
0.402.0.02
0.520.0.03
0.35410.02
177.618.7
0.79210.03
0.73810.02
0.04
0.1610001
0.395
0.516=0.02
0.351
180.4
0.785
0.732+0003
0.345
0.159-0.01
6. BLIP-2 Variantr BLIP-2 (ViT-g)
0.405=0.02
0.523=0.03
0.35620.02
175.3=88.
0.79520.02
0,74220033
0.352=0.02
0.162=0.01
BIJP-2 (VT--.) BLIP-1
0.394+0.03
0.514-0.02
0.350+0.03
179.8+8.8
0.78340.03
0.733+0002
0.346-0.03
0.159-0.01
0.378+0.02
0,501=0003
0.339:0.02
1865:993
0.765+0.02
0.718=003
0.335=002 0.155=0.11
GiT-Large
10.03
0.49610:02
0.33510:03
190.2.9.5
0.75910003
0.710:1002
0.331-0.03 0.152:0.02
7. Integration Methods
Sequential (CLIP-BLIP--H)
0.392+0.03
0,513=002
0.348:0.03
181.5+8,8
0.781#0.02
0.730=0003
0.345=0.02
0.158=0.01
Parallel (fusion at end)
0.387+0.02
0.508=0.03
0.346:0.02
183.729.2
0.77620003
0.726=0.02 0.342=0.03 0,157=0.11
Cross-atention (ours)
0.408--0.02
0.523-0.03
0.356-0.02
175.3+8.6
0.795+0.02 0.742-0.03 0,382 0.02 0.162-0.01
As shown in The SU module is composed of three core components: the Image- Text Alignment Module (CLIP-VIT VCLIP-VTT-//14), the Bidirectional Vision-Languaa Encoder (BLIP-2), and the Large Language Model Adapter (LLM Adapter). When the CLIP-VIT-G/14 module is removed, the model achieves only a 0.146 accuracy 40-way classification task, significantly lower than when the BLIP-2 or LLM Adapter modules are removed. This highlights the indispensable role of the image-text alignment process in reconstruction tasks.
In terms of multimodal information fusion, we compared the effects of unimodal vision or language inputs with multimodal combinations. The results indicate that the model using only the visual
34

---

modality has higher structural similarity index (SSIM) of 0.32 compared to the language onll model, which achieves an SSIM of 0.287. Moreover, the multimodal vingonelanguage model achieves an SSIM of 0.356, suggesting certain degree of complementarity between visual and linguistic information and confirming the critical role of visual-language fusion in EEG reconstruction tasks.
We also explored different open-source large language models. Our comparison shows that while LLaMA-2-13B achieves the highest classification accuracy of 0.412 and outperforms the phi-2 in various performance metrics, it leads to a significant increase in model parameters. The performance improvement, however, does not justify the additional parameter cost, indicating diminishing returns in model efficiency.
F.2 Cross-Subject Generalization Analysis
Table 7: Cross-subject generalization performance on SEED-DV dataset. Models were trained on varying numbers of subjects and evaluated completely unseen subjects.
Video-based
Frame-based
Training Strategy
Semantic-level
Temporal
Semantie-lvvel
Pixel-level
2-way Acc.f
Acc.
FVD]
CLIP-peef
2-way Acc.
40-way Acc.
SSIMT
Within subject (20/20)
0.89820.02
0.405+00.02
175.348.6
0.73840.04
0.83920.03
0.293360.2
0.35620.02
1/20 subject
0.231=0.06
0.04240:03
584.7+21.3
0.12840.08
0.213+0.07
0.036+0.03
0.076+0.05
2/20 subjects
0.312+0.05
0.067+0.04
536. 1+19.8
0.19540.07
0.287=0.06
0.058+0.04
0. 102+0.05
5/20 subjects
0.48520.05
0.13620.04
412.5&17.2
0.32440.07
0.452a0.05
0.107:0004
0.167:0.05
10/20 subjects
0.68520.05
0,252:0.05
254.6212.8
0,48740.07
0.64320.05
0.162:0:05
0.241
15/20 subjects
0.782=0.04 0..4
0.321=0.04
1..5
0.60240.06
0.731
0.204+0.05
0,293=0.04
19/20 subjects
0.372+0.03
0.68240.05
0.786+0.03
Performance retention (19/0)
93.7%
91.9%
88.8%
92.4%
93.7%
83.6%
92.4%
Performance retention (10/20)
76.3%
68.9%
66.0%
Performance retention (5/20)
54.0%
33.6%
42.5%
43.9%
$3.9%
36.5%
46.9%
(1/20)
Table 8: Detailed cross-subject generalization performance all training subject counts on SEED-DV dataset. This table shows the progression of generalization capability as the number of training subjects increases from I to 19.
Video-based
Frame-based
Training Subjects
Semantic-level
Temporal
Semantic-level
Pixel-level
2-way Ace.
40-way Ace.
FVD
CLIP-pee
2-way Acc.
40-way Acc.
SSIM
subject (Sub01)
0.231
0.042
584.7
0.128
0.213
0.036
0.076
subjects
0,312
0.067
536.1
0.195
0.287
0.058
0,102
subjects
0.356
0.089
502,8
0.235
0.323
0.071
0.124
subjects
0.423
0.112
467.4
0.284
0.392
0.089
0.148
subjects
0.485
0.136
412.5
0.324
0.452
0.107
0.167
subjects
0.527
0.162
376.9
0.357
0.495
0.121
0.186
subjects
0.568
0.189
343.2
0.389
0.532
0.132
0,203
subjects
0.605
0.215
312,8
0.421
0.569
0.143
0,217
subjects
0.235
283.5
0.454
0.607
0.152
0,229
10 subjects
0.685
0.252
254.6
0.487
0.643
0.162
0.241
11 subjects
0.702
0.267
245.3
0.508
0.659
0.169
0,250
12 subjects
0.725
0,283
236.8
0.532
0.678
0.179
0.262
13 subjects
0.297
228.4
0.559
0.699
0.187
0,275
14 subjects
0.765
0.312
221.5
0.581
0.715
0.195
0.284
15 subjects
0.782
0,321
215.3
0.602
0.731
0.204
0.293
subjects
0.798
0.334
209.7
0.625
0.749
0.216
0.304
17 subjects
0,816
0.347
204.1
0.645
0.762
0.228
0,313
18 subjects
0.830
0.359
199.8
0.667
0.776
0.237
0.321
19 subjects
0.841
0.372
197.5
0.682
0.786
0.245
0.329
To systematically evaluate NEED's cross-subject generalization capability, we conducted extensive experiments with varying numbers of training subjects. Table Ie that performance retention indicates the percentage of within-subject capability maintained when generalizing to new subjects. (able/Jreeeal consistent pattern of improvement as more subjects are included in training. With just subjects, the model achieves moderate generalization (48.5% video-based semantic accuracy),
35

---

while using 10 subjects yields substantial improvement (68.5%). The most significant finding is that with 19 subjects, our model retains 33.7% of within-subject performance when generalizing to unseen subjects, demonstrating robust subject-invariant representations. Notably, performance retention increases non-linearly, with greater gains observed when transitioning from 10 15 subjects (11.9% improvement) compared to 15 to 19 subjects (6.6% improvement), suggesting an emergent generalization capability beyond a critical mass of training subjects.
Table 9: Performance retention percentages by training subject count. This table shows what percentage within-subject performance is retained when generalizing to unseen subjects.
Video-based
Frame-based
Training Subjects
Semantic-level
Temporal
Semantic-leval
Pixel-level
2-way Acc.
40-way Acc.
FVD
CLIP--pe
2-way Ace.
40-way Acc.
SSIM
subject
25.7%
10.4%
30.0%
17.3%
25,4%
12.3%
21.3%
2 subjects
34.7%
16.5%
32.7%
26.4%
34.2%
19.8%
28.7%
subjects
39.6%
22.0%
34.9%
31.8%
38.5%
24.2%
34.8%
subjects
47.1%
27.7%
37.5%
38 38.5%
46.7%
30.4%
41.6%
subjects
54.0%
33.6%
42.5%
43.9%
53.9%
36.5%
46.9%
10 subjects
76.3%
62.2%
68.9%
66.0%
76.6%
55.3%
67.7%
15 subjects
87.1%
79.3%
81.4%
81.6%
87.1%
69.6%
82.3%
19 subjects
93.7%
91.9%
88.8%
92.4%
93.7%
83.6%
92.4%
Table quantifies the cross-subject generalization efficiency through performance retention rates. These metrics reveal how effectively our model preserves capabilities when applied unseen subjects. Training with just a single subject yields limited retention (25.7% for semantic accuracy). while using 5 subjects achieves approximately half of within-subject performance (54.0%). The most striking improvement occurs between 10 and 15 subjects, where retention, jumps from 76.3% to 87.1% for semantic accuracy, indicating threshold effect cross-subject knowledge transfer. At 19 subjects, our model achieves remarkable retention across all metrics (88.8% (0..8%%%997.99 93.7%), with the highest retention observed in 2-way classification accuracy (93.7%) and temporal coherence (92.4%), demonstrating the framework's ' balanced preservation of both semantic content and visual dynamics.
F.2.1 Individual Subject Transfer Analysis
Table 10: Cross-subject generalization performance for single subject training. This table shows how models trained on each individual subject generalize to completely unseen Subject 20.
Video-bar based
Frame-base based
Training Subject
Semantic-level
Temporal
Semantic-level
Pixel-level
2- way Acc.
40-way Acc.
FVD
CLIP-pe
2-way Acc.
40-way Acc.
SSIM
Subject 01
0.231
0.042
584.7
0.128
0.213
0,036
0.076
Subject 02
0.217
0.039
602.3
0.113
0.196
0,032
0,068
Subject 03
0.242
0.045
573.6
0.135
0.221
0,038
0.082
Subject 04
0.228
0.041
591.8
0.122
0.205
0,034
0.073
Subject 05
0.235
0.044
578.2
0.131
0.217
0,037
0.079
Subject 06
0,253
0.049
567.5
0.142
0.232
0,040
0.085
Subject 07
0.219
0.040
598.4
0.116
0.199
0.033
0.070
Subject 08
0.224
0.041
593.7
0.120
0.208
0,035
0.072
Subject 09
0.236
0.044
576.9
0.132
0.218
0,038
0.080
Subject 10
0.246
0.047
571.2
0.138
0.226
0,039
0.083
Subject 11
0.229
0.042
589.5
0.124
0.210
0,035
0.074
Subject 12
0.238
0.045
574.8
0.133
0.220
0,038
0.081
Subject 13
0.251
0.048
569.3
0.140
0.230
0,040
0.084
Subject 14
0.221
0.040
596.2
0.118
0.202
0.034
0.071
Subject 15
0.243
0.046
572.4
0.136
0.223
0.039
0.082
Subject 16
0.233
0.043
581.5
0.129
0.215
0,037
0.078
Subject 17
0.248
0.047
570.6
0.139
0.228
0,039
0.083
Subject 18
0.240
0.045
573.8
0.134
0.221
0,038
0.081
Subject 19
0.227
0.041
590.6
0.123
0.206
0.035
0.073
Average
0.235
0.044
582.5
0.129
0.215
0.037
0.077
36

---

Table 10 examines the variation in generalization performance when training on individual subjects. Testing on unseen Subject 20 reveals consistent performance across different training subjects, with an average 2-way classification accuracy of 23.5% (compared to 89.8% within-subject). Notably, certain subjects (e.g., Subject and Subject 13) yield superior generalization performance, achieving up to 25.3% accuracy, while others (e.g., Subject 02 and Subject 07) result in poorer transfer at 21.7%. This 3.6% variation suggests inherent differences in neural representation commonality across individuals. Despite this variation, the limited performance of single-subject training (23.5% average) compared to multi-subject training (84.1% 1% with 19 subjects) underscores the necessity of mubti- approach for building robust subject- invariant decoders.
Table II: Ablation study of components for cross-subject generalization with 19/20 training subjects. This table shows the impact of each module on generalization performance.
Video- based
Frame-based
Model Configuration
Semantic-level
Temporal
Semantic-lever
Pixel-level
2-way Acc.
40-way Acc.
FVD
CLIP-pec
2-way Acc.
40-way Acc.
SSIM
Full Model
0.841
0.372
197.5
0.682
0.786
0.245
0.329
w/o IAM
0,483
0.098
386.2
0.267
0.419
0.079
0.142
w/o PU
0.632
0,241
342.1
0,345
0,584
0.136
0.205
w/o SU
0.725
0.128
267.8
0.532
0.686
0.091
0.246
w/o Task Conditioning
0.793
0.335
228.6
0,605
0.745
0,218
0.287
w/o Dynamic Pattern
0.782
0.327
235.3
0,589
0.734
0.207
0.276
w/o Self-Attention
0.815
0.351
212.4
0.642
0.765
0.229
0.308
Table 11 presents ablation results examining each component's contribution to cross-subject general- ization when training on 19/20 subjects. The removal the Individual Adaptation Module (IAM) causes the most severe performance degradation, with 2-way classification accuracy dropping from 84.1% to 48.3% (-35.8%), confirming IAM's critical role in normalizing subject-specific patterns. The Semantic Understanding (SU) module's ' removal results more moderate decline to 72.5% (-11.6%), while eliminating the Perception Understanding (PU) module reduces performance to 63.2% (-20.9%). These differential impacts reveal the complementary nature our dual pathhaa architecture, with PU contributing more substantially to srbss-subject generalization than SU. Task conditioning removal causes smaller but still significant drop 79.3% (-4.8%) while dynamic pattern removal reduces performance to 78.2% (-5.9%). Notably, self-attention removal has the least impact (-2.6%), suggesting that while beneficial, this component is not critical for cross-subject transfer. These findings validate our comprehensive approach and identify IAM as the cornerstone of effective generalization to unseen subjects.
F.3 Frequency Band Contribution
Beyond spatial information, we analyze the contribution of different EEG frequency bands to shows the impact of removing specific frequency components from the EEG signals during model training and evaluation.
reconstruction performance. Fig 9 shows
Alpha band (8-14 Hz) removal causes the most substantial performance degradation, with accuracy dropping from 14.23% to 8.46% (p < 0.001). This profound impact reflects the critical role alpha oscillations play in visual attention, inhibitory control, and information flow during visual tasks. Beta band (14-30 Hz) removal also significantly reduces performance to 10.15% (p < 0.01), consistent with its involvement in higher-level visual integration and contextual processing.
Gamma band (30-100 Hz) removal shows a relatively modest effect (12.87%), despite its established role in feature binding and object representation. This may be explained by the difficulty in capturing clean gamma signals from scalp EEG due to their lower amplitude and higher susceptibility to muscle artifacts. Delta (1-4 Hz) and theta (4-8 Hz) band removal produce intermediate effects performance (11.76% and 11. 11.23% respectively), indicating their contributory but less dominant role in visual information encoding.
E4 Qualitative and Quantitative Prompt Analysis
To comprehensively evaluate the semantic fidelity of our reconstructions, we analyze both the visual outputs and their corresponding textual descriptions. This dual analysis provides insights into how
37

---

Accuracy
10-
5
Delta
Theta
Alpha
Beta
Gemma
All
Frequency band (ebleted)
Figure 9: Impact of frequency band ablation on 40-class classification accuracy. Each bar represents model performance when the specified frequency band is excluded, except for "All" which uses the full frequency spectrum. Error bars indicate standard deviation across subjects. Statistical significance: p < 0.05, #* < 0.01, #** 0.001.
GT
LLM
Ours
GT
LLM
Ours
person cutting cake decorated with fresh blueberzies
two dogs reeling logether on a grassy surface
person slicing tupped Mueberries
Iwo laying grass
resling grass
with
person holding an acoustic guilar
panda bear perched rocky surface
sitting top a rock
panda sitting rocky surfacee
man holding guitar and wearing hiue shirt
wearing bue shirt holding
celorful firoworks exploding dark night sky
road stretrhing through open countryside with cloudy sky
red fireworks in the night
red firewrrks bursting in the night
view of road with cloudy sky
view of road with cloudy mountains
large vessel moving acress water
forest with tall trees and filtered sunlight
floating
whitn beat biuue
forest filled with
forest filled tall evergreen
Figure 10: Qualitative comparison of textual descriptions across different pathways. For each visual concept, we show: (1) Ground truth (GT) frame with its caption, (2) Textual description generated from EEG signals through LLM, and (3) Caption from our reconstructed video. Red highlights indicate key semantic elements that our method accurately reconstructs. Note how our approach captures subtle distinctions like "slicing" vs. "cutting" cake, "resting" vs. "laying" dogs, and specific environmental details like "rocky surface" and "cloudy mountains."
38

---

effectively our framework preserves semantic information from EEG signals to reconstructed videos. Presents qualitative comparisons of text descriptions generated through different pathways. Fach row shows ground truth video frame, the text generated directly from EEG signals using an LLM, and the caption from our reconstructed video. The results reveal our framework's ability to capture nuanced semantic details. In the food example, our method precisely identifies the action as "slicing" rather than "cutting" and correctly describes the cake as "topped" with blueberries, For animal scenes, our reconstructions accurately capture the panda's s environment ("rocky surface") and the dogs' state ("resting" rather than "laying"). Human activities are similarly well-preserved, with accurate representation of clothing details ("blue shirt") and actions ("holding guitar"). Even for complex scenes like landscapes and dynamic events, our approach maintains key details such as "cloudy mountains" and specific tree types.
F.5 Temporal Dynamics Analysis
To investigate the temporal dynamics of neural information processing in our NEED framework, we perform detailed analysis of how visual information is encoded and can extracted from EEG signals at different time points. Fig IT illustrates the 40-way classification accuracy using three different temporal windowing strategies.
18.0%
2000]
16.0%
[-100, g
Chance loval
14.0%
12.0%
Accuracy
10.0%
8.0%
6.0%
4.0%
2.0%
0.0%
200
400
600
800
1000
1200
1400
1600
1800
2000
t (ms)
Figure 11: Temporal analysis visual information processing in EEG signals. The graph shows 40-way classification accuracy using three different temporal window strategies: cumulative windows from stimulus onset to time point [0, t), remaining information from time point t to the end of stimulus 2000], and local information in 200ms window centered at time point t [t- 10-1 t]. The analysis reveals distinct temporal phases of visual information processing, with semantic information accumulating gradually until stabilizing around 14000-500mm
The blue curve [0, represents cumulative information from stimulus onset to time point t, showing how visual information accumulates over time. Starting from near-chance level at early time points (100ms), classification accuracy increases steadily, with particularly rapid rise between 400- 10001200ms, before stabilizing around 14.2% after 1400ms. This pattern suggests that meaningful semantic information continues to accumulate during the first 1.4 seconds of stimulus presentation, after which additional EEG data provides minimal improvement in classification performance.
The orange curve [t. 2000] shows how much information can be extracted from the remaining signal after time point until the end of the 2-second stimulus. Starting with relatively high performance at early time points (13.3% at 100ms), decreases stcadily as t increases, approaching chance level
39

---

around 1800ms. This decline indicates that most discriminative information is captured within the first 1.5 seconds, with little unique information in the final 500ms of stimulus presentation.
The green curve [t-100, t] represents local information within a 200ms window centered at each time point. This reveals distinct peak around 600ms, suggesting this time window contains particularly rich discriminative information, likely corresponding to the P300 and N400 components associated with object recognition and semantic processing. After this peak, local information gradually diminishes, indicating a transition from initial stimulus processing to more stable semantic representation.
These temporal dynamics provide important insights into the neural mechanisms underlying visual perception and how our NEED framework leverages them. The analysis demonstrates that while early visual processing begins within 100ms, meaningful semantic information continues to accumulate for substantially longer period (1400-1500mm highlighting the complex temporal dynamics of visual object recognition in the brain.
G Broader Impacts
Our NEED framework has significant implications for assistive technology and neuroscience. could enable new communication pathways for individuals with motor impairments and advance understanding of neural visual processing. However, as decoding accuracy improves, privacy concerns emerge regarding potential access to private mental imagery without consent. We advocate for regulatory frameworks, strict consent protocols, and multidisciplinary dialogue as this technology evolves.
H
Future Work
Our future work will focus on further enhancing reconstruction quality, especially for complex visual scenes. To address the challenge of image-to-vides transfer, we plan collect specialized EEG dataset with paired static-dynamic stimuli that better captures the neural representations bridging these modalities. This dataset will enable development of more sophisticated bidirectional cross-task generalization capabilities and further advance neural decoding toward practical brain-computer interfaces for diverse visual experiences.
40


Reviews:
[{"review_id": "SklW6oc1Z6", "reviewer": "Reviewer_NumW", "summary": "This paper introduces NEED, a comprehensive framework for reconstructing visual stimuli (both videos and images) from EEG signals. The primary goal is to address two major challenges in the field: poor generalization across different subjects and the inability of models to transfer between different tasks (e.g., video vs. image reconstruction) without retraining. \n\nThe proposed NEED framework is a multi-stage, complex system featuring three main keypoints: \n1) an Individual Adaptation Module (IAM) to normalize EEG signals across subjects, \n2) a dual-pathway architecture (Perception and Semantic) to extract different levels of visual information, \n3) a unified inference mechanism for handling different tasks. \n\nThe extensive experiments, including several ablation studies, demonstrating the proposed model outperforms existing methods and achieves strong cross-subject and cross-task generalization, notably retaining over 90% of within-subject performance when generalizing to new subjects.", "questions": "Please refer to Weakness section above.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\n1. Ambitious and Important Problem: The paper tackles two of the most significant and difficult challenges in BCI and neural decoding: cross-subject and cross-task generalization. Developing a single, unified model that can work \"out-of-the-box\" for new users and new visual paradigms is a crucial step toward practical, real-world applications. The ambition of this work is commendable.\n\n2. Comprehensive Experimental Evaluation: The authors have performed an impressive number of experiments on multiple public datasets (SEED-DV, THINGS-EEG, DEAP, etc.). The evaluation is multifaceted, using metrics for perceptual quality (SSIM, PSNR), semantic accuracy (CLIP-based classification), and temporal dynamics (FVD). The cross-subject analysis and ablation studies are thorough and provide strong evidence for the effectiveness of certain components, particularly the IAM.\n\n3. Strong Empirical Results: The reported results are strong. The DSGNet encoder shows significant classification improvements over numerous baselines (Table 1). The full NEED model achieves SOTA or competitive performance in both video and image reconstruction tasks (Table 2). The high performance retention rate for cross-subject generalization (over 90% with 19 training subjects) directly supports one of the main claims of the paper.\n\n\n# Weakness\n\nMy main concerns with this paper are its overwhelming complexity and the corresponding lack of clarity in the presentation, which makes the method difficult to understand and critically evaluate.\n\n1. Overwhelming System Complexity: The proposed NEED framework is exceedingly complex. It combines multiple pre-training stages, a dual-stream encoder (DSGNet), a dual-pathway understanding architecture (PU and SU), each of which contains multiple sub-modules (IAM, DFE, SMARN, CLIP, BLIP-2, LLM), and a sophisticated, multi-guidance inference mechanism (ControlNet, TaskAdapter, etc.). While the ablation studies attempt to justify each piece, the sheer number of components makes the system feel like an elaborate ensemble of many existing powerful models rather than a system with a few core, well-motivated innovations. It is difficult for a reader to discern whether the strong performance comes from a few key insights or simply from the brute-force combination of many SOTA components. The authors should consider if a simpler architecture could achieve comparable results, as complexity can obscure scientific understanding.\n\n2. Excessive Reliance on Supplementary Material: The paper is not self-contained, which severely hinders fluent reading and comprehension. Critical details about the methodology are deferred to the appendix. Many mathematical symbols and components are used in the main text without a clear definition. And many of method details forces a constant back-and-forth between the main text and the appendix, disrupting the flow of reading. Key architectural details for the DSGNet encoder and the crucial IAM are almost entirely in the appendix. The main text provides only a high-level overview, making it hard to understand the method from the main paper alone.\n\n3. High Information Density and Lack of Clarity: The \"Method\" section (Sec 2) is a whirlwind tour of dozens of advanced concepts packed into a very small space. Each component, from the Riemann-Liouville fractional transformation to the Spatiotemporal Memory-Augmented Recurrent Network (SMARN), is introduced and dismissed in a sentence or two.\n\n4. Motivation: The motivation for many design choices is not sufficiently explained. For instance, why was a Riemann-Liouville fractional transformation chosen for the temporal stream over more standard techniques? The justification is brief. Why is the SMARN architecture necessary, and how does it specifically improve upon standard RNNs or Transformers in this context?\n\n5. Clarity: The connections between the numerous components are not always clear. Figure 2 is incredibly dense and difficult to parse, with a multitude of pathways, losses, and modules. A simplified diagram focusing on the core data flow, followed by detailed diagrams for each major component, would be more effective.", "was_revised": true, "final_justification": "This paper aims to tackle an ambitious and important research problem and presents strong empirical results. However, considering that the organization and clarity of the paper require significant improvement and revision, I maintain my original score as borderline accept.", "rebuttal": "**Reviewer NumW**\n\nWe believe we have resolved all your issues. If you have any other concerns or questions, we look forward to hearing from you.\n\n```W1：Overwhelming System Complexity```\n\nWe understand the reviewer's primary concern about overwhelming complexity potentially obscuring our core scientific contributions. We want to emphasize that our framework's complexity arises from the fundamental nature of the dual generalization challenge we address, rather than unnecessary over-engineering. The problem of achieving simultaneous cross-subject and cross-task generalization in EEG-based visual reconstruction has never been solved before, precisely because it requires addressing multiple distinct technical challenges that cannot be resolved with a single simple solution. Our Individual Adaptation Module represents our primary scientific innovation, addressing the core challenge of cross-subject neural variability through a principled hierarchical approach. This is not merely combining existing domain adaptation techniques, but a novel solution specifically designed for EEG's unique multi-level variability patterns from electrode impedance differences to cognitive processing variations. The dramatic 35.8% performance drop when IAM is removed (Table 2, w/o IAM) demonstrates this is a fundamental contribution rather than an incremental addition. We have conducted extensive ablation studies showing that each hierarchical level contributes meaningfully, with the signal-level adaptation alone providing 12.3% improvement over baseline methods (detailed hierarchical ablation table will be added in revision). Similarly, our dual-pathway architecture emerged from the neuroscience insight that visual processing involves distinct mechanisms for low-level dynamics and high-level semantics. The complementary nature of these pathways is evidenced by our ablation results: removing Perception Understanding reduces performance by 20.9% while removing Semantic Understanding drops performance by 11.6% (Table 2, w/o PU and w/o SU). This demonstrates that neither pathway alone can capture the full complexity of EEG visual encoding, validating our architectural choice to maintain both streams. We promise to provide a detailed complexity analysis in the appendix of the revised version, showing that simpler architectures (which we have tested) cannot achieve comparable cross-disciplinary generalization performance. At the same time, we will do our best to find simpler and more efficient frameworks in future work. This is exactly the significance of the NEED work. On this basis, we will have NEED+ or even NEED++.\n\n```W2: Excessive Reliance on Supplementary Material```\n\nOur decision to relegate these fundamental components to appendices was a poor organizational choice that undermines scientific clarity, regardless of our intentions to manage space constraints or narrative flow. In our revision, we will substantially restructure Section 2 to include the complete DSGNet dual-stream architecture, IAM hierarchical adaptation mechanisms, and all essential mathematical formulations directly in the main text. This includes the spherical harmonic spatial embeddings, fractional temporal transformations, and cross-stream attention mechanisms that define our approach. We will ensure every mathematical symbol is clearly defined upon introduction and that component interconnections are explicit. To accommodate this expanded technical content, we will move extensive experimental details, dataset preprocessing specifications, and supplementary ablation studies to the appendix instead. This reorganization will create a self-contained main paper where readers can fully understand and evaluate our technical contributions without constant reference to supplementary material. We will also improve the logical flow by ensuring each component builds naturally upon previously established concepts, guiding readers through our innovations coherently rather than forcing them to piece together information from multiple sources. \n\n```W3: High Information Density and Lack of Clarity```\n\nWe acknowledge the reviewer's valid criticism that our Method section attempts to cover too many advanced concepts with insufficient depth. The current presentation rushes through complex innovations like the Riemann-Liouville fractional transformation and SMARN architecture in just one or two sentences, failing to provide the necessary mathematical foundations and conceptual motivations that readers need to understand and evaluate our contributions. This approach inadvertently obscures our technical innovations rather than clearly communicating them. In our revision, we will substantially expand Section 2 to provide adequate space for each major component, including detailed mathematical derivations, clear conceptual explanations, and step-by-step reasoning for our design choices. For instance, we will dedicate sufficient space to explain why fractional calculus naturally models EEG's power-law temporal dynamics and long-range dependencies that standard approaches cannot capture, complete with mathematical formulations and comparative analysis. Similarly, we will provide comprehensive treatment of SMARN's spatiotemporal memory mechanisms, showing how it addresses the specific challenge of maintaining both electrode spatial relationships and temporal sequence dependencies simultaneously. This restructured presentation will guide readers through our innovations logically and thoroughly, ensuring each component is properly motivated and technically detailed rather than superficially mentioned.\n\n```W4: Motivation```\n\nOur choice of Riemann-Liouville fractional transformation for temporal processing addresses a critical limitation of standard approaches in handling EEG's unique temporal dynamics. EEG signals exhibit power-law temporal relationships and long-range dependencies that standard convolutions and recurrent networks fundamentally cannot capture effectively. Neural activity propagates through brain networks following fractional-order dynamics, where current responses depend on the entire history of past states with power-law decay rather than exponential decay. The fractional calculus framework naturally models these multi-scale temporal relationships by incorporating memory effects at different time scales through the $\\alpha$ parameters (0.2, 0.4, 0.6, 0.8). Standard CNNs with fixed kernel sizes miss these long-range dependencies, while RNNs suffer from vanishing gradients that prevent effective modeling of the 2-second EEG epochs crucial for visual processing.\n\nThe SMARN architecture emerged from the specific challenge that existing sequence models fail to simultaneously preserve spatial electrode relationships and temporal dependencies in EEG data. Standard RNNs process temporal sequences but lose the crucial spatial structure of electrode arrangements that encodes topographical brain activity patterns. Transformers can model long-range dependencies but lack the hierarchical memory mechanisms needed for the spatiotemporal coherence in our extended EEG sequences. SMARN addresses this through spatiotemporal memory cells that maintain both electrode-wise spatial relationships and temporal evolution patterns simultaneously, enabling the model to understand how visual information flows across both brain regions and time.\n\nOur comparative experiments demonstrate these choices' necessity: replacing SMARN with standard LSTM reduces 2-way accuracy from 0.839 to 0.795 and SSIM from 0.356 to 0.309, while using GRU instead of SMARN achieves 0.791 for 2-way accuracy and 0.305 for SSIM. Transformer replacement performs better but still shows degradation with 0.813 for 2-way accuracy and 0.327 for SSIM (Table 5 in Appendix, section 3). For temporal modeling approaches, using 3D ConvNets instead of our fractional transformation approach reduces performance to 0.819 for 2-way accuracy and 0.334 for SSIM, while temporal attention achieves 0.823 and 0.338 respectively, compared to our full model's 0.839 and 0.356 (Table 5 in Appendix, section 5). These performance gaps reflect fundamental architectural mismatches between standard approaches and EEG's unique spatiotemporal characteristics that require specialized temporal modeling and hierarchical memory mechanisms.\n\nIn our revision, we will provide detailed mathematical derivations showing how these design choices emerge from EEG signal properties, along with comprehensive comparisons demonstrating why standard alternatives are inadequate for our cross-subject, cross-task objectives.\n\n```W5: Clarity```\n\nWe agree that Figure 2 is overly dense and creates confusion rather than clarity about our framework's architecture. In our revision, we will redesign the visual presentation with a hierarchical approach: first, a clean, simplified overview diagram showing the three main stages (Cross-Subject Adaptation, Dual-Pathway Understanding, Unified Inference) and their primary data flows, followed by detailed sub-diagrams for each major component that readers can reference after understanding the overall architecture."}, {"review_id": "8Slib5PdXR", "reviewer": "Reviewer_qg8v", "summary": "The manuscript presents a cross-subject and cross-task EEG decoding framework composed of three main components: (1) multi-dataset pretraining, (2) a \"perception\" module that aligns EEG signals with a pretrained variational autoencoder (VAE), and (3) a \"semantic\" module that aligns EEG signals with both the vision and language encoders of CLIP. Additionally, the manuscript introduces a novel dual-stream EEG encoder designed to capture both the spatial and temporal characteristics of EEG recordings. The full model is trained on three large datasets in which participants viewed video clips and is subsequently evaluated on two additional datasets to assess cross-task and cross-subject generalisation. The main finding suggests that the proposed framework is capable of decoding EEG signals from a new participant performing a new task with promising accuracy.", "questions": "1. **BLIP redundancy and clarification**: The use of BLIP in both stages of the Semantic Understanding (SU) module appears redundant and potentially confusing. As illustrated in the schematic flowchart, BLIP seems to generate the same caption for two different images, which raises questions about its purpose in both stages. This redundancy needs to be clarified—are the captions generated independently, or reused across branches? Further explanation would help clarify the model's flow and the role of each BLIP instance.\n\n2. **Comparative evaluation with prior work**: To demonstrate the cross-subject generalisation capabilities of IAM, it would be valuable to directly compare its performance with existing EEG decoding frameworks such as NICE, ATM, or IDES. These previous methods often report strong within-subject decoding but struggle with generalisation across participants. A quantitative comparison would clarify how much IAM improves over these methods and could arguably represent the most impactful contribution of the paper. This deserves deeper experimental validation.\n\n3. **IAM loss components**: The IAM framework includes two main loss components: reconstruction and discrimination. However, the manuscript does not clarify the relative contribution of each to learning dataset-independent representations. An ablation or sensitivity analysis examining the effect of removing or varying the weight of each term would be informative. Similarly, it would be helpful to understand how each component of the IAM module, as illustrated in Figure 7, contributes to the overall performance.\n\n4. **Unclear evaluation setup in Table 1**: It is not clear which dataset is used for the evaluation results reported in Table 1. Are the competing methods trained and tested on the same dataset, and under the same experimental conditions? If this table is intended to demonstrate architectural effectiveness, it would also be appropriate to report the number of parameters and computational cost associated with each method.\n\n5. **Meaningfulness of comparisons in Table 2**: The results in Table 2 compare performance across different neuroimaging techniques, but it is unclear whether these results are based on the same set of images or tasks. If the underlying image sets differ, then the comparisons may not be meaningful. Please clarify whether the input stimuli are standardised across modalities, and if not, discuss the implications for interpreting these results.\n\n6. **Ablation insights and engineering influence**: The performance gap between the full model and individual ablation studies is substantial. This raises the question of whether any single component is inherently effective, or whether the observed gains primarily result from the specific combination and integration of all components. This is a critical question—does the strength of the model lie in its individual modules, or in the way they are engineered together? The manuscript would benefit from a deeper discussion of this point.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 4, "originality": 4, "strengths_and_weaknesses": "1. **Strong potential**: The manuscript addresses the challenging and important problem of decoding EEG signals across both participants and tasks. The proposed framework has significant potential and, with suitable modifications, could be extended to other neuroimaging modalities.\n\n2. **Text organisation**: The manuscript frequently refers back and forth between the main text and appendices, which disrupts the flow and makes it difficult to follow the authors' line of reasoning. In order to understand key details, I had to keep multiple tabs open simultaneously. This significantly affects the readability and accessibility of the work.\n\n3. **Overloaded scope**: Although the manuscript includes many valuable contributions, their presentation within a single compressed manuscript makes it difficult to clearly understand or scientifically assess the individual impact of each component.\n\n4. **Terminology**: The use of \"perception\" and \"semantic understanding\" throughout the manuscript is confusing. I suggest adopting more precise terms such as *visual* and *language features*, which better reflect the training objectives of the respective modules. The term \"perception\" typically refers to sensory processing (e.g., visual, auditory), while \"semantic\" can refer to either visual scene understanding (e.g., semantic segmentation) or linguistic meaning—both of which are overloaded in this context.\n\n5. **Missing details on VAE**: There is no information provided about the frozen VAE model. Was it trained by the authors on the same image dataset, or is it an existing state-of-the-art pretrained VAE? This needs clarification.\n\n6. **Reproducibility concerns**: In its current form, the manuscript lacks sufficient implementation details to allow reproducibility. Although the authors state they will release code upon acceptance, this is not a substitute for a reproducible submission.\n\n7. **Frozen adapter and prompting**: The role of the frozen adapter is unclear, and the manuscript does not explain what it refers to or how it functions. Additionally, the impact of prompt engineering on the large language model is not discussed, though this could significantly affect performance.\n\n8. **Dataset inconsistencies**: There are inconsistencies in the description of the THINGS EEG dataset. The stimuli in this dataset are presented for 100 ms with 100 ms inter-stimulus intervals, and include 16,540 images shown 4 times for training, and 200 images shown 80 times for testing. This does not align with the manuscript’s claim of “22,000 static images” and 500 ms stimulus duration, which should be corrected or clarified.\n\n9. **Appendix figure numbering**: Figures in the appendices should be labelled using a consistent scheme (e.g., A1, A2, etc.) rather than continuing the numbering from the main manuscript.\n\n10. **Figure clarity**: Figure 7 (in the appendices) contains many unexplained abbreviations, which makes it difficult to interpret. A more comprehensive caption and clearer legend would greatly improve readability.\n\n11. **Algorithm 1 clarity**: Algorithm 1 is not self-explanatory and would benefit from brief comments to aid understanding. Ideally, it should align more closely with the schematic in Figure 2, using consistent terminology and structure.\n\n12. **Need for a schematic of DSGNet**: A schematic representation of DSGNet would be highly informative and would help readers understand the model architecture at a glance.", "was_revised": true, "final_justification": "Based on the author's responses during the rebuttal period, considering the new insights and necessary changes to be included in the camera-ready version, I am increasing my rating to 5. I believe the manuscript presents an important new approach to neuroimaging decoding across participants by demonstrating the significance of pretraining on large datasets.", "rebuttal": "**Reviewer qg8v**\n\nWe believe we have resolved all your issues. If you have any other concerns or questions, we look forward to hearing from you.\n\n```Weakness```\n\nWe sincerely appreciate the reviewer's detailed identification of these presentation and technical clarity issues. We acknowledge all these concerns and commit to comprehensive revisions addressing each point. We will restructure the paper to be self-contained by moving critical technical details to the main text, adopt the reviewer's terminological suggestion to replace \"perception\" and \"semantic understanding\" with \"visual features\" and \"language features\" which better reflect our modules' training objectives, provide complete implementation details for reproducibility including VAE model specifications and frozen adapter clarifications, correct the THINGS-EEG dataset description regarding stimulus timing and duration, implement consistent figure numbering across appendices, enhance Figure 7 with comprehensive captions and clearer abbreviation explanations, improve Algorithm 1 with detailed comments, and create a dedicated DSGNet architectural diagram to help readers understand our dual-stream encoder design.\n\n```Q1:BLIP redundancy and clarification```\n\nWe understand the reviewer's confusion about the apparent BLIP-2 redundancy in our Semantic Understanding module, which stems from our unclear presentation in the flowchart. The two BLIP-2 instances utilize the same model architecture but serve fundamentally different purposes by processing entirely different inputs at distinct stages of our pipeline. The first BLIP-2 component processes the predicted keyframes generated from our EEG-to-visual reconstruction pathway, creating captions that describe what our model believes it has reconstructed from the neural signals. The second BLIP-2 instance operates on the ground truth visual content to generate reference captions for training supervision and cross-modal alignment during the hierarchical semantic contrastive learning process. These are completely independent caption generation processes with different inputs and purposes, not redundant processing of the same content. During training, we need both pathways to enable our cross-modal semantic alignment objective - aligning EEG-derived features with both reconstructed visual content and reference textual descriptions. During inference, only the first pathway operates since ground truth content is unavailable. Our flowchart design failed to clearly distinguish these different data flows and processing contexts, creating the impression of redundant processing when the pathways actually serve complementary roles in our semantic understanding architecture. In our revision, we will enhance the schematic with different color coding and clearer annotations for each BLIP-2 instance to explicitly distinguish their processing flows and functional roles.\n\n```Q2:Comparative evaluation with prior work```\n\nWe appreciate this valuable suggestion for evaluating IAM's cross-subject generalization capabilities through direct integration with existing frameworks. The reviewer correctly identifies that demonstrating IAM's effectiveness within established EEG decoding architectures would provide stronger validation of our core contribution. We have conducted preliminary experiments integrating our IAM module into existing frameworks including NICE and ATM-style architectures while maintaining their original decoder structures. When IAM is incorporated into NICE's domain adaptation framework, cross-subject performance retention improves from 62.1% to 79.4% on the same evaluation protocol. Similarly, integrating IAM into ATM's subject-invariant feature learning approach increases cross-subject classification accuracy by 11.7% while requiring 35% fewer training subjects to achieve comparable within-subject performance(detailed table will be added in revision). These controlled experiments demonstrate that IAM's hierarchical adaptation strategy provides substantial improvements over existing single-level domain adaptation approaches, regardless of the underlying decoder architecture. The consistent improvements across different base frameworks indicate that our multi-level adaptation approach (signal, spectral, temporal, semantic) more effectively addresses the complex variability patterns inherent in cross-subject EEG signals. In our revision, we will include a dedicated experimental section showing IAM's integration into multiple existing frameworks, providing comprehensive quantitative comparisons that isolate IAM's specific contributions to cross-subject generalization. \n\n```Q3:IAM loss components```\n\nOur ablation analysis reveals that both reconstruction and discrimination losses play crucial but distinct roles in learning dataset-independent representations. The reconstruction loss ($L_{recon}$) ensures that normalized EEG features retain essential visual information content, while the adversarial discrimination loss ($L_{adv}$) specifically removes subject-specific patterns that hinder generalization. When we remove the discrimination component entirely, cross-subject performance drops by 23.1%, indicating its critical role in subject disentanglement. Conversely, removing reconstruction loss reduces performance by 31.4%, demonstrating that preserving visual content is equally essential. Our sensitivity analysis shows optimal performance occurs with loss weights $\\alpha$ =1.0, $\\beta$=0.5, $\\gamma$=0.1, where increasing $\\beta$ beyond 0.6 causes over-regularization that removes useful individual neural patterns along with harmful subject-specific artifacts. Regarding the hierarchical components in Figure 7, each adaptation level contributes meaningfully to overall performance. Signal-level adaptation provides the largest individual contribution (12.3% improvement), addressing basic electrode and impedance differences between subjects. Spectral-level adaptation adds 8.7% improvement by normalizing frequency-specific neural response patterns, while temporal-level adaptation contributes 6.4% by handling timing differences in neural processing. The semantic-level adaptation, though providing the smallest individual gain (4.1%), proves crucial for maintaining high-level visual content consistency across subjects. The dynamic gating mechanism that combines these levels adaptively contributes an additional 3.2% improvement over simple weighted averaging. In our revision, we will include detailed ablation tables showing these individual contributions and sensitivity curves for the loss weight parameters to provide complete transparency about each component's role in achieving robust cross-subject generalization.\n\n```Q4:Unclear evaluation setup in Table 1```\n\nWe acknowledge the lack of clarity regarding the evaluation setup in Table 1. All competing methods were evaluated on the SEED-DV dataset using the same experimental protocol as established in the EEG2video benchmark, ensuring fair comparison across all baselines. We implemented each baseline method using their published architectures and trained them under identical conditions with consistent data splits and preprocessing protocols. In our revision, we will explicitly clarify the dataset and evaluation protocol used in Table 1, and include a supplementary table detailing parameter counts and computational costs for complete transparency.\n\n```Q5:Meaningfulness of comparisons in Table 2```\n\nWe acknowledge that the methods in Table 2 use different datasets, which requires clarification for proper interpretation. For video reconstruction, all EEG-based methods are evaluated on SEED-DV, while fMRI methods use their respective datasets due to modality-specific requirements. For image reconstruction, our cross-task evaluation uses THINGS-EEG while other methods use their original datasets. While this prevents direct stimulus-level comparison, we included these results to provide context relative to the broader neural decoding field. The key insight is that our approach achieves competitive EEG-domain performance while demonstrating unique cross-task generalization capabilities from video reconstruction (SEED-DV) to image reconstruction (THINGS-EEG) without retraining. In our revision, we will clearly annotate the specific datasets used for each method in Table 2 and reframe this comparison to emphasize our EEG-domain contributions while providing appropriate cross-modal context.\n\n```Q6:Ablation insights and engineering influence```\n\nWe appreciate this fundamental question about whether our model's strength derives from individual module effectiveness or their integrated combination. Our ablation results reveal that while each component contributes meaningfully to overall performance, the substantial performance gaps between individual ablations and the full model indicate that integration and interaction between modules is equally critical. For instance, removing the IAM causes a 35.8% performance drop, demonstrating its individual importance, but the full model's performance significantly exceeds what would be predicted by simply summing individual component contributions. This suggests that our modules exhibit synergistic effects where their combination creates capabilities that none possess individually.\n\nThe engineering integration is particularly crucial for our dual generalization objectives. The IAM normalizes subject-specific patterns, but its effectiveness depends on receiving well-structured inputs from DSGNet's dual-stream encoding. Similarly, the dual-pathway understanding modules complement each other - PU captures low-level visual dynamics that inform SU's semantic processing, while SU provides high-level context that guides PU's mechanisms. The unified inference mechanism leverages both pathways' outputs through fusion weights that adapt based on visual content complexity."}, {"review_id": "CfsGLp1YHE", "reviewer": "Reviewer_ynvm", "summary": "This paper presents a unified framework, NEED, designed to address cross-subject variability and taskspecificity in EEG-based visual reconstruciton. This framework supports zero-shot transfer between video and image tasks.", "questions": "1. What is the purpose behind the design of the Cross-Stream Integration mechanism as presented in the paper? For instance, why have you chosen to treat as a KV? What specific performance improvements does the addition of this MASK bring to the model?\n2. In the decoding module, the paper proposes modifications to common loss functions, such as incorporating the L2 norm based on temporal gradients or the Gram matrix. What is the purpose of these modifications? How would the model's performance be affected without them?\n3. In Table 2, was the performance of the proposed framework tested on the complete SEED-DV dataset, or was it only tested on the 10-class dataset?", "limitations": "No, the limitations should be included for discussions as well.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1. The authors propose a pretrained Individual Adaptation Module to address subject variability and a unified inference mechanism to generate reconstructions across different tasks.\n2. The authors provide a detailed analysis of the impact of using different electrode combinations from brain regions on the reconstruction performance in video and image reconstruction tasks.\nWeaknesses\n1. The presentation of the paper needs further improvement, particularly in the Method section. There are many undefined symbols, which make it difficult to understand.\n2. There is a lack of a quantitative comparison in the video and image reconstruction tasks. The paper includes fMRI-based methods in the comparison, despite not using an fMRI dataset. Additionally, the experimental setup is not clearly stated, making it difficult to reasonably justify the model's performance on the SEED-DV dataset.", "was_revised": true, "final_justification": "The paper targeted the core problem in the interesting topic of video and image reconstructtion from EEG. The authors actively engaged in the rebuttal and addressed my comments well. The paper could be accepted.", "rebuttal": "**Reviewer ynvm**\n\nWe believe we have resolved all your issues. If you have any other concerns or questions, we look forward to hearing from you.\n\n```Weakness```\n\nRegarding the Method section, we recognize that many mathematical symbols lack proper definition, creating barriers to understanding our technical contributions. In our revision, we will ensure every symbol is clearly defined upon introduction and provide comprehensive mathematical formulations directly in the main text rather than deferring to appendices. For the quantitative comparison concerns, we included fMRI-based methods in Table 2 to provide broader context about neural decoding performance across modalities, though we acknowledge this creates confusion about direct comparability. Our EEG-based methods are all evaluated on SEED-DV using the standard protocol established by EEG2video, while fMRI methods use their respective datasets due to modality-specific requirements. We will clarify the experimental setup more explicitly, annotate the specific datasets used for each method in Table 2, and reframe the comparison to emphasize our contributions within the EEG domain while providing appropriate context about cross-modal performance differences. Additionally, we will include more detailed experimental protocols and evaluation procedures to ensure complete transparency about our model's performance assessment.\n\n```Limitations```\n\nWe will add limitations to discussions and eventually put them into the main text.\n\n```Q1:What is the purpose behind the design of the Cross-Stream Integration mechanism...```\n\nThe asymmetric Query-Key-Value assignment reflects the fundamental nature of EEG visual processing, where spatial electrode patterns represent the current neural state (\"what\" information is being processed) while temporal dynamics encode the evolution and context of this information (\"when\" and \"how\" it develops). By using spatial features as queries, we allow the model to actively search through temporal sequences to find the most relevant temporal patterns that correspond to current spatial activations. This design choice emerged from neuroscience principles: spatial electrode configurations provide a snapshot of current brain state, but understanding visual content requires integrating this snapshot with temporal context from the memory bank provided by temporal Key-Value pairs. This asymmetric attention design proved more effective than symmetric approaches because it preserves the directional relationship between spatial states and temporal evolution. Our experiments show that reversing this assignment (temporal as Q, spatial as KV) reduces performance by 4.3%, while symmetric attention decreases accuracy by 2.8%, validating our architectural choice(detailed table will be added in revision). The temporal stream essentially serves as a contextual memory that the spatial stream queries to retrieve relevant dynamic information for visual reconstruction. The multi-dimensional masking strategy ($M_{spatial} \\odot M_{temporal} \\odot M_{frequency}$) serves critical purposes beyond regularization, significantly improving robustness to real-world EEG artifacts that are inevitable in practical applications. EEG signals frequently suffer from electrode disconnections, temporal signal loss, and frequency-specific noise that severely degrade reconstruction quality. Our masking training forces the model to learn robust representations that don't over-rely on specific electrodes or frequency bands, crucial for cross-subject generalization where individual differences in signal quality are common. Performance improvements are substantial: compared to simple feature concatenation, our attention-based integration improves 40-way classification accuracy and SSIM reconstruction quality, while the masking strategy provides additional classification improvement and reduces cross-subject performance variance (detailed cross-stream integration ablation table will be added in revision).\n\n```Q2:In the decoding module, the paper proposes modifications to common loss functions... ```\n\nThe temporal gradient L2 norm ($\\lambda\\|\\nabla_t Z_0 - \\nabla_t V_1\\|_2^2$) in our Bidirectional Dynamic Contrastive loss ensures that EEG representations and visual features maintain similar temporal dynamics, not just static alignment. EEG signals inherently capture the brain's response to dynamic visual changes, but standard contrastive learning only aligns feature representations without considering their temporal evolution patterns. This temporal constraint is critical because visual perception involves continuous neural adaptation to changing stimuli, and this temporal structure contains essential motion and transition information that static alignment cannot capture.\n\nThe Gram matrix term ($\\beta \\cdot \\|G(Z_s)-G(C_k)\\|_F$) in our Hierarchical Semantic Contrastive loss captures structural relationships between semantic features rather than just individual feature values. The Gram matrix computes correlations between different feature dimensions, effectively encoding the relational patterns of semantic representations. This is particularly important for cross-subject generalization because while individuals might have varying neural activation magnitudes, the structural relationships between semantic concepts tend to be more consistent across subjects. For example, understanding \"dog in grass\" requires recognizing both individual concepts and their spatial-semantic relationships.\n\nPerformance impact is substantial when these modifications are removed. Without the temporal gradient term, video reconstruction loses temporal coherence with FVD increasing from 175.3 to 195.3 and CLIP-pcc dropping from 0.738 to 0.689, indicating significant degradation in temporal consistency (Table 5 in Appendix, section 4). Removing the Gram matrix constraint more dramatically affects cross-subject generalization, with 40-way classification accuracy dropping from 0.372 to 0.298 when tested on unseen subjects (detailed semantic loss ablation will be added in revision). These modifications work synergistically to capture both the dynamic aspects essential for video reconstruction and the structural semantic relationships necessary for robust cross-subject transfer\n\n```Q3:In Table 2, was the performance of the proposed framework tested on the complete SEED-DV dataset...```\n\nThe performance results in Table 2 for our NEED framework were obtained using the complete SEED-DV dataset with all 40 visual categories, not a reduced subset. Our full model achieved 0.898±0.02 for 2-way classification and 0.405±0.02 for 40-way classification on the complete dataset. We have also conducted experiments on 10-class subsets where our method achieves higher absolute performance (0.623±0.03 for 10-way classification, detailed table will be added in revision), but we report the full 40-class results to provide a more rigorous assessment of our cross-subject and cross-task generalization capabilities. Testing on the complete semantic diversity is necessary to meaningfully validate our claims about handling inter-subject variability in neural visual processing across the full range of visual categories."}, {"review_id": "8jAN1KEfBk", "reviewer": "Reviewer_KAiD", "summary": "This paper proposes a novel framework named NEED to address the challenging problem of zero-shot cross-subject and cross-task generalization in EEG-based visual reconstruction. The framework comprises three core components: an Individual Adaptation Module (IAM) for normalizing inter-subject variability, a Dual-Stream EEG Encoder (DSGNet) for capturing both spatial and temporal features, and a Unified Inference Mechanism to enable task-adaptive reasoning. Extensive experiments on visual perception classification and visual reconstruction tasks demonstrate that NEED consistently outperforms existing baselines, highlighting its strong generalization ability and robustness.", "questions": "1. It is quite surprising that DSGNet achieves over a 100% improvement in 40-class classification accuracy on the SEED-DV dataset compared to baseline models. Could you provide more details on the experimental setup and share the corresponding code for DSGNet? Additionally, which specific design choices contribute to such a substantial performance gain?\n2. EEG2Video employs GLMNet as its EEG encoder, whereas NEED uses DSGNet. Although DSGNet clearly outperforms GLMNet in video perception classification, NEED does not show a correspondingly large improvement in video reconstruction performance compared to EEG2Video. Have you considered controlling for the encoder variable by using GLMNet within the NEED framework to allow for a more direct and fair comparison of the reconstruction capabilities?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strength\n\n1. This work introduces the first unified framework, NEED, that achieves zero-shot generalization across both subjects and tasks in EEG decoding, addressing two fundamental challenges in the field.\n\n2. The proposed approach is comprehensively designed, encompassing critical aspects such as signal preprocessing, spatiotemporal feature extraction, and multimodal fusion.\n\n3. Extensive evaluations on benchmark datasets demonstrate significant performance improvements, particularly under cross-subject and cross-task scenarios. Thorough ablation studies (including IAM, PU, and SU modules) validate the contribution of each component and confirm the robustness of the overall framework.\n\n\n### Weakness\n\n1. The overall framework contains numerous modules, but the motivations behind some components are not clearly articulated. For example, in the Temporal Stream Processing part of DSGNet, the rationale for introducing the generalized Riemann-Liouville fractional transformation is not sufficiently explained.\n\n2. Some mathematical formulations lack complete variable definitions, which affects clarity and reproducibility. For instance, in Section 2.2 (Line 99), the multi-objective loss function is presented without adequate explanation of its variables. Similarly, in Section 2.3 (Line 140), the source of the enhanced visual features V1 is unclear.\n\n3. The section titles and their content are misaligned: Section 4.1 is titled \"Visual Perception Benchmark\", yet it only includes video perception results and omits image perception. Conversely, Section 4.2 is titled \"Video Reconstruction Results\", but includes both video and image reconstruction tasks. Renaming it to \"Visual Reconstruction Results\" would better reflect the content.\n\n4. The manuscript contains a number of typographical and formatting issues, such as:\n   - Most subscripts in equations are not enclosed using `\\textbf{}` for consistent formatting;\n   - Punctuation errors, e.g., missing periods in Lines 72 and 107, an extra period in Line 214, and a missing space in the formula in Line 98;\n   - Inconsistent figure references—some include a period after \"Fig\", while others do not. A thorough proofreading is recommended to ensure consistency.", "was_revised": true, "final_justification": "The clarification issues are resolved. And the authors commit to including the complete DSGNet implementation code link in the final version to ensure full reproducibility and enable community validation of these results.", "rebuttal": "**Reviewer KAiD**\n\nWe believe we have resolved all your issues. If you have any other concerns or questions, we look forward to hearing from you.\n\n```Weakness```\n\nWe acknowledge these important technical presentation issues and appreciate the reviewer's careful attention to detail. We recognize that our motivation for the Riemann-Liouville fractional transformation and other component choices requires clearer articulation - these design decisions reflect fundamental insights about EEG's unique temporal dynamics that we inadequately communicated. Regarding mathematical clarity, we will ensure all variables in formulations like our multi-objective loss function and enhanced visual features V1 are properly defined with complete explanations. We also acknowledge the section title misalignments and will restructure appropriately, renaming sections to accurately reflect their content scope covering both video and image reconstruction tasks. Finally, we will address all typographical and formatting issues including proper equation subscript formatting using \\textbf{}, correcting punctuation errors throughout Lines 72-214, and establishing consistent figure reference formatting. \n\n```Q1:It is quite surprising that DSGNet achieves over a 100% improvement...```\n\nThe significant performance improvement  reflects our dual-stream architecture's ability to address fundamental limitations in existing EEG classification approaches. Most baseline methods treat spatial and temporal information separately or use simple concatenation, missing the crucial spatiotemporal interactions that encode visual information in neural signals. Our DSGNet simultaneously captures spatial electrode relationships through spherical harmonic embeddings that model the topographical structure of EEG recordings and multi-scale temporal dynamics via fractional transformations with multiple alpha parameters that naturally model EEG's power-law temporal relationships and long-range dependencies that standard convolutions cannot capture. The cross-stream attention mechanism then dynamically integrates these complementary streams by allowing spatial patterns to query relevant temporal contexts, enabling the model to adaptively focus on different temporal scales based on the visual content being processed. \n\nThe substantial relative gains also reflect the inherent difficulty of 40-class visual classification from EEG signals, where even modest architectural improvements translate to large performance increases due to challenging baseline performance levels near the 2.5% chance level. Our multi-dimensional masking strategy during training simulates real-world EEG artifacts by randomly masking spatial electrodes, temporal segments, and frequency bands, forcing the model to learn robust representations that don't over-rely on specific signal components, which significantly improves generalization to degraded signals that commonly occur in practical applications. Additionally, our graph convolutional processing of electrode relationships and dilated temporal convolutions with increasing receptive fields enable the model to capture both local and global spatiotemporal patterns that are essential for discriminating between diverse visual categories. We commit to including the complete DSGNet implementation code link in the final version to ensure full reproducibility and enable community validation of these results.\n\n```Q2:EEG2Video employs GLMNet as its EEG encoder, whereas NEED uses DSGNet...```\n\nThis apparent discrepancy reflects the different requirements and bottlenecks in classification versus reconstruction tasks. Classification performance primarily depends on the encoder's ability to extract discriminative features, where DSGNet's dual-stream architecture and cross-stream attention provide clear advantages. However, video reconstruction performance is influenced by multiple components beyond the encoder, including the decoder architecture, loss functions, and conditioning mechanisms. In reconstruction tasks, factors like temporal coherence, visual fidelity, and semantic consistency become equally important, and improvements in encoder performance don't necessarily translate linearly to reconstruction quality.\n\nRegarding the suggestion to use GLMNet within our NEED framework for controlled comparison, we conducted preliminary experiments replacing DSGNet with GLMNet while maintaining our IAM, dual-pathway understanding, and unified inference components. Results show that NEED with GLMNet achieves 0.867±0.03 for 2-way video classification compared to our 0.898±0.02 with DSGNet, and reconstruction SSIM of 0.321±0.03 versus our 0.356±0.02, confirming that DSGNet contributes meaningfully but not exclusively to our performance gains(detailed table will be added in revision). This suggests that our framework's advantages stem from the synergistic combination of multiple innovations rather than encoder improvements alone. We will include this controlled comparison in our revision to provide clearer attribution of performance improvements across different components of our framework."}]

Author rebuttal:

Meta-review: This paper proposes a framework for the challenging problem of EEG-based visual reconstruction, focusing on cross-subject and cross-task generalization. After a thorough review process, including a highly constructive author rebuttal, the recommendation is to accept this paper for a poster presentation.

The decision is based on the soundness of the core technical contribution, which became evident after the author-reviewer discussion. The initial submission suffered from presentation and clarity issues, but the authors' detailed rebuttal convincingly addressed the reviewers' concerns and clarified the value of the work.

### Summary

The paper introduces the "NEED" framework, designed to perform zero-shot reconstruction of visual stimuli from EEG signals, generalizing across both subjects and tasks (video to image). The work's primary contributions are:

- An Individual Adaptation Module (IAM) to normalize inter-subject EEG variability.
- A dual-pathway EEG encoder (DSGNet) for processing both perceptual and semantic information.
- A unified inference mechanism for task generalization.

The experimental results validate these claims, showing that the model outperforms prior methods and, importantly, maintains a high percentage of its performance (>90%) when applied to unseen subjects.

### Strengths
There was a consensus among the reviewers on the paper's key strengths:
- The paper tackles a well-established and difficult challenge in brain-computer interfaces. 
- The experimental evaluation is extensive. The quantitative results, particularly the cross-subject generalization performance, are compelling and support the central claims.
- The proposed combination of modules into a unified framework to address the dual-generalization problem is a novel contribution.

### Weaknesses
The initial reviews identified several major weaknesses, primarily related to the quality of the presentation:
- The original manuscript was not self-contained. Critical details about the methodology were located in the appendix, which hindered a clear understanding of the work.
- The proposed framework is highly complex. The paper did not adequately explain the rationale behind many of its architectural choices, leaving reviewers to question if a simpler approach might suffice.
- The initial submission contained undefined symbols, ambiguous figures, and unclear experimental details, raising concerns about reproducibility.

### Reasons for Recommendation
The recommendation for acceptance is principally driven by the paper's technical substance and a good rebuttal. The identified weaknesses are largely confined to the paper's exposition and are deemed correctable.

The recommendation is for a poster presentation. While the technical contribution is solid, the paper required a substantial amount of clarification and revision during the review cycle to reach an acceptable state.

### Summary of Discussion and Rebuttal
The author-reviewer discussion was effective. The initial set of reviews was borderline, reflecting the conflict between the interesting results and the flawed presentation. The authors' rebuttal systematically addressed the reviewers' main points:
- The authors acknowledged the organizational flaws and provided a clear plan to restructure the paper, promising to integrate all essential technical details into the main text.
- In addition to providing better justifications, the authors conducted new ablation studies during the rebuttal period. This new evidence demonstrated the utility of their specific design choices over more standard alternatives, directly addressing reviewer concerns about the model's complexity.
- Responding to reviewer requests, the authors ran new experiments to better isolate the contributions of their proposed modules (e.g., integrating their IAM into prior frameworks). This proactive effort significantly strengthened the paper's claims.

The quality of the rebuttal convinced Reviewer qg8v to raise their score to "Accept." The final comments from other reviewers indicated their primary concerns were also resolved, pending the promised revisions. The authors' commitment to releasing their code helps to ensure the work will be reproducible.
Decision: Accept (poster)