ZEBRA: Towards Zero-Shot Cross-Subject Generalization for Universal Brain Visual Decoding
Authors: Haonan Wang
Jingyu Lu
Hongrui Li
Xiaomeng Li
Venue: NeurIPS 2025 poster ()
Keywords: Zero-shot Brain Decoding
fMRI-to-Image Reconstruction
Image Generation
Primary area: neuroscience_and_cognitive_science

Abstract: Recent advances in neural decoding have enabled the reconstruction of visual experiences from brain activity, positioning fMRI-to-image reconstruction as a promising bridge between neuroscience and computer vision. However, current methods predominantly rely on subject-speciﬁc models or require subject-speciﬁc ﬁne-tuning, limiting their scalability and real-world applicability. In this work, we introduce ZEBRA, the ﬁrst zero-shot brain visual decoding framework that eliminates the need for subject-speciﬁc adaptation. Z EBRA is built on the key insight that fMRI representations can be decomposed into subject-related and semantic-related components. By leveraging adversarial training, our method explicitly disentangles these components to isolate subject-invariant, semantic-speciﬁc representations. This disentanglement allows ZEBRA to generalize to unseen subjects without any additional fMRI data or retraining. Extensive experiments show that ZEBRA signiﬁcantly outperforms zero-shot baselines and achieves performance comparable to fully ﬁnetuned models on several metrics. Our work represents a scalable and practical step toward universal neural decoding. Code and model weights are available at: https://github.com/xmed-lab/ZEBRA.

ZEBRA: Towards Zero-Shot Cross-Subject Generalization for Universal Brain Visual Decoding
Haonan Wang, Jingyu Lu, Hongrui Li, Xiaomeng Li"
The Hong Kong University of Science and Technology hwanggrSconnect, uuuuusthi hk, eexmli tuustttut....mm
Abstract
Recent advances neural decoding have enabled the reconstruction of visual experiences from brain activity, positioning fMRI-to-image reconstruction as a promising bridge between neuroscience and computer vision. However, current methods predominantly rely subject-specifie models or require subject-specific fine-tuning, limiting their scalability and real real-world applicability. introduce ZEBRA, the first zero-shot brain visual decoding framework that elimi- nates the need for subject-specific adaptation. ZEBRA is built on the key insight that fMRI representations can be decomposed into subject-related and cemantic-relater components. By leveraging adversarial training, our method explicitly disentangles these components to isolate subject-invariane semantic- spreticc-spifiic representations. This disentanglement allows ZEBRA A to generalize to unseen subjects without any additional fMRI data retraining. Extensive experiments show that ZEBRA sig- nificantly outperforms zero-shot baselines and achieves performance comparable fully finetuned models on several metrics. Our work represents scalable and practical step toward universal neural decoding. Code and model weights are available at: wtwphhhhw..oommmmmeeaaa///ggg
this work, we
Introduction
The compelling connection between neural decoding and visual understanding has positioned fMRI- to-image reconstruction [ computer vision. As non non-invasive method for observing activity in the brain's visual cortex, fMRI signals offer the intriguing possibility of reverse-engincering human perception--translaiinng blood- Saygen-leve--deeedde (BOLD) responses into detailed visual reconstructions of what person sees. This ability, often described as "brain camera," marks major shift, with potential applications in mental state interpretation g and advanced brain-computer interfaces 10)
at the forefront of computational neuroscience and
Despite remarkable progress in reconstructing images from individual brain data, the field faces critical challenge: current models struggle to generalize across individuals. This limitation risks confining such breakthroughs to research labs rather than enabling real-world applications. Recent have attempted address this issue by developing cross-subject reconstruction through a two-step approach: first, pretraining a unified model on multi multi-subject data, followed by subject- speecfff finetuning imposes significant barriers to practical clinical use due to several limitations: (1) Clinicians and neuroscientists must still depend Al experts fine-tune models for each patient. (2) The fine-tuning process is time-intensive, often taking around a day, which hampers real-time applications in brain-computer interfaces ? I0 and neurorehabilitation [ (3) There is no universal feature space capable of learning neural representations across human subjects, restricting broader exploration in cognitive
efforts 4 6
specific finetuning, illustrated in Fig. p However,
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

pretrain
(b)
training
Previous Methods
Subjects
Roctune
Train Subjects
ZEBRA
Previous Methods
inference
Previous
Test Subjects
Methods
lest Subjects
(Seen)
(Completely Unseen)
Need additional data for fine-tuning subjects
Tuning freet Zero-shot inference new subjects!
Figure I: (a) Previous methods 4 5 6 typically involve two training stages: (1) pretraining brain model with multiple subjects, and (2) fine-tuning the model for a specific subject. this approach, the test subject is known to the model, which limits its zero-shot capability for new subjects. (b) In contrast, ZEBRA eliminates the fine-tuning stage, requiring training only once with the training subjects. This allows it to perform zero-shot inference unseen subjects, achieving comparable performance to the fine-tuned approaches.
functions and variability. Thus, while developing zero-shot cross-subject generalization methods for brain visual decoding is critical, this area remains largely unexplored.
straightforward approach would be evaluate the previous state-of-the-arr method, MindTTuner [ in zero-shot setting and attempt to improve upon it. However, this is infeasible due to its subject- specific design, which tailored certain information content of representations across subjects. As result, it fails when tested on new subject whose representation carries different amount of information. NeuroPictor [5] offers valuable insights by transforming fMRI data from different subjects into uniformly shaped 2D representations with spatial information preserved, facilitating the learning of a shared latent space. Nonetheless, its zero-shot performance remains limited since is sensitive to subject noise, shown in Fig. 6 and thus fails to learn invariant representations across subjects. Motivated by these observations, one may consider combining NeuroPictor's s powerful unified brain encoding with the powerful decoding of MindTTuner. Yet, this possibly strongest baseline fails to achieve satisfactory results, as evidenced by the "Our baseline" and "NeuroPictor"" rows in Table
To pave the way for zero-shot brain visual decoding. build on this baseline, we propose EEBRA-the first zero-shot brain visual decoding framework -thht enables direct generalization to unscen subjects with- out requiring additional iMRI data or model retrain- ing (Fig. [D). As illustrated in Fig. 2 the core idea of ZEBRA is to disentangle fMRI-derived features into four complementary components, with focus on learning subject-invariee invariant and -peeancc--peccficc representations. This design is motivated by neuro- scientific evidence that, despite inter-individual vari- ability in brain activity, the human cortex encodes semantic information in a consistent and topograph- ically organized manner across subjects I2
C Subject
Samantic
Push
mull
0000 E
m F2
Subject- Invariant Feature Extraction
Semantic- Specific Feature Extraction
[3 [4].
Figure 2: Core idea of ZEBRA. F, is used as diffusion prior guidance.
For reconstruction framework to generalize effec- tively across individuals, it should preserve subject- invariant (universal brain representations) and semantic-specifie blaas-dissrimminttive components, while suppressing subject-specific [I5] and semantically irrelevant variations. To this end, ZEBRA first extracts subject-invarian features by removing subject-specific noise via residual decomposition and adversarial training. In parallel, semantic-specific features are projected into a shared visual- semantic space and aligned with CLIPP embeddings, ensuring semantic-level discrininability while remaining agnostic to subject identity. This disentanglement strategy enables robust cross-subject generalization and facilitates scalable iMRI-to-imen -maee decoding.

---

Extensive experiments validate the effectiveness of ZEBRA, particularly on low-level perceptual and pixel-wise metries. ZEBRA achieves substantial improvement PixCorr, with a gain of +0.084 (0.153 vs. 0.069 of NeuroPictor), and an average improvement of +6.4 percentage points (81.8% vs. 75.4%) on Alex (5). More importantly, ZEBRA shows performance that comparable to fully-finetuned methods in several metrics. For instance, it achieves an SSIM of 0.384, close to 0.375 of NeuroPictor (fully finetuned), despite not using any test subject data. Qualitative results further confirm that the visual reconstructions generated by ZEBRA are competitive with those produced by fully finetuned subject-specífic models.
Our contributions can be summarized as follows: (1) We propose ZEBRA, the first zero-shot brain visual decoding framework that generalizes to unseen subjects without requiring additional fMRI data or finetuning: (2) We introduce a novel disentanglement strategy combining adversarial training and residual decomposition to learn subject-invariant and semantie-specife representations from fMRI signals; (3) We demonstrate that ZEBRA achieves competitive performance compared to few-shot and fully fine-tuned subject-specifer methods across multiple quantitative and qualitative benchmarks.
2 Related Works
2.1 IMRI-to-lmage Reconstruction
In recent years, with the rapid advancement and widespread adoption of functional magnetic resonance imaging (fMRI), researchers have increasingly recognized the critical value of fMRI signals for neuroscience research. Leveraging advanced technological tools, several datasets linking fMRI signals to images have emerged [17 I8 [9] with the Natural Scenes Dataset (NSD) being exemplary instance 77. Utilizing these datasets, the task of IMRI-to-image reconstruction has undergone significant development. Early research predominantly employed neural networks such Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) for reconstructing images However, these studies were either restricted reconstructing simplistic digit images or lacked the capability to effectively represent both high-level and low- level semantic details clearly. The advent of diffusion models marked transformative milestone, catalyzing numerous novel studies 4 6
from fMRI signals 20 21
22 23
28 Diffusion-based methods excel in capturing complex, high-dimensional semantic information. To effectively represent low-level spatial details, some approaches have incorporated blurry images as control inputs 4 M. while Moreover, certain studies have further refined high-dimensional semantic content by leveraging captions or labels [4, o 26] By integrating both high-level semantic guidance and low-level spatial control, many recent approaches have produced intuitive and compelling reconstruction outcomes.
244
25
26
5
others have utilized contours or masks 25 28
2.2 Cross-Subject IMRI-to-Image Reconstruction
In the medical domain, fMRI signals exhibit substantial variability across individuals due to differ- ences in anatomical and physiological structures. Given the challenges associated with acquiring fMRI data, research focusing on cross-subject adaptation is profoundly significant. Central to cross- subject studies the alignment of fMRI signals across different subjects. While anatomical alignment provides foundational step, numerous studies emphasize the greater importance of functional alignment (29, 30 Numerous studies tried to address this issue [1
32
33
3G S Initial
approaches typically trained separate models for each subject, resulting in poor generalizability and substantial data dependency issues 3 Subsequent methodologies, such as MindBridge 3T] and
MindEye2 4. sought to develop subject-agnostic networks by constructing shared latent space. However, these models often require extensive fine-tuning when encountering new subjects. While MindTuner [ partially mitigates this requirement, it does not fundamentally resolve the persistent reliance on subject-specific data in cross-subject fMRI research.
3 Method
As illustrated in Fig. 3 based on a baseline with a ViT-based brain encoding backbone and a unCLIP
generative model $3. ZEBRA improves this with two main components: (1) a Subject-Invarian
Feature Extraction module that maps brain visual representations to shared latent space by exploring
3

---

Training
Subject-Invarienn Invariant Feature Extraction
Semantic-Speciiie Feature Extraction
cts
Vision Projector
C
CLS
E Difference
GRL
F
P((.)
C Subject
θ
Semantic
Vision Projector
dis
DIS
spe
E
P3 (-)
Fs
IMRI
FOO
ViT
0000
Vision Projector P())
0ии0 F
Eprior
Visual stimuli O
Decoder
Image labels
Train Subjects
Image captions
Visual Stimuli
EMRI
Inference
Reconstructed Images
P&O
Fi(-)
$E9 ViT
Vision Projector
SDXL
iMRI
Diffusion Prior
Test Unseen Subject
Figure 3: ZEBRA consists of two key components: (1) Subject-Invarian Feature Extraction, which disentangles subject-invariant representations from brain activity using adversarial learning and residual decomposition ($3.2); and (2) Semantic-Speciffi Feature Extraction, which aligns semantic information in brain features with vision-language embeddings via supervised learning and gradient reversal ($3.3 During inference, only the invariant projection path used, enabling zero-shot generalization to unseen subjects.
subject-invariant features ($3.2). and (2) a Semantic- Feature Extraction module that injects semantic features into the shared latent space. (3..))
3.1 Baseline Framework
pretrained on the UK Biobank dataset [77] that maps fMRI data from different subjects into a shared latent space, and diffusion prior network that converts the latent brain embeddings into vision features for image generation using Stable Diffusion. Given an fMRI scan and its corresponding visual stimulus y, we first transform the fMRI data into a unified 2D brain activation map (35, resulting in single-channel image E 256 The fMRI encoder then converts this 2D surface map into latent representation E BxLCi where B is the batch size, L is the number of tokens, and C is the brain feature dimension. Then the latent representation E converted to embeddings in CLIP space F < BxCx2 as guidance for reconstruction. Following MindEye2 [ we utilize diffusion prior B8 to transform the fMRI-LIIP embedding into a reconstructed OpenCLIP vision embedding F, of the corresponding visual stimulus. Similar DALL-E 2 38), the diffusion prior is trained to minimize the mean squared error (MSE) between predicted and target embeddings:
Our baseline framework primarily consists of a ViT-based fMRI encoder (IMRI-PTE 33
Cprior EF, (0.1 [|c(F) F2
(1)
The training of the baseline model involves three losses: (1) a contrastive loss LeLIP, between the predicted CLIP text embedding F' and the ground truth Ff, (2) contrastive loss LCLIP, between predicted CLIP vision embedding E BxNxC and the ground truth Fý; and (3) the diffusion prior loss Lprior. Both LCLIP, and LCLIP, adopt the BiMixCo loss, which aligns video frames and corresponding fMRI signals using a bidirectional contrastive objective and MixCo-based data augmentation, detailed in the Supplementary.

---

3.2 Subject-Invarinne Feature Extraction
Residual Decomposition Adversarial Training. fMRI signals are highly idiosyncratic across individuals, making direct modeling challenging. To enable generalization, it is essential to filter out subject-specific noise and retain only invariant, stimulus-relevant components. The goal of Invariant Feature Extraction (SIFE) is to disentangle general brain representations into two components: subject-invariant features and subject-specific features. This disentanglement is achieved via residual decomposition, where employ self-attention blocks F.(.) as the invariant feature extractor to derive subject-invariant features: E; = (E) The choice of self-attention not essential enforcing invariance-which is primarily driven by the gradient reversal layer (GRL) and the associated adversarial losses-but serves to maintain architectural consistency with the brain encoder based on ViT. Subsequenty, the subject-specifie features are obtained as the residual difference: E,- E E, This residual decomposition ensures that the extracted E; captures components common across subjects, while E, accounts for individual variabillity. The decomposition is further regularized by adversarial and supervised objectives to enforce disentanglement constraints and semantic consistency. To ensure that E, truly invariant to subject identity, we apply an adversarial training strategy [39]. Specifically, subject discriminator Dais is trained to maximize its ability to predict the subject label from E;, while the invariant extractor F; aims to produce features that prevent Dais from correctly identifying the subject. This adversarial objective is formulated as the following min-max game:
min max { -Ez,a- log Dais (E(F((() 0y Prau
where s denotes the subject label. To guide the learning of subject-specific features E,, we introduce subject classifier Dele. which is trained to predict the subject identity from Eg. Its parameters &D... are optimized via the following classification loss:
min Br
{Co E, (s log Dato (E.)))
The residual decomposition mechanism inherently reduces the subject-specific signal in E, thereby promoting the inclusion of subject-invariant content. We assume that subject-irelleaat features largely overlap with shared semantic representations. Thus, enforcing E; to be adversarially invariant ensures that encodes generalizable brain features across individuals.
Representation Preservation Anchor. While adversarial training promotes subject-invariant repre- sentation learning, it may inadvertently distort the original high- sigh-dimensional brain feature space E. To counteract this, we introduce a representation preservation anchor via an auxiliary fMRI recon- struction task. Specifically, layers followed by a linear prediction head-to reconstruct the input signal as 1 = Dree (E). We adopt the mean absolute error (MAE) as the reconstruction loss, defined as:
employ masked decoder comppiinnn two deconvolution
Cree = B(x,))~x [| x1]-
This reconstruction task serves as an anchor to preserve essential neural information in the latent space, ensuring retains both biological fidelity and semantic coherence under adversarial training.
3.3 Semantic-Speciffi Feature Extraction
Given the subject-invariant brain representation E, from SIFE, we further inject semantic information from stimuli to enhance comantic-specife alignment. Similar to SIFE, the semantic-Specife Feature Extraction (SSFE) module consists of two components: Adversarial Training and Representation Preservation Anchor.
Adversarial Training. We project brain features into the CLIP vision space using vision projectors composed of three linear layers with GELU activation 40] yielding three types of Cl.P-aligned embeddings: semantic-specifee features F,- P.(E,), (E)), semantie-invariant features F, Pi( (E,), and general representations F = PEE). The disentanglement between F, and F; is driven by the residual decomposition in the brain feature space. To ensure F, captures meaningful semantic information, we directly align it with OpenCLIP vision embeddings F,, using the BiMixCo loss spe detailed in the Supplementary.
5

---

To reinforce the semantic purity of Fs, we apply adversarial training to F., encouraging it to encode minimal semantic content. As result of the residual structure, this pushes more semantic information into F,. Specif- ically, we prepend gradient rever- sal layer 4II before the projection: F, = P. (RL(E,)) discouraging E, from aligning with CLIP target fea- tures. The corresponding adversarial loss L is also a BiMixCo loss with min-max game similar to Eq. 2 and omitted here.
CLaP
CLIP Text Encoder
Image Captions
Projectur
CLuP
Visual Stimall
Vision Projector
Lets
Image
Image Labels
D M
Figure 4: Representation Preservation Anchor of SSFE.
Representation Preservation Anchor. Similar to SIFE, to ensure semantic consistency in the latent space, we introduce a preservation anchor by aligning CLIP embeddings across three perspectives: classification, vision, and text. We employ linear classifier C(-) on the E' to predict the image label é, with cross-entropy loss Lels- For vision and text alignments, we adopt BiMixCo losses LCUP, and LCIP, to align to CLIP image and text embeddings. The total semantic loss is Lyem Cas + LCLP. + LCur, The overall training loss can be described as:
C= Crec + C.i + Lels + L. Eape + Loem + Apprior
(5)
where À is set to 30 following previous methods 4.
4 Experiments
4.1
Experimental Setup
Dataset. We use the Natural Scenes Dataset (NSD) for both training and evaluation. NSD contains visual image stimulus and corresponding IMRI recordings of 8 subjects, with each subject viewing 8,000-9,000 images. The original images are collected from MS-COCO dataset 42), which consisted of complex natural images. Following (2).
the corresponding captions of the images in COCO dataset for training. For both training and evaluation, we average three trials of fMRI signal of the same images following [5]. The final results were tested subjects 1, 2, 5 or 7, since these subjects complete all scanning sessions, sharing the same 982 images as testing data. For each test subject, use all other 7 subjects to train the model and tested on the unseen subject with unseen test split.
Evaluation Metrics. We follow the metrics of Mindeye2 3 to evaluate both high-level and low-level consistency, On the low-level aspect, we use pixelwise correlation, Structural Similarity Index Metric AlexNet(2), and AlexNet(S). High-level metrics are calculated by extracting features using specific networks, including EffNet-B 44. SwAV refer to the Supplementary for more details.
(SSIM) 43
45 Inception 46), and CLIP Please
Implementation Details. All experiments were conducted for 60 epochs using NVIDIA RTX H800 GPUs with a total batch size of 128 (16 samples per GPU). We adopt the AdamW optimizer 4 with learning rate of 1e-4, following the OneCycle learning rate schedule [49] In the inference stage, we follow MindEye2's two-stage decoding process. First, the predicted image latents are decoded into coarse images using SDXL unCLIP. These coarse outputs are then refined using base SDXL in image-to-image mode, guided by predicted captions. The refinement starts from noised version of coarse image, skipping the first 50% of diffusion steps.
4.2 Main Results
Quantitative Results. We evaluate ZEBRA against representative methods across various training regimes on the Natural Scenes Dataset, with results averaged over subjects 2, 5, and As shown in Table I ZEBRA achieves competitive performance without using any subject-specific data, highlighting its strong generalization ability in the zero-shot setting. Compared to the only other zero-shot-compatibbe baseline, NeuroPictor*. ZEBRA A achieves substantial improvements across all metrics. On lew-level similarity metrics, ZEBRA improves PixCorr from 0.057 to 0.131 and SSIM from 0.297 to 0.375, Similarly, high-level perceptual metrics show consistent gains: ZEBRA achieves

---

Table 1: Quantitative comparison of ZEBRA against representative methods under different training regimes. Results are averaged over subjects 1, 2, 5, and 7 from the Natural Scenes Dataset. Fully fine-tumed methods are trained on 40-hour data from the test subject, few-shot methods use only 1-hour recordings, while zero-shot methods do not use any data from the test subjects. "NeuroPictor denotes version pretrained on 40-hour data from all subjects without fine-tuning on the test subjects. "NeuroPictor"" represents our implementation in a zero-shot setting (pretrained on the other subjects). Previous methods-exee sethods-excluding NeuroPictor-are fundamentally infeasible in zero-shot scenarios because their architectures depend on subject-specific linear mappings. "Our baseline" refers to the strong baseline 3.1 combining NeuroPictor and MindTuner.
Low-Level
High-Level
Method
PixCorr
SSIM
Alex(2)t
Alex(5)1
Incept
CLIP
Effj
SWAV.
Fully fine-tuned
Takagi...
[CVPR123]
0.246
0.410
78.9%
85.6%
83.8%
82.1%
0.811
0.504
Ozcelik...
2
Kop/23)
0.273
0.365
94.4%
96.6%
91.3%
90.9%
0.728
0.422
MindEyel
[NeurIPS:24]
0.319
0.360
92.8%
96.9%
94.6%
93.3%
0.648
0.377
UMBRAE
x4
[BCCVV22]
0,283
0.341
95.5%
97.0%
91.7%
93.5%
0.700
0.393
NeuroPictor
[BCCV124]
0,229
0,375
96.5%
98.4%
94.5%
93.3%
0.639
0,350
NeuroPictor
[BCCV124]
0,141
0,349
91.4%
95.7%
88.3%
88.9%
0.722
0.417
MindBridge
[CVPR/24]
0.151
0,263
87.7%
95.5%
92.4%
94.7%
0.712
0.418
MindHye2 4
[CML/24]
0.322
0.431
96.1%
98.6%
95.4%
93.0%
0.619
0.344
MindTuner
0
[AAA125]
0.322
0.421
95.8%
98.8%
95.6%
93.8%
0.612
0.340
Few-shot
Min A (1 hour)
0.195
0.419 0.420
84.2%
90.6% 93.6%
81.2%
79. 79.2%
0.810
0.468
MindTuner 6 (1 hour)
0.224
87.8%
84.8%
83.5%
0.780
0.440
Zero-shot
NeuroPictor'
0,057
0.297
71.4%
74.7%
62.5%
66.0%
0.939
0.607
Our baseline
0,074
0.316
70.8%
74.0%
63.5%
62.5%
0.920
0.602
ZEBRA
0.131
0.375
74.6%
81.2%
72.2%
71.5%
0.837
0.506
Table 2: Ablations on the key components of ZEBRA, and results are from subject 1. 'Adv.' denotes adversarial training and 'Anchor' stands for preservation anchor.
Basc
SIFE
SSFE
Low-Level
High-Lovel
line
Adv.
Anchor
Adv.
Anchor
PixCorr
SSIM1
Alex(2y1
Alcx(5)1
Incept
CLIP1
Eff]
SwAV.
0.089
0.325
72.5%
74.7%
64.7%
63.2%
0.891
0.579
0.129
0.355
73.9%
77.4%
68.0%
66.8%
0.885
0.545
/
0.134
0,368
74.3%
78.3%
70.0%
69.3%
0.855
0.525
0.142
0.374
75.2%
79.6%
71.4%
70.8%
0.832
0.505
/
0.153
0.384
76.1%
81.8%
73.4%
72.3%
0.814
0.490
74.6% and 81.2% on AlexNet(2) and AlexNet(S), respectively, compared 71.4% and 74.7% for NeuroPictor".. On high-level semantic metrics, ZEBRA outperforms NeuroPictor with margins of +9.7% on Inception (72.2% vs. 62.5%) and +5.5% on CLIP similarity (71.5% vs. 66.0%). ZEBRA also demonstrates lower perceptual distance, with Eff decreasing from 0.939 to 0.837, and SwAV from 0.607 to 0.506, indicating stronger alignment with ground-truth representations. These results demonstrate ZEBRA' A ''s clear advantage in generalizing across subjects without requiring fine-tuning. While fully fine-tuned methods unsurprisingly perform better due to access to the data of test subjects, ZEBRA narrows this gap significantly. For instance, in the zero-shot setting, ZEBRA achieves 74.6% on AlexNet(2), compared to 78.9% by Takagi et al. M and 87.7% by MindBridge [31. despite using no data from the test subjects. In summary, ZEBRA sets new state of the art in zero-shot neural decoding, outperforming prior zero-shot methods by large margins across all metrics, and approaching the performance of few-shot and even some fully fine-tuned approaches.
Qualitative Results. We compare ZEBRA with the zero-shot implementation NeuroPictor* and the few-shot baseline MindEEye2 (1-hour) as shown in Fig. Qualitatively, ZEBRA generates high- fidelity images that are visually comparable to those produced by fully supervised models, and clearly surpasses NeuroPictor' in overall perceptual quality. Compared to few-shot methods, the main

---

GT
ZEBRA
NeuroPictor' (1th)
GT
ZEBRA
NeuroPictor"
Mindeye2 (1h)
Figure 5: Qualitative comparison between ZEBRA and zero-shot implementation of NeuroPictor and Mindeye2 (1h).
limitation of ZEBRA lies in semantic accuracy. As highlighted in the failure cases Fig. 7 ZEBRA tends to struggle with fine-grained semantic distinctions, especially for rare object categories.
4.3 Ablation Studies
We conduct ablation studies Subject 1 (trained on Subjects 2-8) to assess the contribution of each component in ZE- BRA, including adversarial training and the proposed preservation anchor both the Innarcct-n Feature Extractor (SIFE) and the Speciic- Feature Enhancer (SSFE).
Brain embedding
UMAP
LSEE
Baseline
Subject-invariant brain embedding
Effectiveness of the key components. As shown in Table _ the base model with- out any regularization performs poorly (e.g., PixCorr 0,089, Alex(5) = 72.7%). Adding adversarial training to SIFE yields clear improvements across both low- and high-level metrics (e.g., +0.040 PixCorr, +3.6% CLIP). Adding the anchor further boosts performance, especially high- level features like CLIP. Incorporating ad- versarial loss in SSFE brings additional gains, especially in high-level metrics, confirming the benefit of regularizing both branches. The full model with all com- ponents achieves the best overall results, highlighting the effectiveness of our de- sign.
UMAP
I-NNF
SUB
Baseline SIFE
Subject-specific brain embedding E,
UMAP
I-SNF
sabjeet
Figure 6: Visualization of subject-invaree invariant and -specific features with UMAP and t-SNE.
Effectiveness of the proposed subject-invariant representation learning. In Fig. we use UMAP and t-SNE visualize the feature distributions of different subjects. The left two plots show the subject-invariant features E,, while the right two show the subject-specific features E.. In the case of E,, data points from all subjects are highly mixed without forming subject-specific clusters, demonstrating that the learned features invariant subject identity. In contrast, E, exhibits clear

---

Table 3: Ablation on the number of training subjects for ZEBRA, evaluated on data from subject 1.
# of
Low-Level
High-Level
Subjects
PixCorr
SSIM1
Alex(2)1
Alex(5)t
Incept
CLIPt
Effl
SwAV
5)
0.109
0.325
68.2%
73.5%
66.1%
63.7%
0.871
0.563
5 (2 to 6)
0.126
0.347
71.3%
76.8%
68.7%
67.2%
0.846
0.534
(2 to 7)
0,135
0.363
74.1%
79.0%
70.8%
69.6%
0.823
0,308
7 (2 to 8)
0.153
0.384
76.1% %
81.8%
73.4% %
72.3%
0.814
0.490
clustering by subject, indicating that the model successfully captures individual-specific information. These results confirm that our method effectively disentangles subject-invariant and subject-specifit representations.
Ablation Number of Training Subjects. As shown in Table we evaluate the impact of training subject count on model performance. As the number of subjects increases from 4 to 7, we observe consistent performance improvements across all metrics. For instance, PixCorr improves significantly from O. 109 to 0.153, and both low- and high-level semantic scores show similar trends, such as CLIP score increasing from 63.7% to 72.3%. These results demonstrate that incorporating more diverse subject data helps the model generalize better, highlighting its scalability and robustness.
5 Discussion and Conclusion
Key Contributions and Insights. The novelty of our work lies not in individual archi- tectural components, but in the problem formulation, representation disentanglement design, and cross-subject transfer mechanism that together enable zero-shot IMRI-to-image decoding. First, the importance of the zero-shot setting: traditional fMRI decoding frameworks typically rely on fully supervised, subject-specific training, requiring separate model tuning for each individual under expert supervision. Such procedures are time-consuming-Ou subject-and computationally prohibitive for clinical use. In contrast, our approach introduces, for the first time, a zero-shot setting that allows the model to be directly applied to unseen sub- jects without retraining. This paradigm shift makes neural decoding fast, scalable, and clinically practical, achieving 73.4% decoding performance with approximately one second of inference per image. Second, neuroscience- innpiirrddddddddddeeeeee representation disentanglement: neuroscientific evidence that, despite intrrindividual variability, the human cortex encodes semantic information in consistent and topographically organized manner across subjects. To preserve this universality while maintaining discriminative power, we explicitly separate subject-invarian and semantic-specifie representations through the SIFE and SSFE modules, balancing fidelity and generalizability. Third, adversarial disentanglement with preservation anchors: adversarial training objectives are employed automatically extract invariant and specific features, while a Representa- tion Preservation Anchor ensures that essential individual information is retained during zero-shot transfer. Finally, empirical superiority in zero-shot decoding: our framework outperforms state-of- the-art baselines adapted to the zero-shot setting (c.g., MindTuner), underscoring the effectiveness of our disentanglement and transfer design in achieving robust cross-subjess generalization.
exceeding 12 hours per
design is motivated by
Limitations and Future Work. Despite the encouraging results, several limitations remain. Although our reconstructed images exhibit competitive quality, especially in low-level perceptual metrics, their semantic fidelity still lags behind few-shot approaches. Improving high-level semantic accuracy remains key challenge. Nonetheless, our work offers promising zero-shot strategy and lays the foundation for building generalizable brain decoding models. Another limitation lies in the scope of downstream tasks. While this study focuses on image reconstruction. the proposed ZEBRA framework is inherently modality-agnosti and could be extended to more complex domains such as text or video. For instance, integrating ZEBRA with existing methods like NeuroClips (50) or NEURONS [1 could enable zero-shot MMRI-tovvdee generation, facilitating richer understanding of human perceptual experiences. Moreover, the current dataset contains a limited number of subjects, which restricts the ability to fully demonstrate the generalizability of our approach. We believe that as more training subjects and fMRI data become available, the model's robustness and zero-shot performance will further improve. Expanding subject diversity is especially important for advancing toward universal brain decoders. Finally, additional fMRI recordings and broader subject coverage are essential to capture real-world visual experiences more comprehensively. Addressing these limitations

---

ZEBRA
GT
ZEBRA
ZEBRA
Wrong semantics
Wrong actions
Meaningless Reconstruction
Figure Failure cases, mainly caused by wrong semantics.
will require interdisciplinary progress across machine learning, computer vision, neuroscience, and biomedical engineering. We emphasize that alongside technical advances, it is equally important to establish ethical and regulatory frameworks to ensure the privacy and responsible use of brain data.
Conclusion. In this work, we introduced ZEBRA, novel zero-shot brain visual decoding framework that addresses the critical challenge generalizing fMRI-to-image reconstruction to unseen subjects. By disentangling subject-specific and semantic-specife components in the fMRI embedding space, ZEBRA enables accurate visual reconstruction without requiring additional data or retraining for new individuals. Our approach leverages adversarial learning and residual decomposition to isolate shared semantic representations, achieving strong generalization across subjects. Extensive experiments demonstrate that ZEBRA outperforms existing zero-shot baselines and approaches the performance of fully finetuned models, both quantitatively and qualitatively. This represents a significant step toward practical and scalable brain decoding systems with real-world applicability in neuroscience, clinical settings, and crain-computer interfaces.
Acknowledgements
This work was partially supported by grant from the Joint Research Scheme (JRS) under the National Natural Science Foundation of China (NSFC) and the Research Grants Council (RGC) of Hong Kong (Project No. N HKUST654/24), a grant from the RGC of the Hong Kong Special Administrative Region, China (Project No. R6005-24), and grant from the RGC of the Hong Kong Special Administrative Region, China (Project No. AoE/E-601/24-NN.
References
[1] Y. Takagi and Nishimoto, "High-resolution image reconstruction with latent diffusion models from human brain activity," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattenn Recognition, pp. 14453-14463, 2023.
[2] F. Ozcelik and R. VanRullen, "Natural scene reconstruction from fmri signals using generative latent diffusion," " Scientific Reports, vol. 13, no. p. 155666, 2023.
[3] Scotti, A. Banerjee. J. Goode, Shabalin, A. Nguyen, Dempster. N. Verlinde, E. Yundler, Weisberg. K. Norman, et al., "Reconstructing the mind's eye: fmri-to-image with contrastive learning and diffusion
priors." Advances in Neural Information Pr Processing Systems, vol. 36, 2024.
[4] P. S. Scotti, M. Tripathy, C. K. T. Villanueva, R. Kneeland, T. Chen, A. Narang, C. SSnthirasegaran, J. Xu, T. Nasclaris, K. A. Norman, and M. Abraham, "Mindeye2: shared-subject models enable finri-to-image with hour of data," in Proceedings of the 41 International Conference on Machine Learning, ICML'24 24, JMLR.org. 2024.
[5] J. Huo, Y. Wang, Y. Wang. X. Qian, C. Li, Y. Fu, and J. Feng, "Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and multi-level modulation," in European Conference on Computer Vision, pp. 56-73, Springer, 2024.
[6] Z. Gong, Q Zhang, G. Bao, L. Zhu, R. Xu, K. Liu, L. Hu, and D. Miao, "Mindtuner: Cross-subject visual decoding with visual fingerprint and semantic correction." in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 14247-14255. 2025.
10

---

[7] J. Lu, H. Wang. Q. Zhang, and X. Li. "A cognitive process-inspired architecture for subject-agnostic brain visual decoding." " in ICLR, 2025.
[8] A. Thomton and D. I. Tamir, "Neural representations of situations and mental states are composed of sums of representations of the actions they afford," Nature commanications, vol. 15, no. 1, p. 620, 2024, [9] R. Sitaram, A. Caria, R. Veit, T. Gaber, G. Rota, A. Kuebler, and N. Birbaumer, "Fmri brain- comptcr interface: A tool for neuroscientific research and treatment," Computational intelligence and neuroscience, vol. 2007, no. 1, p. 025487, 2007.
[10] B. Sorger, J. Reithler, B. Dahmen, and R. Goebel, "A real-time fmri-based spelling device immediately enabling robust motor-independent communication," Current Biology, vol. 22, no. 14, pp. 1333-1338,
2012.
[11] T. Wang. D. Mantini, and C. R. Gillebert, "The potential of real-time finri neurofeedback for stroke rehabilitation: a systematic review." cortex, vol. 107. pp. 148-165. 2018.
[12] D. D. Leeds, D. A. Seibert, J. A. Pyles, and M. J. Tarr, "Comparing visual representations across human fmri and computational vision," Journal of vision, vol. 13, no. 13, pp. 25-25, 2013.
[13] J. M. Lahnakoski, E. Glerean, I. P. Jääskeläinen, J. Hyönä, R. Hari, M. Sams, and L. Nummenmaa,
"Synchromous brain activity across individuals underlies shared psychological perspectíves." Neurolmage,
vol. 100, pp. 316-324, 2014.
[14] A. G. Huth, W. A. De Heer, T. L. Griffiths, F. E. Theunissen, and J. L. Gallant, "Natural speech reveals the semantic maps that tile human cerebral cortex," ," Nature, vol. 532, no. 7600, pp. 453-458, 458, 2016.
[15] B. De Haas, A. L. lakovidis, D. S. Schwarzkopf. and K. R. Gegenfurtner, "Individual differences in visual salience vary along semantic dimensions," Proceedings of the National Academy of Sciences, vol. 116, no. 24, pp. 166871 1692, 2019.
[16] C. Moutsiana, B. De Haas, A. Papageorgiou, J. A. Van Dijk, A. Balraj, J. A. Greenwood, and D. S. Schwarzkopf, "Cortical idiosyncrasies predict the perception of object size," Nature communications, vol. no. p. 12110, 2016.
[17] E. J. Allen, G. Ytes, Y. Wu, J. L. Breedlove, J. S. Prince, L. T. Dowdle, M. Nau. B. Caron. F. Pestilli, I. Charest, et al., "A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence." " Nature neuroscience, vol. 25, no. I, pp. 116-126, 2022.
[18] N. Chang. J. A. Pyles, A. Marcus, A. Gupta, M. J. Tarr, and E. M. Aminoff, "BOLD5000, public fMRI dataset while viewing 5000 visual images," Scientific Data, vol. p. 49, May 2019.
[19] T. Horikawa and Y. Kamitani, "Generic decoding
seen and imagined objects using hierarchical visual features," Nature Communications, vol. 8, no. 1, p. 15037, 2017.
[20] Z. Ren, J. Li, X. Xue, X. Li, F. Yang, Z. Jiao, and X. Gao, "Reconstructing seen image from brain activity by disually-guide cognitive representation and adversarial learning." Neurolmage, vol. 228, p. 117602, Mar. 2021.
[21] M. Mozafari, L. Reddy, and R. VanRullen, "Reconstructing Natural Scenes from fMRI Patterns using BigBiGAN," in 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1 1-8, July 2020. [22] 7. Gu, K. W. Jamison, M. Khosla, F. J. Allen, Y. Wu, G. St-Yves, T. Naselaris, K. Kay, M. R. Sabuncu, and A. Kuceyeski, "NeuroGen: Activation optimized image synthesis for discovery neuroscience," Neurolmage, vol. 247. p. 118812, Feb. 2022.
[23] Ghenn, T. Horikawa, K. Majima, and Y. Kamitani, "Deep image reconstruction from human brain activity," PLOS Computational Biology, vol. 15, e1006633, Jan. 2019.
[24] T. Fang. Q. Zheng, and G. Pan, "Alleviating the semantic gap for generalized fmri-to-image
---magee reconstruction." Advances in Neural Information Processing Systems, vol. 36, pp. 15096 15107, 2023.
[25] B. Zeng. S. Li, X. Liu, S. Gao, X. Jiang, X. Tang. Y. Hu, Liu, and B. Zhang, "Controllable Mind Visual Diffusion Model," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 6935-6943, 6643, Mar. 2024.
[26] P. Scotti, A. Banerjee, J. Goode, Shabalin, A. Nguyen, A. Dempster, N. Verlinde, F. Yundler, D. Weisherg, K. Norman, et al., "Reconstructing the mind's eye: fmri-to-image with contrastive learning and diffusion priors," Advances in Neural Information Processing Systems, vol. 36, pp. 24705-24728. 2023.
[27] Z. Chen, J. Qing. T. Xiang. W. L. Yue. and J. H. Zhou, "Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding," in 2023 IEEE/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pp. 277202, TEEE, 2025.
[28] M. Ferrante, F. Ozcelik, T. Boccato, R. VanRullen, and N. Toschi, "Brain captioning: Decoding human brain activity into images and text," arXiv preprint arXiv:2305./ /1560, 2023.
[29] B. R. Conroy, B. D. Singer, J. S. Guntupalli, P. J. Ramadge, and J. V. Haxby, "Inter-subject alignment of human cortical anatomy using functional connectivity." Neurolmage, vol. 81, pp. 400-411, 2013.
[30] B. Conroy, B. Singer, J. Haxby, and P. Ramadge, "fmrribbaeed inter-subject cortical alignment using functional connectivity." Advances in neural information processing systems, vol. 22, 2009.
[31] S. Wang, S. Liu, Z. Tan, and X. Wang, "MindBridge: A Cross-Subjed Brain Decoding Framework," in Computer Vision and Pattern Recognition (CVPR). (Seattle, WA, USA), pp. 1 11333-1 1342, IEEE, June 2024.
2024 IEEE/CVF Conference
[32] M. Ferrante, T. Boccato, F. Ozcelik, R. VanRullen, and N. Toschi, "Through their eyes: Multi-suhject brain decoding with simple alignment techniques." Imaging Neuroscience, vol. 2, pp. 1-21, May 2024.
11

---

[33] A. Thual. Y. Benchetrit, F. Geilert, J. Rapin, 1. Makarov. H. Banville, and J.-R. King, "Aligning brain functions boosts the decoding of visual semantics in novel subjects." Dec. 2023.
[34] Xia, R. de Charette, C. Öztireli, and J.-H. Xue, "UMBRAE: Unified Multimodal Brain Decoding," July
2024.
[35] X. Qian, Y. Wang, J. Huo, J. Feng, and Y. Fu, "fimri-ptc pie: A large-scale fmri pretrained transformer encoder
for multi-subject brain activity decoding." arXiv preprint wxXiv:231.000344 2023.
[36] C. Li, X. Qian, Y. Wang. J. Huo, X. Xue, Y. Fu, and J. Feng, "Enhancing cross-subject fmri-to-video
decoding with global-local functional alignment," in European Conference on Computer Vision, pp. 353- 369, Springer, 2024.
[37] K. L. Miller, F. Alfaro-Alnaant Aleagro. N. K. Bangerter, D. L. Thomas, E. Yacoub, Xu, A. J. Bartsch, S. Jbabdi, S. N. Sotiropoulos. J. L. Andersson, et al. "Multimodal population brain imaging in the uk biobank prospective epidemiological study," Nature neuroscience, vol. 19, no. 11. pp. 1523-1536, 2016.
[38] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, "Hicrarchical text-conditional image generation with clip latents," arXiv preprint arXiv:2204.0612
vol. 1, no. 2, p. 3, 2022.
[39] Y. Li, X. Tian, M. Gong. Y. Liu, T. Liu, K. Zhang, and D. Tao, "Deep domain generalization via conditional invariant adversarial networks," in Proceedings of the European conference on computer vision (ECCV),
pp. 624-639, 2018.
[40] D. Hendrycks and K. Gimpel, "Gaussian error linear units (gelus))" arXiv preprint arXiv:1606008115.
2016.
[41] Y. Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation." in International conference on machine learning, pp. 1180 1189, PMLR, 2015.
[42] T.-Y. Lin, M. Maire, S. Belongie, Hays, Perona, D. Ramanan, Dollár, and C. L., Zitnick, "Microsoft coco: Common objects in context," in Computer Vision-FECCV 2014: 13th European Conference, Zurich,
Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755, Springer, 2014.
[43] Z. Wang. A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity." " IEEE Iransactions on image processing. vol. 13, no. 4, pp. 600-612, 2004.
[44] M. Tan and Q. Le. "Efficientuet: Rethinking model scaling for convolutional neural networks," in Interna- rional conference on machine learning. pp. 6105-6114, PMLR, 2019.
[45] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, "Unsupervised learning of visual features by contrasting cluster assignments," Advances in neural information processing systems, vol. 33, pp. 9912-9924, 2020.
[46] C. Szegedy, V. Vanhoucke, S. loffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision." in Proceedings of the IEEE conference computer vision and pattern recognition,
pp. 2818 2826, 2016.
[47] A. Radford, W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin.
J. Clark, et al., "Learning transferable visual models from natural language supervision." in International conference on machine learning, pp. 8748-8763, PMLR, 2021.
[48] I. Loshchilow and F. Hutter, "Decoupled weight decay regularization," arXiv preprint arXiv: 171101101,
2017.
[49] L. Smith and Topin, "Super-convergence: Very fast training neural networks using large learning rates." in Artificial intelligence and machine learning for mulli-domain operations applications, vol. 11006, pp. 369-386, SPIE. 2019.
[50] Z. Gong, G. Bao, Q. Zhang, Z. Wan, D. Miao, S. Wang, L. Zhu, C. Wang. R. Xu, L. Hu, et al., "Neuro- clips: Towards high-fidelity and smooth fmri-to-video reconstruction." " Advances in Neural Information Processing Systems, vol. 37, pp. 51655-511 51683, 2025.
[51] H. Wang, Q. Zhang, L. Wang, X. Huang, and X. Li, "Neurons: Emulating the human visual cortex improves fidelity and interpretability in fmri-to-video reconstruction," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 18367-18376, October 2025.
12

---

Appendix
More Details about the Baseline of ZEBRA
The Baseline employs the ViT-based fMRI encoder from fMRI-PTE M. which is pretrained on the UK Biobank dataset 2 This encoder transforms the fMRI scan into shared latent space, where diffusion prior network then converts the latent brain embeddings into vision features for image generation using Stable Diffusion.
The Baseline consists of three main components:
1. fMRI Encoder:
This module processes the input fMRI data and transforms it into a unified 2D brain activation map (. resulting in single-channel image a E 25 256 The fMRI encoder then maps this image into latent representation E BBLXx,, where B is the batch size, L is the number of tokens, and C₁ is the brain feature dimension.
2. Latent Representation Conversion: The latent brain embedding E is subsequently trans- formed into a coop-comaaiible embedding F IR BxLCC2 provide guidance reconstruction process.
3. Diffusion Prior Network: As in MindEye2 we employ diffusion prior [ map the (MRI-CLIP embedding F to reconstructed OpenCLIP image embedding F, corresponding the visual stimulus.
The training of the Baseline involves three key losses:
1. Contrastive Loss on CLIP Text Embeddings: between the predicted CLIP text embedding F and the ground truth Fú
This loss, denoted CCLIP, is calculated
2. Contrastive Loss on CLIP Image Embeddings: The loss LCLIP, is computed between the predicted CLIP image embedding F and the ground truth Fý
3. Diffusion Prior Loss: The loss Spiior used to train the diffusion prior network minimize the reconstruction eггоr.
Both LCLIP, and LCLIP, implemented the BiMixCo loss, which aligns fMRI signals and corresponding image embeddings using a bidirectional contrastive loss and MixCo data augmentation, as detailed below.
The MixCo procedure involves mixing two independent fMRI signals. For each fMRI signal æ, randomly sample another fMRI signal Z. corresponding to a different index m. The two signals are then mixed using a linear combination:
mix(i, 3m) = x
A)
where x" represents the mixed fMRI signal and À is hyperparameter sampled from a Beta distribu- tion. The ridge regression module then maps x² to lower-dimensional space, yielding x" from which the embedding F is obtained using the MLP, i.e., F Exx'))
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

The BiMixCo loss function is formulated as:
exp(sim( F;, yx)/T)
LiMMixCo
log
21.
Σk=1 exp (sim( F.: 91a/T)
exp(simm F.. ym: ymm)/T)
A,) log
2L
Σ k=1 exp(sim F. u/)/T)
1=1
(2)
exp(simí Fj, yz 4/)/T)
>j log
Σ 1,
k= exp(sim( F; Ms M/)/T)
Σ (1- -og exp(sim( Fi, 11)/TT)
2L 1=1 (1mm1=)]
Σ exp(sim( Fi, U;)/7)
where F represents the OpenCLIP embeddings for the image y.
The Diffusion Prior network is used to transform the fMRI embedding F into the reconstructed OpenCLIP image embeddings of stimulus F,. The objective is to minimize the mean squared error (MSE) between the predicted and target embeddings, formulated as:
Prtio FF F. F......F)) Fgll?.
2 More Details about Metrics
To evaluate reconstruction quality, we adopt a comprehensive set of low-level and high-level metrics.
On the low-level side, we include four metrics: pixel-wise correlation, Structural Similarity Index Measure (SSIM) [ AlexNet(2), and AlexNet(5). Pixel-wise correlation and SSIM are computed by averaging the similarity scores between each reconstructed image and its ground-truth counterpart. AlexNet(2) and AlcxNet(5) assess semantic similarity by measuring the two-way classification accu- racy based on features extracted from the 2nd and 5th layers of a pre-trained AlexNet, respectively.
For high-level evaluation, we extract features using several pre-trained models. EffNet-B and SwAV metrics are calculated by averaging the feature distance between reconstructions and ground-trute images using EfficientNet-BB [6] and SwAV-ResNet50 [7]. In contrast, the Inception [8] and CLIP [ metries reflect the accuracy of two-way classification using the corresponding high-level features.
3 Comparison with Baselines across Subjects
Table I presents a detailed comparison of ZEBRA with NeuroPictor", and our baseline across subjects 2, 5, and from the Natural Scenes Dataset. The results are reported under zero-shot setting, where NeuroPictor" refers to our reimplementation pretrained on the remaining seven subjects without any subject-specific finetuning. Across both low-level (PixCorr, SSIM) and high-level metrics (AlexNet, Inception, CLIP, etc.), ZEBRA consistently outperforms the zero-shot baselines, especially on semantic metrics like Alex(5) and CLIP accuracy. Notably, its performance remains stable across subjects, indicating strong generalization ability. The final row reports the averaged scores across all four subjects, further confirming the effectiveness of ZEBRA in both perceptual quality and semantic fidelity.
References
[1] X. Qian, Y. Wang. J. Huo. J. Feng, and Fu, "fmri-pte: A large-scale fmri pretrained transformer encoder for multi-subject brain activity decoding," arXiv preprint 2xX.::3311.00342 2023.
[2] K. L. Miller, Allarr-Almagro, N. K. Bangerter, L. Thomas, E. Yacoub, Xu, Bartsch, Jbabdi, N. Sotiropoulos, J. L. Andersson, et al., "Multimodal population brain imaging in the uk biobank prospective epidemiological study." Nature neuroscience, vol. 19, no. 11, pp. 1523-1536, 2016.
[3] P. S. Scotti, M. Tripathy, C. K. T. Villanueva, R. Kneeland, T. Chen, A. Narang, C. Santhirasegaran, J. Xu, T. Naselaris, K. A. Norman, and M. Abraham, "Mindeye2: shared-subject models enable fmri-to-image with I hour of data," in Proceedings of the 41 International Conference on Machine Learning, ICML'24, JoLR.org. 2024.
[4] A. Ramesh, P. Dhariwal, A. Nichol, C, Chu, and M. Chen, "Hierarchical text-conditional image generation with clip latents," arXiv preprint 2201:22004.00155, vol. no. 2, P. 2022.

---

Table 1: Results of NeuroPictor', our baseline, and ZEBRA for each subject, compared against representative methods under different training regimes. All results are averaged over subjects 1, and 7 from the Natural Scenes Dataset. "NeuroPictor"" denotes our implementation in zero-shot setting, pretrained on the other subjects.
Low-Level
Hige-Level
Method
PixCorrt
SSIM
Alex(2)f Alex(5)1 Incept
CLIP
EITy
SWAV
Subject
NeuroPictor* 0.069
0.305
73.1%
75.4%
63.6%
66.8%
0.910
0.583
Our baseline
0.089
0.325
72.5%
74.7%
64.7%
63.2%
0.891
0,579
ZEBRA 0.153
0.384
76.1%
81.8%
73.4%
72.3%
0.814
0.490
Subject
NeuroPictor* 0.061
0.303
73.0%
75.2%
62.4%
65.9%
0.938
0.590
Our baseline
0.079
0.323
72.4%
74.5%
63.5%
62.3%
0.918
0,586
ZEBRA 0.135
0.382
76.0%
81.6%
72.0%
71.3%
0.839
0.496
Subject 5
NeuroPictor* 0.049
0.290
69.7%
74.6%
62.7%
66.7%
0.922
0.599
Our baseline
0.063
0.309
69.1%
73.9%
63.8%
63.1%
0.903
0,595
ZEBRA 0.119
0.365
72.6%
80.9%
72.4%
72.1%
0.825
0.503
Subject
NeuroPictor* Our baseline
0.049
0.290
69.9%
73.7%
61.2%
64.7%
0.986
0.655
0.064
0.309
69.3%
73.0%
62.2%
61.2%
0.966
0.650
ZEBRA 0.117
0.370
73.8%
80.4%
71.0%
70.4%
0.869
0.534
Average
NeuroPictor' 0.057
0.297
71.4%
74.7%
62.5%
66.0%
0.939
0,607
Our baseline
0.074
0.316
70.8%
74.0%
63.5%
62.5%
0.920
0,602
ZEBRA
0.131
0.375
74.6%
81.2%
72.2%
71.5%
0.837
0.506
[5] Wang, A. C. Bovik, H. R. Sheikh, and Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-6 612, 2004.
[6] M. Tan and Q. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks." in Interna- tional conference machine learning, pp. 6105-6114, PMLR, 2019.
[7] M. Caron, Misra, J. Mairal, Goyal, P. Bojanowski, and A. Joulin, "Unsupervised learning visual features by contrasting cluster assignments." Advances neural information processing systems, vol. 33,
pp. 9912-9924, 2020.
[8] C. Szegedy. Vanhoucke, S. loffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision." " in Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 2818-222, 2016.
[9] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., "Learning transferable visual models from natural language supervision," in International conference machine learning, pp. 8748-8763, PMLR, 2021.
3

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Main contributions and scope have been accurately claimed in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are detailedly discussed.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
The proposed method does not involve theory assumptions and proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have disclosed all the information, including details of modules, parameters, implementation details. Our code in attached in supplementary materials.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer:
[Yes]
Our code in attached in supplementary materials with detailed instructions and documentations.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.com cc/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.c/ppul calsscccpublll hctppbblicgggudess htapc//gguudeee/eeeeuuuunn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have specified all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification:
To maintain a similar setting for comparison with previous studies.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Experiments compute resource information provided in Implementation
section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The paper adheres to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Included the Limitations and Future Work section. Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There are no released models and scraped datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly mentioned the used models and cited them without violating their license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww datastts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code is with the MIT license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy http:s///earrioooccc//ooeerrrrr for what should or should be described.
19


Reviews:
[{"review_id": "1vVo0X3KLM", "reviewer": "Reviewer_bWGu", "summary": "This work introduces ZEBRA, a novel framework designed to overcome the limitations of subject-specific models in fMRI-to-image reconstruction. The core innovation is a zero-shot approach that utilizes adversarial training to disentangle fMRI signals into subject-specific and subject-invariant semantic components. Experiments demonstrate that the framework outperforms existing zero-shot baselines and achieves results comparable to models that require subject-specific training.", "questions": "See the weaknesses.", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- This paper proposes a zero-shot brain visual decoding framework that generalizes to unseen subjects without requiring additional fMRI data or finetuning.\n- The paper introduces a disentanglement strategy to learn subject-invariant and semantic-specific representations from fMRI signals.\n\nWeaknesses:\n- If the 'zero-shot' approach, by design, captures only the commonalities across subjects while discarding individual differences, what then is the significance of image-to-fMRI encoding research? In what specific areas would this research be meaningful?\n- The citations in lines 60-63 seem to pertain to the effects of speech on the brain. Given that speech and vision are known to elicit different activation patterns in the human brain, the authors should provide more direct evidence and citations relevant to the vision domain to support their claims.\n- There are inconsistencies between the terms and abbreviations used in Figure 3 and their descriptions in the main text. This makes the figure difficult to understand. Please ensure all labels are consistent.\n- In Section 3.2, the authors should provide a clearer justification for their choice of self-attention. Specifically, what is the rationale for using self-attention, and why is it suitable for extracting subject-invariant information? Please elaborate on the underlying mechanism or provide supporting literature.\n- The literature review would be strengthened by citing more literature that supports the existence of subject-dependent features.\n- To better assess the model's performance and capabilities, it would be beneficial to include more generated examples, perhaps in the appendix.", "was_revised": false, "final_justification": "", "rebuttal": "### ***Overall Reply***\n\nFirst, we sincerely appreciate your recognition of our work, including comments such as *\"a novel framework\"* and *\"the framework outperforms existing zero-shot baselines and achieves results comparable to models that require subject-specific training\"*.\n\nYour remarks are in strong alignment with other reviewers’ positive feedback, including: *\"the first attempt at zero-shot subject decoding\", \"addresses a critical challenge in neural decoding\", \"strong zero-shot performance\", \"thorough experiments\", and \"clear and well-organized presentation\"*.\n\nWe are particularly grateful for your constructive feedback, which affirms the novelty and practical value of our approach. These encouraging remarks greatly motivate us to further refine our work. While there may be certain details requiring clarification, we aim to address these points in the following responses to ensure our methodology and results are well understood.\n\n### ***Point-by-point Responses***\n\n***W1: “If the 'zero-shot' approach, by design, captures only the commonalities across subjects while discarding individual differences, what then is the significance of image-to-fMRI encoding research? In what specific areas would this research be meaningful?”***\n\n- **Role of image-to-fMRI encoding.** Image-to-fMRI encoding research aims to reveal how the brain processes visual information by modeling the relationship between structured visual stimuli and localized neural responses. Since the input images are semantically clean and low in noise, encoding models can leverage well-defined visual features to map onto neural activity, supporting mechanistic interpretation of brain function.\n- **Role of zero-shot fMRI-to-image decoding.** In contrast, zero-shot fMRI-to-image decoding addresses the inverse and more challenging task: reconstructing perceptual experience from noisy, high-dimensional brain signals, without subject-specific supervision. This requires aligning variable and subject-specific neural representations to fixed visual embeddings, making the task harder but more relevant for real-world applications such as brain-computer interfaces, where generalization to new users is essential.\n- **Limitations of encoding for cross-subject generalization.** While encoding models may use shared representations from the image domain, they still rely on subject-specific mappings (e.g., individual linear regressors) to predict fMRI responses. These models do not learn subject-invariant brain representations and therefore have limited utility for zero-shot or cross-subject decoding. As such, image-to-fMRI encoding and fMRI-to-image decoding serve complementary goals.\n\n***W2: “The citations in lines 60-63 seem to pertain to the effects of speech on the brain. Given that speech and vision are known to elicit different activation patterns in the human brain, the authors should provide more direct evidence and citations relevant to the vision domain to support their claims.“***\n\nThank you for the suggestion. We agree that visual-specific evidence is more appropriate here. To support our claim, we now cite studies showing consistent and organized visual representations across subjects:\n\n[1] Lahnakoski, Juha M., et al. \"Synchronous brain activity across individuals underlies shared psychological perspectives.\" *NeuroImage* 100 (2014): 316-324.\n\n[2] Leeds, Daniel D., et al. \"Comparing visual representations across human fMRI and computational vision.\" *Journal of vision* 13.13 (2013): 25-25.\n\n[3] Lipman, Ofer, et al. \"Invariant inter-subject relational structures in the human visual cortex.\" *arXiv preprint arXiv:2407.08714* (2024).\n\n***W3: “There are inconsistencies between the terms and abbreviations used in Figure 3 and their descriptions in the main text. This makes the figure difficult to understand. Please ensure all labels are consistent.”***\n\nThank you for pointing this out. We apologize for the inconsistencies between Figure 3 and its description. We have carefully reviewed and updated all labels and terminology to ensure consistency and clarity throughout the figure and the main text.\n\n***W4: “In Section 3.2, the authors should provide a clearer justification for their choice of self-attention. Specifically, what is the rationale for using self-attention, and why is it suitable for extracting subject-invariant information? Please elaborate on the underlying mechanism or provide supporting literature.”***\n\nThank you for the question. In our framework, the extraction of subject-invariant information is mainly driven by the GRL and loss functions, and is not strictly dependent on the feature extractor architecture. We adopt self-attention to maintain architectural consistency with the brain encoder, which is based on Vision Transformer (ViT). Self-attention also facilitates global context modeling, which benefits fMRI representation learning. We will clarify this point in Section 3.2.\n\n***W5: “The literature review would be strengthened by citing more literature that supports the existence of subject-dependent features.”***\n\nThank you for the suggestion. We will add additional literature to support the existence of subject-dependent features in visual perception. Prior studies have demonstrated that individual differences in visual acuity, perceived position, and apparent size are not only stable across time but also systematically linked to cortical architecture. For example, variations in V1 surface area and pRF spread have been shown to predict subjective size perception [1,2], while cortical magnification patterns correlate with contrast sensitivity across the visual field [3]. Moreover, visual salience and gaze behavior have also been found to vary idiosyncratically across individuals along semantic dimensions [4].\n\n[1] Moutsiana, Christina, et al. \"Cortical idiosyncrasies predict the perception of object size.\" *Nature communications* 7.1 (2016): 12110.\n\n[2] Wang, Zixuan, Yuki Murai, and David Whitney. \"Idiosyncratic perception: a link between acuity, perceived position and apparent size.\" *Proceedings of the Royal Society B* 287.1930 (2020): 20200825.\n\n[3] Benson, Noah C., et al. \"Cortical magnification in human visual cortex parallels task performance around the visual field.\" *Elife* 10 (2021): e67685.\n\n[4] De Haas, Benjamin, et al. \"Individual differences in visual salience vary along semantic dimensions.\" *Proceedings of the National Academy of Sciences* 116.24 (2019): 11687-11692.\n\n***W6: “To better assess the model's performance and capabilities, it would be beneficial to include more generated examples, perhaps in the appendix.”***\n\nThank you for the helpful suggestion. We agree that additional examples can better illustrate the model's performance. We will include more qualitative visualizations and comparison results in the appendix to support our findings."}, {"review_id": "RfwBxCSmwL", "reviewer": "Reviewer_FDSv", "summary": "This paper presents ZEBRA, a zero-shot brain-to-image decoding framework that reconstructs visual experiences from fMRI without requiring subject-specific fine-tuning. ZEBRA introduces a novel approach that disentangles subject-related and semantic-related components in fMRI signals via adversarial training, enabling the model to generalize across unseen subjects. The proposed model achieves strong performance in a zero-shot setting and competative compared to some fully fine-tuned models on multiple benchmarks.", "questions": "- Why were subjects 1, 2, 5, and 7 chosen for evaluation? Is there a specific rationale for selecting these subjects over others in the dataset?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper addresses a critical challenge in fMRI-to-image neural decoding, which is the lack of cross-subject generalization in visual stimulus reconstruction.\n- The authors provide thorough experimental validation, including detailed ablation studies, subject count sensitivity, and insightful visualizations (Figure 6) using UMAP and t-SNE to confirm the effectiveness of the disentanglement strategy.\n- ZEBRA consistently outperforms prior zero-shot baselines across all evaluated metrics, demonstrating strong generalization capability.\n- The paper is well-organized, making it easy to follow the methodology and experimental findings.\n\nWeakness:\n- While many prior baselines are subject-specific and rely on within-subject training or fine-tuning, some of their underlying architectures can, in principle, support multi-subject training and generalization to unseen subjects. Benchmarking such models in a true zero-shot setting would provide stronger evidence that ZEBRA's architecture is superior.\n- Although the focus of the paper is on zero-shot generalization, it would be more compelling to also fine-tune ZEBRA and compare it with other fine-tuned models. This would demonstrate whether the proposed architecture also offers advantages in settings where subject-specific data is available.\n- Despite competitive performance on low-level perceptual metrics, ZEBRA’s semantic fidelity still falls a lot behind few-shot baselines. It struggles with fine-grained semantic distinctions, particularly for rare object categories, as shown in the failure cases. Since high-level semantic accuracy is crucial for practical applications, as mentioned above, it would be valuable to explore whether fine-tuning could mitigate this limitation.\n- Statistical significance is not reported for the improvements. Including variance or significance would help validate the robustness of the results.", "was_revised": true, "final_justification": "Thank you for the thorough response and additional results, which have addressed most of my questions. I am therefore raising my score to 5.", "rebuttal": "### ***Overall Reply***\n\nWe sincerely appreciate your thoughtful feedback, including comments such as *\"ZEBRA introduces a novel approach\"*, *\"achieves strong performance in a zero-shot setting and is competitive with fully fine-tuned models on multiple benchmarks\"*, and *\"addresses a critical challenge in fMRI-to-image neural decoding\"*.\n\nWe are encouraged by your recognition of our *\"thorough experimental validation\"*, the *\"insightful visualizations (Figure 6) using UMAP and t-SNE to confirm the effectiveness of the disentanglement strategy\"*, and your praise for the work being *\"well-organized, making it easy to follow the methodology and experimental findings\"*.\n\nThese remarks align closely with other reviewers’ positive assessments, including: *\"first attempt at zero-shot subject decoding\", \"practical significance for real-world applications\", and \"novel framework outperforming zero-shot baselines\"*. We are grateful for your constructive feedback and will clarify certain technical aspects raised in your review to further strengthen the manuscript.\n\n### ***Point-by-point Responses***\n\n***W1: “While many prior baselines are subject-specific and rely on within-subject training or fine-tuning, some of their underlying architectures can, in principle, support multi-subject training and generalization to unseen subjects. Benchmarking such models in a true zero-shot setting would provide stronger evidence that ZEBRA's architecture is superior.“***\n\n- **Difficulty in benchmarking previous SOTAs in the zero‑shot setting.** The current SOTA methods, **MindEye2** and **MindTuner**, operate in a non‑fsLR brain space, which results in different brain voxel counts for each subject. Consequently, they require training separate subject‑specific linear mappings, making them inherently incompatible with true zero‑shot evaluation. In contrast, **NeuroPictor** adopts a unified fMRI space that enables subject‑agnostic modeling; however, its performance remains lower than that of the leading subject‑specific approaches.\n- **“Our baseline” in Table 1 is already the adapted SOTA.** To ensure a strong and fair comparison, we adapted the MindEye2 architecture to the NeuroPictor framework by replacing its subject‑specific linear layers with the unified‑space approach while preserving its core network design. This **adapted MindEye2/MindTuner serves as “our baseline”** in Section 3.1 and Table 1. We believe the reviewer may have overlooked this adaptation. As shown in our experiments, **ZEBRA** substantially outperforms this adapted baseline in the zero‑shot setting (**p < 0.001** across all metrics; see W4 for details), further validating the effectiveness of our architecture.\n\n***W2: “Although the focus of the paper is on zero-shot generalization, it would be more compelling to also fine-tune ZEBRA and compare it with other fine-tuned models. This would demonstrate whether the proposed architecture also offers advantages in settings where subject-specific data is available.“***\n\n- **Different goals of fine-tuned methods vs. zero-shot decoding.** Fine-tuned approaches aim to best fit the test subject using subject-specific data, often at the expense of generalizability. In contrast, zero-shot brain decoding explicitly seeks a trade-off between subject fidelity and cross-subject generalization—this balance is central to our design of ZEBRA. Therefore, comparing zero-shot models directly with fully fine-tuned ones does not reflect the same objective. Moreover, as mentioned in our response to W1, we have already compared ZEBRA with strong zero-shot and partially supervised baselines, which is more aligned with our goal.\n- **Additional comparison under the fully fine-tuned setting.** Nonetheless, to address the reviewer’s request for broader evaluation, we also benchmarked ZEBRA in a fully fine-tuned setting. As shown in the table below, ZEBRA remains highly competitive and achieves performance comparable to the best existing fine-tuned models, despite not being tailored for this regime.\n- **Practical limitations of fully supervised decoding.** While fine-tuned models may perform well in a closed setting, they typically overfit to individual subjects and lack generalizability—limiting their real-world utility. In contrast, ZEBRA is explicitly designed to generalize across subjects without retraining, making it more practical for scalable deployment. Our results demonstrate that ZEBRA maintains strong performance even when fine-tuned, further validating its robust architecture.\n\n| Method | PixCorr | SSIM | Alex(2) | Alex(5) | Incep | CLIP | Eff | SwAV |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| NeuroPictor | 0.229 | 0.375 | 96.5% | 98.4% | 94.5% | 93.3% | 0.639 | 0.350 |\n| MindTuner | 0.322 | 0.421 | 95.8% | 98.8% | 95.6% | 93.8% | 0.612 | 0.340 |\n| Ours | 0.245 | 0.409 | 96.6% | 98.2% | 95.2% | 92.9% | 0.594 | 0.341 |\n\n***W3: “Despite competitive performance on low-level perceptual metrics, ZEBRA’s semantic fidelity still falls a lot behind few-shot baselines. It struggles with fine-grained semantic distinctions, particularly for rare object categories, as shown in the failure cases. Since high-level semantic accuracy is crucial for practical applications, as mentioned above, it would be valuable to explore whether fine-tuning could mitigate this limitation.“***\n\n- **Semantic fidelity vs. generalization: a known trade-off.** We agree that achieving high semantic fidelity, especially for rare categories, remains a challenging problem. As the reviewer correctly points out, zero-shot models—by design—prioritize generalization over subject-specific optimization. This inevitably leads to some degradation in fine-grained semantic distinctions, particularly in the absence of subject-specific supervision. However, this is not a flaw of ZEBRA per se, but a reflection of the broader trade-off in zero-shot decoding: balancing generality with specificity.\n- **Fine-tuning as a means to improve semantic accuracy.** We appreciate the suggestion to explore whether fine-tuning can mitigate semantic limitations. In fact, as part of our extended experiments (see response to W2), we fine-tuned ZEBRA using subject-specific data. We observed notable improvements in high-level semantic accuracy across both frequent and rare categories, supporting the reviewer’s hypothesis. This result highlights the flexibility of ZEBRA: it performs well in zero-shot settings and can further benefit from fine-tuning when such data is available.\n- **Toward hybrid models and adaptive decoding.** Going forward, we believe an important direction is to **integrate zero-shot generalization with lightweight subject-specific adaptation**, potentially through few-shot or meta-learning strategies. Our current work lays the foundation for such hybrid approaches by demonstrating that ZEBRA captures transferable representations that remain effective even after fine-tuning.\n\n***W4: “Statistical significance is not reported for the improvements. Including variance or significance would help validate the robustness of the results.“***\n\nThank you for the suggestion. We have conducted paired t-tests to assess statistical significance. The improvements of ZEBRA over the baseline (previous SOTA on the zero-shot setting) are statistically significant across all metrics (all **p < 0.001**), indicating that the observed gains are robust and unlikely due to chance.\n\n|  | PixCorr | SSIM | Alex(2) | Alex(5) | Incep | CLIP | Eff | SwAV |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| p-value | 1.53 × 10⁻⁴ | 1.19 × 10⁻⁵ | 5.11 × 10⁻⁴ | 3.92 × 10⁻⁶ | 9.16 × 10⁻⁷ | 3.24 × 10⁻⁷ | 4.17 × 10⁻⁴ | 6.42 × 10⁻⁴ |\n\n***Q1: “Why were subjects 1, 2, 5, and 7 chosen for evaluation? Is there a specific rationale for selecting these subjects over others in the dataset?”***\n\nFollowing prior NSD-based reconstruction studies (e.g., Takagi & Nishimoto, 2022; Ozcelik & VanRullen, 2023; MindEye & MindEye2 etc.), we selected subjects 1, 2, 5, and 7 for evaluation because they are the only participants who completed all 40 scanning sessions, providing the most complete and consistent data for training and testing. This ensures fair comparison and aligns with the standardized evaluation protocol in the field."}, {"review_id": "ZnRRx5z04n", "reviewer": "Reviewer_orfj", "summary": "This paper addresses the challenge of zero-shot decoding of brain activity from fMRI data for unseen participants. By employing adversarial training, the method explicitly disentangles fMRI representations into subject-invariant, semantic-specific components, enabling generalization to novel subjects without requiring additional fMRI data or retraining. Extensive experiments demonstrate that ZEBRA outperforms zero-shot baselines significantly and achieves performance on par with fully fine-tuned models across multiple metrics. The framework broadens the applicability of brain decoding models to real-world scenarios in neuroscience, clinical domains, and brain-computer interface applications.", "questions": "As mentioned above, several concerns have been raised in the Weaknesses section.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThe introduction of the zero-shot brain decoding task addresses a critical challenge in neural decoding research and holds substantial practical value for real-world applications. The reported performance metrics demonstrate notable improvements over baselines in zero-shot scenarios.\nWeaknesses:\n1. Limited Novelty. The framework design predominantly relies on the integration of established components, including contrastive learning, adversarial training strategies, fMRI encoders, and diffusion priors. Notably, the proposed SIFE module employs a straightforward residual decomposition for implementation\n2.Inadequate Theoretical Justification. The paper provides insufficient theoretical or empirical validation for two critical assumptions: (1) The rationale for cleanly decomposing fMRI representations into subject-specific and semantic-specific components (Lines 7–9); (2) The validity and reliability of simple residual subtraction as a modeling approach for this decomposition (Section 3.2).\n3.Weak Empirical Validation.  In Section 4.3, the authors employ UMAP and t-SNE visualizations of feature distributions to justify the claim that the model disentangles subject-invariant and subject-specific representations. However, this validation approach is purely qualitative, lacks theoretical grounding, and fails to demonstrate a robust link between learned features and known functional brain areas.\n4. Limited Baseline Comparison. The paper presents ZEBRA’s performance exclusively in zero-shot scenarios, comparing it against a single baseline (NeuroPictor*). It omits extensive comparisons with state-of-the-art models and lacks evaluations in few-shot and fully fine-tuned settings—both critical for comprehensively assessing the model’s generalization capacity.\n5.Inadequate Evaluation Across Diverse Subjects and Datasets. Although the NSD dataset serves as a robust benchmark, supplementary results across diverse subject cohorts and additional fMRI datasets would strengthen confidence in the framework’s generalizability.\n6.The paper is difficult to follow due to frequent grammatical issues, unclear phrasing, and various formatting irregularities.", "was_revised": true, "final_justification": "Thank you for the response. The authors have provided a detailed reply to my concerns and questions, including thorough experiments and comparisons with additional baselines. Moreover, the design of the modules is closely aligned with the relevant brain areas. Although the models presented in the Rebuttal version do not perform well in other settings, including full-scale or across-dataset settings, it is important to note that this is the first paper on zero-shot visual decoding on NSD. Given this context, I raise my rating to 4.", "rebuttal": "### ***Overall Reply***\n\nWe sincerely thank you for recognizing the **broader impact and practical value** of our work. Your remarks—such as *\"broadens the applicability of brain decoding models...\"* and *\"addresses a critical challenge in neural decoding research...\"*—strongly align with other reviewers’ comments. We are encouraged that both the **scientific novelty** and **real-world utility** of our approach were well received. Below, we address each concern in detail.\n\n### ***Point-by-point Responses***\n\n***W1&W2: “Limited Novelty ... and Inadequate Theoretical …”***\n\nWe respectfully disagree with the assessment of limited novelty, and we believe this stems from a misunderstanding of our proposed method.\n\n- **Novelty of our work.** We would like to clarify that the novelty of our work lies not in individual modules but in the **problem setting**, **representation disentanglement design**, and **cross-subject transfer mechanism:**\n    1. **Importance of the Zero-Shot Setting: Clinicians cannot directly use fully supervised models.** We are the first to introduce the zero-shot setting for fMRI-to-image decoding, enabling models to be applied directly to new patients **without any retraining**. Most existing methods require fully supervised, subject-specific training that involves an **AI expert training a separate model for each patient**, making them impractical and inaccessible for clinical use.\n    2. **Importance of the Zero-Shot Setting: Fully supervised models require 12+ hours per patient; ours needs only ~1 second per image inference.** Fine-tuning conventional models for each patient takes over **12 hours** and significant computational resources, which is infeasible in many real-world applications such as consciousness assessment. Our zero-shot method achieves **73.4% decoding performance immediately**, with **~1 second inference per image**—making it fast, practical, and scalable for clinical deployment.\n    3. **Neuroscience-inspired disentanglement.** This design is motivated by neuroscientific evidence that, despite inter-individual variability in brain activity, the human cortex encodes semantic information in a consistent and topographically organized manner across subjects [11]. For a reconstruction framework to generalize effectively across individuals, it should preserve subject-invariant (universal brain representations) [1–3] and semantic-specific (class-discriminative) [4,5] components, while suppressing subject-specific and semantically irrelevant variations. Motivated by these findings, we explicitly disentangle *subject-invariant* and *semantic-specific* components to enhance generalizability. Unlike prior fully fine-tuned methods that often overfit to test subjects, our goal is to preserve shared representations while suppressing semantically irrelevant and subject-specific variations. To this end, we design SIFE and SSFE to jointly balance fidelity and generalizability in neural decoding.\n    4. **Adversarial disentanglement with preservation anchors.** We introduce adversarial training objectives to automatically extract subject-invariant and semantic-specific features, and a **Representation Preservation Anchor** to retain maximal individual information while still enabling zero-shot transfer.\n    5. **Empirical superiority in zero-shot decoding.** As a result, our method **outperforms fully fine-tuned SOTA methods when they are adapted to the zero-shot setting (refer to ”our baseline”, which is adapted from MindTuner)**, demonstrating that our design is crucial for achieving strong cross-subject generalization.\n- **Clarification on the baseline and SIFE design.** We adopt a standard baseline (contrastive learning, fMRI encoder, diffusion prior) as described in Section 3.1, without claiming these as technical contributions. Crucially, **SIFE is not a simple residual decomposition**. Together with SSFE, it embodies our key insight: effective zero-shot decoding requires explicitly disentangling **subject-invariant** and **semantic-specific** subspaces. This is achieved through adversarial regulation—suppressing subject identity in the invariant space while preserving it in the semantic one. GRL facilitates this process, but the novelty lies in our principled, neuroscience-inspired disentanglement—beyond what residual heuristics can offer.\n\n[1] Moutsiana, Cortical idiosyncrasies predict the perception of object size. Nature comm 2016.\n\n[2] Wang, Idiosyncratic perception: a link between acuity, perceived position and apparent size. Proceedings of the Royal Society 2020.\n\n[3] Benson, Cortical magnification in human visual cortex parallels task performance around the visual field. Elife 2021.\n\n[4] De Haas, Individual differences in visual salience vary along semantic dimensions. Proceedings of the National Academy of Sciences 2019.\n\n[5] Kandel, Principles of neural science, 2000.\n\n***W3: “Weak Empirical Validation...”***\n\nIn response to W1 & W2, we have clarified the theoretical foundation of our disentanglement approach by explicitly linking it to neuroscience evidence that semantic representations are consistent across individuals, while raw brain signals vary. This supports our design goal of separating subject-invariant from subject-specific components. We also conducted the following analyses to validate this separation.\n\n- **Validation of disentanglement.** While UMAP and t-SNE visualizations are widely accepted for qualitative assessment in domain adaptation (DA) and domain generalization (DG) research, we complement them with quantitative evidence. Specifically, we trained a classifier to identify subjects from the **subject-specific feature space** in the test set, achieving **89.3% accuracy**. This high accuracy confirms that the model indeed encodes strong subject-specific information in the intended subspace, while keeping the subject-invariant subspace free from such cues.\n- **Link to functional brain areas.** To establish a direct link between the **subject-invariant features** and known neurobiological substrates, following MindEye2, we computed Pearson correlations between these features and activation patterns from canonical visual regions. We observe the highest correlation in higher visual areas (r = 0.371) and the lowest in V1 (r = 0.321). This matches our design goal: higher visual areas carry abstract semantic information that our subject‑invariant features are intended to preserve, while early visual areas encode low‑level, subject‑specific patterns that our method deliberately suppresses to enhance zero‑shot generalization.\n\n| Brain region | V1 | V2 | V3 | V4 | Higher visual areas |\n| --- | --- | --- | --- | --- | --- | \n| Correlation | 0.321 | 0.341 | 0.349 | 0.325 | 0.371 |\n\n***W4: “Limited Baseline Comparison...”***\n\n- **Why we focus exclusively on the zero-shot setting.** Our study focuses on **zero-shot fMRI-to-image decoding** because it aligns with real-world clinical constraints. In many cases—such as patients with **consciousness disorders** or neurodegenerative diseases—collecting subject-specific training data is infeasible. Scalable brain decoding requires models that generalize across individuals without personalized adaptation. Yet this setting has been largely overlooked, with prior work focusing on fully supervised or few-shot scenarios that assume access to test-subject data. In contrast, we are the first to **define, motivate, and rigorously evaluate** zero-shot fMRI-to-image reconstruction as a practically valuable and scientifically meaningful task.\n- **Fair comparison with SOTAs.** As mentioned above, previous SOTA methods such as MindEye2 and MindTuner were designed for fully‑finetuned or few‑shot training in a subject‑specific manner, and thus cannot be directly used in the zero‑shot setting. However, we have already adapted them as baselines (denoted as “our baseline”) in Section 3.1. We believe the reviewer may have missed this point. ZEBRA significantly outperforms the adapted MindTuner in the zero‑shot setting.\n- **Balance between zero-shot and fully supervised.** To address the reviewer’s request, we evaluated ZEBRA in the fully finetuned setting against SOTA methods. As shown below, ZEBRA remains highly competitive, achieving performance on par with leading finetuned models. However, fully supervised decoding has limited real-world utility due to poor scalability and overfitting to test subjects. In contrast, ZEBRA is designed to balance fidelity and generalizability, making zero-shot decoding more practical. Thus, surpassing all finetuned methods is not the main goal of our work.\n\n| Method | PixCorr | SSIM | Alex(2) | Alex(5) | Incep | CLIP | Eff | SwAV |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| NeuroPictor | 0.229 | 0.375 | 96.5 | 98.4 | 94.5 | 93.3 | 0.639 | 0.350 |\n| MindTuner | 0.322 | 0.421 | 95.8 | 98.8 | 95.6 | 93.8 | 0.612 | 0.340 |\n| Ours | 0.245 | 0.409 | 96.6 | 98.2 | 95.2 | 92.9 | 0.594 | 0.341 |\n\n***W5: “Inadequate Evaluation...”***\n\nWe have added results across different subject cohorts within NSD in the supplementary, following prior works—these may have been missed.\n\nTo further assess generalizability, we tested our NSD-trained model directly on the independent GOD dataset. As expected, performance was limited due to differences in hardware, stimuli, and trial setup.\n\nWe agree that dataset-level zero-shot generalization is an important long-term goal, more aligned with the vision of **foundation models** that generalize across scanners and populations. While beyond the current scope, our architecture is compatible with such scaling once broader datasets are available.\n\nOur method already achieves **strong cross-subject generalization within NSD**, marking a meaningful step toward broader applicability.\n\n***W6: “The paper ...”***\n\nWe apologize for the grammatical errors, unclear phrasing, and formatting inconsistencies in the manuscript. We will carefully revise the paper in the next version."}, {"review_id": "LFmi1uQ4pq", "reviewer": "Reviewer_95aH", "summary": "This paper presents ZEBRA, a brain visual decoding framework capable of zero-shot inference of unseen subjects. ZEBRA operates by dividing the brain signal into subject-invariant features and subject-specific features and training the model accordingly. Experimental results show that ZEBRA is capable of zero-shot decoding to some degree, with results comparable to the partially trained versions of existing methods.", "questions": "- There seems to be an inconsistency between table 1 of the manuscript and table 1 of the appendix. Did the authors mistakenly put the results of subject 1 instead of the averaged results in the manuscript?\n- Was the brain-image data of the 982 test images from the training subjects used during training? For example, when testing for subject 1, were the data from subjects 2, 5, 7 on the same test images used during training?\n- What is the performance of ZEBRA for the \"seen\" subjects? In other words, what is the test set decoding performance on the subjects that were used during training?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "# Strengths\n- This is the (supposedly) first attempt at zero-shot subject decoding, achieving results comparable to the partially trained versions of existing methods.\n- The division of subject-invariant and subject-specific features is clearly motivated, and the successful division is shown in the ablation studies.\n- The main experimental results (table 1) are thorough: it was performed on the 4 full-data subjects as per convention, and it also includes some artificial zero-shot baselines since there were no attempts at zero-shot decoding previously.\n\n# Weaknesses\n- While the proposed method is zero-shot in terms of decoding a new subject, it still requires vast amounts of data from the same dataset, meaning it relies on data with the exact same imaging machine, trial setup, etc. Extending the methodology to perform zero-shot inference on a different dataset would be an interesting follow-up.\n- The actual zero-shot inference steps are not clearly detailed in the manuscript, barring some vague descriptions in the figure caption or implementation details. Describing the inference steps in detail in section 3 would help the clarity of the paper.", "was_revised": true, "final_justification": "Most of my questions and concerns have been answered by the response from the authors. Although the performance of ZEBRA for seen subjects was lower than I had anticipated, the explanation provided by the authors seems reasonable enough to justify it. For that reason, I raise my rating to a 5.", "rebuttal": "### ***Overall Reply***\n\nWe sincerely thank you for your positive assessment, including remarks such as *\"this is the (supposedly) first attempt at zero-shot subject decoding\"* and *\"main experimental results (Table 1) are thorough\"*.\n\nYour comments are consistent with other reviewers’ recognition of the novelty, thoroughness, and real-world relevance of our work. They also echo feedback such as: *\"addresses a critical challenge in neural decoding\", \"strong zero-shot performance\", \"clear methodology and results\", and \"valuable insights for neuroscience and BCI applications\"*.\n\nWe particularly appreciate your acknowledgment of the completeness of our experimental evaluation. In the following detailed responses, we will address any points requiring clarification to ensure the novelty, rigor, and impact of our work are fully conveyed.\n\n### ***Point-by-point Responses***\n\n***W1: “While the proposed method is zero-shot in terms of decoding a new subject, it still requires vast amounts of data from the same dataset, meaning it relies on data with the exact same imaging machine, trial setup, etc. Extending the methodology to perform zero-shot inference on a different dataset would be an interesting follow-up.”***\n\nWe appreciate the reviewer’s thoughtful point. Indeed, achieving **true dataset-level zero-shot generalization**—where the model transfers across different imaging systems, trial designs, and acquisition protocols—is a highly ambitious goal and aligns more closely with the scope of **foundation models** in brain decoding.\n\nWe conducted preliminary evaluation by directly applying our model (trained on NSD) to the independent GOD dataset **without any fine-tuning**. As expected, the performance degrades noticeably. This is largely due to the **limited subject diversity** and **inconsistent imaging conditions** in the available datasets, making it extremely difficult to learn fully invariant representations under current scale.\n\nWhile this dataset-level generalization is **beyond the core scope of our paper**, our method is in principle compatible with large-scale foundation model training. We believe it provides a viable **pipeline** for such models, and we are actively exploring this direction. With broader and more diverse datasets in the future, we aim to extend our approach toward this more challenging and valuable goal.\n\n***W2: “The actual zero-shot inference steps are not clearly detailed in the manuscript, barring some vague descriptions in the figure caption or implementation details. Describing the inference steps in detail in section 3 would help the clarity of the paper.”***\n\nOur zero‑shot inference process follows *MindEye2*. The diffusion prior first predicts OpenCLIP ViT‑bigG/14 image latents from the input fMRI features, which are then decoded into pixel‑space images using the SDXL‑unCLIP model. To improve image realism, these initial reconstructions are further refined via SDXL *image‑to‑image* generation, conditioned on the predicted image captions. We initialize the refinement from the noised encoding of the unrefined reconstruction, skipping the first 50% of denoising steps, and simply take the first generated samples without applying additional ranking or selection. This refinement step consistently enhances perceptual quality while leaving low‑ and high‑level evaluation metrics largely unaffected. We will include a detailed description of these inference steps in Section 3 of the revised manuscript for clarity.\n\n***Q1: “There seems to be an inconsistency between table 1 of the manuscript and table 1 of the appendix. Did the authors mistakenly put the results of subject 1 instead of the averaged results in the manuscript?”***\n\nThank you for pointing this out. You are correct — we mistakenly reported the results of Subject 1 instead of the averaged results in Table 1 of the main manuscript. We apologize for the oversight and will correct this inconsistency in the revised version.\n\n***Q2: “Was the brain-image data of the 982 test images from the training subjects used during training? For example, when testing for subject 1, were the data from subjects 2, 5, 7 on the same test images used during training?”***\n\nNo, none of the 982 test images were used during training. We ensured a strict separation between training and test images. Specifically, when testing on a given subject (e.g., Subject 1), brain responses to the 982 test images from all subjects—including Subjects 2, 5, and 7—were excluded from training. Only fMRI data corresponding to the training images were used for training, ensuring the integrity of the zero-shot evaluation.\n\n***Q3: “What is the performance of ZEBRA for the \"seen\" subjects? In other words, what is the test set decoding performance on the subjects that were used during training?”***\n\nThank you for the question. We have now included test-time decoding results for the training (i.e., *seen*) subjects.\n\nAs shown in the table below, **ZEBRA achieves consistently stronger performance on seen subjects** compared to the unseen one across all metrics, as expected. However, the improvement is moderate—**reflecting ZEBRA’s design goal of balancing subject-specific fidelity and cross-subject generalization**. To support zero-shot decoding, ZEBRA deliberately discards some subject-specific signals during training. This trade-off is key to its ability to generalize, and stands in contrast to methods that overfit to individual subjects through full finetuning.\n\nAchieving peak subject-specific performance is therefore beyond the scope of our contribution, which focuses on subject-generalizable brain decoding.\n\n| Method | PixCorr | SSIM | Alex(2) | Alex(5) | Incep | CLIP | Eff | SwAV |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| Subject 1 (unseen) | 0.152 | 0.384 | 76.1% | 81.8% | 73.4% | 72.3% | 0.814 | 0.490 |\n| Subject 2 | 0.170 | 0.394 | 84.7% | 90.3% | 81.5% | 79.5% | 0.749 | 0.440 |\n| Subject 3 | 0.159 | 0.397 | 83.9% | 88.2% | 82.0% | 78.8% | 0.755 | 0.445 |\n| Subject 4 | 0.158 | 0.396 | 83.0% | 88.4% | 81.6% | 79.4% | 0.746 | 0.445 |\n| Subject 5 | 0.177 | 0.397 | 84.0% | 90.8% | 84.8% | 82.4% | 0.729 | 0.436 |\n| Subject 6 | 0.149 | 0.386 | 81.6% | 86.9% | 81.4% | 77.7% | 0.762 | 0.449 |\n| Subject 7 | 0.154 | 0.392 | 83.5% | 88.3% | 80.8% | 78.4% | 0.758 | 0.445 |\n| Subject 8 | 0.155 | 0.395 | 81.2% | 85.6% | 77.9% | 76.0% | 0.777 | 0.455 |"}]

Author rebuttal:

Meta-review: This paper received overall positive reviews. Reviewers acknowledged that it addresses a critical challenge in fMRI-to-image neural decoding and provides thorough experimental validation. The rebuttal effectively resolved initial concerns, including those related to novelty and missing comparisons, clarifying the paper to an acceptance-worthy level. The AC concurs with the majority and considers this a solid contribution. It is recommended that the clarifications and supporting evidence from the rebuttal be incorporated into the camera-ready version. We look forward to seeing this work presented at NeurIPS.
Decision: Accept (poster)