Bridging Brains and Concepts: Interpretable Visual Decoding from fMRI with Semantic Bottlenecks
Authors: Sara Cammarota
Matteo Ferrante
Nicola Toschi
Venue: NeurIPS 2025 poster ()
Keywords: Brain Decoding
Interpretability
Vision
Neuroscience
Primary area: neuroscience_and_cognitive_science
TL;DR: We introduce a brain decoding framework that maps fMRI data to interpretable semantic concepts for image reconstruction. This pipeline maintains high reconstruction quality, revealing voxel-level concept activations, enabling neuroscientific insight.

Abstract: Decoding of visual stimuli from noninvasive neuroimaging techniques such as functional magnetic resonance (fMRI) has advanced rapidly in the last years; yet, most high-performing brain decoding models rely on complicated, non-interpretable latent spaces. In this study we present an interpretable brain decoding framework that inserts a semantic bottleneck into BrainDiffuser, a well established, simple and linear decoding pipeline. We firstly produce a $214-\text{dimensional}$ binary interpretable space $\mathcal{L}$ for images, in which each dimension answers to a specific question about the image (e.g., "Is there a person?", "Is it outdoors?"). A first ridge regression maps voxel activity to this semantic space. Because this mapping is linear, its weight matrix can be visualized as maps of voxel importance for each dimension of $\mathcal{L}$, revealing which cortical regions influence mostly each semantic dimension. A second regression then transforms these concept vectors into CLIP embeddings required to produce the final decoded image, conditioning the BrainDiffuser model. We found that voxel-wise weight maps for individual questions are highly consistent with canonical category-selective regions in the visual cortex (face, bodies, places, words), simultaneously revealing that activation distributions, not merely location, bear semantic meaning in the brain. Visual brain decoding performances are only slightly lower compared to the original BrainDiffuser metrics (e.g., the CLIP similarity is decreased by $\leq 4$% for the four subjects), yet offering substantial gains in interpretability and neuroscientific insights. These results show that our interpretable brain decoding pipeline enables voxel-level analysis of semantic representations in the human brain without sacrificing decoding accuracy.

Bridging Brains and Concepts: Interpretable Visual Decoding from fMRI with Semantic Bottlenecks
Sara Cammarota'
Matteo Ferrante"
Department of Biomedicine and Prevention University of Rome, Tor Vergata
Tether Evo
Department of Biomedicine and Prevention
Viale Montpellier, 1 Rome (IT) sara. cammarota romam.rrmm2.ccmmm eu
University of Rome, Tor Vergata matteo werrante@tether.cr
Nicola Toschi
Department of Biomedicine and Prevention University of Rome, Tor Vergata Martinos Center For Biomedical Imaging MGH and Harvard Medical School (USA) toschi @med. uniroma2. it
Abstract
Decoding visual stimuli from noninvasive neuroimaging techniques such func- tional magnetic resonance (fMRI) has advanced rapidly in the last years; yet, most perforref brain decoding models rely on complicated, non-interpretable latent spaces. In this study we present an interpretable brain decoding framework that inserts a semantic bottleneck into BrainDiffuser, well established, simple and linear decoding pipeline. We firstly produce 214 dimensional binary in- terpretable space for images, in which each dimension answers to a specific question about the image (e.g.. "Is there person?", "Is it outdoors?"). A first ridge regression maps voxel activity to this semantic space. Because this mapping is linear, its weight matrix can be visualized as maps of voxel importance for each dimension of C, revealing which cortical regions influence mostly each semantic dimension. second regression then transforms these concept vectors into CLIP embeddings required to produce the final decoded image. conditioning the Brain- Diffuser model. We found that voxel-wise weight maps for individual questions are highly consistent with canonical category-selective regions in the visual cortex (face, bodies, places, words), simultaneously revealing that activation distributions, not merely location, bear semantic meaning in the brain. Visual brain decoding performance are only slightly lower compared to the original BrainDiffuser metrics (e.g., the CLIP similarity is decreased by < 4% for the four subjects), yet offering substantial gains interpretability and neuroscientific insights. These results show that our interpretable brain decoding pipeline enables voxel-level analysis of semantic representations in the human brain without sacrificing decoding accuracy.
Introduction
Decoding the contents of the human mind from neural activity one of the main challenges of contemporary neuroscience. Brain decoding refers to the attempt to classify, retrieve or reconstruct the stimuli that elicited certain neural response, an effort translate the language of the human
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

mind into a representation of the external world (visual scenes, linguistic input, music or other
experiential domains)
48 25 23
26
35
39
46].
Recent progress in the field of Artificial Intelligence, together with increased availability of large, 28
high-quality open neuroimaging data [2] 7 22
36 29). have significantly propelled the field
forward 39 Very quickly it became possible to decode visual [41
18 20 21). language 5 16 content from brain activity, and state of the
10, 44
related [ 1 46). semantic [5] and music 14
art models are capable of doing so even when the neural correlates are measured with non-invasive techniques such as functional magnetic resonance imaging (fMRI) which infers neural activity by measuring the blood oxygen level dependent (BOLD) signals across the brain. Even though fMRI is limited by low temporal resolution, a consequence of the slowly evolving hemodynamic response function underlying BOLD signal, it presents substantial advantages: its spatial resolution is extremely precise, it allows for whole brain coverage and it is non invasive. These factors led fMRI to become an indispensable tool for cognitive neuroscience, with fMRI datasets being made accessible and well performing encoding and decoding models being proposed in recent literature. One of the most widely used datasets for brain decoding is the Natural Scenes Dataset (NSD) (2). the product of a large-scale data collection initiative following the principles of "intensive (MRI" (27). This approach shifts the focus from traditional "wide fMRI" studies (many subjects, few trials) to "deep" or "intensive fMRI" (few subjects, many trials), enabling the development of subject-specife models that leverage machine learning and facilitate experiments supporting a broad range of neuroscientific hypotheses. For these reasons, we selected the NSD as the foundation for our work.
On visual brain decoding specifically, a majority of works have converged on dual-stream architec- ture combined with a generative model: a semantic stream predicts high-level representations of the stimuli while a structural stream maintains structural features of the image, and generative model uses these predictions to produce the final stimuli reconstruction [32 37 41), These models have proven very effective and have demonstrated very high performance, especially on the NSD dataset. Deep learning approaches like these have revolutionized the field; however, a striking result is that simple linear regressions from brain activity to latent semantic representations (such as those obtained by CLIP) can achieve good, eeglalggner performance (38 42). This result may be explained by the simultancous action of macroscopic neural dynamics, such as time and space averaging and noise, which could mask the nonlinearities of smaller scales. At the macroscopic fMRI scale, the representation of concepts in the human brain may resemble the vector space of self-supervised large models like CLIP, effectively making linear models very efficient, other than robust, easy to train and potentially interpretable. Nonetheless, linear mapping between brain voxels and the entangled CLIP feature space still doesn't allow for straightforward interpretation.
Here we introduce an interpretable brain-decoding framework that bridges this gap. Inspired by a recent related study on language and the idea of factorizing linear models in concept-specifee components, propose approach to map brain activity into human-rcadable embedding space that serves as middle ground between neural correlates and the semantic embeddings of CLIP. Each dimension of this intermediate space corresponds to visual concept (such as people, motion, words), allowing us to gain interpretability on the input space, eventually advancing our knowledge of the brain. We insert this framework within BrainDiffuser (41), J well-established reconstruction pipeline, allowing us to benchmark our results and ultimately to test whether interpretability impacts image reconstruction performance.
Related Work Our work falls within the umbrella of the recent field of brain decoding literature, which demonstrated that it is possible to decode images, videos, language and music from brain activity measured via fMRI [41] 10
16 9]. The central idea of this work that the representations of external stimuli in the human brain and their latent counterparts in large-scale models share some similarities, and with sufficient (stimulus, neural activity| } pairs the two representations could be related to each other. Two recent pieces of literature, in particular, align with our study (6] 34). First, [6] elegantly showed that language brain encoding models can achieve good performance when non-interpretable sentence embeddings are replaced by interpretable vectors whose dimensions represent the answer to different questions (e.g. "Does the sentence describe a relationship between people?", "Is the sentence grammatically complex?"). This way of crafting the interpretable latent space really resonates with the way we design our interpretable embeddings for images. Second, the BrainBits study (34) demonstrated that it is possible to impose bottlenecks in a visual stimuli decoding pipeline without losing much performance, suggesting that
13 46 15
14
5
2

---

brain representation could be more compact and lower dimensional compared to the high- nigh-dimensional embeddings produced by large-scale multimodal models such as CLIP. While a vast majority of literature focuses on maximizing decoding performance leveraging several non-interpretable spaces, here we take inspiration from(6) [6] (4) and adopt a slightly different perspective: given the high performance reached by visual brain decoding models, we leverage the power of these predictive models to probe the structure of conceptual representations in the brain space.
Contributions The contributions of this work are: (i) we introduce an interpretable semantic bottleneck, interpretable by design, that preserves the information required for high-quality image reconstruction while providing each dimension with a clear meaning; (ii) we propose a pipeline to produce robust interpretable embeddings from a set of images, starting with the generation of a number of questions about the NSD carried out by an LLM, GPT-4o, followed by the answering of these questions by another model, BLIP-2: (iii) we focus on the semantic stream rather than the structural stream, decomposing the brain to semantic mapping into two separate maps: brain to interpretable space, and interpretable space to semantic CLIP space. This yields voxel-level insights that are easily visualized and compared to known activation patterns in the human brain. (iv) we show that the proposed model produces stable, anatomically plausible concept maps across subjects; we furthermore observe that the distribution of voxel-patters strength (not only the location of the activations) carries important information.
Main findings The derived interpretability maps align well with canonical regions in the visual cortex significantly and for several concepts. The results also highlight the presence of distributed networks of voxel activations, reinforcing the idea that high-level representations are encoded in our brains by large patterns of coactivations rather than small, isolated hotspots. Our approach combines good reconstructing accuracy with built-in transparency, offering a new way to fine-grained studies of semantic representations in the human visual cortex.
2 Methods
Data We used the Natural Scenes Dataset (2 a deep fMRI dataset encompassing eight healthy adult subjects who performed continuous recognition task thousands of images from the CÓCO dataset. The data were acquired in high-resolution with ultra- altra-high-fiee (7T) strength. https:://natuaasceeeeee
NSD data can be requested with other works employing the same dataset, we considered in our analysis only those subjects who completed all trials (subj01, subj02, subj05, subj07). We obtained training set of 8859 images and 24980 IMRI trials per subject, while the test set included 982 images and 2770 IMRI trials per subject. Because stimuli were presented to subjects up three times, the corresponding fMRI trials were averaged. The fMRI signal, in 1.8 mm resolution, was masked using the NSDGeneral region-of- interest mask (this includes many areas of the visual cortex), which reduced spatial dimensionality to approximately 15000 voxels per subject. On the temporal dimensionality, we used NSD-supplie beta weights from general linear model with fitted hemodynamic response functions; this condensed each voxel's 's response to single value per stimulus. Full acquisition and preprocessing details are available in the original NSD publication (2)
org/ To facilitate comparison
Classical visual decoding pipeline To develop our interpretable pipeline, we build on a classical brain decoding method for images, BrainDiffuser(41] We choose BraiuDiffueer because of its strong performance, modular architecture, and simplicity of implementation. Its clarity and flexibility make well-suited for integrating our interpretable semantic bottleneck without introducing additional confounds. BrainDiffuser is a well-established brain decoding framework that includes a two- stage scene reconstruction pipeline encompassing different latent representations of the images (VDVAE[TT) and C3). ). The VDVAE is hierarchical variational auto-encoder that stacks series of conditionally dependent latent layers, allowing each layer to capture a different aspect of image structure, from fine details to coarse global information. The original BrainDiffuser exploits the first 31 of the 75 VDVAE layers. At training time, images are fed to the pretrained VDVAE encoder and latent variables corresponding to these first 31 layers are concatenated. ridge regression model is trained to predict these concatenated VDVAE latents from fMRI activations. At test time, the trained regressor used to predict Z from the new fMRL. latents are then fed to the pretrained VDVAE decoder, resulting in 64 x 64 low-level reconstruction of the image, which is structurally
3

---

Initial guess
Estimatedd VOVAE z
VOVAE Docoder
Regressor
Z
Estimated CLIPText and CUIVVision embeddings
Stimulus
Regressor
Versatile Diffusion
Final reconstruction
Figure 1: The BrainDiffuser model reconstructs stimuli from fMRI data. A first, structural stream (top) is trained to estimate the representations of a VDVAE while second, semantic stream (bottom) estimates CLIP-Text and CLIP-Vision latent features from neural data Z. Both estimations are done with linear models. The VDVAE latents are fed the VDVAE decoder produce a first initial guess of the image, which is given as input to Versatile Diffusion together with the estimated CLIP- and CLIP- Vision vectors obtain the final reconstruction.
similar to the original picture, but lacks fine semantic detail and still unrecognizable. This serves as an initial guess for the second stage of the BrainDiffuser. The second reconstruction stage leverages Versatile Diffusion (VD) [49], a latent diffusion model which allows conditioning the generation process on both text and image features to guide the reverse diffusion process. The reverse diffusion of VD can also be initialized with latent variables obtained by an image. Stage of the BrainDiffuser consist in the training of two additional regression models: the first one maps fMRI patterns to CI.IP-Vision embeddings of the stimuli, and the second one maps fMRI data CLIP--Tes -..ttTxxt features (obtained from the captions of the COCO dataset corresponding to each NSD stimulus).
At testing time, also the initial guess is used: it is encoded by the AutoKI Encoder of the pretrained Versatile Diffusion model and noise is added to the obtained latent vector in 37 steps; the so obtained noisy vector is then fed as initialization the diffusion model and denoised for 37 steps, while conditioning on the predicted CLIP-Text and CLIP- VisVinnon features h. The result the diffusion process is then fed to the AutoKI. Decoder of the pretrained Versatile Diffusion model and the final image reconstruction is obtained. A schematic representation of the BrainDiffuser is available in Figure For further details, we refer to the original publication (41).
Notably, in the second stage the original BrainDiffuser brain activity Z is mapped into the estimated CLIP embeddings by means of a linear model = ZW where W has shape (N CL. dimension). Because the CLIP embeddings are not interpretable. it is difficult to understand the meaning of the model weight matrix W.
Interpretable space To understand how brain activity is converted into the representations given by the CLIP embeddings, we factorize the decoding weight matrix W in two separate matrices A of size noooess a and B size CLIP dimension: such that W AB. The core idea is that we can design an dimensional, semantically interpretable latent space L so that A brain activity + L and B C - CLIP space. This decomposition allows for direct inspection of A, whose entries reveal the individual contribution of each brain voxel to every latent dimension. Therefore, the factorization exposes a linear map onto semantic dimensions, enabling neuroscientific interpretation.
To design meaningful latent space L we adopt two-step pipeline. First we leverage GPT4-o [40] to create set of overlappeppp questions with binary answer (yes no) that well describe the images of the NSD dataset (e.g. "Is there person in the image?", "Is the image taken outdoors?", "Is there more than one subject?" etc.). This was done by providing the LLM with the captions for all images and the prompt "Given the following set of captions, each representing an image, generate a set of 256 binary questions that are suitable to well describe the content the images. Generate questions that are diverse and non-overlapping, and that describe completely possible all the images. These should maximize your information about the image content. The prompt asks for 256 questions. This number was set with the idea in mind of imposing bottleneck that reduces

---

"Brain to Int" Model training
"Int to CLIP Model training
BLIP2
Torget
Regressor
VEVAE:
Dascooer
li tene soppoee
BLIP2
a
Mere peosent
Questtons
a www.o
Regressor
Excrrdtedd embeediing
Versotes
Regnessor
Interpretable Brain Diffuser Inference
Reginessor
Estimotedd VOVAL tntents
Z
Regressor
Eetimotedd ond CUPPiison emberldings
Regressor
Diftusion
Cetimoted Interpretobie vector
Figure 2: Overview of the different models. Top row: (1) "Brain to Int" Model training: training pipeline of the model mapping from brain activity - intepretable space L. Each stimulus image is fed to BLIP-2 along with a fixed set 214 yes/no questions. The model's answers form the concept vector our interpretable target space. The "Brain to Int" model learns to reconstruct these vectors from brain activity; (2) "Int to CLIP" Model training: training pipeline of the model mapping from - CLIP/VDVAE space. This model learns map brain activity to VDVAE and CLIP embeddings which are used to generate the reconstructed image. Bottom row: "Interpretable BrainDiffuser Inference": the whole inference pipeline of our interpretable model. Neural data Z is first mapped to the interpretable space L via the "Brain to Int" model and then mapped to CLIP/VDVAE space via the "Int to CLIP" model). The final image is generated via Versatile Diffusion, using the estimated CLIP/VDVAE embeddings.
the dimensionality between 20% and 30% of the original CLIP dimensionality (768), knowing from previous literature result that bottleneck of 50 dimension is already able to retain most of the performance [34]. After manual inspection of LLM generated questions, removing duplicate questions, we select reduced set of 214 questions. Each dimension of L corresponds the answer of a specific question. The complete set of questions is available in the Supplementary Materials. Later, BLIP-2[30] was employed answer the 214 questions sequentially for each image in the dataset with the following question-specift prompt: "Question: [one question, for example 'is the subject smiling? 2'] Answer:". For every image, the model was queried 214 times, once per question, producing a binary ("Yes"/"No"") answer for each. The answers were converted to binary values (1 for "Yes", 0 for "No"),, resulting 21 2imedimesia interpretable vector representation for each image.
Figure Di is a scheme of the interpretable brain decoding pipeline.
We verify that L preserves CLIP-level semantics by learning ridge-regression alignment to CLIP- Vision and measuring o image-retrieval accuracy. The full protocol and results appear in Appendix
6.1
Reconstructing images from interpretable embeddings We first estimate the linear map B C CLIP by duplicating the BrainDiffuser architecture and training a Ridge regression to reconstruct the images from ground-truth interpretable embeddings (instead of brain activity). We used 5-fold cross-validation to determine the best values of the regularization coefficient a over logarithmically spaced values in the interval 10-3 S a S 10a. This latent-to-image model has two purposes. First, it verifies that the designed latent space is sufficiently informative, as high reconstruction quality indicates that the latent space encodes important properties of the NSD images. Moreover, this model is subsequently frozen and applied to the interpretable embeddings inferred from brain activity to generate the final reconstructed images.
We use Python library Himalaya Ridge regression with the standard svd solver.
5

---

Learning the map from brain activity to L We fit a ridge regression mode! - to estimate the normalized coordinates (because - CLIP expects normalized imputs) of the interpretable space L. from brain activity. 5-fold cross-validation was used to determine the optimal values of the regularization coefficient a over logarithmically spaced values the interval 10-6 S a S 10% The learned map A, one per subject, is interpretable by design: each i th column encodes the contribution of every brain voxel to the - th coordinate in L, i.c. the answer to each th binary question.
Evaluation We assess the quality of the images generated by our interpretable model and we compare them to the classical, non-interpretable BrainDiffuser based on low- and high-level metrics. Low-level metrics include PixCorr, SSIM, MSE, Cosine Similarity, 2-way accuracy in AlexNet latent space, while high-level metrics include 2-way accuracy in Inception V3 and CLIP latent spaces, correlation distance in EfficientNet and SwAV spaces, as well as 50- way-to- Accuracy using ViT-H/14 (1000 repetitions per image), replicating the original publication evaluation [41)
Visualization Visualization of brain regions involved most in the estimation of each dimension of C was achieved by first mapping each Region Of Interest (ROI) to the actual anatomical space of each participant. We then register these coordinates to the MNI-152 standard space, providing a common frame for group level analyses and comparison with canonical atlases. We compared the resulting maps to visual cortex regions knowingly related to concepts such as bodies, faces, words and places. The analysis was performed in Python using the Nilearn neuro-imaging library [ For final visualizations, we kept only the top 4% of most influential voxels and discarded any cluster smaller than 100 voxels to suppress noise. The decision to focus on the top 4% of most influential voxels was made empirically, guided by two considerations. First, this threshold yields a voxel count that comparable to that of well-characterized functional regions in the dataset, such as face-, body,, place-, and word-selective areas, each comprising a similar number of voxels 2]. Second, this proportion provides a clear and interpretable visualization.
All experiments and model training were carried out on a server equipped with 8 NVIDIA H100 GPUs, 2 TB of RAM, and 256 CPU threads. The extraction of interpretable latent embeddings with BLIP-2 took approximately 24 hours per subject and the training of the whole interpretable model took less than 30 minutes per subject. Inference time for all models is approximately 3 seconds per decoded image. Code is available at htttps://githin com/ /araaaamaaooaa Bridging-Braing and Concepts
3 Results
Tables and 2 present the results of the interpretable model evaluation compared to the original BrfinDiffuser and to the intermediate model "Int to CLIP". This last model maps the true interpretable vectors to the CLIP-Text and CLIP- Vision embeddings and serves as a check that the designed interpretable embeddings are good enough for image reconstruction. The tables report, respectively, low- and high-level metrics on the decoded images of the test set for the four subjects.
Results show that our interpretable model (Int-BD) retains good reconstructing performance (c.g.. on average we report- -4% on CLIP score and 88% on 50-way-top-1 Accuracy with respect the original BrainDiff),, confirming that interpretability does not affect excessively the reconstructing accuracy. It important to notice that, even Int to CLIP obtains the best high-level performance across all models, this is not a brain decoding model and only serves as an indication of how well L represents NSD images. Its superior accuracy stems from being the only model to bypass the noisy fMRI data, mapping directly the true interpretable embeddings to the CLIP space.
Figure 3 reports collection of image reconstructions obtained with the three pipelines for different subjects. Images reconstructed by our interpretable model show visual fidelity and similarity to the ground truth images, comparable to the ones achieved by the original, non-interpretable BrainDiffuser.
Figure 4 subset of the interpretable embedding dimensions. We compare the activated voxels to the standard ROIS related to bodies, faces, places and text/words. The so-obtained activation maps exhibit good similarity with standard concept-related regions.
displays displays the column entries of the interpretable matrix A, mapped to the MNI-512 space, for
See Footnote 1.

---

Table 1: Quantitative analysis of reconstructed images for all subjects. Here are reported low-level image metrics. Upward arrow (t): higher better; downward arrow (J): lower better.
Subject
Model
PixCorr
SSIM
MSE.
Cosine Sim.
AlexNet(2) f AlexNet(55)
BrainDiffuser
0.285
0.359
0.119
0.801
0.948
0,966
Subj01
Int-BD Int to CLIP
0.153
0.320
0.140
0.767
0.795
0,869
0.156
0.309
0.148
0.757
0.813
0.894
BrainDiffuser
0.241
0.355
0.126
0.789
0.931
0.956
Subj02
Int-BI) Int to CLIP
0.149
0.319
0.141
0.764
0.783
0.866
0,155
0.311
0.149
0.755
0.819
0.897
BruinDiffuser
0.214
0.344
0.779
0.910
0,950
Subj05
Int-BD Int to CLIP
0.154
0.320
0.140
0.765
0.798
0.878
0.154
0.312
0.149
0.758
0.808
0.894
BrainDiffuser
0.213
0.345
0.134
0.781
0.901
0.940
Subj07
Int-BI
0.142
0.317
0.143
0.761
0.787
0.861
Int to CLIP
0.160
0.311
0.148
0.756
0.817
0.895
Table 2: Quantitative analysis of reconstructed images for all subjects with high-level metries. Upward arrow (t): higher better; downward arrow (L): lower better.
Subject
Model
Incep
CLIP
EffNet Dist.
SwAV Dist.
50-way-top-1 Acc. 1
BrainDiffuser
0.914
0.923
0.710
0.407
0.53 0.23
Suhj01
Int-BD
0.851
0.885
0.782
0.470
0.46 - 0.23
Int CLIP
0.942
0,955
0.655
0.412
0.69 0.20
BrainDiffuser
0.909
0,907
0.725
0.417
0.52 :: 0.23
Subj02
Int-BD Int to CLIP
0.842
0.869
0.798
0.480
0.42 0.23
0.946
0,952
0.661
0.412
0.70 0.20
BrainDiffuser
0.917
0.928
0.702
0.409
0.55 0.22
Subj05
Int-BD Int to CLIP
0.860
0.886
0.771
0.466
0.47 + 0.23
0.941
0,951
0.668
0.417
0.68 = 0,20
BrainDiffuser
0.886
0.900
0.736
0.430
0.50 0.21
Subj07
Int-BD
0.831
0,868
0,801
0.484
0.43 :: 0.23
Int CLIP
0.933
0.950
0.668
0.416
0.68 0.20
We also find that the specific distribution of voxel activations within the same region varies with each sub-concept examined. For instance, in Figure the reference region bodies (second group from the top) displays distinct activation patterns depending on the specific question. Even though all activations tend to overlap with the reference regions, the distribution voxel intensity varies depending the specific sub-concept. The question "Is someone running in the image?" results in widespread pattern of low-level activations, interrupted by a few cluster of highly active voxels. In contrast, the question related to dancing yields more intense activation of those same clusters, with little or no activation in the surrounding voxels. Similar comments can be made when comparing place-related activations (first group from the top in Figure 4. such the questions "Is the image taken outdoors?". "Is the image taken on a street?". We don't observe a sharp distinction in the activations for faces and bodies. Neuroscientific studies have shown that face- and body-selective areas are anatomically close in the human (and monkey) high-level visual cortex and often co-activate when a whole person animal is present 477. This is simply explainable because, under normal circumstances, faces and bodies are seen together. For many of the questions related to bodies and faces we observed an overlap with both face and body regions. For example, the questions "Is the subject sitting?" and "Is the subject running?" show strong overlap with body-selective regions but also notable overlap with face areas.
4 Discussion
In this study we have presented an interpretable model for the decoding of visual stimuli from brain fMRT data. Prior work has largely explored how to obtain performance gains over ofate-of-the-ar models; our study has taken complementary approach. We believe that incremental advancement
7

---

Stimulus
Brain Diffuser
Interpretable Brain Diff
Stimulus
Brain Diffuser
Int. CLIP Diffuser
Interpretable Brain Diff
Diffuser
Figure 3: Comparison of our reconstruction results (columns 4, 8) to the original stimuli (columns 1, 5), the images reconstructed by the original BrainDiffuser (columns 2, 6) and those reconstructed by the Int to CLIP model (columns 3, 7). Results presented here refer to subj01.
is an incredibly valuable path of research. However, the field of brain decoding is starting to show signs of performance saturation. In this work, rather than contributing to the field solely with incremental performance gains, focused on combining sate-of-theeart decoding accuracy with clear, interpretable mechanisms, with the ultimate goal of unlocking deep insights into human brain neural funetions. By mapping voxel signals them into CLIP embeddings, we reformulate decoding as a two-step problem, creating a bridge between low-level BOLD signals and human concepts. We have found that this semantic bottleneck allows for the extraction of rich neuroscientific insights with minimal performance loss, as the generated images incurred in only minimal decline in both low- and high-level metrics.
camman-readable semantic space before projecting
Concept-specific voxel activations found in this study align well with canonical regions of the visual cortex, supporting their validity. Furthermore, the spread pattern of informative voxel activations reinforces the hypothesis of distributed rather than localized semantic encoding in the brain. As highlighted by the activation maps, the patterns of co-activations carry crucial information that is not completely expressed by the voxels location only.
To the best of our knowledge, this is the first complete brain decoding pipeline that introduces truly interpretable intermediate representation. While post-hoc explainability methods such as Grad-CAM 45). SHAP 333, or attention-based saliency can technically be applied to existing models, these techniques generally rely on the assumption that the model's output space is inherently interpretable. In the context of brain decoding, this is often not the case: most models operate between two abstract spaces neural activity and CLIP-like embeddings making it challenging to extract meaningful insights from such explanations. This limitation motivated the design of our semantic bottleneck. By explicitly structuring the intermediate representation around human-interpretablle concepts, framework provides more transparent link between brain activity and visual content. We believe this principled integration interpretability into the core model architecture offers more robust and actionable understanding than standard post-hoe techniques.
Implications for neuroscience Our work carries several implications for neuroscience. First, the semantic latent space offers link from neural data to concepts, enabling voxel-level neuroscientific insights. Alignments with established ROIs support the validity of the decoding approach. Moreover, our findings support the hypothesis that, fMRI scale, brain representations of visual concepts can be linearly approximated in semantic spaces similar to those learned by modern rision-language models,
8

---

places RQI
image taken outdoors?
image?
image street?
there city skyline visible?
there trees imago?
NSDrhodies ROI
Are the people the image sitting?
Is there a person the image?
running Image?
Is someone dancing the image?
Is someone walking in the image?
NSD faces ROI
is someone wearing hat?
the main subject smiling?
main subject looking directly at the camera?
NSD words ROI
Does the image contain any text or signage?
Figure Activation maps projected MNI-512 space, compared to category-specifer reference maps. Sections correspond to places (top left). bodies (top right), faces (bottom left) and words (bottom right). Within each section, the blue map shown first presents the reference for that category. Other images show the activations derived from the interpretable matrix A. All maps are from subject01.
such as CLIP. Finally, the observed distribution of activation patterns underscores the complexity and redundancy of semantic representations the brain.
Model Interpretability vs. Performance Trade-off The introduction of semantic bottleneck inevitably leads to a reduction in reconstruction quality. This is expected: the prompts are derived from human-authored COCO captions via BLIIP-2 and GPT-4o, and human descriptions typically emphasize semantic content over geometrie layout. Nevertheless, curated question set does

---

include some geometry-oriented items. However, the measured drop in performance is small and offset by significant gains interpretability and neuroscientific observations. The Int to CLIP model serves an upper bound, revealing that the chosen interpretable space preserves most of the information required for faithful reconstructions.
Methodological strengths The use of a large and high-quality dataset (NSD) ensures robust training and evaluation of the results obtained. The two-stage regression approach (brain - interpretable space - CLIP space) decouples decoding from semantic interpretation, allowing for model transparency. Importantly, the proposed framework architecture-agnosice the semantic bottleneck operates prior to the final CLIP embedding prediction, making it compatible with any decoding pipeline that maps brain activity to CLIP or similar multimodal embeddings, which is common among many decoding pipelines. Using GPT-4o and BLIP-2 2 for the generation of question-answer pairs makes the process of generating the interpretable space automated and scalable. Finally, the consistency of voxel activation patterns across different subjects confirms the robustness and generalizability of the method.
Limitations Of course, this approach comes with some limitations. First, the introduction of the semantic bottleneck causes slight performance degradation in both low- and high-level reconstruction metrics compared to the non-interpretable BrainDiffuser baseline. Even though our main target is the semantic content of the reconstructed images some questions related to the picture geometry and the spatial positioning of the subjects are present in the chosen set. However, is possible that these questions do not capture the whole spectrum of features present in complex NSD images, yielding decreased low-level metrics of reconstructed images. This leads to a second limitation of the approach: interpretability is constrained by the relevance and completeness of the generated question set, and the extension of interpretability for structural information is worth to be explored in the future. We also note that the overall quality of the pipeline limited by the performance of the VQA model used to build the interpretable representations of the images, BLIP-2 in this case. Finally, our study is currently limited to the NSD dataset and visual decoding tasks. Whether it possible to generalize to other modalities remains to be demonstrated.
Future directions Possible paths for future research include an expansion of the interpretable space to incorporate a wider and more diverse set of concepts, possibly hierarchical, in order to produce a more granular representation of the images, ultimately allowing for increased image reconstruction performance and, more importantly, a wider analysis of activation maps. The application of this framework to other datasets and modalities (such as auditory decoding or language processing) is possible, as well as its extension to future imagery data in which participants imagine visual scenes, objects, or concepts instead of viewing them directly, when these will be available. Finally, moving beyond pure semantics and integrating greater set of structural interpretability elements (e.g. object layout, scene geometry, subjects position) could help in clarifying how the brain represents complex visual information.
Privacy and Ethics considerations With the advancement of brain decoding and image reconstruc- tion technologies, there is potential for misuse and privacy violations. Even though current models require the cooperation of the subjects and are strongly limited by the nature of the training data, future improvements may expand capabilities and it's important to ensure responsible use of brain data.
5 Conclusions
We investigated whether interpretability in visual brain decoding can coexist with high-fidelity image reconstruction. Our two-stage framework, initially mapping IMRI data into a semantic latent space and subsequently projecting into the CLIP latent space, quantitatively shows that this kind of model transparency imposes only modest performance drop with respect to classical pipelines, while bringing the advantage of built-in interpretability. Moreover, the category-selective voxel patterns we found overlap with canonical hotspots on the visual cortex, providing neuroscientific validation and aligning with the idea that semantic content is distributed in the human brain. These results offer a practical way to study how concepts are represented in the brain.
10

---

References
[1] Alexandre Abraham et al. "Machine learning for neuroimaging with scikit-learn.. In: Frontiers
in Neuroinformatics Volume 8 2014 (2014). ISSN: 1662-5196. DOI: 10 3389 fninf 2014. 00014 URL: https: www..//wwww..... articles/ 10. 3389/f38nnnnn fminff 2014. .00014
www frontiersin org/ www.rrrassseuoooooooornooo.a.aaa http://uuuoooooooi......o
[2] Emily J Allen al. "A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence". In: Nature Neuroscience 25.1 (Jan. 2022), pp. 116-126.
[3] Richard Antonello, Aditya Vaidya, and Alexander G. Huth. Scaling laws for language encoding models in SMRI. 2024. arXiv: 2305 11863 [cs. CL] URL: https : / arxiv. org/ abs
2305 11863
[4]
Hubert Banville al. Scaling laws for decoding images from brain activity. 2025. arXiv:
2501 15322 [eess. IV] URL: pettss//arxcer org/ wwrg/abs/25011. 15322
[5] Ludovic Bellier et al. "Music can be reconstructed from human auditory cortex activity using nonlinear decoding models". en. In: PLoS Biol. 21.8 (Aug. 2023), e3002176.
[6] Vinamra Benara et al. Crafting Interpretable Embeddings by Asking LLMs Questions. 2024.
arxiv: 2405 167166711 14 [cs. CL] URL: https: /ttp::///aaxvvv arxiv. org/ abs 2405 16714
[7] Nadine Chang al. "BOLD5000, public IMRI dataset while viewing 5000 visual images".
In: Scientific Data 6.1 (May 2019), 49. ISSN: 2052-4463, DOL: 10. 1038/s41599 019
0052-3 URL: httpt:ttt///so http:///aio..////// /0rg/10.. 10. 1038/
[8] Ting Chen et al. Simple Framework for Contrustive Learning of Visual Reprexentations. 2020. arXiv: 2002. 05709 [cs LG]] URL: https://arxiv httpaarwwwi..gigaaaaaaaaee arrg/bbs/000/ 05709
[9] Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity. 2023, arXiv: 2305. 11675 [cs. CV]
[10] Zijiao Chen et al. Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding. 2023, arXiv: 2211. 06956 [cs [cs.CV]] URL: https: pestps://arxiy
org/ abs/2211 221122211111 06956
[11] Rewon Child. Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images. 2021. arXiv: 2011 10650 [cs. [c].LG]
URL: https: : arxiv. org/ abaaaba/aa 2011
10650
[12] David Cox and R. Savoy. "Functional magnetic resonance imaging (fMRI) "brain reading": Detecting and classifying distributed patterns of fMRI activity in human visual cortex". In: Neurolmage 19 (July 2003), pp. 261-70. DOI: 10. 001.//011111-000 (11
[13] Alexandre Défossez et al. "Decoding speech perception from non-invasive brain recordings".
In: Nature Machine Intelligence 5.10 (Oct. 2023), pp. 1097-1107.
[14] Timo I. Denk et al. Brain2Music: Reconstructing Music from Human Brain Activity. 2023.
Xiv: 2307 11078 [q-bio.No NC]]
[15] Matteo Ferrante, Tommaso Boccato, and Nicola Toschi. Semantic Brain Decoding: from fMRI
to conceptually similar image reconstruction of visual stimuli. 2023, arXiv: 2212. 06726
URL: https:///axiiv org/ 2aaba/22122 00726
[16] Matteo Ferrante, Matteo Ciferri, and Nicola Toschi. RB Rhythm and Brain: Cross-subject Decoding of Music from Human Brain Activity. 2024. arXiv: 2406 15537 [q-bio.No URL.:
https://arxixs org/ abs 2406/2406..0 15537
[17] Matteo Ferrante al. "Multimodal decoding of human brain activity into images and text". In: Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models. Ed. by Marco Fumero et al. Vol. 243. Proceedings of Machine Learning Research. PMIR, 15 Dec 2024, pp. 87-101. URT:: htt.t:tt:::::pceeeeeeeee
mlr press/ h/4/////r33//eeee// ffrraateeeaaaa...m html
[18] Matteo Ferrante al. "Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro- neuro-inspired inspired brain decoding model". In: Journal of Neural Engineering 21.4 (2024). p. 046001. DO1: 10. 1088 1740 111111711111221111 2552/ad593c
[19] Matteo Ferrante et al. "Through their eyes: Multi-subject brain decoding with simple alignment techniques". In: Imaging Neuroscience 2 (May 2024), pp. 1 1-21. ISSN: 2837-6056. DOI:
10. wt..166immaag. / L_a\__aaa___________________ _00170 URL: http::ttt::// h/tssp::/ddoo.ooo org 1 wtt.:/11111mmm.........
00170
[20] Mattco Ferrante et al. Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion. 2024. arXiv:
9411.09723 [cs.CV)
URL:
org/ abs/ 24ts/ 2411.0111999 09723

---

[21] Guy Gaziv et al. "Self-supervised Natural Image Reconstruction and Large- Semantic Classification from Brain Activity". en. In: Neurolmage 254 (July 2022). p. 119121. ISSN: 10538119. Dor: 10. 1016/j wwwrooiageeeeeeee..... 2022. 119121 URL: https: https:://tsiii //1inkinghub
@lsevier.com/rte olapppaa...ooooerrreeeee/aa/aaap (visited on 11/13/2022).
[22]
Zhengxin Gong et al. "Nanarad@OeeiccDattaee"""" OpenNeuro, 2023. DOI: doi: 10. 18112
[23] James V. Haxby al. "Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex". In: Science 293.5539 (2001), pp. 2430. DOr: 10. 1126/
science 1663736 eprint: https: /
//www www science org doi /ttp/pdf/10/// 1126 science 1126/ science. 1063736
1063736 URL: httph::ttt//ttt http://ww.ccieeeeec wnc.enc....cccccccccc........c.c wwtp:/t.t.../ooo///t.t/oo/oooaat
[24] John-Dylan Haynes and Geraint Rees. "Predicting the orientation of invisible stimuli from
activity in human primary visual cortex". In: Nature Neuroscience 8.5 (May 2005). pp. 686- 691. ISSN: 1546-1726. DOI: 10 1038/nnneeen 1445 URL: Ittps://t org/10. (038//mm444
[25] Yukiyasu Kamitani and Frank Tong. "Decoding the visual and subjective contents of the human brain".. In: Nature Neuroscience 8.5 (May 2005), pp. 679-685. ISSN: 1546-1726, DOI:
10.1038/nn 1444 URL: http::ttts//ooioooooooo org/ 1038/nn1444
[26] Kendrick N. Kay et al. "Identifyyng natural images from human brain activity". In: Nature 452.7185 (Mar. 2008), pp. 352-355. ISSN: 1476-4687. D01: 10. 1ew.1008/nattt ntt//////ture007113 URL:
attps://doi. org/ 10. 1038/ 1000///a/tt/// n////nature6613
[27] Eline R. Kupers et al. "Principles of intensive human neuroimaging". In: Trends in Neu- rosciences 47.1 (2024), pp. 856-864.
ISSN: 0166-2236. Do1: 10. 1016/j tins 2024
09 011 URL.: https / www sciencedirect. com science / article pii /
50166223624001225
[28]
Benjamin Lahner et al. "Modeling short visual events through the BOLD moments video fMRI dataset and metadata". In: Nature Communications 15.1 (July 2024), p. 6241. ISSN: 2041-1723. DOI: 10. 1038/ 0009/54144 -41467 024---00---- 50310-3
URL: https: org/ 10. 1038/ 500888//1447--222
[29]
Amanda LeBel et al. "A natural language fMRI dataset for voxelwise encoding models". In: Scientific Data 10.1 (Aug. 2023), 555. ISSN: 2052-4463, Dot: 10. 1038/ 1038/$41597-025 s41597-023
02437-z URL: https://doi org/10. 1038 /00000//44400 023-41997720-27777---
[30] Junnan Li et al. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image
Encoders and Large Language Models. 2023,
arXiv: 2301 12597 [cs.CV]
URL: https
//arxiv. org/ /ass//331...2 12597
[31] Sikun Lin, Thomas Sprague, and Ambuj K Singh. Mind Reader: Reconstructing complex images from brain activities. 2022. arXiv: 2210. 01769 [q-bio.No NC]]
[32] Yizhuo Lu al. MindDiffuser: Controlled Image Reconstruction from Human Brain Activity
with Semantic and Structural Diffusion. 2023. arXiv: 2308 04249 Cv)
URL: https
//arxiv. .rgg abs/2308. 04249
[33] Scott Lundberg and Su-In Lee, A Unified Approach to Interpreting Model Predictions. 2017.
arXiv: 1705. 07874
[cs. AI] URL: httptt://asxiiiii org/ has///ss/111 1105 0ttp//1055007777444.
[34] David Mayo et al. "BrainBits: How Much of the Brain are Generative Reconstruc-
tion Methods Using?" In: Advances in Neural Information Processing Systems. Ed. by
A. Globerson al. Vol. 37. Curran Associates, Inc., 2024, pp. 54396-54420. URL:
https proceeding neurips cc paper files paper / 2024 file /
Paper- Conf Conference...oocoooooccccoo..... pdf
[35] Yoichi Miyawaki et al. "Visual Image Reconstruction from Human Brain Activity using Combination of Multiscale Local Image Decoders". In: Neuron 60.5 (Dec. 2008), pp. 915-929,
ISSN: 0896-6273.
DOI: 10. 1016/j neuron. 2008 004 URL: httpp:://ti..ooo
1016/j. newron. 2008 10.
[36] Tomoya Nakai, Naoko Koide-Majima, and Shinji Nishimoto. "Music genre neuroimaging dataset". In: Data in Brief 40 (2022). p. 107675. ISSN: 2352-3409 DO1: https: org/
10. 10.6/1 dib. 2021 107675 URL: https: www sci sciencedirect. com science/ article/pp1/222525000100039
[37] Thomas Naselaris et al. "Bayesian Reconstruction of Natural Images from Human Brain
Activity" In: Newron 63.6 (2009), pp. 902-915. ISSN: 0896-6273. DOT: chttps:/// //dooi
org/ 10. 1016 neuron. 2009 09. 006 URI.: https::::: www. sciencedirect. com/
12

---

[38] Erfan Nozari et al. "Macroscopic resting-state brain dynamics are best described by linear models". In: Nature Biomedical Engineering 8.1 (Jan. 2024), pp. 68 68-84.
[39] Subba Reddy Oota et al. Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey). 2024. arXiv: 2307 10246 Ng-
URL: /stps://arrivv org/
abs/ 2oo/ 10246
[40]
OpenAl et al. GPT-4o System Card. 2024. arXiv: 2410 21276 [cs.CL]
URL: https
/ //arxiv.oo org/ wwtpaaa/....// 21276
[41]
Furkan Ozcelik and Rufin VanRullen. "Natural scene reconstruction from fMRI signals using generative latent diffusion". In: Scientific Reports 13.1 (Sept. 2023), 15666.
[42] Steven T Piantadosi al. "Why concepts (probably) vectors". In: Trends Cogn. Sci.
28.9 (Sept. 2024), pp. 844-856.
[43] Alec Radford et al. Learning Transferable Visual Models From Natural Language Supervision.
2021. arXiv: 2103. 00020 [cs.CV]] URL: https:: /twxiv.xivv./////i///x/////ava./ org/ httg//bb22110...00
[44] Paul Scotti et al. Reconstructing the Mind's s Eye: MMRI-t-Imaae with Contrastive Learning and Diffusion Priors. 2023. arXiv: 2305 18274 [cs.CV]
URL: httts:://axxii/ /abs/
18274!
[45] Ramprasaath R. Selvaraju et al. "Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization". In: International Journal of Computer Vision 128.2 (Oct. 2019), pp. 336-359, ISSN; 1573-1405, DOr: 100.1007//11122--11--00222 7/áx. doi 111.://0110//1111-11111 01228-77
URL: http:
[46] J. Tang, A. LeBel, S. Jain, et al. "Semantic reconstruction of continuous language from non- invasive brain recordings". In: Nature Neuroscience 26 (2023), pp. 858-866 DO1: 10. 1038/ -41593- 023 01304-94
[47] Jessica Taubert et al. "One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system". In: Brain Struct. Funct. 227.4 ..4 (May 2022), pp. 1423-1438.
[48] Bertrand Thirion et al. "Inverse retinotopy: Inferring the visual content of images from brain activation patterns". In: Neurolmage 33.4 (2006), pp. 1104-1116.
ISSN: 1053-8119, DOI:
https // doi org 10. 1016/ neuroimage 2006 06 062 URL: https // www
sciencedirect. com wwt.c//cw../c///tc.ccc..ooocccee cttp://ciiaeeeeaaaaaaaaa/a//SSSS /tt.c//ee/pp../111111100000031
[49] Xingqian Xu et al. Versatile Diffussion: Text, Images and Variations All in One Diffusion Model.
2024. aiv: 2211 08332 [cs.CV]
URL: https: //rxxviivvv/.///////////a///aaa/ wwerii..org/aaba abs/ 2aabs/2211 2211 08332
13

---

Appendix
6.1 Evaluation of interpretable space C
To determine whether the latent space L retains enough semantic detail to eventually be mapped reliably into the CLIP space by our Interpretable Bifiseif we carried the following analysis. For every image in the training and test partitions of NSD we extracted its CLIP- VisPon embedding CLIP and its interpretable embedding 1 E L. We then fit linear ridge-regression model CLIP, selecting the regularization coefficient on a logarithmic grid from 10 to 10 by 5-fold eross- velisaaalii..nn.. After training. the model projected each test embedding into predicted CLIP vector è BI. Finally, we performed t-nearest-neighhon retrieval with k: E (1, 5, 10, 15). For every test image we queried the original CLIP index with è and recorded how often the ground-truth image appeared within the top-k returned results. The retrieval accuracy calculated in such way quantifies how faithfully information in can be transferred to CLIP. obtained for the four values of k. These results confirms the quality of the designed interpretable representations, further supported by the fact that, even when the ground truth image is absent from the top-k list, the nearest neighbors retrieved by the mapped embeddings typically show strong similarity with the query. Figure 5 displays a set of representative examples that illustrate this similarity.
B B -
the accuracies
Table 3: Top-k-NearesttNeigee Neiihors retrieval accuracy achieved after projecting latent embeddings C into CLIP-Visie space and querying the original CLIP index.
Metric
Accuracy
Top-I-NN
29%
Top-5-NN
59%
Top-10-NN
74%
Top-15
82%
Chance level
0.1%
Query
Retrieved
Query
Retrieved
Query
Retrieved
Figure 5: For 9 randomly selected test images (left in each pair), show the nearest neighbor returned by I-NN search the original CLIP index after projecting the latent embedding L into CLIP space (right in each pair).
14

---

6.2 Reconstruction examples
Below, we present series of images reconstructed by our interpretable model, along with their corresponding images generated by the original BrainDiffuser and the Iur to CLIP model. These reconstructed images refer subjects 2. 5 and 7, respectively. For the results relative to subject please refer to the main section of the article under "Results."
Stimulus
Brain
Int. to CLIP Diffuser
Interpretable Brain Diff
Stimulus
Brain Diffuser
Int. to CLIP Diffuser
Interpretable Brain DIff
Diffuser
FLE
Figure 6: Comparison of our reconstruction results (columns 4, 8) the original stimuli (columns 5), the images reconstructed by the original BrainDiffuser (columns 2, 6) and those reconstructed by the Int CLIP model (columns 7). Results presented here refer to subj02.
15

---

Stimulus
Brain Diffuser
Diffuser
interpretable Brain Diff
Stimulus
Diffuser
Diffuser
Interpretable. Riff
Figure 7: Comparison of our reconstruction results (columns 4, 8) to the original stimuli (columns 1, 5), the images reconstructed by the original BrainDiffuser (columns 2, 6) and those reconstructed by the Int to CLIP model (columns 7). Results presented here refer to subj05.
Stimulus
Brain Diffuser
Int. to CLIP Ditfuser
Interpretable Brain Diff
Stimulus
Diffuser
Int. to
Diffuser
Interpretable Brain Diff
Figure 8: Comparison of our reconstruction results (columns 4, 8) to the original stimuli (columns 1, 5), the images reconstructed by the original BrainDiffuser (columns 2, 6) and those reconstructed by the Int to CLIP model (columns 3, 7). Results presented here refer to subj07.
16

---

6.3 Brain Regions Visualizations
In this section we present activation maps derived from the interpretable model for subjects 2, and 7, visualized in MNI-512 space and compared against category-specific reference maps. For the corresponding figure related to subject 1, please see the "Results" section of this article.
NSD
image outdoors?
there desert the image?
is the image taken on a street?
there city skyline visible?
there imago?
NSDthodies RO!
Are the people the image sitting?
Is there person the image?
Is someane running in the Image?
dancing Image?
Is someone walking in the image?
NSD Faces ROI
someone wearing hat?
the main subiece smiline?
Is the main subject looking directly at the camera?
NSD words ROJ
Does the image contain any text signage?
Figure 9: Activation maps for subject 2, projected MNI-512 space and compared to category- specific reference maps. Sections correspond to places (top left), bodies (top right), faces (bottom left) and words (bottom right). Within each section, the blue map shown first presents the reference for that category. Other images show the activations derived from the interpretable matrix A.
17

---

image outdoors?
image?
is the image taken on a streed?
there city skyline visible?
image?
NSD bodies RO!
Are the people the image sitting?
is there a person the image?
Is someone running in the image?
someone dancing the image?
Is someone walkinng in the image?
NSD Faces RO!
someone wearing hat?
the main subject smiling?
Is the main subject looking directly at the camera?
NSD words RO]
Does the image contain text signago?
Figure 10: Activation maps, for subject 5, projected MNI-512 space and compared category- specific reference maps. Sections correspond to places (top left), bodies (top right), faces (bottom left) and words (bottom right). Within each section, the blue map shown first presents the reference for that category. Other images show the activations derived from the interpretable matrix A.
18

---

image outdoors?
image?
the image taken on a streed?
there city skyline visible?
image?
NSD bodies RO!
Are the people the image sitting?
is there a person the image?
Is someone running in the image?
someone dancing the image?
Is someone walkinn in the image?
NSD Faces RO!
someone wearing hat?
the main subject smiling?
Is the main subject looking directly at the camera?
NSD words ROJ
Does the image contain any text signage?
Figure II: Activation maps for subject 7, projected in MNI-512 space and compared to category- specific reference maps. Sections correspond to places (top left), bodies (top right), faces (bottom left) and words (bottom right). Within each section, the blue map shown first presents the reference for that category. Other images show the activations derived from the interpretable matrix A.
19

---

6.4 Complete list of questions
1. Is there person in the image?
2. Is there more than one subject?
3. Is the image taken indoors?
4. Is the image taken outdoors?
5. Is the image related sports?
6. Is the image related to animals?
7. Is the image related to transportation?
8. Does the image contain water bodies like a sea, ocean, or river?
9. Are there any buildings visible the image?
10. Are there trees in the image?
11. Is the image related to food?
12. Does the image contain any text or signage?
13. Is the image taken during the daytime?
14. Is the image taken night?
15. Are vehicles present in the image?
16. Are there any children in the image?
17. Are the people in the image standing?
Are people in the image sitting?
19. Is the image related celebrations or parties?
20. Is the image related to bathroom restroom?
21. Is there a surfboard in the image?
22. Is someone riding a wave in the image?
23. Are people sitting on rooftop?
24. Does the image show toilet?
25. Are there zebras in the image?
26. Is there a shower visible in the image?
27. Are there animals standing on dirt ground?
28. Is there sand in the image?
29. Is there mountain in the image?
30. Are there cars parked in the image?
31. Does the image include bicycles?
32. Are there birds in the image?
33. Are there plants flowers visible?
34. Is someone holding umbrella?
35. Is someone wearing hat?
36. Is the image taken on a beach?
37. Are there boats in the image?
38. Is there bridge in the image?
39. Are there stairs in the image?
40. Is there a window visible in the image?
41. Is there a table in the image?
42. Is there chair in the image?
43. Is there a bed in the image?
20

---

44. Is there a lamp in the image?
45. Are there cups or glasses visible?
46. Are there plates visible?
47. Is there a computer or laptop in the image?
48. Is there mobile phone visible?
49. Are there books or papers in the image?
50. Are there shelves in the image?
51. Is there a refrigerator in the image?
52. Is there microwave in the image?
53. Is there stove oven visible?
54. Is there a washing machine in the image?
55. Are there curtains in the image?
56. Are there rugs or carpets visible?
57. Are mirrors present in the image?
58. Are there paintings or artworks the walls?
59. Is there a clock visible in the image?
60. Is there a park in the image?
61. Is the image set in a forest?
62. Is there snow in the image?
63. Is it raining in the image?
64. Is there fog or mist in the image?
65. Are there mountains in the background?
66. Is there a desert in the image?
67. Is there city skyline visible?
68. Is the image taken on a street?
69. Is there a marketplace in the image?
70. Are there shops or stalls visible?
71. Are there fences or barriers visible?
72. Is the image taken an amusement park?
73. Are there swings or playground equipment visible?
74. Are there stairs or escalators in the image?
75. Is the image taken subway or train station?
76. Are there traffic lights in the image?
77. Is someone swimming in the image?
78. Is someone running in the image?
79. Is someone walking in the image?
80. Is someone cycling in the image?
81. Is someone playing musical instrument?
82. Is someone reading in the image?
83. Is someone cooking in the image?
84. Is someone eating or drinking?
85. Is someone talking another person?
86. Is someone taking a photo?
87. Is someone holding object?
21

---

88. Is someone painting or drawing?
89. Is someone driving a vehicle?
90. Is someone fishing in the image?
91. Is someone dancing in the image?
92. Are people wearing jackets coats?
93. Are people wearing swimsuits?
94. Are people wearing uniforms?
95. Are people wearing traditional clothing?
96. Are hats or caps visible?
97. Is the image symmetrical?
98. Does the image contain bright colors?
99. Are shadows visible in the image?
100. Are reflections visible in the image?
101. Is there smoke fire in the image?
102. Does the image contain unusual patterns?
103. Are there pets in the image?
104. Are there insects visible in the image?
105. Does the image have signs damage or destruction?
106. Are there fences railings visible?
107. Is the main subject human?
108. Is the main subject an animal?
109. Is the main subject a man?
110. Is the main subject a woman?
111. Is the main subject child?
112. Is the main subject elderly?
113. Is the main subject a group of people?
114. Is the main subject alone?
115. Is the main subject smiling?
116. Is the main subject interacting with someone?
117. Is the main subject looking directly at the camera?
118. Is the main subject facing away from the camera?
119. Is the main subject partially visible (cropped)?
120. Is the main subject wearing formal clothing?
121. Is the main subject wearing casual clothing?
122. Is the main subject carrying an object?
123. Is the main subject holding tool?
124. Is the main subject using electronic devices?
125. Is the main subject sitting the ground?
126. Is the main subject climbing?
127. Is the animal a mammal?
128. Is the animal a bird?
129. Is the animal reptile?
130. Is the animal an amphibian?
131. Is the animal fish?
22

---

132. Is the animal a pet?
133. Is the animal a farm animal?
134. Is the animal a wild animal?
135. Is the animal flying?
136. Is the animal swimming?
137. Is the animal cating?
138. Is the animal drinking water?
139. Is the animal alone?
140. Are there multiple animals?
141. Are the animals interacting with each other?
142. Is the animal domesticated?
143. Is the main subject an object?
144. Is the object made of wood?
145. Is the object made of metal?
146. Is the object made of plastic?
147. Is the object broken?
148. Is the object old or vintage?
149. Is the object modern?
150. Is the object electronic?
151. Is the object artistic decorative?
152. Is the object used for work or utility?
153. Is the main subject in the center of the image?
154. Is the main subject on the left side?
155. Is the main subject the right side?
156. Is the main subject near the top?
157. Is the main subject near the bottom?
158. Is the main subject partially out of the frame?
159. Is there a background behind the main subject?
160. Is the main subject framed by other objects?
161. Is the main subject closer to the foreground?
162. Is the main subject farther in the background?
163. Does the image appear staged?
164. Does the image look candid natural?
165. Is there movement captured in the image?
166. Is the image static?
167. Does the image look artistic or abstract?
168. Is the setting rural?
169. Is the setting urban?
170. Is the setting domestic?
171. Is the setting industrial?
172. Is the setting natural?
173. Is it sunny in the image?
174. Is it cloudy in the image?
175. Is there rain in the image?
23

---

176. Is it snowing in the image?
177. Are there visible shadows in the image?
178. Does the image depict sunset or sunrise?
179. Is the image taken during golden hour?
180. Is someone dancing?
181. Is someone playing sports?
182. Is someone cooking?
183. Is someone cleaning?
184. Is someone fixing or repairing something?
185. Is someone driving?
186. Is someone hiking?
187. Is someone fishing?
188. Is the image black and white?
189. Is the image edited filtere??
190. Does the image use high contrast?
191. Is the image blurred?
192. Does the image have reflections?
193. Is the subject interacting with animals?
194. Is the subject interacting with machines?
195. Is the subject interacting with nature?
196. Is the subject interacting with others?
197. Is the subject interacting with water?
198. Is the subject wearing sunglasses?
199. Is the subject wearing a hat?
200. Is the subject wearing a uniform?
201. Is the subject wearing shoes?
202. Is the subject barefoot?
203. Is there a car visible?
204. Is there a bicycle visible?
205. Is there a bus visible?
206. Is there train visible?
207. Is there airplane visible?
208. Is there a boat visible?
209. Is there a motorcycle visible?
210. Is there symmetry the image?
211. Is there repetition or patterns in the image?
212. Does the image depict destruction ruins?
213. Are there visible tools or instruments?
214. Is there a stage or performance area visible?
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction state the main contributions of this article: the introduction an interpretable semantic bottleneck in a brain decoding pipeline, the use of GPT-4o and BLIP-2 to construct the interpretable space, and the demonstration that this approach enables voxel-Ievel insights with only minimal performance degradation. These claims are supported by the results presented in the article, including performance comparisons and voxel pattern alignment with known brain regions. Limitations and scope are also acknowledged in the "Introduction" and "Discussion" sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Limitations are addressed in a dedicated "Limitations" section within the Discussion. Potential privacy and ethical considerations are discussed in the "Privacy and Ethics Considerations' paragraph of the same section.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specificatiin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
25

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
This article does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All details the interpretable model, including data preprocessing and visual- ization procedures, are fully disclosed in the "Methods" section to ensure the reproducibility of results. Open access to the code also provided.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The complete code is publicly released, accompanied by detailed instructions to facilitate results reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (kttps://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ cac//pulic//ae mublic/guides:
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Standard train and test split for the Natural Scenes Dataset were used. This is described in the "Data" paragraph within the "Methods" section. The paragraphs titled "Reconstructing images from interpretable embeddings" and "Learning the map from brain activity to L", within the same section, describe the selection of regularization parameters and the number of cross-validns coossvalldationn folds used. Additional information provided regarding the Python libraries used. Full access to the source code is also made available.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
27

---

Answer: [Yes]
Justification: Whenever applicable, results are accompanied by error bars. The only case in which results show variability is the 50-way-tops Accuracy in Table 2 and standard deviations over 1000 repetitions, averaged over test set images, are provided.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Hardware specifications are provided at the end of the "Methods" section. Training and inference time details for all models are mentioned within the same paragraph.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research adheres completely to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
28

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The topic is addressed in paragraph "Privacy and Ethics considerations' of the "Discussion" " section. There are no anticipations of negative impact from this specific study. However, we emphasize the need for responsible use of brain data, especially as brain decoding models improve.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data released part of this work. The model is not anticipated to present significant risks of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
29

---

Answer: [Yes]
Justification: Creators of the Natural Scenes Dataset were explicitly mentioned in the "Methods" section, paragraph "Data". Usage of NSD complies fully with the terms of use. Original creators of the BrainDiffuser 41 model were properly cited and credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: A GitHub repository is provided with the code used in the experiments.
Although the code is not documented to the level of a standalone library, it includes basic instructions for reproducing the main results.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This research does not involve crowdsourcing and it does not involve research with human subjects. As such, there were no instructions, tasks, or compensation provided to participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
30

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No human participants were involved in this research. Therefore there were no risks, disclosures, or ethical review processes applicable to the study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [Yes]
The use of GPT-4o is described in the "Interpretable Space" paragraph of the "Methods" section, where it is employed to automatically generate a set of questions suitable for characterizing the Natural Scenes Dataset. In the same section, the of BLIP-2 is detailed for automatically answering the 214 questions for each image in the dataset.
Justification:
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
co/ /ttp://p......oooeeeee/////////0
31


Reviews:
[{"review_id": "ZHsNv17psA", "reviewer": "Reviewer_YAzb", "summary": "This paper presents a semantically-interpretable fMRI-> image decoder model by mapping the brain responses to a set of 214 binary questions (such as “Is someone swimming in the image?”). This not-so-latent 214-D space is then used as the target for a linear regression from the fMRI regions to the binary vector describing the image. In addition, a linear mapping from this space to CLIP embeddings is trained (using a pre-trained Brain Diffuser model), so that the brain-to-image mapping is via this human-interpretable space. This bridges the gap between latent spaces and human concepts, making the interpretation of brain activations more semantic. In particular, they show that the linear regression weights from relevant questions map onto brain regions known to represent bodies, faces, places, and words.\n\nFinally, they show that the reconstruction quality of the images is only somewhat reduced from using a more opaque latent space.", "questions": "Did you do any manual evaluation of the performance of BLIP on your questions? It seems like you should select a random sample of question-answer pairs to estimate the performance, as this could impact your results.\n\nWhy didn’t you use logistic regression to predict the semantic features? You’re predicting a binary variable, so it seems like logistic regression is much more appropriate.\n\nSome of the questions presuppose a person in the image (such as questions 34-35, 77-94, 180-187, 193-202, etc.), some more than others (e.g., is the subject barefoot). Did these also correspond to face and/or body areas? \n\nWhy did you only look at one question with respect to face areas?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Quality\n+ the quality of the reconstructed images is still quite good compared to the original Brain Diffuser results.\n+ The idea of using a semantic space is an interesting innovation.\n\nClarity\n\n+ The model is well-described.\n- It was unclear to me exactly how the questions and answers were constructed using GPT-o. What was the prompt used? \n- can you explain what the 2-way accuracy in Alexnet latent space refers to? Similarly, 2-way and 50-way in the following sentences.\n\nThere are a bunch of places where the English could be improved.\nEverywhere in the ms: “performances” should be replaced by “performance”. There are a few specific ones that need further adjustment, listed below.\nline 52: dataset . -> dataset.\nline 60: wouldn’t allow -> doesn’t allow\nline 69: performances -> performance (and everywhere else this word is used).\nline 85: performances -> performance by\nline 93: a LLM -> an LLM\nline 136: to condition -> conditioning\nline 165: with in mind the idea -> with the idea in mind \nline 166: reduce -> reduces\nline 167: from previous -> from a previous\nline 203: models -> model\nlines 204-205: “The extraction of interpretable latent embeddings with BLIP2 took approximately 24 hours per subject” Why per subject? Didn’t all subjects see the same images? \nline 214: reconstructing performances -> reconstruction performance\nline 216-217: reconstructing accuracies -> reconstruction accuracy.\nline 297: the training data nature -> the nature of the training data\nline 304-305: \nMoreover, the found category-selective voxel patterns overlap with canonical hotspots \n->\nMoreover, the category-selective voxel patterns we found overlap with canonical hotspots\n\nSignificance\n\n+ This is a nice way to extract semantics from brain images. But have you compared this to the stuff Tom Mitchell did back in the day of training models to answer simply semantic questions from brain images? There might be better questions in his work.\n\nOriginality\n\n+ I don’t know (besides Tom Mitchell’s work) of anyone else who has done this.", "was_revised": true, "final_justification": "This is an exciting paper, and gives very good results. I have read the authors' response to my review, and they have clarified all of my concerns.", "rebuttal": "We thank the reviewer for their valuable comments and insights. We proceed to clarify the points raised in the following and in the revised manuscript. \n\n**Question/answer pair generation:**\nWe thank the reviewer for pointing this out and appreciate the opportunity to clarify.\nFor each subject in the Natural Scenes Dataset (NSD), we aimed to generate a compact, interpretable representation of each image. To do so, we first extracted the first COCO caption associated with each image in the NSD training and test set. As a reminder, these captions were originally created by human annotators and are part of the COCO dataset, which underlies NSD.\nWe collected all these captions and submitted them to GPT-4o with the following prompt:\n“Given the following set of captions, each representing an image, generate a set of 256 binary questions that are suitable to well describe the content of the images. Generate questions that are diverse and non-overlapping, and that describe as completely as possible all the images. These should maximize your information about the image content.”\nThe set generated by GPT was not perfect; we manually inspected all questions to remove duplicates and semantically overlapping entries. This resulted in a final set of $214$ unique binary questions.\nThese questions were then posed sequentially to the BLIP-2 vision-language model for each image in the dataset with the following question-specific prompt: “Question: [one question, for example ‘is the subject smiling?’] Answer:”.\nFor every image, the model was queried 214 times, once per question, producing a binary (“Yes”/“No”) answer for each. The answers were converted to binary values (1 for “Yes”, 0 for “No”), resulting in a 214-dimensional interpretable vector representation for each image.\nThis procedure was repeated independently for each subject in NSD, because in the original experiment images of the training sets were different across subjects, whereas the test images were shared by all subjects.\n\n**Clarifications on the used metrics:**\nRegarding the 2‑way and 50‑way accuracy metrics: these metrics evaluate how well reconstructed images preserve the semantic content of the original images using ViT, a pretrained Vision Transformer, as a stand-in for human perception. For each original image, the model predicts its most likely class (serving as a pseudo ground truth). We then classify the reconstructed version of the same image with the same model to obtain its class probabilities. Instead of evaluating accuracy over all ImageNet classes, we create smaller, simulated classification tasks. In one case, we compare the true class against just one randomly chosen incorrect class (2‑way accuracy); in another, we compare it against 49 randomly chosen incorrect classes (50‑way accuracy). These tasks are repeated 1000 times with different randomly chosen distractor classes to estimate the mean and variability of the accuracy for each reconstructed image. Averaging across all images produces an overall measure of how much class‑level semantic information is preserved in the reconstructions. \nAlexNet(2) and AlexNet(5) report the same 2‑way identification protocol but carried out in feature space rather than label space. We extract activations from layer 2 or layer 5 of a pre‑trained AlexNet and evaluate whether the reconstruction is closer to its true partner than to a single randomly chosen distractor image. Repeating the distractor draw for all possible images provides a robust estimate of identification accuracy at two levels of visual abstraction: layer 2 emphasises low‑level edges/textures, whereas layer 5 captures object‑level semantics.\n\n**Questions:**\n\n**Did you do any manual evaluation of the performance of BLIP on your questions?**\nWe thank the reviewer for this very interesting and constructive suggestion. We agree that manually evaluating the performance of BLIP on our question set would provide valuable insights, as the final results are indeed influenced by the choice of the underlying VQA model. While we have not performed such an analysis yet, we plan on including it in the revised manuscript, if accepted, for a subset of question–answer pairs (5\\% of the test set images selected randomly) to provide the readers with an estimate of its accuracy. Furthermore, we will explicitly acknowledge in the limitations section that the quality of the interpretable space is impacted by the VQA model. Improving the quality of the interpretable space, both from the question design point of view and on the VQA model used, is an important future direction. \n\n**Why didn’t you use logistic regression to predict the semantic features?** \nWe agree that logistic regression seems like a natural choice for this type of problem; in the initial stages of this study we indeed experimented with predicting semantic features using logistic regression. However, when we compared its performance with that of Ridge regression trained to predict the standardized interpretable vectors directly, we found that the Ridge approach gave better results. Specifically, after thresholding, the linear regression model achieved a mean label accuracy of $0.83$, an F1 micro score of $0.78$, and a macro F1 score of $0.47$, with a Hamming loss of $0.17$. Its ROC-AUC was $0.91$ (micro) and $0.77$ (macro). In contrast, logistic regression performed worse, with a mean label accuracy of $0.63$, an F1 micro score of $0.56$, and a macro F1 score of $0.49$, along with a higher Hamming loss of $0.37$ and lower ROC-AUC values of $0.68$ (micro) and $0.71$ (macro). \n\n**Some of the questions presuppose a person in the image (such as questions 34-35, 77-94, 180-187, 193-202, etc.), some more than others (e.g., is the subject barefoot). Did these also correspond to face and/or body areas?**\nAll the questions mentioned by the reviewer exhibited activation patterns that matched both body and face selective areas. It is important to underline that we do not expect a sharp distinction in the activations when referring to face and body. Neuroscientific studies have shown that face- and body-selective areas are anatomically close in the human (and monkey) high-level visual cortex and often co-activate when a whole person or animal is present (see for example Taubert et al., 2022). We cite directly from Taubert et. al: “The vast majority of research revealing the functional profiles of face and body regions, as well as their location, have used static images or dynamic stimuli in which the face and body are cropped separately from each other. This act of disassembling whole people or animals in this way is a peculiar practice that is unique to the study of body parts compared with other objects. As emphasized in the introduction, under normal circumstances faces and bodies are seen together.”\nOur findings are consistent with this: for many of the questions related to bodies, we observed overlap with both face and body regions. For example, in Figure 4 of the submitted manuscript, the questions “Is the subject sitting?” and “Is the subject running?” show strong overlap with body-selective regions but also notable overlap with face areas. In the revised manuscript, if accepted, we will include a broader set of activation maps to better illustrate this phenomenon, as well as a more detailed explanation of the overlapping body/face activations. \n\nTaubert J, Ritchie JB, Ungerleider LG, Baker CI. One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system. Brain Struct Funct. 2022 May;227(4):1423-1438. doi: 10.1007/s00429-021-02420-7. Epub 2021 Nov 18. PMID: 34792643.\n\n**Why did you only look at one question with respect to face areas?**\nThis point is closely related to the previous question. We will include a wider set of activation maps in the revised manuscript, if accepted. We included only one question because most of the questions presupposing a subject referred to it in its entirety., while relatively few questions specifically targeted the face. One of these face-focused questions was the one we included in Figure 4 (‘Is the subject smiling?’) but other examples are ‘Is the subject wearing a hat?’ and ‘Is the main subject looking directly at the camera?’.\n\nFinally, we thank the reviewer for mentioning Tom Mitchell’s work. We are familiar with some of his studies (Mitchell T. et al., 2004), which we also find extremely interesting and related in spirit. However, the experimental paradigm in that study differs substantially from ours. In Mitchell et al. (2004), participants in a controlled setting were explicitly engaged in tasks directly tied to the output categories, for example deciding whether a visually presented word belonged to one of a small set of predefined semantic categories. In contrast, our study focuses on naturalistic image viewing: participants passively viewed complex real-world scenes while performing a continuous recognition task, reporting if they had already seen an image during the experiment. This task is unrelated to any particular semantic feature of the scene. Despite this difference, we similarly train models to answer a broad range of  visual questions directly from fMRI. Another distinction is that Mitchell et al. (2004) used feature selection to isolate predictive voxels, whereas we train a linear model directly and interpret voxel weights for each question without explicit voxel selection. To the best of our knowledge, we are not aware of work from Mitchell’s group using semantically unconstrained visual questions. We would be happy to learn of any specific studies or question sets from Mitchell’s group that we may have overlooked, as they could provide useful context or additional comparisons for our study.\n\nMitchell, T.M., Hutchinson, R., Niculescu, R.S. et al. Learning to Decode Cognitive States from Brain Images. Machine Learning 57, 145–175 (2004). https://doi.org/10.1023/B:MACH.0000035475.85309.1b"}, {"review_id": "7YdABODTc0", "reviewer": "Reviewer_zVsQ", "summary": "This paper introduces a novel framework for interpretable visual decoding from fMRI data. The central idea is to address the \"black box\" nature of high-performing brain decoding models by inserting a human-readable \"semantic bottleneck\" into the existing BrainDiffuser pipeline. The authors factor the complex mapping from fMRI voxels to CLIP embeddings into two simpler, linear stages: a first mapping from brain activity to a 214-dimensional interpretable concept space, and a second mapping from this concept space to the CLIP embeddings. The interpretable space is constructed from a set of binary questions about image content (e.g., \"Is there a person?\"). The key contribution is that the first linear mapping provides direct, voxel-level insights into how semantic concepts are represented in the brain, with only a minor trade-off in final image reconstruction quality.", "questions": "- To improve clarity, would the authors consider revising Figure 2 to explicitly separate the training procedures for models A and B from the final, end-to-end inference pipeline? This would help readers better understand the elegant two-stage data flow.\n- The current framework seems to require retraining the Concepts -> CLIP model (B) from scratch if the interpretable space is expanded. Have the authors considered more efficient update strategies, such as fine-tuning or modular approaches, to add new concepts without complete retraining?\n- The original BrainDiffuser uses a structural stream to generate an \"initial guess.\" It's not entirely clear from the text how this structural component interacts with the new interpretable semantic stream. Does the Int Brain Diff model still use the VDVAE-based initial guess, or does it rely solely on the CLIP embeddings derived from the interpretable space? Clarifying this would help in understanding the source of the performance drop in low-level metrics.\n- The interpretable space was set to 214 dimensions based on \"manual inspection\" (Section 2, \"Interpretable space\"). Could the authors elaborate on the criteria used for this selection and whether they explored how varying the number of concepts affects the trade-off between reconstruction quality and the quality of the neuroscientific insights?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n**Elegant Solution to a Key Problem:** The paper addresses the critical trade-off between performance and interpretability in brain decoding. The proposed semantic bottleneck is a simple yet powerful method for gaining neuroscientific insights without a major loss in decoding accuracy.\n\n**Novel Method for Creating Interpretable Space:** The use of LLMs (GPT-4o and BLIP-2) to automatically generate a human-readable semantic space is a scalable and innovative approach that moves beyond hand-crafted features (Section 2, \"Interpretable space\").\nStrong Neuroscientific Validation: The primary claim of interpretability is well-supported by the results in Figure 4. The alignment of the learned concept maps with canonical Regions of Interest (faces, places, bodies, words) provides compelling evidence that the model is capturing meaningful brain representations.\n\n**Thorough Quantitative Evaluation:** The authors provide a comprehensive comparison against the BrainDiffuser baseline using a wide range of low-level and high-level metrics (Tables 1 and 2). The inclusion of the \"Int to CLIP\" control model is a clever experimental design choice that effectively demonstrates the semantic richness of the interpretable space itself.\nReveals Deeper Brain Representation Patterns: The finding that activation patterns within a region vary for sub-concepts (e.g., running vs. dancing in the 'bodies' ROI, Figure 4) offers a nuanced insight into how the brain encodes information, supporting the theory of distributed representations.\n\n**Weaknesses and Possible Improvements**\n\n**Clarity of the Model Diagram (Figure 2):** The manuscript would benefit significantly from a revision of Figure 2. The current figure ambiguously merges the training process for the Concepts -> CLIP model (B) with the final inference pipeline. This makes it difficult to understand that the top and bottom rows represent distinct processes, how models A and B are trained with their respective targets, and that model B is frozen and reused. A clearer diagram separating these stages would greatly improve the paper's accessibility.\n\n**Scalability of the Interpretable Space:** The current framework requires retraining the Concepts -> CLIP model (B) from scratch if the set of questions is expanded. This is a practical limitation for future research aiming to build on this work with more granular concepts. The discussion could be strengthened by acknowledging this and suggesting potential avenues for more efficient updates.\n\n**Limited Discussion on Structural Information:** The paper notes that low-level reconstruction metrics see a larger performance drop (Table 1) and that the question set might not capture the \"whole spectrum of features.\" The work is heavily focused on semantic concepts. An improvement would be to discuss more concretely how structural concepts (e.g., object layout, scene geometry)—a key component of the original BrainDiffuser's success via its VDVAE stream—could be integrated into this interpretable framework.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. Below, we address the points raised under Weaknesses and provide point-by-point responses to the reviewer’s questions.\n\n**Clarity of Figure 2**\nWe agree that the current version of Figure 2 may be unclear about the distinct training procedures for models A and B with the final inference pipeline. We appreciate the reviewer highlighting this concern. In response, we have designed a revised version of the figure that clearly separates the training stages (for the Brain → Concepts and Concepts → CLIP mappings) from the final end-to-end decoding process. The updated figure segregates (i) Brain → Concepts training, (ii) Concepts → CLIP alignment, and (iii) inference—using colour‑coded backgrounds so the ‘two B blocks’ can no longer be confused. The new figure disentangles the role of each component and clarifies that model B is frozen after training and reused in inference. While we are unable to include the revised figure during the rebuttal phase due to formatting constraints, we will incorporate this improvement in the final version of the paper, pending acceptance.\n\n**Scalability of the Interpretable Space**\nWe appreciate this important observation regarding the scalability and extensibility of the semantic concept space. Currently, the Concepts → CLIP mapping (model B) must be retrained from scratch if the dimensionality of the concept space changes. This is a limitation of the current design, which we will explicitly discuss in the revised manuscript. (Please ssee reply to question 2 below)\n\n**Limited Representation of Structural Information**\nWe agree that our current interpretable space is predominantly semantically driven and lacks an adequate representation of structural or spatial features, which are critical for low-level image fidelity. As the reviewer correctly points out and as we touched upon in the discussion this likely contributes to the greater performance drop in low-level reconstruction metrics compared to high-level semantic alignment.\nThis limitation is mainly due to two factors: (i) our question-generation pipeline relies on LLM-based summarization of human annotators captions; these tend to emphasize semantic content over spatial layout or geometry and (ii) formulating fine-grained structural questions in a generalizable and scalable way is nontrivial, especially with natural images that lack consistent object placement (see reply 2 and 3). We will include these comments in the discussion of the revised manuscript, if accepted.\n\n**Questions:**\n\n1) As noted above, we have revised Figure 2 to separate training and inference stages, clearly indicating the training flow for models A and B and the reuse of model B during inference. \n\n2) We thank the reviewer for raising this important point regarding the extensibility of the interpretable concept space. In the current formulation, expanding or modifying the set of questions (semantic concepts) requires retraining the Concepts → CLIP mapping (model B) due to changes in input dimensionality. This presents a practical limitation for researchers who may wish to refine or scale the semantic bottleneck for different experimental goals or datasets.\nTo this end, our framework was intentionally designed to be simple and modular, with both mappings - A and B - implemented as linear ridge regressions. This allows for interpretability and analytical tractability but currently lacks built-in flexibility for dynamic updates to the concept space.\nNonetheless, we believe the design does admit potential extensions that could make it more scalable:\n- For applications where only a subset of concepts is needed (e.g., to test minimal sets for decoding or neuroscientific interpretability), model $A$ can be trivially subsetted by selecting the corresponding rows of its output. However, model $B$ would need to be retrained to project from this reduced concept space to the CLIP embedding space.\n- On the other hand, additional questions could be introduced by training a new regressor (model $A_2$) to predict only the added concept dimensions from brain activity. The outputs of model $A$ and $A_2$ could then be concatenated to form an augmented brain-to-concept representation (model $A^*$). However, model B must still be retrained to accommodate the new input dimensionality. In principle, this retraining could be implemented as fine-tuning (rather than from scratch), assuming the new concepts form a small perturbation of the original representation. \n- Another avenue for future work could be  to explore the pretraining model $B$ on a much larger, overcomplete set of interpretable concepts and later sparsifying it for specific use cases.\nWe will elaborate on these ideas in the revised manuscript to better inform future work on the composability and scalability of interpretable decoders.\n\n3) We appreciate the reviewer’s observation on the limited representation of structural information in our interpretable concept space. This limitation is acknowledged and contributes to the reduced performance on low-level reconstruction metrics (e.g., MSE, SSIM) compared to high-level semantic alignment. Although our framework retains the structural stream from the original BrainDiffuser (based on VDVAE), the semantic bottleneck remains biased toward high-level content.\nThe bias toward semantic content in our interpretable space stems from its construction: questions are automatically generated from human-written COCO captions using BLIP-2 and GPT-4o, which typically highlight object presence rather than spatial arrangement. Natural language favors concise descriptions of what is in the image over how elements are positioned, making structural features underrepresented. Additionally, questions related to structural features of the image are harder to formulate at scale, as they require more complex, context-dependent descriptions of object relationships, which are rarely found in the source captions.\nIn future work, we aim to address this limitation by enriching the dataset with spatially detailed captions (e.g., via re-captioning with multimodal LLMs) and regenerating the question set, or by explicitly designing structural questions. While our main goal was to demonstrate the feasibility of a semantic bottleneck for interpretable brain decoding, we agree that incorporating structural information remains a key direction for future research.\n\n4) About the question regarding the rationale behind the choice of $214$ dimensions for the interpretable space: the decision was guided by a combination of practical and conceptual considerations. First, from the perspective of prior literature in cognitive neuroscience and language-based encoding models [1], relatively low-dimensional semantic feature spaces - often ranging from 30 to 100 dimensions - have been shown to capture a substantial portion of the explainable variance in neural responses. These findings suggested that high decoding performance does not necessarily require an extremely high-dimensional representation, especially when the features are carefully chosen for their informativeness and relevance.\nSecond, follow-ups [2] the original BrainDiffuser framework demonstrated strong performance using low-rank approximations of the CLIP embedding space, often in the range of 100 to 200 dimensions. This indicated that a large fraction of the meaningful visual-semantic information could be retained even after significant dimensionality reduction. We therefore hypothesized that a semantically curated bottleneck in a similar dimensional range would be sufficient to preserve downstream decoding quality, while offering the added benefit of interpretability.\nThird, and perhaps most importantly, our decision was informed by manual inspection of the candidate questions generated through our automated BLIP-2 + GPT-4o pipeline. After running the question-generation pipeline on the image captions, we selected a set of $214$ binary questions that appeared semantically diverse and representative of common image content categories. This process was not performance-optimized; our goal was to create a broad, human-readable semantic space to test the framework’s feasibility in a realistic, scalable setting with automated rather than hand-tuned concept generation. We agree with the reviewer that exploring the effect of concept space dimensionality on both reconstruction quality and neuroscientific insight is a rich direction for future research. For instance, one might ask: how many questions are necessary to achieve a particular level of decoding performance? Is there a minimal concept set that still enables interpretable decoding above chance? Conversely, how much performance can be recovered by progressively increasing the number of concepts, and is it possible to match or even exceed the vanilla BrainDiffuser performance using only interpretable representations?\nWhile we are eager to investigate these questions, doing so at scale was beyond the scope of the current study due to computational constraints. Each configuration requires regenerating question-answer pairs using BLIP-2 across the full training set- a process that currently takes several days on several high-end H100 GPUs - and retraining both $A$ and $B$ mappings. We will discuss these limitations and opportunities in the revised manuscript to better contextualize the current design choices and motivate future extensions of the framework.\n\n\n\n[1] Benara V, Singh C, Morris JX, Antonello RJ, Stoica I, Huth AG, Gao J. Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions. Adv Neural Inf Process Syst. 2024;37:124137-124162. PMID: 40276238; PMCID: PMC12021422.\n\n[2] Mayo, D., Wang, C., Harbin, A., Alabdulkareem, A., Shaw, A., Katz, B., & Barbu, A. (2024). BrainBits: How Much of the Brain are Generative Reconstruction Methods Using? In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Main Conference Track."}, {"review_id": "TcDuJpoAP0", "reviewer": "Reviewer_mU3X", "summary": "This work proposes an interpretable framework to decode images from fMRI signals. They add a semantic bottleneck to an existing brain-decoding model (braindiffuser). This semantic bottleneck is basically a binary answer to 214 different questions related to images, which are generated by  LLM using the caption of each image. The authors evaluated the reconstruction performance of this model and also visualized interpretable activation maps. The reconstruction results don't get improved but the idea is claimed to be one step towards interpretable brain decoding models.", "questions": "* How did you choose the top 4% of most influential voxels? What do you define as the most influential voxels?  \n* Can you provide any more insights on your results in interpretability? Elaborate on why you think your method is actually capturing semantic information. Can you provide any quantitative metric for that?  \n* The motivation behind choosing Braindiffuser is not clear to me. Can you elaborate on that?  \n* In Figure 2, the ridge model (B) is not clear to me. Why do you have two B models? The first row seems to be confusing.(Also, aesthetics can get improved here)\n* Can you compare your results regarding interpretability with any existing method?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n* The paper is well-written and easy to follow.  \n* The proposed idea is interesting.  \n* It seems that a good amount of detail is provided on the method.\n\n**Weaknesses:**\n\n* The biggest flaw of the paper, in my opinion, is that although interpretability is the most important part of the paper, evaluations regarding that are rather limited and unconvincing. I would expect some sort of structured, summarized, and quantitative analysis on interpretable results besides the provided activation maps.  The provided activation maps for a subset of questions do not provide that much information. I have a hard time believing the method is actually interpretable without any aggregated quantitative results.\n* The discussion on the results of interpretability is limited. Authors can provide better insight on that.\n* The quality of schematic figures can be improved.\n* There is no comparison with any other interpretable methods.", "was_revised": true, "final_justification": "The authors significantly addressed my concerns and questions, and I'm willing to raise my score from 2->4.", "rebuttal": "We thank the reviewer for their comments and efforts towards improving our manuscript.\n\n**Weakness:**\nWe thank the reviewer for highlighting this important point. We agree that interpretability is a central contribution of our work, and we acknowledge that the initial version of the manuscript emphasized qualitative results - particularly activation maps - over structured quantitative evaluations. This was motivated in part by the exploratory and conceptual nature of this work, where we aimed to demonstrate the feasibility of incorporating a semantic bottleneck into a full decoding pipeline. However, we fully agree that strengthening the quantitative analysis of interpretability would make the contribution more convincing.\n\nIt is also worth noting that many interpretability methods in deep learning - such as attention visualizations, saliency maps, or Grad-CAM - are primarily evaluated qualitatively, often by inspecting whether visualizations align with expected input regions. Our work shares a similar spirit, but with a critical difference: our goal is not just to interpret a model’s output, but to interpret brain activity itself, through the lens of semantically meaningful intermediate representations. This makes interpretability not just a feature of the method, but the core objective.\n\nTo better support our claims, we conducted a series of new analyses that will be included in the revised version:\n\n**ROI-based Dice Score Analysis:** \nWe computed the Dice score between thresholded activation maps for concept-specific questions (e.g., body-related) and the corresponding functional ROIs (e.g., “bodies” ROI from NSD). Across multiple thresholds (50th, 75th, 80th, and 90th percentiles), we consistently observed that concepts like “body”, “face”, “place”, and “word” aligned more strongly with their expected ROIs than with unrelated ones, across all subjects.\n\n**ROI-wise activation decoding:** \nInspired by the original BrainDiffuser analysis, we averaged activation maps across semantically grouped questions (e.g., all “face” questions), binarized them within NSD ROIs, and decoded them using the standard BrainDiffuser pipeline. The resulting reconstructions were qualitatively consistent with the original concept category, providing further evidence that the interpretable space captures meaningful structure.\n\n**Activation maximization analysis:**\nFor each question, we identified test-set samples that produced the highest activation in the corresponding semantic dimension (via the trained Brain → Concept model). We asked three human raters to assess whether the associated images reflected the target concept. A success was recorded if at least 6 of the top 10 samples matched the intended concept. Across concepts and subjects, we observed a strong match rate ($67$%), supporting the idea that each dimension of the semantic space reliably corresponds to distinct visual categories.\n\n**Accuracy of answers in interpretable space:**\nWe evaluated the accuracy of the predicted answers in the interpretable space by comparing them to ground-truth annotations derived from BLIP-2’s binary responses. This allowed us to assess the fidelity of semantic decoding independently of the final image reconstruction. (Please see our detailed response to Question 4 for results.)\n\nWe will integrate these new analyses and expand the discussion accordingly in the final version of the paper, if accepted. We hope these additions address the reviewer’s concerns and help clarify the genuine interpretability enabled by our framework.\n\n**Questions:**\n\n1) Thank you for the question. The decision to focus on the top $4\\%$ of most influential voxels was made empirically, guided by two considerations. First, this threshold yields a voxel count that is comparable to that of well-characterized functional regions in the dataset, such as face-, body-, place-, and word-selective areas, each comprising a similar number of voxels ([2]). Second, this proportion provides a clear and interpretable visualization, which is particularly important given that our analysis is qualitative in nature. Importantly, our goal here is not to make strong quantitative claims about voxel importance but rather to offer intuitive insights into how different semantic concepts may be represented across the brain. Prior work in decoding and cognitive neuroscience suggests that while some high-level categories (e.g., faces, places) can be decoded from localized clusters, more fine-grained or abstract concepts often involve distributed representations (for example see ROI-wise experiment in Brain-Diffuser or BrainDive papers) and that’s why decoders benefit from large high dimensional voxel count in visual cortex. By visualizing the most influential voxels per concept, we aim to illustrate these patterns and highlight their correspondence with known functional regions. We will clarify this rationale more explicitly in the revised manuscript, if accepted.\n\n2) We thank the reviewer for pointing out the need for greater clarity in the schematic figures. We agree that the original Figure 2 may not sufficiently differentiate between the distinct training phases of models A and B and their integration during inference. In response, we have redesigned this figure to more clearly illustrate the separation between the two training stages - (1) Brain → Concepts and (2) Concepts → CLIP - and the final inference pipeline. The revised schematic emphasizes that model $B$ is trained independently and frozen before being reused at inference time. Additionally, we improved Figure 1 to depict the full decoding pipeline more explicitly, making the overall flow from brain activity to image reconstruction easier to follow. Due to formatting constraints during the rebuttal phase, we are unable to include the updated figures here, but we will incorporate both revisions in the final version of the paper, pending acceptance.\n\n3) Thank you for the question. We chose BrainDiffuser because of its strong performance, modular architecture, and simplicity of implementation, being basically a set of Ridge Regression between brain data and CLIP and VDVAE embeddings. Its clarity and flexibility made it well-suited for integrating our interpretable semantic bottleneck without introducing additional confounds. Our framework is architecture-agnostic: the semantic bottleneck is inserted before the CLIP embedding stage, which is common to most recent brain-to-image pipelines. We will clarify this rationale more explicitly in the revised manuscript. Another reason is that it was recently demonstrated that decoders based on linear architectures better generalize across different tasks, for example being able to decode imagery vision tasks [1] when trained on the same subjects, generalizing better than more complex and non-linear architectures.\n\n4)  To assess the quality of the predictions in the interpretable concept space, we compared the model’s outputs against binary labels derived from BLIP-2 responses on the same set of images. Specifically, we thresholded the continuous outputs of the Brain → Concept model and computed several standard classification metrics across all $214$ questions. Our results are as follows:  Mean Label Accuracy: $0.83$, F1 Score (Micro): $0.78$, F1 Score (Macro): $0.47$, ROC-AUC (Micro): $0.91$, ROC-AUC (Macro):  $0.77$. These results indicate that the model is able to accurately decode high-level semantic attributes from brain activity, achieving strong overall performance in both classification accuracy and discriminability. The lower macro F1 score reflects the varying difficulty across concepts, as some binary questions are inherently more ambiguous or less visually salient. Nevertheless, this evaluation confirms that the interpretable space reliably captures meaningful semantic structure. We will include these results and clarify their implications in the revised manuscript, if accepted.\n\n5) To the best of our knowledge, this is the first complete brain decoding pipeline that introduces a truly interpretable intermediate representation. While post-hoc explainability methods - such as Grad-CAM, SHAP, or attention-based saliency - can technically be applied to existing models, these techniques generally rely on the assumption that the model’s output space is inherently interpretable. In the context of brain decoding, this is often not the case: most models operate between two abstract spaces - neural activity and CLIP-like embeddings - making it challenging to extract meaningful insights from such explanations. This limitation motivated the design of our semantic bottleneck. By explicitly structuring the intermediate representation around human-interpretable concepts, our framework provides a more transparent link between brain activity and visual content. We believe this principled integration of interpretability into the core model architecture offers a more robust and actionable understanding than standard post-hoc techniques, and we will highlight this more clearly in the revised manuscript.\n\nIt's important to note that the scientific discourse around interpretability in brain decoding models is still developing and our work aims to contribute to this conversation by proposing a simple and human-readable approach. While we acknowledge the limitations and are working to address them through additional analyses, we would kindly ask the reviewer: are there specific improvements or additions that, in your view, would make this work sufficiently valuable to be part of the ongoing scientific dialogue?\nWe see this paper as a starting point rather than a final answer, and we would be grateful for your perspective on what would elevate the contribution further.\n\n[1] Kneeland R et al., 2025. NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery. \n[2] Allen, E.J. et al., 2022 A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence."}, {"review_id": "ROl2NG6SFt", "reviewer": "Reviewer_9695", "summary": "This paper proposes an interpretable fMRI-based visual decoding framework that integrates a semantic bottleneck into the established BrainDiffuser pipeline. The bottleneck consists of a 214-dimensional interpretable space, where each dimension corresponds to a yes/no question about the image (e.g., “Is there a person?”, “Is it outdoors?”). The authors demonstrate that the proposed model maintains competitive decoding accuracy. The voxel-wise contribution to semantic dimensions is visualized and aligns well with canonical category-selective regions in the visual cortex.", "questions": "1. Have the authors considered removing the reliance on pre-defined binary labels and instead allowing the model to directly generate human-readable explanations via a large language model (LLM)? Such an approach could potentially bypass the human bottleneck of manually crafting or filtering semantic dimensions, and allow for more flexible and context-aware interpretability.\n\n2. From a machine learning perspective, the structure of a network optimized for interpretability may differ significantly from one optimized purely for performance. In this case, the original BrainDiffuser model is essentially being “reshaped” to fit the interpretability constraints—possibly at the cost of performance, as the observed degradation suggests. This raises a broader concern: are we truly interpreting how high-performing models behave, or are we just interpreting a specially trained, interpretable variant? Is there any potential approach to interpret existing high-performing models without retraining them specifically for interpretability?\n\n3. Is it possible to train an image generator conditioned directly on the semantic bottleneck without using CLIP as an intermediate space? Could this allow a fully interpretable path: brain to concepts (bottleneck) to image?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Introducing a semantic bottleneck between fMRI and CLIP embeddings is a  feasible and concise way for interpretable decoding of brain activity. Although it cannot cover all perspectives, it can provide a certain degree of explanation at the semantic level that people are more concerned about.\n2. From an experimental standpoint, the network maintains performance even with the added interpretability constraint, with only a minor drop in accuracy. Another commendable aspect is the inclusion of an upper-bound reference using ground-truth semantic labels (Int-to-CLIP). This model achieves better high-level semantic reconstruction, which is expected and provides a meaningful benchmark to evaluate the effectiveness of the proposed interpretable space. Beside that, authors also provide all the codes.\n3. The authors provide brain region activation maps that support neuroscience validity. It can be seen that the voxel regions that support network semantic discrimination are consistent with the relevant regions considered in neuroscience.\n\n\nWeaknesses\n1. As the authors mentioned, the current interpretability relies on manually defined semantic information, specifically in the form of binary yes/no questions. This design inevitably omits many other potentially informative features, such as color, spatial layout, or fine-grained scene attributes. As a result, the scope of interpretation is limited, and the framework may miss more detailed or nuanced insights.\n2. While the proposed methodology and research objective are indeed novel, the paper lacks a concrete articulation of the broader scientific significance. The current claims regarding the benefits of interpretability remain fairly general—for example, suggesting that the model \"advances neuroscientific insight\"—but these insights appear to align with already well-established findings in neuroscience. The added value of the proposed approach in uncovering new neural representations remains unclear.\n3. The study compares the proposed model only against a single baseline (BrainDiffuser) with a relatively fixed architecture and training regime. This limited experimental design makes it difficult to assess the generalizability of the method. A broader comparison with alternative architectures or decoding paradigms would help strengthen the case for robustness and practical applicability.", "was_revised": true, "final_justification": "The authors addressed most of my concerns. The paper is overall good. I maintain my original rating.", "rebuttal": "We thank the reviewer for the insightful comments.\n\n**Weakness 1:**\nWe thank the reviewer for highlighting the limited scope of interpretable features in our semantic bottleneck. Our framework relies on binary questions automatically generated from COCO captions using BLIP-2 and GPT-4o. These captions focus mainly on object presence rather than spatial layout or fine-grained attributes, so our interpretable space reflects this bias.\n\nTo ensure scalability across the large NSD dataset, we prioritized automated over manual generation. While effective for semantic coverage, this omits key perceptual dimensions. Extending the concept space to include structural and visual attributes could improve interpretability but requires more detailed language or structured templates not easily derived from captions. We will note this limitation and highlight future work to incorporate such features for more nuanced analyses of brain representations, especially in early visual areas.\n\n**Weakness 2:**\nWe appreciate the reviewer’s concern regarding the articulation of broader scientific significance and the novelty of the neuroscientific insights derived from our model. Our primary aim in this work was to introduce a novel, interpretable framework for brain decoding to enable voxel-level understanding of semantic representations through an explicitly human-readable bottleneck. \n\nTo validate this framework, we deliberately focused on a simple validation: testing whether the concept activation maps recovered by the model align with well-established functional specializations in the brain (face, place, body, and word-selective regions). That our model successfully recovers these canonical regions using linear mappings is not intended as a novel neuroscientific discovery per se, but rather as a validation of the framework’s interpretability and fidelity.\n\nWe agree that the true potential of this approach lies in its ability to support more exploratory, fine-grained analyses such as identifying sub-regional distinctions within known ROIs or discovering previously overlooked patterns of representation. Indeed, our results in Figure 4 hint at this direction, as we observe meaningful variation in sub-concept activations (e.g., different bodily actions like “running” vs. “dancing” within the body-selective region), which may offer new insights into the distributed coding of semantic information in the cortex. \n\nTo illustrate this potential we performed exploratory analyses inspired by the approach of the BrainDiffuser authors (section 3.6 in [1]) as well as BrainDiVE ([2]). Specifically, we aggregated activation maps across semantically related questions (e.g., all “face”-related questions) to obtain binary masks and used these to generate image reconstructions through the BrainDiffuser decoding pipeline. These reconstructions qualitatively reflected the expected semantic content (e.g., bodies or faces), indicating that the interpretable space preserves meaningful structure. \n\nFurthermore, we performed activation maximization analyses, identifying test images that elicited the strongest responses along specific semantic dimensions and asking three human raters to verify their relevance. The high correspondence between targeted concepts and top-ranked images ($67\\%$) suggests that individual semantic dimensions are interpretable and consistent. While our aim here was to validate the approach rather than to make specific neuroscientific claims, these analyses demonstrate that this framework can, in principle, be used to probe cortical representations in new ways and potentially reveal novel insights.\nWe will revise the manuscript to clarify this distinction between validating the approach and the future potential of the method for generating new hypotheses about brain representations.\n\n[1] Ozcelik, F. et al., Natural scene reconstruction from fMRI signals using generative latent diffusion. Sci Rep 13, 15666 (2023). https://doi.org/10.1038/s41598-023-42891-8\n\n[2] Luo, A. F. et al., 2023 Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models. https://arxiv.org/abs/2306.03089\n\n**Weakness 3:** \nThank you for this valuable point. We agree that the current comparison is limited to BrainDiffuser. We selected this model due to its strong performance, architectural clarity, and modular design, which allowed for a straightforward integration of our interpretable bottleneck. While a broader evaluation across multiple decoding architectures would indeed strengthen claims of generalizability, it was beyond the scope of this initial study.\n\nImportantly, the proposed framework is architecture-agnostic: the semantic bottleneck operates prior to the final CLIP embedding prediction, making it compatible with any decoding pipeline that maps brain activity to CLIP or similar multimodal embeddings (which is common among almost all the decoding pipelines). The decomposition into models $A$ and $B$ is modular by design, and can be easily incorporated into alternative or more complex architectures in future work.\n\n**Question 1:**\nWe thank the reviewer for this interesting suggestion. The idea of leveraging a LLM to directly generate human-readable explanations from brain activity is indeed an intriguing and complementary direction. In some sense, this reverses our current approach: rather than constraining brain decoding through a predefined interpretable bottleneck, such a method would aim to decode free-form natural language directly.\n\nWhile prior work (references below) has explored LLM-based decoding of image captions or descriptive labels from neural data, such approaches tend to prioritize semantic fluency over interpretability. That is, while the outputs may appear meaningful or coherent, they often lack transparency in how specific neural features contribute to the generated explanation. In contrast, our framework explicitly exposes voxel-level contributions to discrete, interpretable concepts, making it possible to localize and visualize how different semantic dimensions are represented in the brain.\n\nThat said, we agree this is a promising future direction. For example, fine-tuning an LLM to accept fMRI-derived embeddings - rather than image embeddings - as input could enable flexible, interactive decoding, where attention over fMRI embeddings could provide some form of interpretability. However, this would require fundamentally different modeling assumptions and training regimes, which we feel lies beyond the scope of the present work. We will include this discussion in the revised manuscript to highlight it as a compelling avenue for future research.\n\nA minimal set of works targeting the problem in this way:\n\nQiu, W. et al., 2025 MindLLM: A subject-agnostic and versatile model for fMRI‑to‑text decoding. In Proceedings of the 41st International Conference on Machine Learning (ICML 2025) arXiv:2502.15786.\n\nMindLLM combines a neuroscience-informed encoder with a large language model to decode natural language from brain activity in a subject-agnostic way. Differently from our work, their output is a caption of the image derived from brain activity. \n\nHuang, W. (2024). BrainChat: Decoding semantic information from fMRI using vision–language pretrained models. arXiv:2406.07584\n\nBrainChat uses vision-language models like CoCa to generate image captions and answer questions directly from fMRI embeddings. Similarly, the output is a caption. \n\nYe Z. et al., Generative language reconstruction from brain recordings, Nature Communications (2025). DOI:10.1038/s42003-025-07731-7. \n\nThis study proposes a similar idea, but based on language, demonstrating how fMRI-derived embeddings can be fed into autoregressive LLMs to reconstruct continuous language, capturing fluent, context-aware text beyond candidate-based decoding. \n\nLuo AF et al., 2024 BrainDive: Uncovering and editing inner thoughts via fMRI-to-text decoding. arXiv:2312.00776\n\nBrainDive proposes a diffusion-based approach to generate images that maximize the predicted activity of specific brain regions, providing a form of inverse interpretability (you choose the region and get as output the image that is most likely to activate that region).\n\nDiffently from these and other work in the literature we target a specific problem: decoding images from brain activity in an interpretable way, by providing a minimal but effective modification to a robust decoding pipeline.\n\n**Question 2:**\nTo the best of our knowledge, this is the first full brain decoding pipeline that incorporates a truly interpretable intermediate representation. While post-hoc explainability methods (e.g., Grad-CAM, SHAP, attention-based saliency) could, in principle, be applied to existing high-performing models, they typically assume an output space that is itself interpretable. In brain decoding, however, models often map between two inherently abstract spaces - brain activity and CLIP embeddings - making such explanations difficult to interpret meaningfully. This is precisely why we designed our framework around an explicit semantic bottleneck: to enable interpretable mappings at both the input and output levels, rather than relying on opaque latent dimensions.\n\n**Question 3:**\nThank you for the interesting suggestion. In principle, it is indeed possible to fine-tune a diffusion model to condition directly on the semantic bottleneck, bypassing CLIP and enabling a fully interpretable end-to-end path from brain activity to concepts to images. However, in practice, this would require training the generator on a new set of conditioning vectors, and with the current dataset size, our previous experience in similar settings taught us that even efficient fine-tuning approaches like LoRA may not be sufficient to support high-quality image synthesis. For this reason, we considered such an extension beyond the scope of the present work, though we agree it represents an exciting direction for future research."}]

Author rebuttal:

Meta-review: In this work, the authors propose a method for interpretable visual decoding from fMRI data. Initially, some reviewers had some concerns over the paper and gave negative recommendations. Yet, the authors did a good job during the rebuttal and and the end all the reviewers have positive opinions on the paper. The AC read through the manuscript, all reviews, the discussion, and the rebuttal. The authors are highly encouraged to improve the paper quality according to the reviewers' feedback in the camera-ready version. The AC decided to accept this submission.
Decision: Accept (poster)