Neuro-Symbolic Decoding of Neural Activity
Authors: Yanchen Wang
Joy Hsu
Ehsan Adeli
Jiajun Wu
Venue: ICLR 2026 Poster (2026)
Keywords: neural decoding
concept grounding
neuro-symbolic learning
Primary area: applications to neuroscience & cognitive science
TL;DR: A neuro-symbolic framework for fMRI decoding with concept grounding.

Abstract: We propose NEURONA, a neuro-symbolic framework for fMRI decoding and concept grounding in neural activity. Leveraging image- and video-based fMRI question-answering datasets, NEURONA learns to decode interacting concepts from visual stimuli based on patterns of fMRI responses, integrating symbolic reasoning and compositional execution with fMRI grounding across brain regions. We demonstrate that incorporating structural priors (e.g., compositional predicate-argument dependencies between concepts) into the decoding process significantly improves both decoding accuracy over precise queries, and notably, generalization to unseen queries at test time. With NEURONA, we highlight neuro-symbolic frameworks as promising tools for understanding neural activity.

Published as a conference paper at ICLR 2026
NEURO-SYMBOLIC DECODING OF NEURAL ACTIVITY
Yanchen Wang"
Joy Hsu" Stanford University
Ehsan Adeli
Jiajun Wa
Columbia University
Stanford University
Stanford University
ABSTRACT
We propose NEURONA, a neuro-symbolic framework for fMRI decoding and concept grounding in neural activity. Leveraging image- and video-based fMRI prestion-ansswrrig datasets, NEURONA learns to decode interacting concepts from visual stimuli based on patterns of fMRI responses, integrating symbolic reasoning and compositional execution with fMRI grounding across brain regions. We demonstrate that incorporating structural priors (e.g., compositional predicate- argument dependencies between concepts) into the decoding process significantly improves both decoding accuracy over precise queries, and notably, generalization unseen queries test time. With NEURONA, we highlight neuro-symbolic frameworks as promising tools for understanding neural activity. Our code is available atttesss//ttaoo.coo////aaaaageee
INTRODUCTION
long-standing hypothesis in cognitive science, the Language of Thought (LoT) hypothesis (Fodor) 1975), proposes that human cognition operates over structured representations that compose sys- tematically. Rather than treating concepts as isolated units, many theories propose that the brain organizes knowledge into compositional structures- -ursssssscchhhe as predicates and their arguments- that enable flexible inference. In this work, we study whether explicitly modeling such structure can improve neural decoding from functional magnetic resonance imaging (fMRI), with the goal of predicting high-level semantic content from patterns of brain activity (Mitchell et al.) 2008). More broadly, we aim to assess whether incorporating structural priors yields decoding models that are more accurate, precise, and generalizable.
There has been extensive literature on semantic representation and concept decoding from fMRI in the past decades, with several influential works studying how concepts are organized and grounded across the cortex (Mitchell et al. 2008 Palatucei et al., 2009 Huth et al.) 2016 Pereira et al. 20 8). Recent advances in machine learning have further accelerated data-driven approaches to neural decoding. However, most large-scale fMRI decoding studies focus either isolated concepts or
holistic stimulus reconstruction (Nishimoto et al.] [011 Naselaris et al.) 2011 Chen et al.] 2023a
Takagi & Nishimoto 2023 Scotti al. 2023] Chen al. 2023b), leaving open central question: how can we decode high-level relational
between multiple visual concepts and the relations that bind them from neural responses? Concretely, we ask whether decoding predicate- driven concepts (e.g., holding) can be improved by explicitly accounting for their constituent arguments (e.g., person and baseball li-bat) t)) and their representations across brain regions.
To explore these questions, we leverage rich data from image- and video-based fMRI datasets, which naturally encode complex and compositional semantics. Naturalistic stimuli such as images and videos often involve multiple interacting concepts (e.g., person holding a baseball bat), making them well-suited for probing how to decode entities and their relations from neural activity. Hence, we propose challenging fMRI question answering (IMRI-QA QA) datasets derived from BOLD5000 (Chang ct al., 2019) and CNeuroMod (Gifford et al./ 2024; Boyle et al. 2023), pairing 1MRI recordings with structured queries about the corresponding visual stimuli (c.g.. "Is there a person holding baseball bat?"). Our BOLD5000-QA and CNeuroMod-QA datasets evaluate decoding of me-ggrined,, compositional visual semanties from fMRI responses.
"Equal contribution.
I Equal mentorship.

---

Published as a conference paper at ICLR 2026
We find that neither simple linear models nor purely end-to-end neural decoding models are sufficient for solving this task. Linear models lack the capacity to capture interactions between multiple interacting components Naselaris al. (201T). while large neural decoders (c.g., those with language model backbones) tend to encode stimuli holistically, without explicitly modeling modular concepts or their relations, leading coarse alignment between neural activity and language Takagi & Nishimoto (202)). To address these limitations, we propose a neuro-symbolic approach to IMRI-QA that integrates the compositionality of symbolic systems with the expressivity of neural networks: each query is mapped to symbolic expression that composes concepts, and brain activity is routed through corresponding concept modules (implemented as neural networks) to produce an answer. Crucially, we incorporate various structural priors into the model's decoding process, by defining candidate entity representations neural activity and specifying how predicate-level concepts should compose over these entity representations according to the symbolic expression. From this paradigm, we propose NEURONA, a NEURO- symbolic framework for decoding in Neural Activity, which integrate symbolic reasoning and compositional execution with IMRI grounding.
We evaluate NEURONA on BOLDS000-QA and CNeuroMod-Q2s and demonstrate that our neuro- symbolic framework significantly outperforms baseline neural decoding methods, and importantly, exhibits strong generalization to unseen test queries. Ablation studies further highlight the functional role of compositional structure: conditioning predicate modules the regions associated with their subject and object arguments consistently yields large performance gains in neural decoding. These priors guide the model recover high-level semantics conditioned on constituent entity groundings. showing that relational meaning is better predicted across multiple co-activated brain regions via its arguments, rather than localized to single region or multiple regions without guidance.
To summarize, the main contributions of this work are:
We introduce the BOLD5000-QA and CNeuroMod--Qs datasets for fMRI question answering, pairing fMRI responses with compositional queries derived from visual stimuli to emphasize decoding of fine-grained visual semantics.
We propose NEURONA as neuro-symbolic framework for neural decoding that integrates structural priors over concept composition with grounding in neural activity.
We show that NEURONA significantly outperforms strong neural decoding baselines and general- izes better to unseen compositional queries, and identify predicate-argument dependencies key driver of performance gains through targeted ablations.
RELATED WORKS
Visual decoding from fMRL. Reconstructing visual content from fMRT signals has become central research focus of works in the field, with many approaches leveraging state-of-the-ard generative backbones for the task, following early studies (Thirion et al., 2006 Miyawaki et al. 2008 Kay ct al. 2008 Nasclaris et al.) 2009: Nishimoto et al. 2011). Takagi et al. demonstrated that a pre- trained diffusion model can reconstruct high-resolution images from fMRI (Takagi & Nishimoto 2023). MinD-Vi Vis uses masked brain modeling with a diffusion model for semantically faithful image generation (Chen et al. 2023a). MindEye projects fMRI into a CLIP embedding space and applies diffusion prior for pixel-level synthesis (Scotti et al., 2023). Extending to video, MinDD-ideo and NeuroCLIPP incorporate spatiotemporal masked modeling and keyframe-pent keyframe--erreepioo flow cues, respectively, into diffusion-based reconstruction (Chen al. 2023b; Gong et al. (024).. These visual reconstruction works focus on recovering stimulus appearance from neural data; in contrast, rather than generating pixel-level images or videos, our work targets decoding of fine- mme-grained semantic concepts and their relations from neural activity.
Concept grounding. Several influential works have focused on how semantic information is represented and organized across the cortex. As a representative work, Huth et al. used voxel- wise encoding models with natural narrative stimuli to construct semantic atlas, showing that different semantic domains selectively ground to distinct brain regions (Huth et al.] 2016). Mitchell et al. predicted fMRI patterns for concrete nouns using corpus-derived semantic features, showing generalization to unseen words (Mitchell et al. 2008). SOC enabled zero-shot decoding by mapping fMRI semantic codes and recognizing novel object categories (Palatucci et al. 2009). Pereira et al.
2

---

Published as a conference paper at ICLR 2026
introduced general decoder that maps fMRI into shared semantic space, enabling generalization from limited data (Pereira et al. 2018). Beyond semantic mapping, several studies have also explored how concepts are organized in the brain (Frankland & Greene 2015 Eichenbaum 2001.. For
example, abstract concept processing and rule-like representations have been linked to prefrontal
and medial temporal regions across tasks (Quiroga et al.] 2005; Rey al. 2013 Tian al. 2024 Dijksterhuis et al. (2024). Additionally, single-neuron recordings in the human prefrontal cortex have revealed neurons that encode abstract task rules independently of sensory or motor details (Mian et al. 2014). In contrast to these prior works, our approach emphasizes compositional concept grounding the neural decoding process, with focus on functional compositionality, where we explicitly model not only individual concepts, but the relations between them. This modeling enables a targeted study of how predicate-leva meaning can be decoded through structured grounding over brain regions.
IMRI-question answering. Recent works have explored using fMRI data for question-answeringg by integrating large vision-language models (VLMs). These methods typically map neural activity to visual embeddings, then generate answers using pre-trained VLMs. For example, SDRecon (Takagi)
& Nishimoto 2023) projects fMRI signals into BLIP (Li et al., (022) embeddings for captioning:
BrainCap (Ferrante et al., 2223) maps fMRI to GIT (Wang et al. 2022) features for visual description;
and UMBRAE (Xia et al. 2024) aligns fMRI to multimodal embeddings with subject-specific
tokenization and answers questions via LLaVA (Liu et al. 2023). These methods commonly use
BLEU scores (Papineni et al. 2002) to measure alignment with ground-truth text, but they do not verify whether the predicted answer captures exact underlyng concepts and relational structure. In contrast, our framework first grounds fMRI activity to modular concept representations and then performs structured reasoning over the resulting symbolic form, enabling precise, accurate, and generalizable question answering.
METHOD
3.1 NEURO-SYMBOLIC FRAMEWORK
We introduce NEURONA, a neuro-symbolic framework for neural decoding with concept grounding in fMRI activity. Neuro-symboll models decompose queries into symbolic expressions over concepts, and then differentiably execute those expressions over input representations, using learned concept grounding modules to perform a variety of downstream tasks (Yi et al. 2018 Mao et al.) 2019 Hsu et al. 2023b Mao al. 2025). In this paradigm, each concept (e-g.. pe sson, holdi ng) is associated with a small neural network that maps entity-centric representations from input data to a predicted semantic signal, enabling intermediate grounding to be learned from weak supervision of downstream tasks. A differentiable executor composes concept module outputs according to the structure of the symbolic expression, allowing end-to-end training to predict the final answer.
In this work, we propose NEURONA differentiable neuro-symbolic model that adapts and extends the LEFT framework (Hsu et al. (203a)) to the fMRI decoding setting. LEFT was originally designed for concept grounding across visual domains (e.g., 2D images, 3D scenes), where set of candidate entities (e.g., object proposals) is typically specified a priori. In contrast, our setting introduces unique challenge: for IMRI there is no canonical notion of entities analogous to objects in scene: the relevant neural representations for a given concept are not provided by supervision and must be inferred. Accordingly, NEURONA jointly (i) evaluates hypotheses over candidate neural entities derived from parcellated fMRI signals and (ii) learns how these entities should be composed to support precise decoding. To make differentiable program execution compatible with this uncertainty. NEURONA introduces execution semantics that replace hard selection with soft, role-conditioned aggregation mechanism that guides predicate grounding, which we detail below sections.
Concretely, we train NEURONA on the fMRI question answering (fMRI-QA) task (See Figure D Each example consists of two elements. The first is a symbolic query about the visual stimulus e (e.g., "Is there person holding baseball bat?"). The second is an fMRI recording f E NxT measured while the stimulus is viewed, where N is the number of imput parcels and T is the number of time points. The goal is to predict answer a, either Boolean label (e.g.. True/False) classification over an answer concept vocabulary. Crucially, no supervision is provided on which brain regions are relevant to each concept.
3

---

Published as a conference paper at ICLR 2026
Visual stimulus
Symbolli expression pursed from stimulus
Weakly supervised concept grounding fMRI-QA
(xdstssholddnngppeesnn baseball bat)) holding
Possible grounding concept holding
Hypotheses candidate entities
Exceulor
OOOOOCO Multi region
Single region
Linear Layer
Yes
IMRI signal
regions guided by argraments haselblll har:
person:
00000
SomMorA
DonAAtti
ContB
ContC
Figure 1: NEURONA is a neuro-symbolic framework for neural decoding that parses each query into a symbolic expression and maps the accompanying fMRI recording into candidate parcel-level embeddings. It grounds concepts in the expression to these candidate parcels with learned linear concept modules, optionally guided by predicate-argument structure, and composes the grounded scores to answer the question. Supervision is provided only by the final answer, which enables learning of intermediate groundings.
To define candidate neural entities, we map the fine-grained networks to P functional networks using a standard functional atlas. We experiment with Yco-7 and Yco-17 17 YYo et al. 201I), DiFuMo-64 and DiFuMo- 128 (Dadi et al., (2020), and Schaefer-100 (Schaefer et al. 2018) atlases, where P is 7, 17, 64, 128, 100 respectively. All atlases yield consistently strong performance with NEURONA: in the main text report results using Yeo-17 17, and provide results across atlases in Appendix A After parcellation, we obtain P parcel-level fMRI signals and encode them into a unified set & of parcellation embeddings (ei,..., ep). These parcel embeddings define candidate neural entities onto which concepts may ground.
In NEURONA, each concept is associated with a grounding module (i.c., a neural network) that maps from the candidate entity embeddings to concept-specific spores-intuitively identifying which parcels (or parcel pairs) best support predicting that concept. A differentiable executor then composes grounded concepts according to the symbolic expression to produce a final output a (a Boolean value distribution over answer classes). This modular structure enables NEURONA to decode compositional semantics from fMRI and to test hypotheses about how concept grounding should be guided across distributed neural signals without requiring direct supervision on intermediate groundings. In the following sections, we describe our grounding formulation, the hypotheses test, and the training objective.
3.2 GROUNDING CONCEPTS ONTO NEURAL ACTIVITY
Given parcel embeddings & = (e1,..., erl. where each entity is embedded to dimension d, concept grounding aims to assign each concept, out of C' total concepts, score over these candidate entity embeddings.
Unary concepts. To model unary concepts (e.g., subjects and objects such as person and baseball-batis we score each concept over entity embeddings E using a linear projection with weights Wunary c and bias Dunary gC, produeing concept-specific evidence over parcels. For each entity Ep. the predicted logits are computed via this linear layer; the overall predicted logits Zunary E I PXC is the grounding logits of all concepts, and the grounding score for concept c is Gunary (c) = [Z1e...., [Pe] R²,, where zp.c denotes the logit for concept c at parcel
Relational concepts. To model relational concepts (e.g., predicates such as ho holding). we compute logits over parcel pairs. We augment the entity embeddings & with learnall embeddings &, R Pxd and form & E IR Px2d, where &, = @ E, and @ denotes concatenation. Here, E, provides features that represent each parcel. For each pair of parcels (i, concatenate their embeddings ec, ecy
4

---

Published as a conference paper at ICLR 2026
and apply a learnable transformation Wair R4dxd to obtain pairwise representation E R We then apply a linear classifier to compute the logits for each pair, yielding Ziinary R PxPC which represents the grounding logits of all concepts. The grounding score for concept is Giinnry (c) = Pusses PxP.
From groundings to answers. The grounded scores are used (and optionally composed) to answer IMRI-QA queries. Let Cs. Cos and Cp denote the subject, object, and predicate concepts in a query. For Boolean queries, optionally condition the predicate grounding Giinar((c)) the unary groundings Gonary( (Ca) and Gunary(C,, (Co), and aggregate the resulting scores. Then, we apply sigmoid over the scores, and threshold the result produce binary decision in inference.
For concept classification queries with answer vocabulary V, we compute unary and relational logits for each candidate answer E V from the precomputed tensors, treating them the unary similarity Sunary and relational similarity Suinary scores as follows
Sunary =
[suul E PXIM, Suinary = (ayjelsssssPeve E
E PPPxx|V]
(1)
When subject and object groundings are available, we compute guided scores
Squided
unary
Gunary (co) Sunary + Gunary (co) Sunary E B
(2)
guiddd
Gunary(( (c&) (Gunary(co))
Sbinary) E VI
(3)
binary
TR. VI with the final predicted concept selected by i arg maX+ev Stfnal This yields unified, differentiable formulation for grounding unary and relational concepts for both Boolean and vocabulary-baas bbssed queries.
and combine them as Stinal
uuary
+ S binary
sguided
3.3 TESTING HYPOTHESES OF GROUNDING STRUCTURES
We evaluate five hypotheses about how concepts may be grounded over parcels and parcel pairs. Because NEURONA is modular, we can explicitly test guided grounding by conditioning predicate representations on their arguments rather than treating each concept independently. Since relational groundings are defined over parcel pairs, we compare hypotheses in per-parcel space by reducing pairwise scores via aggregation Gbiuary (c): P Ej=1Ca LGrinary (c)]ij-
HI: Single-region localized. Concepts are best supported by single region, where the grounding score is defined as Ghi(c)- Gunary (c).
H2: Multi-region co-activation. Concepts are best supported by co-activation across region pairs, where the grounding score is defined as GHz(e,)) Gbinary (cp)-
H3: Subject-guided multi-region. Predicate grounding guided by subject evidence, with the grounding score defined as GHs(Cyp) = Guinary (cp) + +G Gumary-(Ca (c.).
H4: Object-guided multi-region. Predicate grounding is guided by object evidence, with the grounding score defined as GHa(C,)) Guinary (cp) Gunary
H5: Full argument-guided multi-region. Predicate grounding is guided by both arguments, with the grounding score defined as GHs(Cp) = Guiuary (cp) + Gunaey (c.) + Gunary(Co))
Together, these hypotheses define different ways of structuring grounding over E, allowing to test whether structure-guide guided grounding improves decoding. Full derivations and experimental details are provided in Appendix G
5

---

Published as a conference paper at ICLR 2026
BOLDS000-QA
CNeuroMod-QA
CAR
Q: there a red molocycle?
Q: there a woman talking with a man? exists(Object, lanbda x: wonan(x)) talk(x, iota(Object, lambda y: nan(y)))) A: Yes
P: exists(Object, lambda x: red(x) and notercycle(x)) A: No
O: What the action between the person and motorcycle? Pt describe(Action, lanbda k; action(k, iota (0bjeet,, lanbda person(x)) and lota(Object, Lambda notoreycle(y))) A: Riding
Q: What the position between couch plant?
P: descrtteePassttion, larbda positian(k, tota(Chject, lanbda cauch(x)) tota(Oeject, larbda y: plant(y))))
A: Reside
Figure 2: We include example queries and dataset distribution overviews for BOLDS000-QA and CNcuroMod-QA: both datasets span diverse queries and tasks.
3.4 TRAINING OBJECTIVE
We train NEURONA the fMRI-QA setting, where ground-truth answers are provided for each query, but no supervision is provided intermediate concept groundings, none are available. We optimize a standard cross-entropy objective over the model's predicted answer distribution LCF = Σ k= log (âx), ). where à E RRK is the predicted probability distribution over K answer classes after executing the full symbolic expression, and a is the ground-trut truth label.
4 DATASETS
To train and evaluate NEURONA, we construct fMRI question answering (IMRI-QA) datasets by adapting existing large-scale IMRI-vision datasets. Our goal is to derive structured, compositional supervision from rich visual stimuli. To do so, for each stimuli, we use a pre-trained vision-language model produce scene graph that captures both object-level and relational semantics, for example, unary entities such as person, baseball 11-batll.at... and higher-arity predicates describing their interaction such as ho holding (person, basebal 1 b-bat)..
We then convert the scene graph into a set of structured question- -nnweer pairs, where each question corresponds symbolic query, and each answer provides supervision for neural decoding. For example, given the above scene graph, we construct questions such as: "What is the relation between person and baseball-bat?.. We additionally generate negative samples by sampling concepts from other stimuli in the dataset. This process yields diverse fMRI-QU -QA examples covering both unary enti- ties and higher-arity relations, with precise answers. We apply this procedure to BOLD5000 (Chang et al. 2019) and CNeuroMod (Gifford et al. 2024 Boyle et al. (203) (See Figure 2 to create the 5500D5000-QA and CNeuroMod-QU datasets. More dataset details can be found Appendix [F
BOLD5000-QA. The BOLD5000 dataset large- fMRI dataset collected while participants viewed naturalistic images. The dataset consists of approximately 5, 000 distinct images from three datasets: Scene Images (Xiao et al. 2010), COCO (Lin et al.] 2014). and ImageNet (Deng al.) 2009). Four participants viewed these images while undergoing whole-brain fMRI scanning. To create BOLD5000- QA, we follow the process above, and generate queries containing 4, 258 unary and 135 relational concepts, with 133, 146 train and 2, 095 test examples.
The CNeuroMod dataset is large-scale fMRI dataset that includes recordings of participants watching full-length naturalistic videos. Specifically, we build upon the Friends dataset, and set season to 5 to be the train samples, and unseen season 6 to be the test. To create CNeuroMod-Or -AA, we sample video frames based on motion energy, defined as the absolute difference between consecutive frames. We then extract changes, and construct corresponding symbolic queries. The CNeuroMod QA dataset includes 1, .66 unary and 106 relational concepts, with 157, 046 train and 30, 059 test examples.
CNeuroMod-QA.
graphs for each sampled frame. aggregate the

---

Published as a conference paper at ICLR 2026
Table I: We evaluate NEURONA BOLDD5000- QA and CNeuroMod-QA, comparing its perfor-
mance to prior IMRI language decoding models and linear baseline.
BOLD5000-QA
CNcuroMod- QA
Method
Overall
Action
Position
T/F
Overall
Action
Position
T/F
Linear 0.4754
0.4692
0.2069
0.1778
0.5260
0.4638
0.3043
0.1285
0.5192
UMBRAE
0.2069
0.1746
0.5340
0.4642
0.1432
0.1762
0.5378
SDRecon
0.4711
0.2414
0.1937
0.5248
0.4430
0.1350
0.1481
0.5238
BrainCap
0.4773
0.1937
0.1724
0.5551
0.4417
0.1257
0.1477
0.5112
NEURONA (Ours)
0.7041
0.6207
0.5079
0.7407
0.7046
0.6514
0.5746
0.7250
Table 2: Generalization results of NEURONA and prior work on unseen queries.
BOLDS000-QA
CNeuroMod- QA
Method
Overall
Action
Position
T/F
Overall
Action
Position
T/F
Linear
0.4587
0.0690
0.0794
0.5231
0.4143
0.0398
0.0323
0.5003
UMBRAE
0.4501
0.1379
0.0984
0.5191
0.4502
0.1205
0.1384
0.5225
SDRecon
0.4702
0.2414
0.1937
0.5237
0.4341
0.1236
0.1473
0.5008
BrainCap
0.4754
0.1724
0.1937
0.5311
0.4347
0.1198
0.1402
0.5042
NEURONA (Ours)
0.6840
0.6207
0.4984
0.7184
0.6583
0.4365
0.5261
0.6991
RESULTS
We evaluate NEURONA on neural decoding accuracy and on the consistency of its intermediate concept groundings in the below sections. We present quantitative results in Section 3.1 qualitative
analyses Section 5.2 and a discussion of limitations and implications in Section 5.3
5.1 QUANTITATIVE PERFORMANCE
We measure NEURONA's performance on both the BOLD5000-QA and CNeuroMod-QA datasets, with full predicate wredicat-arguument
argument guidance the underlying grounding structure. We compare against bascline fMRI decoding methods, test gencralization to unseen compositional queries, conduct ablations that correspond to our grounding hypotheses, and report quantitative consistency metric for concept grounding. Additional results across atlases, across subjects, across tasks, and detailed analyses are provided in the Appendix.
Comparison to prior works. We first evaluate NEURONA the IMRI question answering (fMRI-QA) task, compared against existing decoding methods: a linear baseline, UMBRAE (Xia
et al., 2024). SDRecon TTaagi & Nishimoto, (023), and BrainCap (Ferrante al. 2023). All models are trained on subject CSII (BOLD5000) and subject sub-01 (CNeuroMod) As shown in Table NEURONA consistently outperforms prior works across both the BOLD5000-QA and
CNeuroMod- datasets. Relative to the strongest prior method, NEURONA achieves a 47% relative improvement. These results suggest that linear methods and purely end-to-end decoding pipelines both struggle with the fine-grained concept and relation decoding required by fMRI-QA. In particular, NEURONA demonstrates significant gains on queries about actions and positions, which involve precise relational reasoning over subject and object roles. This highlights the strength of our neuro-symbolic framework, which explicitly grounds neural activity using predicate-argument structure to answer relational queries.
Generalization to unseen compositions. We further measure NEURONA's 's ability to generalize to unseen compositions of concepts at test time, which robustly evaluates whether learned groundings support systematic recombination. Specifically, we construct evaluation splits where all training and testing queries are disjoint, with no overlapping combinations of entities and relations, forcing models to generalize compositionally. For example, the training set may contain only queries such as in front of (person, sur fboardd, while the test set includes held-out compositions such as in front of (surfboard, person). As shown in Table 12, NEURONA achieves the strongest

---

Published as a conference paper at ICLR 2026
Table 3: We ablate hypotheses about the structure of concept grounding BOL.D5000-Q2 QA..
BOLDS000QQ
Method
Overall
Action
Position
T/FF
Single region Multi-region (MR)
0.61151 + 0.0161
0.2973 + 0.0361
0.2005 = 0.0047
0.7293 # 0.0191
0.6476
0.2973 0.
0.7324 0.0056
Subject guided MR
0.6678 1: 0.0029
0.2881 1. 0.0299
0.3469 1 0.0134
0.7308 1. 0.0024
Ouject-guided MR
0.6733
0.0051
0.3892 0.0752
0.3429 = 0.0164
0.7363 + 0.0045
Full argument guided MR
0. 0.7102 0.0053
0.5965 0.0322
0.5378 0.0135
0.7425 1 0.0057
Table 4: We evaluate our hypotheses about structural priors CNeuroMod-QA.
CNeuroMod- QA
Method
Overall
Action
Position
T/F
Single region
0.6429 + 0.0013
0.2165 + 0.0193
0.2445 = 0.0009
0.7400 + 0.0016
Multi-region (MR)
0.6162 0.0027
0.2165
0.0009
0.70412 0.0023
Subject-guide MR
0.6265 + 0.0042
0.2339 1: 0.0043
0.3722 = 0.0045
0.7008 1 0.0051
Sohject-guidor
0.0019
0.4933
0.0172
0.7149 1 0.0045
Full argument-guided MR
0.7189 + 0.0009
0.6422 0.0072
0.5931
0.0084
0.7417 + 0.0005
performance across both datasets, significantly outperforming baselines. Notably, prior methods show significant performance degradation, often falling near-chance levels when generalizing to unseen compositions. Language model-based baselines retain slightly better performance due to their of pre-trained language models, which carry general linguistic priors. However, since they lack explicit concept grounding, their performance remains well below NEURONA. Overall, these results suggest that NEURONA learns meaningful groundings that support robust generalization to novel decoding queries.
Ablations and hypotheses testing. We conduct ablations corresponding to our grounding hy- potheses, and evaluate how they affect decoding performance. We summarize results Table and Table compare single-region grounding, multi-region grounding without guidance, and guided grounding based on subject and/or object arguments. We report mean and standard deviation across subjects in BOLD5000 QA and 3 subjects in CNeuroMod-O With NEURONA, answer the following questions.
DOES AULTI-REGION GROUNDING IMPROVE PERFORMANCE? We first test whether allowing concepts to ground combinations of regions improves decoding performance relative to single- region grounding. As shown in Table 3 and Table 4 we observe that multi-region grounding alone does not yield consistent gains over single-region grounding in overall accuracy. This especially evident in vocabulary classification tasks, where both variants tend to overfit to frequent labels rather than capturing fine-grained distinctions.
DOES GUIDED GROUNDING IMPROVE PERFORMANCE? Next, we evaluate whether guiding multi- region grounding using the subject and object arguments of relational concepts improves decoding. As shown, guided variants consistently outperform unguided multi-region baselines. We find that object-guided grounding is generally more effective than subject-guided grounding, and that using both arguments yields the best performance, notably on action and position queries that require precise relational reasoning. These results suggest that while multi-region grounding expands the representational space, explicit structural guidance based on predicate-argumen argummnt dependencies is crucial for accurate and generalizable decoding. The consistent results on both natural image and video datasets validate NEURONA ability to conduct complex neural decoding with structural priors.
Concept grounding consistency. As there is ground truth for intermediate groundings, introduce consistency metric that measures whether the same concept tends to ground to similar regions across different IMRI-QA instances. We calculate consistency as follows. Let concept c appear in N' QA examples, and let the predicted grounding in the i-th example be a set of regions (c) S {1...., P}. where P is the total number of regions, and each B (c) is the set of regions selected as the grounding for concept in that example. We compute the frequency count for each
8

---

Published as a conference paper at ICLR 2026
region as
Count(r)
Σ
1| E (c).
(4)
Then, the score for concept c defined as
Count(r)
Consistency(c) =
(5)
R
NN
FER
where is the set of all regions that appear in any grounding c, and Count(r) N is the fraction times
region was selected. Our proposed metric captures how concentrated a concept's groundings are: a score of 1.0 indicates that all instances select the same region set, whereas lower scores indicate greater variability.
We report consistency scores over all concepts in BOLD5000-QA and CNeuroMod-QA As baseline, we define a null model that randomly assigns region groundings using multinomial distribution, preserving the exact total sample count for each concept while distributing sam- ples uniformly across all regions. This ensures that the null model retains the same structure as the observed data (i.e., same total grounding counts per concept) while randomizing only the region assignments. We run this null model 10 times and report the mean and standard deviation in Table We see that NEURONA achieves significantly higher consistency than the null baseline, suggesting that its learned groundings are reproducible across stimuli, rather than arising from chance.
Table 5: Consistency of concept grounding be- tween NEURONA and multinomial null model.
BOL BO0.D5000-A
CNeumMod-QA
Null NEURONA
0.6267 + 0.0067
0.6424 1 0.0057
0.8475 + 0.0186
O. 0.8592 + 0.0084
5.2 QUALITATIVE ANALYSES
Finally, we qualitatively examine how NEURONA decodes relational concepts via intermediate groundings, focusing on how predicate groundings vary with different subject-object pairs. Figure 3 shows examples from both BOLD5000-QA and CNeuroMod-QA, where we project grounding scores onto network parcellations that define candidate entities. We observe that the same predicate (e.g.. hold or look), can be best decoded from different regions depending on the object. For exam- ple, in BOLD5000, holding (person, kite) is best decoded using the Control B network, while ho holding (person, sur fboard) relies on both the Somatomotor B and Control A net- works. This pattern suggests that predicate decoding benefits from regument-dependeer modulation, consistent with our quantitative results guided grounding.
Interestingly, the model-inferred groundings are not confined to early visual areas. For exam- ple, objects such as baseball-bal and sur surfboard often receive high grounding scores in somatomotrr-asocitee networks. This qualitative pattern is broadly consistent with prior work linking perception of action-related objects with engagement of motor and premotor systems (Mar) tim 2007: Gallese et al. 1996). In addition, both hold and look are frequently decoded using prefrontal networks (e.g., the dorsal attention and salience/ventral attention networks), which have been associated with high-level cognitive control and abstract rule processing (Miller & Cohen 2001 Quiroga al. 20055: Tian et al. 2024). However, emphasize that these observations reflect model- dependent decoding patterns rather than direct estimates of underlying encoding representations.
Additional visualizations are provided Appendix E
5.3 DISCUSSION
Our findings show that incorporating compositional structure into the neural decoding process can significantly improve fMRI-QA performance. While NEURONA does not establish representational compositionality in neural activity, the gains from modeling predicate- argument structure as an inductive bias provide evidence that compositional assumptions can inform neural decoding. At the same time, our analyses highlight limitation of current neuroimaging resources. We lack the combinatorial data necessary to better validate representational compositionality, Existing visual fMRI datasets Allen et al. (2022); Chang et al. (2019); Horikawa & Kamitani (2017) provide rich naturalistic stimuli (the "whole") ) but rarely include the systematic, combinatorial manipulations needed to directly test how isolated "parts" compose. In addition, participants in datasets engaged

---

Published as a conference paper at ICLR 2026
BOLDS300-Q4
CNearoMMee-Q44
person
LITe
person
purtboard
votan
Clectes
HOBUT
belolperson, kite)
halr(person,, surfooard)
Losk(vomun, clothes)
Took/anenn, mas)
Protent
sarfboard
ciothes
Figure 3: We show examples of learned grounding from NEURONA. On both BOLD5000-QA and CNeuroMod-QA, that predicate concepts ground to regions that their constituent objects arguments are grounded to, following hierarchical predicate-aguumett structure.
only in passive viewing, and symbolic expressions were derived through automated scene-graph parsing rather than participant-driven reasoning, limiting the cognitive conclusions that can be drawn. Moreover, we restrict candidate regions predefined parcellations from established atlases (e.g.. Yeo-7, Yeo-17, DiFuMo-64, DiFuMo- 128, and Schaefer- 100), which are widely used but necessarily coarse. In the Appendix. report performance BOLD5000-Q2 and CNeuroMod-OU across atlases, effect sizes between atlases, and concept grounding consistency across atlases. These results demonstrate robust decoding across different levels of spatial granularity, suggesting that results are not tied to specific parcellation choices. However, we believe that scaling NEURONA to incorporate whole-brain voxel-level data promising next step.
CONCLUSION
We propose NEURONA, neuro-symbolic framework for neural decoding and concept grounding in IMRI activity. By leveraging symbolic reasoning and compositional execution with fMRI grounding, NEURONA supports precise decoding and significantly improved generalization. Experiments on BOLD5000-QA and CNeuroMod-Ou MM demonstrate that NEURONA outperforms strong baselines and generalizes to held-out queries, with argument-guide grounding yielding the largest gains. These results suggest that incorporating predicate-arge predicate-argument argument dependencies structural prior can improve relational decoding, and highlight neuro-symbolic modeling as promising approach for for building and analyzing fMRI decoding models.
REPRODUCIAILLIT STATEMENT
We release our code at attp:s//ggtthub..mm///aaaaaae///
ACKNOWLEDGMENTS
This work is in part supported by the Stanford Institute for Human-Centered AI (HAI), NIH A0189991,,, AFOSR YIP 249-50-23-1-0127. ONR 100014-23-1-2355. ONR YIP N00014- 24-1-2117, ONR MURI 4-0014444---2748, and NSF RI #221 1258. JH is also supported by the Enight-Hennessy Fellowship and the NSF Graduate Research Fellowship.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Emily J Allen, Ghislain St-Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince. Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et A Massive 7T fMRI Dataset to Bridge Cognitive Neuroscience and Artificial Intelligence. Nature neuroscience, 25(1):116-126) 2022.
Julie Boyle, Basile Pinsard, Valentina Borghesani, Francois Paugam, Elizabeth DuPre, and Pierre Bellec. The Courtois NeuroMod project: Quality Assessment of the Initial Data Release (2020). In 2023 Conference on Cognitive Computational Neuroscience, pp. 2023-1602, 2023.
Nadine Chang, John A Pyles, Austin Marcus, Abhinav Gupta, Michael J Tarr, and Elissa M Aminoff. BOLD5000, Public fMRI Dataset While Viewing 5000 Visual Images. Scientific data, 6(1):49, 2019.
Zijiao Chen, Jiaxin Qing, Tiange Xiang. Wan Lin Yue, and Juan Helen Zhou Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
22710-22720, 2023a.
Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity. Advances in Neural Information Processing Systems, 36: 24841 -24558, 2023b.
Kamalaker Dadi, Gaël Varoquaux, Antonia Machlouzaridde--Sallt,, Krzysztof J Gorgolewski, Demian Wassermann, Bertrand Thirion, and Arthur Mensch. Fine-grain Atlases of Functional Modes for IMRI Analysis. Neurolmage, 221:1 17126, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-scale Hierarchical Image Database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Kingma Diederik. Adam: A Method for Stochastic Optimization. (No Title), 2014.
Doris Dijksterhuis, Matthew W Self, Jessy K Possel, Judith C Peters, ECW van Straaten, Sander Idema, Johannes C Baaijen, Sandra MA van der Salm, Erik Aarnoutse, Nicole CE van Klink, et al. Pronouns Reactivate Conceptual Representations in Human Hippocampal Neurons. Science, 385(6716):148-14435 2024.
Howard Eichenbaum. The Hippocampus and Declarative Memory: Cognitive Mechanisms and Neural Codes. Behavioural brain research, 199--2)) 2001.
Matteo Ferrante, Tommaso Boccato, Furkan Ozcelik, Rufin VanRullen, and Nicola Toschi. Multi- modal Decoding of Human Brain activity into Images and Text. In UniReps: the First Workshop on Unifying Representations Neural Models, 2023.
Jerry Fodor. The Language of Thought. Harvard University Press, 1975.
Steven M Frankland and Joshua D Greene. An Architecture for Encoding Sentence Meaning in Left Mid-superior Temporal Cortex, Proceedings of the National Academy of Sciences, 112(37): 11732-11737,200
Vittorio Gallese, Luciano Fadiga, Leonardo Fogassi, and Giacomo Rizzolatti. Action Recognition in the Premotor Cortex. Brain, 11(())5933609) 1996.
Alessandro T Gifford, Domenic Bersch, Marie St-Laurent, Basile Pinsard, Julie Boyle, Lune Bellec, Aude Oliva, Gemma Roig, and Radoslaw M Cichy. The Algonauts Project 2025 Challenge: How the Human Brain Makes Sense of Multimodal Movies. arXiv preprint arXiv:2501.0mm0 2024.
11

---

Published as a conference paper at ICLR 2026
Zixuan Gong, Guangyin Bao, Qi Zhang, Zhongwei Wan, Duoqian Miao, Shoujin Wang, Lei Zhu, Changwei Wang, Rongtao Xu, Liang Hu, et al. NeuroClips: Towards High-fidelity and Smooth IMRI-to-Videe Video Reconstruction. Advances in Neural Information Processing Systems, 37:51655- 51683, 2024.
Tomoyasu Horikawa and Yukiyasu Kamitani. Generic Decoding of Seen and Imagined Objects Using Hierarchical Visual Features. Nature communications, 8(1):15037, 2017.
Joy Hsu, Jiayuan Mao, Josh Tenenbaum, and Jiajun Wu. What's Left? Concept Grounding with Logic-Enhanced Foundation Models. Advances in Neural Information Processing Systems, 36: 2023a.
Joy Hsu, Jiayuan Mao, and Jiajun Wu. NS3D: Neuro-Symbolic Grounding of 3D Objects and Rela- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2012-2612222 2023b.
Alexander G Huth, Wendy De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack Gallant. Natural Speech Reveals the Semantic Maps that Tile Human Cerebral Cortex. Nature, 532(7600):453-458, 2016.
Kendrick N Kay, Thomas Naselaris, Ryan J Prenger, and Jack L Gallant. Identifying Natural Images from Human Brain Activity. Nature, 452(71855)352-3555 2008.
Aaron Kucyi, Amy Daitch, Omri Raccah, Baotian Zhao, Chao Zhang, Michael Esterman, Michael Zeineh, Casey H Halpern, Kai Zhang, Jianguo Zhang, et al. Electrophysiological Dynamics of Antagonistic Brain Networks Reflect Attentional Fluctuations. Nature communications, 11(1):325, 2020.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping Language-ime image Pre- training for Unified Vision- language Understanding and Generation. In International conference on machine learning. pp. 12888 12900. PMLR, 2022.
Tsung-Yr Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In Computer Visi 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning. Advances in neural information processing systems, 011344892-34116 2023.
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The Neuro- Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences from Natural Supervision. arXiv preprint arXiv: 10:004//28444 12544, 2019.
Jiayuan Mao, Joshua Tenenbaum, and Jiajun Wu. Neuro-Symbolic Concepts. arXiv preprint arXiv:2505.0619) 2025.
Alex Martin. The Representation of Object Concepts in the Brain. Annu. Rev. Psychol., 58(1):25-45 45, 2007.
Vinod Menon. 20 Years of the Default Mode Network: A Review and Synthesis. Neuron, 11 111(16): 2469-2487, 2023.
Matthew K Mian, Sameer A Sheth, Shaun R Patel, Konstantinos Spiliopoulos, Emad N Eskandar, and Ziv Williams. Encoding of Rules by Neurons in the Human Dorsolateral Prefrontal Cortex. Cerebral cortex, 24(3):807-816, 2014.
Earl K Miller and Jonathan Cohen. An Integrative Theory of Prefrontal Cortex function. Annual review of neuroscience. 24(1):167-222, 2001.
Tom M Mitchell, Svetlana V Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L Malave, Robert A Mason, and Marcel Adam Just. Predicting Human Brain Activity Associated with the 2008.
Meanings of Nouns. science, 19115880
12

---

Published as a conference paper at ICLR 2026
Yoichi Miyawaki, Hajime Uchida, Okito Yamashita, Masa-aki Sato, Yusuke Morito, Hiroki C Tanabe, Norihiro Sadato, and Yukiyasu Kamitani. Visual Image Reconstruction from Human Brain Activity Using Combination of Multiscale Local Image Decoders. Neuron, 60(5):915-925 2008
Thomas Naselaris, Ryan Prenger, Kendrick N Kay, Michael Oliver, and Jack L Gallant. Bayesian Reconstruction of Natural Images from Human Brain Activity, Newron, 63(6):902-915, 2009.
Thomas Naselaris, Kendrick N Kay, Shinji Nishimoto, and Jack L. Gallant. Encoding and Decoding in IMRL. Neuroimage, 56(2))400-40,, 2011.
Shinji Nishimoto, An T Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu, and Jack L Gallant. Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current biology, 21(19):164111644 2011.
Mark Palatucci, Dean Pomerleau, Geoffrey E Hinton, and Tom M Mitchell. Zero-shot Learning with Semantic Output Codes. Advances in neural information processing systems, 22, 2009.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th annual meeting of the Association for Compulational Linguistics, pp. 311-318, 2002.
Francisco Pereira, Bin Lou, Brianna Pritchett, Samuel Ritter, Samuel J Gershman, Nancy Kanwisher, Matthew Botvinick, and Evelina Fedorenko. Toward a Universal Decoder of Linguistic Meaning from Brain Activation. Nature communications, 9(1):963, 2018.
R Quian Quiroga, Leila Reddy, Gabriel Kreiman, Christof Koch, and Itzhak Fried. Invariant Visual Representation by Single Neurons in the Human Brain. Nature, 12570045)1100211100 2005.
Hernan G Rey, Matias Ison, Carlos Pedreira, Antonio Valentin, Gonzalo Alarcon, Richard Selway, Mark Richardson, and Rodrigo Quian Quiroga. Single-cell Recordings in the Human Medial Temporal Lobe. Journal of anatomy, 227
2015.
Alexander Schaefer, Ru Kong, Evan M Gordon, Timothy O Laumann, Xi-Nian Zuo, Avram J Holmes, Simon B Eickhoff, and BT Thomas Yeo. Local-global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cerebral cortex, 28(9):3095-31 2018.
Paul Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Aidan Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. Reconstructing the Mind's 's Eye: IMRI-to-Image with Contrastive Learning and Diffusion Priors. Advances in Neural Information Processing Systems, 36:24705-24722 247228, 2023.
Yu Takagi and Shinji Nishimoto. High-Resolution Image Reconstruction with Latent Diffusion Models from Human Brain Activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14453-14463, 2023.
Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jessica Dubois, Jean-Baptiste Poline, Denis Lebihan, and Stanislas Dehaene. Inverse Retinotopy: Inferring the Visual Content of Images from Brain Activation Patterns. Neuroimage, 33(4): 104-1116, 2006.
Zhenghe Tian, Jingwen Chen, Cong Zhang. Bin Min, Bo Xu, and Liping Wang. Mental Programming of Spatial Sequences in Working Memory in the Macaque Frontal Cortex. Science, 385(6716): cadp6091, 2024.
Jianfeng Wang. Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. GIT: Generative Image-to-text Transformer for Vision and Language. arXiv preprint arXiv:2205.14100 2022.
Yanchen Wang, Adam Turnbull, Tiange Xiang, Yunlong Xu, Sa Zhou, Adnan Masoud, Shekoofeh Azizi, Feng Vankee Lin, and Ehsan Adeli. Decoding Visual Experience and Mapping Semantics through Whole-Bran Analysis Using fMRI Foundation Models. arXiv preprint 1:11122411.07121 2024.
Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing-Hao Xue. Umbrae: Unified Multimodal Brain Decoding. In European Conference on Computer Vision, pp. 242-259. Springer, 2024.
13

---

Published as a conference paper at ICLR 2026
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. SUN Database: Large-scale Scene Recognition from Abbey to Zoo. 2010 JEEE computer society conference on computer vision and pattern recognition, pp. 3485-3492 IEEE, 2010.
BT Thomas Yeo, Fenna M Krienen, Jorge Sepulcre, Mert Sabuneu, Danial Lashkari, Marisa Hollinshead, Joshua L Roffman, Jordan W Smoller, Lilla Zöllei, Jonathan R Polimeni, et al. The Organization the Human Cerebral Cortex Estimated by Intrinsic Functional Connectivity. Journal of neurophysiology, 2011.
Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. Neural- symbolic VQA: Disentangling Reasoning from Vision and Language Understanding. Advances in neural information processing systems, 31, 2018.
14

---

Published as a conference paper at ICLR 2026
Supplementary for: Neuro-Symbolic Decoding of Neural Activity
The appendix is organized into eight main sections. Appendix includees experiments results across atlases: performance, effect sizes, concept grounding consistency. Appendix ( results across subjects: cross-subject decoding and cross-subject consistency. Appendix Appendix([includer inicuuue additional experiments results for cross-dataset transfer and fMRI retrieval tasks. Appendix D includes in-depth analyses of network ablations, statistical tests across hypotheses, fine-grained generalization, predicate argument binding, and detailed concept accuracy. Appendix additional visualizations of NEURONA's grounding results. Appendix F presents more examples illustrating our IMRI-QA datasets and detail the data generation process. Appendix G describes the training procedure of NEURONA, the implementation of baseline methods, and the setup of our hypothesis ablation experiments. Appendix BoaddixxHJeeture detailss our ethics statement. Here, we also note that use large language models to make minor improvements writing.
iincudes experiments
Appendix/EJrrovidess prpovide
A EXPERIMENTS ACROSS ATLASES
A.1 PERFORMANCE ACROSS ATLASES
We report performance of NEURONA across atlases show robustness of decoding. We map parcellated fMRI signals (1024 regions for BOLD5000, 1000 for CNeuroMod) to multiple atlases, including Y and Yco-17 Yeo et 2011). DiFuMo-64 and DiFuMo-122 128 (Dadi al., 2020). and Schaefer- 100 (Schaefer et al.) 2018 then train NEURONA on these candidate entities. Results in Table 6 and Table 7 show that NEURONA consistently learns across these atlases, and still significantly outperforms prior works in decoding accuracy. Yeo-17 17 yields the highest accuracy among tested atlases, followed by DiFuMo- and Schaefer:100
Table 6: NEURONA's performance with coarse and fine-grained atlases on BOLD5000-QA
BOLD5000-QA Overall
Action
Position
T/F
Yeo-7
0.6864
0.5517
0.4730
0.7270
Yeo-17
0.7041
0.6207
0.5079
0.7407
DiFuMo-64
0.6992
0.5517
0.5524
0.7282
DiFuMo- 128
0.7026
0.5862
0.5460
0.7327
Table 7: NEURONA's 's performance with coarse and fine- iiee-graind atlases CNeuroMod-QA
CNeuroMod-QA Overall
Action
Position
T/F
Yeo-7
0.6969
0,6459
0.5577
0.7180
Yeo-17
0.7046
0.6514
0.5746
0.7250
Schaefer-100 100
0.7043
0.6549
0.5614
0.7258
A.2 EFFECT SIZES BETWEEN ATLASES
To additionally evaluate the robustness of NEURONA to different parcellations, we compute Cohen's 's effect sizes between QA predictions from different atlases. For each atlas pair, we compute paired effect sizes using QA predictions across the test set. As seen in Table 8 and Table effect sizes consistently small, showing that NEURONA is robust to the choice of atlas and performs reliably across range of parcellations.
A.3 CONCEPT GROUNDING CONSISTENCY ACROSS ATLASES
In Table 10 and Table II we report consistency scores averaged over concepts in BOLD5000 and CNeuroMod, under multiple atlas configurations: Yeo-7, Yeo-17, 17, DiFuMo-64, DiFuMo-
15

---

Published as a conference paper at ICLR 2026
Table 8: Effect sizes between atlases in BOLDS000-QA.
BOLD5000-QA
Yeo-7
Yeo-17
Difumo-64
Difumo-128
Yeo-7
-0.017
-0.133
-0.012
Yeo-17
0.017
-0.116
0.006
Difummo644
0.133
0.116
0.121
Difumo-
0.012
-0,006
-0.121
Table 9: Effect sizes between atlases in CNeuroMod-QA.
CNeuroMod-QA Yeo-7
Yeo-17
Schaefer-100
Yeo-7
0.094
0.089
Yeo-17
-0,094
-0.006
Schaefer-100
-0.089
0.006
Table 10: Concept grounding consistency in BOLD5000-QA.
BOLD5000-QA
Overall
Unary Concept
Relational Concept
Yeo-7 Null
0.5738
Yeo-7 NEURONA
0.8207
0.8283
0.7343
Yeo-17 17 Null
0.5357
Yeo-17 17 NEURONA
0,8220
0.8351
0.6646
DiFuMo-64 Null DiFuMo-64 NEURONA
0,5075
0.8462
0.8644
0.6064
DiFuMo- 128 Null
0,5039
DiFuMo-122 128 NEURONA
0.8224
0,8380
0.6241
Table 11: Concept grounding consistency in CNeuroMod-QA.
CNeuroMod-QA
Overall
Unary Concept
Relational Concept
Yeo-7 Null
0.5740
Yeo-7 NEURONA
0.8437
0.8564
0.7563
Yeo-17 17 Null
0.5358
Yeo-17 17 NEURONA
0.8700
0.8967
0.6812
Schaefer- 100 Null
0.5029
Schaefer-100 NEURONA
0.8346
0.8695
0.5838
and Schaefer- 100. NEURONA achieves consistently high grounding consistency across all atlases, significantly above the null baseline. Unary concepts show higher consistency than relational ones, as expected due to their simpler structure. Across the atlases, all show consistent results, with DiFuMo-64 best for BOLD5000 and Yeo-17 best for CNeuroMod. NEURONA'S 's concept grounding reproducible way across different stimuli and parcellations.
16

---

Published as a conference paper at ICLR 2026
Table 12: Results across subjects for BOLD5000-QA and CNcuroMod-QA.
BOLD5000-QA
Overall
Action
Position
T/F
Linear
0.4702 0.0073
0.0677 0.0815
0.1817 0.0289
0.5280 0.0068
UMBRAE
0,4948 0.0098
0.2488 0.0572
0.2092 = 0.0401
0.5401 = 0.0097
SDRecon
0,4751 0.0064
0.2887 + 0.0499
0.2005 = 0,0047
0,5266 + 0.0075
BrainCap
0.4842 0.0052
0.2412 + 0.0470
0.2005 0.0047
0.5383 = 0.0061
NEURONA
0.7102 0.0053
0.5965 0.0322
0.5378 0.0135
0.7425 0.0057
CNeuroMod QA
Overall
Action
Position
T/F
Linear
0,4579 = 0.0142
0.2078
0.0400
0.1680 = 0,0338
0,5184 = 0.0153
UMBRAE
0.4588 0.0425
0.1588 0.0102
0.1705 0.0220
0.5225 = 0.0248
SDRecon
0.4368 0.0012
0.1428 0.0010
0.1554 0.0010
0.5012 - 0.0024
BrainCap
0.4422 0.0026
0.1245 0.0099
0.1497 = 0.0074
0.5110 = 0.0024
NEURONA
0.7189 0.0009
0.6422 0.0072
0.5931 = 0.0084
0.7417 = 0.0005
Table 13: Generalization results across subjects for BOLDS000-QA and CNeuroMod-QA.
BOLD5000-QA
Overall
Action
Position
T/F
Linear
0,4595 0.0093
0,0308 = 0.0312
0.1291 = 0,0545
0,5250 = 0,0059
UMBRAE
0.4886 0. 0068
0.1214 0.0217 0.0217
0.0914 0.0212
0.5392 0,0080
SDRecon
0.4752 0.0064
0.2973 0.0361
0.2005 0.0047
0.5266 0.0075
BrainCap
0.4838 0.0055
0.2412 0.0470
0.1996 = 0.0040
0.5380 = 0.0064
NEURONA
0.6812 0.0055
0.4952 0.0682
0,4696 + 0.0190
0.7217 = 0.0057
CNeuroMod QA
Overall
Action
Position
T/F
Linear
0.4206 0.0129
0.0462 0.0654
0.1045 0.0667
0.4986 0.0052 0.0052
UMBRAE
0,4487 0.0040
0.1107 0.0156
0.1386 + 0.0047
0.5247 : 0.0035
SDRecon
0,4365 0.0014
0.1407 + 0.0008
0.1560 + 0,0012
0,5012 = 0.0019
BrainCap
0.4382 0.0012
0.1217 0.0070
0.1455 0.0060
0.5069 = 0.0003
NEURONA
0.6676 0.0119
0.2916 0.0878
0.5377 0.0215
0.7260 = 0.0072
Table 14: Cross-subject consistency of concept grounding.
BOLD5000-QA
CNeuroMod-QA
Null (random)
0.5276
0,5267
Null (multinomial)
0.6045
0.6481
NEURONA 0.7000
0.7027
B EXPERIMENTS ACROSS SUBJECTS
B.1 CROSS-SUBJESS DECODING
We include evaluation over all subjects on both BOLD5000-QA (4 subjects) and CNeuroMod QA (3 subjects). For each dataset, we train an individual model for each subject and report the mean standard deviation across subjects. We evaluate neural decoding performance in Table 12 and our generalization split in Table [3 prior works.
We see that NEURONA continues to significantly outperform all
B.2 CROSS-SURIEES CONSISTENCY
In addition, evaluate cross-subject consistency in concept groundings. To evaluate whether concepts are grounded similarly across individuals, we aggregate grounding scores for each concept across all subjects and queries (for concept appearing N times per subject, we obtain N samples where S' is the number of subjects). We then compute cross-subjec consistency metric that measures the similarity of the spatial grounding patterns for each concept across individuals. We evaluate NEURONA against the multinomial null model and an additional null model that randomly assigns each concept to a region subset via uniform sampling that preserves each concept's distribution across regions. Our results in Table 14 show that NEURONA's 's cross-subject grounding consistency scores, averaged across concepts, are higher than both null models (p < 0.001), demonstrating that NEURONA's 's concept groundings converge to more similar sets of regions across participants.
17

---

Published as a conference paper at ICLR 2026
C ADDITIONAL TASKS
C.1 CROSS-DATASEE TRANSFER EXPERIMENTS
We report cross-dataset generalization performance by training our model on BOLD5000-QA and evaluating it on the CNeuroMod- -A test set. Since BOLD5000 spans broader concept space, we selected overlapping queries across datasets. the CNeuroMod test set, this includes 1, 169 queries for the action task, 2, 600 for the position task, and 23, 038 for the T/F task (out of full test set sizes of 2, 912, 2, 661, and 24, 486, respectively).
In Table 15| we compare NEURONA to UMBRAE (Xia et al. 2024), the top performing baseline model. NEURONA significantly outperforms UMBRAE across all queries, demonstrating stronger cross-dataset robustness and generalization. Notably, while overall performance of NEURONA drops, largely due to a performance gap on T/F queries, accuracy on action and position tasks remains high, indicating some degree of cross-dataset transfer. This drop is expected, subject-specific model and there is substantial variance across subjects. Additionally, the two datasets differ in preprocessing pipelines: BOLD5000 uses the DiFuMo- 1024 parcellation, while CNeuroMod uses the Schaefer- 1000 atlas. This difference requires us to apply padding to align the feature dimensions when evaluating on CNeuroMod. Furthermore, some concepts in CNeuroMod, such as telephone, occur infrequently in BOLDS000, which limits NEURONA's 's ability to generalize to them. Nonetheless, we find that NEURONA maintains strong performance on queries such as action decoding, suggesting meaningful transfer of motor-related neural representations across datasets.
model is trained
Table 15: Cross-dataset generalization results, where models are trained on BOLD5000 and tested on CNeuroMod.
Overall
Action
Position
T/F
UMBRAE
0.4494 0.5535
0.0106
0.0485 0.5246
0.5036 0.5481
NEURONA (Ours)
0.7237
C.2 FMRI RETRIEVAL RESULTS
Here, we detail results from fMRI retrieval task, where we adapt NEURONA to retrieve the corresponding fMRI given symbolic query. We use positive queries to ensure that the concept and fMRI are well aligned. Each concept is represented one-hot vector over the full concept vocabulary. from which a concept embedding obtained using a small MLP. We then follow the structure of the symbolic expression by applying aggregation operation to combine the specified concepts in the query. A parallel MLP encodes the fMRI input into an fMRI embedding, and we train the embedding spaces jointly using Case-bbsed contrastive loss. In this setting, we achieve test top-1 retrieval accuracy of 0.1325 and a top-5 accuracy of 0.3012, substantially outperforming a random-choice baseline (top~1: 0.0120, top-5: 0.0602).
18

---

Published as a conference paper at ICLR 2026
Table 16: Network ablations to evaluate the contributions of each network decoding performance.
Selected network
Overall
Action
Position
T/F
VisCent VisPeri
0.6750 + 0.0067
0.3577 = 0.0459
0.3765 + 0.0159
0.7330 = 0.0072
SomMotA/B
0,6724 = 0,0069
0.3480 = 0.0317
0.3620 + 0.0131
0.7326 + 0.0079
DorsAutnA/B
0,6753 0.0073
0.3281 = 0.0786
0.3850 # 0.0282
0.7324 0.007
SalVentAutAA/B
0.6709 0.0069
0.2910 0.0655
0.3839 = 0.0267
0.7281 = 0.0060
LimbicA/B
0.6669 0.0101
0.3379 0.0918
0.3512 = 0.0603
0.7282 0.0044
ContA/B/C
0.6785 = 0.0038
0.3380 = 0.0808
0,4129 + 0.0173
0.7310 = 0.0035
DefaultA/B/C TempPar
0,6804 = 0,0080
0.3248 = 0.0752
0.4357 = 0.0324
0.7297 = 0.0066
All
0.7102 0.0053
0.5965 0.0322
0.5378 + 0.0135
0.7425 + 0.0057
BOLD5000 Hypothesis Statistics
Milcous Signad-iani Aeealts
Lffice (Coher's
2-14
4111
411
Figure 4: Statistical tests evaluating the effects of the five hypotheses across subjects in BOLD5000.
D ADDITIONAL ANALYSES
D.1 NETWORK ABLATIONS
To evaluate how each functional network contributes to decoding performance, we perform a network- level ablation in which NEURONA was trained using only a constrained subset Yeo-7 networks at time, isolating the contribution of each network. In Table 16 we see that the top performing networks are the Default Mode, Control, Dorsal Attention, and Visual networks, all which have been closely linked to visual processing and high-level perceptual representations in prior works Menon (2023);
Kucyi et al. (2020); Miyawaki et al. (2008)
D.2 STATISTICAL TESTS ACROSS HYPOTHESES
We conduct additional statistical analyses for our hypotheses ablations to more rigorously evaluate the effects of the five hypotheses across subjects. Specifically, we use the Wilcoxon signed-rant test to assess statistical significance and Cohen's is d to estimate effect size. For each subject, we collect accuracies across all metrics, and perform comparisons between hypotheses using these values. The BOLD5000 and CNcuroMod results are summarized in Figure Hnd Figure Across both datasets, NEURONA's full argument-gumm -guided multi-region hypothesis (H5) consistently shows statistically significant improvements over alternative hypotheses.
D.3 FINE-GRAINED GENERALIZATION ANALYSES
Our generalization experiments on both BOLD5000 (subject-CSII) and CNeuroMod (subject-01) include argument swapping, predicate transfer, and role systematicity. In Table 17 report fine- grained systematicity tests across each these settings. We see that NEURONA shows strong performance each category, with relatively small drops from all to unseen combinations,
19

---

Published as a conference paper at ICLR 2026
CNeuroMod Hypothesis Ablation Statistics
Wilcooon Sprec-Rack Readts
Offect Siee (Cavers
SCM
PAN
Figure 5: Statistical tests evaluating the effects of the five hypotheses across subjects in CNeuroMod.
Table 17: Fine-grained generalization analyses in BOLD5000-QA and CNeuroMod-QA.
BOLD5000-QA
Argument swapping
Predicate transfer
Role systematicity
Other
All (T/F)
Unseen
0.7451
0.75
0.7391
0.7096
0.7184
All
0.7562
0.6667
0.8030
0.7393
0.7407
CNeuroMod-O QA
Argument swapping
Predicate transfer
Role systematicity
Other
All (T/F)
Unseen
0.7526
0.7333 0.5088
0.7390 0.7020
0.6842
0.6991
All
0.7011
0.7307
0.7250
Table 18: Results of predicate argument binding.
BOLD5000-QA
CNeuroMod QA
Unguided-preddd predicate subject
-0.0940
0.0230
predicate /object
0.0048
0,0019
Unguided predicate predicate
peguidde-prediccte
0.0336
0.1488
Guuded-prediees subjeet Guided--pedicate objject
0.2020
0.1095
0.2975
0.2552
D.4 PREDICATE ARGUMENT BINDING
We investigate whether unguided predicate representations contain decodable information about their arguments, by computing pairwise correlations between concept groundings across brain regions. For each relational query (e.g.. hold (person, baseball) ), we extract grounding score vectors across functional networks for the subject, object, unguided predicate, and guided predicate. We then compute the correlation matrix between these grounding patterns across all relational queries both BOLD5000 (4 subjects) and CNeuroMod (3 subjects).
we report the mean correlations, and see minimal correlation between unguided predicate groundings and their subject (-0.0940) object (0.0048) arguments in BOLD5000, with similarly low correlations in CNeuroMod (0.0230 for subject, 0.0019 for object). This suggests that when no structural guidance is provided, predicate representations do not bind specific arguments.
In Table T8
In contrast, guided predicate representations showed substantially higher correlations with both subjects (0.2020 in BOLD5000, 0.1095 in CNeuroMod) and objects (0.2975 in BOLD5000, 0.2552 in CNeuroMod). This increase correlation suggests that NEURONA'S 's explicit guidance helps decode relations from predicate-argument interactions.
20

---

Published as a conference paper at ICLR 2026
Table 19: Accuracy breakdown between unary and relational concepts in BOLD5000-QA,
Table 20: Accuracy breakdown between unary
and relational concepts in CNeuroMod-QNn
BOLD5000-QA
Overall
Unary
Relation
CNeuroMod- QQA
Overall
Unary
Relation
Yeo-7
0.727
0.717
0.751
Yea-7
0.718
0.696
0.754
Yco-17
0.740
0.732
0.760
Yeo-17
0.725
0.707
0.754
DiFuMo-64
0.728
0.727
0.728
Schaefer- 100
0.725
0.708
0.754
DiFuMo-128
0.732
0.733
0.730
Probulalkyy Canfiaka Matric for divETAALL Vocabalery Quarkos
#isted
4.0
12.0
02
00
30
3000
04
33
28
30
06
90
06
06
08
100
16.2
000
160
Prodeced
Figure 6: Accuracy confusion matrix across concepts in BOLD5000-QA.
Probubiiyy Confusion Matric for OIERALLL vocalblarry
Mlatedd
00
452
50
MA
900
BACK
63
100
900
52
VOA
30X
30
16.00
0.
23.4
00
00
967
2.5
06
00
35.
21N
24
Figure 7: Accuracy confusion matrix across concepts in CNeuroMod-QA
D.5 DETAILED CONCEPT ACCURACY
In Table 19 and Table 20
report QA accuracy for unary and relational concepts separately across BOLD5000 and CNeuroMod, to analyze whether query structure affects performance. We see that that performance is generally stable across concept types, and across multiple atlases.
In Figure 6 and Figure we illustrate confusion matrices for both BOLD5000 and CNeuroMod. We see that, expected, while many concepts are reliably decoded (c.g., visually distinctive actions), some still cause confusion (e.g. spatial relations that are semantically similar).
21

---

Published as a conference paper at ICLR 2026
E CONCEPT GROUNDING VISUALIZATIONS
In Figure 8
we present concept grounding examples from the BOLD5000 (Chang et al. 2019) and
CNeuroMod (Gifford et 2024 Boyle al. 2023 datasets.
BOLDS000-QA
leaves
branch
surround branch)
boat
on(bost, water)
driveway
beside(grass, driveway)
column
floor
attached tocnodd,o floor)
trash-can
nirror
next_tonnirron, trash-can)
CNeuroMoi-OK
ran
table
in frontt (f table) Lo
BOn
sofa
sit_ ontonn safa)
table
in_ frofront.oom pront_of(worun, table)
onvosan,,,,,,, sofa)
Figure 8: We show examples of learned concept grounding by NEURONA on BOLDS000-QA and CNeuroMod-QA across subject, object, and predicate concepts.
22

---

Published as a conference paper at ICLR 2026
F DATASETS
F.1 LICENSE FOR EXISTING DATASETS
We train NEURONA on the BOLD5000 (Chan al. 2019) and CNcuroMod (Giftord al. 2024 Boyle et al. 2023) datasets, which are both licensed under the Creative Commons 0 License. More information can be found on their websites: BOLD5000 and CNeuroMod
F.2 FMRI-QA DATASETS
BOLD5000-QA. We utilize the BOLD5000 dataset, which has been preprocessed and aligned
with image stimuli following WAVE (Wang et al., 2024). The fMRI data has a shape of [5, 1024], representing 5 TRs and 1024 brain regions. We preprocessed, image-aligned fMRI data provided here Each TR (repetition time) is 2 seconds, resulting in chunk duration of 10 seconds (5 x 2s). We account for a hemodynamic lag of TRs. All four subject pairs are included. following the same rait-test split as in previous studies.
CNeuroMod-QA. We use the CNcuroMod dataset preprocessed by the Algonauts Challenge Giffford et al. 2024 Boyle et al. 2023). The fMRI data has a TRR of 1.49 seconds and shape of [5. 1000 representing 5 TRs and 1000 brain regions based on the Schaefer- atlas (Schaefer et al. 2018).
This yields chunk duration of 7.45 seconds (5 x 1.49s), with hemodynamic lag of 3 TRs. For each chunk, we select the most motion-informative video frame by computing motion energy as absolute difference between consecutive frames. The chunks are extracted from Friends episodes, with seasons 1-5 used for training and the unseen season reserved for testing. We use 1, 0.00 fMRI-video chunks per season, resulting in 5, 000 training samples and 1, 000 testing samples.
We provide additional examples from our datasets in Figure
G EXPERIMENT DETAILS
G.1 TRAIN SETTINGS
We train and evaluate NEURONA the specified training and test sets for both BOLD5000 (Chang) iet al., (011)) and CNeuroMod (Gifford et al. 2024 Boyle et al. [2023). Training is conducted for 100 epochs using the Adam optimizer (Diederik) 2014). with learning rate 0.001 and batch size 32.
G.2 COMPUTE RESOURCES
Since NEURONA consists of lightweight convolutional neural network for IMRI feature extraction followed by linear classifier for concept grounding and execution, its computational requirements minimal. All experiments are conducted on single NVIDIA A100 GPU, with training completing in approximately 30 minutes. Data loading is parallelized using 16 CPU workers, and the system uses 64 GB of RAM.
G.3 BASELINE IMPLEMENTATIONS
We describe the implementation details of the baseline models compared in our study below. In all methods, we treat the fMRI input as sequence of length 5, with each time step as a token.
We tokenize the input query using the BERT tokenizer (Devlin al. 2019) and pad all sequences to a fixed length. The tokens are then encoded using a linear layer. We concatenate the fMRI token sequence and the query token sequence, and apply final linear classification layer to predict the output (either a binary T/F answer or a vocabulary token).
Linear.
We implement SDRecon Takagi & Nishimoto, 2023) following the official repository [ A ridge regression model aligns fMRI features with image embeddings, which are then passed to VQA-GIT language model (Wang al. 2022) to generate answers. We set the ridge regularization
SDRecon.
https:: / thub. com /ttps////////////o/o////ooaaraaa
23

---

Published as a conference paper at ICLR 2026
BOLD5000-QA
Q: Is there dog?
Q: Are there skis on the snow?
A: Yes
Yes
Q: Is bus on road?
What is the action between the person and the wine glass?
A: No
A: Hold
Q: What is the action between the sheep and the grass?
Q: Is there a moon?
A: No
A: Fat
Q: Is there person riding the motorcycle?
Q: there helmet?
A: No
Q: Is there bear?
Q: Is there a woman siting on the sofa?
A: Yes
A: No
CNcuroMod-QA
Q: Is there bed?
Q: there a man in green?
A: Yes
A: Yes
Q: Is there man sitting on the desk?
Q: What is the position between the door and the man? Beside
A: No
Q: Is there man looking at the painting?
Q: What is the position between the paper and the desk? A: On
A: No
Q: What the action between the man and the chair?
there woman?
A: Yes
A: Sit
Q: Is there red shirt?
Q: What is the action between the and woman?
A: Yes
A: Talk
Figure Examples of queries in BOLD5000-QA and CNeuroMod-O0 -A.-QA.
24

---

Published as a conference paper at ICLR 2026
parameter to À 20. A custom parser (described below) is used to map the generated language response to valid prediction.
BrainCap. BrainCap similarly uses a linear encoder to align fMRI features with visual embed- dings (Ferrante et al., 2023). The aligned embeddings are passed to BLIP language model (Li et al., 2022) to generate answers. We apply the same parser as in SDRecon to extract final predictions from the language output. We follow the implementation of the official repository
UMBRAE. UMBRAE leverages wansformer-base encoder to map fMRI features to image
embeddings (Xia et al. 2024). These embeddings are then passed to LLaVA (Liu et al. 2023) for language-based prediction, followed by response parsing. We follow the implementation of the official repository!
We implement rule-based parser to convert language model outputs into structured predictions. The parser first cleans the text by removing punctuation, digits, and formatting inconsistencies It identifies binary answers ("yes" or "no"" when the query requires. For other queries, it extracts the first valid word from predefined vocabulary. If no valid word is found, it defaults to the most common answers of "on" for spatial queries or "hold" otherwise. For all image-groundes baselines, the ground-truth image embeddings are derived from the visual encoder of vision- model for BOLD5000. We use the embedding of the first selected video frame the ground truth for CNcuroMod.
G.4 ABLATION DETAILS
In this section, provide the full definitions of the grounding hypotheses introduced in the main paper.
G.4.1 ENTITY PROCESSING
To enable concept grounding neural activity f NxT, first map the fine- fine-grained networks N 1024 to P functional networks defined by the given atlas. This results in network-speciffc fMRI signals (/1...., fr), where each fp Rm, represents the aggregated signal from mp fine-grained regions assigned to network p. Since the number of regions my vary across networks, we apply a linear stitcher to project each fp E Rm, xT to a fixed-dimensional representation ep E IR where = 256, using network-specific linear projections W, E " xd such that Ep = W fp. This produces a unified set & of P embeddings {e1..... Ep). which are then processed by a small 1-D convolutional encoder form parcellation embeddings. From these base embeddings, propose hypotheses of candidate regions from which concepts can be grounded.
G.4.2 GENERAL GROUNDING FORMULATION
We define the unary grounding score for a concept as Gunary (c) RR and the binary (pairwise) grounding score Gbinary (c) R PxP, Since Giinary (c) is defined over parcel pairs, we reduce it to unary space via
1 P [Grinary (c) E R..
Goinary (c);
Σ
P
==1
We then define a fused grounding score that combines unary and reduced binary evidence:
G(c) Gunary (c) Ebinary (c) E PR..
(6)
G.4.3 HYPOTHESES DEFINITIONS
Let Eps Co and e, be the concepts for predicate (e.g., holding), subject (e.g., person), and object (e.g., basebal ll-bat, respectively.
https:://gtthube http:://ggthh...cee///mmoonnanna http://///.araaacaiciioioooo httpp:://gittubb.co////eeeee https///////aiiooc////aaarrrraar
25

---

GH1 (c,) Gunary (ca). Published as a conference paper at ICLR 2026
HI: Single-region grounding.
Concepts grounded single brain region:
Chi(c) = Gunary (cp),
GHr(Co) = Gunary (co).
H2: Multi-region co-activation.
Concepts are grounded through co-activation across region pairs:
GH2 = Giiinry (cp); Ghz(Ca) = Gunary (c&), Ghz(Co) = Gunary (Co).
(8)
H3: Predicate conditioned on subject. Predicate representations are guided by the activation of the subject region:
Gus(Cp) Gbiiary (Cp) + Gunary
GHa(Cs) = Gunary (C)),
(9)
GH1((C) Gunary (Co)-
H4:
Predicate conditioned on object. Predicate representations are guided by the activation of the object region:
Gna(Cp) Giiinary (Cp) + Gunary GHa(CC) = Gunary (ca);
(10)
GH4(Cc)) Gunary (Co).
H5: Full rrguuent-guided grounding. Our proposed method combines multi-region grounding with subject and object guidance. The grounding scores are defined as:
Gus(c)) G(cp) + G(c,) + G(c.),
Gus(c,) = G(c,),
(1I)
Gus = G(c.).
These formulations enable systematic evaluation of how structural priors affect downstream neural decoding accuracy.
H ETHICS STATEMENT
This work uses only publicly released fMRI datasets, and focuses on decoding concepts from visual stimuli rather than personal traits identity, which reduces the risk of individual fingerprinting. However, we acknowledge the inherent risks in building architectures that enable neural decoding from participants. We emphasize that our method intended solely for scientific analysis, and should not be used for identification or inference of sensitive personal attributes from neural activity.
26


Reviews:
[{"review_id": "fUCjUWqejJ", "reviewer": "Reviewer_fNVL", "summary": "The paper proposes NEURONA, a neuro-symbolic framework for decoding compositional concepts from fMRI by parsing each visual stimulus into a symbolic expression (e.g., predicate–argument relations) and grounding its unary and relational concepts to candidate neural entities derived from cortical parcellations. The authors focused on using QA accuracy as the proxy for concept groundings capabilities. The proposed approach extends LEFT to fMRI-QA so that weak supervision from question-answer pairs can train disentangled concept groundings and an end-to-end executor that composes those groundings to answer queries. The authors conduct experiments to show that the proposed method improves accuracy and generalizes to unseen compositions on BOLD5000-QA and CNeuroMod-QA.", "questions": "Since fillers ultimately come from token sequences, I am wondering have the authors thought about how robust are role actions to multi-token fillers or subword boundary changes?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I am not familiar with tasks with fMRI, but to the best of my knowledge the framework is new in bringing predicate–argument guidance into an fMRI-QA decoder and testing it via five alternatives within one executor.\n\nThe overall evaluation setup looks correct to me: the compositional split is appropriate; train and test use disjoint entity–relation pairs, so it measures true generalization. The ablations are clean and isolate the source of gains: unguided multi-region grounding adds little over a single region, while subject/object guidance and full guidance help most, especially for actions and spatial queries. Results are robust and strong across multiple cortical atlases. The qualitative maps are sensible and aid interpretation. \n\nOverall I think the paper is overall clearly written and the ideas are intuitive to understand despite my unfamiliarities with fMRI-related tasks.", "weaknesses": "The primary concern is the novelty about the proposed approach. \n\nThe proposed method assembles known pieces: a LEFT-style executor, VLM-derived scene graphs, and standard cortical parcellations. The main new element is predicate–argument guidance and the within-executor hypothesis family that tests it. I acknolwedge that these are thoughtful design choices, but they are not a new model class or theory. That said, the paper still adds value: it gives a clean empirical test of the idea, uses a compositional split that rules out memorization, and offers consistency and cross-atlas checks that others can reuse. \n\nIn short, the work is incremental in mechanism yet still useful in evidence and evaluation protocol for the research commnuity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v8eDhKAjqV", "reviewer": "Reviewer_RD4u", "summary": "This paper proposes NEURONA, a neuro-symbolic framework for grounding compositional concepts in neural activity using fMRI data. The approach extends Logic-Enhanced Foundation Model (Hsu et al. 2023) to fMRI-based question answering, testing different hypotheses about how concepts are grounded in brain regions through hierarchical predicate-argument dependencies. The authors demonstrate improved decoding accuracy on constructed fMRI-QA datasets derived from BOLD5000 and CNeuroMod, showing particular gains on compositional queries involving actions and spatial relationships.\n\nThe version of the paper as of now should be rejected because (1) the central claims about discovering compositional organization in the brain are not well supported by the evidence, (2) the experimental design conflates task performance improvements with evidence of neural representational structure, (3) the \"grounding\" methodology does not actually demonstrate meaningful concept-to-brain mappings, and (4) key claims about compositionality lack proper validation through decomposability or systematic structural tests. The technical framework represents a potentially valuable contribution to neural decoding, but the interpretive claims about brain organization require much stronger evidence than currently provided.", "questions": "1.\tCan the authors implement encoding models built from the parsed concepts and use them as grounding/representations?\n2.\tIf strong claims about neurosymbolic representations are to be made: Can fine-grained systematicity tests be done? Is it possible for the evaluator to be inverted to make the output decomposable after binding?\n3.\tCan better null models be used (with random concept assignments)?\n4.\tIf your answer is no to points 1 and 2, can the focus be changed to methodological contribution and decoding improvements rather than strong claims about brain organization?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "Despite fundamental issues, the paper has notable technical strengths:\n\n1. A well-defined evaluator logic for creating conjunctions that could help in better decoding. \n2. Strong empirical performance: 47% relative improvement over baselines is substantial, and the generalization to unseen compositional queries is genuinely impressive evidence that the learned representations support novel combinations.\n3. Cross-dataset validation: Testing on both image and video datasets with consistent results strengthens the technical contribution.", "weaknesses": "1. The Core Interpretive Problem:\n\nThe paper's fundamental flaw is the interpretive leap from 'symbolic structure improves neural decoding' to 'relational meaning in the brain emerges from structured activations guided by hierarchical predicate-argument structure.' The authors consistently conflate improved task performance with evidence of compositional neural mechanisms, but these are different claims requiring different types of evidence.\nThe 'grounding' results are classification logits indicating which brain regions are statistically useful for predicting concept labels—not neural representations in a meaningfully neuroscientific sense. Classification weights in multivariate neuroimaging do not reflect the underlying neural patterns or information content; they indicate statistical utility for discrimination, not representational structure (see Haufe et al. 2014 NeuroImage). True neural representations involve patterns of activity across voxels that can be decoded, analyzed geometrically, and related to cognitive processes. Furthermore, the authors mischaracterize the prior literature on concept grounding. The foundational studies cited (Huth et al., Mitchell et al., Palatucci et al.) use forward encoding models that predict brain activity from semantic features, allowing legitimate claims about neural encoding. The paper should better distinguish its claims from prior work on concept grounding and be more precise about what type of evidence would support compositional neural organization vs. improved decoding.\nThe distinction between functional and representational compositionality should be made explicit early in the paper. The current presentation conflates these concepts (e.g., see Lake & Baroni 2018 ICML)\n\n2. Experimental Design Issues:\n\nThe datasets retrofit compositional reasoning onto passive viewing data without evidence that participants were performing compositional operations. BOLD5000 participants simply viewed images—they were not actively reasoning about spatial relationships or compositional structure. The \"ground truth\" comes from automated scene graph parsing, not actual participant cognition. This creates a fundamental disconnect between the experimental setup and claims about compositional cognition in the brain. The danger is that improved decoding may simply reflect visual feature correlations rather than compositional neural processes - without evidence that participants engaged compositional reasoning, there's no basis for compositional interpretations of the neural patterns.\nWithout participants actively performing compositional reasoning during scanning, there is no basis for claiming that improved decoding of these post-hoc questions reflects compositional neural organization. The authors need to demonstrate that participants were actually engaging compositional processes, not just that symbolic priors help classify brain signals.\n\nThe dataset construction follows machine learning conventions (scene graphs → QA tasks) which is appropriate for developing better reasoning systems. However, this methodology is insufficient for claims about brain organization. Without experimental evidence that participants engaged compositional reasoning during scanning, improved decoding of post-hoc compositional questions cannot support claims about compositional neural mechanisms. The authors are applying machine learning validation criteria (task performance) to neuroscientific claims (brain organization) without the experimental controls required for the latter.\nBoth the issues 1 and 2 could be addressed if voxel/region-wise encoding models (akin to Huth et al.) could be constructed for the concepts from the scene graph parsing and the predicted responses corresponding to the concepts from the model can be treated as grounding/representations and the rest of methodology could be applied (transformation into unary and binary embedding space and executor logic).\n\n3. Missing Evidence for Compositionality:\n\nWhile the paper implements compositional operations in its executor, it does not demonstrate compositional neural representations. Key missing evidence includes:\n\n3.1 Decomposability: True neuro-symbolic compositional systems (tensor products, vector symbolic architectures) maintain constituent structure that can be systematically recovered through inverse operations. In contrast, the executor combines groundings through non-invertible operations that destroy constituent information. There is no principled way to recover individual concept groundings from the final compositional output, demonstrating that the system implements functional composition without compositional representation. I consider the evaluator logic is a good conjunction recipe but not sure about neuro-symbolic composition though given lack of invertibility. And given that it resembles conjunctive representations, I wasn’t surprised that H5 showed greatest evidence.\n\n3.2 Systematic decomposition: While the authors demonstrate generalization to novel combinations by ensuring disjoint train/test sets, they lack fine-grained analysis of systematic recombination patterns. True compositionality requires systematic relationships between constituents and compounds across all concept types - for example, if the system learns holding(person, bat) and beside(person, tree), it should systematically handle holding(person, tree) and beside(person, bat) with predictable performance relationships. The paper shows aggregate performance on unseen combinations but doesn't analyze whether learned representations support the algebraic manipulation of concepts that characterizes genuine compositional systems. While dataset constraints may limit the extent to which systematic recombination tests are feasible, the authors don't report what subset of such analyses their data could support or acknowledge these limitations when making compositionality claims. Without demonstrating systematic performance patterns across specific recombination types (argument swapping, predicate transfer, role systematicity), the generalization results, while impressive, don't constitute strong evidence for compositional representation.\n\n4. Methodological Concerns:\n\nThe consistency metric (how often concepts ground to same regions) is presented as validation, but high consistency could simply reflect stable solutions to the classification task rather than meaningful neural organization (similar to the Haufe et al. critique above). Without independent validation against known neuroscience or comparison to null models with random concept assignments, these results cannot support claims about brain organization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B1hF1uKZFn", "reviewer": "Reviewer_FGNG", "summary": "This paper presents a neuro-symbolic framework for compositional concept grounding and reasoning using fMRI data. The model aims to decode not only object-level but also relational representations from regional brain activity. The authors evaluate NEURONA on 2 visual fMRI dataset, showing improved decoding and question-answering performance compared to previous fMRI decoding methods (SDRecon, BrainCap, UMBRAE). The work further analyzes the impact of multi-region grounding and relational reasoning through ablation studies.\n\nOverall, the paper is conceptually interesting and methodologically competent, but the empirical depth is relatively limited in statistical validation and subject generalization, and the interpretive link between symbolic composition and neural mechanisms could be made more rigorous. . Additional analyses are highly encouraged.", "questions": "1. Have you tried cross-subject evaluations?\n2. In the multi-region grounding setup, is there any redundancy or competition between regions (e.g., correlated activations leading to similar grounding weights)?\n3. How consistent are the learned concept groundings across subjects — do similar regions emerge for the same concept?\n4. Do claims of compositional grounding align with theories of hierarchical organization of the brain?\n5. How might this framework extend to other domains (e.g., motor planning)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well motivated to tackle the puzzle of interpretable concept grounding in neural data, with an emphasis on compositional structure.\n2. The results are consistent across two datasets and multiple concept types.\n3. The ablation analyses meaningfully test hypotheses about region-level grounding and compositional generalization.\n4. It positions neuro-symbolic modeling as a promising route to study structured neural semantics beyond pixel-level reconstruction.", "weaknesses": "1.The results in Tables 1–2 are trained and tested on one subject per dataset, which limits generalizability. Multi-subject or cross-subject validation is essential for neuroscientific conclusions.\n2. Grounding is restricted to coarse atlas parcels; voxel-level analyses would strengthen claims about spatial specificity of “modular concepts.”\n3. Results in table 1 and 2 are using models trained from one subject in each dataset\n4. It remains unclear how much each region or network contributes to concept decoding; e.g., whether “multi-region grounding” corresponds to interpretable brain modules.\n5. While means ± SD are reported, formal statistical tests are absent, so robustness to inter-individual variability is uncertain.\n6. Without ground-truth region–concept pairs, “consistent” grounding might reflect network-level co-activation rather than explicit compositional structure.\n7. Qualitative examples are interesting but could be complemented by quantitative measures (e.g., concept-wise accuracy)\n8. Writing is generally clear, though sections 3–5 are dense and occasionally repeat prior claims; discussion could better delineate cognitive vs. computational interpretations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K0JxdB7d1d", "reviewer": "Reviewer_U1Ce", "summary": "The MS introduces a neuro-symbolic fMRI-QA framework that links unary (objects) and relational (predicates) concepts to brain-region embeddings. They show that argument-guided learning of the predicates (i.e., conditioning on subject/object) improves decoding, especially for action/position queries. This is an interesting result indicating the system is role-sensitive and is learning from brain data something like event representations rather than argument-agnostic predicate maps. \n\nNote: I assess the work mainly for its **value in advancing understanding of brain data and brain-research methods**, rather than for engineering improvements in QA performance from fMRI decoders.", "questions": "I was wondering if the predicate representation in the unguided setting contains decodable information about the object arguments. If that is the case, it would hint that the predicate embedding/code binds to specific objects.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The system learns in a way that understands event semantics, i.e., decoding what is the subject doing to the object.  in fact, while the framing of the work is in terms of learning relational concepts, but I think it's equally fair to say they are learning event semantics, which are driven by relations applied to objects, and the objective is to learn the general semantics. \n2. The findings implicitly supports the neurobiological position *against* searching for a general (neurobiological) concept-brain-location in a context independent way, but conditionalizing it on the subject/predicate. This has a major implication for theories that aim to identify brain regions for 'verb meaning' independent of context\n3. The neuro-symbolic decomposition is clear and the MS is readable beyond the neuro-symbolic community.", "weaknesses": "* There is a main limitation for using the results to inform brain studies, grounded in the use of *Atlases*: using an atlas as the feature basis can (*i*) induce correlations among features because of spatial smoothing and parcellation choices (e.g., 2 brain areas might encode the image in the same way introducing collinearity among features), and (*ii*) allocate variance to parcels that are not informative with respect to the task domain, which limits interpretability and statistical sensitivity. \nTo my thinking, a better, domain-aligned approach is to start from image-by-voxel data and apply PCA (or other dimensionality reduction; e.g., dictionary learning) prior to training. This would match the feature space to the dataset, perhaps increase precision, and certainly provide more information on distributed brain networks relevant to the concepts. Currently, interpretation is forced to be in the Atlas domain, which is not a state of art basis for brain representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)