Beyond Grid-Locked Voxels: Neural Response Functions for Continuous Brain Encoding
Authors: Haomiao Chen
Keith W Jamison
Mert R. Sabuncu
Amy Kuceyeski
Venue: ICLR 2026 Poster (2026)
Keywords: Neural encoding model
Computational Neuroscience
Neuroimaging
Medical imaging
Implicit neural representation
Primary area: applications to neuroscience & cognitive science

Abstract: Neural encoding models aim to predict fMRI-measured brain responses to natural images. fMRI data is acquired as a 3D volume of voxels, where each voxel has a defined spatial location in the brain. However, conventional encoding models often flatten this volume into a 1D vector and treat voxel responses as independent outputs. This removes spatial context, discards anatomical information, and ties each model to a subject-specific voxel grid. We introduce the NRF Neural Response Function, a framework that models fMRI activity as a continuous function over anatomical space rather than a flat vector of voxels. NRF represents brain activity as a continuous implicit function: given an image and a spatial coordinate (x, y, z) in standardized MNI space, the model predicts the response at that location. This formulation decouples predictions from the training grid, supports querying at arbitrary spatial resolutions, and enables resolution-agnostic analyses. By grounding the model in anatomical space, NRF exploits two key properties of brain responses: (1) local smoothness—neighboring voxels exhibit similar response patterns; modeling responses continuously captures these correlations and improves data efficiency, and (2) cross-subject alignment—MNI coordinates unify data across individuals, allowing a model pretrained on one subject to be fine-tuned on new subjects. In experiments, NRF outperformed baseline models in both intrasubject encoding and cross-subject adaptation. Achieving high performance while reducing the data size needed by orders of magnitude. To our knowledge, NRF is the first anatomically aware encoding model to move beyond flattened voxels, learning a continuous mapping from images to brain responses in 3D space.

Published as a conference paper at ICLR 2026
BEYOND GRID-LOCKED VOXELS: NEURAL RE- SPONSE FUNCTIONS FOR CONTINUOUS BRAIN EN- CODING
Haomiao Chen Keith W. Jamison", Mert R. Sabuncu & Amy Kuceyeskil.5 Commell University, CComell Tech, Weill Cornell Medicine hc8 hc8728cornell. edu, [wj2001@mest cornell. edu,
mabuucuu@oorell..dd.. amx2012 20e20012@mm...mmmm cornell.edu
ABSTRACT
Neural encoding models aim to predict fMRI-measured brain responses to natural images. fMRI data is acquired as 3D volume of voxels, where each voxel has a defined spatial location in the brain. However, conventional encoding models of- ten flatten this volume into ID vector and treat voxel responses as independent outputs. This removes spatial context, discards anatomical information, and ties each model to subject-specific voxel grid. We introduce the NRF Neural Re- sponse Function, a framework that models fMRI activity as a continuous function over anatomical space rather than flat vector of voxels. NRF represents brain activity as a continuous implicit function: given an image and a spatial coordi- nate (3r, y, 2)= in standardized MNI space, the model predicts the response at that location. This formulation decouples predictions from the training grid, supports querying arbitrary spatial resolutions, and enables resolution-agnost -agnttsin- analyses. By grounding the model in anatomical space, NRF exploits two key properties of brain responses: (1) local smoothness smootnness-neiihboriigg voxels exhibit similar response patterns; modeling responses continuously captures these correlations and improves data efficiency, and (2) cross-subject alignment -MNI coordinates unify data across individuals, allowing model pretrained one subject to be fine-tuned on new subjects. experiments, NRF outperformed baseline models in both intrasubject encoding and cross-subject adaptation. Achieving high per- formance while reducing the data size needed by orders of magnitude. To our knowledge, NRF is the first anatomically aware encoding model move beyond flattened voxels, learning a continuous mapping from images to brain responses 3D space. Code and project site: https:: /ttp//ggttubbbb com/ haomi 108/NRE
INTRODUCTION
A major goal in computational neuroscience is to understand how the human brain maps visual stim- uli into neural activity. Neural encoding models aim to address this by predicting neural responses- typically measured by fMRI IMRI-from visual stimuli. These models offer powerful tools for analyzing high-dimensional brain data and probing the representations encoded in the visual system (Downing
al. 2001; Epstein & Kanwisher 1998 Gu et al. 2022 Heeger & Ress
2002
Kanwisher al.,
1997 Naselaris et al.] |0 et al. | 2012).
However, the real-world utility of current neural encoding models remains limited. Current neu-
ral encoding models represent fMRI responses as ID vector in RR,, where n is subject-specific
voxel count (Naselaris et al. 2015) St Yves Naselaris, 2018 Wang et al.) [0023] Yamins et al.
2014). This discrete formulation has two critical limitations: 1) Ignoring 3D structure. By flatten- ing [MRI volumes into ID vectors, conventional models discard spatial information. This removes local context: anatomically adjacent voxels, which are often functionally correlated, are instead treated as independent outputs. 2) Subject specific. Each model is tied to the voxel grid of a single subject, making it non- transferable across individuals. Because voxel counts differ across brains, the output dimensionality conventional models and their corresponding architectures are tied to a single subject. As result, knowledge learned from one individual cannot be directly transferred to

---

Published as a conference paper at ICLR 2026
Individual Subject NRF
Training Sunject
Frozen
Subject
Trainable
Subject coordinate
vaxel
Subject NRF model
Positional Encoding
Oe
Predicted subjectí vonel, response
19672
Image M
Imego embedding
Novel Subject Adaptation
Ensembiee model
Finetune
+
Sudgett made
KSF model
XW
Predicted new subject Vovel Response
Data
Figure 1: Overview of NRE. Top: Individual-subjeer NRF. Brain responses are modeled as a con- tinuous function of both image features and anatomical coordinates in MNI space. NRF maps image features voxel responses while capturing correlations between neighboring voxels through shared anatomical coordinates. Bottom: New-subject adaptation. For novel subject, start from pre- trained individual-subjec NRF (trained on the other subjects) and then fine-tune this model using only small amount of data from the new subject. The figure shows both the pretrained and the fine-tuned NRFs to make explicit how the base model adapted during transfer. Predictions from multiple fine-tuned base models then combined via voxel-wise ensembling to better capture subject-specific variability. NRF thus moves beyond grid-locked voxel models, providing a continu- ous, anatomically grounded representation that supports both data-efficient single-subject encoding and flexible cross-subject transfer.
another, forcing each new subject to require training a separate model from scratch. Consequently, these models fail to exploit the 3D geometry of brain activity, wasting statistical power and requiring more data learn accurate mappings.
These issues are particularly problematic in realistic applications, where data are scarce. Unlike large-scale efforts such as NSD (Allen et al., (022), which collected tens of thousands of trials per subject over year-long scanning period, most studies can only acquire few hundred trials per subject due to cost and time constraints. Thus, the inefficiencies of discrete neural encoding models amplified in real-world low-data regimes. In short, previous encoding models grid-locked: they can only predict responses at the discrete sampling points they were trained on, for a single subject, and at one resolution.
However, the human brain a continuous 3D structure. Within each subject, neighboring voxels also exhibit similar response patterns, reflecting the spatial smoothness of neural activity. Despite indi- vidual variability, the visual cortex is highly conserved across people :areas such as the fusiform face area (FFA) and extrastriate body area (EBA) consistently respond to the stimulus categories,
2

---

Published as a conference paper at ICLR 2026
and these regions align well across subjects when mapped into standardized anatomical templates such as MNI space (Heeger & Ress 2002 Kanwisher et al. 1997. Downing et al. 2001] Epstein &
Kanwisher 1998 Heeger & Ress 2002 Naselaris et al. [201] Huth et 2012). Ignoring this orga-
nization- both local smoothness and cross-subject correspondences - discards valuable structure
that could enable more efficient learning and better generalization.
To address these limitations, we propose the Neural Response Function (NRF), a coordinate- neural encoding model that predicts IMRI responses as a continuous function over anatomical space. Given a stimulus M and a spatial coordinate x = (I, y, 2) in standardized MNI space, NRF outputs the predicted brain response r at that location:
(M,, T. R E R.
This formulation directly addresses the limitations of previous models:
Exploiting local smoothness. By conditioning predictions on anatomical coordinates, NRF incorporates the 3D spatial structure of fMRI data. This allows nearby voxels that are anatomically connected and functionally correlated to share information instead of be- ing treated independent outputs. As result, NRF captures local smoothness in brain responses and achieves greater data efficiency.
Efficient cross-subject adaptation. NRF grounds predictions in standardized MNI space, unifying responses across subjects in shared coordinate system. This enables direct trans- fer: a model pretrained on one subject can be adapted to a new individual with only mini- mal fine-tuning, We further introduce a finetune-ensemble ensemble strategy that leverages multiple pretrained models to boost adaptation accuracy, reducing the need for extensive subject- specific data collection in real-world settings.
aggnosii modeling. By defining responses in continuous space, NRF decou- ples predictions from the specific voxel grid used during training. The model can be queried at arbitrary spatial coordinates, independent of the voxel size or sampling scheme used during data collection. This allows for the seamless integration of data acquired at varying resolutions and opens the door to building general-purpose brain models that move closer to a functional digital twin of the human visual system.
Through experiments, we demonstrate that NRF provides novel unatomy-grounde framework for neural encoding, offering new paradigm for efficient and generalizable brain modeling.
2 RELATED WORK
Neural encoding models. IMRI encoding models have been extensively studied over the past two decades (Mitchell et al. 2008 Huth et al., 2016 Gu et al., 2022 Tang al.) 2023 Kay et al,, 2008 Güçlü & Van Gerven 2015: Naselaris et al. 2015). Most existing approaches treat fMRI data as discrete, formulating the problem as a regression task that maps image features to voxel- wise responses (Naselaris et al., 2011: Han et al. 22019; Wang et al.
work has also explored stronger transformer-baue based encoders (Adeli et al.J 2025 Beliy et al., 2024). In these models, fMRI responses are flattened as 1-D vector, and each voxel is treated independently, ignoring the 3D anatomical structure of the brain. While some approaches incorporate spatial priors, such fitting multi- multi-parammttrr models for spatial frequency mapping (Broderick et al. 2022) or using Bayesian templates to warp retinotopic maps (Benson & Winawer] (2018). However, general encoding frameworks largely fail to exploit the local smoothness and cross-subject correspondences inherent in brain activity. To our knowledge, NRF is one of the first anatomically aware encoding models that formulates image to fMRI prediction mapping as a continuous function over 3D brain space, leveraging anatomical structure to improve data efficiency and generalization.
Neural decoding models. A parallel line work focuses on reconstructing or decoding visual stimuli from fMRI signals. Early approaches (Scotti et al. 2024) typically utilize subject-specific MLPs that cannot naturally generalize across individuals due to varying voxel counts and layouts.
Recent methods address this via adaptive pooling for cross-subject decoding (Wang et al.) 2024) or by incorporating voxel (x2y.z) coordinates into attention mechanisms via positional encodings (Quu et al. 2025). While these methods highlight the importance of anatomical alignment, they
3

---

Published as a conference paper at ICLR 2026
pursue fundamentally different goal: decoding information from measured, discrete voxel grids. Even with spatial cues, their operations remain tied to the specific voxels acquired. In contrast, we address the encoding problem by treating the brain continuous 3D structure. Rather than using coordinates as auxiliary features for discrete voxels, NRF models neural activity as a continuous function in brain space. This formulation enables resolution-agnosiic querying at arbitrary positions and facilitates seamless cross-subject adaptation within a unified anatomical coordinate system.
Brain semantic mapping. A rich line of research has mapped the semantic organization of the cortex using voxel-wise encoding models (Huth et al. 2016 Deniz et al. 2019). These studies typ- ically train independent encoders on flattened 1D response vectors and project learned selectivities onto the cortical surface post hoc for visualization. While this pipeline uses 3D spatial information during analysis, the encoding models themselves do not incorporate anatomical structure during training; each voxel is modeled independently, ignoring its physical location. NRF differs by in- jecting anatomical structure directly into the learning process. Rather than using coordinates solely for downstream visualization, NRF conditions the encoding function on 3D (z. y, 2) coordinates and learns a continuous neural field over standardized brain space. This allows the model to oper- ationalize the spatial principles revealed in prior studies, such as the fact that nearby voxels exhibit smoothly varying semantic selectivity. By modeling neural activity as spatially smooth function rather than a set of independent points, NRF achieves greater data efficiency and enables flexible cross-subject adaptation without the need to resample volumes onto shared grid.
Implicit neural representation Implicit neural representations((NR) have emerged as power- ful paradigm for modeling continuous signals in computer vision and graphics. Instead of storing data on fixed grids, INRs represent signals such as images (Sitzmann et al., 2020) and 3D shapes (Park al.) 2019 Mildenhall al., [202T] Chen & Zhang Mescheder 2019) as contin- uous functions parameterized by neural networks. A key advantage of this framework is its ability to capture fine-grained structure and support resolution-agnostic queries. Inspired by this line of work, we adopt a similar coordinate-based formulation for fMRI encoding. Unlike prior voxel-wise models that discretize the brain into subject-specific grids, our approach treats brain responses as continuous function over standardized anatomical coordinates. To our knowledge, NRF is the first attempt to bring the implicit representation framework to computational neuroscience, enabling anatomically aware, resolutiionagnostti modeling of fMRI responses.
METHOD
3.1 MODELING NEURAL RESPONSE AS IMPLICIT NEURAL REPRESENTATION
Current encoding models can be summarized in two steps: flatten neural response into 1D vectors, then train an encoding model that takes an image or its embedding as input and directly outputs the predicted response flattened vector in R" Ignoring the 3D spatial information and forcing models to be trained separately for each subject. This leads to poor data efficiency. Our key insight is that brain response should be modeled in its anatomical context. We represent the brain response mapping as a continuous function over MNI coordinates, a standardized anatomical space. For- mally, given a stimulus image M1 and spatial coordinate x (I, y, 2) E R the Neural Response Function (NRF) outputs the predicted fMRI response i E IR at that location:
( x) =
Rather than outputting a fixed-length vector tied to a particular subject's voxel grid, NRF predicts the response at any coordinate (z, y. 2) R3. This shift from ID discrete outputs to 3D spatial coordinate conditioned continuous predictions makes the model anatomically aware and able to exploit spatial smoothness during training and inference. Because & is defined over R3, it can be queried at arbitrary spatial resolutions, independent of the voxel grid or sampling scheme used during acquisition. This enables flexible data analysis: 1MRI responses can be resampled seamlessly at different resolutions, supporting resolution-agnostic modeling and analysis.
Architecture. We instantiate & using a two-component design. The first component, G, the image feature extraction block. It extracts multi-scale features from the stimulus image M, capturing both
4

---

Published as a conference paper at ICLR 2026
low-level and high-level representations. These features fused together to obtain final image embedding G(M). The second component an implicit neural representation predictor P that conditions on both ((M) and the spatial coordinate x. The coordinate is first encoded using Fourier features (Tancik et al. 2020):
(xx) = [cos( (2 x), sin (2Tb) x)...., cos(( sin sin(2
where by are sampled from an isotropic Gaussian. Finally, G(M) and 7(x) are concatenated and passed through an MLP predictor P:
(M, = P(G(M),
This design makes the mapping explicitly anatomy aware by conditioning on both image content and spatial location. Details can be found in the Appendix A.I
Model Training. The model trained end-to-end with all components learned together, where the objective of the model is to correctly predict the voxel activation on each input image. Training batches are constructed from 32 randomly selected images, where for each image, we randomly sample 2000 voxels (out of 13000-15000 voxels) along with their corresponding fMRI activations for prediction. The model is trained using the Adam optimizer with a learning rate of 3e-3, For the loss function, we employ the same loss in (Beliy al. 2019), convex combination of mean square error and cosine similarity between the predicted response f and ground truth fMRI, r. The fMRI loss is defined as:
((f,r) = a)lf + +a* cos(
Where a is set to 0.1 during training, which balances absolute error minimization (via MSE) with representational alignment (via cosine similarity).
3.2 CROSS SUBJECT TRANSFER
A major challenge in training visual encoding models is the limited availability of subject-specific data. Collecting fMRI responses for thousands images requires many hours of scanning, often across multiple sessions, and is infeasible in most clinical experimental settings. In practice, new subjects often contribute only a few hundred trials. Discrete neural encoding models underperform in this regime because they tied subject-specific voxel grids: each subject requires training new model from scratch, and knowledge cannot be transferred directly across individuals.
NRF overcomes this limitation by being voxel-grid agnostic. Since responses are defined con- tinuous function over standardized MNI space, subjects are naturally aligned in shared anatomical coordinate system. This enables direct transfer: model trained on one subject can be adapted to another without voxel-wise resampling, on the new subject's coordinates and responses. Unlike clas- sical voxel-wise shhhelh-whihh rigidly tie the representation to subject-specific grid-NRk -NRF learns a continuous, anatomically grounded representation that could flexibly generalize across individuals with only minimal data. To exploit this property, we adopt a two-step adaptation strategy:
Finetuning. A pretrained NRF is fine-tuned on the new subject's limited data, using their MNI coordinates and measured responses. The two components of NRF & are the feature extractor G and MLP predictor P. G encodes the visual stimulus into representation. while P maps this representation and the spatial coordinate to the predicted brain response. Both G and P benefit from adaptation, since individuals vary in both how visual content is processed and how it is mapped to anatomy. Therefore, we perform full end-to-end finetuning of both components on the new subject's data.
voxel-wise Ensemble. To further improve performance on new subjects, we perform a voxel-wist ensemble of the predictions from different finetuned models. Similar the personalized ensem- ble approach in (Gu et al. 2022). this strategy maximizes predictive performance while preserv- ing inter-subject variability improve model personalization. Specifically, for each voxel v, let (F(...) =((.1) ,...,,!! ((K.0) denote the predictions of finetuned base models for the ith image. We then learn voxel-specific weights w.,k (one per base model, for the th base model) and bias b.
5

---

Published as a conference paper at ICLR 2026
by solving least-squares regression on the limited new subject training data:
min (mock Duy
Woek p(R,)) by)
where 766) the measured response voxel v to image i, and the number adaptation samples, At inference time, the final prediction for voxel v is given by the weighted ensemble:
Wo,k p(h) + b
This ensemble leverages common neural structure while accounting for subject-specific variability, yielding higher accuracy in the low-data regime and producing a more personalized model for each subject.
EXPERIMENTAL SETUP
4.1 DATASETS AND PREPROCESSING
We the Natural Scene Dataset (NSD) (Allen al. 2022), which includes whole-brain 7T fMRT data from 8 subjects who viewed 10,000 natural scene images from the MS COCO dataset, repeated 1-3 times. The brain activations were computed using the GLMSingle algorithm (Prince et al. 2022). and each voxel's response value is normalized per session (μ = 0, 1). The brain activation to repeated images within subject was averaged. The Neural response function(NRF) was trained using 9,000 unique images per subject, with around 1,000 images used for testing model accuracy via voxel-wise pearson correlation (r). Since we are focusing on the visual cortex regions, apply the official nsdgeneral region-offinterest (ROI) mask, which spans visual regions ranging from the early visual cortex to higher visual areas. Our evaluation focuses on Subj01, Subj02, Subj05, and Subj07 because these subjects completed all experiment sessions.
4.2 EVALUATION METRICS.
To quantitatively compare with other models, we assess model performance across two levels.
Voxel-Level Metrics: To quantify prediction accuracy, we compute the voxel-wise Pearson cor- relation (r) and voxel-wise mean square error (MSE) across all testing images.
Semantic-Level Metrics. Following prior work (Bao et al. 2025), we evaluate the semantic fi- delity of predicted responses using MindEye2 (Scotti et al. (024). pretrained fMRI-to-image de- coder. Given input image, we first predict its fMRI response using NRF; the predicted response then fed into MindEye2, which reconstructs the corresponding visual stimulus. We compare these reconstructions against the ground-trute images presented during data collection. We employ suite of image reconstruction metrics. PixCorr and SSIM quantify low-level visual fidelity, while Alex(2) and (5) measure similarity in early and deeper layers of AlexNet. To evaluate higher-level mantic alignment, we compute Incep, CLIP, Eff, and SWAV scores, which assess the representational correspondence between reconstructed and original images in diverse semantic embedding spaces. Additional details are provided in Appendix A.3
5 RESULTS
5.1
INDIVIDUAL SUBJECTS ENCODING
We first evaluated NRF's neural prediction capability for single-subject data. Training a separate model for each of the 4 subjects and comparing the average neural prediction accuracy across sub- jects. For comparison, we selected two representative encoding models as baseline comparison. The linear regression model from the BrainDIVE (Luo al. 2023) and the fWRF (Feature-Weighted

---

Published as a conference paper at ICLR 2026
LEAM
0.2
Mamber Traiiing inager
Figure 2: Prediction accuracy (Pearson correlation r) in low data regime. Single-subject mod- els. NRF consistently outperforms baseline models when trained on limited samples from scratch, highlighting the benefit of its continuous mapping Seappngg.Results are shown for the mean of the median voxel correlation across four subjects, with error bars indicating the standard error of the mean (SEM). b. Cross-subject transfer. Voxel-level prediction accuracy visualized face of subject 7. When pretrained base models from other subjects are available, the NRF finetune ensemble further improves performance over NRF scratch and baselines, showing clear gains across visual regions.
the cortical sur-
Receptive Field) encoder (St- Yves & Naselaris 2018). Details about the baseline model are in Ap- pendix A 2 We also took the result for full data encoding performance from MindSimulator (Bao
al. 2025
We first evaluate NRF under limited-data conditions, since practical applications rarely have ac- cess to the tens of thousands of trials collected in large-scale datasets such as NSD. As shown in Figure p. NRF achieves significantly higher accuracy than baseline models when trained on small numbers of images. Remarkably, with only 200 training samples, NRF outperforms baselines trained on more than 800 images. We attribute this data efficiency to the anatomical awareness of NRF: by conditioning on spatial coordinates, the model can exploit the smoothness of fMRI sponses and learn more effectively from scarce data. This neuroscience-inspired design makes NRF particularly well-suited for realistic, low-data regimes.
Next, we evaluate NRF in the full-data setting(~99 training images). Quantitative results, sum- marized in Table [ show that NRF outperforms baselines on voxel-wise prediction metrics while achieving comparable performance on semantic-level evaluations. In addition, we observed that some baselines, such as fWRF, achieve unusually high semantic-level scores, some cases even surpassing reconstructions from measured IMRL. We attribute this to decoder bias: IWRF outputs, while less neurally accurate, may align more closely with the pretrained decoder's distribution, thereby inflating semantic metries. These results indicate that semantic-level metries should be in- terpreted as a coarse indication of reconstruction quality rather than strict basis for comparing encoding models. Further discussion regarding these metries is included in Appendix[A.3
Figure provides a qualitative assessment via image reconstructions: the decoded images capture low-level visual features and high-level semantic categories with high fidelity to the ground truth. These results demonstrate that NRF maintains high voxel-level accuracy while also preserving se- mantic information, confirming its effectiveness across both limited and full data regimes. Ap- pendix A.4 presents detailed per-subject results along with the voxel-wise accuracy distributions, and Appendix A.5 reports the statistical significance analyses.
5.2 NEW SUBJECT ADAPTATION
More importantly, NRF enables cross-subject transfer, allowing knowledge learned from one subject to be adapted to new subjects-a a critical property given that collecting fMRI data for new individu- als is both resource-intensive and time-consuming. To evaluate this capability, with 20, 200, and 800 images, corresponding to approximately 4, 40, and 160 minutes of scanning time. Three subjects used for pretraining base models, and fourth subject held out for
tested adaptation

---

Published as a conference paper at ICLR 2026
Method
Voxel-Level
Semantic-Level (via decoding)
Pearsonf
MSEL
PixCortt
SSIMI
Alex(2)t
Alex(S)
IncepT"
CLIPT
Eff.
SwAV]
Measured IMRI
0.322
0.431
96.1%
98.6%
95.4%
93.0%
0.619
0.344
Linear Regression
0.323
0.353
0.186
0.271
86.1%
95.0%
90.2%
84.5%
0.750
0.417
PWRF
0.343
0.361
0,303
0.341
96.9%
99.1%
96.2%
91.9%
0.614
0.356
MindSimulator (Trials=1)
0.345
0.403
0,194
0.296
89.0%
96.2%
92.3%
90.3%
0.702
0.399
MindSimulator (Trials=5)
0.355
0.385
0.201
0.298
89.6%
96.8%
93.2%
91.2%
0.688
0.393
NRF (our method)
0.358
0.345
0.261
0.371
91.6%
96.3%
92.1%
89.3%
0.706
0.400
Table 1:
Evaluation results of fMRI prediction accuracy for the model trained the full dataset. All reported voxel-level metric values are reported as the per-subject median across voxels, and the table shows the mean of these medians across the subjects. Additional subject-wise results and full per-voxel distributions are provided in Appendix A.4
GT
Measured IMRI
Linear regression
PWRF
NRF(ours)
Figure 3: Visualization comparison between different neural encoding models and NRF. GT seen during data collection. Measured fMRI = decoded image using measured fMRI. Reconstructions from PRE-predected responses preserve both low-level visual details and high-level semantic content of the stimuli. Results shown for Subject 1.
adaptation. For the new subject, we applied fine-tuning followed by voxel-wise regression ensemble using the limited data. As baseline, we compared against the "NRF scratch" approach, where a new NRF is trained entirely from the same limited dataset without pretraining. Across all data conditions, fine-tuning ensemble consistently outperformed NRF scratch, confirming that NRF's anatomically grounded formulation enables efficient cross-subject transfer, reducing the need for ex- tensive subject-specific data while maintaining high predictive fidelity. The qualitative comparison is shown in Table Prediction accuracy comparison across different methods is shown in Figure 2p.. Notably, in the very low-data regime, finetuning ensemble achieved strong semantic-level decod- ing performance. This shows that the strategy not only improves voxel-wise prediction but also preserves subject variability, enabling predicted responses that more faithfully capture the semantic content of visual stimuli.
5.3 PROBING ANATOMICAL AWARENESS
To verify the mechanisms behind NRF's performance, particularly in data-constrained settings, we probe its reliance on two fundamental properties of fMRI data: (i) the local spatial continuity of voxel responses within a subject, and (ii) the anatomical alignment across subjects. We conduct controlled ablation experiments by introducing structural perturbations that systematically disrupt these factors. NRF's efficiency stems from its anatomical grounding, performance should degrade significantly when these structural priors violated.
8

---

Published as a conference paper at ICLR 2026
Training Images
Method
Voxel-Lovel
Semantic- Levell (via decoding)
Peanson]
MSE,
PixCorri
SSIMT
Alex/2)t
Alex(5)1
Incep?
CLIP1
BM]
SwAVU
Full
NRF subject 7 (all data)
0.269
0.348
0.244
0.367
0.880
0.936
0.892
0.846
0.768
0.445
NRF scratch
0.076
0.417
0.060
0.195
0.597
0.545
0.962
NRF finctune ousembbe
0.114
0.445
0.186
0.366
0.750
0.792
0.732
0.729
0.868
0.515
NRF scratch
0,180
0.394
0.159
0.284
0.760
0.813
0.774
0.716
0.857
0.315
NRF finetune ensemble
0.227
0.390
0.255
0.372
0.908
0.957
0,913
0.873
0.729
0.425
NRF sernich
0.220
0.376
0,188
0.313
0.856
0.926
0.878
0.834
0.772
0.452
NRF finetune cusemble
0.251
0.372
0.269
0.382
0.927
0.970
0.922
0.895
0.700
0.408
Table 2: New-subject adaptation with limited data (20, 200, 800 images). Results are shown for adapting the NRF pretrained on subjects 1, 2, and 5 to subject 7 using finetuning + ensemble. Per- formance is reported as the median across all voxels adapted NRF consistently outperforms training from scratch, and with only 200 images, it even ex- ceeds the performance of model trained on the full dataset.
subject for voxel-level metries. The
Shuffle coordinate
Shifting coordinate
Shifting coordinate
a
b
EBA
V1
sndveaie stufts
206
189
000
Number of training images
Figure 4: Probing anatomical awareness in NRF. (a) Disrupting spatial smoothness by shuffling co- ordinate-response pairings reduced accuracy, especially in low-data regimes, confirming that NRF relies on local continuity in brain responses. (b)(c) Breaking cross-subject alignment by shifting MNI coordinates degraded transfer, with the largest effect under limited data, showing that anatom- ical correspondence is critical for efficient adaptation.
Disrupting Local Smoothness via Voxel Shuffting. To test whether NRF's data efficiency stems from exploiting spatial continuity, disrupted the natural smoothness of IMRI data by shuffling co- ordinate-response pairings. Voxel responses were randomly reassigned to MNI coordinates, break- ing correlations between neighboring voxels. We performed two variants of this perturbation: (i) global shuffling, randomizing pairings across the entire visual cortex, and (ii) ROI-wise shuffling, randomizing only within each ROI. Traditional voxel-wise models should be unaffected, since they treat voxels independently. In contrast, NRF relies on coordinate conditioning, and as expected, its performance dropped sharply in low-data regimes, with global shuffling producing the largest drop. This confirms that NRF's improvements are driven by its ability to leverage local smoothness in brain responses. Shown in Figure 41a).
Disrupting Cross-Subject Alignment. To the importance of anatomical correspondence for transfer, we disrupted MNI alignment by shifting voxel coordinates between subjects. Specifically, a model pretrained on Subject was finetuned on Subject 7 using responses from EBA and V1. Dur- ing finetuning. the MNI coordinates were shifted while remaining within the subject's 's brain range, breaking the cross-subject anatomy alignment. Compared to finetuning with aligned coordinates, coordinate shifting substantially degraded cross-subject transfer. The effect was most pronounced in low-data regimes: with only a small number of finetuning samples, the misaligned model failed to adapt, whereas alignment enabled effective transfer. With more data, the model gradually com- pensated for the misalignment, but still required far more samples to match the aligned case. These results demonstrate that NRF's cross- subsett generalization depends critically ment. Without it, transfer is possible but far less data-efficient. To avoid artificial overlap after shifting, finetuning performed using ROI-restricted data rather than the full brain.
anatomical align-

---

Published as a conference paper at ICLR 2026
Method
Vexel-Level
Semantie-Lvel (via decoding)
Pearson?
MSEU
PixCort
SSIMT
Alex(2y1
Alcx(5it
IncepTt
CLIPT
Em1
SwAVL
NRF finetune cusemble
0.227
0.390
0.255
0.372
0.908
0.957
91.3%
87.3%
0.729
0.425
NRT finetune average
0.253
0.367
0.167
0,283
0.784
0.852
80.4%:
74.9%
0.848
0.514
NRF finetune hase (subj] subj))
0.386
0.375
0.897
90.8%
0.735
NRT finetune bave (suby2 subg7)
0.232
0.379
0.243
0.366
0,885
0,937
87.1%
82.4%
0.779
0.457
NRF linetune base (suby5 subb7))
0.389
0.874
82.4%
0.775
Table 3: Ablation on voxel-wise regression ensemble. We report the result for adapting from subjects 1,2,5 to subject 7 with 200 images.
5.4 ABLATION STUDY
voxel-wise ensemble A key component for new subject adaptation is voxel-wise regression en- semble, where each voxel is fit with a linear regression model to optimally combine predictions from multiple fine-tuned base models. This approach improves prediction accuracy while preserv- ing subject-specifie variability. Table 3 compares voxel-wise regression against single fine-tuned base models and simple averaging. While simple averaging slightly boosts voxel-wise prediction accuracy. it hinders subject variability and produces predicted fMRI signals with reduced semantic fidelity, leading to lower decoding performance. In contrast, voxel-wise regression leverages com- plementary information across base models in flexible, voxel-specific way, achieving both higher voxel-level accuracy and stronger semantic-level decoding results. This suggests that the ensemble does not merely act as noisetreduction filter but effectively "mixes" specialized knowledge from different source subjects to better represent the unique functional profile of the target individual. Additional ablation results are included in the Appendix A.7
CONCLUSION
In this work, we introduced the Neural Response Function (NRF), an anatomically aware neural en- coding model that represents fMRI activity as a continuous function over MNI coordinates. Unlike conventional voxel-wise models, NRF leverages spatial smoothness and cross-subject alignment to achieve accurate predictions in low-data regimes and to support efficient subject adaptation. Cru- cially, its continuous formulation moves beyond grid-locked voxels, allowing predictions arbitrary spatial resolutions and across individuals. In this sense, NRF serves as a resolution-agnoss twin of the brain: a unified, flexible representation that integrates data across scales and subjects. These advances offer path toward efficient, generalizable, and anatomically grounded neural encoding.
digital
REFERENCES
Hossein Adeli, Minni Sun, and Nikolaus Kriegeskorte. Transformer brain encoders explain human high-level visual responses. arXiv preprint arXiv:2505./7329, 2025. 2025.
Emily J Allen, Ghislain St- Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al. A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1): 15(1):116-126, 16-126,
2022.
Guangyin Bao, Qi Zhang, Zixuan Gong, Zhuojia Wu, and Duoqian Miao. Mindsimulator: Exploring brain concept localization via synthetic fmri. arXiv preprint arXiv:2503.0235/, 2025.
Roman Beliy, Guy Gaziv, Assaf Hoogi. Francesca Strappini, Tal Golan, and Michal Irani. From voxels to pixels and back: Self-supervision in natural-image reconstruction from fmri. Advances in Neural Information Processing Systems, 32, 2019.
Roman Beliy, Navve Wasserman, Amit Zalcher, and Michal Irani. The wisdom a crowd of brains: A universal brain encoder. arXiv preprint 2tt.:2400.22299999 2024.
Noah Benson and Jonathan Winawer. Bayesian analysis of retinotopic maps. elife, 7:c40224, 2018.
10

---

Published as a conference paper at ICLR 2026
William F Broderick, Eero P Simoncelli, and Jonathan Winawer. Mapping spatial frequency prefer- ences across human primary visual cortex. Journal of Vision, 2 2022.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 5939-5948, 2019.
Fatma Deniz, Anwar O Nunez-Elizalde, Alexander G Huth, and Jack L Gallant. The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality. Journal of Neuroscience, 39(39):722--7736 2019.
Paul E Downing, Yuhong Jiang, Miles Shuman, and Nancy Kanwisher. A cortical area selective for visual processing of the human body. Science, 293(5539):2470-222 2001.
Russell Epstein and Nancy Kanwisher. A cortical representation of the local visual environment. Nature, 39266676)55986600, 1998.
Zijin Gu, Keith Jamison, Mert Sabuncu, and Amy Kuceyeski. Personalized visual encoding model construction with small data. Communications Biology, 5(1):1382, 2022.
Umut Güçlü and Marcel AJ Van Gerven. Deep neural networks reveal gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27):10005-10014 2015.
Kuan Han, Haiguang Wen, Junxing Shi, Kun-Han Lu, Yizhen Zhang, Di Fu, and Zhongming Liu. Variational autoencoder: An unsupervised model for encoding and decoding fmri activity in visual cortex. Neurolmage, 198: 128-1623 2019.
David J Heeger and David Ress. What does fmri tell us about neuronal activity? Nature reviews neuroscience, 3(2):142-155) 2002.
Alexander G Huth, Shinji Nishimoto, An T Vu, and Jack L Gallant. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6): 1210-124,, 2012.
Alexander G Huth, Wendy De Heer, Thomas Griffiths, Frédéric Theunissen, and Jack Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532 (7600):453-44)) 458, 2016.
Nancy Kanwisher, Josh McDermott, and Marvin M Chun. The fusiform face area: module in human extrastriate cortex specialized for face perception. Journal of neuroscience, ((11):4302- 4311, 1997.
Kendrick N Kay, Thomas Naselaris, Ryan Prenger. and Jack Gallant, Identifying natural images from human brain activity. Nature, 452(71855:352-355, 2008.
Andrew Luo, Maggie Henderson, Leila Wehbe, and Michael Tarr. Brain diffusion for visual explo- ration: Cortical discovery using large scale generative models. Advances in Newral Information Processing Systems, 36:75740-75781 2023.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Oc- cupancy networks: Learning 3d reconstruction in function space. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460 14700 2019.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Tom M Mitchell, Svetlana V Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L Malave, Robert A Mason, and Marcel Adam Just. Predicting human brain activity associated with the meanings of nouns. science, 320(5880):1191-111 2008
Thomas Naselaris, Kendrick N Kay, Shinji Nishimoto, and Jack Gallant. Encoding and decoding in fmri. Neuroimage, 56(2):400-410, 2011.

---

Published as a conference paper at ICLR 2026
Thomas Naselaris, Cheryl A Olman, Dustin E Stansbury, Kamil Ugurbil, and Jack L Gallant. A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes. Neuroimage, 105:215-201 2015.
Jeong Joon Park. Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 165-174, 2019.
Jacob Prince, lan Charest, Jan W Kurzawski, John A Pyles, Michael Tarr, and Kendrick Kay. Improving the accuracy of single-trial fmri response estimates using glmsingle. Elife, 11:777599, 2022.
Weikang Qiu, Zheng Huang, Haoyu Hu, Aosong Feng, Yujun Yan, and Rex Ying. Mindllm: A subject-agnostic and versatile model for fmri-to-text decoding. arXiv preprint 25X70 2025.
Paul S Scotti, Mihir Tripathy, Cesar Kadir Torrico Villanueva, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. Mindeye2: Shared-subjec, models enable fmriito-image with 1 hour of data. arXiv preprint arXiv:2403.1 / 1207, 2024.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation funetions. Advances in neural information processing systems, 33:7462-74733 2020.
Ghislain St-Y and Thomas Naselaris. The feature-weighted receptive field: an interpretable encoding model for complex feature spaces. Neurolmage, 580:188-202, 2018.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Kridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
Jerry Tang, Meng Du, Vy Vo, Vasudev Lal, and Alexander Huth. Brain encoding models based multimodal transformers can transfer across language and vision. Advances in Neural Information Processing Systems, 36:29654-29666, 2023.
Aria Y Wang. Kendrick Kay, Thomas Naselaris, Michael Tarr, and Leila Wehbe. Better models of human high-level visual cortex emerge from natural language supervision with large and diverse dataset. Nature Machine Intelligence, (415):1415-1426, 2023.
Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subjes crose-subject brain decoding framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11333-1 1342, 2024.
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. Performance-optiminn hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 1 111(23)88619-88)) 8624, 2014.
12

---

Published as a conference paper at ICLR 2026
A
APPENDIX
Sections
1. Additional details on NRF A.1
2. Additional details baseline models
A.2
2
3. Additional details evaluation metrics
A.3
4. Additional results on single subject NRF
A.4
5. Statistical significance analysis A.5
6. Additional results on new subject adaptation
A.6
7. Additional ablation results
A.7
8. High-Resolution NRF Querying A.8
13

---

Published as a conference paper at ICLR 2026
A.1 ADDITIONAL DETAILS ON NRF
Image Feature Extraction Block. We leverage the pretrained OpenAl CLIP ViT-B/16 to obtain multiscale image features. Representations are extracted from the 3rd and 6th transformer layers, each yielding features of shape (196x 768), along with the final CLIP embedding of shape (1 512). The two intermediate feature maps are each processed by separate two-layer projection modules with identical architecture: the first layer reduces the dimensionality from (196 x 768) to (196 x 256), and the second compresses this to (1 256) vector. These two compact embeddings are then concatenated with the (1 x 512) CLIP embedding to form the fused multiscale image representation G(M).
MLP Predictor. The predictor is goordinate-conditionn MLP that takes input both the Fourier positional encoding of the MNI coordinate and the fused image embedding G(M). It out- puts single scalar-the predicted fMRI response that voxel location. We an 8-layer MLP with hidden dimension 4096, applying ReLU activations after each layer except the final output. Further ablations on model architecture are included in Appendix A.7
A.2 ADDITIONAL DETAILS ON BASELINE MODELS
FWRF encoding model The encoder uses AlexNet the base feature extractor, processing 227x227 pixel input images normalized to [0.1]. Feature selection retains the top 256 features per layer based on variance across the training data. The receptive field model employs 3x3 spatial grid with aperture size 0.8, covering RF sizes from 0.15 to 0.25 across 2 logarithmically-spann scales, yielding 18 total RF candidates per voxel. Ridge regression optimization uses regularization parameters À E [10*, 109] with adaptive holdout validation.
Linear Regression Encoding. For linear regression baseline we used the same encoding model as (Luo et al.) 2023). Specifically, we extract the (1 x 512) CLIP embedding from OpenAl CLIP V/T-B/16 and directly map it to the voxel dimension (e.g.. 15,724 voxels) using linear layer. The model is trained for 150 epochs with the Adam optimizer, with a learning rate that decays linearly from x 10-1, During inference, we select the checkpoint that achieves the lowest validation MSE.
14

---

Published as a conference paper at ICLR 2026
A.3 ADDITIONAL DETAILS ON EVALUATION METRICS
We used the evaluation metrics for decoded image evaluation from MindEye2 (Scotti et al. 2024) directly. Given input image, we first predict its fMRI response using NRE; the predicted response is then fed into MindEye2, which reconstructs the corresponding visual stimulus. We compare these reconstructions against the ground-trute images presented during data collection.
PixCorr measures the pixel-wise correlation between the ground-truth image and the reconstruc- tion. SSIM refers the Structural Similarity Index, which evaluates perceptual similarity between ground-truth and reconstructed images. Alex(2), Alex(5), Incep, and CLIP are two-way identifica- tion metries (chance = 50%) based on feature similarity. Specifically, Alex(2) uses features from the 2nd layer of AlexNet, Alex(S) from the 5th layer of AlexNet, Incep from the final pooling layer of Inception V3, and CLIP from the final layer of CLIP ViT-I/14. In two-way identification, the task is to decide whether the voxel embedding is closer to its paired image embedding selected image embedding, reported as percent correct. Eff and SwAV denote representational simi- larity metries, computed as the average correlation distance between voxel embeddings and features extracted from EfficientNet-Bi B1 and SwAVVRessNe50,, respectively.
randomly
Interpretation of Semantic-Level Metries The semantic-level reconstruction metrics reported in the main paper (PixCorr. SSIM, AlexNet features, Inception, CLIP, EfficientNet, SwAV) are com- puted using the MindEye2 fMRI-to-image decoder, which reconstructs visual stimuli from predicted fMRI responses. We included these semantic metrics to provide additional qualitative insight into the behavior of the generated or selected images, following the evaluation format adopted in recent prior work, including MindSimulator (Bao et al.) 2025), published at ICLR 2025. Our intention was to maintain consistency with existing evaluation practices, rather than to treat these metrics as decisive evidence for model comparison. While useful for assessing whether predicted responses support plausible reconstructions, these metrics therefore require careful interpretation.
Because MindEye2 is itself a trained neural decoder, it introduces its representational biases. As a result, we observe counterintuitive cases in which fWRF predictions score higher than the actual measured fMRI responses that TWRF predictions are more neuronally accurate; rather, it reflects that the decoder's internal feature space more compatible with certain statistical properties of the predicted responses than with the true fMRI signals.
semantic reconstruction metrics. This does not imply
This behavior demonstrates that semantic-level decoding metrics are not suitable primary mea- sures of encoding quality. Instead, they should be viewed as diagnostic tools for assessing whether predicted responses can support plausible image reconstructions.
Accordingly, all core model comparisons and statistical conclusions in the paper rely exclusively on voxel-level encoding metries (e.g., Pearson correlation), which directly measure correspondence to ground-trum neural responses and are unaffected by decoder bias. Semantic reconstruction results are included for completeness and visualization, but should not be interpreted as primary indicators of model performance.
15

---

Published as a conference paper at ICLR 2026
A.4 ADDITIONAL RESULTS SINGLE SUBJECT NRF
Here we report the single subject NRF evaluation for all subjects (Subject 1, 5, 7) in table 4 and the Mean SEM across subjects in table
Voxel-Level
Semantie-Leve (via decoding)
Subject
Method
Pearson"
MSET
PixCont
SSIMT
Alex(2)1
Alex(51f
Incep'l1
CLIPT
E11]
SwAV
Subject
Linear Regression
0.311
0.364
0.180
0.243
86.0%
94.3%
89.8%
84.2%
0.759
0.428
OWRF
0.341
0.363
0,304
0.342
96.8%
99.0%
96.2%
92.0%%
0.615
0.357
NRF (ours)
0.361
0.349
0.324
0.387
95.6%
98.3%
94.1%
91.9%
0.680
0.396
Subject
Linear Regression
0.323
0.358
0.164
0.253
85.2%
94.2%
89.5%
83.9%.
0.764
0.424
[WRF
0.347
0.364
0.219
0.222
96.9%
98.9%
95.3%)
91.1%
0.635
0.359
NRF (ours)
0.368
0.347
0,240
0.351
89.3%
95.4%
88.1%
87.5%
0.767
0.442
Subject
Linear Regression
0.378
0.340
0.201
0,302
87.0%
97.6%%
91.0%
85.2%
0.734
0.405
IWRF
0.413
0.357
0,345
0,403
96.7%
99.1%
96.9%
92.22%
0.599
0.352
NRF (ours)
0.425
0.346
0,236
0.379
93.5%
97.9%%
97.1%
93.1
0.609
0.317
Subject
Linear Regression
0.267
0.348
0.197
0.285
86.3%
94.1%
90.6%
84.7%:
0.745
0.412
DWRF
0,269
0.360
0.346
0.399
97.4%
99.3%
96.3%)
92.2%
0.609
0.357
NRF (ours)
0.278
0.328
0.244
0.367
88.0%
93.6%
89.2%:
84.6%
0.768
0.445
Table 4: Single-subject NRF results for S1, S2, S5, and S7. For voxel-level metries, the median values are reported.
Marhedl
Peasuet
MSB
PoCher
SS0MT
CUIP
FiN]
SwoAV
Linzar Regrossoe
0,323 0.211
0.353
0.005
0.186 a 0.00
0.371 0014
86 16.19:0.5
00.2:103%
84.35
0.250
0.007
0-417 0005
PWRR
0.039
0:301
0.000
0.305
0.000
WAS500 76
UA14
0.08
0.011
Table 5: Mean + SEM across all subjects. For voxel-level metrics, the mean of the per-subject medians is reported.
NRF Validation Performance per Subject
0.8
IQR (25th 75th percentile)
Median
Pearson Correlation (r)
0.2
0.1
0.0
$2
S5
S7
Subject
Figure 5: The distribution of the predicted voxel accuracy for 25th 75th percentile for each subject
16

---

Published as a conference paper at ICLR 2026
A.5 STAT SIGNIFICANCE ANALYSIS
This section provides the statistical procedures used to evaluate voxel-wise prediction accuracy. compare models, and analyze robustness across training-set sizes and ROI ablations. All analyses were repeated across multiple training sizes (20-800 images and full data; Fig. 2 For the ROI ablation experiments (Fig. we applied the same paired permutation testing framework compare performance between the original and spatially perturbed coordinate conditions.
voxel-wise model comparison via paired permutation tests. To determine whether NRF signifi- cantly outperforms the baselines, perform voxel- -oxel-wise paired permutation tests between all model pairs (NRF vs. IWRF and NRF vs. Linear). For each voxel, we compute the correlation difference and evaluate its significance by constructing null distribution from 10,000 sign-flip permutations. Two-tailed p-values are corrected using the Benjamini- FDR procedure (a = 0.05), and effect sizes are reported using Cohen's d. Figure 2a reports the subject-level median correlations, while the full statistical results are provided in the following sections. Across all subjects, NRF shows significantly higher voxel-wise correlations than both baselines (DFDR < 0.05), with medium- ta-large effect sizes.
we apply the same paired permutation test to compare NRF performance under the original versus shuffled or shifted ROI coordinate conditions, allowing us to assess whether performance depends on anatomically aligned spatial inputs rather than subject-specific voxel patterns.
For the coordinate ablation experiments Fig.
Voxel-level predictive significance testing. We also assess whether each model produces statis- tically reliable voxel-wise predictions. For each subject and model, Pearson correlations between predicted and ground-truth responses are tested using the correlation t-test, followed by BH-FDR correction (a 0.05). NRF consistently yields larger fraction of significant voxels than both baselines across all subjects, and this pattern holds across all training sizes.
For the ROI ablation experiments (Fig. 4) perform the same voxel-level significance quantify how many voxels remain significant when coordinates are shuffled or spatially shifted. This allows evaluate whether anatomical misalignment degrades the reliability of voxel-wise predictions within targeted functional regions.
17

---

Published as a conference paper at ICLR 2026
A.5.1
SINGLE SUBJECT MODEL STATISTICS ANALYSIS (FIG A
Paired permutation test
Training images
Subject
d (NRF IWRF)
PFDR
Sig?
d (NRF Linear)
PEDR
Sig?
20
S1
0.40
0.000
True
0.48
0.000
Truc
20
S2
-0.01
0.126
False
0.37
0.000
True
20
S5
-0.22
0.000
True
0.14
0.000
Truc
20
S7
0.06
0.000
True
0.24
0.000
True
60
S1
0.21
0.000
True
0.71
0.000
True
60
S2
-0.12
0.000
True
0.50
0,000
True
60
S5
0.08
0.000
True
0.59
0.000
True
60
S7
0.06
0.000
True
0.57
0.000
Truc
200
$1
0.36
0.000
True
0.74
0.000
Truc
200
S2
0.20
0.000
True
0.76
0.000
True
200
$5
-0.06
0.000
True
0.55
0.000
True
200
S7
0.20
0.000
True
0.50
0.000
True
400
$1
0.32
0.000
True
0.68
0.000
True
400
S2
0.30
0.000
True
0.65
0.000
True
400
S5
0.15
0.000
True
0,55
0.000
True
400
S7
0.20
0.000
True
0.51
0.000
True
600
S1
0.26
0.000
True
0.62
0.000
True
600
S2
0.39
0.000
True
0,64
0.000
True
600
S5
0.23
0.000
True
0.51
0.000
Truc
600
S7
0.08
0.000
True
0.40
0.000
True
800
S1
0.47
0.000
True
0.61
0.000
True
800
$2
0.37
0.000
True
0.60
0.000
True
800
S5
0.17
0.000
True
0.43
0.000
True
800
S7
0.03
0.0002
True
0.29
0.000
True
Full
S1
0.30
0.000
True
0.50
0.000
Truc
Full
S2
0.23
0.000
True
0.50
0.000
True
Full
S5
-0.08
0.000
True
0.08
0.000
True
Full
S7
0.25
0.000
True
0.21
0.000
True
Table 6: Effect sizes (Cohen's d). BH-FDR corrected p-values, and significance for NRF vs. fWRF
and NRF vs. Linear. For readability, p-values 0.0001 are shown as 0.000.
Aggregated results
Training images
d (NRF (WRF)
NRF
(WRF)
Sig
(NRF vs Linear)
(NRF vs Linear)
# Sig
20
0.058 0.223
[0.000, 0.126)
3/4
0.308 + 0l.29
0.000, 0.000)
4/4
0,058 0.118
0.000, 0.000
0.593 0.075
0.000, 0.000
4/4
200
0.175 0.151
0.000. 0.000
0.638 + 0.114
0.000, 0.000
4/4
400
0.243 2: 0.070
0.000, 0.000
4/4
0.598 : 0.070
0.000, 0.000
4/4
0,240 0.110
0.000. 0.000
0.543 0.096
0.000, 0.000
800
0.260 0.171
0.000. 0.0002
4/4
0.453 : 0,132
0.000. 0.000
4/4
Full
0.175 0.149
[0.000,
0.000)
4/4
0.321 + 0.183
0.000, 0.000)
4/4
Table 7: Cross-subject mean + SD effect sizes (Cohen's d), BH-FDR corrected p-value ranges across subjects, and the number of subjects with significant differences (p < 0.05). For readability, p-values < 0.0001 are shown as 0.000.
18

---

Published as a conference paper at ICLR 2026
Voxel-Ievel significance testing
Training images
Subject
NRF (% sig)
(WRF (% sig)
Linear (% sig)
# Voxels
20
S1
67.20%
51.93%
48.32%
15,724
20
S2
67.63%
62.91% %
49.37%
14,278
20
S5
69.23%
69.65%
60.9 60.94%
13,039
20
S7
52.38%
50.38%
38.82%
12,682
60
S1
80.60%
73.40%
62.62%
15,724
60
S2
75.83%
69.97%
57.91% %
14,278
60
S5
83.89%
79.15%
69.45%
13,039
60
S7
70.23%
67.32%
53.22%
12,682
200
S1
86.31% %
81.47%
71.97%
15,724
200
$2
85.00%
80.21 %
72.71%
14,278
200
S5
89.32%
88. 88.26%
82.82%
13,039
200
77.69% S7
73.91% %
69.35%
12,682
400
S1
89.12%
87.38%
79.78%
15,724
400
S2
87.23%
84.71% %
80.76%
14,278
400
S5
91.61% %
90.77%
88.35%
13,039
400
S7
84.32%
81.08%
76.72%
12,682
600
89.91% %
88.69%
82.94%
15,724
600
S2
88.76%
86.47%
83.95%
14,278
600
S5
92.64%
91.71% %
91.01%
13,039
600
S7
82.10%
82.58%
79.08%
12,682
800
S1
90.68%
88.44%
84.75%
15,724
800
S2
89.01% %
86.72%
84.65%
14,278
800
S5
93.13%
92.87%
91.84%
13,039
800
S7
84.57%
84.18%
83.14%
12,682
Full
S1
94.29%
93.13%
92.97%
15,724
Full
$2
92.23%
91.06%
90,95%
14,278
Full
S5
95.70%
94.60%
95.97%
13,039
Full
S7
89.90%
88.63%
90.18%
12,682
Table 8: Fraction of voxels with significant prediction correlation (FDR-corrected) across training set sizes and subjects.
A Aggregated results
Training Images
NRF (%)
IWRF (%)
Linear (%)
20
64.61 7.22
58.22 8.70
49.86 8,98
60
77.14 - 5.61
72.46 + 5.41
60.80 = 7.23
200
84.58 4.80
81.21 + 5.58
74.71 + 5.48
400
88.57 3.20
85.49
3.92
81.40 + 4.47
600
88.85 4.20
87.36 3.94
84.75 5.60
800
89.85 4.02
86.09 3.91
Full
93.03 2.48
91.36 + 2.09
92.52 + 2.39
Table 9: Mean SD percentage of significant voxels across subjects for each training size.
19

---

Published as a conference paper at ICLR 2026
A.5.2
SHUFFLE COORDINATE ABLATION STATISTICS ANALYSIS (FIG
Paired permutation test
Training Images
d (Original Local)
PFDR
Sig?
d (Original Global)
PFDR
Sig?
20
0.202
0.000
True
0.816
0.000
True
100
0.422
0.000
True
0.831
0.000
True
200
0.877
0,000
True
1.024
0.000
True
400
0,780
0,000
True
0.956
0.000
True
0.737
0.000
True
0.985
0.000
True
800
0.773
0,000
True
0.892
0.000
True
Full
0.771
0.000
True
1.386
0.000
True
Table 10: Effect sizes (Cohen's d), BH-FDR corrected p-values, and significance for Original vs Coord shuffle Ablations. Original refers original setting, while Local refer to ROI-wise shuffle and Global refer to Whole-Brain shuffle. For readability, p-values < 0.0001 are shown as 0.000.
Voxel-level significance testing
Training images
Original (% sig)
Local (% sig)
Global (% sig)
Voxels
20
71 .55%
57.59%
30.47%
15001
100
86.75%
80.17%
65.11%
15001
200
91.94%
80.67%
74.85%
15001
400
92.61% %
85.21% %
80.99%
15001
600
93.18%
87.37%
83.03%
15001
800
93.66%
88.11%
86.73%
15001
Full
95.31%
91.97%
86.17%
15001
Table 11: Percentage of significant voxels for different settings. Original refers to original setting, while Local refer to ROI-wise shuffle and Global refer to Whole-Brain shuffle. For readability, p- values 0.0001 shown 0.000.
20

---

Published as a conference paper at ICLR 2026
A.5.3
SHIFT COORDINATE ABLATION STATISTICS ANALYSIS (FIG 48, c)
Paired permutation test
Training images
dOriginal EBA vs Shifted EBA)
PFDR
Sig?
20
0.627
0.000
True
100
0.128
0.000
True
200
-0.114
0.000
True
400
0.276
0.000
True
600
0.244
0.000
True
800
0.115
0.000
True
Table 12: Effect sizes (Cohen's d), BH-FDR corrected p-values, and significance for Original vs Coord shift ablation. Original refers original setting, while shifted correspond to finetuning on shifted EBA voxels. For readability, p-values 0.0001 are shown as 0.000.
Training images
d(Original V1 Shifted V1)
PFDR
Sig?
20
0.609
0.000
True
100
0.550
0.000
True
200
0.415
0.000
True
400
0.204
0.000
True
600
0.299
0.000
True
800
0.302
0.000 True
Table 13: Effect sizes (Cohen's d), BH-FDB corrected p-values, and significance for Original vs Coord shift ablation. Original refers to original setting, while shifted correspond to finetuning on shifted VI voxels. For readability, p-values < 0.0001 are shown as 0.000.
21

---

Published as a conference paper at ICLR 2026
Voxel-Ievel significance testing
Training images
Original EBA (% sig)
Shifted EBA (% sig)
Voxels
20
69.16%
50.50%
3184
100
75.88%
69.91 %
3184
200
73.08%
73.71%
3184
400
79.55%
76.73%
3184
600
78.33%
76.35%
3184
800
78.80%
77.10%
3184
Table 14: Percentage of significant EBA voxels for Original EBA and Shifted EBA.
Training images
Original V1 (% sig)
Shifted V1 (% sig)
Voxels
20
73.93%
63.78%
1074
100
85.94%
84.64%
1074
200
85.66%
83.43%
1074
400
89.11%
86.78%
1074
600
88.27%
86.22% 85.75%
1074
800
87.24%
1074
Table 15: Percentage of significant VI voxels for Original V1 and Shifted V1.
22

---

Published as a conference paper at ICLR 2026
A.6 ADDITIONAL SUBJECT ADAPTATION RESULT
Here, we present additional results of new subject adaptation for subjects 1, 2, and in Table 16 Table 17 Table [8 and Table T9 respectively. The results show that our method consistently yields superior performance compared to the scratch method. We show the and the mean # SEM across the four subjects in Table 20
Training Images
Method
Vovel-Level
Semantic- Levell (via decoding)
Pearson"
MSEU
PixCorrt
SSIMF
Alex(2yf
Alex(5yt
IncepT
CLIP
Eif]]
SwAVI
NRF seratch
0.116
0.411
0.023
0.163
0.548
0.552
56.8%
53.9%
0.969
0.660
NRF finetune eusemble
0,184
0.139
68.9%
NRF scratch
0.261
0.377
0.132
0.242
0.750
0.811
73.6%
70.3%
0.892
0.555
200
NRF finetune ensemble
0.306
0.379
0.266
0.375
0.917
0.958
88.2%
85.9%
0.758
0.437
NRF serntcc
0.314
0.369
0.244
0.307
0.915
0.962
90.6%
86. 1%
0.742
0.432
800
NRF finetune ensemble
0.342
0.361
0.316
0.382
0.945
0.980
93.3%
88.7%
0.698
0.404
Table 16: New subject adaptation with limited data (20, 200, 800 images). NRF pretrained on subjects 2,5,7 are used as base models to adapt to subject 1.
Training Insaee
Method
Level-Level
Semantic Levell (via decoding)
Pearson"
MSEL
PixCorrt
SSIMF
Alcx(2)t
Alex(Syt
IncepT
CLIP
Edt]]
SwAVi
NRF scratch
0.124
0.462
0.023
0.344
0.499
0.498
49.6%
49.6%
0.971
0.636
finetune ensemble
0.168
0.140
74.0%
67.9%.
NRF sernichh
0.266
0.386
0.076
0.323
0.617
0.674
62.7%
57.8%
0.944
0,605
200
NRF finetune eusemble
0.317
0.255
0.365
0.966
90.3%
85.5%
0.735
0.427
NRF scratch
0.323
0.372
0.192
0.299
0.885
0.951
87.4%
82.5%
0.778
0.452
800
NRF finetune ensemble
0.356
0.354
0.274
0.374
0.935
0.976
92.4%
88.0%
0.711
0.413
Table 17: New subject adaptation with limited data (20, 200, 800 images). NRF pretrained on subjects 1,5,7 are used as base models to adapt to subject 2.
Training Images
Method
Voxel-Level
Semantic-Level (via docoding)
Pearson"
MSEL
PixCorrt
SSIMI
Alcx(2)t
Alex(5)t
IncepT
CLIP1
Ed]]
SwAV]
NRF scratch
0.128
0.433
0.126
0.213
0.608
0,633
56.6%
56.8%
0.953
0.592
NRF finetune ensemble
0.184
0.470
0.161
0.366
0.695
0.744
70.2%
67.3%
0.889
200 NRF serntch
0.293
0.415
0.113
0.238
0.687
0.719
68.0%
64.3%
0.911
0,573
NRF finetune ensemble
0.353
0.242
84.8%
800
NRT scratch
NRF finetune ensemble
0.400
0.365
0.355
0.370
0,211
0.259
0,383
0.326
0.883
0.916
0.969
0.945
92.8%
89.39
90.9%
87.0%
0.682
0.734
0.401
0.426
Table 18: New subject adaptation with limited data (20, 200, 800 images). NRF pretrained on subjects 1,2,7 are used as base models to adapt to subject 5.
23

---

Published as a conference paper at ICLR 2026
Training
Method
Voxel-Leww
Semantic-I decoding)
Images
Pearsont
MSEJ
PixCortt
SSIMT
Alex(Zy
Allx(5)f
IncepT1 (5)
CLIPT (R)
En[]
SwAV]
Foll
NRF subject 7 fall dain)
0.269
0.348
0.244
0.367
0.880
0.936
89.2%
84.6%
0.768
0.445
NRF scratch
0.076
0.417
0.060
0.195
0.564
0.597
$4.9%
54.5%
0.962
0.621
ensemble
0.366
0.750
0.792
73.2%
200
NRF scratch
0.180
0.394
0.159
0.284
0.260
0.813
77.4%
71.6%
0.857
0.515
NRF linetune ensemble
0.227
0.390
0.255
0.372
0.908
0.987
91.3%
87.3%
0.729
0.425
NRF scratch
0.220
0.376
0.188
0,313
0.856
0.926
87.8%
83.4%
0,772
0:152
ensemble
Table 19: New subject adaptation with limited data (20, 200, 800 images). NRF pretrained on subjects 1,2,5 are used as base models adapt to subject 7.
Imaga
Mehol
Nend-LLrd
Scczatio-Led doondag))
SRF amads
6.11 2002
0.8122000
0108 =0/024
I422 20044)
0.355 0.00
14.
087
0.000 = 00044
0U227 2-0014
cosnidde
088.1.1
200
NRF cxnembte
NBF wesh
6.901 1:107
6:3594 0res
0.20 0.30
0U904
0.00 -2007
2000
N372 2.0022
0379 4-0
0.791-0000 0.3
8008
6:754 0/05
6.00
70.1133
0:0-331
0001 8-014
11434-2:0/00
0.5084-0.000
500
SRF fasuue exsonide
SBE wnald
6,337 0.081
0,672
0,900
2 0.204
RL865
0 0309
0 2.005
0811 2:000
0.500 1350-200022
0.301
0.58220000
0.02
0.000
0,044 0.006
0,90:00000
555.207
$2.7202
868-5..1
0.507 = B01L
0008 4 8.006
(U441 4-0.00
0.407 0-0.003
Table 20: Mean + SEM across subjects (S1, S2, 85, S7) for all training sizes and methods.
24

---

Published as a conference paper at ICLR 2026
A.7 ADDITIONAL ABLATION RESULTS
Finetuning strategy During finetuning, there are two options: the projection model and the image feature merger. We explored finetuning both/prooector-onlyy and the feature extraction block only. We finetuned subject NRF with 800 images from subject data. The results are shown in the Table A.7
The result suggests that finetuning should be done on both components to get the maximum performance boost.
Finetune strategy
Vovel-Level
Semantic-Lever decoding)
Peursont
MSEJ
PixCorff
SSIMI
Alex(2)1
Alex(5)t
IncepTT
CLIP1
Eff]
SwAV
Both
0.234
0.361
0.254
0.377
0,910
0.958
90.9%
86.7%
0.730
0.424
Image extractor only
0.104
0.394
0.102
0.274
0.613
0.636
59.2%
55.6%.
0.958
0.596
Projector only
0.233
0.364
0.249
0.361
0.894
88.8%
83.8%
0.766
0.446
Table 21: Ablation different finetuning strategies. Models pre-trained on subject finetuned on 800 images from subject 7.
To investigate the effect of model architecture on neural response prediction ac- curacy, we conducted an ablation study by varying the number of layers and the hidden dimension of the MLP projector. As shown in Table we computed 4, 8, 16-layer configurations under subject- specific settings on subject1. The results show that 8-layer model results in the best performance, which is also the setting we utilized for our experiments.
Number of layers
Layers
Vevel-Level
Semantic Levvetic-Level (via decoding)
Pearsonf
MSEI
PixCorrt
SSIM
Alex(2)1
Alex(5)
Incep'Tt
CLIPt
Eff]
SwAV]
0.358
0.348
0.258
0.351
0.914
0.961
89.7%
85.1%
0.748
0.437
0.360
0.350
0.324
0.387
0.956
0.983
94.1%
89.9%
0.680
0.396
0.349
0.353
0.331
0.385
0.956
0.983
93.99%
89.9%
0.678
0.395
Table 22: Ablation of the number layers the MLP projector. Model trained on subject data.
Hidden dimension We also conducted ablation study by varying the hidden dimension of the MLP projector. As shown in Table, we computed for hidden dimension 2048, 4096, 8192 con- figurations under subject-specific settings subject1. The results show that model with hidden dimension = 4096 results in the best performance, which also the setting we utilized for our experiments.
Hidden dimension
Voxel-Level
Semantic-Level (via decoding)
Pearson
MSE.
PixCont
SSIM
Alex(2)t
Alex(Sy)
IncepTt
CLIPt
Eff]
SwAV
0.358
0.348
0.253
0.353
0.903
0.948
87.4%
83.2%
0.777
0.451
4096
0.360
0.350
0.324
0.387
0.956
94.1%
89.9%
0.680
0.396
8192
0,353
0.353
0.323
0.386
0.955
0,981
94.1%
89.9%
0.683
0.396
Table 23: Ablation on hidden layer dimension of the MLP projector. data.
Model trained on subject
25

---

Published as a conference paper at ICLR 2026
A.8 NIGH-RESOLUTION NRF QUERYING
Once trained, NRF behaves as a continuous "digital twin" of the voxel response field: given an image embedding and any 3D anatomical coordinate, NRF directly outputs the predicted response at that location. Unlike traditional voxel-wise models, NRF does not depend on the discrete voxel grid of the acquired fMRI data.
Because NRF learns a continuous neural field over standardized brain space, it can be queried on spatial grids of arbitrary resolution e.g., 0.5 mm finer-withowd any volumetric resampling, terpolation, or reconstruction of the fMRI volume. This completely bypasses the computationally expensive and resolution resolution-dependent resampling pipeline typically required for aligning or upsam- pling fMRI data.
Figure illustrates this capability by comparing NRF predictions evaluated on 0.5 mm grid with the original measured responses acquired at 1.8 mm resolution.
NRF Queried Response func0.55m slice
Original measured response func1.88m 1mcm..mm slice
1.5
1.0
Respo nse Value
0.5
0.0
-0.5
-1.0
-1.
Figure 6: NRF enables higher-resolution querying of the learned response field. Because NRF models brain activity as a continuous function over 3D anatomical space, can be queried at arbi- trary spatial locations after training. Shown here is an example slice from Subject 1: left. NRF pre- dictions queried on high-resolution 0.5 mm grid; right, the original measured fMRI response at the native 1.8 mm voxel resolution. NRF's continuous formulation enables principled, high-resolution resampling of the predicted response field beyond the acquired voxel grid.
26

---

Published as a conference paper at ICLR 2026
B USE OF LLMS
LLMss were used only for polishing grammar and writing clarity.
C ETHIC STATEMENT
Our research adheres to the ICLR Code of Ethics. All experiments this paper are conducted using open-source datasets, and no potential ethical concerns are associated with this work.
D REPRODUCIBILITY STATEMENT
All preprocessed data, code, and model parameters used in research will be made publicly available upon publication. Detailed protocols for data preprocessing, model training, and evaluation have been provided our manuscript, enabling independent reproduction.
27


Reviews:
[{"review_id": "odIC4Z4KXX", "reviewer": "Reviewer_mipv", "summary": "This paper introduces the Neural Response Function (NRF), a novel neural encoding model that represents fMRI activity as a continuous function over 3D anatomical space. Unlike conventional models that flatten brain data into 1D vectors, NRF is an implicit neural representation that takes a visual stimulus and a standardized MNI coordinate (x, y, z) as input to predict the response at that specific location. This anatomically-aware formulation allows the model to leverage the spatial smoothness of fMRI data and enables efficient cross-subject transfer, as the MNI coordinate system is shared across individuals.", "questions": "1. Performance in Full-Data Regime: Why does the significant performance gap observed in the low-data regime (Fig 2a) largely disappear in the full-data setting (Table 1)? Does the strong spatial-smoothness prior perhaps act as a regularizer that helps when data is scarce but hinders the model from capturing fine-grained, non-smooth voxel-specific details when data is abundant?\n\n2. Reliability of Semantic Metrics: Given that the fWRF baseline outperforms the \"Measured fMRI\" ground truth on several semantic metrics, how should we interpret these results? Does this not suggest the MindEye2 decoder is (over)fitted to fWRF-like distributions, making this entire family of metrics unsuitable for comparing these encoders?\n\nI will consider raising my score if all my concerns are well addressed by the authors.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novel and Elegant Formulation: The core contribution, modeling brain responses as a continuous, coordinate-based implicit function, is a novel and elegant departure from the standard \"grid-locked\" voxel-wise approach.\n\n2. Solves a Key Practical Problem (Low Data): The model's strongest results are in the low-data regime, where it substantially outperforms baselines. This is a highly relevant contribution, as most fMRI studies outside of massive public datasets are data-scarce. The ability to exploit spatial priors (smoothness) clearly improves data efficiency.\n\n3. Effective Cross-Subject Transfer: The paper successfully demonstrates a practical framework for cross-subject adaptation. By grounding the model in MNI space, it provides a principled way to transfer knowledge, and the finetune-ensemble strategy yields impressive performance with minimal data from a new subject.\n\n4. Strong Empirical Validation: The probing experiments are commendable. By intentionally breaking the model's assumptions (shuffling coordinates) and observing the expected performance drop, the authors provide strong evidence that the model's gains are in fact due to its leveraging of spatial smoothness and anatomical alignment.", "weaknesses": "- Unaddressed Computational Cost: The per-coordinate query architecture seems significantly less scalable than standard single-pass encoders. The paper provides no analysis of this computational overhead or its implications for full-brain, high-resolution inference.\n\n- Incomplete Related Work Context: The paper emphasizes its cross-subject adaptation. However, the related work section overlooks that many recent neural decoding models (e.g., 'MindBridge' [1]) also incorporate encoding mechanisms and explicitly address the challenge of adapting to new subjects. The Related Work section would be more comprehensive with a discussion of these approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BQnGBvj0pF", "reviewer": "Reviewer_vmAw", "summary": "This work is part of a broader research effort to study voxelwise encoding models for visual stimuli. While prior vision encoding studies typically used linear mapping between image features and voxel activations of image stimuli. The current work advances the prior work by proposing a Neural Response Function (NRF) that takes both voxel location (x,y,z coordinates from a 3D volume) and image features as input, and uses an MLP(multi-layer perceptron) to predict fMRI brain activity. The authors claim that including voxel coordinates provides local spatial smoothing, capturing neighboring voxels, and an MNI space brings all subjects in common space possible for cross-subject predictions. Using the NRF approach, the evaluation focuses on comparing the individual subjects with baseline models such as linear regression and feature-weighted receptive field (fWRF) method for both brain encoding and brain decoding. Furthermore, the authors test cross-subject transfer: train on one subject and then fine-tuned on a new subject for adaptation. Overall, proposed NRF shows strong encoding and decoding performance under low-data regimes, provides an anatomically aware neural encoding model, generalizes both spatial smoothness and supports cross-subject alignment.\n\n**Contributions:**\n\n* *MNI space voxel coordinates for neural encoding:* The study uses both spatial coordinates (x,y,z) of each voxel and image stimuli representations from image encoder and use these representations in NRF to predict fMRI brain response to visual stimuli. Specifically, incorporating anatomical coordinates offers local spatial smoothing and facilitates cross-subject alignment, which is methodologically novel, relative to prior work that treats voxels as flattened 1D activations. \n* *Comprehensive evaluation:* The study evaluates proposed NRF’s brain predictive performance on individual subjects and compares it against baseline models including linear regression and fWRF. For each encoding model, the authors report correlation and MSE, for decoding, they use image retrieval metrics (e.g., SSIM, PixCorr). Further, they compare cross-subject transfer by training one subject with varying data scales and fine-tuning on a new subject, quantifying new subject adaptation as a function of training steps and dataset size, with an emphasis on low-data regimes.\n\n**Technical summary:**\nThis is primarily an empirical study, and its methodology involves the following components:\n* *Mapping spatial coordinates:* The authors define a continuous function over MNI coordinates (common space for all subjects). For each voxel coordinate r =(x,y,z), the function computes a positional encoding ϕ(r) using fourier positional encoding method (which is popular in Transformer language models). The positional encoding vector is used in the NRF function during encoding to predict fMRI response.\n* *Neural Response Function:* To train the NRF, the model takes two inputs: Image stimulus representations from image encoder and positional encoding features (ϕ(r)) for each voxel coordinate r. These vectors are concatenated and passed as input to MLP that predicts target fMRI response at spatial coordinate r. The model is trained end-to-end with a weighted sum of convex combination of mean square error and cosine similarity between the predicted response  and ground truth fMRI; optimised using Adam. \n* *Cross-subject transfer:* The authors use a two-step strategy: (i) end-to-end fine-tuning- it involves fine-tuning a pretrained NRF model on a new subject's limited data and performing end-to-end fine-tuning. (ii) Voxel-wise ensembling: In this step,  for each voxel, they make predictions from K fine-tuned models for an image by learning voxel-specific weights for each fine-tuned model.\n\n**Experimental design/evaluation:**\n* *Subject-specific encoding:* The authors evaluate the capability of NRF by training subject-specific models and compare them with two baselines: linear regression and fWRF. To probe NRF under limited data samples, they train with 20, 40, 200, 400, 600, 800 and full images and report performance across four subjects. \n* *New subject adaptation:* This analysis evaluates cross-subject transfer, where NRF trained on three subjects with 20, 200, and 800 training images, then test on the held-out fourth. For the fourth subject, the authors fine-tune the pretrained NRF and compare it to an NRF trained from scratch. They also perform voxel-wise ensembling: for each voxel, predictions from K fine-tuned models are combined using voxel-specific weights.\n* *Probing anatomical awareness:* To test whether NRF really uses spatial continuity and cross-subject anatomy, the authors perturb the coordinates: they disrupt local smoothness by shuffling voxel locations within a subject, and shifting MNI coordinates across subjects to break the alignment. Performance drops under these perturbations, indicating both spatial continuity and anatomical alignment are crucial for better NRF performance.\n\n\n**Main findings:**\nAccording to the authors’ interpretation, the main findings are as follows:\n* Under subject-specific encoding across different training data regimes, with only 200 training samples, NRF outperforms baselines trained on more than 800 images, indicating stronger performance in low-data regimes.\n* During cross-subject transfer, NRF with fine-tuning + ensemble consistently outperformed NRF scratch, indicating the anatomically conditioned formulation enables efficient cross-subject transfer with limited target data while maintaining predictive accuracy.\n* Shuffling local-spatial continuity and shifting MNI coordinates both reduces NRF performance, which implies NRF relies on local spatial continuity and cross-subject anatomical alignment. Overall, authors argue that NRF serves as a resolution-agnostic representation that can integrate data across scales and subjects.", "questions": "I would like to thank the authors for the interesting comparison between coordinate-conditioned NRF (leveraging local spatial smoothing via voxel coordinates) and linear regression models with flattened 1D brain responses for fMRI encoding during visual stimuli. However, there are several points that I believe require further attention/work. I have divided these into major issues, which should be prioritized, and minor ones, which should be addressed for a strong version of current work.\n\n**Major Comments/Questions**\n* *Motivation and positioning:* While cross-subject transfer and multi-subject models are possible with surface-level alignment (fsaverage surface). To strengthen the claims, I recommend authors to add: (i) multisubject models in average baseline, and (ii) compare NRF model in MNI space vs. multi-subject average baseline. If NRF still outperforms the fsaverage baseline, especially in low-data and cross-subject transfer settings, that would strengthen the local spatial smoothing claim and support the conclusions.\n* *Statistical significance and clarity of Fig 2:* Fig2a reports subject-specific encoding model performance for two baselines (linear regression and fWRF) and proposed NRF. However, no statistical significance tests are provided. From Fig 2a, it is hard to conclude that NRF is significantly better than baselines. I suggest authors following\n     * Please perform paired permutation/bootstrapped tests with Benjamini–Hochberg FDR correction (across voxels or per-subject summaries), report the adjusted p-values and effect sizes, and clearly indicate which model pairs are statistically significant.\n     * Select voxels which are significant for each model and report percentage of significant voxels across subject-level summaries.\n     * Repeat above two steps  while varying the number of training images and report significant results.\n     * Also clarify what evaluation metric is reported in Fig 2b. There is an inconsistency of evaluation metrics mentioned in methodology that R^2 as accuracy and Peason correlation as voxel-level evaluation metric. \n     * Similarly, no statistical significance tests are provided for Fig 4. From Fig 4, it is hard to conclude that NRF is significantly better than ROI shifted results for EBA and V1 regions.\n* *Reporting variability:*  Please report standard deviations (or standard errors) alongside mean/median values to provide a clearer picture of variability in Table 1and 2. Further, fix the caption in each Table to state explicitly whether values are means (or medians) across the 4 subjects.\n     * Table 3 shows transfer only for Subject 7. Please run the analysis for every target subject (leave-one-subject-out), and report mean ± SD (or SEM) per target subject as well as the overall average. A full source->target transfer matrix (heatmap) with uncertainty and FDR-corrected paired tests would make the results much more robust.  \n* *Clarity on fMRI-to-Image reconstruction:*  While authors report image reconstruction results in Table 2 and 3 with semantic-level metrics, and qualitative analysis of images in Fig 3 using MindEye2, there are no details about what information exactly is fed to MindEye2 from NRF model. \n     * Please add a small description about fMRI-to-Image decoding\n     * Also, in Table 2, NRF trails fWRF and MindSimulator on decoding. If decoding is not the paper’s strength, I recommend authors to reposition decoding as a diagnostic, highlighting where NRF adds value to the paper.\n* *Why an MLP (NRF) instead of a linear/ridge model?*\n     * The authors do not provide any justification why a nonlinear MLP is necessary once after concatenation of image features + positional encodings. A ridge regression as baseline on the same inputs would test whether the gains come from nonlinear interactions or simply from adding coordinates.\n\n**Minor Comments/Typos**\nWhile addressing the following points may not be critical to the paper’s core contributions, doing so would enhance the overall quality. \n* I would appreciate authors to add little details for decoding setup.\n* Clarify bottom panel Figure 1: It is unclear why both the pretrained NRF and the fine-tuned NRF are shown for Subject 1. Please state explicitly in the caption: Need better clarification in Figure 1 caption.\n* Line 42, 46, 97: Follow proper citation format: e.g., “Downing et al. (2001) -> (Downing et al. 2001)”. Please correct all over the paper and use appropriate citation format: \\citep{} or \\cite{}. \n* Line50: 2)Subject specific -> 2) Subject specific\n* Line 103: Please add citation for this sentence: “Despite individual variability->”\n* Line 152: define abbreviation: Implicit neural representations -> Implicit neural representations (INR)\n* In Table captions, specify whether metrics are per-voxel median or mean over voxels and how subject aggregation is done.\n* Figure 2b: please add evaluation metric on colorbar\n* Since NSD provides pycortex flatmaps; please add 2D cortical flatmaps to complement Fig. 2b and make differences visible: Show absolute performance maps (e.g., voxel-wise Pearson r or R^2) for 200 vs. 800 images for each model (linear, fWRF, NRF) with the same color scale across panels. Current Fig 2b, the brain maps look similar across models.\n* Move Fig. 3 to the appendix. Align Fig. 2 and Fig. 4 layouts. Use the same panel grid, and increase the font size of legends and axis-labels.\n\n**General Advice**\nThe manuscript presents a Neural Response Function (NRF) that uses voxel coordinates for local-spatial smoothing and combining with image features to perform visual encoding on NSD dataset. Using NRF, the authors evaluate subject-specific models, low-data regimes, and cross-subject transfer. However, the current version lacks strong alignment baselines (ideally, an fsaverage/surface baseline), and shows metric inconsistencies (R^2 vs. Pearson/MSE), and reports results without any statistical significance testing. The decoding section is under-specified and NRF underperforms relative to baselines. Figures would benefit from reorganization and clearer labeling, and tables should include subject-level variability. Addressing these points and the above mentioned weaknesses and major comments would make the work stronger.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "I found this work to have the following strengths:\n* *Clarity:* The manuscript introduction, dataset, and key methodological details are well written and well structured. The pipeline top panel of Figure 1 is easy to follow and clearly shows a pretrained NRF for a single subject. Later, the NRF formulation is described clearly combining an image-embedding vector with a positional encoding.  The model training and cross-subject transfer are also explained well, including how fine-tuning is performed on the pretrained NRF. The results section clearly reports subject-specific encoding performance, compares NRF with two baselines, applies the approach to cross-subject transfer, and then presents perturbation analyses that probe the proposed NRF.\n* *Originality:* The idea of using common-space MNI coordinates to build positional encodings and combine with image features for handling local-spatial smoothing in a neural encoding model to predict fMRI is a simple idea that is well known in brain decoding but remains underexplored in brain encoding. Prior brain encoding studies typically use flattened 1D brain responses and learn a subject-specific ridge-regression model. In contrast, NRF provides a cross-subject model that supports subject adaptation and outperforms baselines in low-data regimes.\n* *Significance:* This work is significant in that it contributes to a better understanding of the contribution of local-spatial smoothing and anatomical alignment are crucial for better encoding performance under low-data regimes. The comprehensive analysis on the NSD dataset shows that NRF serves as a general-purpose brain model that supports cross-subject transfer and can integrate data collected at different spatial resolutions.", "weaknesses": "From my perspective, the primary weaknesses of this study arise from the lack of comparison with prior literature, and limited evaluation:\n* *Limited methodological novelty:* The paper claims that NRF is “the first anatomically aware encoding model to move beyond flattened voxels, learning a continuous mapping from images to brain responses in 3D space”. However, prior study, MindLLM (Qiu et al. 2025) already uses voxel coordinates with Fourier positional encoding for embedding vectors for each voxel coordinate (x,y,z). Given this, I recommend authors to soften the word “first claim” and clarify how NRF differs from prior work. Further, the authors ignore the prior study, and do not cite or compare against MindLLM.\n\nQiu et al. 2025, MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding, ICML-2025\n\n* *Lack of motivation and position:* The authors argue that flattened neural 1D response vectors make them forced to train subject-specific models and it leads to poor data efficiency. However, this claim is not fully convincing. In general, the 3D volume can be mapped onto common space (e.g., MNI or fsaverage/fsaverage5/6), enabling multi-subject models and cross-subject transfer. Further, prior work does not ignore 3D spatial information as they use localizers to map language/Visual ROI voxels [Huth et al. 2016, Deniz et al. 2019]. I suggest authors clarify why conditioning on voxel coordinates within NRF is necessary beyond these standard options, and what extra benefit it provides (ideally with a direct comparison). \n\n[Huth et al. 2016] Natural speech reveals the semantic maps that tile human cerebral cortex, Nature 2016\n\n[Deniz et al. 2019] The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality, Journal of Neuroscience 2019\n\n* *Inconsistency of evaluation metrics:* The dataset section 4.1 (line 263) states that the NRF model is trained and tested with model accuracy via R^2, while line 271 states that Pearson correlation and mean-squared error as voxel-level metrics. There are some results reported with Pearson correlation in Fig2a, Table1, and 2, while Fig2b shows prediction accuracy. Therefore it is unclear, which metrics are primary, and what they reported in results? I recommend authors to clarify the primary evaluation metric and use them consistently across Tables and Figs.\n* *No statistical significance reports:* The paper reports no statistical significance tests (e.g., bootstrap testing with FDR correction or paired permutation) to determine whether model differences are significant. Without any significance tests, the performance of NRF over baselines should be considered as descriptive rather strong empirical evidence.\n* *No standard error or standard deviation across subjects:* The results reported in Table 1-2 state that all metrics are calculated across 4 subjects, without accompanying measures of variability such as standard deviation or standard error. These metrics are critical for evaluating the robustness of the findings. \nFor a complete and detailed account of both major and minor issues, please refer to the “Questions” section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1cFN9ar9dY", "reviewer": "Reviewer_vXEp", "summary": "The authors propose an implicit neural representation method for fMRI data, called neural response function (NRF). The proposed method utilizes local smoothness and cross-subject alignment to learn a function that can be used to predict the voxel-level response to a given stimulus continuously over R^3 space. This is presented as a significant improvement over the current state of the art methods that generally flatten 3d voxel space into a 1d vector and independently predict brain-response at each location. Which means current approaches break the anatomical reality where voxel level response is often spatially correlated and it also means cross-subject prediction is impossible as two voxels for two different subjects aren't comparable. NRF uses positional embeddings (with learnable coefficients) to embed voxel locations and then concatenates those with learnable image (stimulus) embedding and then passes these through a learnable NN model to predict the response for a subject's given voxel to the given image. Once pretrained on a single subject, the model can then be finetuned on partial data from other subjects to predict that subject's brain response. Multiple such pretrained models can be combined using regression and finetuned on a new subject to improve this prediction even further.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The idea as well is as the presentation of the paper is very strong. This is a significant improvement over current methods of fMRI encoding. Making sure that voxel responses are anatomically correlated respects the realities of fMRI data in a way that treating voxels independently does not. It's also impressive how the implementation is fairly straightforward and as the experiments show, it works.", "weaknesses": "One weakness is that the results are demonstrated on a single dataset. Generalization across subjects is again, within the same dataset. I think the method would become an even better sell if it can be shown to generalize on other datasets, especially across sites. As that generalizability is a hard problem in the fMRI space.\n\nThe other weakness is in the ultimate usefulness, the fact that it can generalize to a new subject is great, but that still requires some data from that subject. From a practical standpoint that means you still will need to put the subject through a scanner. If we take that as an inevitability, what we'd like to see is that the only data we need is from some sort of standardized or just from the beginning of the scan - so that the subject can then be let go after a short amount of scanning. With the current set-up, (I assume) random portions of the new participant's data are being used for finetuning. That might not be practical in reality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iZK2cZn9uq", "reviewer": "Reviewer_vUPt", "summary": "This paper adapts Neural Radiance Fields (NeRF) for the fMRI-to-image encoding task. In contrast to the common approach of predicting each voxel independently, this method predicts all voxel activations in a coupled manner. This strategy yields locally smooth predictions and facilitates efficient transfer learning.", "questions": "- Can your approach be combined with competitive encoding models? \n- Does your method enable an increase in the spatial resolution of the fMRI signal, or is the benefit of local smoothness limited to the predictions themselves?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an original approach to the challenging task of predicting fMRI activations in a structured and meaningful way.\nThe framework also facilitates an interesting subject adaptation method.", "weaknesses": "- The reported encoding performance is very low. On the NSD dataset, a decent model for the selected voxels should yield correlations above 0.5 for subjects 1, 2, and 5, with strong models achieving values around 0.6. Notably, other works (e.g., [1, 2]) report significantly better performance on a wider selection of NSD voxels.\n\n- Applying the fWRF approach to layers from DinoV2/CLIP, combined with LoRa adaptation and well-tuned hyperparameters, would outperform the reported results by a larger margin than the one shown between the NRF method and other baselines.\n\n- The fWRF model outperforms the proposed approach on all but one reported decoding metrics.\n\nReferences:\n\n[1] Adeli, H., Sun, M., & Kriegeskorte, N. (2025). Transformer brain encoders explain human high-level visual responses. arXiv preprint arXiv:2505.17329.\n\n[2] Beliy, R., Wasserman, N., Zalcher, A., & Irani, M. (2024). The Wisdom of a Crowd of Brains: A Universal Brain Encoder. arXiv preprint arXiv:2406.12179.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)