Disentangling Superpositions: Interpretable Brain Encoding Model with Sparse Concept Atoms
Authors: Alicia Zeng
Jack L. Gallant
Venue: NeurIPS 2025 poster ()
Keywords: Interpretability
Neuroscience
fMRI
Encoding Models
Superposition
Primary area: neuroscience_and_cognitive_science

Abstract: Encoding models using word embeddings or artificial neural network (ANN) features reliably predict brain responses to naturalistic stimuli, yet interpreting these models remains challenging. A central limitation is superposition: distinct semantic features become entangled along correlated directions in dense embeddings when latent features outnumber embedding dimensions. This entanglement renders regression weights non-identifiable—different combinations of semantic directions can produce identical predictions, precluding principled interpretation of voxel selectivity. To address this, we introduce the Sparse Concept Encoding Model, which transforms dense embeddings into a higher-dimensional, sparse, non-negative space of learned concept atoms. This transformation yields an axis-aligned semantic basis where each dimension corresponds to an interpretable concept, enabling direct readout of conceptual selectivity from voxel weights. When applied to fMRI data collected during story listening, our model matches the prediction performance of conventional dense models while substantially enhancing interpretability. It enables novel neuroscientific analyses such as disentangling overlapping cortical representations of time, space, and number, and revealing structured similarity among distributed conceptual maps. This framework offers a scalable and interpretable bridge between ANN-derived features and human conceptual representations in the brain.

Disentangling Superpositions: Interpretable Brain Encoding Model with Sparse Concept Atoms
Alicia Zeng
Jack Gallant
Biophysics Program University of California, Berkeley Berkeley, CA 94720 Litrtwe@berkeley. edu
Department of Neuroscience University California, Berkeley
Berkeley, CA 94720 gallant@berkeley edu
Abstract
Encoding models using word embeddings or artificial neural network (ANN) fea- tures reliably predict brain responses naturalistic stimuli, yet interpreting these models remains challenging. A central limitation is superposition: distinct seman- tic features become entangled along correlated directions in dense embeddings when latent features outnumber embedding dimensions. This entanglement renders regression weights non-identifiable- different combinations of semantic directions can produce identical predictions, precluding principled interpretation of voxel se- lectivity. To address this, we introduce the Sparse Concept Encoding Model, which transforms dense embeddings into a higher-dimensional sparse, non-negative space of learned concept atoms. This transformation yields an axis-aligned semantic basis where each dimension corresponds to an interpretable concept, enabling direct readout of conceptual selectivity from voxel weights. When applied fMRI data collected during story listening, our model matches the prediction performance of conventional dense models while substantially enhancing interpretability. enables novel neuroscientific analyses such as disentangling overlapping cortical repre- sentations of time, space, and number, and revealing structured similarity among distributed conceptual maps. This framework offers scalable and interpretable bridge between ANN-derived features and human conceptual representations the brain.
1 Introduction
Artificial neural networks (ANNs) were originally inspired by the brain, yet progress in machine learning has far outpaced advances in understanding brain function. A central bottleneck in neuro- science is data: while many large-scale datasets are available for training machine learning models, brain recordings remain scarce and costly. To overcome this limitation, ncuroscientists increasingly machine learning models to study how the brain represents information. A common framework for this approach is the encoding model, which predicts brain activity from quantitative stimulus
features using linear regression [22] 27 15 35 [66]. Early encoding models relied manually-
defined features, such as labeled objects or actions movies [23 55 34). Recent encoding models
instead leverage features derived from machine learning, such word embeddings activations
26). When used as regressors predict brain responses naturalistic stimuli such narrative stories, machine-eamming-derie features consistently outperform manually-defined features. In particular, features from deeper neural
from large language models (LLMs) [52 30 37 57
I
24
20,
28 50
networks achieve higher predictive accuracy 1 45).
Despite their success in applications like speech decoding 29 51 47 ANN-based encoding models
remain limited in their ability to advance fundamental neuroscience due to poor interpretability.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Dense Embedding
Concopt Atom
Aris
Time
Number
Family
Weight Vector
Weight Vector
Space
Embedding
Concept Space
Axis 2
Femaleness
Dense Embedding
Color
Time
Concept: Atom Axis Number
Axis
Dense Feature Space
Sparse Concept Feature Space
The Sparse Concept Encoding Model resolves superposition in dense feature spaces. Left: In dense embedding space, semantic concepts represented by correlated linear combinations of basis directions. This superposition introduces ambiguity: a voxel's weight vector may project positively onto multiple concept directions simply because those directions are themselves correlated. Right: The Sparse Concept Encoding Model recovers the underlying semantic directions and assigns each an independent axis, transforming the dense space into an interpretable, axis-aligned representation. This enables voxel tuning to be directly read out along concept dimensions.
Figure 1:
Mapping ANN-derived representations onto brain activity often amounts to explaining one black box with another, yielding little insight into the underlying neural mechanisms. However, many core questions in neuroscience, such where and how abstract concepts are represented in the brain, demand models with interpretable internal structure.
In this work, address the interpretability challenges inherent to dense feature representations in encoding models. Previous studies using word embeddings or ANN-based features often assume that regression weights in these high-dimensional spaces directly interpretable 22 show that this assumption fundamentally flawed. Specifically, we demonstrate that superposition, the entanglement of multiple latent semantic features when their number exceeds the embedding dimensionality, introduces intrinsic ambiguity in model interpretation []6] 8). When semantic features are superposed, regression weights become non-identifiable distinct combinations of semantic features can produce the same predicted brain responses. This makes it impossible to reliably identify which specific concepts voxel encodes, limiting the utility of dense models for neuroscientific inquiries. To overcome this limitation, we propose new framework that preserves the predictive power dense features while significantly enhancing model interpretability. Our method disentangles superposed representations into sparse, semantically-aligned concept dimensions, allowing clearer link between model structure and brain activity. The framework generalizes naturally across wide range of representations, such LLM embeddings, and across recording modalities, such as clcctrocorticograhyy (ECoG) and magnetocnceehalloraptyy (MEG).
50
54 13). We
Main Contributions (1) We show that regression weights learned from dense features are inherently non-identifiable. Using Bayesian analysis, we trace this ambiguity to the implicit prior induced by the geometry of dense embedding spaces. (2) We introduce the Sparse Concept Encoding Model, which transforms dense features into a higher-dimensional. sparse space composed of learned concept atoms. (3) We propose Vector Norm Reparameterization (VNR), a preprocessing method that enhances interpretability of learned concept atoms by expanding the relative spread of low-norm vectors while compressing the range of high-norm vectors. (4) We apply the Sparse Concept Encoding Model to naturalistic fMRI dataset and demonstrate two neuroscientific analyses that are infeasible with dense models: (i) disentangling overlapping cortical representations of time, space, and number, and (ii) comparing concept representations across the cortex to reveal structured relationships among distributed conceptual maps.

---

3
Concept Atom 361 "Trail": hilly, uphill, jog, trail, mile
+
2
X
Concept Atom 924 "Walkabiltt": rambla, subway, boulevard, avenue, street
Walking
+-
2
X
Concept Atom 384 'Present Continuous Tense" accusing. owning, artticizing. wasting
+
2
X
Concept Atom 456 'Body position": sistion".sitting crouching, kneeling crouched, standing
+ 1
X
Concept Atom 997 "Space" centre, near, situated located km
Figure 2: Sparse dictionary learning decomposes each word into a sparse, non-negative linear combination of concept atoms. This example shows the sparse decomposition of the word "walking" using the Iearned concept atom dictionary. Each contributing atom is shown with its index, a descriptive label (chosen heuristically by the authors), and its top activating stimulus words from the story transcripts.
Theories
2.1 Sparse Coding
Sparse coding, also known as sparse dictionary learning, was originally proposed as a model for how the visual cortex efficiently represents natural scenes [8] 39]. The central idea is that many natural signals, such as images, sounds, or words, can be expressed as linear combinations of a small number of latent components, often called basis factors atoms. For example, the word "queen" can be decomposed into two latent factors: "royalty" and "femaleness", These features tend to occur independently: "royalty" appears in "king" and "castle", while "femaleness" is present in "daughter" and "mother". Since most words activate only a small number such factors, the resulting representations are sparse.
Formally, each input x E RP is approximated as linear combination atoms from an overcomplete p Xj Ei=1 U. The coefficient vector a, E R is constrained to be sparse, with most entries equal to zero. Overcompleteness (m >> p) allows the dictionary to capture fine-grained structure in the data but renders the linear system underdetermined. Sparsity resolves this ambiguity by favoring solutions that use only a small subset atoms. Specifi- cally, the dictionary and coefficients are jointly learned by minimizing regularized least-squares objective:
dictionary {Φ, ) C IR, where
min (a.)
Xj
la
2.2 Sparse Coding For Interpreting Word Embeddings and Deep Neural Networks
Word embeddings are known to encode semantic features as linear directions in activation spaces. The canonical example by Mikolov et [32], "king" "man" " "woman" Rs "queen", showed that vector arithmetic on word embeddings captures analogies. This suggests that certain linear directions correspond to interpretable semantic concepts ("royalty" and "femaleness"), Arora et al. 6) (2) provided a theoretical explanation for the linear structure through a random-walk, on-discourses model, where word vectors are generated from mixtures of latent discourse vectors. Arora et al. (2018) K further argued that sparse coding provides a natural method to recover these latent discourse directions from dense embeddings [I7]. Subsequent studies applying sparse coding to word embeddings have indeed revealed dictionary atoms that correspond to interpretable semantic and syntactic features 3 59).
Recently, sparse coding has been adopted in mechanistic interpretability research on deep neural networks (DNNs), which aims decompose models into intera-niterrretablle components [43] 46).
Early efforts focused on analyzing individual neurons, under the assumption that each neuron corresponds to a single interpretable feature 25 36 5. However, empirical studies revealed that many neurons respond to multiple, often unrelated features, phenomenon known as polysemanticity [36]. To explain this, researchers proposed the superposition hypothesis [16 g]: when the number of latent features exceeds the number of available neurons, the network encodes each feature as a distinct

---

Dense Encoding Model
Dense Embbddingg
Embeddiing
Number stimulus word
four
Detee tribaaddnng
downsamated
to manch
fly fralic
Dense Weight Vecior
Number of samppho
messuremnt remeeeettttt
embadding dimension
Dense embecotnn dimension
Number vovels Weights
Number of vavals Measurements
Sparse Dictionary Learning
Number Time Space
outside
Number
Numberor
nine hours
Number words a
Spanse feature aimension
words
Time Donse embeading
ambedising Dense Embedding Matrix
Sparse con Teature dimensioon Sparse Encoding Matrix
Concept Atoms
Sparse Concept Encoding Model
Concept Atom (Time)
Concept Atom (Space)
Number Time Space
Concapt Atsm Aci (humbar))
Number
four
Sperse Concept Weight Vector Space
stimulus words
downsempted
high
fy
Number
Space Concept Feature Dimension
Number samples
to match
Time
mess rements
frllic
Space
mansionn
Number
Sparse Feature Matrix
Vowel Weights
[MRI Measurementts
Figure 3: The Sparse Concept Encoding Model produces interpretable voxel weights by placing dense features with learned concept atoms. Top: Dense Encoding Model: Stimuli are represented using dense word embeddings. resulting in voxel weights that lie in entangled feature space where semantic directions overlap. Middle: Sparse Dictionary Learning: Dense embeddings transformed into higher-dimensional space of sparse concept atoms, each defining distinct, interpretable semantic axis. Bottom: Sparse Concept Encoding Model: Stimuli are re-encoded as sparse combinations of concept atoms. The resulting voxel weights defined with respect to disentangled concept-atom axes, enabling direct and interpretable readout of semantic selectivity.
linear combination of neurons. Superposition thus emerges naturally as an efficient representational strategy under data sparsity and limited capacity. Inspired by earlier work word embeddings, recent studies have applied sparse coding to dense DNN activations to recover interpretable basis factors [10 58 21 In partieular, sparse autoencoders trained large language model (LLM) activations have shown that a few hundred neurons can represent thousands of distinct, interpretable features in superposition [
2.3 Limitations of Dense Encoding Models
The seminal work of Huth et al. [22 used word embeddings to represent narrative stimuli and revealed widespread conceptual representations across the human cortex. This approach catalyzed growing body research using machine-leaming-deere
features to predict brain activity. These studies typically employ regularized linear regression to fit a separate model for each voxel (a volumetric unit of brain tissue measuring approximately x 3 x millimeters), producing a weight vector that
characterizes the voxel's feature selectivity [15] 35 30).

---

However, interpreting regression weights from dense encoding models presents a fundamental ambiguity. In dense embeddings (Fig. left), the coordinate axes are not inherently interpretable; semantic concepts are represented by correlated linear combinations of basis directions. This entangled structure complicates interpretation, as the relationship between voxel weights and semantic concepts becomes opaque. Conventional analyses often rely on heuristic assumptions: for example, inferring that a voxel is selective for a concept if its weight vector projects positively onto that concept's s direction. But such inferences are unreliable when concept directions are correlated. A voxel aligned with number, for instance, may appear to also respond to time simply because the two concept directions femaleness and space in the diagram, may be strongly anti-correlated, making unclear whether a voxel is activated by one, suppressed by the other, or both. These ambiguities undermine principled interpretation: a positive projection does not necessarily imply true selectivity to a concept; observed overlaps in cortical maps may reflect either genuine multi-concept selectivity or spurious effects induced by embedding geometry. The root of this ambiguity is formalized in the Theorem of Non-Identifanl Von-Identifiabilityy in Dense Embedding Spaces (Appendix [B) In contrast, an ideal feature space would have an interpretable basis in which each axis corresponds to a distinct semantic concept (Fig. right). In such a space, voxel's tuning could be directly read from its projections along concept axes. This axis-aligned structure is the interpretability goal we aim to achieve with the Sparse Concept Encoding Model.
positively correlated. More problematically,, many concept pairs, such as
2.4 The Sparse Concept Encoding Model
To circumvent the identifiability limitations of dense embeddings, we aim to recover the latent semantic directions and assign each its own coordinate, effectively lifting the dense feature space into an interpretable, axis-aligned representation (Fig. [ right). We achieve this through non-negative sparse dictionary learning applied to a large text corpus, producing an overcomplete set of semantic axes that we term concept atoms. Specifically, we expand the original d-dimensional embedding space into an m-dimensional space (m > d), where each axis corresponds to a distinct concept atom (Fig. 3] middle). Each stimulus vector is then re-encoded as sparse, non non-negttive linear combination of these atoms (see Fig. for an example). This transformed representation replaces the original dense features in the encoding model. In the resulting sparse feature space, voxel weights become directly interpretable: each dimension aligns with a distinct semantic concept, enabling principled readout of conceptual selectivity from the corresponding weight values (Fig. 3 bottom). This structure facilitates clean disentanglement of weights for voxels jointly selective for multiple concepts. The non-negativity constraint further sharpens interpretability by ensuring that positive weights indicate activation and negative weights indicate suppression. Together, these properties enable clear interpretation of conceptual selectivity across the cortex.
Both the Sparse and Dense Encoding Models can be interpreted under a Bayesian regression frame- work [35]. From this view, the Dense Model corresponds to the Sparse Model with non-spherical Gaussian prior whose covariance captures correlations among concept directions in the dense space. A formal derivation is provided in the Appendix B
3 Experiments
3.1 Voxelwise Modeling and Feature Construction
We evaluated the Sparse Concept Encoding Model on a publicly available fMRI dataset in which participants listened to naturalistic autobiographical audio stories (22). Our analysis follows the voxelwise encoding model framework, emphasizing high-resolution. within-subject modeling in which model training and evaluation are performed on separate data from the same individual [[5].
We present cortical maps for two representative participants and report group-level statisties across all seven participants.
To construct the dense feature space, we used pre-trained 300-dimensional GloVe word embed- dings [11. The corresponding sparse feature space was obtained using non-negative sparse dictionary learning, following the procedure of Zhang et al. [59] Using their public implementation, we learned a 1, d.000-dimeesiionll overcomplete dictionary of concept atoms from the GloVe vectors. For each atom, we identified the top-activating words from the story transcripts to aid interpretability. Figure
5

---

shows illustrative examples, and a broader list of 50 representative concept atoms is provided in Appendix[[]
key question is whether the dense feature space already provides sufficient semantic separation for voxel weights to be interpreted directly. If distinct concepts were represented along nearly orthogonal directions, sparse transformation would be unnecessary. To evaluate this, we examined the pairwise cosine similarities among the 1,000 learned concept atoms in the original 300-dimensional Glo Ve space. For reference, 1,000 random unit vectors in 300-dimensional space are expected to be nearly orthogonal: their pairwise cosine similarities have mean μ O, standard deviation 0.058, and only 0.03% of pairs exceed 0.2 in magnitude. In contrast, the concept atoms exhibited substantial overlap. with mean pairwise similarity ب 0.01, standard deviation pairs exceeding 0.2 (Fig. 4b). This structured correlation pattern indicates that the dense feature space represents concepts along moderately correlated directions, reaffirming the need for sparse, axis-aligned representation.
0.075, and 2.18% of
3.2 Improving Embedding Geometry with Vector Norm Reparameterization
known property of dense word embeddings is that high-frequency words tend lie closer the origin, likely because their vectors represent averages over more diverse contextual usages 44 2].
We observe that sparse coding often captures many high-frequency words with different semantic meanings within the same dictionary atoms. To improve separability among tigh-frequency words, we developed preprocessing technique called Vector Norm Reparameterization (VNR). which differentially rescales each word vector according to its norm, expanding the relative spread of vectors near the origin while compressing those among high-norm vectors. Quantitative comparisons with standard preprocessing methods (Appendix G show that VNR improves WordNet purity, reconstruction error, and sparsity. Based on these results, we adopted VNR for all subsequent analyses.
3.3 Comparing Predictive Performance of Dense and Sparse Models
To compare the predictive performance and interpretability of the Dense Encoding Model and the Sparse Concept Encoding Model, we fit voxelwise encoding models using two types of semantic features: 300-dimensional dense embeddings for the Dense Model and 1.000-dimensional sparse embeddings for the Sparse Model. Both models were trained using regularized linear regression with cross- vosss-valiationn and evaluated on held-out test story using the coefficient of determination (R²) as the performance metric. Statistical significance was assessed at the voxel level using 9,999 block permutations with false discovery rate (FDR) correction at p < 0.05 (Appendix [F). To visualize prediction performance, we plotted voxelwise v RR values onto cortical flatmaps [18]. The Sparse Model achieved strong prediction accuracy across temporal, parietal, and prefrontal cortices in both participants (Fig. 4p-c). To directly compare models, we plotted voxelwise VR scores across all significantly predicted voxels (Fig. fl-e). Across subjects, the mean difference in prediction performance between the transformed and original models was -0.00047 + 0.00071, with no statistically significant difference detected (paired f66 = -1.77, = 0.13). These results indicate that the Sparse Concept Encoding Model achieves comparable prediction accuracy to that of the Dense Encoding Model while substantially improving interpretabilltyy
3.4 Concept-Level Cortical Maps from the Sparse Concept Encoding Model
Prior work has shown that abstract concepts are represented in the brain through widespread, spatially overlapping cortical patterns rather than anatomically localized regions [23 22). However, dense encoding models lack principled way to disentangle the cortical representations of individual concepts. The Sparse Concept Encoding Model addresses this by representing each concept as a distinct axis in sparse feature space. These axes, or concept atoms, correspond to interpretable semantic dimensions, which can be identified by inspecting the tcp-activating stimulus words (see Appendix C Table I]. In this and Section (36]] focus on the 20 atoms with the highest average cortical activation, as they provide the most robust signal for analyzing concept-level representations.
To assess whether the Sparse Concept Encoding Model yields consistent and interpretable cortical maps for individual semantic concepts, we examined two illustrative concept atoms, indices 456 and 1 115, drawn from the top 20 atoms (Fig. 5 These atoms are interpretable as representing the

---

a)
Pairwise Cosine Similarities Between Concept Atoms
Prediction Performance
b)
d)
Subject 01
Subject 01
(R) Dowe Esscerattoee
0.0
0.3
Subject 02
23
-2
.2
Subject
Figure 4: The Sparse Concept Encoding Model preserves prediction accuracy while improving interpretability. (a) Pairwise cosine similarities between all 1,000 concept atoms in the dense embedding space reveal structured correlations, with many atom pairs showing moderate cosine similarity. (b-c) Cortical maps of prediction accuracy (v R2) for the Sparse Concept Encoding Model in Subject 01 and Subject 02, showing strong performance across lateral temporal (LTC), medial parietal (MPC). and prefrontal (PFC) regions. (d-e) Scatter plots comparing prediction accuracy between the Sparse and Dense Encoding Models across significantly predicted voxels.
concepts of "body position" and "family", respectively. For each atom, we listed the most strongly activating stimulus words and, for two participants, visualized voxelwise regression weights along the corresponding concept-atom axis on cortical flatmaps. The resulting maps revealed bilateral, distributed patterns of selectivity that are consistent across individuals.
As baseline, we compared these maps to those derived from the Dense Encoding Model. Since the dense model lacks principled method for generating concept-level maps, prior work has relied on heuristic strategies. Common approaches include: (1) assigning each voxel to the concept whose embedding vector has the highest cosine similarity its weight vector, which can underestimate mixed selectivity; and (2) projecting voxel weights onto individual semantic directions, which can overestimate selectivity due correlations in dense space. In this work, we implemented hybrid heuristic: labeling voxel as selective for concept if that concept ranked among its top- 10 cosine similarity matches. Maps produced by the dense model were less spatially coherent, less bilaterally symmetric, and more variable across individuals (Appendix H)
To quantify inter-subject consistency across models, we projected each participant's cortical maps onto the common fsaverage surface and computed pairwise cosine similarities across all seven individuals. Averaged across the top 20 concept atoms, the Sparse Concept Encoding Model yielded substantially higher inter-subject consistency (mean + SD 0.26 + 0.04) compared to the Dense Encoding Model (0.09 = 0.04). These results indicate that the sparse model produces more consistent semantic representations across subjects.
3.5 Case Study: Disentangling Time, Space, and Number Representations
A longstanding hypothesis in cognitive neuroscience posits that the brain encodes time, space, and number using shared magnitude system [I] 53 [2]. While this theory is supported by robust behavioral and psychophysical evidence, neuroimaging results have been inconsistent (9, 48). The Sparse Concept Encoding Model enables a novel test of this hypothesis by assigning time, space, and number independent axes in shared semantic space, allowing direct comparison of their cortical representations. To evaluate this, we identified the concept atoms most associated with each domain based their top-activating stimulus words (Fig. 6). To normalize for variation in
7

---

Stands sidewalk
standing
rou
as
couch
Loot Shindd
its
d)
family
friends birthday
brother stepmot
ATOM 456
lekks
aislewaiting
ATOM 115
grandfather
dad
mom
brother's
eett front tood
parents friend
father COUS inddaughee
Sholding
sitting leans
naked
siibiigg ngsson husband
wirent wiffff
sister
crouchone page
desk wilking
lay
lying
"relatives grandparent
Sit ledge sat
walked
mother
girlffriend
b)
e)
MPC
UP
Lure
Septerts
Anterior
Subiect
Subject 01
c)
NPC
Subject 02
-001
.01
Subject 02
Figure 5: The Sparse Concept Encoding Model yields consistent, interpretable cortical maps for individual semantic concepts. (a, d) Two example concept atoms (indices 456 and 1 115) from the top 20 most strongly activated atoms, interpreted as representing the concepts of "body position" and "family," respectively. Each word cloud shows the top-activating stimulus words for the corresponding atom. (b-c, e-f) Voxelwise regression weights projected onto the selected concept-atom axis for Subject 01 and Subject 02. Red indicates positive selectivity and blue negative. Opacity is scaled by cross-validated prediction accuracy, and only voxels with significant prediction performance beakkknnff-estimateee p 0.05) are shown. Both atoms exhibit distributed, bilaterally symmetric selectivity patterns that are consistent across individuals. These results show that the Sparse Concept Encoding Model successfully disentangles individual concept representations and produces coherent cortical maps.
activation magnitude across concept atoms, we selected the top 5,000 voxels with the highest positive regression weights for each cortical map. The resulting composite maps revealed localized, bilateral overlap among the three conceptual domains, indicated in white. These patterns were consistent across individuals. Notably, regions of three-way overlap included the bilateral intraparietal suleus (IPS) and bilateral inferior frontal gyrus (IFG), areas previously implicated in shared magnitude processing [9] 48). To quantifyy the extent of overlap, we calculated intersection-over ovver-union (IoU) scores for each concept pair and their three-way intersection 49 9 48). Averaged across seven participants, IoU scores were 0.34 0.05 for ame-space, 0.32 + 0.07 for time-numbie 0.32 + 0.07 for space-number, and 0.18 + 0.05 for voxels jointly selective for all three. These findings support the shared magnitude system hypothesis, offering fMRI evidence from naturalistic stimuli enabled by novel and interpretable modeling framework.
3.6 Case Study: Comparing Cortical Representations of Different Concepts
Understanding how the brain encodes relationships between abstract concepts is a central goal in cognitive neuroscience. example, are logical composition concepts such as "combine", "overlap", and "exclude" represented similarly to body position concepts like "lie," "sit," and "slouch"? To address this, we compared cortical tuning maps across the top 20 concept atoms. For each atom, generated cortical maps described in Section 3.4 and computed pairwise cosine similarities

---

a)
d)
three
fifffeen
thousand
eleven four
MI
five
eighteen,
seven fifty
PFC
nine sixty several
dozen
few
sixteen
twentyeighy
six
hundred
nineteen
ninety
forty
two weteeeighteeenteeee
seventy
WTTC
LTC
b)
nake everyday morning
arrived
day
weekends late
Subject 01
shifts a aoon
Siperor
hours
lunch
tonight
hour
today y
appot
apolnteent
monday ynight
until
Anterio
Time
Space
Anteror
weeting
yesterday
evening
anytine
waking
closed
tuesday
TOC
IIFC
PF
villageaway
train
hal
complex
Teay
opposite
minute,
places
close
shopping
restaurants
park
center
stop
airport
LTC
LTC
distance
around
anous
map
milewalkmi
base
es
town
wi weahinnear dr
iveoussie
Subject 02
Figure 6: Sparse Concept Encoding Model reveals overlapping cortical representations of time, space, and number. (a-c) Word clouds showing the top 30 stimulus words that most strongly activate three selected concept atoms, corresponding to the abstract domains of number (red), time (green), and space (blue). (d-e) Composite cortical maps for Subject 01 and Subject 02 showing voxelwise selectivity to each concept atom. Each map displays the top 5,000 voxels with significant prediction performance and positive regression weights (jackknife-estmmatee and blue denote selectivity to number, time, and space. respectively; overlapping responses are shown in secondary colors, and white indicates joint selectivity to all three. Some inter-individual variation is present, as expected in within-subjed, subject functional maps derived from voxelwise models using naturalistic stimuli. These maps reveal localized, bilateral three-way overlap among time, space, and number representations, most notably in the intraparietal sulcus (IPS) and inferior frontal gyrus (IFG). This pattern is consistent with the hypothesis that these domains are represented by shared magnitude system in the human brain.
p < 0.05) per atom. Red, green,
between all maps. yielding 20 x 20 concept-level similarity matrix. We applied hierarchical clustering to the similarity matrix from Subject 01 and visualized the reordered matrix alongside its dendrogram. Using the atom ordering, we then plotted the similarity matrix for Subject 02. The clustering patterns were qualitatively consistent across subjects and quantitatively stable across all subjects: the subjects was 0.70 + 0.08. To ensure this structure was not inherited from the input embeddings, we computed pairwise Pearson correlations between sparse feature vectors across the 190 concept atom pairs. These correlations were uniformly low (|r = 0.00 + 0.10; 85% < 0.10), confirming that the Sparse Concept Encoding Model effectively disentangles semantic directions. In contrast, the same concept directions in the original dense GloVe space showed positive moderate correlations (0.16 + 0.14; 87% 0), suggesting dense embeddings obscure fine grainer distinctions between semantic directions. These findings suggest that the brain organizes abstract concepts according to a structured similarity geometry, which becomes observable once the feature representations are disentangled.
off-diagonal Pearson correlation between similarity matrices across
4 Discussion, Limitations, and Broader Impact
We introduced the Sparse Concept Encoding Model, framework that addresses the interpretability challenges arising from superposition in dense feature spaces. Our experiments focused fMRI

---

a)
b) Subject 01, pairwise cosine d) Pairwise cosine similarities
similarities of cortical maps
in the dense embedding
Subject 01, hierarchical clustering solution based
862
862
218
218
pairwise cosine
586
585
similarities of cortical maps
626
825
670
870
893 Present Tense Action
540
579
335 Romantic Emotions
01
61
505
505
218 Female identity
585 Appointment Booking
456
456
115 Families and Relatives
369
369
626 Social Roles
562
562
670 Personal Narrative
10 N00 1000000 POND-II
254 Past Tense Actions
c) Subject 02, pairwise cosine
e) Pairwise correlations in the
672 Life History
similarities of cortical maps
sparse feature matrix
549 Present Tense Existence
798 Time Durations
335
335
210
218
81 Time Moments
115
115
505 Numbers
626
626
Singular Physical Objects
672
254
254
677
301 Plural Physical Objects
790
750
81
81
456 Body Position
261 Present Tense Intents
301
301
304 Expressions of Expediency
304
304
369 Oppositional Conjunctions
562 Probability Descriptors
300
115
790
905
Figure 7: Sparse Concept Encoding Model reveals consistent structure in concept-level cortical representations across subjects. (a) Dendrogram from hierarchical clustering of Subject 01 based on pairwise cosine similarities between cortical tuning maps for the 20 most strongly represented concept atoms. Concept labels were heuristically assigned by summarizing the top-activating stimulus words for each atom. (b, e) Concept-level cortical representation similarity matrices for Subject 01 and Subject 02. Each matrix shows pairwise cosine similarity between cortical maps for the 20 concept atoms, reordered using the clustering from Subject 01. The structure is qualitatively consistent across subjects, indicating shared cortical representational structure. (d) Cosine similarity matrix between the same 20 atoms the original dense Glo Ve embedding space, showing structured correlations that would confound interpretation under the Dense Encoding Model. (e) Pearson correlation matrix of sparse feature vectors used to train the Sparse Concept Encoding Model. Off-diagonal values are uniformly low, confirming that the sparse transformation yields disentangled semantic dimensions.
data and static word embeddings, but the framework is broadly applicable to domains requiring interpretable mappings from entangled features. One current limitation is that the model does not incorporate linguistic context, as it relies on non-contextual word embeddings. Extending to contextualized representations from large language models is a promising direction for future work (24).
Although concept atoms improve interpretability, they may inherit structural biases from the original embeddings, potentially amplifying harmful stereotypes. Moreover, while this method could enhance brain Saii-coompter interface technologies by enabling more accurate semantic decoding, it raises ethical concerns around mental privacy and the potential misuse of neural decoding tools [19].
Acknowledgments and Disclosure of Funding
We thank Bruno Olshausen and Yubei Chen for helpful discussions. This work was supported in part by the National Eye Institute (NET) of the National Institutes of Health under award R01 EY031455.
10

---

References
[1] Richard Antonello, Aditya Vaidya, and Alexander Huth. Scaling laws for language encoding models in IMRI. Advances in Neural Information Processing Systems, 36:21895-21907, 2023.
[2] Sanjeey Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A latent variable model approach to PMI-based word embeddings. Transactions of the Association for Computational Linguistics, 4:385-399, 2016.
[3] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy, 2018, arXiv preprint 16Xiv:6601.0320
[4] Vanda Balogh, Gábor Berend, Dimitrios I. Diochnos, and György Turán. Understanding the semantic content of sparse word embeddings using commonsense knowledge base. Proceedings of the AAAI Conference on Artificial Intelligence, 3465):739--7406, 2020.
[5] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Toraalba. Network dissection: Quantify- ing interpretability of deep visual representations, 2017, arXiv preprint sxXiv:174.005996
[6] Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1) (1299 300, 1995.
[7] Laurent Bonnasse-Gatle and Christophe Pallier. finri predictors based on language models of inereas- ing complexity recover brain left lateralization. Advances in Neural Information Processing Systems, 17:122331-222663 2024.
[8] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby. Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticityy; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023, bttps:/trrassoomrr-iiicct put::000mmmmoooommmmi hew.ann/ifeetttur//nee....
[9] Giorgia Cona, Martin Wiener, Francesco Allegrini, and Cristina Searpazza. Gradient organization of space, time, and numbers in the brain: A meta-analysis of neuroimaging studies. Neuropsychology review, 34( 34(3):721-737, 2024.
[10] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models, 2023, arXiv preprint arXiv:2309.08600
[11] Stanislas Dehacne. The number sense: how the mind creates mathematics. Oxford University Press, New York, 1997.
[12] Stanislas Dehaene and Elizabeth M. Brannon. Space, time, and number: a kantian research program. Trends in Cognitive Sciences, 14(12)511-51) 201,
[13] Fatma Demiz, Anwar O. Nunez-Elizalde, Alexander G. Huth, and Jack L. Gallant. The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality. Journal of Neuroscience, 39(39))7722-7736 2019.
[14] Tom Dupré la Tour, Michael Eickenberg. Anwar O. Munez-Elizalde and Jack L. Gallant. Feature-space selection with banded ridge regression. Neurolmage, 197288, 2022.
[15] Tom Dupré la Tour, Matteo Visconti di Oleggio Castello, and Jack L. Gallant. The voxelwise encoding model framework: A tutorial introduction fitting encoding models to IMRI data. Imaging Neuroscience, 3:imag a 00575, 2025.
[16] Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan. Shauna Kravec, Zac Hatfield-Dodee Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse. Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022. httpp:///rranaffomee-iiirre wttp://uwt.t..pb//////////////// (tto:///ttommmde//iiiee moodel/indes. html
[17] Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah A. Smith. Sparse overcomplete word vector representations. In Chengqing Zong and Michael Strube, editors, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1491 1500, Beijing, China, 2015. Association for Computational Linguistics.
11

---

[18] James S. Gao, Alexander G. Huth, Mark D. Lescroart, and Jack L. Gallant. Pycortex: an interactive surface visualizer for IMRL Frontiers in Neurvinformatics, 9, 2015.
[19] Sara Goering, Eran Klein, Laura Specker Sullivan, Anna Wexler, Blaise Agüera y Arcas, Guoqiang Bi, Jose Carmena, Joseph Fins, Phoebe Friesen, Jack Gallant, Jane E. Huggins, Philipp Kellmeyer, Adam Marblestone, Christine Mitchell, Erik Parens, Michelle Pham, Alan Rubel, Norihiro Sadato, Mina Teicher, David Wasserman, Meredith Whittaker. Jonathan Wolpaw, and Rafael Yuste. Recommendations for responsible development and application of neurotechnologies. Neumethics, 14(3):365-386, 2021.
[20] Aricl Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Lora Fanda, Wemer Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Roi Reichart, Sasha Devore, Adeen Flinker, Liat Hasenfratz, Omer Levy, Avinatan Hassidim, Michael Brenner, Yossi Matias, Kenneth A. Norman, Orrin Devinsky, and Uri Hasson. Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3):369 380, 2022.
[21] Wes Gurnee, Neel Nanda. Matthew Pauly, Katherine Harvey, Dmiiri Troitskii, and Dimitris Bertsimas. Finding neurons a haystack: Case studics with sparse probing, 2023, arXiv preprint ar arXiv:2305.016
[22] Alexander G. Huth, Wendy A. de Heer, Thomas L. Griffiths, Frédéric E. Theunissen, and Jack L. Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600):453 -453-458, 2016.
[23] Alexander G. Huth, Shinji Nishimoto, An T. Vu, and Jack L. Gallant. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, (616))1210--1 1220000-12,222221 2012.
[24] Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fMRI. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[25] Andrej Karpathy, Justin Johnson, and Li Fei-Fei. Visualizing and understanding recurrent networks, 2015. arXiv preprint #rXiv:1506.0207:
[26] Sreejan Kumar, Theodore R. Sumers, Takateru Yamakoshi, Ariel Goldstein, Uri Hasson, Kenneth A. Norman, Thomas I.. Griffiths, Robert D. Hawkins, and Samuel A. Nastase. Shared functional specialization in transformer-hased language models and the human brain, 2023.
[27] Matthew K. Leonard, Laura Gwilliams, Kristin K. Sellers, Jason E. Chung, Duo Xu, Gavin Mischler, Nima Mesgarani, Marleen Welkenhuysen, Barundeb Dutta, and Edward F. Chang. Large-scale single-neuron speech sound encoding across the depth of human cortex. Nature, 62677999),533-662, 2024.
[28] Yuanning Li. Gopala K. Anumanchipalli, Abdelrahman Mohamed, Peili Chen, Laurel H. Cameey. Junfeng Lu, Jinsong Wu, and Edward F. Chang. Dissecting neural computations in the human auditory pathway using deep neural networks for speech. Nature Neuroscience, 26 26(12):221-22)) 2023.
[29] Kaylo T. Littlejohn, Cheol Jun Cho, Jessie Liu, Alexander Silva, Bohan Yu, Vanessa Anderson, Cady M. Kurtz-Miott, Samantha Brosler, Anshul P. Kashyap, Irina P. Hallinan, Adit Shah, Adelyn Tu Tu-Chan, Karunesh Ganguly, David A. Moses, Edward F. Chang, and Gopala K. Anumanchipalli. A strcaming brain-to-voice neuroprosthesis to restore naturalistic communication. Nature Neuroscience, 2824)900-912, 2025.
[30] Mackenzie Weygandt Mathis, Adriana Perez Rotondo, Edward F. Chang, Andreas S. Tolias, and Alexander Mathis. Decoding the brain: From neural representations to mechanistic models. Cell, 187211)5511-5582, 2024.
[31] Emily X. Meschke, Mattco Visconti di Oleggio Castello, Tom Dupré la Tour, and Jack L. Gallant. Model connectivity: leveraging the power of encoding models to overcome the limitations of functional connectivity. 2023.
[32] Tomas Mikolov. Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Lucy Vanderwende, Hal Daumé III. and Katrin Kirchhoff, editors, Pruceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia, 2013. Association for Computational Linguistics.
[33] George A. Miller. WordNet: lexical database for english. Commun. ACM, 38(11):39 41, 1995.
12

---

[34] Tomoya Nakai and Shinji Nishimoto. Quantitative models reveal the organization of diverse cognitive functions the brain. Nature communications, [I(1):1142, 2020.
[35] Anwar O. Nunez-Elizalde, Alexander G. Huth, and Jack L. Gallant. Voxelwise encoding models with non-spherical multivariate normal priors. Neurolmage, 197:482-492, 2019.
[36] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization.
Distill,
1/di: 2017.
[37] Michael Oliver, Michele Winter, Tom Dupré la Tour, Michael Eickenberg, and Jack L. Gallant. A hollogiaall-innpinee hierarchical convolutional energy model predicts v4 responses to natural videos, 2024.
[38] Bruno A. Olshausen and David Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583))607-609 1996.
[39] Bruno A. Olshausen and David J. Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research, 37(23):331 131-3325, 1997.
[40] Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. WordNet:similarity measuring the relatedness of concepts. In Demonstration Papers at HL.7-NACCL. 2004, pages 38-41, Boston, Massachusetts, USA, 2004. Association for Computational Linguisties.
[41] Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word repre- sentation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar, 2014. Association for Computational Linguistics.
[42] Belinda Phipson and Gordon K. Smyth. Permutation p-values should never be zera: calculating exact p-values when permutations are randomly drawn. Statistical Applications Genetics and Molecular Biology, 9:Article39, 2010.
[43] Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield- Toward transparent Al: A survey interpreting the inner structures of deep neural networks, 2023. arXiv preprint arXiv:2207. 132443
[44] Adriaan MJ Schakel and Benjamin Wilson. Measuring word significance using distributed representations of words. arXiv preprint arxiv:/500 02297, 2015.
[45] Martin Schrimpf. Idan Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko. Artificial neural networks accurately predict language processing in the brain, 2020.
[46] Lee Sharkey. Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq. Nicholas Goldowsky- Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic inter- pretability. arXiv preprint arXiv:2501 012220011.144.... 2025.
[47] Alexander B. Silva, Jessie R. Liu, Sean L. Metzger, Ilina Bhaya- Groosann Maximilian Dougherty, Margaret P. Seaton, Kaylo T. Littlejohn, Adelyn Tu-Chan, Karunesh Ganguly, David A. Moses, and Edward F. Chang. A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages. Nature Biomedical Engineering, 8(8)977 911199911222 2024.
[48] Hazal Şimsek- Henne,,-Uvver, Burcu networks underlying magnitude perception: specific meta-analysis fmri studies. Cerebral Cortex. bha7).b 2025.
Beyza Doganay, Fuat Balcı, and Metehan Çięek. Neural
[49] Kenny Skagerlund, Thomas Karlsson, and UIf Träff. Magnitude processing in the brain: an fmri study of time, space, and numerosity as a shared cortical system. Frontiers in human neuroscience, 10:500, 2016.
[50] Jerry Tang, Meng Du, Vy Vo, Vasudev Lal, and Alexander Huth. Brain encoding models based on multimodal transformers can transfer across language and vision. Advances in Neural Information Processing Systems, 36:29654-29999 22196666266666..-.1 2023.
[51] Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G. Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26(5):858-866, 2023.
[52] Tom Dupre la Tour, Michael Lu, Michael Eickenberg, and Jack L. Gallant. A tiner mapping of convolutional neural network layers to the visual cortex. In SVRHM 2021 Workshop@ NewrIPS, 2021.
13

---

[53] Vincent Walsh. A theory of magnitude: common cortical metrics of time, space and quantity. Trends in Cognitive Sciences, (11)):483488) 2003.
[54] Leila Wehbe, Alexander G Huth, Fatma Deniz, J Gao, Marie-Luise Kieseler, and Jack I. Gallant. Bold predictions: Automated simulation of fmri experiments. NeurIPS Demonstr: Track, 2016.
[55] Leila Wehbe, Brian Murphy, Partha Talukdar, Alona Fyshe, Aaditya Ramdas, and Tom Mitchell. Simulta- neously uncovering the patterns of brain regions involved in different story reading subprocesses. PloS One, 9(1 ((11)ee112 12575, 2014.
[56] Michael C.-K. Wu, Stephen V. David, and Jack L. Gallant. COMPLETE FUNCTIONAL CHARACTERL- ZATION OF SENSORY NEURONS BY SYSTEM IDENTIFICATION, Annual Review of Neuroscience, 29:477-505, 2006.
[57] Daniel Yamins and James DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19(3):356-366 2016.
[58] Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors, 2023. arXiv preprint 11X999
[59] Juexiao Zhang, Yubei Chen, Brian Cheung, and Bruno A. Olshausen. Word embedding visualization via dictionary learning, 2021. arXiv preprint arXiv: 1910.03833.
14

---

A Experimental Data and Encoding Model Pipeline
A.1 Data Availability
The raw (MRI dataset used in this study is publicly available via the Gallant Lab GIN repository Preprocessed fMRI data and aligned stimulus features are also available on OSF
All data used in this study-excluding anatomical MRI scans-have been made publicly accessible. Anatomical images were withheld to protect participant privacy. To ensure full reproducibility. we provide transformation matrices that map volumetric data to individual cortical flatmaps, enabling accurate surface-based visualization of the results.
A.2
Experimental Procedure
The dataset used in this study was originally collected for previously published experiment [22] All procedures related to participant recruitment, scanning parameters, and data acquisition were described in that work. Briefly, each participant completed eleven fMRI scans while listening to narrative audio stories from The Moth Radio Hour, in which individuals recount autobiographical experiences before a live audience. Each scan featured a single story lasting 10-15 minutes, with 10 seconds of silence preceding and following the audio.
A.3 Dataset Structure and Test/Train Split
The full dataset comprised eleven stories per participant. Ten stories were used for training the encoding models, yielding approximately 125 minutes data per subject. One remaining story was held out for testing and presented twice to each participant. To improve the signal- signal-to-noise ratio in test data, BOLD responses across two repetitions were averaged. Final model evaluations were conducted on this averaged 10-minute test set.
A.4 [MRI Preprocessing
Functional MRI data were preprocessed using the same pipeline as described in previously published study (31]. The resulting fMRI response matrices consisted of approximately 60,000 60,000080,000 80,000 voxels per subject. All analyses were conducted in each subject's native voxel space, with no spatial smoothing applied.
A.5 Feature Construction
Low-Level Acoustic Features. To control for correlations between semantic content and acoustic
structure, we included 41 nuisance regressors following 13
39 phoneme-rate features reflecting the frequency of each English phoneme over time
A single feature representing total phoneme rate
A single feature representing word rate
Semantic Features. Each word the stimulus transcript was transformed into two parallel semantic representations:
3000dimmnnsionll dense Glo Ve embeddings
41
1.000-dimennioonal sparse concept embeddings derived via dictionary learning from GloVe
All features were resampled to match the fMRI acquisition rate using a 3-lobe Lanczos low-pass filter with cutoff of 0.249 Hz (the Nyquist frequency). This resulted in training feature matrices of size 3.717 x 300 for the Dense Encoding Model and 3,717 x 1,000 for the Sparse Concept Encoding Model.
15

---

To account for hemodynamic lag. finite impulse response (FIR) model was applied to the design
matrix. Each feature time series was delayed by 1, 2. 3, and TRs (corresponding 4, 6, and seconds) and concatenated, allowing the model to estimate linear temporal kernel per feature [15
A.6 Feature Normalization
All stimulus features demeaned over time within each scan except for the 400-dimensional GloVe embeddings. These embeddings were left unchanged to maintain a consistent origin across runs. Because cosine similarity depends vector direction relative to fixed origin, shifting the origin across runs would undermine interpretability. In contrast, sparse concept features and low-level acoustic features were demeaned. For these features, mean-centering does not affect the interpretation of voxel weights.
A.7 Model Estimation
Voxelwise encoding models were estimated using banded ridge regression [4, which applies separate regularization to the semantic and low-level feature subspaces. For each voxel, two regularization hyperparameters (Asemanti and levell were optimized using leave-one-run-our cross-validation on the training set.
The procedure was as follows:
The 10 training runs were split into 9 training and validation run, iterated over all folds.
For each fold, 20 logarithmically spaced values (from 10 to 10²²) were tested for each hyperparameter.
Prediction accuracy was computed on the held-out run.
The hyperparameter pair yielding the highest average prediction accuracy across folds was selected separately for each voxel.
Model fitting was implemented using the Himalaya Python library 14
which enables efficient
banded ridge regression on both CPUs and GPUs. Training required approximately 30-60 minutes per subject on a single NVIDIA RTX A6000 GPU.
A.8
Model Evaluation
Final models were evaluated on the held-out test story using the average blood-oxygen-level. dependent (BOLD) responses from two repeated presentations. Prediction accuracy computed as the coefficient of determination (IP2) between predicted and observed fMRI responses, independently for each voxel.
A.9 Postprocessing of Semantic Weights
Model weights were first averaged across the five FIR delays to yield a single value per feature voxel. To remove differences in overall scale introduced by voxel-specific ridge regularization, each semantic weight vector was normalized to unit length. Vectors were then scaled by the voxel's cross-validated prediction accuracy, allowing downstream analyses to reflect both tuning direction and model reliability. Negative accuracies were clipped to zero before scaling.
A.10 Software Implementation
All analyses were implemented in Python using custom framework called sparseconcept, which openly available on GitHub[ [T] The analysis pipeline relies on standard scientific Python libraries including numpy, scipy, matplotlib, scikit-learn, statsmodels and pycortex, and uses the himalaya package with Py backend for efficient voxelwise model fitting.
httep://gicabbbcccm/gglllaaaaaaa
16

---

B Non-identifiability of the Dense Encoding Model and Bayesian Interpretation of Dense Sparse Encoding Models
B.1 Theorem (Non-identifiability in Dense Embedding Spaces)
Let {f] C Rd be unit-norm concept atoms, and define the Gram matrix:
G (G,,li,== Gij (f.,f))
Suppose a voxel's weight vector is tuned to linear combination of these atoms:
Or2 f.
(a.,....a T
Taking inner products with each fy yields the observed projections:
(w, f()
Ga.
(w, fe)
If G' is invertible, then can be uniquely recovered. However, if G' is singular, the system is non- identifiable multiple combinations of concept atoms can yield the same weight vector. In the ideal case where all are orthogonal, G I and a can be directly recovered.
B.2 Bayesian Interpretation of Sparse and Dense Encoding Models
Both the Sparse Concept Encoding Model and the Dense Encoding Model admit Bayesian linear regression interpretation [35]. This view makes explicit the prior assumptions each model imposes on voxel tuning over semantic features.
Sparse Concept Encoding Model. Each stimulus is represented by sparse, non-negative activation over learned concept atoms, yielding design matrix Z 7'xm, Voxel responses are fit with ridge regression
B arg min BERm
lly 2 |||
which corresponds to MAP inference under an isotropic Gaussian prior
B ~ 00, 'T).
This prior assumes independent contributions of each concept atom with equal variance.
Dense Encoding Model. Let the dense feature matrix be factorized as X -Φ, where Roxx contains the concept atoms as row vectors in the dense embedding space, and fit ridge regression in dense space:
arg min ||y Zdw
Al| 1l
w WERL
Define the implied sparse-space weights 3 = dw E Rm As shown below, dense ridge is equivalent sparse-space regression with structured (generally degenerate) Gaussian prior
e ~ 3~0,
When 4Φ T is singular, this is a degenerate Gaussian supported on C(Ø); its MAP penalty is 3 (# T + Intuitively, atoms with similar directions (large dot product) will biased to have coupled weights under the prior. This coupling can improve prediction when related concepts truly co-activate, but complicates interpretation: voxel weights may reflect embedding correlations rather than true semantic selectivity.
17

---

B.2.1 Proof
Start from dense ridge:
arg min lly 24w A//w|le-
w WR
Let B &w. Then Ily Zdww y 2/ and feasible , satisfy 3 E C(Φ).
Case A: rank( = (injective). For each ,3 E C(Φ) there is a unique w such that dw = B. given by
(Φ -) - T
Substituting into the objective yields
min lly 2 AB (Φ4) ) BEC(
arg
m and is invertible, then C(+) = Rm and ($C T)+ = (Φ T)-1.
Case B: rank( < d (non-injective). For fixed B E C()), the constraint 4w B admits infinitely many solutions. Moreover, w satisfying 2w = B yield the same prediction Z Z3, so the data-fit term is constant over this solution Dense ridge therefore selects the solution with minimal norm,
minn arg min w: +
[
which is given by the Moore-Penrose pseudoinverse,
min BiB
p = T
Substituting Wmin into the regularizer gives
min |w | 3 ( T #
and thus recovers the same sparse-space objective as in Case A:
e arg min BEC(
||y 2 + ABT ( +
C Top 50 Sparse Concept Atoms and Associated Words
We identified the 50 concept atoms most strongly expressed in the Sparse Concept Encoding Model, based on their average activation across voxels. During analysis, we found that a small number of atoms exhibited high activation correlations with many others the sparse feature matrix. This redun- dancy complicates interpretation by making it difficult to isolate distinct conceptual contributions to neural responses. To address this, we excluded highly redundant atoms from all subsequent analyses. Specifically, we computed the pairwise Pearson correlation matrix across all 1,000 sparse feature dimensions and calculated the average absolute correlation of each atom with all others. Atoms whose average correlation exceeded 70% of the maximum observed value were removed. This thresholding procedure yielded decorrelated subset of concept atoms better suited for interpreting voxelwise model weights. From the remaining atoms, we computed the mean absolute activation across voxels using the postprocessed model weights. The atoms with the highest average activations were selected, as they were most strongly engaged across the brain. To interpret the semantic content of each atom, we identified the most strongly associated words from the training stories. Only words with activation value greater than 0.2 for given atom were retained. For each selected atom, the 20 highest-ranker words (sorted by activation strength) are listed in Table [ along with the atom's original index from the full 1, 1.000-dimeniional dictionary.
18

---

Table 1: Fifty concept atoms with the highest average cortical activation, listed with their top associated words.
Index
Top Associated Words
505
cleven, sixteen, nineteen, fifteen, seventeen, twenty, ten, eighteen, thirty, sixty usually, often, also, normally, may, actually, always, might, definitely, sometimes
562
254
jumped, ran, came, pulled, blew, proceeded, look, stood, fell, stopped
115
grandparents, sister, dad, cousins, brother, siblings, mom, grandfather, mother, stepmother Iearn, want, able, take, join, to, need, begin, wish, go
261
585
call, telephone, appointment, phone, us, anytime, ask, queries, answering. number business, personal, home, city, family, school, music, game, woman, life
670
369
although, though, however, despite, but, yet, anyway, surprisingly, somewhat, far
549
is, becomes, an, itself, a, exists, another, comes, represents, was cars, dolls, sticks, boxes, pillows, shirts, cans, beds, are, cards
301
218
herself, queen, goddess, her, waitress, she, sister, mother, woman, whore
672
died, since, graduated, moved, dated, began, until, lived, existed, was easy, easier, quick, simple, instantly, allows, handy, lets, solution, fast standing, sitting, stood, sat, sit, crouch, aisle, stands, lying, behind
304
456
798
months, hours, days, weeks, ago, years, minutes, hour, month, year cow, bird, hat, cat, tiny, bag, ball, pink, baby, tree
719
893
pulls, sees, leans, asks, grabs, takes, waits, thinks, goes, gets morning, afterooon, evening, weekends, monday, tuesday, lunch, night, hour, day
81
626
guy, sailor, boy, woman, waitress, man, doctor, stranger, mister, kid cry, heart, eyes, soul, breath, lover, alive, dream, cruel, smile acknowledged, insisted, suggested, explained, said, admitted, saying. recounted, claimed, told thanks, hello, sorry, hey, bye, hi, dear, thank, yeah, ya am, wrote, by, at, on, subject, anonymous, views, writes, abuse discuss, topic, explored, similarities, about, examining, issues, examine, subject, dealing
335
172
707
530
184
148
toward, towards, forward, fro, slowly, direction, moving, into, back, headed steve, dave, andy, tim, rob, kevin, matt, bob, todd, nick from, this, notes, ago
486
733
244
insanely, remarkably, incredibly, terribly, extremely, perennially, oddly, horribly, surprisingly, quite
494
gon, outta, ya, gotta, lotta, em, wanma, gonna, fo, shit page, here, visit, details, check, listed, list, information, above, see near, close, distance, airport, center, miles, opposite, minutes, village, mile
17
997
385
um, em, ch, logal, atc, nova, do many, other, several, common, similar, different, such, particularly, most, certain
750
321
awesome, great, amazing, lovely, nice, beautiful, gorgeous, brilliant, definitely, neat talking, doing, enjoying, pushing, jumping, stopping, taking, chasing, throwing, keeping everywhere, anywhere, happening, in, within, exist, occurs, occurred, exists, somewhere brother. himself, father, son, his. grandfather, marquis, him, minister, afterwards families, residents, people. adults, students, women, children, relatives, men, fellow director, vice, professor, consultant, assistant, manager, says, said, joined, counselor teachers, school, teacher, kindergarten, classroom, grades, grade, education, students, gifted drank, counseled, listened, cared, worshipped, hated, travelled, liked, ate, lived hands, finger, arms, arm, cheek, shoulders, thumb, rubbing, squeezes, ears caused, cause, due, occurs, result, serious, suffered, failure, loss, problems shorts, shirts, jeans, shirt, pants, jacket, cropped, vest, lauren, casual bedroom, upstairs, room, downstairs, bathroom, bods, lobby, floor, closet, living stabbed, stabbing, victim, fled, suspicious, blaze, killed, raped, man, missing
384
766
782
640
890
730
610
867
8
119
264
886
970
our, proud, committed, recognize, mission, ourselves, deeply, understands, tradition, honor thin, tricky, slightly, tight, somewhat, bit, awkward, too, notoriously, low
853
700
alabama, virginia, arizona, florida, california, georgia, texas, vermont, washington, wyoming
842
event, held, attended, events, conference, baoth, invited, host, weekend, fair
D Cortical Representations of Time, Number, and Space
D.1 Semantic Category Selection
To identify concept atoms associated with the semantic domains of time, number, and space, we began by defining representative seed word lists for each category. Time-related words included second, hour, day, week, month, year, today, and ago; number-related terms included one, two, three,
19

---

ten, fifteen,,, hundred, and thousand: and spatial expressions included near.far, distance, mile, outside, map, and direction.
For each category, we ranked all 1,000 concept atoms by their average activation across the corre- sponding seed words and manually examined the top five. We retained atoms whose top associated words were semantically aligned with the target category, using inclusive criteria to ensure coverage of relevant conceptual variations. Table summarizes the selected concept atoms along with their highest-activation words.
Table 2:
Concept atoms with strongest activation for semantic categories of time, number, and space.
Factor Index
Category
Top Associated Words
Time
morning, afternoon, evening, weekends, monday, tuesday, lunch, night, hour, day, tonight, today, late, shifts, everyday, yesterday, hours, appointment, am, closed months, hours, days, weeks, ago, years, minutes, hour, month, year, week, day, sec, seconds, spent, intervening, semester, last, weekends
798
Time
times,
880
Time
seventies, sixties, twenties, late, earliest, grunge, subsequent, years, tenure, fame, nineteen, persisted, prior, later, existed, ac- claim
394
Number Number
double, single, two, pair, simultaneous, four, three eleven, sixteen, nineteen, fifteen, seventeen, twenty, ten, eighteen, thirty, sixty, seven, nine, eight, eighty, six, fifty, five, forty, hun- dred, seventy digs, career, nine, tied, seven, sixth, eight, points, fifth, record, freshman, impressive, five, six, mark, finishes, shy, leads, four,
505
602
Number
straight
353
Space
threshold. average, calculated, absolute, scale, higher, slope, hu- midity, altitude, amount, low, normal, mean, distance, lower, level, speed, above, cumulative, readings near, close, distance, airport, center, miles, opposite, minutes, village, mile, town, park, map, away, within, walk, train, drive, outside, halfway
997
Space
These concept atoms capture distinet subtypes of meaning within each semantic domain. For example, factor 81 emphasizes time points typically used in "ar" contexts (e.g., at night, at a.m.); factor 798 encodes temporal durations often found in "for" constructions (e.g.. for days, for weeks); and factor 880 reflects temporal eras or historical references (e.g., seventies, grunge, persisted).
For the main analysis in Section 3.5 we selected one representative concept atom from each semantic category- factor 81 (time), factor 505 (number), and factor 997 (space). While our primary results focus on these exemplars, preliminary inspection of other atoms within the same semantic domains suggests they evoke qualitatively similar cortical activation patterns. To assess the similarity of these representations more systematically, we computed cosine similarities between cortical weight maps for all pairwise combinations within three semantic groups: Time (81, 798, 880), Number (394, 505), and Space (353, 997). Number (394-505) and Space (353-997) pairs exhibited consistently high similarity (0.502 + 0.063 and 0.495 + 0.088, respectively), whereas Time pairs showed moderate consistency (e.g., 81-798: 0.431 + 0.138; 798-880: 0.457 + 0.098). These within-group similarities were substantially higher than the background distribution of across-group factor pairs, which averaged just 0.048 + 0.013. These findings provide quantitative support for the functional clustering of concept atoms within semantic domains and highlight the potential for future research to explore the structure and organization of these representational spaces more deeply.
E Heuristic Concept-Level Cortical Maps from the Dense Encoding Model
This appendix presents heuristic concept-specific cortical maps generated from the Dense Encoding Model, used for qualitative comparison against the Sparse Concept Encoding Model, as discussed in Sections 3.5 and 3.6 Because the Dense Encoding Model produces voxel weight vectors in
20

---

a)
stands standing
sidewalkron rows
as
leeepwaits
d)
family
brothersstemotte
ATOM 456
aislewaiting
couch
eenino
ATOM 115
friends agrandd ther dad
m mom
brothers
@stand front S tood
parents friend
father COUS
brothee
&holding sitting leans
Seak
crouchonstape lay
sit ledge sat walked
naked
lying
mother
'relatives
siblings
parent
grandparent
son
wife
dear
husband
sister
girltriend
yourger
leved
b)
ILTIS
Superoo
Subject 01
Subject 01
Antertor
c)
LTCE
Subject 02
Subject 02
Figure 8: Heuristic concept-level cortical maps from the Dense Encoding Model for concept atoms 456 (a-c) and 115 (d-f), shown for Subject 01 and Subject 02. Panels (a, d) display word clouds for the top words associated with each atom. Panels (b-c, e-f) show voxelwise cortical maps, thresholded using the heuristic method deseribed in Appendix Compared maps in Section (F44 5). these maps are less spatially coherent and more variable across subjects, highlighting the interpretability gains achieved by modeling directly in sparse concept space.
the corresponding
continuous, entangled semantic space, it does assign explicit concept axes. To approximate the cortical representation of a specific concept atom, we applied the following heuristic procedure:
Select target concept atom. We began with target concept atom and retrieved its top 30 activating words using the learned sparse factor embeddings.
Project dense embeddings onto voxel weights. We loaded voxel weights from the Dense Encoding Model. The mean dense embedding the selected concept atom's 's top words was computed and projected onto each voxel's weight vector, yielding an estimate of that voxel's tuning strength to the concept.
Select significant voxels. We retained voxels that satisfied both of the following: (a) positive mean rross-validated prediction scores, and (b) positive projection for the selected concept. This step produced a subset of candidate voxels potentially selective for the concept.
Label voxels by semantic similarity. For each candidate voxel, computed cosine similarity between its weight vector and all 1,000 sparse concept atoms. We then identified the top 10 most similar atoms for each voxel.
Mask concept-relevant voxels. Voxels were marked as selective for the concept the original target atom appeared among their top-10 matches.
This procedure was applied independently each concept examined Sections 3.5 and 3.6 While
these heuristic maps provide a rough approximation of concept tuning in the dense model, they
21

---

a)
d)
three
fifteen
thousand
eleven four
MIP
five
eighteen,
seven fifty
PFC
PF
nine sixty several
dozen
few
sixteen
twentyeightythiry
six
hundred
nineteen
ninety
two weeeeeighteeenteeee
seventy
LTC
LTC
b)
nake everyday morning
arrived
day
weekends late
Subject 01
shifts a non
hours
lunch
tonight
hour
today
y
appot
appointeent
monday Maniannghh
until
night
Anteroo
Time
Space
Anteror
weeting
yesterday
evening
anytine
waking
closed
tuesday
TOL
PFC
INFC
PF
villageaway
train
hal
complex
Teay
opposite
minute.
places
close
shopping
restaurants
park
center
stop
airport
LTC
LTC
distance
around
anous
map
millewakkmi
base
es
town
wi nhin dr
drivooutiide
Subject 02
Figure 9: Heuristic composite cortical maps from the Dense Encoding Model for three concept atoms representing number (red), time (green), and space (blue). Word clouds (a-c) show the top stimulus words most strongly associated with each concept. Panels (d-e) show voxelwise cortical maps for Subject 01 and Subject 02. Colors indicate voxels selectively tuned to each concept (red, green, blue) or their overlaps (white indicating joint selectivity). This figure parallels the analysis Figure of the main paper (Section 3.5), but reveals substantially less spatial overlap between concepts, highlighting the limitations of the heuristic method.
less spatially coherent, less symmetric across hemispheres, and more variable across individuals than maps produced by the Sparse Concept Encoding Model. These differences, visualized in Figures and underscore the interpretability gains afforded by modeling directly in sparse concept space.
Statistical Significance Maps from Permutation Testing
This appendix presents binary voxelwise significance maps obtained using non-parane con-parametric permuta- tion testing assess prediction accuracy the test set. To construct voxelwise null distribution, performed -on-paraettric permutation testing. Specifically, the joint predictions of the banded ridge regression model, defined the sum of predictions from the lexical-semantic and low-level acoustic feature spaces, were randomly permuted over time within each voxel. To account for temporal autocorrelation in the BOLD signal, we used a block permutation strategy following [3]. permuting non-overlapping blocks of 10 consecutive time points (equivalent to 20 seconds of data). For each permutation, prediction accuracy was recomputed, and this process was repeated 9,999 times to generate a null distribution of accuracy values for each voxel. An empirical p-value was then com- puted per voxel by counting the number of permutations in which the permuted prediction accuracy exceeded the observed accuracy from the original (unpermuted) test data. To avoid zero- zero-valued valued p-values, we added one both the numerator and the denominator, following the method of [42]. To correct for multiple comparisons across voxels, the empirical p-values were adjusted using the Benjanini-Hochberg false discovery rate (FDR) procedure (6]. FDR correction was applied using mul multipletess function from the statsmodels Python package with method=' bb? Finally, a binary mask of statistically significant voxels was generated by thresholding the FDR-corrected valuee map p 0.05.
22

---

a)
b)
Suparior
Superoo
Subject 01
Subject 02
Anterior
Anterior
Figure 10: Binary statistical significance maps of prediction accuracy for the Sparse Concept Encoding Model in Subject 01 and Subject 02, highlighting significant prediction performance in temporal, parictal, and prefronnal regions.
Vector Norm Reparameterization
Rescaled Norms
Identity Line
Rescaled Vector Norm
Original Vector Norm
Figure II: Effect of VNR on embedding norms. Each point represents a word vector before and after applying VNR. The dashed line indicates the identity (no change). VNR is preprocessing method that enhances the interpretability of learned concept atoms by expanding the relative spread of vectors near the origin while compressing those among high-norm vectors.
G Evaluation of Embedding Preprocessing Methods
While most learned concept atoms captured coherent semantic themes, observed that subset activated broadly across many high-frequency words used in disparate contexts. We hypothesize that this ambiguity stems from known property of word embeddings: high-frequency words tend have smaller norms and cluster near the origin in Euclidean space, reducing their separability. To address this. developed custom preprocessing method called Vector Norm Reparameterization (VNR). VNR improves the geometric structure of the embedding space by nonlinearly rescaling each word vector's norm while preserving its direction (Fig. IT). For word vector v, we define the scale factor 2
9 tanh
1
and the reparameterized vector becomes v [
This transformation compresses large norms more than small ones, effectively expanding semantically dense regions and reducing the influence of rare words.
To evaluate the effectiveness of VNR, we compared against three alternative preprocessing strategies applied prior to dictionary learning:
23

---

1. Unmodified GloVe vectors
2. L2 normalization to the unit hypersphere
3. ZCA whitening
Each method was evaluated using the following metrics:
Reconstruction Error. We computed the relative Frobenius reconstruction error [[ Xdensel P where Xeesse Txxd contains the original dense word-embedding features, Z R is the learned sparse, non-negative concept-atom activation matrix, and Φ E Rmxd contains the corresponding concept atoms (rows of @). This metric measures how well the sparse concept representation reconstructs the dense embedding space.
2 |> |
Sparsity. For each word, we calculated the fraction of zero entries in its 1.000-dimensional sparse code, then averaged this value across the stimuli vocabulary. Higher values indicate sparser represen- lations.
WordNet Purity. To assess semantic coherence, we identified the top 50 lighest-activation words for each concept atom and computed the mean pairwise WordNet similarity among them, using both path similarity and Wu-Palmer similarity [33 40). Semantically meaningful atoms are expected to group together words from similar lexical fields, as defined in WordNet's ontology.
Results. As shown in Table VNR achieved the lowest reconstruction error while maintaining high sparsity and the strongest semantic coherence across both WordNet- based metrics. Notably, L2 normalization resulted extremely poor reconstruction performance, likely due to its constraint that all word vectors and dictionary atoms lie the unit hypersphere, limiting the model's ability to encode useful norm variation. Based on these results, we adopted VNR as the default preprocessing method in all subsequent experiments.
Table 3: Comparison of preprocessing strategies for word embeddings prior to sparse coding.
Embedding Version
Reconstruction Error
Sparsity
WN Purity (Path)
WN Purity (WuP)
Raw GloVe
0.4922
0.9723
0.1158
0.3571
VNR 12-normalized
0.4083
0.9677
0.1171
0.3579
7.2403
0.9640
0.1166
0.3563
Ahhttenee
0.6714
0.9610
0,1113
0,3412
24

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The description is accurate and does not overstate or misrepresent the scope or contributions of the work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed at length in last section the main paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
25

---

Answer: [Yes]
Justification: Mathematical discussions on dense feature representations and the advantages of sparse concept features aree provided in AAppeddix
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification; Appendix Ajdeccribes all experimental settings, including feature construction, model training. hyperparameter tuning, and evaluation procedures, Model pipeline is clearly explained the main paper.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
26

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The and preprocessed data publicly available on GIN and OSF Appendix A] An anonymized version of the full codebase is included in the supplement and will be released publicly upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
public/guided SoodeeSubmissioooll1eeo
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips http:cccpccccccgcccccoooccccccee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Training, validation, and test splits are described in and Appendix A The cross-validation scheme, regularization grid, and hyperparameter optimization procedures are all fully specified.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Statistical significance is assessed for prediction accuracy on test data using paraameetriic permutation testing with FDR correction Appendix F Pairwise comparisons are evaluated with t-tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
27

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Appendix A specifies computation resources used for model fitting. Code dependencies and libraries are also descrihed.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp//////eerpppe.cpppppp/////e
Answer: [Yes]
Justification: We have reviewed the code of conduct and anonymized our paper, code and data submission.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The last section in the main paper discusses implications for brain-maching interface applications, concerns about mental privacy, as well as dangers of inherent biases in pretrained embeddings.
28

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., dep.oymmly of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification; As a safeguard, we chose not release anatomical brain images due to privacy concerns, but we provide transformation matrices that map volumetric data to cortical flatmaps to support visualization and interpretation.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
for word embeddings, (59 for sparse dictionary learning on word embeddings, and 22 for the data set. Data and code are redistributed under the
The paper cites 41
CC-BY 4.0 license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Details about the dataset and model are thoroughly described in the main paper and supplementary materials. The code is well-commented, and the repository includes a README with detailed instructions for usage and data handling.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work used publicly available fMRI data from previously published study and did not conduct new human subject research.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
30

---

Justification:
This work used publicly available, anonymized data. No new human subject experiments were conducted as part of this study.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy [ttrge///oorriggcc/////ffeeeeerp for what should or should not be described.
31


Reviews:
[{"review_id": "G4tA68DpZB", "reviewer": "Reviewer_ymzV", "summary": "1. The authors learn a mapping from 300-D GloVe word embeddings to 1000-D sparse vectors, each representing coefficients of an overcomplete basis for the embedding space, via a dictionary learning framework. The basis vectors can be intepreted as semantic concepts by retrieving words assigned a large coefficient for that vector.\n\n2. They then regress fMRI recordings from a listening task using the sparse vectors as input and test on held-out stories.\n\n3. They find that (i) sparse embeddings show similar brain encoding performance to dense embeddings while offering more intepretability, (ii) the regression weight vectors corresponding to select concepts, when projected on to the brain, are visually similar across two subjects, and (iii) there is visual overlap between the projections corresponding to \"time\", \"space\", and \"number\" concepts.", "questions": "None.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The approach can inspire future work on gaining a deeper understanding of brain activity based on known semantic concepts in the stimulus.\n\nWeaknesses:\n1. Lack of novelty: None of the methods in the paper are novel. Sparse coding is a well-known approach to extract interpretable features from neural network layers, including LLMs (as described in Section 2.2). The specific dictionary learning framework used was introduced in previous work (as mentioned in line 163). The resulting regression analysis of fMRI recordings is also standard.\n\n2. Lack of rigor:\\\n**a)** In lines 159-160, the authors say they leave validation on more participants to future work and present only \"exploratory findings\", which falls short of typical NeurIPS empirical standards. While the exploratory results may serve as qualitative examples, it is crucial to present a more robust quantitative analysis, which includes (i) clearly defining evaluation metrics, (ii) utilizing data from all (seven) participants, and (iii) reporting the variance in the results across participants whenever possible. Moreover, this all needs to be included in the main paper, not pushed to the appendix.\\\n**b)** Section 3.4 claims inter-subject consistency of per-concept cortical maps obtained from the learned weight matrix when using sparse embeddings. However, the only evidence for this is a visualization (Figure 5) for two selected concepts and two participants. The same applies to the claim of increased inter-subject variability (line 222) when using dense embeddings. The authors should instead quantify the variability across subjects (aggregated over concepts) when using sparse vs. dense embeddings to provide more tangible and reliable evidence for their claims. They could possibly also examine brain regions where the variability is more pronounced using either method.\\\n**c)** Section 3.5 attempts to show overlap in the neurons responding to \"time\", \"space\", and \"number\" concepts. Again, the authors have not quantified the apparent overlap, instead resorting to a visualization (Figure 6). A straightforward solution is to report some kind of intersection over union (IoU) of the neurons responding to the three concepts.\\\n**d)** Section 3.6 mentions \"similar clustering structure\" (line 253), \"correlations were uniformly low\" (line 257), and \"moderate positive correlations\" (line 260). None of these are quantified in the paper.\\\n**e)** Lines 186-187 state that Vector Norm Reparameterization qualitatively leads to \"sharper\" concepts. However, no evidence is provided for this claim in the main paper. There is also no derivation for the form of the rescaling in the appendix.\n\n3. Section 2.4, presumably meant to describe the method, has no content.\n\n4. There are no references to the literature in (i) lines 47-48 for the claim that each neuron in the brain likely responds to multiple semantic concepts and (ii) lines 182-183 for the claim that the embeddings for high-frequency words tend to cluster near the origin.\n\n5. In lines 61-63, the authors claim to present the first mathematical framework showing that several embedding inputs may be mapped to the same predicted brain response. However, this is an expected result in multi-output linear regression since an exact one-to-one map would require the rank of the learned regression weight matrix to match the number of columns, which is not guaranteed in practice.\n\n6. The paper uses GloVe vectors, which provide non-contextual representations of words, while participants hear words in context. This represents a lack of consistency between the inputs and outputs of the regression. While acknowledged as a limitation in lines 272-276, I do not see a reason for not using LLM-based embeddings in the analysis given their ease of access (e.g., through Hugging Face) and the sheer number of open-source LLMs available in 2025.\n\n7. Section 3.1 does not seem to add much value to the paper as it is already known that dictionary learning can produce basis vectors capturing interpretable concepts, as explained in Section 2.2. There is no need to show \"consistency with prior findings\" (line 168) since this section is simply replicating prior findings. A simple visualization would suffice to emphasize this point.\n\n8. The intention of Section 3.6 is not clear. It starts (lines 241-244) by wanting to investigate the similarity between brain representations of different abstract concepts. However, there is no reference to these \"abstract concepts\" anywhere else in the section. It is not clear how \"the top 20 concept atoms\" (line 247) analyzed are linked with the abstract concepts.\n\n9. The authors state that a limitation of dense embedding-based brain encoding is that each component contains a combination of multiple semantic features, rendering the results less interpretable. However, lines 47-48 and 142-143 imply that voxels behave in a similar manner, i.e., a single voxel may respond to multiple concepts. The paper lacks any discussion of this – and its effect on interpretable brain encoding – despite the emphasis on interpretability throughout the paper.\n\n10. Minor:\\\n**a)** \"where\" in line 89 is in math mode.\\\n**b)** \"Fig 2a\" => \"Fig 3a\" (line 131).\\\n**c)** \"the the\" => \"the\" (line 152).\\\n**d)** \"Figure 3\" => \"Figure 2\" (line 169).\\\n**e)** \"Appendix D\" => \"Appendix F\" (line 223).\\\n**f)** \"Section 3.5\" => \"Section 3.4\" (line 248).\\\n**g)** There is no reference to Figure 5 in Section 3.4.\\\n**h)** There is no reference to Figure 6 in Section 3.5.\\\n**i)** \"Subiect\" => \"Subject\" (Figure 5b).\\\n**j)** \"ATOM 81\" => \"ATOM 997\" (Figure 6c).\\\n**k)** \"Space\" and \"Time\" colors are flipped in the Venn diagram in Figure 6.", "was_revised": true, "final_justification": "The authors have improved their empirical evaluation as suggested, so I will raise my score to 3 (Borderline reject). \n\nI am not willing to go higher after the rebuttal since they: \n1. Overstate the contribution of a ''first mathematical framework'' while the paper contains no mathematical or theoretical novelty.\n2. Claim that dense embeddings *only enable* heuristic analysis while not substantiating this adequately (e.g., by trying a bunch of dense-embedding based methods and showing that they all fall short). \n3. Only report experiments with GloVe even when highlighted as a major limitation for interpretability. The authors point out that sparsifying an LLM is non-trivial, but I never suggested that the LLM itself be sparsified, just that the chosen dictionary learning approach be applied on top of LLM-based embeddings instead of non-contextual embeddings. I believe this is simple to implement by drawing on prior work and it seems the authors have severely misrepresented my concern.", "rebuttal": "We thank reviewer ymzV for the detailed evaluation, thoughtful comments, and meticulous feedback on technical and editorial issues. We have carefully considered all critiques and are confident we can address them fully and that revision will substantially strengthen the work. \n\nWhile we agree with most of the reviewer’s concerns, we respectfully disagree with two key criticisms: (1) inadequate novelty, and (2) insufficient number of subjects and cross-participant analysis. For brevity, we will respond in detail to (1) in our rebuttal to buH2 and (2) here:\n\n> …the authors say they leave validation on more participants to future work and present only \"exploratory findings\", which falls short of typical NeurIPS empirical standards... \n\nWe appreciate the reviewer’s emphasis on robust quantitative analysis. However, this concern may partially reflect a misunderstanding of the modeling framework adopted.\n\nOur analysis adopts the Voxelwise Encoding Model (VEM) framework [7], which emphasizes high-resolution, within-subject modeling using large volumes of data per participant. VEM differs from traditional group-level fMRI designs, which typically entail collecting short recordings (around 20 minutes) from many participants (≥ 20) and evaluating hypotheses by averaging voxel responses across individuals. These traditional approaches lack held-out test evaluation and rely heavily on point-null hypothesis testing, which has been shown to suffer from reproducibility issues. The VEM framework by contrast, entails extensive data collection (around 3 hours) from each participant to enable voxelwise regression models to be fit independently in each subject’s native cortical space. Model performance is evaluated on held-out test data. Thus, each participant serves as a full replication of the hypothesis test. Group-level prevalence can thus be assessed by projecting subject-specific model parameters to a common template surface.\n\nGiven the limited conference format, we prioritized within-subject analyses. These analyses adhere to best practices in data science: model weights are estimated using cross-validation on a training dataset; predictive performance is evaluated on held-out test data; confidence intervals of model weights are computed via bootstrapping; and statistical significance of prediction is assessed through nonparametric permutation testing. Detailed methods are in Appendix A. rather than the main text because the goal of this paper is to introduce a broadly applicable modeling framework rather than to present new neuroscientific discoveries. That said, we agree group-level results can provide valuable complementary evidence. In the revision, we are happy to include analyses from all seven participants and report cross-subject consistency and variability in model performance and cortical maps if the reviewers feel this is essential.\n\n> … authors claim to present the first mathematical framework showing several embedding inputs may be mapped to the same predicted brain response... this is an expected result in multi-output linear regression since exact one-to-one map would require the rank of the learned regression weight matrix to match the number of columns, …\n\nWe believe this comment may reflect a misunderstanding of our central claim. The reviewer references “multi-output linear regression” and “rank of the regression weight matrix”. However, our argument pertains to single-output linear regression, which is standard in voxelwise encoding models. We also do not argue that underdetermined regression problems have multiple solutions. This is a well-known fact often addressed by regularization techniques such as ridge regression. Rather, we highlight a more subtle issue: even when the regression problem is well-posed and the solution is unique, the interpretation of that solution becomes ambiguous when dense embeddings are used. Because semantic directions in dense spaces are often correlated due to superposition, a single weight vector can correspond to multiple plausible combinations of underlying semantic features. In other words, the solution is mathematically unique but semantically non-identifiable.\n\nAlthough the mathematical proof is straightforward, this interpretability problem has been largely overlooked. In prior studies, weight vectors from dense encoding models were assumed to reflect meaningful voxel tuning, and standard practices such as nearest-neighbor word retrieval were used to interpret them. To our knowledge, we are the first to identify and formalize this problem. In the revision, we will clarify the scope of our argument and emphasize that our contribution is not to present a novel theoretical result, but to identify and formalize a conceptual flaw in a widely used modeling practice.\n\n> The authors state a limitation of dense embedding-based brain encoding is that each component contains a combination of multiple semantic features…\n\nWe appreciate this comment and agree that clarifying how voxel multi-selectivity affects interpretability is central to our work. In fact, our proposed sparse framework is specifically designed to address this issue. The limitation of dense embeddings is not that voxels respond to multiple concepts but that dense models fail to disentangle the contributions of each concept due to the superposition of semantic directions. To make this connection clearer, we have revised Section 2.3 (Limitations of Dense Encoding Models) and restored the missing content of Section 2.4, which introduces the Sparse Concept Encoding Model. Both revised sections are included in our response to Reviewer buH2. We deeply apologize for the confusion caused by the omission.\n\n> The paper uses GloVe vectors, which provide non-contextual representations of words…\n\nWe chose GloVe for two main reasons. First, our goal was to isolate and examine the interpretability gap between dense and sparse encoding models in direct comparison with prior work. Word embeddings such as GloVe have been widely used in previous voxelwise encoding model studies, making them a natural baseline. Second, contextual embeddings introduce additional modeling decisions (e.g., contextual window length) that would distract from the core contribution of our paper: identifying and resolving the interpretability problem in dense-to-sparse encoding. That said, we fully agree that applying our method to LLM-based embeddings is a natural next step. In fact, we are currently working on extending this work using gemma-2. If the reviewers believe including discussions of contextual embeddings is crucial to the acceptance of this work, we are happy to include these in the revision. \n\n> …authors have not quantified the apparent overlap…solution is to report some kind of intersection over union (IoU) of neurons…\n\n> Section 3.6 mentions \"similar clustering structure\"...None of these are quantified...\n\nWe truly appreciate these suggestions and agree these sections should include quantitative descriptions in addition to visualization. In revision we will include textual descriptions of these statistics. \n\n> Section 3.4 claims inter-subject consistency of per-concept cortical maps obtained from the learned weight matrix when using sparse embeddings…\n\nWe appreciate the reviewer’s suggestion. As our study adopts the VEM framework, our main focus has been on within-subject modeling. While we do present qualitative evidence of cross-subject consistency in Figure 5, we acknowledge that the current manuscript lacks a quantitative group-level analysis. That said, inter-subject consistency is not a central claim of our work. Moreover, for the dense encoding model, the comparison maps are derived from heuristic procedures and vary substantially depending on the method used, as is discussed in rebuttal to buH2. Nonetheless, we agree that adding group-level metrics could strengthen the manuscript. If the reviewers believe this is necessary for acceptance, we are happy to fit models on all seven participants and report inter-subject similarity metrics in the revision.\n\n> Section 3.1 does not seem to add much value to the paper as it is already known that dictionary learning can produce basis vectors capturing interpretable concepts…\n\nWe appreciate this suggestion and agree Section 3.1 should be condensed. One of the original goals of the section was to show that many semantic directions in dense embeddings are moderately correlated, which motivates the need for a disentangled feature space. In revision we will streamline this section. \n\n> The intention of Section 3.6 is not clear…\n\nWe thank the reviewer for highlighting this lack of clarity. We agree the document did not sufficiently explain the link between these categories and the selected concept atoms. To clarify: each learned concept atom corresponds to a distinct semantic dimension, identified by inspecting the top stimulus words associated with that atom (see Appendix Table 1). These atoms can be interpreted as representing abstract concepts, such as “oppositional conjunctions” or “body positions.” In Section 3.6, we focused on the 20 atoms with the highest average cortical activation, as they provide the most robust signal for cross-atom comparisons. In revision we will clarify the connection between concept atoms and abstract categories. Specifically, we will clearly state that the selected atoms correspond to interpretable abstract concepts; reference Appendix Table 1, which lists the top 50 atoms; and move Figure 7 (the labeled dendrogram and clustered similarity matrix) into the main text. \n> Lines…state Vector Norm Reparameterization qualitatively leads to \"sharper\" concepts. However, no evidence is provided… \n\nIn the revision, we can include a table of subjective assessments demonstrating that Vector Norm Reparameterization indeed yields more semantically distinct concepts, as well as a figure which intuitively demonstrates the way this rescaling works."}, {"review_id": "bwCwilyPTS", "reviewer": "Reviewer_ZoiN", "summary": "The paper presents e Sparse Concept Encoding Model, a novel method to perform encoding analysis in fMRI data. The contribution lies in using dictionary learning to decompose dense word embeddings (GloVE) into a sparse set of quasi-orthoghonal concept atoms.\n\nThanks to such decomposition, the weight vectors resulting of the encoding linear model become directly interpretable. In such way, SCEM provides with key insights about where in the brain different concepts are encoded across patients.\n\nThe results show that space, time and number representations are encoded in the same brain regions,hinting towards a unified magnitude system. The paper also shows a consistent anatomical mapping of different concepts (other than space, time and number) across patients.", "questions": "What is the interpretation of blue vs red in Figure 5? It seems like the brain regions are consistent across patients  but not the colors.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- The paper is well written, the motivation, aim, methods and results are well conveyed.\n- The method provides with a principled way to better interpret encoding models\n- The results look consistent and coherent with previous work\n\n## Weaknesses\n\n- It is not very clear to me if such \"unified magnitude system\" can be concluded from figure 6. While there is quite a bit of overlap (white), the maps look a bit disordered", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer ZoiN for their thoughtful review and for highlighting several key strengths of our work. We are especially grateful for their recognition of the principled nature of our method and the clarity of our presentation.\n\n> It is not very clear to me if such \"unified magnitude system\" can be concluded from figure 6. While there is quite a bit of overlap (white), the maps look a bit disordered\n\nWe appreciate this observation. We agree that the maps in Figure 6 appear noisy. However, this level of variability is expected in within-subject functional maps generated from VEM studies using naturalistic stimuli [1,7, 9,10]. As a non-invasive imaging technique, fMRI inherently suffers from low signal-to-noise ratios. Group-level analyses, which project participant-specific weights to a common template and averaging across individuals, typically yield smoother and more structured maps. However, this comes at the cost of reduced sensitivity to individual variability and can obscure meaningful discoveries [7]. As noted in the rebuttal to ymzV, we are happy to include group-level maps in the revision if the reviewers believe this would strengthen the evaluation of the results.\n\n> What is the interpretation of blue vs red in Figure 5? It seems like the brain regions are consistent across patients but not the colors.\n\nWe thank the reviewer for this question. In Figure 5, red indicates positive tuning and blue indicates negative tuning to a given concept atom. That is, red voxels show increased predicted responses when the concept is present in the stimulus, while blue voxels show decreased predicted responses (relative to the mean, since responses have been mean-centered for each voxel). In our revision, we will revised Sections 2.3 and 2.4 to make this interpretation more clear. Please see the revisions in our rebuttal to Reviewer buH2.\n\nRegarding the apparent color inconsistency across participants, much of this inconsistency is expected from individual differences in cortical anatomy and functional organization. The 2D flatmaps are created by flatting each subject’s cortical surface through a manual cutting procedure [1,7], which introduces further variations in spatial alignment. If one examines the inflated cortical surface renderings below the flatmaps in Figure 5, the spatial correspondence of activated regions across participants might be more evident. \n\nReferences\n\n[1] Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453-458.\n\n[7] Dupré la Tour, T., Visconti di Oleggio Castello, M., & Gallant, J. L. (2025). The Voxelwise Encoding Model framework: a tutorial introduction to fitting encoding models to fMRI data. Imaging Neuroscience, 3, imag_a_00575.\n\n[9] Deniz, F., Nunez-Elizalde, A. O., Huth, A. G., & Gallant, J. L. (2019). The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality. Journal of Neuroscience, 39(39), 7722-7736.\n\n[10] Meschke, E. X., Castello, M. V. D. O., Tour, T. D. L., & Gallant, J. L. (2023). Model connectivity: leveraging the power of encoding models to overcome the limitations of functional connectivity. BioRxiv, 2023-07."}, {"review_id": "dXSbnwiiUn", "reviewer": "Reviewer_iQoW", "summary": "The goal of the study is to develop more interpretable fMRI encoding models. The authors provided mathematical and empirical demonstrations that dense embeddings lead to ambiguities in model weight interpretation. The authors then developed a theoretically motivated approach that decomposes the dense embeddings into sparse combinations of concept atoms. The authors estimated encoding models using the sparse coefficient vectors as stimulus features and demonstrated that sparse models achieve similar predictive performance to dense models. Finally the authors analyzed the sparse model weights to visualize how concepts are encoded across the cortex.", "questions": "In 3.5 could it be the case that the sparse model predicts more overlap for all concepts (rather than just time, space, and number concepts)? As a control could you compare the sparse and dense models on a different set of concepts that are actually hypothesized to be separate in the brain?\n\nThe interpretations will depend on the actual atoms that are chosen for each concept. In 3.5 the authors chose one representative atom for each concept (time, space, and number) and said that the others atoms provided similar results (line 534). Is there a way to quantify this similarity?\n\nThe overall prediction performance is similar but are there any brain areas where prediction performance differs between the dense and sparse models?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe sparse concept encoding model is clearly motivated and the authors provided a convincing argument that sparse models are more interpretable than dense models. The sparse models were able to attain comparable performance to dense models. The vector norm reparameterization technique for differentiating between high frequency words could be useful for future studies. The case studies demonstrate how sparse models can be used to address long standing questions in cognitive neuroscience.\n\nWeaknesses\n\nThe main benefit of the sparse model is interpretability but this can be difficult to evaluate. For instance the authors show that the dense model predicts more overlap across time, space, and number concepts than the sparse model, and they claim that this finding is more consistent with known behavioral and psychophysical results. However it would also be helpful to evaluate which model is more consistent with known neuroimaging results. It appears that the Huth et al. data contains separate ROIs, is there a way to compare the dense and sparse models to the ROIs?", "was_revised": true, "final_justification": "The authors have adequately answered my questions about the study. They have also provided thoughtful responses to the other reviewers that clarify the motivation of the study. I think that this is an interesting study and will keep my score at 5.", "rebuttal": "We sincerely thank Reviewer iQoW for their thoughtful and constructive review. We especially appreciate their insightful questions based on a clear theoretical understanding of the work and methods therein. We hope that our responses to these questions provide additional helpful context for the panel’s overall evaluation. \n\n> The main benefit of the sparse model is interpretability but this can be difficult to evaluate. For instance the authors show that the dense model predicts more overlap across time, space, and number concepts than the sparse model, and they claim that this finding is more consistent with known behavioral and psychophysical results. However it would also be helpful to evaluate which model is more consistent with known neuroimaging results. It appears that the Huth et al. data contains separate ROIs, is there a way to compare the dense and sparse models to the ROIs?\n\nWe thank the reviewer for this thoughtful suggestion. The Huth et al. dataset includes standard regions of interest (ROIs) localized using widely adopted functional localizers. These include visual category ROIs, auditory cortex ROIs and motor ROIs. These ROIs are well-established for sensory and motor functions. However, there are no standardized ROIs for abstract concepts such as time, space, or family. While some prior studies have implicated the intraparietal sulcus (IPS) in number and spatial processing, and medial prefrontal and parietal cortices in social or autobiographical concepts, these findings are not consistently replicable. This reflects a broader challenge in cognitive neuroscience. Traditional task-based fMRI studies often collect ~10 minutes of data per subject using isolated stimuli and analyze group-level averages. This approach has faced reproducibility issues, as results often fail to generalize across datasets or paradigms. By contrast, the Huth et al. dataset used in our study adopts a naturalistic paradigm, collecting multiple hours of rich, continuous story-listening data per participant. Their data reveal more distributed and overlapping cortical representations of conceptual content, but also makes ROI-based validation more difficult. Our work is designed to address this challenge. Our proposed Sparse Concept Encoding Model enables interpretable voxel-level tuning even for abstract concepts that lack well-defined ROIs. \n\nIn the revision, we will clarify that most abstract concepts do not have standardized ROIs. We will also reference prior neuroimaging findings on the cortical representation of number, time, and space, and include qualitative comparisons with the corresponding maps produced by our models.\n\n> In 3.5 could it be the case that the sparse model predicts more overlap for all concepts (rather than just time, space, and number concepts)? As a control could you compare the sparse and dense models on a different set of concepts that are actually hypothesized to be separate in the brain?\n\nThis is an excellent question! As we explain in the paper, dense embedding models lack a principled method for generating concept-level cortical maps. To enable rough comparison, Appendix F reports heuristic maps for the dense encoding model. These are not ground-truth concept maps, but approximations constructed using heuristic methods. Prior work has used several such heuristics, including:\n\n* Cosine similarity: assigning each voxel the concept whose direction is nearest to its weight vector [2,9,10]. This often underestimates overlap, as mixed selectivity is ignored.\n\n* Projection: projecting voxel weights onto semantic directions of interest [1,11]. This often overestimates overlap due to spurious correlations caused by superposition.\n\nIn our paper, we adopt a middle-ground heuristic, labeling a voxel as selective for a concept if that concept appears in its top-10 cosine similarity matches. If we use top-1, overlap becomes minimal; with top-20, it increases. This sensitivity illustrates why concept overlap under dense models is heuristic-dependent and not robust. In revision, we will make this clearer in the main text and can include additional comparisons using different thresholds in the appendix if reviewers would find this helpful.\n\n> The overall prediction performance is similar but are there any brain areas where prediction performance differs between the dense and sparse models?\n\nThank you for this helpful question! It can best be answered in the form of a flatmap showing the difference in prediction performance between the two models across the cortical surface, similar to Figure 9 in Deniz et al [9]. While NeurIPS rebuttal guidelines do not permit inclusion of figures, we have conducted this analysis and found no consistent spatial pattern indicating one model outperforms the other across specific cortical regions. In the revision, we will include this differential performance map in the appendix to support this observation.\n\n> The interpretations will depend on the actual atoms that are chosen for each concept. In 3.5 the authors chose one representative atom for each concept (time, space, and number) and said that the others atoms provided similar results (line 534). Is there a way to quantify this similarity?\n\nWe can quantify the similarities by calculating the cosine similarities of cortical maps for atoms within each concept group (e.g. “time - duration” and “time - moments”), and compare these values with the distribution of cosine similarities for all possible pairs of atoms. In revision we will report the statistics.\n\nReferences\n\n[1] Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453-458.\n\n[2] Tang, J., Du, M., Vo, V., Lal, V., & Huth, A. (2023). Brain encoding models based on multimodal transformers can transfer across language and vision. Advances in Neural Information Processing Systems, 36, 29654-29666.\n\n[9] Deniz, F., Nunez-Elizalde, A. O., Huth, A. G., & Gallant, J. L. (2019). The representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality. Journal of Neuroscience, 39(39), 7722-7736.\n\n[10] Meschke, E. X., Castello, M. V. D. O., Tour, T. D. L., & Gallant, J. L. (2023). Model connectivity: leveraging the power of encoding models to overcome the limitations of functional connectivity. BioRxiv, 2023-07.\n\n[11] Wehbe, L., Huth, A. G., Deniz, F., Gao, J., Kieseler, M. L., & Gallant, J. L. (2016). BOLD predictions: Automated simulation of fMRI experiments. NeurIPS Demonstr. Track."}, {"review_id": "mTg8JQyfIl", "reviewer": "Reviewer_buH2", "summary": "The paper proposes Sparse Concept Encoding Model in order to the interpretability of the encoding models for predicting brain activity. While dense feature models are accurate, they lack interpretability because feature representations of different concepts can overlap due to superposition. Their model transforms dense embeddings into a high-dimensional, sparse space composed of learned, interpretable concept atoms. The model matches the performance of conventional dense models on fMRI data, while enabling interpretation of features that dense models don't have access to.", "questions": "* In Section 3.2, \"we observed a subset that activated broadly across many high-frequency words used in divergent contexts.\" Can you elaborate more on this, either by visualizations or providing logic on why this could happen?\n* line 89, \"where\" should not be in latex mode.\n* line 147, which appendix?\n* Section 2.4 has a title but with missing content", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* The paper is clearly written, with beautiful visualizations\n* The method is elegant and simple and seems to work\n\nWeaknesses:\n* The novelty of the model needs more justification, since the model seems like a conventional dictionary learning method. If the novelity is not the main selling point, the implications for neuroscience should be elaborated on more.\n* \"Limitations of Dense encoding models\" is a bit verbose.", "was_revised": true, "final_justification": "I would like to thank the authors for their clarification. I would like to raise my score to 4.", "rebuttal": "The main criticism from buH2 is inadequate novelty, a concern also raised by ymzV. We believe these concerns may stem from misunderstandings about the core goals and contributions of the paper. Below, we restate the intended contribution and its relevance to the NeurIPS neuroscience community.\n\nOur central contribution is to identify a previously overlooked limitation of brain encoding models that use artificial neural network (ANN) features, demonstrate its empirical consequences, and propose a practical solution that enables principled neuroscientific interpretation. Specifically, we formalize the impact of superposition in ANN-based feature spaces: when the number of latent semantic features exceeds the embedding dimensionality, those features become entangled in correlated directions. Thus, the regression weight vector learned for a voxel is non-identifiable: multiple distinct combinations of semantic features can yield the same predicted brain response. This ambiguity makes it impossible to reliably determine which specific concepts a voxel is tuned to, and ultimately limits the utility of dense models for answering neuroscientific questions. Our proposed framework preserves the predictive utility of ANN-derived features while restoring the interpretability necessary for scientific discovery. The framework can be generalized to a wide range of representations (e.g., LLM embeddings) and recording modalities (e.g., ECoG, MEG, or intracranial recordings).\n\nThis limitation is broadly relevant to a growing body of work, including several recent papers published at NeurIPS [1,2,3,11], that uses ANN-based features in linear encoding models to predict brain activity. These works often rely on the implicit assumption that learned weight vectors in dense embedding spaces are directly interpretable. Our paper demonstrates that this assumption is flawed. By formalizing this limitation and providing a principled alternative, our work directly engages with this line of research and contributes to an ongoing discussion in the NeurIPS neuroscience community [4,5,6] about how ANN-based brain models should be interpreted and evaluated. \n\n\n> The novelty of the model needs more justification, since the model seems like a conventional dictionary learning method. If the novelity is not the main selling point, the implications for neuroscience should be elaborated on more.\n\nWe acknowledge that the individual techniques employed in our approach, dictionary learning and voxelwise regression, are not themselves novel. However, this is, to our knowledge, the first study to formally define the semantic non-identifiability problem in ANN-based brain models, show how it undermines standard interpretability practices, and offer a widely-applicable solution. \n\nWe recognize this framing was insufficiently emphasized in the original submission. In the revision, we will: (1) move summaries of the non-identifiability argument and Bayesian framing from Appendix C to the main text; (2) revise Section 2.3 (Limitations of Dense Encoding Models) to clarify the motivation (the full revised section is included in our response to Reviewer buH2); (3) restore the omitted content of Section 2.4, which critically introduces the Sparse Concept Encoding Model; and (4) update the introduction to explicitly state that our goal is to correct a conceptual flaw in a widely used modeling framework rather than to propose a novel algorithm or report new neuroscientific findings. Together, these changes should serve to more clearly highlight the conceptual insight and neuroscientific significance of this work.\n\n\n> \"Limitations of Dense encoding models\" is a bit verbose.\n\nWe appreciate this feedback. We have revised the second paragraph of Section 2.3 to clarify our core argument. \n\nRevised Section 2.3: Limitations of Dense Encoding Models (second paragraph)\n\nEncoding models based on dense ANN embeddings face a fundamental limitation: semantic directions are entangled due to superposition, making it difficult to reliably determine which concepts a voxel is truly selective for. To illustrate, consider two contrasting cases of feature spaces used in voxelwise encoding models. In the ideal case, the feature space is constructed with an interpretable basis, where each axis corresponds to a distinct semantic concept (Figure 1, right). A voxel’s regression weight vector can then be interpreted directly: each coefficient reflects sensitivity to a specific concept. Even when predictors are correlated, standard techniques such as variance partitioning can disentangle their contributions. This is the interpretability goal enabled by the Sparse Concept Encoding Model.\n\nIn contrast, dense embeddings such as GloVe (Figure 1, left) do not provide an interpretable basis. Semantic concepts are encoded by correlated linear combinations of basis directions rather than coordinate axes. When a voxel is jointly tuned to multiple concepts (e.g., “time” and “color”), its weight vector lies between their directions. Cosine-based interpretability methods may misidentify this vector as responding to an unrelated concept (e.g., “space”), leading to incorrect conclusions. These methods implicitly assume that each voxel is tuned to a single semantic direction.However, this assumption is inconsistent with decades of neuroscience findings showing that voxels often respond to multiple distinct features. Dense models obscure this multi-concept tuning, limiting their interpretability.\n\n\n> Section 2.4 has a title but with missing content\n\nThank you for catching this blunder. To our deepest regret, we mistakenly submitted a non-final version of the manuscript, which included several severe organizational blunders and formatting oversights, including the omission of Section 2.4, inconsistent figure references, and missing citations (ymzV, buH2). We sincerely apologize for any confusion this may have caused and appreciate the reviewers’ patience and attention to detail. We are committed to carefully addressing all these issues, and if the paper is accepted, we will make every effort to ensure that the camera-ready version is error-free. We feel certain these revisions will significantly improve the clarity and impact of the paper. The corrected version of Section 2.4 is included below.\n\nSection 2.4: Sparse Concept Encoding Model\n\nTo resolve the identifiability limitations of dense embeddings, we construct a sparse, overcomplete semantic space. Specifically, we expand the original d-dimensional embedding space into an m-dimensional space ( m>d) using sparse dictionary learning. Each axis in this expanded space corresponds to a distinct semantic direction, which we term a concept atom. These atoms are learned from a large text corpus using non-negative sparse coding (Fig. 2b). After dictionary learning, each stimulus word is re-encoded as a sparse, non-negative linear combination of concept atoms. This sparse feature matrix replaces the dense feature matrix in the voxelwise encoding model pipeline (Fig. 2c). Because the dictionary is overcomplete and approximately orthogonal, the resulting Gram matrix G=I. This structure ensures each voxel’s weight vector provides a direct ,interpretable readout of its tuning to individual concept atoms. The non-negativity constraint in dictionary learning ensures interpretability: positive weights reflect tuning to specific concepts, while negative weights imply suppression. Crucially, since each concept atom occupies its own axis in the new space, voxels jointly selective for multiple semantic features can be cleanly disentangled. This model thus enables unambiguous, voxel-level interpretation of conceptual tuning for each voxel.\n\n> In Section 3.2, \"we observed a subset that activated broadly across many high-frequency words used in divergent contexts.\" Can you elaborate more on this…\n\nIn dense embeddings like GloVe or FastText, high-frequency words tend to lie in similar directions—not due to shared semantics, but because of frequency-related statistical patterns [12]. High-frequency words also tend to lie closer to the origin, likely because their embedding vectors represent the average over more diverse contexts [13]. Sparse coding picks up on these shared directions, yielding atoms that broadly activate across semantically diverse but frequent words. \n\nReferences\n\n[1] Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453-458.\n\n[2] Tang, J., Du, M., Vo, V., Lal, V., & Huth, A. (2023). Brain encoding models based on multimodal transformers can transfer across language and vision. Advances in Neural Information Processing Systems, 36, 29654-29666.\n\n[3] Wang, A., Tarr, M., & Wehbe, L. (2019). Neural taskonomy: Inferring the similarity of task-derived representations from brain activity. Advances in neural information processing systems, 32.\n\n[4] Schaeffer, R., Khona, M., Chandra, S., Ostrow, M., Miranda, B., & Koyejo, S. (2024, October). Position: maximizing neural regression scores may not identify good models of the brain. In UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models.\n\n[5] Canatar, A., Feather, J., Wakhloo, A., & Chung, S. (2023). A spectral theory of neural prediction and alignment. Advances in Neural Information Processing Systems, 36, 47052-47080.\n\n[6] Ostrow, M., Eisen, A., Kozachkov, L., & Fiete, I. (2023). Beyond geometry: Comparing the temporal structure of computation in neural circuits with dynamical similarity analysis. Advances in Neural Information Processing Systems, 36, 33824-33837.\n\n[11] Wehbe, L., Huth, A. G., Deniz, F., Gao, J., Kieseler, M. L., & Gallant, J. L. (2016). BOLD predictions: Automated simulation of fMRI experiments. NeurIPS Demonstr. Track."}]

Author rebuttal:

Meta-review: This paper proposes to use sparse positive decompositions of dense learned embeddings to achieve more interpretable mappings from learned representations to neural data. The method is sensible and clearly presented, and the qualitative results are suggestive.

 While most the reviewers were satisfied with the authors' responses, I find myself still somewhat sympathetic to several of reviewer ymzV's concerns; it is not entirely clear to me from the qualitative analyses presented that this model is better than those used in prior work (even the simple clustering method originally used by Huth et al., 2016), and the authors don't seem to make a compelling case for this point (e.g. showing that the dense embeddings *do not* reveal the same structure in the neural data). The closest they come is the mathematical argument, but 1) I don't find the mathematical argument for non-identifiability to be particularly novel (it basically just says that the null space of a non-invertible matrix is non-trivial, which is just a fundamental fact about linear algebra), and 2) showing that another method can fail to recover full information in some cases is not a sufficient argument for why a new method is better. Sparse factorization can *also* be non-identifiable in some cases. I'd encourage the authors to take these issues into consideration in revising the paper.

On the whole, I find this paper to be just over the threshold of interestingness to accept, despite its limitations and framing issues.
Decision: Accept (poster)