Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement
Authors: Linyang He
Tianjun Zhong
Richard Antonello
Gavin Mischler
Micah Goldblum
Nima Mesgarani
Venue: NeurIPS 2025 poster ()
Keywords: (Cognitive/Neuroscience) Neural Coding
(Cognitive/Neuroscience) Language
Primary area: neuroscience_and_cognitive_science
TL;DR: We isolate “reasoning embeddings” from LLMs using a residual method and show they uniquely predict brain activity, revealing distinct neural correlates of reasoning beyond shallow linguistic features.

Abstract: Understanding how the human brain progresses from processing simple linguistic inputs to performing high-level reasoning is a fundamental challenge in neuroscience. While modern large language models (LLMs) are increasingly used to model neural responses to language, their internal representations are highly "entangled," mixing information about lexicon, syntax, meaning, and reasoning. This entanglement biases conventional brain encoding analyses toward linguistically shallow features (e.g., lexicon and syntax), making it difficult to isolate the neural substrates of cognitively deeper processes. Here, we introduce a residual disentanglement method that computationally isolates these components. By first probing an LM to identify feature-specific layers, our method iteratively regresses out lower-level representations to produce four nearly orthogonal embeddings for lexicon, syntax, meaning, and, critically, reasoning. We used these disentangled embeddings to model intracranial (ECoG) brain recordings from neurosurgical patients listening to natural speech. We show that: 1) This isolated reasoning embedding exhibits unique predictive power, accounting for variance in neural activity not explained by other linguistic features and even extending to the recruitment of visual regions beyond classical language areas. 2) The neural signature for reasoning is temporally distinct, peaking later (~350-400ms) than signals related to lexicon, syntax, and meaning, consistent with its position atop a processing hierarchy. 3) Standard, non-disentangled LLM embeddings can be misleading, as their predictive success is primarily attributable to linguistically shallow features, masking the more subtle contributions of deeper cognitive processing. Our work provides compelling neural evidence for an abstract reasoning computation during language comprehension and offers a robust framework for mapping distinct cognitive functions from artificial models to the human brain.

Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement
Linyang H. Tianjun Zhong Richard Antonello 1.2
Gavin Mischler Micah Goldblum² Nima Mesgarani
"Zuckerman Mind Brain Behavior Institute, Columbia University Department of Electrical Engineering, Columbia University 'Department of Computer Science, Columbia University
Correspondence: linyang he@columbia edu nima@ee. columbia. edu
Abstract
Understanding how the human brain progresses from processing simple linguistic imputs to performing high-level reasoning is fundamental challenge in neuro- science. While modern large language models (LLMs) are increasingly used to model neural responses to language, their internal representations are highly "entangled," mixing information about lexicon, syntax, meaning, and reasoning. This entanglement biases conventional brain encoding analyses toward linguisti- cally shallow features (e.g., lexicon and syntax), making difficult to isolate the neural substrates of cognitively deeper processes. Here, we introduce a residual disentanglement method that computationally isolates these components. By first probing LM identify speecffc layers, method iteratively regresses out lower-level representations to produce four nearly orthogonal embeddings for lexicon, syntax, meaning, and, critically, reasoning. We used these disentangled embeddings to model intracranial (ECoG) brain recordings from neurosurgical patients listening natural speech. We show that: 1) This isolated reasoning bedding exhibits unique predictive power, accounting for variance in neural activity not explained by other linguistic features and extending the recruitment of visual regions beyond classical language areas. 2) The neural signature for reasoning temporally distinct, peaking later (-350-400ms) than signals related to lexicon, syntax, and meaning, consistent with its position atop a processing hierarchy. 3) Standard, non-disentangled LLM embeddings can be misleading, as their predictive success is primarily attributable to linguistically shallow features, masking the more subtle contributions of deeper cognitive processing. Our work provides compelling neural evidence for abstract reasoning computation dur- ing language comprehension and offers a robust framework for mapping distinct cognitive functions from artificial models to the human brain
A growing body of work has shown that LLMs exhibit strong representational alignment with human brain activity during language comprehension. These studies employ language encoding models, powerful predictors of measured brain activity in response
language stimulus [41 19 10 20 38
5 34 7
27 2 3 g 9.. Typically, these models are created by inputting language stimulus into LLM, to create contextual embedding of the stimulus using the models' internal hidden state.
A linear mapping is then learned between these hidden states and the measured brain response to that stimulus. Modern language encoding models represent some of the most advanced computational approaches for studying neural processing across sensory modalities, and they provide a unique window into the semantic processing landscape of human cortex.
"Equal Contribution.
"Code available here
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Original Holban Stades
Prodeced Holhon
Reoadaal Embedsing
Probing Tasks
Lager
D/Ni222 5-148
LUMe' Merarcical Representation
Symap
Layer
che aany
Ridge Reggaaston
thcc abae
H
Prodeng
Ar ses cantaer
Adtioaato on heat bot.
Symtox Layer
Sewiing ensintionn
A
Meaning Layer
Aegil
ontigmner Thantor 201 cantiaa Bail
Reasering
Probing Baszit
Fidipe Regression
Prediceed Natuvl Sigaal
Lextcon Feature &
M
Syntax Feature E,
Uassing
Rassor Sauation
Meaning Feature
Syrtan
Reasoning Feature E.
Full Feature E,
20
Dun.
Localization
Naural Signaly
Figure I: a) Hierarchical representations in an LLM. Transformer layers accumulate information in order: lexical features emerge first, followed by syntax, contextual meaning and eventually higher- order reasoning, with still-richer knowledge continuing in later layers. b) Minimal-pair probing tasks. Three diagnostic sentence sets separately test syntax, concept meaning and multi premme reasoning. c) Layer localization from probing curves. We define L, the earliest layer where syntax performance saturates while meaning is still low; Lm - layer where meaning saturates but reasoning has not yet emerged; and L, layer where reasoning performance plateaus. These identified layers through probing will be used in later analyses. d) Feature disentangling across layers. Starting from the localized layers, iteratively regress lower-level features out of higher ones. Details of residual embedding constructions could be found in Algorithm [ four orthogonal embeddings that isolate lexicon, syntax, meaning and reasoning information. e) Brain encoding with purified features. Each residual feature fed into a ridge encoder to predict high-gamma ECoG responses of Podcast listening. Comparing predicted and actual neural signals reveals the spatiotemporal distribution of cortical activity uniquely associated with lexicon, syntax, meaning and reasoning representations.
Residual disentangling yields
However, most work that has studied this brain-LLM alignment has focused semantics and low- level phonological relationships [1 39 32]: little is known about the degree to which LLMs and the brain align in their higher-level reasoning processes. One reason is historical: reasoning capabilities have only recently emerged robustly in modern LLMs [21]. Another issue is methodological Most prior studies have largely treated language encoding models monolithic black boxes that map entire hidden states to brain responses without disentangling the specific linguistic or cognitive functions represented within them, such lexical features, syntactic structure, semantic meaning, or reasoning processes.
Recent findings [26] show that when models trained to perform multiple tasks of varying com- plexity, their internal representations disproportionately favor simpler and more linearly extractable features, even when accuracy on complex tasks is equally high. This suggests that in brain alignment studies, unsegmented LLM features may reflect a bias toward lexical or syntactic components simply because they computationally easier to represent.
To move beyond these biases and probe the deeper relationship between machine and human reason- ing, we introduce novel approach: residual reasoning embeddings. Rather than comparing raw LLM activations brain activations, we isolate the reasoning-specific component of L.MM hidden states by separating them from discrete lexical, syntactic, and meaning features. This residual embedding

---

captures the unique portion of variance in the initial embedding that is attributable to high-level reasoning information.
We then use these residual reasoning embeddings to build encoding models that predict human electrocoricooraahicc (ECoG) brain activity during tasks that require inference. We demonstrate that these reasoning representations align with distinct spatiotemporal neural patterns, peaking later in time and engaging more frontal and visual regions than lexical or syntactic signals, suggesting shared computational hierarchy between LLMs and the human brain. Furthermore, we show that full LLM embeddings are biased toward lower-level features, and that only through disentanglement can massoning-related brain activity be meaningfully isolated. Together. our results offer new lens on both model interpretability and the neural basis of abstract linguistic reasoning.
2 Methods
2.1 Probing Datasets
To identify the LLM layers most specialized for encoding syntactic, semantic, and reasoning features, we apply feature-specific probing using two diagnostic datasets. For syntactic probing, we use the Benchmark of Linguistic Minimal Pairs (BLiMP) [40,, which evaluates grammatical sensitivity across 67 controlled paradigms. By training classifiers on hidden states from each layer of LLMs, we determine where syntactic competence emerges and stabilizes. For meaning and reasoning probing,
the Conceptual Minimal Pair Sentences (COMPS) dataset [33], which tests con- ceptual understanding and property inheritance. Its controlled sentence-pair design allows us to distinguish between surface-level semantic association and inference-based reasoning. We further extend reasoning evaluation with ProntoQA 35) and WinoGrande [22] to assess multi-hop deductive and commonsense reasoning abilities respectively. These datasets together enable hierarchical dissection of linguistic representations. Detailed dataset structures, task formulations, and examples provided in Appendix A
2.2 Language Model
Our main experiments use the Qwen2.5-14B model, a 7.7aaaaareetrrr transformer with 48 layers [36..
Each layer outputs hidden states of dimension 5120. The model supports maximum context length of 131k tokens, though our experiments only use context sizes 50 tokens. We use the base model without any instruction tuning task-specific fine- tie-ttming, and employ it both to probe the layer-wise emergence of linguistic and reasoning features and to perform brain encoding analyses.
To assess the robustness of our probing pipeline, we additionally examined other models across the Qwen family, spanning sizes from 1.8B to 14B parameters and multiple generations (Qwenl, Qwen1.5, Qwen2, Qwen2.5, and Qwen3) 4 42 36 377. We evaluated all models and found that Qwen2.5- 14B exhibited the strongest reasoning capability, which why we selected our primary model for subsequent analyses (see Appendix C.2
2.3 Minimal Pair Probing
We apply minimal pair probing [14 15]
identify the LLM layers most specialized for syntactic, meaning, and reasoning features. For each probing dataset, we extract sentence representations from each layer by feeding the sentence in isolation and taking the hidden state of its final token. We then use these sentence representations to train a logistic regression classifier to predict the correct item in each minimal pair. Model performance at given layer measured by the normalized accuracy score of this classifier.
Let Hi RRax denote the matrix of sentence embeddings at layer I, where each row corresponds to the final-token hidden state of one sentence in the minimal pair dataset. To determine the most specialized layer for a given feature, we define the saturation layer as the earliest layer where performance plateaus:
Ly = min {1 | > 1, Acc (Hy - P (HH, < <E}, I I (s, Vl'
where D, = BLIMP, Dm = COMPS-BASE, and DD COOPSSWWUS-SAA and e is small threshold representing the tolerance for marginal improvement. The hidden states at the identified
3

---

Algorithm 1 Construction of Feature- Sperittic Residual Embeddings
1: Input: LLM hidden states {H, f 11=00 for each token; probing datascts D., Dm, D,
2: Output: Feature-specific embeddings E,, E., Em, E,
3: Perform probing with D. Dm. D, to find saturation layers:
4: La f- symlax saturation layer from D3
5: Li meaning saturation layer from 2.m.
6: Ly reasoning saturation layer from D,
7:
I f-
8: Define lexical embedding: Ej <- HI.'>
9:
for cach (Lrow, Lnigh) in (LL, L.), (Ler Lmm); (Ler L do
10: Train ridge regression to predict Hugg from H
Compute residual embedding: E Lup (H
Assign E.g. Em: E, accordingly
saturation layers, denoted as H, HL,, Hm Hm and H, He,, used to construct feature-specific embeddings the next stage.
2.4 Feature-Specific Embeddings
Building on the saturation layers identified via minimal pair probing, we construct four feature- specific embeddings to isolate distinct types of linguistic information: lexical, syntactic, meaning, and reasoning. Since hidden states various layers contain overlapping linguistic information, remove contributions from earlier representations to disentangle the targeted feature.
Lexical embedding. We define the lexical embedding F E Rdd the hidden state layer the LLM. As this layer follows token embeddings directly, it reflects uncontextualized lexical properties.
Residual embeddings for syntax, meaning, and reasoning. For the remaining features, syntactic, meaning, and reasoning. construct residual embeddings by removing lower-level contributions from higher-layer representations. For instance, to isolate reasoning information, we remove the meaning contribution from the layer where reasoning saturates:
Fyy = H, gr (Hm), where = arg min H,, W H 11 + [
where 9r a ridge regression trained via 4-fold cross validation on a podcast corpus described the next paragraph (.4). The procedure applies compute F,, and Fm. Specifically:
Em Hm 9s (Ha)
(meaning minus syntactic) (syntax minus lcxical)
E, = H. gi (H))
This yields feature-specific representations that are aligned with the linguistic hierarchy (Appendix and minimally confounded by lower-level signals.
Dataset for residual regression training. To extract feature-specific residual embeddings, we train ridge regression models that require large number of training samples. However, the transcript that will later be used for neural alignment, which is drawn from a single 30-minute podcast episode, is short support stable regression. To address this, train the models 16 transcribed episodes from the same podeast series, including the episode used in the alignment analysis [30]. This 160k-token dataset enables regression without PCA, preserving richer structure in the hidden states. We then apply the trained models to the target transcript to extract residuals for encoding analysis.
expanded corpus of
2.5 Encoding Model
ECoG dataset. After constructing feature-specific embeddings, we assess their neural alignment us- ing the Podcast BCoG dataset 43). This dataset contains high-gamma band (70-200 Hz) intracranial recordings from nine participants as they listened to a 30-minute narrative podcast. It includes 1,330

---

electrodes and time-aligned word-level transcript, making it ideal for testing how lexical, syntactic, meaning. and reasoning embeddings align with neural activity during language comprehension.
As described in Section 2.4 we train ridge regression models on an expanded podcast corpus to isolate feature-specific residuals. We then apply these models to the ECoG-alligne transeript to extract feature-specific embeddings, which are used to predict neural responses. Each embedding is aligned to individual word onsets, and for each word, neural signals are epoched in =2s window and downsampled to 32 Hz. yielding & = 128 time bins per event. Let X E IRaxd be the matrix of input embeddings (lexical, syntactic, meaning. or reasoning) across word-aligned tokens, and y E (e-((e-)) the corresponding ECoG response matrix across electrodes and /. time lags. We fit:
W arg min ||Y XWI + a
W
with a selected via 5-fold cross-validation over a log-spaced grid, and = bootstrap resamples per fold using contiguous chunks of length I 32. Model performance is quantified by Pearson correlation between predicted and actual signals. Temporal profiles are obtained by averaging over channels each lag; spatial maps visualize per-channel peak correlations 3D brain coordinates.
Word-rate feature regress out. We controlled for generic acoustic onset responses by adding a two-column word-rate covariate (word onsets and syllable-rate) every ridge model. All variance- partitioning steps therefore quantify the variance explained beyond that attributable to mere word onsets. Let Ruull denote the cross-validated prediction correlation achieved using the combined feature set (embedding + word rate), and let Rwe denote the correlation using only the word rate features. Assuming approximate orthogonality between the feature sets, we estimated the unique contribution of the embedding features as: Rembed This operation projects the full correlation vector onto the embedding-one -nly axis, removing variance explained by word rate features. The assumption of orthogonality approximately satisfied due to the preprocessing pipeline, and helps to prevent over-attribution of shared variance.
sign(Rral) /max(0, R R.,).
Null distribution and responsiveness criterion. Considering that different channels and features have varying signal-to-noise ratios (SNRs), we constructed subject-electrode tribution to assess whether feature block explains neural activity beyond chance and enable cross-electrode analysis. This was done by shuffling the feature rows 500 times while keeping the word-onset covariates fixed. For every shuffle we refit the ridge model and recorded the peak correlation R. Because Pearson correlations are bounded and skewed near +1, we applied the standard Fisher z-transform (atanh) to all correlations, computed the shuffle mean and s.d. z-space, and converted the true correlation to z-score. Electrodes with 2 3.95 (one-tailed = .05, conferroni-correete across N 1268 electrodes) were deemed responsive, corresponding to values exceeding 3.95 standard deviations above the shuffle mean.
electrode -pecrfid-ssecific null dis-
3 Probing Results
Applying our probing pipeline to Qwen2.5-14B, we observe a clear progression in the emergence of representational features. Syntax saturates earliest at layer 6, followed by meaning layer 20, and reasoning only at the deeper layer 30. This ordering highlights that low-level linguistic structure is captured quickly, whereas high-level reasoning requires substantially more depth.
To verify that this pattern is not specific to single model, we extended the probing pipeline across multiple Qwen families spanning different sizes and generations. The same emergence order was consistently observed, with only one exception in Qwen-1.8B, where syntax and meaning saturated at the same layer. Full cross-model analyses are reported in Appendix B
4 Disentanglement Validation
4.1 Matrix-Level Orthogonality Residual Embeddings
We justify the approximate orthogonality of the feature-specific embeddings Et, E., Em, Ex at the level of their embedding matrices. Each residual embedding can be represented as an n X d matrix, where is the number of tokens in the dataset and d is the embedding dimension. Orthogonality here
5

---

is considered along the sample axis: the n-dimensional feature columns of one residual embedding matrix are approximately uncorrelated with those of another.
This follows from the progressive emergence of linguistic features across L.M layers (Table I syntax peaks early, meaning in mid layers, and reasoning in later layers. Once feature reaches saturation, its accuracy score remains stable in deeper layers, indicating that later representations retain earlier features. As result, representations like H. already embed information from H1 and H., making regression from Hm alone comparably as informative as from all three:
9r (m)) = g (H,, Ho, Hml)
E, = H g ([H, Hg: Hm)) =: E,
Since E,, is the residual of linear projection onto [Hi, Ha, Hm,, it orthogonal each: Hm
E 1 Hi,
Fy Hs, Ey
Each residual embedding E, E {E, Es. Em} J is a linear combination of earlier hidden states (e.g.,
Em Hm 9m (H,) Hm Wm H.). By the bilinearity of covariance, we have:
Cov(EJ, E,) = Cov(H, W,H,, E = Cov(H,, E.) W, Cov(H2, E,) 0
whenever E, H and H4, for appropriate j. E {1. s, m). This implies:
(F F.j) Vj E m.
Applying this proof across all residual stages, we conclude approximate mutual orthogonality of residual embedding matrices:
Bk) 0 for all k.
This sample-axis orthogonality ensures that residual embedding matrices contribute non-overlapping signals across the dataset, which is precisely the property required for the subsequent brain encoding analysis.
4.2 Token-Level Orthogonality via Cosine Similarity
While Section 4.1 establishes matrix-level orthogonality across the dataset, we also test orthogonality at the level of individual tokens. For each token i E (1....,N), let Ej, E. E E, denote the four feature-specifie residual vectors. We compute 4 4 cosine similarity matrix C₁, take its absolute value [Cr). and average across samples:
[Clia = |EJ|| |
C= N
Σ
j, E (l,sm,r) Cal, 1=1
In the ideal case of perfect disentanglement. off-diagonal entries of C' would be zero, indicating orthogonality between different embeddings. Figure compares the pairwise mean absolute cosine similarity among raw hidden states at the four saturation layers (top) and among the corresponding residual embeddings (bottom). While the hidden states show substantial overlap, especially between meaning and reasoning layers (C 0.751), the residual embeddings exhibit near-zero off off-diagonal similarity(all <= 0.045) across all pairs.
Thus, Section 4.1 and Section 4.2 together provide complementary evidence: residuals are linearly novel across the sample axis, and they also align with near-orthogonal directions at the token level. These findings confirm that residualization yields disentangled representations of lexical, syntactic, semantic, and reasoning information.
4.3 Feature Probing on Residual Embeddings
While mutual decorrelation ensures that residual embeddings are separated from one another, each residual should also preserve information relevant to its intended linguistic feature. We evaluate this by reapplying the same probing tasks used to define feature emergence. Since the lexical embedding E2 directly extracted rather than residualized, we focus on the syntactic (F,,,, meaning (F,,). and reasoning (E,) residuals.

---

Mban Absolute Cosine Similarity
Residual Embeddings
1.0
1.0
0.03
0.8
0.5
Aimilani
Residual Disentanglement
0.6
Similarity
Ho
0,029
0.751
0.4
0.013
0.030
-.000
0.043
0.4
0.0
0.0
H
H
Hm
H+
6
Em
E
b
Feature Probing Accuracy
ressoning mesning syntax
0.927
0.977
0.968
C.8.52
0.647
0.612
0.8
Probing Task
reasiring meening
0.6
0.0
Residual Disentanglament
Probing
0.589
0.427
0.474
0.682
0.2
0.448
0.532
0.716
0.2
H:
Hin
Hr
Er Embeddings
Hidden States
Figure 2: Pairwise cosine similarity among representations before (left) and after (right) residual disentanglement. The hidden states at feature saturation layers (H1, Hs, Hm, H,) exhibit substantial overlap. In contrast, the residual embeddings (Ei, Es, Em, E,) show near-zero off-diagonal simi- larity. b) Feature probing results on hidden states (left) and residual embeddings (right), shown as accuracy. Each residual embedding achieves the highest performance its corresponding task, while performing worse on unrelated tasks.
For each case, we train logistic regression classifier on the corresponding residual embedding and report accuracy scores:
9. = [ E3 + by),
ms,
where (-) is the sigmoid function and E E Rd is the residual embedding for token
Results are shown in Figure 2p. Each residual achieves the highest performance on its own task (e.g.. F, syntax probing, Fm meaning, E, reasoning) while performing worse unrelated tasks. The diagonal entries retain accuracy similar to the hidden-state baseline, whereas the off-diagonal entries drop substantially, with decreases of 33.8% (syntax task evaluated the meaning residual), 36.8% (syntax task evaluated on the reasoning residual), and 19.4% (meaning task evaluated on the reasoning residual). These findings demonstrate that the residual embeddings are meaningfully disentangled and capture feature-specific information rather than reflecting general task complexity. more detailed analysis of cross-task accuracies, including Bag-of-Wil -ord baseline, provided in Appendix D
Summary of Disentanglement Validation Across three complementary tests (matrix-level or- thogonality proof, laken-wise orthogonality computation, and probing the residual embeddings), residualization consistently produces four nearly orthogonal, interpretable embeddings corresponding to distinct linguistic levels. These embeddings form the basis for our subsequent neural encoding experiments.
5
Encoding Results
5.1 Shallow features dominate neural prediction, but lexical activations are sparse.
Across time-aligned word events, encoding models based on shallower linguistic features, such as lexical identity and syntactic structure, showed the strongest neural correlations (Figure 3), Electrodes
7

---

Peak Corre Carioaation across Features
b
Peak Correlation by Subject and Feature
0.30
Hean
0.250
Locccnn
Miaaring
Full
Syntax
Cornelaion
Corneaatoo
0.200
0.20
0.375
0.15
0.125
0.100
0.05
0.050
Lenkon (nw93)
Syntax [n=265]
Meaning
Beasaring
Full (n-375)
(n=290) (n=2381
Im-1331
(2-2341
(mm1s4)
Inmion
(nm1700
Inm321
(mm281)
Feature
Subject
Figure 3: a) Peak correlation across linguistic feature models. As described in the Methods section, electrodes with > 3.95 (one-tailed a .05, Bonferroni-correctt across 1268 electrodes) were deemed responsive, corresponding to values exceeding 3,95 standard deviations above the shuffle mean. Boxplots indicate the median and interquartile range across activated electrodes with red lines indicating means additionally. Asterisks mark significant differences between adjacent features (Welch's t-test; p < 0.001), showing that lexical and syntax features yield significantly higher correlations than high-level linguistic representations. b) Peak correlation by subject and feature. The number of responsive electrodes for each subject shown below the x-axis. Lexical features generally exhibit higher correlations, indicating stronger neural alignment for lower-level linguistic representations.
responsive to these features exhibited markedly higher peak correlation values than those driven by meaning or reasoning embeddings (Welch's I-test, p < 0.001). confirming that lower-level linguistic information accounts for a disproportionate share of brain- However, despite its high correlation strength, the lexicon embedding activated the fewest electrodes overall, suggesting that its predictive precision arises from a smaller, more specialized subset of cortical sites. In contrast, deeper features such as meaning and reasoning recruited broader but less strongly tuned populations. This dissociation between activation magnitude and spatial extent indicates that shallow representations, though limited in coverage, achieve tighter alignment within localized neural circuits, whereas deeper representations engage distributed cortical networks with weaker linear correspondence. Moreover, without disentanglement, the temporal dynamics in Figure reveal that the full embedding is dominated by lexical and syntactic signals, whose early and sharp peaks mask the slower, high-level temporal signatures of meaning and reasoning. This highlights the necessity of residual separation for isolating deeper cognitive processes from low-level linguistic confounds.
alignment under linear encoding.
5.2 Reasoning embedding shows different temporal pattern compared to Lexicon, Syntax and Meaning.
The temporal correlation profiles of Figure illustrate a clear processing hierarchy among linguistic representations. Syntactic signals exhibit an early rise that begins before word onset and reach their maximum slightly pre-onset, suggesting that large portion of predictive activity for upcoming words might be syntactically driven. Lexical features show a sharp and temporally confined peak immediately after onset, consistent with rapid low-level mapping between acoustic input and lexical identity. Meaning representations start to increase even before onset but peak afterward, indicating that contextual semanties contribute both anticipatory prediction and continued integration once the word is heard. In contrast, reasoning signals begin to rise only after onset and reach their peak around 362 ms, reflecting delayed, high-level computation that likely involves integrative or inferential processes beyond immediate linguistic parsing. Finally, the full embedding closely tracks the envelope of the four disentangled features, mirroring their composite dynamies and reinforcing the reliability of our residual separation procedure. Together, these temporal patterns reveal cascading progression from predictive syntactic structure to contextual semantics, and finally to post-lexical reasoning operations in the human cortex.

---

Temporal per Channel
Temporal Profile
Syrtuc
Plarring
Lonksn
330ms
36ms
20
8.135
Charme (2007) 1000
Coreeltton
0.100
6071
330m DECRO
10
15
6050
6425
-03 0.2
Figure 4: a) Average correlation time courses for the top 10% most responsive electrodes reveal distinct temporal dynamics across linguistic features. Responsive electrodes were identified following the strategy described the Methods section. Syntactic signals rise before word onset, followed by meaning, lexicon and reasoning, which peak latest at ~~36 ms. Significance markers (x) indicate time points where correlations are significantly greater than zero (one-tailed (-test, FDR-corrected < 0.05). b) Temporal profile of individual electrodes selective to each feature.: Lexicon, Syntax, Meaning, Reasoning, and Full. Electrodes were selected the same way as panel a.
a
Lexicom
Syntax
Moaning
Roasoning
rull
b Dominant
Yeunteg
Syrtxx
Loncon
Figure 5; a) Spatial distribution of feature-selective responsive electrodes (MNI space). Displayed electrodes are those that met the responsiveness criterion defined in the Methods section. For visualization, z-scores were clipped to 3-6 to prevent extreme values from dominating the color and obscuring spatial patterns. b) For each electrode, the feature with the highest peak z-score among Lexicon, Syntax, Meaning, and Reasoning models is shown its dominant feature. Colored dots indicate the dominant model. Across all electrodes, the counts of dominant features were: Syntax 166, Meaning 161, Reasoning 128, and Lexicon = 42.
scale
5.3 Reasoning recruits more than language area compared to low-level aspects.
As shown in Figure 3 shallow linguistic features such lexicon exhibit high peak correlations but remain confined to classical language regions, including the inferior frontal gyrus (IFG) and superior temporal gyrus (STG). The strong yet spatially restricted activations indicate that lexical representations are predominantly localized within the traditional perisylvian language network, reflecting low-level linguistic encoding. In contrast, syntax, meaning, and particularly reasoning embeddings engage broader set of cortical regions. While all three maintain robust responses in IFG and STG, massoning-selectvve electrodes extend anteriorly into the superior frontal gyrus (SFG) and posteriorly into the visual cortex, showing significant activation compared with other

---

a
b
0.250
Lesicon
0.13
Syntax
Meaning
0/12
Ressoring
Peak Corr (selected)
0.200
0.11
0.175
0.10
0.150
0.09
0.125
0.00
0.100
0.075
0.06
0.95
Supertor Gyrus
Vianoul
Oroital
Oparcular
Anberior
Midcle
Fosterior
Visual
Visual Cortex
Figure 6: a) Peak correlation across cortical regions and feature models. Responsive electrodes were selected based on bilateral significance above the null shuffle distribution, a 0.05. The analysis focuses on language-related cortical regions, including the inferior frontal gyrus (IFG), superior temporal gyrus (STG), and surrounding Heschl's Gyrus (HG), well the superior frontal gyrus and visual cortex. The visual cortex group includes electrodes located in the superior occipital sulcus, transverse occipital sulcus, calcarine sulcus, occipital pole, superior occipital gyrus, middle occipital gyrus, and inferior occipital gyrus and sulcus. b) Welch's 's t-test (one-tailed) revealed that electrodes in visual cortex activated by the Reasoning feature showed significantly higher correlations than those activated by all other features (p < 0.001 for ***** < 0.05 for ***).
features (Welch's t-test; p < 0.001; Figure 6). This expanded recruitment beyond canonical language areas suggests that reasoning might involve higher-order cognitive operations that integrate linguistic input with abstract inference, visual imagery, and executive control. Such cross-modal engagement underscores that reastnd brain activity cannot be fully explained by linguistic processing alone, but rather reflects the neural substrates of generalizable, multimodal cognition 25 12].
Discussion
By isolating distinct types of linguistic information using residual embeddings, we disentangled the contributions of lexicon, syntax, meaning, and reasoning to the neural encoding of natural speech. This revealed new insights into the spatial and temporal organization of linguistic features that
cannot be captured by full LLM embeddings used in prior studies (20 38 13 34 17 27 2 3
8, 9 32, 6). Previous work has tried hierarchical approaches to decompose low-level linguistic However, our study the first to apply hierarchical disentanglement framework to large language models by first localizing feature emergence layers through probing and then performing residual decoupling construct layer- wise feature representations, thereby demonstrating its utility for isolating higher-order reasoning representations.
and acoustic features in the brain (Keshishian
23 24
Our findings show that lower-level features, such as syntax and lexicon, elicit stronger neural activations than higher-level features like meaning or reasoning. This reflects both the stronger linear accessibility of shallow features and an inherent bias in full LLM embeddings, which primarily capture syntactic regularities while obscuring deeper computations [26]. Residual disentanglement reveals clear spatiotemporal hierarchies: shallow features emerge earlier and localize to classical language regions (IFG, STG), whereas reasoning signals arise later and extend beyond the perisylvian network into SFG and visual cortex. These results indicate that reasoning engages broader cortical systems that integrate linguistic processing with higher-level cognitive and multimodal functions. Together, our framework provides principled approach for separating linguistic and cognitive components L. and mapping them onto the human brain, offering an alignment-base perspective on how language and reasoning may correspond between artificial and neural systems.
10

---

Acknowledgment
biorxiv This work is funded by the National Institutes of Health (NIH- NIDCD) and grant from Marie-Josée and Henry R. Kravis.
References
[1] Richard Antonello, Javier S Turek, Vy Vo, and Alexander Huth. Low-dimensional structure in the space of language representations is reflected in brain responses. Advances in neural information processing systems, 34:8332-8344, 2021.
[2] Richard Antonello, Aditya Vaidya, and Alexander Huth. Scaling laws for language encoding models in fmri. Advances in Neural Information Processing Systems, 06:21895-21900 2023.
[3] Khai Loong Aw and Mariya Toneva. Training language models to summarize narratives improves brain alignment. arXiv preprint arXiv:2212.10898 2022.
[4] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16666 2023.
[5] Jonathan R Brennan, Chris Dyer, Adhiguna Kuncoro, and John T Hale. Localizing syntactic predictions using recurrent neural network grammars. Neuropsychologia, 146: 107479, 2020.
[6] Charlotte Caucheteux and Réan-Rémi King. Brains and algorithms partially converge in natural language processing. Communications biology, 5(1):134, 2022.
[7] Catherine Chen, Tom Dupré la Tour, Jack Gallant, Daniel Klein, and Fatma Deniz. The cortical representation of language timescales is shared between reading and listening. bioRxiv, pages 2023-01, 2023.
[8] Peili Chen, Linyang He, Fu, Lu Fan, Edward F Chang, and Yuanning Li. Do self-supervised speech and language models extract similar representations human brain? In ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2225-2229, IEEE, 2024.
[9] Peili Chen, Shiji Xiang, Linyang He, Edward F. Chang, and Yuanning Li. Convergent rep- resentations and spatiotemporal dynamics of speech and language in brain and deep neural networks. bioRxiv, 2024. doi: 10. 101/2024. 12.28...2222 630582. htt.:tts:: htg//oontenteearly////0 wttpaaarraaa/////////a/e///
12. 28. 630582
[10] Wendy A de Heer, Alexander G Huth, Thomas I. Griffiths, Jack L. Gallant, and Frédéric Theunissen. The hierarchical cortical organization of human speech processing. Journal of Neuroscience, 37(27):6539-6557, 2017.
[11] Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. Neural language models as psycholinguistic subjects: Representations of syntactic state. arXiv preprint arXiv: 1903, 190.003600000 2019.
[12] Giorgio Ganis, William L. Thompson, and Stephen M. Kosslyn. Brain areas underlying visual mental imagery and perception: An IMRI study. Cognitive Brain Research, 20(2):226-241, 2004. doi: 10. wwt..oobaiinsss.00..2....
[13] Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, et al. Shared computational principles for language processing in humans and deep language models. Nature neuroscience. 25(3):369380, 2022.
[14] Linyang He, Peili Chen, Ercong Nie, Yuanning Li, and Jonathan R Brennan. Decoding probing: Revealing internal linguistic structures in neural language models using minimal pairs. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LLEC-CCOLIE 2024), pages 1488-4497, 2024.
11

---

[15] Linyang He, Ercong Nie, Helmut Schmid, Hinrich Schütze, Nima Mesgarani, and Jonathan Brennan. Large language models as neurolinguistic subjects: Discrepancy between performance and competence. In Findings of the Association for Computational Linguistics: ACI. 2025, pages 19284-19302, 2025.
[16] Linyang He, Qiaolin Wang. Xilin Jiang, and Nima Mesgarani. Layer-wise minimal pair probing reveals contextual grammatical--onncepuus hierarchy in speech representations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 15326-35341 2025.
[17] Micha Heilbron, Kristijan Armeni, Jan-Mathijs Schoffelen, Peter Hagoort, and Floris De Lange. A hierarchy of linguistic predictions during natural language comprehension. Proceedings the National Academy of Sciences, (9(32):e2011668 19, 2022.
[18] John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Pupers), pages 4129-4111 4138, 2019.
[19] Alexander G Huth, Wendy A De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack L Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532 (7600):453-45) 458, 2016.
[20] Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fmri. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Cur- ran Associates, Inc., 2018.
neurips.c. httpppscccc/pppee////////ee ppf
[21] Zixuan Ke, Fangkai Jiao, Yifei Ming. Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in Ilm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037, 2025.
[22] Sakaguchi Keisuke, Le Bras Ronan, Bhagavatula Chandra, and Choi Yejin. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv: 1907. 10641, 2019.
[23] Menoua Keshishian, Serdar Akkol, Jose Herrero, Stephan Bickel, Ashesh D Mehta, and Nima Mesgarani. Joint, distributed and hierarchically organized encoding of linguistic features in the human auditory cortex. Nature human behaviour, 7(5):740-77 2023.
[24] Menoua Keshishian, Gavin Mischler. Samuel Thomas, Brian Kingsbury, Stephan Bickel, Ashesh D Mchta, and Nima Mesgarani. Parallel hierarchical encoding of linguistic repre- sentations in the human auditory cortex and recurrent automatic speech recognition systems. bioRxiv, 2025.
[25] Stephen M. Kosslyn, William I.. Thompson, and Nathaniel M. Alpert. Neural systems shared by visual imagery and visual perception: A PET study. Neurolmage, 6(4):320-334, 1997. doi: 10. 10.610000...1//// 1997.0295.
[26] Andrew Kyle Lampinen, Stephanie C. Y. Chan, and Katherine Hermann. Learned feature representations are biased by complexity, learning order, position, and more. Transactions on Machine Learning Research, September 2024. URL
aY2nsgE97a
[27] Yuanning Li, Gopala K Anumanchipalli, AbdeIrahman Mohamed, Peili Chen, Laurel H Carney, Junfeng Lu. Jinsong Wu, and Edward F Chang. Dissecting neural computations in the human auditory pathway using deep neural networks for speech. Nature Neuroscience, 26(12):221- 2225, 2023.
[28] Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. Assessing the ability of Istms to learn syntax- sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521-535, 2016.
12

---

[29] Christopher D Manning, Kevin Clark, John Hewitt, Urvashi Khandelwal, and Omer Levy. Emer- gent linguistic structure in artificial neural networks trained by sulf-viupn Proceedings of the National Academy of Sciences, 1 117448):3046-33005
2020.
[30] Huanru Henry Mao, Shuyang Li, Julian McAuley, and Garrison Cottrell. Speech recognition and multi-speaker diarization of long conversations. arXiv preprint 20t.::000508077. 2020.
[31] Rebecca Marvin and Tal Linzen. Targeted syntactic evaluation of language models. In Pro- ceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018, Association for Computational Linguistics. doi: 10. L/DI8- 1151. URL https: patttss//accanteee awloocoonnntooyyyyyoooomooo org/ B01//D181111 1151
pages 1192-1202, Brussels. Belgium,
[32] Gavin Mischler, Yinghao Aaron Li, Stephan Bickel, Ashesh D Mehta, and Nima Mesgarani. Contextual feature extraction hierarchies converge in large language models and the brain. Nature Machine Intelligence,
2024.
[33] Kanishka Misra, Julia Rayz, and Allyson Ettinger. Comps: Conceptual minimal pair sentences for testing robust property knowledge and its inheritance in pre-trained language models. In Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2928-2949, 2023.
[34] Subba Reddy Oota, Manish Gupta, and Mariya Toneva. Joint processing of linguistic properties in brains and language models. arXiv preprint (rXXiv22212.88//1 2022.
[35] Abulhair Saparov and He He. Language models greedy reasoners: A systematic formal analysis of cfain-of-thought. arXiv preprint arXiv:2210.01244 2023.
[36] Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15111,, 2025. URL https
Frrxiv.org/ab//4122... 15115
[37]
Qwen Team. Qwen3 technical report, 2025. URL
09388
[38]
Mariya Toneva and Leila Wehbe. Interpreting and improving natural-language processing (in machines) with natural language-processingg (in the brain). In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alccé-Bue, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. /ttpss pdf
[39] Aditya R Vaidya, Shailee Jain, and Alexander G Huth. Self-supervised models of audio effectively explain human cortical responses to speech. arXiv preprini arXiv:2205. 24..2222....2,, 2022.
[40] Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohanancy, Wei Peng. Sheng-Fu Wang, and Samuel R Bowman, Blimp: The benchmark of linguistic minimal pairs for english. Transactions of the Association for Computational Linguistics, 8:377-392, 2020.
[41] Leila Wehbe, Brian Murphy, Partha Talukdar, Alona Fyshe, Aaditya Ramdas, and Tom Mitchell. Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PloS one, 911)xe 2014.
[42] An Yang. Baosong Yang. Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang. Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin. Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang. Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report. arXiv preprins arXiv:2407.10671 2024. httpstt:::/ http://axxvv../// org/ abt//////////0 10671
[43] Zaid Zada, Samuel A Nastase, Bobbi Aubrey, Itamar Jalon, Sebastian Michelmann, Haocheng Wang, Liat Hasenfratz, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Sasha Devore, Adeen Flinker, Orrin Devinsky, Ariel Goldstein, and Uri Hasson. The "podcast" ecog dataset for modeling neural activity during natural language comprehension. bioRxiv, 2025. doi: 10.1000/202 24..22222...2.1..3333332 URL https: / //ooi org/ 1101/2025 02. 14. 638352
13

---

A Detailed Description of Probing Datasets
BLIMP for syntactic probing. To construct feature-specific representations for brain alignment,
first identify LLM layers that specialize in syntactic encoding. Minimal pairs and probing techniques
have been widely used NLP field access LLM's internal representation [1 28 [8] 29 16].
Here we use the Benchmark of Linguistic Minimal Pairs (BLiMP) [40], which evaluates syntactic knowledge through controlled minimal pairs differing in grammaticality across 67 paradigms. Each pair isolates a specific grammatical phenomenon such as subject- verb agreement, reflexive anaphora, or negative polarity. By training a classifier to distinguish acceptable from unacceptable sentences using hidden states from each L.M layer, we detect where syntactic competence emerges and peaks. For example, a task targeting subject-verb agreement:
a. The cats annoy Tim. / b. The cats annoys Tim.
We average classifier performance across all paradigms to determine the saturation layer where syntactic features are stably encoded.
COMPS for Meaning and Reasoning Probing. For meaning and reasoning probing, we use the Conceptual Minimal Pair Sentences (COMPS) dataset (33 Sentence pairs in COMPS differ in whether the subject plausibly inherits property. By probing whether each layer favors the correct concept-property association, we detect the layers where meaning and reasoning abilities peak.
COMPS-BASE evaluates surface-level understanding without requiring inference. For example, given the property "can heat food":
a. An oven can heat food. /
b. A refrigerator can heat food. x
SOMPS-WUGS-DIST increases reasoning complexity by replacing known concepts with nonsense words (e.g., "wug", "dax") and inserting distractor sentences that introduce interference. This design prevents the model from relying on surface-level co-occurrence or positional cues: distractors can be reordered. The model must infer the identity of the nonsense word from context to generalize property inheritance.
a. A wug is an oven. A dax is refrigerator. Therefore, a wug can heat food. / b. A wug is an oven. A dax is refrigerator. Therefore, a dax can heat food. X
This progressive design increases reasoning difficulty and allows us to localize where models transition from semantic to reasoning-lews representation.
Additional reasoning benchmarks. Beyond COMPS, we incorporate two additional reasoning benchmarks validate robustness across task formats: ProntoQA [35], which evaluates multi-hop deductive reasoning, and WinoGrande [22). which targets commonsense coreference resolution. For ProntoQA. we only use examples that require 5-hop deductive inferences to ensure high difficulty, where solving the task requires maintaining consistent symbolic relations across multiple premises and performing compositional logical deduction. Two illustrative examples are:
a. Dogs are cats. Each dog is sour. Vertebrates are dull. Felines are dogs. Felines are not dull. Cows are felines. Each cow is aggressive. Snakes are cows. Snakes are orange. Animals are snakes. Every animal is not luminous. Mammals are animals. Each mammal is hot. Fae is mammal. Therefore, Fae is not dull.
b. Every carnivore is dog. Carnivores angry. Every mammal is carnivore. Each mammal is not red. Each snake is mammal. Each snake is transparent. Cats are snakes. Cats are nervous. Each sheep is not angry. Each animal is a cat. Animals are earthy. Sam is an animal. Therefore, Sam not angry. X
For WinoGrande, which tests commonsense reasoning through contextual coreference resolution, the model must leverage world knowledge and pragmatic inference identify the correct referent under minimal lexical cues. A pair of examples is:
14

---

a. John moved the couch from the garage to the backyard to create space. The garage is small.
b. John moved the couch from the garage the backyard to create space. The backyard is small. X
Together, these datasets complement COMPS by spanning property inheritance, deduction, and monsense inference. Detailed experiments and cross-validation results are provided in Appendix
Bad Probing Task Filtering We employ Bag-of- -ordds (BoW) baseline and exclude tasks where the BoW model achieves an accuracy above 0.6, resulting in a final set of 29 BLiMP tasks (out of 67). tasks remain unchanged.
The meaning (OMPS--AAE) and reasoning (COMPS-WUGS-
This filtering step motivated by fact that BoW readily captures tasks solvable via simple lexical cues, indicating shallow design. By removing these superficially "easy" tasks, ensure that the retained tasks demand genuine syntactic or semantic understanding rather than mere lexicon- based heuristics.
B Hierarchical Emergence of Linguistic Features
In the main text, we reported probing results for Qwen2.5-14s (Section 3). showing that syntax, meaning, and reasoning features saturate layers 6, 20, and 30, respectively. To evaluate the robustness of this emergence pattern, we extended our probing pipeline to 17 models across the Qwen family, including Qwen, Qwenl.5, Qwen2, Qwen2.5, and Qwen3 models.
Across nearly models, we observed the same progression of feature emergence: syntax saturates Owen-1.8B where syntax
earliest, followed by meaning, and then reasoning. The only exception and meaning saturated at the same layer. Table reports the saturation layers identified for all models.
Table Saturation layers for syntax, meaning, and reasoning across Qwen models.
Model
Syntax
Meaning
Reasoning
Qwen-1.8h
Qwen-7B
11
13
1.5
Qwen-14h
20
Qwenl.5-1.88
10
11
14
Qwenl.5-77
Qwenl.5-14B
8
16
20
Qwen2-1.5B
10
Qwen2-7B
7
14
18
Qwen2.5-1.5B
14
18
Qwen2.5-7B
7
15
19
Qwen2.5-14BB
6
20
30
Qwen3-1.7B
16
19
Qwen3-8B
20
23
Qwen3-14B
17
27
C Consistency Across Reasoning Benchmarks
In the main text, reasoning saturation layers were identified using robustness across tasks of varying complexity, we additionally applied two further reasoning probes: 5-hop deductive reasoning task from ProntoQA and the WinoGrande benchmark for commonsense coreference.
To evaluate
We tested all three probes, COMPS- SHEPS-WUGSS WUGS-UUSEEEE DIST, ProntoQA, and WinoGrande, across 17 Qwen models spanning multiple sizes and training modes. Results show strong consistency: average difference in reasoning saturation layers between COMPS-WUGS-DIN WUGS-DIST and ProntoQA was 0.94, and between COMPS-WUGS. DIST and WinoGrande was 0.53. Given overall model depths 25 layers, these differences are negligible and support the robustness of our reasoning probe.
15

---

C.1 Consistent Saturation Layers Across Reasoning Benchmarks.
Sable2report the reasoning saturation layers identified by each probe. The high degree of agreement across the three tasks indicates that reasoning-speciiin representations are consistently localized in deep layers. regardless of task format. The agreement across COMPS-W WUGS-DISS and WinoGrande confirms that our identification of reasoning-specifie saturation layers generalizes beyond a single benchmark. This consistency reinforces the conclusion that reasoning features emerge reliably in deep layers across architectures and reasoning formats.
ProntoQA,
Table 2: Reasoning saturation layers identified by different probes across Qwen models.
Model
ProntoQA
COMPS-WUGS-DIST
WinoGrande
Owen-1.8B
13
14
Qwen-7B
13
15
16
Qwen- 14B
18
20
20
Qwen .5- 55-..BB
13
14
14
Qwen I 557B
14
16
16
Qwenl.5-14P
20
20
22
Qwen2-1.5B
17
17
17
Qwen2-7B
16
18
19
Qwen2.5-1.3
16
18
17
Qwen2.5-77
19
Qwen2.5-14B
29
30
29
Qwen3-1.7B
19
Qwen3-8B
23
23
23
Qwen3-14B
27
C.2 Qwen2.5-14B Achieves the Best Reasoning Performance (Small-Scale Models)
We evaluated the Qwen family on three reasoning benchmarks: ProntoQA (multi-hop deductive reasoning), COMPS WUGS-DIST (property inheritance with distractors), and WinoGrande (com- monsense coreference). T accuracy each benchmark, along with the average across tasks. We observe consistent improvements in reasoning performance with increasing model scale and newer Qwen generations. Among the base models, @wwen2.5-14B achieves the highest average accuracy across benchmarks, reflecting its stronger reasoning capability, which why we select it as the primary model for experiments. These results complement our saturation layer analysis by showing that models allocating greater depth to reasoning layers also demonstrate superior task task-level reasoning performance.
Table 3: Reasoning task performance across Qwen models. The Average column reports the mean accuracy across ProntoQA, COMPS-WUGS-DEE and WinoGrande.
Model
ProntoQA
COMPS-WUGS-DIST
WinoGrande
Average
Twen-1.8B
0.797
0.522
0.523
0.614
Qwen-7B
0.886
0.641
0.602
0.710
Qwen- 14B
0.880
0.695
0.647
0.741
Qwenl 1.eel.5-1.8B 5511.BB
0.792
0.513
0.523
0.609
Qwen 557BB
0.848
0.667
0.603
0.706
Qwen1 15-B
0.910
0.670
0.653
0.744
Qwen2- 1w5n2-1.5BB
0.784
0.586
0.552
0.640
Qwen2-7B
0,851
0.636
0.660
0.716
Qwcn2.5-1.5B
0.783
0.605
0.566
0.651
Qwen2.5-7B
0.879
0.673
0.664
0.739
Qwen2.5-14B
0.922
0.691
0.698
0.770
Owen3-1.7B
0.739
0.500
0.530
0.589
Qwen3-8B
0.876
0.629
0.651
0.719
Owen3-14P
0.912
0.716
0.670
0.766
16

---

D Clarification on Residual Embedding Probing Results
In Section 4.3 we reported that each residual embedding performs best on its corresponding task while performing worse unrelated tasks. Some cross-task accuracies remain non-trivial, particularly on the meaning and reasoning probes. We provide additional clarification here.
Lexical-level baseline. A control experiment with log-oo-wordde (BoW) model, which ignores syntax and word order, achieved 0.665 accuracy on COMPS-BAR BBSS.. This shows that lexical cues alone provide strong baseline for the meaning task, well above random chance (0.5). In this context, the raw scores of the syntax and reasoning residuals on COMPS-BASE (0.589 and 0.605, respectively) in fact below the BoW baseline, indicating that residualization successfully removes spurious lexical and structural information from the meaning embedding. This control confirms that the observed tasks-task scores largely explained by lexical baselines artifacts. The syntax, meaning, and reasoning residuals each retain information most relevant to their target feature, supporting the interpretation that residual embeddings meaningfully disentangled.
E
Supplemented Neuroscience Analysis
E.1
Activated-electrode Overlap Across Linguistic Feature Models
Figure 7 Activated-electrode overlap across feature models. Each cell shows the number of electrodes that were identified as active in both the feature model indicated by the row and that indicated by the column. Diagonal values represent the total number of electrodes active within each model, while off-diagonal values quantify cross-feature overlaps. Notably, high overlaps between Syntax and Meaning (143) and between Full and all subcomponents (around 180) suggest that electrodes responsive to linguistic processing are largely shared across multiple feature spaces. This pattern indicates that while distinct feature models capture partially unique neural activations, substantial commonality exists among them, particularly for integrative language representations.
Activated-electrode Overlap
Syntax Loxcon
93
77
62
63
86
Active set from
62
143
290
128
181
63
118
128
2.38
175
86
185
181
175
375
Syntak
Mearing
Reasoning
Also active
Figure 7: Activated-electrodee overlap across feature models. Each cell shows the number of electrodes active in both feature sets, revealing substantial overlap among higher-level linguistic representations.
E.2 Lateralization Analysis
To quantify hemispheric lateralization, we computed the proportion of active electrodes relative total electrodes in the left and right hemispheres for each linguistic feature. Although the total number of elec- trodes is highly asymmetric across hemispheres (1057 on the left vs. 210 on the right), the normalized activa- tion proportions still reveal clear trends. Specifically,
Table 4: Hemispheric lateralization ratios.
Each proportion represents the fraction of active electrodes relative to total electrodes within each hemisphere (1057 left vs. 211 right). The ratio indicates Left presection
Right propection
Feature
#L
Prop
Prop.R
L/R
Lexicon
76
17
0.07
0.08
0.89
Syntax
0.20
1.10
Meaning
227
63
0.21
0.30
0.72
17
Reasoning
193
0.18
0.24
0.76
Full
300
0.28
0.37
0.76

---

Syntax-related encoding shows a higher proportion of hettsheeip activations (0.21 vs. 0.20), resulting in the highest left/right ratio (1.09) among all fea- tures. In contrast, Meaning and Reasoning features exhibit relatively stronger hemishheeee activations (ratios < 1). Given the uneven electrode coverage across hemispheres, these findings should not be overinterpreted as true right-latcralization for semantic reasoning processing; however, they do suggest stronger left-lateralization effect for shallow representations (lexicon and syntax).
E.3
Activated Channels in All Brain Area
Please refer to Figure
Peak Correlation
Peak Correlation
50
50
020
05
5. Letmpprral traroverse
temp_sup
Litrrppaaae and
parare inf-auprmmer
CLanc spperccntiiall
temp sup tempo
s Longusteeggnd
2 nont, intoperturar
Spentraliocal
cingul
front inFiinngl
+_tnet_sip
5 oibiilh Shased
D. aupostcentral
5. ciroulrr invals ent
Aiw Jemporal
sccrodar_jnsalo..o pup
Porahip
Cometemp Geootemp_jsthooe futioor
Semporal inf
SuD
LojDay Uraln
S_oc_rup_nd tronoversal
temp sup Haa polar
3, o.occootall Liddle
6_tomp_oup 6,- transv
orbital
med and
G tont On-Oratta
cc-temp
Unknown
G Longul Post ventral
pariotal
and 5, Sccceattal, jet
jemporal supp
parat intingurar
Po ocopitl
presummuo
pooiptal_cca
Sgreoentzakifpartt
and bani frantapol
S. gpeetcrrtral
Meriicl
Ins. la and cent lts
orbital oefe-tffect
IN
LitemporaLint
Figure 8: Peak correlation values of encoding performance across all cortical regions for five feature types. Responsive electrodes were selected based on bilateral significance above the null shuffle distribution, a < 0.05.
18

---

F Limitations and Future Directions
Despite these advances, several limitations remain. Our reliance on ECoG data, although temporally precise, provides limited spatial coverage, with insufficient sampling in frontal regions that are crucial for reasoning. Future work should integrate complementary modalities such as fMRI to improve spatial resolution. Also, while Qwen2.5-14P balances scale and accessibility, larger models optimized for multi-step reasoning may reveal stronger or more specialized reasoning signals. While reasoning representations saturate around layer 30 in Qwen2.5-14B the role of subsequent layers remains unclear. We hypothesize that deeper layers may encode more abstract or generative representations, potentially related to discourse organization, narrative planning. world modeling, beyond explicit reasoning. Exploring how these higher-level representations might align with cortical hierarchies or large-scale cognitive organization in the brain presents especially interesting direction for future work.
G Compute Resources and Execution Time
We report here the compute infrastructure and approximate runtime for each stage our pipeline to support reproducibility.
Hidden State Extraction: Performed using 4x NVIDIA L40 GPUs. Extracting hidden states for around 164,000 tokens from the Qwen2.5- 14B model. The extraction took approximately 40 minutes and required around 4 x 30 GB of GPU memory.
Layer-wise Probing: Conducted using 1 NVIDIA L40 GPUs. Each BLiMP task paradigm took 2.2 minutes on average, totaling around 57.2 minutes for the selected 26 paradigms focusing syntax. COMPS-BASH consisting of 49,340 English sentence pairs took 1 hour and 47 minutes on average. COMPS-WUGS WUGS-DIS' consisting of 27,792 pairs took hour average.
Residual Embedding Construction: Ridge regression training was done using Scikit-learn on CPU with 30 GB memory. Each regression model took less than 10 minutes to converge. NVIDIA cuML and GPU training not adopted due lack of support for multi-output ridge regression training.
Brain Encoding (ECoG): Conducted using CPU with self-developed package. Align- ing model activations and training encoding models over all channels and time lags took approximately 5 hours for all experiments done.
Overall, the full pipeline can be reproduced on a modern workstation or cloud instance equipped with 1 NVIDIA L40 GPU 30 GB RAM.
H Licenses and Attribution of External Assets
This work uses several publicly available datasets and language model, all of which are properly cited and used in accordance with their respective licenses. No proprietary or scraped data was used.
BLIMP Dataset 440 suite of syntactic probing tasks for language models.
-URL:
- -License: CC-BY 4.0
COMPS Datasets 33 COMPS-BASE and COMPS-WUGS-DIST are used to assess
semantic and reasoning representations.
URL:btttp:://iihhub com./aaniskkamisrra//aaaaaaaaaaa compa/
-Lieene: Apache License 2.0
The Podcast ECoG Dataset
High-resolution electrocorticooraphic recordings from 43
participants listening to natural podcast.
-UR:
License: CCO
19

---

Expanded Podcast Transcripts [0] Text for additional podcast episodes used to extend ECoG analysis.
URL: httpps///githb..mmmmaallaaaaaa/a
License: No explicit license is specified on the repository. However, the author provides access to the dataset and indicates that it can be downloaded for research purposes. We used the dataset solely for non-commercial academic research and did not redistribute or modify it.
Qwen2.5-14B 36)
Language model for LLM representations extraction.
URR: https: bottps:/huuginggeee
wwwpsree/wwen2.--11111111 14B
License: Apache license 2.0
All datasets and model were used solely for research purposes. We did not modify or redistribute any datasets beyond standard preprocessing steps for experimental use. All license terms were respected.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims the abstract and introduction accurately reflect the paper's contribu- tions. The paper introduces feature-specific disentangling method to isolate reasoning representations in LLMs, applies this to brain encoding with ECoG data, and shows that reasoning-specifie embeddings capture unique variance in neural responses and enable more interpretable alignment with high-level brain activity. These claims are well-supported by the methodology and results.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper explicitly discusses its limitations Section F It acknowledges constraints in the spatial coverage of the ECoG dataset, particularly the lack of frontal region recordings, and suggests extending the work to other neural modalities like fMRL. It also notes the limited scope of reasoning types represented in current probing datasets (c.g.. COMPS), and proposes the use more diverse reasoning tasks in future research.
Justification:
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The theoretical result in Section 4.1 (Mutual Independence Theorem) provides complete derivation under clearly stated assumptions, including progressive layer-wise emergence of linguistic features and linear projection structure. The argument leverages bi- lincarity of covariance to justify approximate orthogonality of residual embeddings. Though approximate, the result is mathematically sound and is further supported by empirical validation in Sections and 4
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: The paper's Section 2.1 (Methods) provides sufficient detail to reproduce the main experimental results that support the paper's claims. It includes descriptions of probing datasets, procedures for identifying saturation layers, residual embedding construction, brain encoding model setup (including regression type, cross-vall crossvvalidation, and alignment procedure), and evaluation metrics. These details are enough to replicate the core pipeline even without access to the original code data.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets used in this study are publicly available. For linguistic probing, we use BLiMP, COMPS-BAR BASE, COMPS-WUGS, and COMPS-WUGS -DST [40] [3]
For brain encoding analysis, transcripts for one episode, and an expanded set of podcast transcripts introduced in 43 Code will be organized and released public platform upon publication.
use the Podcast ECoG dataset, which includes aligned
6. Experimental Setting/Details
21

---

Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
The experimental setup is described in sufficient detail within the main body of the paper, providing the context needed interpret and understand the results. Core elements such as probing setup, regression model, number of bootstrap resamples, and evaluation metries are included in Section Methodss. While not all hyperparameters and implementation details are included in the main paper, we provide the full information in code ensure reproducibility.
Justification:
Guidelines:
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
The paper reports appropriate measures of variability for both probing and brain encoding results. For instance, in Figure 4 encoding correlations over time are accompanied by 95% bootstrap confidence intervals, capturing uncertainty in the temporal alignment between model embeddings and neural signals.
Justification:
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Information about computational resources used for each major experi- ment innt-incl model inference, probing, and brain encoding is provided in Appendix G This includes details on hardware, memory usage, and runtime estimates to support repro- ducibility.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp///eerrrpp.cc///aal///lc/a
Answer: [Yes]
The research presented in this paper fully conforms to the NeurlPS Code of Ethics. All datasets used are publicly available and de-identified, no personally identifi- able information is involved, and the experiments were conducted with consideration for reproducibility, transparency, and scientific integrity.
Justification:
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: The paper does not explicitly discuss societal impacts. This work is primarily foundational and focuses interpretability and alignment between large language models and brain activity. While there are possible downstream societal implications-bote positive (e.g., better understanding of AI and cognition) and negative (e.g., misuse of reasoning representations) -iriiionnsssstsssss these considerations are outside the scope of the current paper and are not addressed directly.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
22

---

Justification: This paper does not release any models or datasets that pose high risk for misuse. All datasets used are publicly available, non-sensitive, and de-identified, and no new pretrained models or data with dual-use concerns are introduced.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets used in this paper-such the BLiMP, COMPS, and Podcast ECoG datasets-ant are properly cited with references to their original publications. Where available, have included dataset URLs and respected the licenses and terms of use. No proprictary or scraped content was used, and we did not modify or redistribute any datasets beyond standard preprocessing for reproducibility. License details are included in Appendix Section
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces a new codebase for residual embedding construction and brain encoding analysis. While the full code will be released publicly upon publication, commit to providing clear documentation alongside it, including environment requirements, usage examples, and configuration details necessary reproduce all key experiments.
Justification:
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve any new crowdsourcing or direct research with human subjects. We use the publicly released Podcast HCoG dataset, which was collected and shared by the original authors in accordance with institutional ethics approvals and participant consent, documented in their publication [43]. All data used are de-identified and comply with the terms use specified by the dataset providers.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve any new research with human subjects. We use the publicly released Podcast ECoG dataset, which was collected by the original authors under appropriate Institutional Review Board (IRB) approval and participant consent, as documented in their publication [43]. No new data collection or interaction with human participants was conducted in this work.
23


Reviews:
[{"review_id": "8uxcdzx9UQ", "reviewer": "Reviewer_vX2N", "summary": "This paper proposes a residual‐disentanglement framework that isolates orthogonal LLM embeddings for lexicon, syntax, meaning, and reasoning, and uses the reasoning‐specific residuals to build encoding models that predict ECoG responses during natural speech.", "questions": "see weakness", "limitations": "see weakness", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe idea is easy to follow and the paper is well written. The method clearly separates four linguistic feature embeddings and validates their independence using cosine‐similarity analysis.\nIt shows that the reasoning embedding captures unique variance in neural responses, peaks later, and engages higher‐level cortical areas.\nIt provides both a mathematical proof of approximate orthogonality and empirical evidence that the residual embeddings are independent.\n\nWeaknesses:\nThe experiments use only ECoG data, which undersamples frontal regions and may limit spatial generalizability.\nThe reasoning probes rely only on COMPS language tasks without widely used advanced reasoning tasks like math and coding.\nThe study focuses only on Qwen2.5-14B, larger or differently trained models (reasoning models like r1 and qwq) may show different layering or reasoning features.", "was_revised": true, "final_justification": "Good for nips, I maintain acceptance.", "rebuttal": "## Response to Reviewer vX2N\nFirst of all, we sincerely thank you for reviewing our paper and providing constructive comments! The followings specifically respond to key concerns you raised in your review.\n\n**Concern 1: Reasoning probes rely only on COMPS**\n\n**Summary of Response:** While our original analysis relied solely on COMPS-WUGS-DIST for identifying reasoning saturation layers, we have now added 5-hop deductive reasoning probes from ProntoQA to strengthen the evaluation. We applied both tasks to 17 Qwen models and found that the saturation layers they identify are closely aligned, with an average difference of only 0.94. This cross-validation suggests that our reasoning probe is robust across different task formats.\n\n**Full Response:** Thank you for highlighting this important point. In the original submission, we relied solely on COMPS-WUGS-DIST—a 1-hop deductive reasoning task—to identify the reasoning saturation layer, defined as the point at which reasoning performance plateaus within the model. While this task is useful for detecting the presence of reasoning capability, we acknowledge that its simplicity may limit generalizability to more complex forms of reasoning.\n\nMotivated by your feedback, we have incorporated a more challenging 5-hop deductive reasoning task from ProntoQA as an additional probe [1]. This task, which involves multi-step reasoning chains, has been widely adopted in recent reasoning trainings and benchmarks [2, 3, 4, 5], and we believe it serves as a meaningful enhancement to our evaluation framework.\n\nWe applied both COMPS-WUGS-DIST and ProntoQA to 17 Qwen models spanning different sizes and versions. The results from the two tasks are highly consistent, with an average difference in reasoning saturation layer of only 0.94—small relative to the overall model depths (ranging from 25 to 49 layers). This consistency supports the robustness of our reasoning probe across tasks of varying complexity.\n\n| Model                        | ProntoQA | COMPS-WUGS-DIST |\n|------------------------------|----------|-----------------|\n| Qwen-1.8B                   | 13       | 14              |\n| Qwen-7B                     | 13       | 15              |\n| Qwen-14B                    | 18       | 20              |\n| Qwen1.5-1.8B                 | 13       | 14              |\n| Qwen1.5-7B                   | 14       | 16              |\n| Qwen1.5-14B                  | 20       | 20              |\n| Qwen2-1.5B                   | 17       | 17              |\n| Qwen2-7B                     | 16       | 18              |\n| Qwen2.5-1.5B                 | 16       | 18              |\n| Qwen2.5-7B                   | 16       | 19              |\n| Qwen2.5-14B                  | 28       | 28              |\n| Qwen3-1.7B-thinking-mode-off | 19       | 19              |\n| Qwen3-8B-thinking-mode-off   | 23       | 23              |\n| Qwen3-14B-thinking-mode-off  | 27       | 27              |\n| Qwen3-1.7B-thinking-mode-on  | 16       | 16              |\n| Qwen3-8B-thinking-mode-on    | 20       | 20              |\n| Qwen3-14B-thinking-mode-on   | 20       | 21              |\n\n**Concern 2: Only Qwen2.5-14B**\n\n**Summary of Response:** We have extended our probing analysis to a wide range of Qwen models beyond Qwen2.5-14B, revealing that while the precise saturation layers vary, all models consistently follow the same emergence order of syntax → meaning → reasoning. We also introduce a notion of relative depth to quantify how much of the model is dedicated to each feature. Our results show a clear trend across model generations: newer models allocate fewer layers to syntax and more to meaning and reasoning, suggesting a shift toward deeper processing of high-level features as model capabilities improve.\n\n**Full Response:** Thank you for raising this important question. You are absolutely right that probing across multiple models is essential to understanding whether feature emergence patterns generalize. Models with different architectures or training corpora may indeed exhibit different internal dynamics. In response, we applied our probing pipeline to a broad range of Qwen models. The results are summarized in the following table. While the precise saturation layers vary, we observe a consistent emergence order across all models—syntax → meaning → reasoning—with Qwen-1.8B being the only exception, where syntax and meaning saturate at the same layer.\n\n| Model                        | Syntax | Meaning | Reasoning |\n|------------------------------|--------|---------|-----------|\n| Qwen-1.8B                    | 11     | 11      | 14        |\n| Qwen-7B                      | 11     | 13      | 15        |\n| Qwen-14B                     | 9      | 16      | 20        |\n| Qwen1.5-1.8B                 | 10     | 11      | 14        |\n| Qwen1.5-7B                   | 9      | 13      | 16        |\n| Qwen1.5-14B                  | 8      | 16      | 20        |\n| Qwen2-1.5B                   | 10     | 14      | 17        |\n| Qwen2-7B                     | 7      | 14      | 18        |\n| Qwen2.5-1.5B                 | 7      | 14      | 18        |\n| Qwen2.5-7B                   | 7      | 15      | 19        |\n| Qwen2.5-14B                  | 6      | 20      | 28        |\n| Qwen3-1.7B-thinking-mode-off | 9      | 16      | 19        |\n| Qwen3-8B-thinking-mode-off   | 7      | 20      | 23        |\n| Qwen3-14B-thinking-mode-off  | 5      | 17      | 27        |\n| Qwen3-1.7B-thinking-mode-on  | 5      | 13      | 16        |\n| Qwen3-8B-thinking-mode-on    | 6      | 13      | 20        |\n| Qwen3-14B-thinking-mode-on   | 4      | 14      | 21        |\n\nFurther, we analyzed broader trends emerging from these results. To do so, we introduce the notion of relative depth, which quantifies the proportion of layers dedicated to encoding each feature. Specifically, we define the relative depth of a feature as the fraction of total layers between the saturation layer of the preceding feature and that of the current one. For instance, in Qwen2.5-14B (which has 48 layers), meaning saturates at layer 20 and reasoning at layer 28, implying that layers 21–28 are primarily devoted to reasoning. Thus, the reasoning relative depth is computed as $(28 - 20) / 48$. More formally, for any feature $x \\in \\{s, m, r\\}$ representing syntax, meaning, or reasoning, we define:\n$$\nDepth_x = \\frac{L_x - L_{prev}}{n}\n$$\nwhere $L_x$ is the saturation layer of feature $x$, $L_{prev}$ is the saturation layer of the preceding feature, and $n$ is the total number of layers in the model. The table below reports these values for several 14B-scale models from the Qwen family.\n\n| Model                       | Syntax Depth | Meaning Depth | Reasoning Depth |\n|-----------------------------|--------------|---------------|-----------------|\n| Qwen-14B                    | 0.225        | 0.175         | 0.1             |\n| Qwen1.5-14B                 | 0.2          | 0.2           | 0.1             |\n| Qwen2.5-14B                 | 0.125        | 0.291         | 0.167           |\n| Qwen3-14B-thinking-mode-off | 0.125        | 0.3           | 0.25            |\n| Qwen3-14B-thinking-mode-on  | 0.1          | 0.25          | 0.175           |\n\nWe observe a clear trend: as models evolve and likely improve in capability, they dedicate a smaller fraction of layers to encoding syntax, while allocating more capacity to meaning and reasoning. This suggests that newer models may encode low-level linguistic features more efficiently, allowing them to devote more layers to higher-level semantic and reasoning processes—potentially contributing to improved performance on complex tasks.\n\n**Concern 3: Only ECoG data**\n\nWe appreciate your feedback. Indeed, assessing how the residual reasoning embedding aligns with brain signals beyond ECoG—such as fMRI or EEG—would be highly valuable, as these modalities offer complementary spatial coverage and signal characteristics. Among the available options, we selected the dataset that best matches our reasoning-focused research question while providing a good balance of spatial and temporal precision.\n\nOur experimental results show that reasoning differs from semantic and syntactic processing within a time window of about 100 ms—a timescale that is difficult to capture with fMRI. Regarding your point on limited frontal coverage, we note that while we have relatively good coverage over the inferior frontal gyrus (IFG), we do lack coverage in the prefrontal cortex, which is traditionally associated with higher-order cognition. This limitation arises because, in clinical settings, electrode placement for epilepsy surgery typically prioritizes coverage over the temporal lobe, where seizure foci are more common.\n\n\n**_We have updated our local draft accordingly and will incorporate these changes into the camera-ready version._**\n\n\n**References**\n\n[1] Abulhair Saparov and He He. Language Models are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought. In International Conference on Learning Representations (ICLR), 2023.\n\n[2] Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with Language Model is Planning with World Model. arXiv preprint arXiv:2305.14992, 2023.\n\n[3] Pan, L., Albalak, A., Wang, X., & Wang, W. Y. (2023). LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning.\n\n[4] Tian, X., Ji, Y., Wang, H., Chen, S., Zhao, S., Peng, Y., Zhao, H., & Li, X. (2025). Not All Correct Answers Are Equal: Why Your Distillation Source Matters. arXiv preprint arXiv:2505.14464v2.\n\n[5] Xu, J., Fei, H., Pan, L., Liu, Q., Lee, M.-L., & Hsu, W. (2024). Faithful Logical Reasoning via Symbolic Chain-of-Thought. arXiv preprint arXiv:2405.18357v2."}, {"review_id": "ItPxGjCp9v", "reviewer": "Reviewer_d6as", "summary": "1. The authors discover LLM layers that maximize linear classification performance on sentence datasets meant to test \"syntax\", \"meaning\", or \"reasoning\".\n\n2. They then extract syntax, meaning, and reasoning word-embeddings by computing inter-layer regression residuals based on a transcribed version of a listening dataset, and empirically show that these embeddings are nearly orthogonal.\n\n3. Finally, they perform regression analyses of recorded brain activity using the extracted embeddings as inputs.\n\n4. They find that (i) lexical / syntax embeddings show higher correlation with brain activity compared to meaning / reasoning embeddings, (ii) reasoning embeddings show peak correlation later in time than the other embeddings, and (iii) reasoning embeddings are correlated with different brain regions compared to lexical / syntax embeddings.", "questions": "1. What happens if one uses the reasoning embeddings $E_r$ to classify sentences in the datasets $D_s$, $D_m$, and $D_r$? If the residuals truly produce disentangled representations, I would expect performance on $D_s$ or $D_m$ to be worse but on $D_r$ to be relatively unaffected, compared to using the non-isolated embeddings $H_r$ as inputs.\n\n2. In Figure 3a, why is the correlation for \"Full\" worse than for \"Lexicon\"?\n\n3. As a qualitative result, could the authors show a few examples from the podcast where a listener would need to reason/visualize to a higher degree? In such cases, do the reasoning embeddings predict this brain activity?", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The \"probing\" method to locate the earliest LLM layers specializing at different skills followed by regressing out one layer’s activations from another to disentangle embeddings is an interesting idea and a novel contribution to my knowledge.\n2. The paper could inspire future work on gaining a deeper understanding of the similarities/differences between the reasoning processes in LLMs and the human brain.\n\nWeaknesses:\n1. No other LLMs used:\\\n**a)** The abstract claims that the reasoning embedding has unique predictive value \"only because the latest LLMs exhibit emergent reasoning behavior\". In that case, it is crucial to repeat the analysis with an older LLM (that lacks as much reasoning ability) to understand if the brain patterns revealed by the \"reasoning embeddings\" are simply because deeper layers contain higher-level features or because the latest LLMs are better at reasoning (and that the improved reasoning is important for the regression analysis).\\\n**b)** Section 2.2 mentions using a Qwen base model without any further alignment. Given the emphasis on \"reasoning\" throughout the paper, I would expect experiments using at least one open-source model from the recent wave of \"reasoning LLMs\" (i.e., those fine-tuned with supervised learning / RL to solve hard problems). I would also expect these results to be compared to the corresponding non-reasoning LLM (the base model). Without these, given recent developments in the field, the paper’s claims may mislead readers in 2025 and beyond.\n\n2. While Section 3.2 empirically shows that the four embeddings are roughly mutually orthogonal, I do not think the math in Section 3.1 explains this result. In a linear regression with $n$ samples, $d$ input features, and scalar targets, the $n$-D vector of residuals will be orthogonal to all (of the $d$) $n$-D feature vectors. I.e., orthogonality is expected along the sample dimension, not the feature dimension (which doesn’t even make sense if $n \\neq d$). However, Section 3.2 shows orthogonality along the feature dimension.\n\n3. Lines 205-206 imply that syntax and meaning embeddings account for the majority of the explained variance in the brain activity. However, lines 208-209, 213, and 246-247 state that low-level linguistic (syntactic, lexical) features have the highest predictive power. These statements seem inconsistent and the intended message is unclear.\n\n4. Insufficient grounding in neuroscience:\\\n**a)** Section 4.2 shows that reasoning embeddings show peak correlation with brain activity later in time and in different brain regions compared to the other embeddings. But what is the scientific significance of this? Without citations to the neuroscience literature on the spatiotemporal patterns involved in human reasoning, it is not clear how to make sense of the regression result.\\\n**b)** Lines 226-228 describe the progression of high-correlation regions with time. How consistent is this with what we expect from the human brain? If this is significant, the authors should back it up with relevant citations.\\\n**c)** Line 230 states the reasoning embeddings allow for \"isolation of reasoning-specific neural signals\". How does the reader know that the regions detected are indeed reasoning-specific? Without references to the neuroscience literature, this sounds like the circular statement 'any neurons that happen to correlate with our reasoning embeddings are reasoning-specific'.\\\n**d)** Other missing citations: (i) lines 235-236 (and 254-255) linking \"anterior bias\" with \"higher-order\" cognition, (ii) lines 237-238 linking \"language areas\" with the SFG, (iii) lines 252-253 linking \"abstraction and salience\" with the SFG and insula, and (iv) line 264 linking \"delayed temporal peak\" with \"integrative reasoning processes\".\\\n**e)** Based on the above, I think the claim in the abstract of providing \"the first brain-relevant representation of reasoning\" is highly misleading.\n\n5. The text in Figure 5 is unreadable.\n\n6. Minor:\\\n**a)** In line 95, $H$ should have dimensions $d\\times n$ (not $n\\times d$) since the equation below line 113 computes $W\\cdot H$, where I presume $W$ has dimensions $d\\times d$. Similarly, in line 108, $E$ should have dimensions $d\\times n$ (not $d$).\\\n**b)** $\\text{Cov}(A, B)$ below line 185 is not defined, does it mean $A\\cdot B^T$?\\\n**c)** Regression residuals are not a new idea in brain encoding studies. I would recommend the authors cite some prior work on this (e.g., https://aclanthology.org/2021.cmcl-1.1.pdf) so the reader has more context.\\\n**d)** It is not clear what lines 280-285 mean and they do not seem to be discussed previously.\\\n**e)** \"feature\" => \"features\" (line 6).\\\n**f)** The figure cross-reference is missing in line 178.\\\n**g)** Figure 5 => Figure 3 (lines 210 and 214).", "was_revised": true, "final_justification": "I like the idea of this paper. But I have concerns with some central claims that may remain unsubstantiated in the final version. These claims are:\n1. That the extracted embeddings identify human reasoning brain patterns only because latest LLMs have improved reasoning capability. To prove this, the authors need to properly benchmark the LLM suite on their ability to isolate reasoning-specific brain regions/signals and show that only the newer LLMs succeed.\n2. That the extracted reasoning embeddings actually identify reasoning-specific signals in the human brain, providing the first brain-relevant representation of reasoning. To prove this, the authors need to show that (i) the regions/signals detected by the reasoning embeddings indeed take center stage in the human reasoning process, and (ii) the other embeddings struggle to identify these same regions.\n\nBoth the above points are missing from the rebuttal, so I think accepting this paper runs the risk of insufficient evidence to support them. None of the other reviewers have engaged with me to disagree. I will therefore maintain 3 (Borderline reject).", "rebuttal": "Thank you very much for your thoughtful and constructive feedback! We highly appreciate your careful review and the insightful comments, which have helped us clarify and strengthen our work. Below, we respond to a few of your main concerns in detail. Due to the length limit, we may not be able to address all questions raised. But please reach out during discussion period for unaddressed questions!\n\n**Concern: No other LLMs used**\n\nWe agree that testing a broader range of models—including earlier-generation LLMs and recent reasoning-aligned variants—is important for understanding what drives the predictive value of reasoning embeddings. To address this, we expanded our probing analysis to Qwen models from 1.8B to 14B parameters, covering multiple architecture revisions. Despite differences in exact saturation layers, all models—except Qwen-1.8B—consistently exhibit the emergence order of syntax → meaning → reasoning.\n\nCritically, we observe that newer and larger models, especially the Qwen3 series, allocate substantially greater relative depth to reasoning. For example, Qwen2.5-14B dedicates 16.7% of its layers to reasoning, while Qwen3-14B (thinking-mode-off) allocates 25%, more than doubling the share compared to earlier versions like Qwen-14B and Qwen1.5-14B (both at 10%). This trend suggests that later models not only preserve the hierarchical emergence but deepen the network's capacity for high-level reasoning features.\n\n| Model | Syntax Depth | Meaning Depth | Reasoning Depth |\n|-|-|-|-|\n| Qwen-14B | 0.225 | 0.175 | 0.1 |\n| Qwen1.5-14B | 0.2 | 0.2 | 0.1 |\n| Qwen2.5-14B | 0.125        | 0.291         | 0.167           |\n| Qwen3-14B-thinking-mode-off | 0.125        | 0.3           | 0.25            |\n| Qwen3-14B-thinking-mode-on  | 0.1          | 0.25          | 0.175           |\n\nWe also highlight that the Qwen3 series—particularly Qwen3-14B with \"thinking mode\"—is explicitly optimized for multi-step reasoning and can be seen as analogous to recent instruction- and RLHF-aligned models. Including Qwen3-14B helps bridge the gap between base models and reasoning-aligned systems.\n\n**Question:** What happens if one uses the reasoning embeddings to classify sentences in the probing datasets.\n\n**Answer:** Theoretically, each embedding should perform well only on its corresponding task—for example, reasoning embeddings should solve the reasoning task, but not syntax or meaning. Using the residual embeddings to classify the probing sentences would yield a 3×3 probing matrix with strong diagonal and weaker off-diagonal values, indicating feature specificity.\n\nAs a baseline, we first show model performance at each feature’s saturation layer, normalized by the task's peak performance achieved by the model. The high values across all layers illustrate the need for residualization to isolate distinct signals.\n\n|                | Syntax Layer | Meaning Layer | Reasoning Layer |\n|----------------|--------------|---------------|-----------------|\n| Syntax Task    | 0.947        | 0.999         | 0.989           |\n| Meaning Task   | 0.812        | 0.995         | 0.995           |\n| Reasoning Task | 0.617        | 0.685         | 0.988           |\n\nAfter applying our residual feature extraction, we observed the following normalized performance matrix:\n\n|                | Syntax Embedding | Meaning Embedding | Reasoning Embedding |\n|----------------|------------------|-------------------|---------------------|\n| Syntax Task    | 0.882            | 0.664             | 0.563               |\n| Meaning Task   | 0.781            | 0.919             | 0.802               |\n| Reasoning Task | 0.648            | 0.770             | 1.036               |\n\nThese results reveal a clear pattern: each embedding performs best on its corresponding task and worse on others, indicating that the residuals are meaningfully disentangled rather than reflecting general complexity. This supports the interpretability and specificity of our residual reasoning representation.\n\n**Concern: Variance Partioning (weekness 3)**\n3. Thank you for pointing this out. We apologize for the lack of clarity in the original manuscript. The apparent inconsistency arises from a key difference in feature dimensionality across analyses. Specifically, during variance partitioning (Lines 165–166), each feature type was reduced to 500/N dimensions (with N=4) to keep the total input dimensionality fixed at 500. In contrast, in the standalone encoding analyses, each feature retained the full 500 dimensions. This trade-off was necessary: while 500 dimensions offer robust encoding performance, using 500 dimensions per feature in variance partitioning would result in a total of 2000 dimensions, which is impractical for neural encoding due to overfitting risks.\nFor the lexicon feature, the discrepancy is further amplified because it is highly correlated with the word rate confound, which we always include and regress out in all analyses (Lines 143–152). After reducing the 500-dimensional zeroth-layer LLM embeddings to 125 dimensions for lexicon, much of the variance beyond word rate is lost, making lexicon appear less predictive in the variance partitioning setting.\nOverall, despite the dimensionality adjustment, the results from variance partitioning remain consistent with those from the encoding analysis for syntactic, semantic, and reasoning features. They collectively suggest that higher-level features contribute less uniquely explained variance and are more easily masked by low-level information, reinforcing the motivation for our disentanglement framework.\n\n**Concern: Neuroscience Grounding (weekness 4)**\na) The reasoning embedding (Er) shows peak correlation with ECoG around 300–350 ms post-stimulus, later than lexical, syntactic, or semantic embeddings. This aligns with the P3b component (250–500 ms), linked to context updating and evaluative decision-making [1][9]. fMRI meta-analyses further implicate a left-lateralized frontoparietal network—particularly dlPFC and IPL—in reasoning [2], suggesting Er captures high-level integrative processes beyond early linguistic stages.\nb) The observed spatiotemporal cascade—from early STG (<200 ms) to mid-latency MTG/IFG and late PFC—matches the P200 (~200 ms), which reflects lexical-semantic recognition [6][8]. This temporal trajectory is consistent with the MUC model of sentence processing [3] and meta-analytic evidence of frontoparietal recruitment during reasoning [4].\nc) To isolate reasoning-specific signals, we regress out variance from lower-level embeddings, yielding an orthogonal Er. This mirrors cognitive subtraction in neuroimaging [5] and ensures that Er’s neural alignment is not confounded by earlier processing stages.\nd) (i) The anterior STG supports high-level language integration [11]. (ii) The frontal aslant tract links Broca’s area with SFG, involved in syntactic planning [12]. (iii) The anterior insula and SFG, key nodes of the salience network, mediate attention and control [13]. (iv) P3b timing aligns with Er’s peak, reinforcing its cognitive role [1].\ne) While prior work aligns whole LLM embeddings with brain data, we uniquely apply residualization to extract reasoning-specific signals and validate their distinct spatiotemporal neural correlates.\n\n\n**Concern: why is the correlation for \"Full\" < \"Lexicon\"?**\nThis is because we took the union of all electrodes activated by any feature, which means some electrodes strongly activated by the lexicon may not show activation in the full-feature set. As layers increase, the LLM’s hidden states abstract away layer‑0 features, so early lexicon-specific activations may no longer be prominent in deeper layer. \n\n**_We have updated our local draft accordingly and will incorporate these changes into the camera-ready version. We hope these clarifications address your concerns, and we would be grateful if you would consider revisiting your rating._**\n\nReferences\nPolich J. Updating P300: an integrative theory of P3a and P3b. Clin Neurophysiol. 2007;118(10):2128–2148.\n\nPrado J, Chadha A, Booth JR. The brain network for deductive reasoning: meta-analysis of 28 neuroimaging studies. J Cogn Neurosci. 2011;23(11):3483–3497.\n\nHagoort P. The memory, unification, and control (MUC) model of language. In: Meyer AS, Wheeldon L, Krott A, eds. Automaticity and Control in Language Processing. Psychology Press; 2007:243–270.\n\nWang L, Zhu X, Yin W. Deductive-reasoning brain networks: coordinate-based meta-analysis. Hum Brain Mapp. 2020;41(18):5017–5034.\n\nFriston KJ, Frith CD, Liddle PF, Frackowiak RSJ. The trouble with cognitive subtraction. Neuroimage. 1996;4(2):97–104.\n\nFedermeier KD, Kutas M. Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. J Exp Psychol Learn Mem Cogn. 2002;28(4):879–891.\n\nLuck SJ, Hillyard SA. Electrophysiological correlates of feature analysis during visual search. Psychophysiology. 1994;31(3):291–308.\n\nHolcomb PJ, Grainger J. On the time course of visual word recognition: an event-related potential investigation. Lang Cogn Process. 2006;21(4):401–439.\n\nComerchero MD, Polich J. P3a and P3b from typical auditory and visual stimuli. Clin Neurophysiol. 1999;110(1):24–44.\n\nKleih SC, Nijboer F, Halder S, Kübler A. P3 amplitude and latency in task-switching paradigms: dissociable components of cognitive control? Brain Res. 2010;1335:142–150.\n\nMellem, Monika S., et al. \"Sentence processing in anterior superior temporal cortex shows a social-emotional bias.\" Neuropsychologia 89 (2016): 217-224.\n\nCatani M, Thiebaut de Schotten M. The frontal aslant tract and supplementary motor area connectivity. Brain. 2012;135(1):198–211.\n\nMenon V, Uddin LQ. Saliency, switching, attention and control: a network model of insula function. Brain Struct Funct. 2010;214(5-6):655–667.\n\nLuck SJ, Hillyard SA. Electrophysiological correlates of semantic processing: the P200. Front Psychol. 2021;12:746813."}, {"review_id": "G1ytrPDo1l", "reviewer": "Reviewer_q2nc", "summary": "This paper introduces a residual disentanglement method to extract four distinct linguistic feature embeddings (lexicon, syntax, meaning, reasoning) from LLMs by identifying saturation layers through probing and iteratively removing lower-level features. The authors apply these embeddings to predict ECoG brain activity during podcast listening, finding that reasoning embeddings show unique spatiotemporal patterns with later temporal peaks and engagement of frontal/visual regions beyond classical language areas.", "questions": "1. How sensitive are the saturation layer identifications to the threshold ε? Have you tested different values and their impact on the final embeddings?\n2. Can you demonstrate that this disentanglement approach works across different model architectures and sizes? The single-model evaluation is a significant limitation.\n3. Beyond property inheritance, how does the method perform on more complex reasoning tasks like multi-step logical reasoning, causal inference, or mathematical reasoning?\n4. The orthogonality proof assumes perfect hierarchical emergence, but what happens when this assumption is violated? How robust is the method to non-hierarchical feature organization?\n5. Can you validate the brain findings using complementary neuroimaging modalities with better spatial coverage (e.g., fMRI) to address the ECoG limitations?", "limitations": "The authors adequately acknowledge ECoG spatial limitations and single-model constraints. However, they could better address the fundamental assumption about hierarchical feature emergence and the limited scope of reasoning evaluation.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The residual disentanglement approach is novel and addresses an important limitation in current brain-LLM alignment studies that treat models as monolithic. The finding that reasoning emerges later temporally (~330ms) and engages different brain regions is compelling and aligns with cognitive theories.\n2. The mathematical framework with the mutual independence theorem provides theoretical grounding, and the empirical validation showing near-zero cosine similarities between residual embeddings supports the disentanglement claim. The results showing shallow features dominate neural encoding are important.\n\nWeaknesses:\n1. The orthogonality assumption is problematic. While low cosine similarities suggest reduced overlap, this doesn't guarantee true independence. The theoretical proof relies on strong assumptions about hierarchical feature emergence that may not hold universally across models or linguistic phenomena.\n2. The evaluation is severely limited by using only one model (Qwen2.5-14B) and narrow reasoning tasks (property inheritance only). This restricts generalizability claims about LLM reasoning representations. The ECoG spatial coverage limitations, particularly in frontal regions crucial for reasoning, weaken the brain-based conclusions.\n3. The layer saturation identification could be sensitive to the threshold, and the assumption that ridge regression creates orthogonal features from hierarchical representations needs stronger justification. The reasoning tasks tested are quite simple and may not capture the complexity of reasoning that modern LLMs can perform.", "was_revised": true, "final_justification": "While some of my questions remain only partially addressed, I believe this paper offers valuable contributions and is suitable for acceptance at NeurIPS. Please ensure that the additional results are included in the revised version of the paper. I am raising my score from 4 to 5.", "rebuttal": "## Response to Reviewer q2nc\n\nThank you for your thoughtful and constructive feedback. We appreciate your careful review and the insightful comments, which have helped us clarify and strengthen our work. Below, we respond to each of your questions in detail.\n\n**Question:** Can you demonstrate that this disentanglement approach works across different model architectures and sizes?\n\n**Response** We expanded our probing analysis to models from 1.8B to 14B parameters across different Qwen collections. Despite differences in exact saturation layers, we consistently observe the emergence order: syntax → meaning → reasoning. Qwen-1.8B is the only exception, where syntax and meaning saturate at the same layer. While we have not yet tested the 32B and 72B model due to resource limits, the observed trends across scales already support the robustness of our approach.\n\n| Model                        | Syntax | Meaning | Reasoning |\n|------------------------------|--------|---------|-----------|\n| Qwen-1.8B                    | 11     | 11      | 14        |\n| Qwen-7B                      | 11     | 13      | 15        |\n| Qwen-14B                     | 9      | 16      | 20        |\n| Qwen1.5-1.8B                 | 10     | 11      | 14        |\n| Qwen1.5-7B                   | 9      | 13      | 16        |\n| Qwen1.5-14B                  | 8      | 16      | 20        |\n| Qwen2-1.5B                   | 10     | 14      | 17        |\n| Qwen2-7B                     | 7      | 14      | 18        |\n| Qwen2.5-1.5B                 | 7      | 14      | 18        |\n| Qwen2.5-7B                   | 7      | 15      | 19        |\n| Qwen2.5-14B                  | 6      | 20      | 28        |\n| Qwen3-1.7B-thinking-mode-off | 9      | 16      | 19        |\n| Qwen3-8B-thinking-mode-off   | 7      | 20      | 23        |\n| Qwen3-14B-thinking-mode-off  | 5      | 17      | 27        |\n| Qwen3-1.7B-thinking-mode-on  | 5      | 13      | 16        |\n| Qwen3-8B-thinking-mode-on    | 6      | 13      | 20        |\n| Qwen3-14B-thinking-mode-on   | 4      | 14      | 21        |\n\n**Question:** How does the method perform on more complex reasoning tasks like multi-step logical reasoning?\n\n**Response:** Our original analysis used COMPS-WUGS-DIST, a 1-hop reasoning task, to identify reasoning saturation layers. While effective at detecting the presence of reasoning, its simplicity may limit generalizability.\n\nMotivated by your feedback, we added ProntoQA—a 5-hop deductive reasoning task widely used in recent benchmarks and trainings—as an additional probe. We applied both tasks to 17 Qwen models of various sizes and found highly consistent results: the reasoning saturation layers from the two tasks differed by only 0.94 on average, a small gap relative to model depths (25–49 layers). This cross-task agreement reinforces the validity of our reasoning probe across tasks of different complexity.\n\n| Model                        | ProntoQA | COMPS-WUGS-DIST |\n|------------------------------|----------|-----------------|\n| Qwen-1.8B                   | 13       | 14              |\n| Qwen-7B                     | 13       | 15              |\n| Qwen-14B                    | 18       | 20              |\n| Qwen1.5-1.8B                 | 13       | 14              |\n| Qwen1.5-7B                   | 14       | 16              |\n| Qwen1.5-14B                  | 20       | 20              |\n| Qwen2-1.5B                   | 17       | 17              |\n| Qwen2-7B                     | 16       | 18              |\n| Qwen2.5-1.5B                 | 16       | 18              |\n| Qwen2.5-7B                   | 16       | 19              |\n| Qwen2.5-14B                  | 28       | 28              |\n| Qwen3-1.7B-thinking-mode-off | 19       | 19              |\n| Qwen3-8B-thinking-mode-off   | 23       | 23              |\n| Qwen3-14B-thinking-mode-off  | 27       | 27              |\n| Qwen3-1.7B-thinking-mode-on  | 16       | 16              |\n| Qwen3-8B-thinking-mode-on    | 20       | 20              |\n| Qwen3-14B-thinking-mode-on   | 20       | 21              |\n\n**Question:** The orthogonality proof assumes perfect hierarchical emergence, but what happens when this assumption is violated?\n\n**Response:** We acknowledge that our residual disentanglement pipeline depends on heuristic feature hierarchy and being able to identify approximate saturation layers for each feature; if no clear saturation exists—e.g., due to overlapping or non-sequential emergence—the method may fail to isolate distinct residuals. However, our results across 17 Qwen models show that the feature hierarchy (syntax → meaning → reasoning) emerges consistently, supporting the pipeline’s applicability in practice. Moreover, prior work has shown that LLMs tend to encode progressively higher-level linguistic features across layers (e.g., Tenney et al.) [1], reinforcing the plausibility of our layered assumption.\n\n**Question:** How sensitive are the saturation layer identifications to the threshold ε?\n\n**Response:** We find that the identified saturation layers are relatively stable across a reasonable range of ε values, typically varying by only 1–2 layers. While we have not yet tested the impact of different ε values on downstream tasks due to time constraints, the overall feature emergence order remains consistent, suggesting that the method is not overly sensitive to the exact threshold. We plan to explore the effect on residual embedding quality in future work.\n\n**Question:** Can you validate the brain findings using fMRI?** \n\n**Response:** Our experimental results show that reasoning differs from semantic and syntactic processing within a time window of less than 100 ms—a timescale that is difficult to capture with fMRI. Nonetheless, it would be valuable to validate this finding with fMRI, given its superior spatial resolution. Unfortunately, this is not feasible within the rebuttal period due to the substantial time required to build a new encoding pipeline.\n\n**_We have updated our local draft accordingly and will incorporate these changes into the camera-ready version._**\n\n**References**\n\n[1] Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 4593–4601, 2019."}, {"review_id": "7UUI2CpQlv", "reviewer": "Reviewer_hFzy", "summary": "This paper presents a residual disentanglement method that identifies feature saturation layers through probing tasks, and using ridge regression to remove lower-level contributions from higher-level features. Results demonstrate that a \"reasoning\" embedding predicts a unique component of brain activation in human ECoG recordings (with later timing and engagement of frontal regions) that is obscured when using conventional embeddings.", "questions": "1. I would suggest testing across the Qwen model family (from smaller 1.8B model to 72B parameters) to provide convergent validity for the reasoning residuals. You should predict that smaller models with poorer reasoning performance should show weaker or absent \"reasoning\" residuals in the disentanglement pipeline, while larger models with demonstrated reasoning capabilities should yield stronger, more distinct reasoning embeddings, and the brain encoding performance of reasoning residuals should scale with the source model's actual reasoning abilities? If a 1.8B model's \"reasoning\" residuals predict brain activity just as well as a 72B model's, it would suggest they're capturing more generic complexity rather than genuine reasoning processes. This would also help test whether the layer localization methodology (finding reasoning \"saturation\" layers) generalizes across model scales, and whether the temporal dynamics (reasoning signals peaking ~330ms) hold only for models that actually demonstrate reasoning capabilities.\n\n2. The COMPS-WUGS-DIST task represents a somewhat impoverished view of \"reasoning\" on its own. Simple property inheritance (\"a wug is an oven -> a wug can heat food\") and variable binding captures only basic logical inference, not the multi-step, abstract, or causal \"situation model\" reasoning typically associated with higher-order cognition. The limits the interpretability of the \"reasoning\" residuals and their neural correlates. I'd suggest using a more diverse set of reasoning benchmarks (e.g., winogrande, multi-step arithmetic) to validate that what are being called \"reasoning\" residuals generalize across diverse reasoning tasks as opposed to being an artifact of this one task instantiation. \n\n3. I'd appreciate a stronger justification for why various assumptions of the residual disentanglement approach should hold empirically. For example, (a) the assumption that the syntax-> meaning -> reasoning hierarchy emerges in a strict sequence doesn't necessarily corresond to how modern transformers process information, (b) the (implicit) assumption that higher-level features are linear combinations of lower-level ones plus orthogonal residuals, (c) the assumption that features \"saturate\" at specific layers and remain stable thereafter (what looks like saturation might just be the limits of the probing task), and (d) while low cosine similarity between residuals tests orthogonality, this doesn't validate that the residuals capture the intended cognitive constructs -- just because they're orthogonal doesn't mean they correspond to cognitively meaningful categories, they might just be isolating arbitrary statistical patterns that happen to correlate with these specific reasoning tasks.\n\n4. I would consider directly testing whether the extracted \"reasoning\" residuals actually causally contribute to reasoning performance. For example, if you corrupt or ablate the reasoning residuals in the original model, does reasoning performance selectively decline while other linguistic capabilities remain intact? This would provide much stronger evidence that the residuals are partitioning out functionally relevant reasoning representations.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe technical execution is rigorous, employing solid ECoG data and a novel residual disentanglement pipeline that includes careful probing, validation of orthogonality, and encoding analyses. The use of a capable modern language model (Qwen-2.5-14B) as opposed to the smaller models used in a lot of other recent work enables the detection of emergent reasoning capabilities that would be impossible with smaller models.The results reveal intriguingly distinct spatiotemporal activation patterns that suggest meaningful differences between linguistic features. \n\nWeaknesses:\n\nI'm concerned that the paper's central claim about isolating \"reasoning\" representations is stronger than what the evidence supports. First, the operationalization of \"reasoning\" through a single, simple task probe (COMPS-WUGS-DIST) does not capture the kind of abstract, multi-step reasoning typically associated with higher-order cognition, limiting the interpretability of the purported reasoning residuals and the corresponding neural analyses. The exciting possibility that these residuals capture genuine reasoning processes requires substantially stronger evidence across diverse reasoning benchmarks and models (see below).", "was_revised": true, "final_justification": "After consideration of the authors' rebuttal, I am updating my score from 3 (Borderline reject) to 4 (Borderline accept).\n\nThe demonstration of convergent saturation layers with ProntoQA substantially strengthens the claim that the method identifies reasoning representations across different task types (not just COMPS). Additionally, extending the probing analysis across 17 Qwen models (1.8B-14B) provides stronger evidence for the generalizability of the syntax -> meaning -> reasoning hierarchy to models of different depths. \n\nThe following issues I raised still remain unresolved: (1) even with the inclusion of ProntoQA, the reasoning tasks still represent relatively narrow logico-deductive reasoning (vs physical or commonsense reasoning). So claims about isolating \"reasoning\" more broadly would benefit from validation on more diverse benchmarks. (2) two suggested experiments remain absent -- testing whether brain encoding performance scales with model reasoning performance capabilities, and causal ablation studies to establish the functional relevance of the residuals.", "rebuttal": "## Response to Reviewer 4 (hFzy)\n**Concern 1: Reasoning Probes Rely Only on COMPS**\n\n**Summary of Response:** To strengthen our evaluation, we supplemented COMPS-WUGS-DIST with a more complex 5-hop reasoning task from ProntoQA. Applied across 17 Qwen models, the two tasks yielded closely aligned reasoning saturation layers.\n\n**Full Response:** Thank you for raising this point. Our original analysis used COMPS-WUGS-DIST, a 1-hop reasoning task, to identify reasoning saturation layers. While effective at detecting the presence of reasoning, its simplicity may limit generalizability.\n\nMotivated by your feedback, we added ProntoQA—a 5-hop deductive reasoning task widely used in recent benchmarks and trainings—as an additional probe. We applied both tasks to 17 Qwen models of various sizes and found highly consistent results: the reasoning saturation layers from the two tasks differed by only 0.94 on average, a small gap relative to model depths (25–49 layers). This cross-task agreement reinforces the validity of our reasoning probe across tasks of different complexity.\n\n| Model                        | ProntoQA | COMPS-WUGS-DIST |\n|------------------------------|----------|-----------------|\n| Qwen-1.8B                   | 13       | 14              |\n| Qwen-7B                     | 13       | 15              |\n| Qwen-14B                    | 18       | 20              |\n| Qwen1.5-1.8B                 | 13       | 14              |\n| Qwen1.5-7B                   | 14       | 16              |\n| Qwen1.5-14B                  | 20       | 20              |\n| Qwen2-1.5B                   | 17       | 17              |\n| Qwen2-7B                     | 16       | 18              |\n| Qwen2.5-1.5B                 | 16       | 18              |\n| Qwen2.5-7B                   | 16       | 19              |\n| Qwen2.5-14B                  | 28       | 28              |\n| Qwen3-1.7B-thinking-mode-off | 19       | 19              |\n| Qwen3-8B-thinking-mode-off   | 23       | 23              |\n| Qwen3-14B-thinking-mode-off  | 27       | 27              |\n| Qwen3-1.7B-thinking-mode-on  | 16       | 16              |\n| Qwen3-8B-thinking-mode-on    | 20       | 20              |\n| Qwen3-14B-thinking-mode-on   | 20       | 21              |\n\n**Concern 2: Probes across Qwen Family**\n\n**Summary of Response:** We extended our probing analysis to Qwen models ranging from 1.8B to 14B parameters. While saturation layers vary, all models follow the same emergence order—syntax → meaning → reasoning. We also reveals a trend: newer models dedicate fewer layers to syntax and more to higher-level features, suggesting a shift toward deeper reasoning as models evolve.\n\n**Full Response:** Thank you for the thoughtful suggestion. We expanded our probing analysis to Qwen models from 1.8B to 14B parameters. Despite differences in exact saturation layers, we consistently observe the emergence order: syntax → meaning → reasoning. Qwen-1.8B is the only exception, where syntax and meaning saturate at the same layer. While we have not yet tested the 32B and 72B model due to resource limits, the observed trends across scales already support the robustness of our findings.\n\n| Model                        | Syntax | Meaning | Reasoning |\n|------------------------------|--------|---------|-----------|\n| Qwen-1.8B                    | 11     | 11      | 14        |\n| Qwen-7B                      | 11     | 13      | 15        |\n| Qwen-14B                     | 9      | 16      | 20        |\n| Qwen1.5-1.8B                 | 10     | 11      | 14        |\n| Qwen1.5-7B                   | 9      | 13      | 16        |\n| Qwen1.5-14B                  | 8      | 16      | 20        |\n| Qwen2-1.5B                   | 10     | 14      | 17        |\n| Qwen2-7B                     | 7      | 14      | 18        |\n| Qwen2.5-1.5B                 | 7      | 14      | 18        |\n| Qwen2.5-7B                   | 7      | 15      | 19        |\n| Qwen2.5-14B                  | 6      | 20      | 28        |\n| Qwen3-1.7B-thinking-mode-off | 9      | 16      | 19        |\n| Qwen3-8B-thinking-mode-off   | 7      | 20      | 23        |\n| Qwen3-14B-thinking-mode-off  | 5      | 17      | 27        |\n| Qwen3-1.7B-thinking-mode-on  | 5      | 13      | 16        |\n| Qwen3-8B-thinking-mode-on    | 6      | 13      | 20        |\n| Qwen3-14B-thinking-mode-on   | 4      | 14      | 21        |\n\nWe also examined broader trends by introducing relative depth, which measures the fraction of layers allocated to each feature. It is defined as the proportion of layers between the saturation point of the previous feature and that of the current one. For any feature $x \\in \\{s, m, r\\}$ representing syntax, meaning, or reasoning, we define:\n$$\nDepth_x = \\frac{L_x - L_{prev}}{n}\n$$\nwhere $L_x$ is the saturation layer of feature $x$, $L_{prev}$ is that of the preceding feature, and $n$ is the total number of layers. The table below reports these values for several 14B-scale Qwen models.\n\n| Model                       | Syntax Depth | Meaning Depth | Reasoning Depth |\n|-----------------------------|--------------|---------------|-----------------|\n| Qwen-14B                    | 0.225        | 0.175         | 0.1             |\n| Qwen1.5-14B                 | 0.2          | 0.2           | 0.1             |\n| Qwen2.5-14B                 | 0.125        | 0.291         | 0.167           |\n| Qwen3-14B-thinking-mode-off | 0.125        | 0.3           | 0.25            |\n| Qwen3-14B-thinking-mode-on  | 0.1          | 0.25          | 0.175           |\n\nWe observe a clear trend: newer models allocate a smaller fraction of layers to syntax and more to meaning and reasoning, suggesting more efficient encoding of low-level features and increased capacity for higher-level processing—potentially enhancing performance on complex tasks.\n\n**Concern 3: Whether Residuals Capture the Intended Cognitive Constructs**\n\n**Summary of Response:** To verify that our residual embeddings capture distinct cognitive features, we evaluated their ability to solve the probing tasks they were derived from. Each embedding (syntax, meaning, reasoning) was tested across all three tasks, forming a 3×3 matrix. The results show that each embedding performs best on its corresponding task and significantly worse on the others.\n\n**Full Response:** Thank you for raising this important point. We agree that it is critical to confirm whether the residual embeddings truly capture syntax, meaning, and reasoning—rather than general model activity.\n\nTo evaluate this, we applied our feature extraction pipeline not only to the podcast corpus (for brain encoding) but also to the question texts from our probing tasks. For each task, we extracted residual embeddings at their respective saturation layers, then trained ridge regression classifiers to predict task answers using each embedding type.\n\nIdeally, each embedding should perform well only on its corresponding task—for example, meaning embeddings should solve the meaning task, but not syntax or reasoning. This forms a 3×3 probing matrix with strong diagonal and weaker off-diagonal values, indicating feature specificity.\n\nAs a baseline, we first show model performance at each feature’s saturation layer, normalized by the task's peak performance. The high values across all layers illustrate the need for residualization to isolate distinct signals.\n\n|                | Syntax Layer | Meaning Layer | Reasoning Layer |\n|----------------|--------------|---------------|-----------------|\n| Syntax Task    | 0.947        | 0.999         | 0.989           |\n| Meaning Task   | 0.812        | 0.995         | 0.995           |\n| Reasoning Task | 0.617        | 0.685         | 0.988           |\n\nAfter applying our residual feature extraction, we observed the following normalized performance matrix:\n\n|                | Syntax Embedding | Meaning Embedding | Reasoning Embedding |\n|----------------|------------------|-------------------|---------------------|\n| Syntax Task    | 0.882            | 0.664             | 0.563               |\n| Meaning Task   | 0.781            | 0.919             | 0.802               |\n| Reasoning Task | 0.648            | 0.770             | 1.036               |\n\nThese results reveal a clear pattern: each embedding performs best on its corresponding task and worse on others, indicating that the residuals are meaningfully disentangled rather than reflecting general complexity. This supports the interpretability and specificity of our residual reasoning representation.\n\n**Concern: Assumptions Behind Residual Disentanglement**\n\n(a) While transformers do not enforce a sequential processing order, our probes across 17 Qwen models consistently reveal a saturation pattern of syntax → meaning → reasoning (with one exception), suggesting a functional hierarchy in learned representations.\n\n(b) Our approach follows Toneva et al. (2022), who similarly used linear residualization to isolate supra-word meaning. We do not assume strict linearity between features, but rather use linear residuals to subtract overlapping components and highlight feature-specific structure. The resulting embeddings align well with both task performance and brain activity, supporting this as a useful approximation.\n\n(c) Saturation layers serve as empirical markers of when task performance plateaus for a given feature. While not implying that representations stop evolving, they identify the point beyond which further layers add little discriminative power. Their correspondence with shifts in residual behavior and brain alignment suggests they offer meaningful, though approximate, indicators of peak feature encoding.\n\n**_We have updated our local draft accordingly and will incorporate these changes into the camera-ready version. We hope these clarifications address your concerns, and we would be grateful if you would consider revisiting your rating._**"}]

Author rebuttal:

Meta-review: This paper presents a residual disentanglement approach for isolating four separate linguistic feature embeddings (lexical, syntactic, semantic, and reasoning) from LLMs by detecting saturation layers via probing and progressively eliminating lower-level features. Subsequently, the paper employs the reasoning-specific residuals to construct encoding models that predict ECoG responses during natural speech processing.

This work proposes a novel and intuitive residual disentanglement framework that isolates four linguistically meaningful feature embeddings (lexical, syntactic, semantic, and reasoning) from LLMs. The approach is theoretically grounded through a mutual independence theorem that guarantees feature separation, while empirical results demonstrate validation—near-zero cosine similarities between residual embeddings provide evidence for successful disentanglement.

However, one reviewer acknowledges the paper's promising idea but maintains concerns about unsubstantiated claims regarding LLM-derived reasoning embeddings specifically mapping to human brain patterns. Thus, the authors are strongly suggested to show the regions/signals detected by the reasoning embeddings indeed take center stage in the human reasoning process, and show the other embeddings struggle to identify these same regions.
Decision: Accept (poster)