Title: Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement
Venue: NeurIPS 2025
Accepted: yes
Keywords: (Cognitive/Neuroscience) Neural Coding

(Cognitive/Neuroscience) Language
Abstract: Understanding how the human brain progresses from processing simple linguistic inputs to performing high-level reasoning is a fundamental challenge in neuroscience. While modern large language models (LLMs) are increasingly used to model neural responses to language, their internal representations are highly "entangled," mixing information about lexicon, syntax, meaning, and reasoning. This entanglement biases conventional brain encoding analyses toward linguistically shallow features (e.g., lexicon and syntax), making it difficult to isolate the neural substrates of cognitively deeper processes. Here, we introduce a residual disentanglement method that computationally isolates these components. By first probing an LM to identify feature-specific layers, our method iteratively regresses out lower-level representations to produce four nearly orthogonal embeddings for lexicon, syntax, meaning, and, critically, reasoning. We used these disentangled embeddings to model intracranial (ECoG) brain recordings from neurosurgical patients listening to natural speech. We show that: 1) This isolated reasoning embedding exhibits unique predictive power, accounting for variance in neural activity not explained by other linguistic features and even extending to the recruitment of visual regions beyond classical language areas. 2) The neural signature for reasoning is temporally distinct, peaking later (~350-400ms) than signals related to lexicon, syntax, and meaning, consistent with its position atop a processing hierarchy. 3) Standard, non-disentangled LLM embeddings can be misleading, as their predictive success is primarily attributable to linguistically shallow features, masking the more subtle contributions of deeper cognitive processing. Our work provides compelling neural evidence for an abstract reasoning computation during language comprehension and offers a robust framework for mapping distinct cognitive functions from artificial models to the human brain.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a residual disentanglement method that identifies feature saturation layers through probing tasks, and using ridge regression to remove lower-level contributions from higher-level features. Results demonstrate that a \"reasoning\" embedding predicts a unique component of brain activation in human ECoG recordings (with later timing and engagement of frontal regions) that is obscured when using conventional embeddings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe technical execution is rigorous, employing solid ECoG data and a novel residual disentanglement pipeline that includes careful probing, validation of orthogonality, and encoding analyses. The use of a capable modern language model (Qwen-2.5-14B) as opposed to the smaller models used in a lot of other recent work enables the detection of emergent reasoning capabilities that would be impossible with smaller models.The results reveal intriguingly distinct spatiotemporal activation patterns that suggest meaningful differences between linguistic features. \n\nWeaknesses:\n\nI'm concerned that the paper's central claim about isolating \"reasoning\" representations is stronger than what the evidence supports. First, the operationalization of \"reasoning\" through a single, simple task probe (COMPS-WUGS-DIST) does not capture the kind of abstract, multi-step reasoning typically associated with higher-order cognition, limiting the interpretability of the purported reasoning residuals and the corresponding neural analyses. The exciting possibility that these residuals capture genuine reasoning processes requires substantially stronger evidence across diverse reasoning benchmarks and models (see below).\n\nQUESTIONS:\n1. I would suggest testing across the Qwen model family (from smaller 1.8B model to 72B parameters) to provide convergent validity for the reasoning residuals. You should predict that smaller models with poorer reasoning performance should show weaker or absent \"reasoning\" residuals in the disentanglement pipeline, while larger models with demonstrated reasoning capabilities should yield stronger, more distinct reasoning embeddings, and the brain encoding performance of reasoning residuals should scale with the source model's actual reasoning abilities? If a 1.8B model's \"reasoning\" residuals predict brain activity just as well as a 72B model's, it would suggest they're capturing more generic complexity rather than genuine reasoning processes. This would also help test whether the layer localization methodology (finding reasoning \"saturation\" layers) generalizes across model scales, and whether the temporal dynamics (reasoning signals peaking ~330ms) hold only for models that actually demonstrate reasoning capabilities.\n\n2. The COMPS-WUGS-DIST task represents a somewhat impoverished view of \"reasoning\" on its own. Simple property inheritance (\"a wug is an oven -> a wug can heat food\") and variable binding captures only basic logical inference, not the multi-step, abstract, or causal \"situation model\" reasoning typically associated with higher-order cognition. The limits the interpretability of the \"reasoning\" residuals and their neural correlates. I'd suggest using a more diverse set of reasoning benchmarks (e.g., winogrande, multi-step arithmetic) to validate that what are being called \"reasoning\" residuals generalize across diverse reasoning tasks as opposed to being an artifact of this one task instantiation. \n\n3. I'd appreciate a stronger justification for why various assumptions of the residual disentanglement approach should hold empirically. For example, (a) the assumption that the syntax-> meaning -> reasoning hierarchy emerges in a strict sequence doesn't necessarily corresond to how modern transformers process information, (b) the (implicit) assumption that higher-level features are linear combinations of lower-level ones plus orthogonal residuals, (c) the assumption that features \"saturate\" at specific layers and remain stable thereafter (what looks like saturation might just be the limits of the probing task), and (d) while low cosine similarity between residuals tests orthogonality, this doesn't validate that the residuals capture the intended cognitive constructs -- just because they're orthogonal doesn't mean they correspond to cognitively meaningful categories, they might just be isolating arbitrary statistical patterns that happen to correlate with these specific reasoning tasks.\n\n4. I would consider directly testing whether the extracted \"reasoning\" residuals actually causally contribute to reasoning performance. For example, if you corrupt or ablate the reasoning residuals in the original model, does reasoning performance selectively decline while other linguistic capabilities remain intact? This would provide much stronger evidence that the residuals are partitioning out functionally relevant reasoning representations.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a residual disentanglement method to extract four distinct linguistic feature embeddings (lexicon, syntax, meaning, reasoning) from LLMs by identifying saturation layers through probing and iteratively removing lower-level features. The authors apply these embeddings to predict ECoG brain activity during podcast listening, finding that reasoning embeddings show unique spatiotemporal patterns with later temporal peaks and engagement of frontal/visual regions beyond classical language areas.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The residual disentanglement approach is novel and addresses an important limitation in current brain-LLM alignment studies that treat models as monolithic. The finding that reasoning emerges later temporally (~330ms) and engages different brain regions is compelling and aligns with cognitive theories.\n2. The mathematical framework with the mutual independence theorem provides theoretical grounding, and the empirical validation showing near-zero cosine similarities between residual embeddings supports the disentanglement claim. The results showing shallow features dominate neural encoding are important.\n\nWeaknesses:\n1. The orthogonality assumption is problematic. While low cosine similarities suggest reduced overlap, this doesn't guarantee true independence. The theoretical proof relies on strong assumptions about hierarchical feature emergence that may not hold universally across models or linguistic phenomena.\n2. The evaluation is severely limited by using only one model (Qwen2.5-14B) and narrow reasoning tasks (property inheritance only). This restricts generalizability claims about LLM reasoning representations. The ECoG spatial coverage limitations, particularly in frontal regions crucial for reasoning, weaken the brain-based conclusions.\n3. The layer saturation identification could be sensitive to the threshold, and the assumption that ridge regression creates orthogonal features from hierarchical representations needs stronger justification. The reasoning tasks tested are quite simple and may not capture the complexity of reasoning that modern LLMs can perform.\n\nQUESTIONS:\n1. How sensitive are the saturation layer identifications to the threshold ε? Have you tested different values and their impact on the final embeddings?\n2. Can you demonstrate that this disentanglement approach works across different model architectures and sizes? The single-model evaluation is a significant limitation.\n3. Beyond property inheritance, how does the method perform on more complex reasoning tasks like multi-step logical reasoning, causal inference, or mathematical reasoning?\n4. The orthogonality proof assumes perfect hierarchical emergence, but what happens when this assumption is violated? How robust is the method to non-hierarchical feature organization?\n5. Can you validate the brain findings using complementary neuroimaging modalities with better spatial coverage (e.g., fMRI) to address the ECoG limitations?\n\nLIMITATIONS:\nThe authors adequately acknowledge ECoG spatial limitations and single-model constraints. However, they could better address the fundamental assumption about hierarchical feature emergence and the limited scope of reasoning evaluation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. The authors discover LLM layers that maximize linear classification performance on sentence datasets meant to test \"syntax\", \"meaning\", or \"reasoning\".\n\n2. They then extract syntax, meaning, and reasoning word-embeddings by computing inter-layer regression residuals based on a transcribed version of a listening dataset, and empirically show that these embeddings are nearly orthogonal.\n\n3. Finally, they perform regression analyses of recorded brain activity using the extracted embeddings as inputs.\n\n4. They find that (i) lexical / syntax embeddings show higher correlation with brain activity compared to meaning / reasoning embeddings, (ii) reasoning embeddings show peak correlation later in time than the other embeddings, and (iii) reasoning embeddings are correlated with different brain regions compared to lexical / syntax embeddings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The \"probing\" method to locate the earliest LLM layers specializing at different skills followed by regressing out one layer’s activations from another to disentangle embeddings is an interesting idea and a novel contribution to my knowledge.\n2. The paper could inspire future work on gaining a deeper understanding of the similarities/differences between the reasoning processes in LLMs and the human brain.\n\nWeaknesses:\n1. No other LLMs used:\\\n**a)** The abstract claims that the reasoning embedding has unique predictive value \"only because the latest LLMs exhibit emergent reasoning behavior\". In that case, it is crucial to repeat the analysis with an older LLM (that lacks as much reasoning ability) to understand if the brain patterns revealed by the \"reasoning embeddings\" are simply because deeper layers contain higher-level features or because the latest LLMs are better at reasoning (and that the improved reasoning is important for the regression analysis).\\\n**b)** Section 2.2 mentions using a Qwen base model without any further alignment. Given the emphasis on \"reasoning\" throughout the paper, I would expect experiments using at least one open-source model from the recent wave of \"reasoning LLMs\" (i.e., those fine-tuned with supervised learning / RL to solve hard problems). I would also expect these results to be compared to the corresponding non-reasoning LLM (the base model). Without these, given recent developments in the field, the paper’s claims may mislead readers in 2025 and beyond.\n\n2. While Section 3.2 empirically shows that the four embeddings are roughly mutually orthogonal, I do not think the math in Section 3.1 explains this result. In a linear regression with $n$ samples, $d$ input features, and scalar targets, the $n$-D vector of residuals will be orthogonal to all (of the $d$) $n$-D feature vectors. I.e., orthogonality is expected along the sample dimension, not the feature dimension (which doesn’t even make sense if $n \\neq d$). However, Section 3.2 shows orthogonality along the feature dimension.\n\n3. Lines 205-206 imply that syntax and meaning embeddings account for the majority of the explained variance in the brain activity. However, lines 208-209, 213, and 246-247 state that low-level linguistic (syntactic, lexical) features have the highest predictive power. These statements seem inconsistent and the intended message is unclear.\n\n4. Insufficient grounding in neuroscience:\\\n**a)** Section 4.2 shows that reasoning embeddings show peak correlation with brain activity later in time and in different brain regions compared to the other embeddings. But what is the scientific significance of this? Without citations to the neuroscience literature on the spatiotemporal patterns involved in human reasoning, it is not clear how to make sense of the regression result.\\\n**b)** Lines 226-228 describe the progression of high-correlation regions with time. How consistent is this with what we expect from the human brain? If this is significant, the authors should back it up with relevant citations.\\\n**c)** Line 230 states the reasoning embeddings allow for \"isolation of reasoning-specific neural signals\". How does the reader know that the regions detected are indeed reasoning-specific? Without references to the neuroscience literature, this sounds like the circular statement 'any neurons that happen to correlate with our reasoning embeddings are reasoning-specific'.\\\n**d)** Other missing citations: (i) lines 235-236 (and 254-255) linking \"anterior bias\" with \"higher-order\" cognition, (ii) lines 237-238 linking \"language areas\" with the SFG, (iii) lines 252-253 linking \"abstraction and salience\" with the SFG and insula, and (iv) line 264 linking \"delayed temporal peak\" with \"integrative reasoning processes\".\\\n**e)** Based on the above, I think the claim in the abstract of providing \"the first brain-relevant representation of reasoning\" is highly misleading.\n\n5. The text in Figure 5 is unreadable.\n\n6. Minor:\\\n**a)** In line 95, $H$ should have dimensions $d\\times n$ (not $n\\times d$) since the equation below line 113 computes $W\\cdot H$, where I presume $W$ has dimensions $d\\times d$. Similarly, in line 108, $E$ should have dimensions $d\\times n$ (not $d$).\\\n**b)** $\\text{Cov}(A, B)$ below line 185 is not defined, does it mean $A\\cdot B^T$?\\\n**c)** Regression residuals are not a new idea in brain encoding studies. I would recommend the authors cite some prior work on this (e.g., https://aclanthology.org/2021.cmcl-1.1.pdf) so the reader has more context.\\\n**d)** It is not clear what lines 280-285 mean and they do not seem to be discussed previously.\\\n**e)** \"feature\" => \"features\" (line 6).\\\n**f)** The figure cross-reference is missing in line 178.\\\n**g)** Figure 5 => Figure 3 (lines 210 and 214).\n\nQUESTIONS:\n1. What happens if one uses the reasoning embeddings $E_r$ to classify sentences in the datasets $D_s$, $D_m$, and $D_r$? If the residuals truly produce disentangled representations, I would expect performance on $D_s$ or $D_m$ to be worse but on $D_r$ to be relatively unaffected, compared to using the non-isolated embeddings $H_r$ as inputs.\n\n2. In Figure 3a, why is the correlation for \"Full\" worse than for \"Lexicon\"?\n\n3. As a qualitative result, could the authors show a few examples from the podcast where a listener would need to reason/visualize to a higher degree? In such cases, do the reasoning embeddings predict this brain activity?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper proposes a residual‐disentanglement framework that isolates orthogonal LLM embeddings for lexicon, syntax, meaning, and reasoning, and uses the reasoning‐specific residuals to build encoding models that predict ECoG responses during natural speech.

STRENGTHS AND WEAKNESSES:
Strengths:
The idea is easy to follow and the paper is well written. The method clearly separates four linguistic feature embeddings and validates their independence using cosine‐similarity analysis.
It shows that the reasoning embedding captures unique variance in neural responses, peaks later, and engages higher‐level cortical areas.
It provides both a mathematical proof of approximate orthogonality and empirical evidence that the residual embeddings are independent.

Weaknesses:
The experiments use only ECoG data, which undersamples frontal regions and may limit spatial generalizability.
The reasoning probes rely only on COMPS language tasks without widely used advanced reasoning tasks like math and coding.
The study focuses only on Qwen2.5-14B, larger or differently trained models (reasoning models like r1 and qwq) may show different layering or reasoning features.

QUESTIONS:
see weakness

LIMITATIONS:
see weakness
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-07