Title: The Mind's Transformer: Computational Neuroanatomy of LLM-Brain Alignment
Primary area: applications to neuroscience & cognitive science
Keywords: language model, neuroscience, brain alignment, fMRI

Abstract: The alignment of Large Language Models (LLMs) and brain activity provides a powerful framework to advance our understanding of cognitive neuroscience and artificial intelligence. In this work, we zoom into the fundamental unit of these models—the transformer block—to provide the first systematic computational neuroanatomy of its internal operations and human brain acitivity during language processing. Analyzing 21 state-of-the-art LLMs across five model families, we extract and evaluate 13 distinct intermediate states per transformer block—from initial layer normalization through attention mechanisms to feed-forward networks (FFNs). Our analysis reveals three key findings: (1) The commonly used hidden states in LLMs are surprisingly suboptimal, with over 90\% of brain voxels in sensory and language regions better explained by previously unexplored intermediate computations; (2) Different computational stages within a single transformer block map to anatomically distinct brain systems, revealing an intra-block hierarchy where early attention states align with sensory cortices while later FFN states correspond to association areas—mirroring the cortical processing hierarchy; (3) Rotary Positional Embeddings (RoPE) specifically enhance alignment along the brain's auditory processing streams. Per-head queries with RoPE best explain 74\% of auditory cortex activity compared to 8\% without RoPE, providing the first neurobiological validation of this architectural component in LLMs. Building on these insights, we propose MindTransformer, a two-stage feature selection framework that learns brain-aligned representations from all intermediate states. MindTransformer achieves state-of-the-art brain alignment performance, with correlation improvements in primary auditory cortex that substantially exceed gains from 456× model scaling. Our computational neuroanatomy approach opens new directions for understanding both biological intelligence through the lens of transformer computations and artificial intelligence through principles of brain organization.

TL;DR: We introduce computational neuroanatomy to map intermediate states in a transformer block to brain regions, revealing intra-block hierarchy and achieving unprecedented auditory cortex alignment with LLMs.