Scaling and context steer LLMs along the same computational path as the human brain
Authors: Joséphine Raugel
Jérémy Rapin
Stéphane d'Ascoli
Valentin Wyart
Jean-Remi King
Venue: NeurIPS 2025 spotlight ()
Keywords: Large Language Models
LLMs
Emergence
Neuroscience
Brain
Alignment
Language
Magnetoencephalography
Primary area: neuroscience_and_cognitive_science
TL;DR: Emergence of an alignment between LLMs' and the brain's computational dynamics, and key factors allowing it : scaling and context size.

Abstract: Recent studies suggest that the representations learned by large language models (LLMs) are partially aligned to those of the human brain. 
However, whether this representational alignment arises from a similar sequence of computations remains elusive. 

In this study, we explore this question by examining temporally-resolved brain signals of participants listening to 10 hours of an audiobook. 
We study these neural dynamics jointly with a benchmark encompassing 17 LLMs varying in size and architecture type. 

Our analyses reveal that LLMs and the brain generate representations in a similar order: specifically, activations in the initial layers of LLMs tend to best align with early brain responses, while the deeper layers of LLMs tend to best align with later brain responses. 

This brain-LLM alignment is consistent across transformers and recurrent architectures. 
However, its emergence depends on both model size and context length. 

Overall, the alignment between LLMs and the brain provides novel elements supporting a partial convergence between biological and artificial neural networks.

Scaling and context steer LLMs along the same computational path as the human brain
Joséphine Raugel
Stéphane d' A'scolli Meta AI
Jérémy Rapin Meta Al
Meta AI
Laboratoire Neurosciences Cognitives et Computationnelles (Inserm U960) Ecole Normale Supérieure PSL.
Valentin Wyart*
Jean-Rémi King
Laboratoire de Neurosciences Cognitives et Computationnelles (Inserm U960) Ecole Normale Supérieure PSL
Meta AI
*shared senior authorship
Abstract
Recent studies suggest that the representations learned by large language models (LLMs) are partially aligned to those of the human brain. However, whether and why this alignment score arises from similar sequence of computations remains elusive. In this study, we explore this question by examining temporally-resol brain signals of participants listening 10 hours an audiobook. We study these neural dynamics jointly with a benchmark encompassing 17 LLMs varying in size and architecture type. Our analyses confirm that LLMs and the brain generate representations in a similar order: specifically, activations in the initial layers of LLMs tend to best align with early brain responses, while the deeper layers LLMs tend to best align with later brain responses. This brain-LLM alignment is consistent across transformers and recurrent architectures. However, its emergence depends on both model size and context length. Overall, this study sheds light on the sequential nature of computations and the factors underlying the partial convergence between biological and artificial neural networks.
Endanice: Rae - Ouez (Je Yae Temporal score: y Ootr (Wa. Layers)
Y
First layor
4 "Orize timel.
"Once upon time.."
Trx 15)
Figure 1: Methods. A. Subjects listened to 10 hours of audio books in the MEG scanner. B. The text input I.M,, e.g. Llama 3-8B. Colors indicate layer depth. To compare this set of biological and artificial neural embeddings, we fit a linear mapping W for each layer, and evaluate its accuracy with Pearson correlation metric: the alignment score Rlayer- Alignment score (Ruyye) of 9 representative layers of Llama 3-8B, as a function of word-onset (t=0). D. The timestep of peaking alignment scores (Tmax, x-axis) plotted for each layer (y-axis). The resulting Temporal score r and associated are printed on the plot.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
While large language models (LLMs) not designed resemble the human brain, recent studies show that their activations share similarities with those of the brain in response to speech [ 2 3 6]. In the same way bats and birds independently evolved wings 7. LLMs and the human brain thus seem to follow a partial convergence [4].
Motivation.
State of the art. Recent studies showed that an anatomical alignment exists between LLM layers and functional regions of the human brain, in the sense that their first layers tend to align with low-level areas of the brain such primary sensory cortices whereas their deeper layers tend to align with higher-level areas such as secondary sensory or associative cortices [
Remaining challenges. While this anatomical alignment between LLMs and the brain is increas- ingly established, the order in which these representations emerge remains poorly understood. While it has been shown by [6] that GPT2-XL exhibits form of temporal alignment, (i) whether this temporal alignment with the brain is systematically found across LLMs, (ii) whether this alignment depends on the type of architecture, (iii) on its size, or (iv) on the length of its context remains currently unknown. In sum, the factors that lead an LLM to adopt computational path analogous to the human brain's s remain unknown.
Approach. To address these issues, we analyze the temporally resolved brain signals of healthy individuals recorded with magnetoeneephhalogapyy (MEG) [I0), while they listened to 10 hours of audiobooks. We then systematically analyze these neural dynamics in conjunction with a benchmark of 17 LLMs varying in architecture, size and training choices.
2 Methods
Problem formalization. We compare (i) the representations of the human brain in response to natural speech, to (ii) the representations of LLMs in response to the corresponding textual input. Brain activity, here measured with magnetoeneephalogaahh (MEG), leads to a high-dimensional time series that depends, in part, on speech input. To test whether the order of computations is similar between the brain and LLMs, we test whether the orders in which representations are generated are correlated between the two systems.
operationally define a neural "representation" as "linearly readable information". As there no one-to-one aligmment between each MEG sensor and each activation of an LLM, compare the representations across these two systems through linear mapping. Specifically, we fit a ridge regression to predict LLM activations (Y E IR (xxd from brain activity (X, E Bwxx);
Linear mapping. Following others 12 13
W = arg min {|lY - XWI + 11
with the number of words, d the number of LLM activations, the number of MEG sensors and i. the time point relative to word onset. For this, we use scikit-learn's 's RidgeCV, with tuning of logarithmically spaced regularization strength through a grid search approach (a 10-4 to 10 tuned for each dimension independently).
Alignment score. To evaluate this alignment score between an LLM and a human brain, we compute, for each time sample relative to word onset, a Pearson correlation between Y and W X, on held-out test set. We repeat this procedure across all five train-test folds of the cross-validation.
Temporal alignment. After computing this LLM-brain alignment score of / = 9 equally spaced layers of the LLM, evaluate whether the time at which the score peaks correlates with the depth of the layer in the LLM considered. We refer to this as temporal alignment. Specifically, we compute, for each layer T the mean of the temporal window during which 2 95%, where R is the normalized alignment score of the layer, obtained by dividing the alignment score by its maximum value across time. Finally, we compute the Pearson correlation between the Tmax and the relative depth of the layers. The result of this correlation is hereafter referred as temporal score.
Brain data and preprocessing. We here focus on a large within-subject MEG dataset publicly available [00. This dataset consists of three healthy participants who listened to 10h of audio books

---

in CTF MEG scanner. To limit the impact of noise we apply a band-pase filter between 0.1 and 20 Hz, down-sample the signal at 30 H0, time-lock the brain responses to individual words. and epoch the corresponding neural data between -2.5 s and +3 relative to word onset using MNE-Python [4]. Finally, we 7-score MEG signals across words, for each MEG channel and each time point independently.
LLM activations and preprocessing. We use selection of SOTA LLMs to ensure a comprehensive evaluation ranging through architectures, scales, design and training choices. Specifically, we
benchmark models such as Llama-3-8B [5] 16]. Llama-3.2 3.2 (IB, 3B)
15
Mistral-7B-vO.
[19], Gemma-7B [20]. Qwen Diven1.5-7B .B57BB (21]. and GPT-2-XL 12 23 the latter serving as historical
reference. We also investigate dynamics at play in SOTA state space models: Mamba- 1.4B-hf [24) 25]
and RecurrentGemma 9B [26]. Additionally, we leverage the Pythia family [27 consisting of 8 models of increasing size and same training setup. Except when explicitly stated, context length 50 words. For each L.MM, we investigate 9 layers linearly distributed between 10% and 90% of the model hierarchy. To mitigate the issue of heterogeneous sizes, we transform these activation patterns with Principal Component Analysis (n=50) with scikit-learn (28)
Text preprocessing. To ensure the processing of the most semantically meaningful words, we study only content words (as opposed to function words), specifically those which belong to the following part-of-speech categories as defined by Spacy [29]: NOUN, VERB, ADJ, ADV. We ensure the replicability of findings for all words (function and content) via control figures in App.
Compute resources. Evaluating the largest L.LM on our dataset extract both activation patterns for all 9 layers and wext-word probability requires 3 V100 GPU-hours and 12 CPU-hours. Pre-processing both LLMs' and brain's activations, fitting and testing decoding model on our dataset, for each layer and each time step along our temporal window of 5.5s requires in total 4 hours x 9 layers x 15CPUs (Allocated memory: 100 GB) 3 subjects 1,620 CPU-hours for the largest LLM. In total, experiments on 17 LLMs and 9 varying contexts required -75 GPU-hours and ~42,500 CPU-hours. An internal cluster was used for all experiments. Licenses for the models and dataset used are presented in App.
3 Results
Alignment score. We compare the representations of LLMs to those of the human brain in response to natural speech. For this, we fit a linear model to predict the LLMs' contextual representations from the MEG activations, time locked to word onset, and we evaluate this linear mapping with a correlation between true and predicted activations on held-out test set. The results show that the alignment scores between each layer of the LLMS and the brain increase around 0.4s post-word onset (Fig. (-C).
Temporal alignment. We next examine the existence of brain-like dynamics of computations in the nine LLMs specified above. On average, these models show a "temporal score" of = 0.99 (p le-06), between the depth of their layers and the MEG responses to words (Fig. [T- This temporal alignment is observed in all studied models, even trn-transforrer models like ReccurrentGemma-99 and 1.mba-1.4B. (Fig. 2C-D). Oldest model GPT2-XI. exhibits the lowest Temporal score, though significant, = 0.85. While being grounded in the same seminal transformer design, GPT2-XI. is smaller and lacks modern advancements present in the other studied transformers [30] 31. When not pretrained, the LLMs do not show this alignment, and encode very poorly the brain activity. Together, these results suggest that the order of computations activated through recent LLMs' layers is similar to the order of computations of the human brain listening to natural speech.
Impact of causality. To assess the impact of contextual directionality, we compare two bidirectional LLMSs BERT (32) and RoBERTa 33 and one bidirectional speech model Wav2vec2.0 [34]- to the previous causal models, more faithful to the brain's 's causal mechanisms of language processing. While alignment scores are comparable, the temporal scores of these bidirectional models are substantially lower than those of causal LLMS (see App.
Impact of model size. Various architectures, scales and training choices all yield above-chance representational and temporal alignments. To identify the factors that impact the emergence of this phenomenon, repeat these analyses family of LLMs that solely vary in model size. For this,
3

---

A
Decodlnng brain scores layers along time. model
B
Cormlation between LLK depth and Tram al model
C
Terpoot slggnnent boin 3000-300005
100
0.05
Motral 15
-0.00
0.04
0.8
Genna 79
-2.03
0.6
Meda
0.00
0.02
0.4
Unmo 3-10
0052
0.2
0.99 p
3e-07
0.00
Gycbol
00.55
Teu:
0.
10
14
Oewen 5-18
zie
Genee-ity
Uama3230
3h
Varbol
10 1/
Wh
tharl
Figure 2: Human brain and LLMs exhibit temporal alignment. Correlation between time of peaking alignment scores (Tiaxe x-axis) and layer depth shows highly significant temporal alignment. A. Alignment scores of representative layers across each of the studied L.MMs, as function of word-onset (t=0). Alignment scores have been averaged across models. In dashed gray curves, layers from unpretrained versions of these models, averaged over models. B. The time steps of peaking alignment scores ('Imax. x-axis) are plotted for each representative layer (y-axis), averaged across models. The Temporal score and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. Error bars across subjects could not be computed due to the low number of subjects and the need to average across subjects to denoise neural data, though we ensure reproducibility of results across subjects in App. G Here, colored error bars indicate standard deviations of the layer-wise distributions of max across the 9 presented models. C. Temporal scores are computed and presented for each model studied independently. An asterix next to the Temporal score indicates the score is significant with < representative layers across each of the 9 presented LLMs, as a function of word-onset (t=0). Each figure presents model studied independently.
5e-3. D. Alignment scores of 9
eight models of increasing scales. These models are trained in the same way, on the same data, in highly controlled setup where only model size varies. We find that both the representational and the temporal alignment increase with model size (Fig. 33). from a non-significant temporal score (r 0.44, p 0.05) for the smallest model of 14M parameters, to highly significant temporal score (r= 0.96, p < Te-4) for the biggest model of 12 billion parameters. The correlation between the temporal score and the log model size reaches = 0.87 (p 0.01). This emergence follows a logarithmic trend, where the temporal and alignment scores of the biggest models tend to plateau. similar trend is observed for alignment score.
we leverage the Pythia family [7]:
Impact of context size. When listening and processing language, humans accumulate narrative context in the form of evidence to extract the richest meaning out of the currently heard word, and anticipate the next one [35). Motivated by this incremental nature of human language processing, we postulate that brain-like inference dynamics in LLMS emerge with context size. To test how context size impacts representational and temporal alignments, model (Llama-3.2 3B) while varying the amount of words in the input context. The results show that representational and temporal alignments increase with context size (r 0.81, p < 5e-2, Fig. 4). from a non- hon-signnfi temporal score without context (r = 0.19, p > 0.5) to a highly significant temporal score for context of 1000 words (I 0.93, p 3e-4). This temporal score increases logarithmically and considerably slows down from context lengths of 50 words. similar - though lower- - correlation is observed for alignment score. We find similar logarithmic increases of both temporal and alignment scores for state-space model Mamha (see in App.
repeat our analyses on a single
Impact of word predictability. Autoregressive LLMs are trained to predict incoming words/tokens. In a similar way, predictive coding theory suggests that the brain anticipates upcoming words 36. Could this similarity be the driving factor of temporal alignment? To test this possibility, evaluate how temporal alignment varies with word's 's predictability: if this hypothesis true,

---

pythio-14m
pythio-160m
pythio-1b
Pythic-6.90
Pytha-120
Pwr
B
2-40
Teat
Figure 3: Temporal alignment emerges with model size. Colors indicate layer depth. A. Each of the 5 figures on the horizontal axis presents results for specific model belonging to the Pythia family and studied independently, of size 14m, 160m, 1b, 6.9b and 12b parameters respectively (left right). The Pythia family hosts 8 models of increasing size, all trained with the same data amount and parameter choices. Figures present evolution of alignment scores Rlaee of 9 representative layers, from 10% 90% of model depth, on the horizontal axis presents results for a specific models belonging to the Pythia family and studied independently. The time steps of peaking alignment scores (T max. x-axis) are plotted for each representative layer (y-axis). The Temporal score and associated are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. C. Temporal and alignment scores as functions of model size, for the 8 models forming the Pythia family. The model names (x-axis) are displayed on a logarithmic scale corresponding to their respective size. The Pearson scores r and associated p quantifying these correlations are printed on the figure. The grey and blue areas indicate the confidence intervals of the regression estimates.
function of word-onset (1=0). B. Each of the 5 figures
A 1-ward context
1D-word context
300-word context
1000-word context
Mirisse priaal
1.00
B
2-0
Cantact Sayl
Tou
Figure 4: Temporal alignment increases with the length of the context provided to the LLM. Colors indicate layer depth. A. Each of the four figures the horizontal axis presents results for a specific context length provided to Llama-3.2 3B, 1-word, 10-word, 500-word and 1000-word contexts respectively (from left right). Figures present evolution of alignment scores Rayer of representative layers, from 10% to 90% model depth, as a function of word-onset (t=0). B. Each of the four figures on the horizontal axis presents results for a specific context length provided to Llama- 3,2 3B. The time steps of peaking alignment scores (T'ax. x-axis) are plotted for each representative layer (y-axis). The Temporal score and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. C. Temporal and alignment scores as functions of context length when given to Llama-3.2 3B, for six context lengths (x (x-axis). Context lengths are displayed on logarithmic scale. The Pearson scores and associated p quantifying these correlations are printed on the figure. The grey and blue areas indicate the confidence intervals of the regression estimates.
highly unpredictable words should exhibit low temporal alignment. To control for the fact that predictability can be impacted by the word being a content or function word, for this analysis we include all words of our dataset. For this, we first retrieve the predictability of each word in its context from the boftmax-transforre logits of Llama-3-8B. We then separate these words into four
5

---

predictability quartiles. Finally, we evaluate temporal alignment for each quartile independently. The results show that most and least predictable words both lead above chance representational and 0.92, le-3 for the quartile "most expected" and r 0.83, p le-2 for the quartile "most surprising" , respectively (Fig- 5). The temporal score for the most surprising quartile does exhibit a lower value and p. However, when computing the Pearson correlation of the layer-wise differences of Taxx between "most expected" and "most surprising quartiles, we do not find significant impact of contextual predictability on Temporal score (p = 0.61, Fig. [5.). This result holds with more layers too (see App. H.. Overall, this control analysis indicates that word prediceedictabiliity alone does not explain the emergence of temporal alignments between LLMs and the brain.
temporal alignments:
A
.056 Quartile most expected
.06
Quartile most surprising
Laper
Layero.2
Layero.a
Lapen
Laper
Layeroa
1
0
Layer
Time (5)
Layers
.... petmuted
C
1.00
1.0
{ 10.50
0.5
r= -0.20
0.00
Se-04
Be-03
p 0.61
0.31
0.36
0.42
0.32
0.35
Tex: (5)
True (5)
-0.2
Difference in T Trax
Figure 5: Temporal alignment holds independently of word predictability. Colors indicate layer depth. A. Alignment scores of representative layers of Llama-3-88. from 10% to 90% of layer depth, function of word-onset (t=0). Alignment score dynamic curves resulting from evaluating only the quartile of most expected words (from context) among the -270 000 words forming the dataset. The contextual predictability of words is computed through Llama-3-8B. B. The time steps of peaking alignment scores (T x-axis) of the quartile of most expected words are plotted for each representative layer (y-axis) of Llama-3-8B. The Temporal score and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. C. Alignment scores of 9 representative layers of Llama-3-8B, from 10% 90% of layer depth, function of word-onset (I=0). Alignment score dynamic curves resulting from evaluating only the quartile of least expected (i.e. more surprising) words from context, among the -270 000 words forming the dataset. The contextual predictability of words is computed through Llama-3-8B. D. The time steps of peaking alignment scores (Tmax, x-axis) of the quartile of least expected words are plotted for each representative layer (y-axis) of Llama-3-88 3-BB. The Temporal score r' and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. F. The pairwise differences between time steps of peaking alignment scores (Difference in T Tmax, x-axis) per layer, between the quartile of most expected words and the quartile of least expected words, for each representative layer (y-axis) of Llama-3-8B. The Pearson score r and associated p quantifying this correlation are printed on the figure. The grey area indicates the confidence intervals of the regression estimate.
Correlation between temporal and alignment scores. We investigate whether, and why, temporal and alignment scores are correlated, while arising from different measurements. We study jointly the temporal scores across all previously studied models clustered by families and context lengths, with their respective maximal alignment score - i.e. through their best predictive layer. When plotting both alignment and temporal scores across families of models (Fig. 6A). we find that both alignment and temporal scores increase with model size and context length. When correlating both alignment and temporal scores, we find that temporal score indeed correlates significantly with alignment score (Pearson score 0.54, p 9c-04) (Fig. 6B). This result indicates that the capacity of an

---

LLM to predict neural signals of the human brain is correlated to how tightly aligned its pathway of computations with the one play in the brain, when processing language.
A
Hose rindee
Mevoilare 1.2 reaes
nodch
Tempporl score
Hugh
600
= 0.54
Models
Alignment score
Figure 6: Temporal and alignment scores significantly correlated. A. Temporal score and alignment score evolve similarly across models and context lengths, along the x-axis, clustered by families of models. Models presented in the format [model name, model size, context length]. B. Temporal score (y-axis) and alignment score (x-axis) are significantly correlated over all models and variance in contexts size presented in this research. The Pearson score and associated p quantifying this correlation are printed on the figure. The grey area indicates the confidence interval of the regression estimate.
Discussion
Temporal alignment. This work investigates whether and when LLMs generate representations in an order similar to the human brain's 'sin's during natural speech listening. This study provides three main contributions. First, order of representations generated by LLMs' layers strongly correlates with the sequence of neural activations observed in the human brain recorded with MEG. This temporal alignment complements previous work on the anatomical alignment observed between LLMs' layers and functional regions of the human brain [5] 8 3 9 I]. and systematize early report of temporal alignment between language models and the brain
A shared computational path. We show that temporal scores is independent of the word pre- dictability (Fig. 5) Beyond indicating the autoregressive generation capacities of the LLM, the temporal score thus seems indicate its brain-like inference dynamics. Additionally, as previously reported [2] 3 37). the best alignment scores are achieved in the intermediary layers. This result suggests that the intermediary representations (layer=0.6 of architecture) as opposed to the input and the prediction are best aligned between brains and LLMs. Finally, while part of the alignment score can explained by the latent representations present from the very first layer accounting for the relatively high alignment score observed from the first layers of the LLM the temporal score accounts for evolving representations, a sequentially ordered mechanism of processing. Together, these results indicate that the brain and LLMs don't 't just share similar representations, but a similar
computational path too. The increase of alignment score more brain-like models may stem from shared sequential structure of computations between the LLMs and the brain, as captured by the temporal score.
Impact of model architecture. Second, this LLM-brain alignment is observed in non-transformer architectures, such as state-space models. While other architectures (e.g. fully connected networks 38 Kolmogorov-Arnot Networks (39] remain be investigated, this finding suggests that the convergence between L.MSs and the brain is innirecture-iddeeenddent If confirmed, this result provides additional evidence that alignment score does not result from trivial inductive bias. It remains to be investigated, however, whether it relates to model objective (next token prediction) or the structure of language 40]
Impact of context and model size. Third, our experiments show that this alignment directly depends on context size and (ii) model size although with saturation beyond 70B parameters models (App. D) These MEG results extend previous works scaling laws in neuro-ai 337 41 42
7

---

In particular, [3 43
showed that LLMs that best predict functional magnetic resonance imaging (fMRI) responses to natural speech are those with the largest amount of parameters. In parallel,
45 showed that context size improved the alignment between brain and fMRI. Here, we further show the effect of context size and model size increase logarithmically, hence pointing to diminishing returns, if not plateau. Extending our analyses to larger LLMs, e.g. Llama 3.3 70B, does not yield major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to plateau effect of scaling laws identifiable with MEG. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or confounded by other uncontrolled variables, such as linguistic performance. For example, in App. ! we find that performance is correlated with temporal alignment for specific conditions LLM with increasing context lengths but not others LLMs with increasing sizes. Disentangling the causal chain that links these factors remains major research avenue.
Working memory. Interestingly, the comparison between State Space Models (SSMs) and LLMs offers new perspective to investigate context-size. Indeed, transformers compute contextual rep- resentations, thanks to a non linear combination of the past context, and hence require very large memory buffers that implausible in the brain. By contrast, SSMs can be thought of recurrent neural networks (RNNs) with hidden states that linearly evolve over time. At each time point, they thus represent the full context with their hidden state an approach presumably similar to the brain. Our results show that SSMs do exhibit modest yet consistent improvements in temporal scores even very large context sizes (App. B and thus show that it is, in fact possible, to build and maintain long context in a single hidden state. However, further research remains necessary to evaluate the degradation of such memory in the absence of meaningful context, such as during the memorization of random digit sequence like phone number a task recognized as highly constrained in human subjects 446.
Limitations. Three main limitations should be highlighted. First, MEG has limited spatial resolution of brain activity. This recording device does not allow single-neuron recording, and is largely unable to pick up deep sources. Consequently, the precise neuronal bases of the present findings remain to be further explored with intracranial recordings. Second, the present dataset only consists of 3 subjects. This design choice was motivated by the fact that subjects each listen to 10 of audiobooks, making this dataset the largest per-subject MEG listening dataset authors found, among multi-subject datasets. As alignment models are trained subject-wise, the biggest possible pur-subject amount of data allows the most precise brain-alignment with LLMs dynamics. While App. Ghoows similar findings can be identified within each of these three subjects, how these alignments vary across individuals thus remains unknown. Finally, the present work only focuses on pretrained text-models. Yet, humans necessarily process language through sensory modality. Consequently, future work remains necessary to investigate the similarities between brains and speech models
47 48 5 9.
The present findings reveal striking similarities between human brain responses and sequential representations in multiple kinds of large language models (LLMs). Yet, their architecture, sensory modality, learning objective, and training regime are remarkably disjoint 49 50 [1], (2). Furthermore, specific features like syntactic structures and semantic roles may be represented fundamentally differently [53] (4). That partially converging computational paths arise despite these differences is thus all the more surprising, and highlights the necessity clarify the computational principles that lead language processing to be partially shared between biological and artificial systems. We hope that systematically investigating the factors that steer LLMs function similarly to the brain will help reduce this major gap, and ultimately chart path toward building artificial systems that leamn as efficiently as the human brain.
Impact.
8

---

Acknowledgments
We warmly thank the authors of [T0 for building the MEG dataset on which we relied for this study. We wish to thank every member of Meta's legal team for helping us to process quickly through the legal process in order to submit in time to the Neurips 2025 conference.

---

References
[1] Aditya R Vaidya, Shailee Jain, and Alexander Huth. Self-supervised models audio effectively explain human cortical responses to speech, 2022.
[2] Shailee Jain, Vy Vo, Shivangi Mahto, Amanda LeBel, Javier S. Turek, and Alexander G. Huth. Interpretable multi-timescale models for predicting fmri responses to continuous natural speech. In Advances in Neural Information Processing Systems, volume 33, pages 13738-13749, 2020.
[3] Mariya Toneva and Leila Wehbe. Interpreting and improving natural-language processing (in machines) with natural language-processingg (in the brain), 2019.
[4] Charlotte Caucheteux and Jean-Rémi King. Brains and algorithms partially converge in natural language processing. Communications biology, 5(1): 2022.
[5] Juliette Millet, Charlotte Caucheteux, Yves Boubenee, Alexandre Gramfort, Ewan Dunbar, Christophe Pallier, Jean-Remi King, et al. Toward realistic model of speech processing in the brain with sulf-suuee learning. 2022.
[6] Ariel Goldstein, Eric Ham, Samuel Nastase, Zaid Zada, Avigail Grinstein-Dabus, Bobbi Aubrey, Mariano Schain, Harshvardhan Gazula, Amir Feder, Werner Doyle, et al. Correspon- dence between the layered structure of deep language models and temporal structure natural language processing in the human brain. BioRxiv, pages 2022-07, 2022.
[7] Neil Shubin, Cliff Tabin, and Sean Carroll. Fossils, genes and the evolution of animal limbs. Nature, 388(6643):633 648, 1997.
[8] Charlotte Caucheteux, Alexandre Gramfort, and Jémn-Rémi King. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nature human behaviour, 7(3):430 441, 2023.
[9] Yuanning Li, Gopala K Anumanchipalli, AbdeIrahman Mohamed, Peili Chen, Laurel H Carney, Junfeng Lu, Jinsong Wu, and Edward F Chang. Dissecting neural computations in the human auditory pathway using deep neural networks for speech. Nature Neuroscience, 26(12):2213- 2225, 2023.
[10] Kristijan Armeni, Umut Güççüü Mareel van Gerven, and Jan Man-Mathijs Schoffelen. A 10-hour within- authin-participann magnetoencephalography narrative dataset to test models of language com- prehension. Scientific Data, 911):278, 2022.
[11] Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:249, 2008
[12] James DiCarlo, Davide Zoccolan, and Nicole C Rust. How does the brain solve visual object recognition? Neuron, 73(3)4415-43, 2012.
[13] Jean-Rémi King and Stanislas Dehaene. Characterizing the dynamics of mental representations: the temporal generalization method. Trends in cognitive sciences, 18(4):203-210) 2014.
[14] Alexandre Gramfort, Martin Luessi, Eric Larson, Denis A Engemann, Daniel Strohmeier, Christian Brodbeck, Roman Goj, Mainak Jas, Teon Brooks, Lauri Parkkonen, al. Meg and eeg data analysis with mne-python. Frontiers in Neuroinformatiis, 7:267, 2013.
[15] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The llama 3 herd of models. 24X 2024.
[16] Meta AI. Llama Open foundation and instruction models. https://huggingfacee meta- meta-1lama/ MallamaMMMtta... Llama- 2024.
[17] Meta AI. Llama 3.2 1b. http:///nngiiunaeeeooorrarlel//m 2024.
[18] Meta AI. Llama 3.2 3b. httpa///nnniinggee.oooereeiiimmm 2024.
[19] Mistral AI. Mistral 7b. http://lgggialee.ooooosoaaaaaaas 2023. 2023.
10

---

[20] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, al. Gemma: Open models based on gemini research and technology, 2024.
[21] Alibaba Cloud. Qwen1.5 7b. https://huggingfaceccm httpaicee..o//ee/eeeeeeeeeeeeee 5-BB 2024.
[22] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask multtass learners, 2019.
[23] OpenAl. Gpt-2 xl. https///hugggingfaccc.oo//op
2019.
[24] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2024.
[25] Stateful AL. Mamba-1.4b-hl. 2024.
/aw.bbee/bbhtttts//ggggggggeeeeg http://cee..osstttttssappeeee//e 1.t.://--maaaaaa-aaa.. hf
https://hugginggfaee
[26] Alexsandar Botev Griffin Team, Soham De, Samuel I, Smith, Anushan Fernando, George- Christian Muraru, Ruba Haroun, and Leonard Berrada al. Recurrentgemma 2b. https
co/ http://aaa.....oooooooo/////rrea
2b 2024.
[27] Stella Biderman, Hailey Schoelkopf. Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling, 2023.
[28] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python. the Journal of machine Learning research, 12:2825-2830, 2011.
[29] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spacy: Industrial- strength natural language processing in python. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 1-7. Association for Computational Linguistics, 2020.
[30] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
[31] Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202 2020.
[32] Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
[33] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach, 2019.
[34] Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: framework for self-supervise learning of speech representations, 2020.
[35] Micha Heilbron, Kristijan Armeni, Jan-Mathijs Schoffelen, Peter Hagoort, and Floris De Lange. A hierarchy of linguistic predictions during natural language comprehension. Proceedings of the National Academy of Sciences, (9(33))e2201968 19, 2022.
[36] Karl Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456):1158366, 2005.
[37] Richard Antonello, Aditya Vaidya, and Alexander Huth. Scaling laws for language encoding models in fmri. Advances in Neural Information Processing Systems, 36:21895-2190 2023.
[38] Alexander G. Schwing and Raquel Urtasun. Fully connected deep structured networks. CoRR, hls/1503.02351, 2015.

---

[39] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačié, Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov networks. arXiv preprint arXiv:2404. 19756, 2024.
[40] Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.0733 07987, 2024.
[41] Stéphane d' Ascoli, Corentin Bel, Jérémy Rapin, Hubert Banville, Yohann Benchetrit, Christophe Pallier, and Jean-Rémi King. Decoding individual words from non-invasive brain recordings across 723 participants. arXiv preprini arXiv:2412.1782) 2024.
[42] Hubert Banville, Yohann Benchetrit, Stéphane d' Ascoli, Jérémy Rapin, and Jean-Rémi King. Scaling laws for decoding images from brain activity. arXiv preprint arXiv:2501. 15322, 2025.
[43] Laurent Connasse-Gahot and Christophe Pallier. fmri predictors based on language models of increasing complexity recover brain left lateralization. Advances in Neural Information Processing Systems, 37:122231-125222 2024.
[44] Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. Deep language algorithms predict semantic comprchension from brain activity. Scientific reports. 12(1):16327, 2022.
[45] Refael Tikochinski, Ariel Goldstein, Yoav Meiri, Uri Hasson, and Roi Reichart. Incremen- tal accumulation of linguistic context in artificial and biological neural networks. Nature communicctionss, 16(1) (603, 2025.
[46] George A Miller. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review, 63(2):81, 1956.
[47] Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units, 2021.
[48] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: framework for self-supervised learning of speech representations. Advances in neural information processing systems, 11112244114666 2020.
[49] Linnea Evanson, Christine Bulteau, Mathilde Chipaux, Georg Dorfmüller, Sarah Ferrand- Sorbets, Emmanuel Raffo, Sarah Rosenberg, Pierre Bourdillon, and Jean-Rémi King. Emergence of language in the developing brain. Manuscript, May 2025. Equal contribution by Pierre Bourdillon and Rémn-Rémi King.
[50] Emmanuel Dupoux. Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner. Cognition, 173:43-59, 2018.
[51] Rachel R Romeo, Joshua Segaran. Julia A Leonard, Sydney T Robinson, Martin R West, Allyson P Mackey, Anastasia Yendiki, Meredith L Rowe, and John DE Gabrieli. Language exposure relates to structural neural connectivity in childhood. Journal of Neuroscience, 38836)77800-7877 2018.
[52] Mohammad Amin Ghanizadeh and Mohammad Javad Dousti. Towards data-efficient language models: A child-inspired approach to language learning. arXiv preprint 1rXXiv2503.046) 2025.
[53] James Fodor, Carsten Murawski, and Shinsuke Suzuki. When word order matters: human brains represent sentence meaning differently from large language models. bioRxiv, pages 2025-07, 2025.
[54] Daria Kryvosheieva and Roger Levy. Controlled evaluation syntactic knowledge in multilin- gual language models. arXiv preprint 2rX1v:2411.07474, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract claims that activations in carly-to-late layers of LLMs best align with early-to-late brain responses when processing word. The abstract also claims that at least two factors allow this emergence in an LLM, size and context. The Result section of the paper show evidences for these claims. P-values are provided in all figures, either explicitly or noted in the form of asterixs, and ensure the significancy of these results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper does discuss three limitations of the work performed by the authors in the section Discussion, paragraph Limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
13

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the suppiemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: The paper fully discloses all the information needed reproduce the main experimental results of the paper, in the Method section, as well as in figure captions and specific subsections for result-specific methods.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification:
The paper is based on an open-source dataset cited in the Introduction section and in the Methods section. The paper cannot currently release the associated code as is based an internal codebase, however said codebase planned be released in short-term future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
Sooeeeee/mooSSSmmmcooa SonodeSubmisssooPoo//..oo ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setting, including training and test details, is presented in the core of the paper in Section Methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper does report statistical significance of every result in the form of P. values in each corresponding figure as well as in the main text of the paper. Error bars are computed when possible, defined in Methods and justified in Methods when not possible to compute (e.g. too few subjects). Confidence intervals are also computed in each figure where technically possible.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information on the computer resources needed to reproduce the experiments in the Methods Section, in paragraph LLM activations and preprocessing.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in this paper conforms to points presented in the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
16

---

Answer: [NA]
Justification: The scope of the paper being foundational research only, the authors cannot find any direct path to any negative applications of the results presented in the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not release any new data or model.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All creators or original owners of assets used in the paper are cited the main text, next to the mentioned asset.
Guidelines:
The answer NA means that the paper does not existing assets.
17

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license ge.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, papprssitthheeeocc thoode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pan re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: The present paper uses only one dataset regarding research with human subjects, and this dataset has already been published for three years, is open-source and available to all, and cited in the paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
18

---

Justification:
The present paper uses only one dataset regarding research with human subjects, and this dataset has already been published for three years, is open-source and available to all, and cited in the paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: An LLM was used only for editing, or formatting purposes, not used as tool in the core methods of this research. LLMs were used in the scope of this research not as tools but as subjects, their dynamics were studied in comparison to human brain's dynamics during language processing as the scope of the present paper, but authors believe the field of research of the paper to not be the scope of this question.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
19

---

Supplementary Material and Technical Appendices
A Encoding scores for two LLMs.
Per channel encoding scores for layer 0.9 Qwen1.5-78
Meta-LLama-3-98 n.2
-10s
14x
Figure 7: Encoding scores for two LLMs. Per-channel encoding scores for the layer 1=0.9 of two LLMs, Qwen 1.5-7B (up) and LLama-3-8B (down), at several time points during word prediction pre word-onset (-1.0s, left), and processing post word onset (0.2s middle, and 1.4s right).
20

---

B Temporal and alignment scores of bidirectional models.
Below are presented results for bidirectional LLMs BERT and RoBERTa, alongside bidirectional speech model Wav2vec2. All three models show alignment scores comparable to larger, more recent and causal LLMs, but much lower, non-significant temporal scores.
Wav2Vec
google-bertbert--bsee-raaaa
Facebookkl/robertaalargg
6015
0.055
Time (s)
Time (s)
Time (s)
12:
1.2
0.75
10
0.00
Layers
0.75
0.00
06
2.06
0.
0.00
0,38
-0.18
p 1.00
0.31
p 0.64
0.0
Timex
Tra
Trax
Figure 8: Temporal and alignment scores of bidirectional models. Each of the 3 figures presents results for a Wav2vec2, BERT and RoBERTa. Upper figures on the horizontal axis present evolution of alignment scores Rayer of 9 representative layers, from 10% to 90% of model depth, of word-onset (t=0). Lower figures on the horizontal axis presents the time steps of peaking alignment scores (T x-axis) plotted against each representative layer (y-axis). The Temporal score r and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate.
function
21

---

C Alignment scores across subjects, per LLM
Qwen1.5-78
Mistral-7B-v0.1
Gemma-7b
Meta-Llama-3-48
Uama-3.2-38
Uama-3.2-4
Recurrentgemma-90
Mamba-1.4b-hf
Gpt2-xl
0.00
0.02
0.04
0.06
0.08
Figure 9: Alignment scores across subjects, per LLM. Maximum alignment scores per model, Each errorbar is standard deviation across the three subjects of the dataset.
22

---

D Temporal and alignment scores of the largest models
mistralai/Mixtrra-877B....1
0.058
0.056
0.1
0.1
0.2
0.2
0.3
0.3
0.4
0.4
0.7
0.8
0.8
0.9
0.9
0.000
0.000
Time (s)
Time (s)
0.6
Layers
Layers
0.6
0.2
= 0.93
r = 0.88 2e-03
0.2
2e-04
0.0
-0.2
max (s)
T max (s)
Figure 10: Temporal and alignment scores of the largest models. Each of the 2 figures presents results for Llama3.3-70B 70ttttrutttt and Mixtral-8x7B--O.. -8-1x1x-.11.... Upper figures on the horizontal axis present evolution of alignment scores Rlaee of representative layers, from 10% 90% of model depth, as a function word-onset (t=0). Lower figures on the horizontal axis presents the time steps of peaking alignment scores (Tmax x-axis) plotted against each representative layer (y-axis). The Temporal score r and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate.
23

---

E Temporal alignment increases with the length of the context provided to State-Space Model Mamba-1.4B
A
1-word context
1D-word context
500 word context
103D-word context
aue cas 1. 2eo37
1.0
6.0
2- 183
Garted Say
Figure 11: Temporal alignment increases with the length of the context provided to State-Space Model Mamba-1.4B. Colors indicate layer depth. A. Each of the four figures the horizontal axis presents results for specific context length provided to State-Space Model Mamba- 1.44, I-word, 10-word, 500-word and 1000-word contexts respectively (from left to right). Figures present evolution of alignment scores Rlayer of 9 representative layers, from 10% to 90% of model depth, as a function of word-onset (t=0), B. Each of the four figures on the horizontal axis presents results for a specific context length provided to Mamba-1 1.4.. The time steps of peaking alignment scores (T x-axis) plotted for each representative layer (y-axis). The Temporal score and associated p are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. C. Temporal and alignment scores as functions of context length when given Mamba- 1.4B, for six context lengths (x-axis). Context lengths are displayed on a logarithmic scale. The Pearson scores and associated p quantifying these correlations are printed on the figure. The grey and blue areas indicate the confidence intervals of the regression estimates.
24

---

Human brain and LLMs exhibit temporal alignment. Analysis performed for all types of words: function and content words
To ensure the processing of the most semantically meaningful words, we often study only content words in the main text of this paper (as opposed to function words), specifically those which belong to the following part-of-speech categories as defined by Spacy: NOUN, VERB, ADJ, ADV. Here, we test for replicability of findings for all words (function and content).
Analysis on both content and function words
Correistion beoweer LUM desth and Tran all moseb
5-10
+0.9
Mittal-7E-CC
+0.77
1.0
-0.1
0.5
Jama 3 38
-0.8
Ulma-3200 2-30
0.96
0.2
Se-05
Parrbe-1.4b-r
-0.70
Gpc2al
Time
0.51
0.30
Tax:
0.41
Tkrppral Peorson
D
Myndar
Figure 12: Human brain and LLMs exhibit temporal alignment. Analysis performed for all types of words: function and content words. Correlation between time peaking alignment scores (Tmxx x-axis) and layer depth shows highly significant temporal alignment. A. Alignment scores of 9 representative layers across each of the studied L.LMs, as function of word-onset (t=0). Alignment scores have been averaged across models. B. The time steps of peaking alignment scores (amax x-axis) are plotted for each representative layer (y-axis), averaged across models. The Temporal score and associated P'aauc are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. Here, colored error bars indicate standard deviations of the layer-wise distributions of Tmax across the 9 presented models. C. Temporal scores computed and presented for each model studied independently. An asterix next to the Temporal score indicates the score is significant with Pvalue < 3. D. Alignment scores of 9 representative layers across each of the 9 presented LLMs, as function of word-onset (t=0). Each figure presents one model studied independently.
25

---

G Human brain and LLMs exhibit temporal alignment - Subject 1, 2 and 3 plotted individually
We ensure reproducibility of results presented in the main text of this paper averaged across subjects this time for individual subjects.
Subject 1
B
comnearion betuuee LMM capeh and Tex vode:
Owand 3-18
Ma tire
-3.61
3
Uare32111
-857
76-07
Marda
0.31
041
632
Subject 2
Condrranneeeee LM seytt and Tex vades
.3 3.
0.25
0.34
10000
Meto 3.C
0.31
0.97 2e-05
0.00
Tour Hi
120
Subject 3
Drood time.
Carmadion betwees LLN depth mode
C
3000
02
0.99
3a-07
0.0
Yea
12
28
Figure 13: Human brain and LLMs exhibit temporal alignment Subject 1, 2 and 3 plotted individually
Correlation between time of peaking alignment scores (T x-axis) and layer depth shows a highly significant temporal alignment. A. Alignment scores of 9 representative layers across each of the studied LLMs, as function of word-onset (t=0). Alignment scores have been averaged across models. B. The time steps of peaking alignment scores (Tmax. x-axis) are plotted for each representative layer (y-axis), averaged across models. The Temporal score and associated Poatue are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. Here, colored error bars indicate standard deviations of the layer-wise distributions of Tmux across the 9 presented models. C. Temporal scores are computed and presented for each model studied independently. An asterix next to the Temporal score indicates the score is significant with Poatue 5e-3. D. Alignment scores of representative layers across each of the 9 presented LLMs, as function of word-onset (t=0). Each figure presents one model studied independently.
26

---

H Temporal alignment holds independently of word predictability
When computing the Pearson correlation of the layer-wise differences of max between "most ex- pected" and "most surprising" quartiles, we do not find a significant impact of contextual predictability on Temporal score. To ensure this result, we perform here the same analysis on twice as many layers across the depth of Llama-3-8B architecture.
.056 Quartile most expected
Quartile most surprising
Orain
E
No significant impact contextual predicuability temporal alignment
lagaru
-2
-1
0
2
-2
Time (6)
D
Time (s)
Lswe
Liver 0c.
1.0
0007
Ligar 0.55
Layers
0.55
0.5
Difference
Ligu 0.4
0.0
Lever 0.0
0.28
0.33
0.38
0.43
0.28
0.33
0.38
0.43
0.48
Tixx
Trea (5)
Figure 14: Temporal alignment holds independently of word predictability. Colors indicate layer depth. A. Alignment scores of 19 representative layers of Llama-3-8B, from 10% to 90% of layer depth, as a function of word-onset (t=0). Alignment score dynamic curves resulting from evaluating only the quartile of most expected words (from context) among the -270 000 words forming the dataset. The contextual predictability words is computed through Llama-3-8B B. The time steps of peaking alignment scores (Tmax. x-axis) of the quartile of most expected words plotted for each representative layer (y-axis) of Llama-3-8B. The Temporal score r and associated Pealue are printed the figure. The grey area indicates the confidence intervals of the regression estimate. C. Aligmment scores of 19 representative layers of Llama-3-8B, from 10% to 90% of layer depth, function of word-onset (1=0). Alignment score dynamic curves resulting from evaluating only the quartile of least expected (i.e. more surprising) words from context, among the ~270 000 words forming the dataset. The contextual predictability of words is computed through Llama-3-8B. D. The time steps of peaking alignment scores (Tmaxx X-axis) of the quartile of least expected words are plotted for each representative layer (y-axis) of Llama-3-8B. The Temporal score r and associated Pvalue are printed on the figure. The grey area indicates the confidence intervals of the regression estimate. E. The pairwise differences between time steps of peaking alignment scores (Difference in Tmax x-axis) per layer, between the quartile of most expected words and the quartile of least expected words, for each representative layer (y-axis) of Llama-3-8B, The Pearson score and associated Pvalue quantifying this correlation are printed on the figure. The grey area indicates the confidence intervals of the regression estimate.
27

---

I Temporal scores along top-3 accuracies evaluated on our dataset for next-token prediction across LLMs
We find that performance is correlated with temporal alignment for specific conditions LLMs with increasing context lengths but not others LLMs with increasing sizes. Regression estimates illustrate how scaling model size or context affect temporal alignment with brain activity. Increasing context lengths significantly correlates with temporal alignment (p 0.01). Increasing model sizes does not significantly correlate with temporal alignment.
A
B
1.0
syShh-14m,8
1.0
Mec Liamo 5 56.
symte-70m, 50
Matae lama-3-00 3-00 2
0.5
symte-165m, 50
Meta Lama-3350
syphis-410m_33 59
Meta 1ama-3-00
pymte-18_85 50
Metil bmm-3-08
s20e
Pymte-3 45,
Tempanal scome
Metii 3mma--10 7
sythis $,
Veda Liana 3 88_15
pytii 20
Medal 3 88,20
Regresico hocsasing sizs
0.2
Vicaa Liama-3-5
Midaa Lumoo 88,
0.0
Regrussion on Indreas contiat
0.5
r = 0.2
-0.2
0.77
0.64
0.01
0.053
0.004
0.006
0.00
0.02
0.54
0,06
n.c8
Top-3 accuresy
Top-3
Figure 15: Temporal scores along top-3 accuracies evaluated on our dataset for next-token prediction across LLMs. A and В show the correlation between Temporal score (y-axis) and top-k accuracy (X-axis), across two subsets of models: A. Pythia models of increasing size, evaluated top-k= = 3. B. Meta-I Meta-Llama-33 Jama-3-88 model with increasing context lengths, evaluated at top-k = 3.
28

---

Licenses
The dataset used in this paper is licensed under Creative Commons Attribution 4.0 International License.
Regarding the models used, here follows list of their licenses:
Qwen/Qwenl. 15--BB: Tongyi Qianwen License
mistrala/MMisttal-BB-.O.I. Apache 2.0 License
googld/gemma+77: Gemma License
mela-lamaa/Maa-llaaa-3BBB: IJama Community License Agreement
metallama/llamaa332333: Llama 3.2 Community License Agreement
mete-laama/llama3322-BBBBL Llama 3.2 Community License Agreement
googlefrecurrenggemnna-bb Gemma License
haate-ppacesmmmmb---Abbbb Apache 2.0 License
openai-oommunityggl--xit MIT License
Pythia family of models: Apache 2.0 License
29


Reviews:
[{"review_id": "gRma5FRrG8", "reviewer": "Reviewer_mo9D", "summary": "This paper aims to assess whether brain---neural network representation alignment is a consequence of the two systems performing similar computations. To this end, authors make a clever experimental design wherein temporally-resolved brain signals are matched with model representations. If representations generate in a similar order, the alignment across layers will reflect that---a finding authors make.", "questions": "Building on one of the minor weaknesses, I would appreciate authors' thoughts on why they found model scaling to not be harmful for achieving temporal alignment. This is in contrast to prior finding on brain representational alignment. My working hypothesis would be that a good modeling of context is necessary to get good temporal alignment, which is not the target of analysis of past work, and hence scaling models eventually saturates alignment score. Alternatively, it could be the case that the temporal alignment score is too generously defined, i.e., it is trivial to achieve high scores on it. Having baselines, e.g., random or relatively less competent models (e.g., GPT-2 small), could help assess this.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths.** I really enjoyed the paper! While arguably the experiments are behavioral in nature, i.e., merely representational alignment measures are used, I found the experimental design very well motivated. The findings themselves are reasonable, though I am somewhat confused why better performing models (often the larger one) show a better temporal alignment score (see questions).\n\n**Weaknesses.** I don't have much to note here. Adding some very minor points below.\n\n- I think the writing can improve on points of discussion w.r.t. past work. For example, on page 4, in section 3.2, I expected authors would need to discuss working memory constraints, since a Transformer's ability to (in theory) keep track of infinitely long contexts is often used to argue against their use for neuroscientific comparisons between LLMs and human brain. Similarly, the impact of model scaling would be worth discussing at several points: authors often saw larger models are performing equally well and scoring high on temporal alignment, which stands in contrast to expectations one would have based on past work that shows scaling model does not improve brain alignment.\n\n- Figure 6's design is somewhat awkward. Individual markers are different models and colors are model families. Connecting these markers with lines does not make sense if you're not trying to put together a trend. Arguably, that single plot in Figure 6a should be three separate plots.", "was_revised": true, "final_justification": "Authors addressed all of my concerns.", "rebuttal": "We thank Reviewer mo9D for their thorough review. We propose to amend several sections of the manuscript to address these issues and add five additional sets of results.\n\n**1. Discussion working memory**\n\nThis is indeed a very good point. We will add the following paragraph:\n\n\n“Interestingly, the comparison between State Space Models (SSMs) and LLMs, offer a new perspective to investigate context-size. Indeed, transformers compute contextual representations, thanks to a non linear combination of the past context, and hence require very large memory buffers that are implausible in the brain. By contrast, SSMs can be thought of as recurrent neural networks (RNNs) with hidden states that linearly evolve over time. At each time point, they thus represent the full context with their hidden state – an approach presumably similar to the brain. Our results show that SSMs do present very large context size improvements, and thus show that it is, in fact possible, to build and maintain a long context in a single hidden state. However, further research remains necessary to evaluate the degradation of such memory in the absence of meaningful context, such as during the memorization of a random digit sequence like a phone number – a task recognized as highly constrained in human subjects (Miller et al. (1956)).”\n- Miller (1956), Psychological Review. \"The magical number seven, plus or minus two: Some limits on our capacity for processing information.\"\n\n**2. Discussion model scaling** \n\nWe agree that we insufficiently credited previous work on scaling laws in neuro-ai. We now amended the discussion as follows:\n\nThird, our experiments show that this alignment directly depends on (i) context size and (ii) model size. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)).\nIn particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.\n\n- Antonello et al. (2023): Antonello et al 2023  NeurIPS. \"Scaling laws for language encoding models in fMRI.\"\n- Bonnasse-Gahot & Pallier (2024): Bonnasse-Gahot & Pallier (2024), NeurIPS. \"fMRI predictors based on language models of increasing complexity recover brain left lateralization.\"\n- Tikochinski et al. (2025): Tikochinski et al. (2025), Nature Communications. \"Incremental accumulation of linguistic context in artificial and biological neural networks.\"\n- d’Ascoli et al. (2024): d’Ascoli et al. (2024), arXiv. \"Decoding individual words from non-invasive brain recordings across 723 participants.\", \n- Banville et al. (2025): Banville et al. (2025), arXiv. \"Scaling laws for decoding images from brain activity.\"\n- Caucheteux et al. (2023): Caucheteux et al. (2023), Nature Human Behaviour. \"Evidence of a predictive coding hierarchy in the human brain listening to speech.\"\n\n\n**3. Redesigned figure 6**\n\nWe agree with this note and propose to separate the figure 6a in three plots. The line linking the markers is not crucial to the figure but its objective is indeed to highlight trends where bigger models and longer contexts induce higher brain-score and temporal-score. Separating the figure in three plots shall make these trends clearer. \n\n\n**4. Baselines as untrained or less competent model such as GPT-2**\n\nThe untrained versions of the models are shown as dashed grey lines in Figure 1 of the manuscript.Their brain scores did not differ significantly from zero over time, resulting in a null temporal score.\n\nAnalyzing a less competent models such as GPT-2-small is indeed a very good point. We conducted this analysis and found a brain-score of 0.054, comparable to more recent and bigger models, but a temporal score of 0.20, which is substantially lower. We further analyze and contextualize this result in the following section, alongside additional analyses.\n\n\n**5. New experiments: Additional models and analyses**\n\nTo address reviewers’ comments, we now run five additional analyses:\n\n1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to the causal models previously presented, more closely reflecting the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.\n\n2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.\n\n3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.\n\n4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.\n\n| Model          | Brain Score | Temporal Score |\n|----------------|-------------|----------------|\n| Wav2Vec2.0     | .055        | .00            |\n| BERT           | .055        | .38            |\n| RoBERTa        | .058        | -.18           |\n| GPT2           | .054        | .20            |\n| GPT2-XL        | .054        | .87            |\n| Mamba          | .061        | .93            |\n| Mistral-7B     | .059        | .96            |\n| LLaMA-3.2-3B   | .055        | .94            |\n| LLaMA-3.3-70B  | .058        | .93            |\n\n\n5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). \nThe results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.\n\n### Temporal Score\n| Context length | 1    | 5    | 10   | 50   | 500  | 1000 |\n|----------------|------|------|------|------|------|------|\n| Mamba-1.4b     | .65  | .61  | .82  | .91  | .91  | .93  |\n| LLaMA-3.2-3B   | .19  | .69  | .82  | .89  | .90  | .93  |\n\n### Brain score\n| Context length | 1     | 5     | 10    | 50    | 500   | 1000 |\n|----------------|-------|-------|-------|-------|-------|-------|\n| Mamba-1.4b     | .035  | .048  | .052  | .054  | .055  | .055  |\n| LLaMA-3.2-3B   | .041  | .054  | .057  | .058  | .059  | .058  |\n\n\nWe would like to thank again Reviewer Mo9D for their insightful comments, and hope this further strengthens our contribution."}, {"review_id": "3bOPLW7KPE", "reviewer": "Reviewer_WHgU", "summary": "This paper investigates the alignment between the computations performed by large language models (LLMs) and the processing observed in the human brain during language comprehension. The authors explore whether the sequence of activations in LLMs, as they process natural language, corresponds to the temporal order of brain activity measured using MEG as individuals listen to an audiobook. The study compares neural dynamics and LLM activations across different model architectures and sizes. The study reports strong temporal alignment. Early layers of the LLMs align with early brain responses, while deeper layers align with later brain activity, a pattern consistent across both transformer and recurrent architectures.", "questions": "No additional questions.", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "This is a very timely and interesting study that sheds light on high-level simialrities between human and machine information processing.\n\nThe paper is well written and clearly structured.\n\nI appreciate the broad model comparison but would have loved to see a larger model included to see whether the trend outlined in figure 2c continues for models in the 70B parameter neighborhood.\n\nIt would have been interesting to contrast the processing order of humans and large models with older ones such as RNNs to test whether processing order is simply always like this or whether generative models are truly different and more similar to humans.\n\nSec 3.2 and Fig 4: It would have been interesting to study very long contexts for SSMs such as Mamba that are not, in principle, limited at all in permissible context length.", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer WHgU for their thorough review. We propose to amend our manuscript to address these issues and add five additional sets of results.\n \n**1. New experiment with large language models** \n\nWe now managed to implement our analysis also on model Llama 3 Instruct (70B). The obtained results don’t outperform results from smaller models, which suggests a limit to the scaling trends observed earlier. We now indicate in the discussion section:\n\nThird, our experiments show that this alignment directly depends on (i) context size and (ii) model size – although with a saturation beyond 70B parameters models. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)).\nIn particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. \nIt remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.\n\n- Antonello et al 2023  NeurIPS. Scaling laws for language encoding models in fMRI.\n- Bonnasse-Gahot & Pallier (2024), NeurIPS. fMRI predictors based on language models of increasing complexity recover brain left lateralization\n- Tikochinski et al. (2025), Nature Communications. Incremental accumulation of linguistic context in artificial and biological neural networks\n- d’Ascoli et al. (2024), arXiv. Decoding individual words from non-invasive brain recordings across 723 participants\n- Banville et al. (2025), arXiv. Scaling laws for decoding images from brain activity\n- Caucheteux et al. (2023), Nature Human Behaviour. Evidence of a predictive coding hierarchy in the human brain listening to speech\n\n \n**2. New experiment on Mamba with context size** \n\nWe now extend our context size experiments to Mamba up to n=1,000 words, where temporal score keeps increasing until 0.93. As comparison, we conduct the same analysis for Llama-3.2, where temporal scores also keep increasing across increasingly long contexts. These elements suggest, with MEG, the very long context sizes do still improve the modeling of brain activity for at least some of the most recent models, State-Space and LLMs. Results are all showcased and contextualized alongside other analyses in the table of **4. New experiments**, part 5).\n\n**3. New experiments on old models and RNNs** \n\nWe now extended our pipeline to older language models including GPT-2 small, BERT and Roberta. These older language models present brain-scores comparable to larger, more recent LLMs, but much lower temporal scores. These results are shown and contextualized in the first table of the next section **4. New experiments**, with additional analyses.\n\nWe also analyze State-Space models (Mamba and RecurrentGemma), which can be viewed as types of RNN with a linear evolution of the hidden state at each layer. Additionally, we propose to add the analysis of a recent RNN, xLSTM (Beck et al 2024), in the final version of the manuscript.\n- Beck et al. (2024), NeurIPS. \"xLSTM: Extended Long Short-Term Memory.\"\n\nOverall, the updated experiments do not change but extend further our original conclusion.\n\n**4. New experiments: Additional models and analyses**\n\nTo address reviewers’ comments, we now run five additional analyses:\n\n1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to causal models previously showcased in the manuscript, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.\n\n2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.\n\n3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.\n\n4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.\n\n| Model          | Brain Score | Temporal Score |\n|----------------|-------------|----------------|\n| Wav2Vec2.0     | .055        | .00            |\n| BERT           | .055        | .38            |\n| RoBERTa        | .058        | -.18           |\n| GPT2           | .054        | .20            |\n| GPT2-XL        | .054        | .87            |\n| Mamba          | .061        | .93            |\n| Mistral-7B     | .059        | .96            |\n| LLaMA-3.2-3B   | .055        | .94            |\n| LLaMA-3.3-70B  | .058        | .93            |\n\n\n5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). \nThe results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.\n\n### Temporal Score\n| Context length | 1    | 5    | 10   | 50   | 500  | 1000 |\n|----------------|------|------|------|------|------|------|\n| Mamba-1.4b     | .65  | .61  | .82  | .91  | .91  | .93  |\n| LLaMA-3.2-3B   | .19  | .69  | .82  | .89  | .90  | .93  |\n\n### Brain score\n| Context length | 1     | 5     | 10    | 50    | 500   | 1000 |\n|----------------|-------|-------|-------|-------|-------|-------|\n| Mamba-1.4b     | .035  | .048  | .052  | .054  | .055  | .055  |\n| LLaMA-3.2-3B   | .041  | .054  | .057  | .058  | .059  | .058  |\n\n\n\nWe would like to thank again Reviewer WHgU for their insightful comments, and hope this further strengthens our contribution."}, {"review_id": "DUbxT4btiF", "reviewer": "Reviewer_tqrA", "summary": "The paper generates a number of language encoding models for MEG and then measures what they refer to as \"temporal alignment\" between the models and the brain. In particular they show that earlier layers of language models have earlier alignment peaks (the time at which the encoding performance is highest) than later layers.  They further show that this correspondence is strongest in largest models, scales with context size, and that this temporal alignment score is itself correlated which general encoding performance.", "questions": "Please clarify the method of computing brain-score. The existing paragraph is somewhat inadequate. \n\nSince everything appears to be averaged across sensors, it is unclear to me what the spatial patterns driving this temporal alignment are. It would be nicer to see how this temporal alignment shifts on a per-sensor basis. I think this would be more along the lines of something that I might be interested to see. The general lack of spatial specificity in the results here is something that could be improved. \n\nAre the temporal alignment trends observed in the paper a property of LLMs specifically? Do the same results apply to speech models like Whisper or Wav2Vec?", "limitations": "See above.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "My primary concern here is about the providence of the temporal alignment score that the authors heavily base their paper around. The observation that later layers of LMs model later times in the brain is not really that surprising - the layers are more lexical (and therefore contain information that is earlier in the language processing pipeline) in the early stages of a model and more contextual in the later layers of a model, which is a natural consequence of the hierarchical manner in which LLM architectures build up their representations. In a similar way, auditory cortex (earlier) is more lexical than prefrontal cortex (later).  In addition to being unsurprising, the observation is also not particular novel; a number of the papers that the authors cite contain the same basic trend. The observation that larger models are more \"temporally aligned\" than smaller models is not something I have explicitly seen mentioned before, but it is also not something I find especially notable, mostly because I would expect larger models to have a more finely-tuned and elongated processing pipeline simply by virtue of being less constrained in parameter count. There is also limited-to-no attention given in the paper to the now fairly extensive literature that already exists regarding scaling laws for encoding models, although I consider this a minor point.\n\nAs another minor point, I was somewhat confused about what the brain-score number that is being plotted is. I understand it to be some sort of Pearson correlation metric, however it is unclear how this metric is averaged across sensors. My assumption is that the Pearson correlation is independently computed for each sensor and then averaged across the brain, since this is the general convention in this field, although please correct me if I am wrong. \n\nI do think the paper is very well written and exceptionally clear. I have little to say about obvious scientific mistakes that are made by the authors. Overall, my general opinion is that this paper has no major scientific issues, but that the results are somewhat low impact given that they appear to be something of a replication. I have no problem with replications, but I am also somewhat unsure whether they belong at NeurIPS. I have given the paper a 3.", "was_revised": true, "final_justification": "I am satisfied with the author's responses to my concerns. I am especially appreciative of the extended literature discussion involving whether these results should be expected based on prior literature, as well as the promise to include per-sensor results and added scaling results. Contingent on these changes being added to the paper, I am willing to change my score to a 4.", "rebuttal": "We thank Reviewer tqrA for their thorough review. We propose to amend several sections of the manuscript to address these issues and add five additional sets of results.\n\n**1. Contribution**\n\nWe agree that several studies had shown a temporal alignment between brain and LLM before – although it is here demonstrated with 20x more data – 10h per participant, compared to 30 min per subject – and with minimal preprocessing – e.g. no localizer or analyses restricted to clusters or regions of interest – compared to both Li et al. (2023) and Goldstein et al. (2022).\n\n- Li et al. (2023), Nature Neuroscience. Dissecting neural computations in the human auditory pathway using deep neural networks for speech\n- Goldstein et al. (2022), BioRxiv. Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain\n\nRather, we start from this independent replication to :\n\n**(1)** Systematically compare 23 diversified language models to brain activity.\n\n**(2)** Use this benchmark to identify the *factors* that cause this temporal alignment (architecture, model size, training quantity, directionality, context length).\n\n**(3)** Successfully test the correlation between higher brain scores and higher temporal alignment, identifying conditions where brain score plateaus while temporal score continues to rise (cf \"6. New Experiments\" for further details).\n\nWhile we believe this contribution is accurately stated in the introduction and discussion, we do agree that the abstract was insufficiently clear on our exact contribution. We thus propose to amend it as follows:\n\n- “However, *whether* this representational alignment arises from a similar sequence of computations remains elusive. “ -> “However, *whether and why* this representational alignment arises...” \n- “Our analyses *reveal* that LLMs and the brain generate representations in a similar order” -> “Our analyses *confirm* ...”\n- “Overall, the alignment between LLMs and the brain provides novel elements supporting a partial convergence between biological and artificial neural networks.” -> “Overall, this study *clarifies the factors underlying the partial convergence* between ...”\n\n**2. Credit to scaling laws studies**\n\n We agree that we insufficiently credited previous work on scaling laws. We now amended the discussion as follows:\n\nThird, our experiments show that this alignment directly depends on (i) context size and (ii) model size. These MEG results extend previous works on scaling laws in neuro-ai (Antonello et al. (2023), Bonnasse-Gahot & Pallier (2024), Tikochinski et al. (2025), d’Ascoli et al. (2024), Banville et al. (2025), Caucheteux et al. (2023)).\nIn particular, Antonello et al. (2023) and Bonnasse-Gahot & Pallier (2024) showed that LLMs that best predict fMRI responses to natural speech are those with the largest amount of parameters. In parallel, Tikochinski et al. (2025) and Caucheteux et al. (2023) showed that context size improved the alignment between brain and fMRI. Here, we show the effect of context size and model size to be nearly loglinearly correlated, hence pointing to diminishing returns, if not a plateau. Together, these findings clarify the specific conditions required for brain-like representations and computations to emerge. It remains unclear, however, whether context and size act directly on the alignment, or are confounded by other uncontrolled variables, such as linguistic performance. For example, in Appendix E, we find that performance is correlated with temporal alignment for specific conditions – LLM with increasing context lengths – but not others – LLMs with increasing sizes. Disentangling the causal chain that links these factors remains a major research avenue.\n\n- Antonello et al 2023  NeurIPS. Scaling laws for language encoding models in fMRI.\n- Bonnasse-Gahot & Pallier (2024), NeurIPS. fMRI predictors based on language models of increasing complexity recover brain left lateralization\n- Tikochinski et al. (2025), Nat. Comm. Incremental accumulation of linguistic context in artificial and biological neural networks\n- d’Ascoli et al. (2024), arXiv. Decoding individual words from non-invasive brain recordings across 723 participants\n- Banville et al. (2025), arXiv. Scaling laws for decoding images from brain activity\n- Caucheteux et al. (2023), Nat. Hum. Behav. Evidence of a predictive coding hierarchy in the human brain listening to speech\n\n**3. Expected results**\n\nWe agree that models with many more layers are likely to have an ‘elongated processing pipeline’. However, this does not affect our analysis, because we sample the same number of layers (n=9, linearly space between 10% and 90% of the architecture) for each model. \nIn addition, to clarify that our results may not be fully expected by the community, we now highlight papers that emphasize the discrepancies between LLMs and the Brain:\n\nYet, their architecture, sensory modality, learning objective, and training regime are remarkably disjoint Romeo et al. (2018), Dupoux (2018), Evanson et al. (2025), Ghanizadeh & Dousti (2025). Furthermore, specific features like syntactic structures and semantic roles may be represented fundamentally differently (Fodor et al. (2025), Kryvosheieva & Levy (2024)). That partially converging computational paths arise despite these differences is thus all the more surprising, and highlights the necessity to clarify the computational principles that lead language processing to be partially shared between biological and artificial systems. \n\n- Romeo et al. (2018), JNeuro. Language exposure relates to structural neural connectivity in childhood\n- Dupoux (2018), Cognition. Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner\n- Evanson et al. (2025), Manuscript. Emergence of language in the developing brain.\n- Ghanizadeh & Dousti (2025), arXiv. Towards data-efficient language models: A child-inspired approach to language learning\n- Fodor et al. (2025), CL, Compositionality and Sentence Meaning: Comparing Semantic Parsing and Transformers on a Challenging Sentence Similarity Dataset\n- Kryvosheieva & Levy (2024), arXiv. Controlled evaluation of syntactic knowledge in multilingual language models\n\n**4. Brain-score averaging and per sensor analysis**\n\nThe brain-score is indeed the result of Pearson scores independently computed for each sensor, then averaged across the brain. We will state this averaging in the Methods section.\n\nAdding the per-sensor analyses is indeed a good idea. We now have these encoding results available and will add them to the manuscript. Note that the low spatial resolution of MEG remains challenging to explore this issue. \n\n**5. New experiments beyond LLMs: Wav2Vec2**\n\nNote that we already use two types of architectures: State Space models (can be thought of as RNNs with a linear dynamics of their hidden state) and Transformers. \n\nWe now include results for speech model Wav2vec2.0, bidirectional, alongside bidirectional LLMs BERT and RoBERTa. All three show brain-scores comparable to larger, more recent and causal LLMs, but much lower temporal scores. These results are shown in the first table of the next section, alongside additional analyses.\n\n**6 New experiments: Additional models and analyses**\n\nTo address reviewers’ comments, we now run five additional analyses:\n\n1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to the causal models previously presented, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.\n\n2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.\n\n3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.\n\n4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.\n\n| Model        | Brain | Temp |\n|--------------|-------|------|\n| Wav2Vec2.0   | .055  | .00  |\n| BERT         | .055  | .38  |\n| RoBERTa      | .058  | -.18 |\n| GPT2         | .054  | .20  |\n| GPT2-XL      | .054  | .87  |\n| Mamba        | .061  | .93  |\n| Mistral-7B   | .059  | .96  |\n| LLaMA-3.2-3B | .055  | .94  |\n| LLaMA-3.3-70B| .058  | .93  |\n\n5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (SSM). \nThe results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.\n\n### Temporal Score\n| Context length | 1    | 5    | 10   | 50   | 500  | 1000 |\n|----------------|------|------|------|------|------|------|\n| Mamba-1.4b     | .65  | .61  | .82  | .91  | .91  | .93  |\n| LLaMA-3.2-3B   | .19  | .69  | .82  | .89  | .90  | .93  |\n\n### Brain score\n| Context length | 1     | 5     | 10    | 50    | 500   | 1000 |\n|----------------|-------|-------|-------|-------|-------|-------|\n| Mamba-1.4b     | .035  | .048  | .052  | .054  | .055  | .055  |\n| LLaMA-3.2-3B   | .041  | .054  | .057  | .058  | .059  | .058  |\n\n\nTogether these MEG results are consistent with the overall trends observed with fMRI (Antonello, et al (2024). \n\nWe thank Reviewer tqrA again for these constructive criticisms which help strengthen this study, and better articulate how it fits the current state-of-the-art."}, {"review_id": "qRtzYsKR2L", "reviewer": "Reviewer_eQDJ", "summary": "The paper focuses on the temporal alignment of LLMs to human brain MEG responses, i.e., later model layers show greatest representational similarity to neural responses at greater times past word onset compared to earlier model layers (temporal alignment). They conduct many analyses to study how temporal alignment of LLMs is related to overall brain alignment, context size, and model size.", "questions": "None", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Interesting finding that LLMs exhibit temporal alignment to human brains. \n- Interesting findings that temporal alignment is correlated to overall brain alignment, context size, and model size\n\nWeaknesses:\n- Pearson correlation of roughly 0.06 as the maximum across different models is quite low, it would be helpful if the paper clarified what the noise ceiling is for the data, to make it easier to understand how well these models are predicting the neural responses.\n- \"To ensure that the strong Temporal scores we find are not led by this parallel between predicting of next word by the model and hearing of next-word by the human, we study the impact of next words’ predictability on temporal alignment.\" However, I think one alternative hypothesis is that the model's next-word prediction aligns with the humans' next-word prediction, not next-word hearing. Since deeper model layers build up representations for next-word prediction, it might align more to next-word prediction that humans are doing, i.e., it might simply suggest a similar objective of next-word prediction, not a similar order of computations between LLMs and humans. Having a similar objective would also explain that there is no significant layer-wise difference of T-max between \"most expected\" and \"least expected\" quartiles, because humans might also find the model's \"most expected\" words to be more expected and the model's \"least expected\" words to be less expected.", "was_revised": true, "final_justification": "Weakness 1 -- partially addressed, I think the Pearson scores are still low, lower than the numbers from prior works they cited. Though, it is helpful that they contextualized that MEG Pearson scores are low in general.\n\nWeakness 2 -- mostly addressed by their comment.", "rebuttal": "We thank Reviewer eQDJ for their thorough review. We propose to amend our manuscript to address these issues and add five additional sets of results.\n\n**1. Pearson correlation**\n\n These Pearson R scores are indeed low, but remain in the range typically observed with electrophysiology. For example, Caucheteux and King (2022) report, from MEG results, brain-scores around R=0.07, peaking at R=0.13 for the best channels. Goldstein et al (2023) report, from intracranial data R=0.2 on the electrodes specifically selected to show language responses. These values are low because (1) the signal-to-noise ratio in electrophysiology is low and (2) there is no denoising procedure (e.g. averaging across repetitions, normalizing by estimated noise ceiling etc). We will add a figure with confidence intervals to illustrate that in spite of being small, these effect sizes are highly reliable. \n- Caucheteux & King (2022), Communications Biology. Brains and algorithms partially converge in natural language processing.\n- Goldstein et al. (2022), BioRxiv. Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain\n\n\n**2. Shared objective vs representations**\n\n This is an interesting remark. To address it, we now amended the discussion as follows:\n\nAs previously reported (Jain et al, (2020), Caucheteux and King (2022), Toneva and Wehbe (2019), Antonello et al. (2023)), the best brain scores are achieved in the intermediary layers. This result suggests that the intermediary representations (layer=60% of architecture) – as opposed to the input and the prediction – are best aligned between brains and LLMs, suggesting that these two systems don’t just share the same objective, but the same computational path too.\n\n- Jain et al. (2020), NeurIPS. \"Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech.\"\n- Caucheteux & King (2022), Communications Biology. \"Brains and algorithms partially converge in natural language processing.\"\n- Toneva & Wehbe (2019), NeurIPS. \"Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain).\"\n- Antonello et al 2023  NeurIPS. \"Scaling laws for language encoding models in fMRI.\"\n\n\n**3. New experiments: Additional models and analyses**\n\n\nTo address reviewers’ comments, we now run five additional analyses:\n\n1. To assess the impact of contextual directionality, we compare two bidirectional LLMs (BERT and RoBERTa) to causal models previously showcased in the manuscript, which reflect more closely the brain’s causal processing of language. While brain scores are comparable, the temporal scores of bidirectional models are substantially lower than those of causal LLMs.\n\n2. We also analyse a speech model, Wav2vec2, bidirectional as well, and compare this model to the other models presented. While brain scores are comparable, the temporal scores of bidirectional Wav2vec2.0 is also substantially lower than those of causal LLMs.\n\n3. We extend our analyses to a “large LLM”, i.e. Llama 3.3 70B. The results suggest that there is no major improvement as compared to smaller LLMs e.g. Llama 3.2 3B, hence pointing to a plateau effect of scaling laws identifiable with MEG.\n\n4. We now added GPT2-S (137M) compared to GPT-XL (1.6B) to study scaling in this older family of models.\n\n| Model          | Brain Score | Temporal Score |\n|----------------|-------------|----------------|\n| Wav2Vec2.0     | .055        | .00            |\n| BERT           | .055        | .38            |\n| RoBERTa        | .058        | -.18           |\n| GPT2           | .054        | .20            |\n| GPT2-XL        | .054        | .87            |\n| Mamba          | .061        | .93            |\n| Mistral-7B     | .059        | .96            |\n| LLaMA-3.2-3B   | .055        | .94            |\n| LLaMA-3.3-70B  | .058        | .93            |\n\n\n5. Finally, we now added analyses on even longer context sizes, with up to 1000 words on both Llama-3.2 and Mamba (State-Space Model). \nThe results indicate that up to 1000 words, longer contexts still lead to an increase of temporal alignment, even when the brain-score plateaus.\n\n### Temporal Score\n| Context length | 1    | 5    | 10   | 50   | 500  | 1000 |\n|----------------|------|------|------|------|------|------|\n| Mamba-1.4b     | .65  | .61  | .82  | .91  | .91  | .93  |\n| LLaMA-3.2-3B   | .19  | .69  | .82  | .89  | .90  | .93  |\n\n### Brain score\n| Context length | 1     | 5     | 10    | 50    | 500   | 1000 |\n|----------------|-------|-------|-------|-------|-------|-------|\n| Mamba-1.4b     | .035  | .048  | .052  | .054  | .055  | .055  |\n| LLaMA-3.2-3B   | .041  | .054  | .057  | .058  | .059  | .058  |\n\n\n\n\nWe would like to thank again Reviewer eQDJ for their insightful comments, and hope this further strengthens our contribution."}]

Author rebuttal:

Meta-review: This paper investigates similarity between human's brain and LLMs by calculated the correlation between activities of LLMs and biological brains when a long text and its corresponding audio are input. The experiments are conducted thoroughly across different type and size of LLM models, and the authors found an interesting phenomenon such that there are strong correlation between LLMs and human's brain and the temporal order of correlation is well aligned with the order of layers.   
  
The reviewers are satisfied with the writing quality and the scientific findings of this paper. The hypothesis the authors made is well supported by the detailed and thorough experiments.
Decision: Accept (spotlight)