Mapping Brains with Language Models: A Survey (2023)

Abstract: Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models. In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics. How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions? Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative. We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.

## introduction advances in neuroimaging technologies have made it possible to better approximate the spatiotemporal profile of the computations responsible for language in the brain @xcite . at the same time, advances in natural language processing have produced language models (lms) with high performance in many tasks @xcite . this progress has motivated scientists to start using state-of-the-art lms to study neural activity in the human brain during language processing @xcite @xcite @xcite . conversely, it has also prompted nlp researchers to start using neuroimaging data to evaluate and improve their models @xcite @xcite @xcite . at the conceptual core of these studies lies the suggestion that representations extracted from nlp models can (partially) explain the signal found in neural data. these representations can be based on co-occurrence counts @xcite @xcite or syntactic and discourse features @xcite . later studies use dense representations such as word embeddings @xcite @xcite and recurrent neural networks to extract contextual stimuli representations @xcite @xcite . more recently, transformer-based architectures have been shown to align even better with neural activity data @xcite @xcite . such work shows that lms can be trained to induce representations that are seemingly predictive of neural recordings or features thereof. however, pursuing the literature, it quickly becomes clear that these papers all rely on different experimental protocols and different metrics @xcite @xcite . so questions are: how much evidence has really been accumulated in support of structural similarities between brains and lms? and more importantly, what exactly, if anything, drives this alignment, and what are we to understand from it? after gathering all the studies, we examine their evaluation metrics and their interrelationships, providing discussions on the corresponding findings. contributions our study provides four major contributions for the wider nlp audience: (a) a detailed review of the literature on mappings between fmri/meg recordings and representations from language models; (b) an overview of the datasets and mapping methods; (c) an analysis of the evaluation setups that have been used to link neural signals with language models and how they relate; (d) a discussion of what drives this representational alignment and what we, as a field, can make of it going forward. terminology first, a brief note on terminology: neural response measurements refer to recordings of the brain activity of subjects reading or listening to language. we focus on (a) functional magnetic resonance imaging (fmri), which measures neuronal activity via blood oxygenation leveldependent contrast and has a high spatial resolution but poor temporal resolution (3-6s) and (b) magnetoencephalography (meg), which involves the measurement of the magnetic field generated by the electrical activity of neurons in the cortex, providing a more accurate resolution of the timing of neuronal activity. voxels refer to the smallest unit of data in a neuroimage, being the three-dimensional equivalent of a pixel in two-dimensional images @xcite . finally, we use brain decoding to refer to predicting stimuli from brain responses (i.e. reading the brain). brain encoding will then refer to predicting brain responses from stimuli. whereas decoding models serve as a test for the presence of information in neural responses, encoding models can be interpreted as process models constraining brain-computational theories @xcite . ## datasets to infer a mapping between language models and brains, researchers rely on datasets in which brain activity is recorded in response to linguistic stimuli. in some studies, the stimuli are single words @xcite or sentences displayed on a screen @xcite . in others, participants read longer stories @xcite @xcite or listened to speech or podcasts @xcite . in this section, we survey work in which neural responses are predicted from linguistic representations. such work typically aims to shed light on how language functions in the brain. one of the earliest studies exploring the mapping between brain and language representations is by @xcite , who trained a linear regression model on a set of word representations extracted from 60 nouns using 115 semantic features based on cooccurrence statistics, to predict the corresponding fmri representations of the same nouns. they use pair-wise matching accuracy evaluation, extracting two words w and w ′ for evaluation, and showed that the predicted fmri for a word w was closer to the real fmri image for w than to the real fmri image for w ′ , at above-chance levels. @xcite also report percentile rank results, ranking predicted fmri images by similarity with the real image of w. we discuss how the metrics relate in §6. the dataset of @xcite is also used by @xcite , who extract linguistic features from part-of-speech taggers, stemmers, and dependency parsers, showing that dependency parsers are the most successful in predicting brain activity. they also use leave-2-out pair-matching as their performance metric. later on, @xcite moved on to predicting brain activation patterns for entire sentences rather than for isolated words. they recorded fmri neural response measurements while participants read a chapter from harry potter and the sorcerer's stone, then extracted a set of 195 features for each word (ranging from semantic, syntactic properties to visual and discourse-level features) to train a comprehensive generative model that would then predict the time series of the fmri activity observed when the participants read that passage. leave-2-out pair-matching accuracy is used for evaluation. @xcite , in contrast, use fmri recordings of participants listening to spoken narrative stories, representing each word in the corpus as a 985-dimensional vector encoding semantic information driven by co-occurrence statistics. they train per-voxel linear regression models and evaluate their predicted per-word fmri images by their per-voxel pearson correlation with the real fmri images, showing that 3-4 dimensions explained a significant amount of variance in the fmri data. @xcite are among the first to use neural language models, using recurrent models to compute contextualized embeddings, hidden state vectors of previous words, and word probabilities. they run their experiments of meg recordings of participants reading harry potter, obtained in a follow-up study to @xcite . from the three sets of representations, they then train linear regression models to predict the meg vectors corresponding to each word, and the regression models are then evaluated by computing pair-matching accuracy. similarly, @xcite evaluates static word embeddings on the data from @xcite , learning linear transformation from word embeddings into an fmri vector space. the predictions are evaluated through mean squared error (mse). jain and huth (2018) evaluate recurrent language models against the fmri dataset from huth et al. @xcite . their findings show that contextual language model representations align significantly better (to brain response) compared to static word embedding models. their evaluation metric is the total sum of explained variance 1 following this, @xcite use attention-based transformer language models for brain mapping. they finetune bert @xcite to predict neural response measurements from the harry potter dataset, showing that the fine-tuned models have representations that encode more brain-activity-relevant language information than the non-finetuned models. they rely on pair-matching accuracy as their performance metric. as in søgaard (2016), zhang et al. (2020) map static word embeddings into the vector space of the neural response measurements (fmri). they introduce a new dataset of such measurements from subjects listening to natural stories. they rely on explained variance as their performance metric. @xcite evaluate word and sequence embeddings from 4 recurrent and attention1 the squared pearson correlation coefficient. we will not distinguish between studies using pearson correlation and studies using explained variance. see appendix a.2. based transformer language models, using the harry potter fmri dataset. they evaluate models across layers, context lengths, and attention types, using pairwise matching accuracy as their performance metric. in a later study, @xcite induce compositional semantic representations of "supra-word meaning" which they then use to predict neural responses across regions of interest, evaluating their models using pearson correlation. also using the harry potter data, @xcite evaluate five models, one static and four contextualized, relying on a variant of representational similarity analysis @xcite . the results suggest that models provide representations of local contexts that are well-aligned to neural measurements. however, as information from further away context is integrated by the models, representations become less aligned to neural measurements. in a large-scale study, @xcite examine the relationships between 43 diverse stateof-the-art neural network models (including embedding models, recurrent models, and transformers) across three datasets (two fmri, one electrocardiography). they rely on a metric they term brain score which involves normalising the pearson correlation by a noise ceiling. their results show that transformer-based models perform better than recurrent or static models, and larger models perform better than smaller ones. similarly, in @xcite , the @xcite fmri and meg datasets are used to compare a variety of transformer architectures. they study how architectural details, training settings, and the linguistic performance of these models independently account for the generation of brain correspondent representations. the results suggest that the better language models are at predicting words from context, the better their activations linearly map onto those of the brain. @xcite evaluate three static and five attention-based transformer models, in combination with four fine-tuning tasks and two machine translation models. they train linear regression models to evaluate their word-level representations against a new fmri dataset from participants listening to podcast stories. they find a low-dimensional structure in language representations that can predict brain responses. in a similar setting, antonello and huth (2022) examine why some features fit the brain data better arguing that the reason is that they capture various linguistic phenomena. reddy and wehbe (2021) evaluate syntactic features in conjunction with bert representations, finding that syntax explains additional variance in brain activity in various parts of the language system, even while controlling for complexity metrics that capture processing load. in a series of studies @xcite aw and toneva (2023) extract feature representations from four attention-based transformer models. they evaluate the impact of fine-tuning on the booksum dataset @xcite . all models are used to predict brain activity on the harry potter data. pairwise matching accuracy and pearson correlation are their performance metrics. @xcite focus more narrowly on variants of gpt-2, showing that improvements in alignment with brain recordings are probably not because of the next-word prediction task or wordlevel semantics, but due to multi-word semantics. their reported metric is pearson correlation. ## performance metrics we present the evaluation metrics used in the above studies and discuss how they relate. see authors data e/d acc p/b rank mse rsa cos sim p@k mitchell et al. (2008) 1 e ✓ ✓ ✓ murphy et al. (2012) 1 e ✓ wehbe et al. (2014a,b) 2,3 e ✓ ✓ huth et al. (2016) 4 e ✓ søgaard (2016) 2 e ✓ pereira et al. (2018) 5 d ✓ ✓ jain and huth (2018) 4 e ✓ toneva and wehbe (2019) 2 e ✓ gauthier and levy (2019) 5 d ✓ ✓ ✓ minnema and herbelot (2019) 5 d ✓ ✓ ✓ ✓ schwartz et al. (2019) 3 e ✓ abnar et al. (2019) 2 e ✓ zhang et al. (2020) 7 e ✓ ✓ schrimpf et al. (2021) 5 e ✓ ✓ abdou et al. (2021) 2,5 d ✓ reddy and wehbe (2021) 2 e ✓ antonello et al. (2021) 9 e ✓ antonello and huth (2022) 9 e ✓ caucheteux et al. (2021, 2022b,a) 8 e ✓ caucheteux and king (2022) 6 e ✓ zou et al. (2022) 1,2 d ✓ oota et al. (2022a,b,c) 8 e ✓ ✓ pasquiou et al. (2022) 10 e ✓ toneva et al. (2022a) 2 e ✓ merlin and toneva (2022) 2 e ✓ pascual et al. (2022) 5 d ✓ ✓ aw and toneva (2023) 2 e ✓ ✓ percentile rank was first used for encoding @xcite , but can also be used for decoding @xcite @xcite . in encoding, the predicted brain image for w is ranked along the predicted images for a set of candidate words w ′ by their similarity to the real (ground truth) image for w. the average rank is then reported. for decoding, they rank word vectors rather than neural response images. note the similarity metric is unspecified, but typically cosine distance is used. mean squared error, the average of the squared differences between word vectors and neural responses, was first used for encoding in søgaard (2016) on a held-out test split. it was also used by @xcite . representational similarity analysis (rsa) was introduced in @xcite as a non-parametric way to characterize structural alignment between the geometries of representations derived from disparate modalities. rsa abstracts away from activity patterns themselves and instead computes representational similarity matrices (rsms), which characterize the information carried by a given representation method through global similarity structure. a rank correlation coefficient is computed between rsms derived from the two spaces, providing a summary statistic indicative of the overall representational alignment between them. being non-parametric, rsa circumvents many of the various methodological weaknesses (such as over fitting, etc.). @xcite @xcite , and @xcite apply (variations of) rsa to investigate the relations between different model components, and then to study the alignment of these components with brain response. cosine similarity was used in @xcite to select between the candidate images in pairwise matching accuracy, as well as in percentile rank and rsa, but the raw cosine similarities between predicted and real images or embeddings can also be used as a metric. @xcite use this metric to quantify how close the predicted word vectors are to the target. finally, @xcite use precision@k, a standard metric in other mapping problems, e.g., cross-lingual word embeddings @xcite . comparisons most metrics are used to evaluate both encoding and decoding models (pairwise matching accuracy, pearson correlation, percentile rank, mse, rsa, cosine distance). results for two of the most widely used metrics -pairwise matching accuracy 5 and percentile rank -tend to be around 0.7-0.8 with generally better results for more recent architectures and larger lms. to draw conclusions across studies relying on different metrics, we need to investigate which metrics are more conservative, and how different metrics relate. pairwise matching accuracy vs. pearson correlation it seems that pairwise matching accuracy tends to increase monotonically with pearson correlation. consider three sets of distances over corresponding point sets, a, b, and c. if a and b are more strongly linearly correlated than a and c, under an optimal linear mapping ω (minimizing point-wise squared error distance), e[(a -bω) 2 ] > e[(a -cω) 2 ]. even in this conservative setting in our synthetic experiments in appendix a.1, the correlation between matching accuracy and percentile rank was very high, ~0.9. pairwise matching accuracy vs. percentile rank both metrics have random baseline scores of 0.5, and they will converge in the limit. if a has a percentile rank of p in a list a, it will be higher than a random member of a p percent of the time. in our experiments in appendix a.1, the correlation converges toward 1.0, with values consistently higher than 0.8 for @xmath0. pairwise matching accuracy vs. precision@k are also positively correlated. perfect score in one entails perfect score in the other, but precision@k can of course be very small for very high values of pairwise matching accuracy (especially if the set of candidate words is big). conversely, we can have 5 when discriminating averages over 20 images @xcite , scores are naturally lower. saturation for high values of k, because matching accuracies higher than n-k n will mean near-perfect precision@k scores. in practice, precision@k (for low values of k) will be much more conservative, however. the correlation coefficient for @xmath1.1) tends to lie around 0.7. relative strength pairwise matching accuracy is a relatively permissible performance metric. to see this, consider the scenario in which all target words can be divided into two equal-sized buckets based on word length (number of characters). say the neural responses capture nothing but this binary distinction between long and short words, but do so perfectly. moreover, our mapping method, e.g., linear regression, learns this from training data. now, from this alone, the pairwise matching accuracy will converge toward µ = 0.75, since our model will do perfectly (1.0) on half of the data, and exhibit random performance (0.5) on the other half. if the neural responses tracked word length (and not just the distinction between short and long words), performance would be even better. in other words, pairwise matching accuracy scores around 0.7-0.8 (observed in the studies above) may only reflect very shallow processing characteristics. the fact that @xcite only observed good results with this metric, led them to adopt a rather critical stance, for good reasons. other metrics are clearly more conservative. for a set of n candidate words, a random mapping will induce a precision@1-score of 1 n . while hubs may inflate scores for larger values, the metric is extremely conservative for small values of k. however, only @xcite use this metric, and they modify the experimental protocol substantially, making the task much easier by providing additional input to a non-linear model. the small improvement from adding neural response input is interesting, but could potentially be explained by shallow processing characteristics. they argue that analogy testing would provide a better evaluation protocol: one would ideally use standard metrics such as semantic relatedness judgment tasks, analogy tasks, etc. @xcite this is not possible due to the limited vocabulary sizes of the available brain datasets such evaluation is possible on small scale, though, and increasingly larger fmri datasets are becoming available (see above). @xcite have identified analogical reasoning in fmri brain activation spaces. the analogies are computed using vector offset and probe the systematicity of how semantic relations are encoded. if a model encodes the capital-of relation systematically, we can retrieve the capital of germany by subtracting the fmri vector for 'paris' from the sum of our the fmri vectors for germany and france. this is the same kind of analogical reasoning found in language models @xcite . @xcite show that the more language models satisfy analogies, the more isomorphic they are. so far, it seems that, with the possible exception of @xcite , there is little evidence for structural similarities, beyond what could be induced by shallow processing characteristics, but what about all the studies that report strong pearson correlations? per-voxel correlation coefficients are low on average, but across the above studies, typically only around 4-40% of the voxels exhibit significant correlations @xcite . since these correlations have been replicated across different datasets, they are generally not disputed, but could still reflect rather shallow processing characteristics. on a more positive note, several studies show that larger (and better) language models align better with neural response measurements @xcite . this suggests that language models in the future may align even better with such measurements, possibly reflecting properties of deep processing. such correlations with model quality and size are positive, making the results reported above more credible. generally, the conclusions we can draw from the above studies are somewhat vague. there are two reasons for this: (i) past studies have relied on permissible (pairwise matching accuracy) and ambiguous (pearson correlation) performance metrics; and (ii) past studies have relied on small-sized datasets. we believe that this calls for a meta-analysis of the above studies. to provide grounds for such a meta-analysis, we have in this section taken steps to compare the metrics used in these studies. we leave it for future work to explore various ways effect sizes can be computed across these studies. ## discussion many studies, summarized above, aim to compare language model representations with neural response measurements using linear mapping mod-els. our main reason to focus on linear mapping models is that they quantify the degree of structural similarity (isomorphism). overall, results suggest that structural similarities between language models and neural responses exist. furthermore, there is good evidence that alignment has correlated positively with model quality and model size, suggesting a certain level of convergence as language models improve. what drives alignment? is alignment driven by deep processing characteristics or by shallow textual characteristics? classical candidates for shallow ones would be word length, frequency, regularity, and part of speech. @xcite , for example, only controlled for part of speech. some authors have presented results to suggest that alignments are driven by syntactic or semantic factors @xcite @xcite , whereas others have claimed some similarities reflect semantic phenomena @xcite . others suggest that alignments reflect deeper similarities between model objectives and predictive processing in human brains @xcite @xcite , but see @xcite for a critical discussion of such work. linguistically-transparent models that allow for a principled decomposition of a model's components into smaller linguistically meaningful units and models that move towards possible neurobiological implementations of neural computation are likely to be key for answering this question @xcite . given the plethora of interpretability methods recently developed, however, we believe that even models which are not intrinsically interpretable can be useful toward this goal. do some models align better? most studies observe that better and larger, contextual models align better with neural responses @xcite . other improvements include fine-tuning on specific tasks @xcite . @xcite outline the impact of model training choices. what metrics? the inconsistent use of performance metrics makes it hard to compare and interpret the results reported in the literature @xcite . we have shown that some metrics are perhaps too permissible to detect structural sim-ilarities between language models and neural responses. we have argued that precision@k is more conservative than most other metrics. minnema and herbelot (2019) have proposed using analogy scores. in the limit (given sufficient analogies), perfect analogical accuracy implies isomorphism @xcite . so do perfect precision@1 and perfect rsa scores. we, therefore, propose giving priority to these performance metrics, not to conflate shallow processing characteristics with deeper, more semantic properties. ## conclusions we surveyed work on linear mappings between neural response measurements and language model representations, with a focus on metrics. in particular, we surveyed a broad range of 30 studies spanning across 10 datasets and 8 metrics. by examining the metrics, and relating them to one another, we attempt to critically assess the accumulated evidence for structural similarity between neural responses and language model representations. we find that similarities with existing models are limited to moderate, and there is a possibility they might be explained by shallow processing characteristics since there is no standardised methodology for employing controls, but also that positive correlations with model quality and size suggest that language models may exhibit deeper similarities with neural responses in years to come.