Speech language models lack important brain-relevant semantics (2024)

Abstract: Despite known differences between reading and listening in the brain, recent work has shown that text-based language models predict both text-evoked and speech-evoked brain activity to an impressive degree. This poses the question of what types of information language models truly predict in the brain. We investigate this question via a direct approach, in which we systematically remove specific low-level stimulus features (textual, speech, and visual) from language model representations to assess their impact on alignment with fMRI brain recordings during reading and listening. Comparing these findings with speech-based language models reveals starkly different effects of low-level features on brain alignment. While text-based models show reduced alignment in early sensory regions post-removal, they retain significant predictive power in late language regions. In contrast, speech-based models maintain strong alignment in early auditory regions even after feature removal but lose all predictive power in late language regions. These results suggest that speech-based models provide insights into additional information processed by early auditory regions, but caution is needed when using them to model processing in late language regions. We make our code publicly available. [https://github.com/subbareddy248/speech-llm-brain]

## introduction an explosion of recent work that investigates the alignment between the human brain and language models shows that text-based language models (e.g. gpt*, bert, etc.) predict both text-and speechevoked brain activity to an impressive degree (text: @xcite @xcite @xcite @xcite ; speech: @xcite @xcite @xcite @xcite @xcite ). this observation holds across late language regions, which are thought to process both text-and speechevoked language @xcite , but also more surprisingly across early sensory cortices, which are shown to be modality-specific @xcite . since text-based language models are trained on written text @xcite @xcite , their impressive performance at predicting the activity in (also referred to as alignment with) early auditory cortices is puzzling. this raises the question of what types of information underlie the brain alignment of language models observed across brain regions. in this work, we investigate this question via a direct approach (see fig. 1 for a schematic). for a number of low-level textual, speech, and visual features, we analyze how the alignment between brain recordings and language model representations is affected by the elimination of information related to these features. we refer to this approach as direct, because it estimates the direct effect of a specific feature on brain alignment. this approach is in contrast to an indirect approach, which first estimates the brain alignment of a model and then independently examines the model's performance on a natural language processing task (e.g. approaches presented by @xcite ; @xcite ). in contrast, our approach can directly estimate the impact of a specific feature on the alignment between the model and the brain recordings by observing the difference in alignment before and after the specific feature is computationally removed from the model representations. we further contrast our findings with speechbased language models, which would be expected to predict speech-evoked brain activity better, provided they model language processing in the brain well. for this purpose, we present a systematic study of the brain alignment across two popular fmri datasets of naturalistic stories (1-reading, 1listening) and different natural language processing models (text vs. speech). we focus on three popular text-based language models (bert (kenton and toutanova, 2019), gpt2 @xcite , and flan-t5 @xcite ) and two speechbased models (wav2vec2.0 @xcite and whisper @xcite )-which have been studied extensively in the nlp-brain alignment literature @xcite @xcite @xcite @xcite . the fmri recordings are openly available @xcite and correspond to 6 participants reading and listening to the same naturalistic stories. this dataset is unique in presenting the same naturalistic stimuli under two sensory modalities. ensuring the same stimuli across sensory modalities is crucial, as differences in stimuli could otherwise account for any observed differences in brain recordings across modalities. we test how eliminating a comprehensive set of low-level textual (number of letters, number of words, word length, etc.), speech (number of phonemes, fbank, mfcc, mel, articulation, phonological features, etc.), and visual features (motion energy) from model representations affects alignment with brain responses. our direct approach leads to three important conclusions: (1) the surprising alignment of models with sensory regions corresponding to the incongruent modality (i.e. text models with auditory regions and speech models with visual regions) is entirely due to low-level stimulus features that are correlated between text and speech (e.g. number of letters and number of phonemes); (2) models align differently with their corresponding sensory regions, and this difference is not explained by low-level stimulus features. text models ex-hibit comparable alignment with visual and auditory regions, entirely explained by low-level textual features. in contrast, speech models display significantly greater alignment with auditory regions than visual regions, and this difference cannot be entirely accounted for by the comprehensive set of low-level stimulus features studied. this underscores the capacity of speech-based language models to capture additional features crucial for early auditory cortex, suggesting their potential in enhancing our understanding of this brain region; ## related work our work is most closely related to that of @xcite , who propose the direct residual approach to study the supra-word meaning of language by removing the contribution of individual words to brain alignment. more recent work uses the same residual approach to investigate the effect of syntactic and semantic properties on brain alignment across layers of a text-based language model @xcite . we complement these works by studying the impact of a wide range of low-level features on brain alignment, and by examining this effect on alignment with both text and speech models. other direct approaches have also been proposed in the literature. most notably, work by @xcite studies the impact of removing information related to word embeddings directly from brain responses on a downstream task. conceptually, the results obtained from this approach and ours should be similar because the feature is completely removed from either the brain alignment input, target, or both, and thus cannot further impact the observed alignment. however, in practice, brain recordings are noisier than model representations, making it more challenging to estimate the removal regression model, especially with a limited sample size. therefore, in our work, we opt to remove features from the model representations rather than from the brain recordings. methods based on variance partitioning @xcite offer another direct approach. our approach is complementary and easily allows relating the feature effect to an estimated noise ceiling, given by cross-subject prediction. our work also relates to a growing literature that investigates the alignment between human brains and language models. a number of studies have used text-based language models to predict both text-evoked and speech-evoked brain activity to an impressive degree @xcite @xcite @xcite @xcite @xcite @xcite @xcite . similarly, the recent advancements in transformer-based models for speech @xcite @xcite have motivated neuroscience researchers to test their brain alignment with speech-evoked brain activity @xcite @xcite @xcite . our approach is complementary and can be used to further understand what types of information underlie the brain alignment of language models, particularly across different brain regions. ## methodology our direct approach to investigate the reasons for brain alignment of language models involves three main steps (see fig. 1 ): (1) removal of interpretable low-level stimulus features from the language model representations; (2) estimating the brain alignment of the language model representations before and after removal of a particular feature; (3) a significance test to conclude whether the difference in estimated brain alignment before and after is significant. ## results we investigate the alignment of text-and speechbased language models during reading and listening, across early sensory processing regions (early visual and early auditory) as well as late language regions. we further use our direct approach to study how different low-level stimulus features impact this alignment. unless otherwise specified, the results presented in the main paper reflect an average across model type, feature type, model layers, voxels, and participants, normalized by the crosssubject prediction accuracy. error bars indicate the standard error of the mean across participants. * * * * * early visual early auditory late language 0 0.2 0.4 0.6 0.8 reading listening rois normalized brain alignment chance performance for each case is indicated by a dashed horizontal line. results for individual models and layers are included in the appendix. ## discussion and conclusion using a direct approach, we evaluated what types of information language models truly predict in brain responses. this is achieved by removing information related to specific low-level stimulus features (textual, speech, and visual) and observing how this perturbation affects the alignment with fmri brain recordings acquired while participants read versus listened to the same naturalistic stories. our direct approach leads to three key conclusions: (1) the unexpected alignment of models with sensory regions associated with the incongruent modality (i.e. text models with auditory regions and speech models with visual regions) is entirely due to low-level stimulus features that are correlated between text and speech (e.g. number of letters and number of phonemes); (2) models exhibit varied alignment with the respective sensory regions, and this variance cannot be ascribed to low-level stimulus features alone. text models exhibit comparable alignment with visual and auditory regions, entirely driven by low-level textual features. in contrast, speech models display significantly greater alignment with auditory regions than visual regions, and this difference cannot be entirely accounted for by the comprehensive set of low-level stimulus features studied. this underscores the capability of speech-based language models to capture additional features crucial for the early auditory cortex, hinting at their potential to enhance our understanding of this brain region; (3) while both text-and speech-based models show substantial alignment with late language re-gions, the impact of low-level stimulus features on text model alignment is marginal, whereas for speech-based models, alignment is entirely driven by these low-level features. since these regions are purported to represent semantic information, this finding implies that contemporary speech-based models lack brain-relevant semantics. furthermore, these results imply that observed similarities between speech-based models and brain recordings in the past @xcite are largely due to low-level information, which is important to take into account when interpreting the similarity between language representations in speech-based models and the brain. this disparity raises questions about the utility of speech models in modeling late language processing, contrasting with the reassuring findings for text-based models for the same regions. our findings also clearly show that despite the growing popularity of text-based and speech-based language models in modeling language in the brain, we are still far from a computational model of the complete information processing steps during either listening or reading. in the future, leveraging the alignment strengths of text-based models in late language regions and speech-based models in early auditory regions may lead to improved end-to-end models of listening in the brain.