Measuring Meaning Composition in the Human Brain with Composition Scores from Large Language Models (2024)

Abstract: The process of meaning composition, wherein smaller units like morphemes or words combine to form the meaning of phrases and sentences, is essential for human sentence comprehension. Despite extensive neurolinguistic research into the brain regions involved in meaning composition, a computational metric to quantify the extent of composition is still lacking. Drawing on the key-value memory interpretation of transformer feed-forward network blocks, we introduce the Composition Score, a novel model-based metric designed to quantify the degree of meaning composition during sentence comprehension. Experimental findings show that this metric correlates with brain clusters associated with word frequency, structural processing, and general sensitivity to words, suggesting the multifaceted nature of meaning composition during human sentence comprehension.

## introduction when encountering words such as "milk" and "pudding", the human mind effortlessly combines them to form a complex concept, such as a milk-flavored pudding. this combinatory process is a fundamental aspect of human language comprehension and production, enabling us to generate an infinite array of meanings from a finite set of words. despite extensive neurolinguistic research into the localization of meaning composition in the human brain @xcite @xcite @xcite @xcite , understanding the detailed mechanism of how a complex meaning is constructed from its components and how it is processed by the human brain has become a challenging problem. one of the primary difficulties lies in the absence of a suitable computational metric to quantify the extent of meaning composition. this absence significantly complicates quantitative analyses of meaning composition in the human brain. recent advancements in large language models (llms) offer promising insights into this problem. by training on large-scale natural language corpora and aligning with human preferences, these computational models achieve unprecedented levels of proficiency in understanding and generating natural languages @xcite @xcite . in addition to their high performance, studies have shown that their internal states correlate with human behavioral and neural data @xcite , suggesting shared principles between their algorithms and the human brain. given this background, it is natural to inquire whether we can develop a computational metric to quantify meaning composition from the internal states of llms. motivated by this inquiry, our study introduces a novel model-based metric, the composition score, to evaluate meaning composition in the human brain. leveraging the key-value memory interpretation of the feed-forward network (ffn) modules in the transformer model @xcite , this metric computes the composition of memory-induced vocabulary distributions within the ffn blocks given an input prefix, thereby reflecting the degree of meaning composition of each word. to assess its validity, we examine the patterns of composition scores using the novel "the little prince" in english and compare them with other control variables such as word frequency and syntactic node count based on top-down, bottomup, and left-corner parsing. additionally, we correlate composition scores with an openly available fmri dataset where participants listened to "the little prince" in the scanner @xcite . our findings reveal that: ## conclusion in this paper, we introduce a novel model-based metric, the composition score to quantify meaning composition. we examine its correlation with human neural activity and identify several brain clusters, offering insights into the process of meaning composition in the brain.