Representational similarity learning reveals a graded multi-dimensional semantic space 1 in the human anterior temporal cortex 2

31 Neurocognitive models of semantic memory have proposed that the ventral anterior temporal 32 lobes (vATLs) encode a graded, distributed, and multidimensional semantic space—yet 33 neuroimaging studies seeking brain regions that encode semantic structure rarely identify these 34 areas. In simulations we show that this discrepancy may arise from a crucial mismatch between 35 theory and analysis approach. Utilizing an analysis recently formulated to investigate graded 36 multidimensional representations, representational similarity learning (RSL), we decoded 37 semantic structure from ECoG data collected from the vATL cortical surface while participants 38 named line drawings of common items. The results reveal a graded, multidimensional semantic 39 space encoded in neural activity across the vATL, which evolves over time and simultaneously 40 expresses both broad and finer-grained semantic structure amongst animate and inanimate 41 concepts. The work resolves the apparent discrepancy within the semantic cognition literature 42 and, more importantly, suggests a new approach to discovering representational structure in 43 neural data more generally. 44 Conflict of interest 45 The Department of Epilepsy, Movement Disorders, and Physiology, Kyoto University Graduate 46 School of Medicine conducts Industry-Academia Collaboration Courses, supported by Eisai Co., 47 Ltd, Nihon Kohden Corporation, Otsuka Pharmaceutical Co., and UCB Japan Co., Ltd. 48 49 MULTI-DIMENSIONAL SEMANTICS IN vATL 3 Introduction 50 If you encounter a wolf when walking home through a dark wood, your mind readily 51 accomplishes some remarkable feats: it anticipates the animal's likely behavior, perhaps slinking 52 closer toward you; it assigns the thing a name, which you might shout to alert others ("wolf! "); 53 and it directs you to change your own plans, maybe running back down the path. These feats 54 arise from the human ability to discern conceptual structure—to realize that the wolf, despite its 55 resemblance to friendly dogs in town, is nevertheless a quite different sort of animal. 56 This ability to represent and exploit conceptual structure is central to human semantic 57 cognition. Such structure is graded in that similarities vary along a continuum: wolves are highly 58 similar to coyotes, partially similar to elk, and quite distinct from birch trees or canoes. It is also 59 multidimensional in that concepts vary along myriad independent components: wolves and dogs 60 are similar in their shapes, parts, movements, and phylogeny, but different in their behaviors, 61 habitats, and diets. To capture these properties of knowledge, computational approaches to 62 semantics often represent concepts with vector spaces: the meaning of a word or image is 63 expressed as a point in an n-dimensional space (or equivalently as an n-dimensional vector) such 64 that the proximity between points expresses the similarity in meaning between the denoted 65 concepts. Cognitive science and machine-learning offer many techniques for estimating semantic 66 vector spaces from natural language (Pereira et al., 2016), feature norms(McRae et al., 2005), or 67 similarity-judgments(Hebart et al., 2020), and these methods have provided a critical empirical 68 foundation for studying human conceptual knowledge. 69 The well-known “hub and spokes” view of semantic representation suggests that the 70 anterior temporal lobes (ATLs) encode a graded, multi-dimensional semantic vector space that 71 expresses conceptual similarity structure for all concepts, extracted across all input and output 72 MULTI-DIMENSIONAL SEMANTICS IN vATL 4 modalities, and from our experiences of each concept across time (Jackson et al., 2021; Lambon 73 Ralph et al., 2017; Patterson et al., 2007). This proposal has been useful for understanding 74 patterns of semantic deficits arising from temporal lobe pathology in fronto-temporal dementia 75 (Acosta-Cabronero et al., 2011; Hodges & Patterson, 2007; Lambon Ralph, 2014; Lambon Ralph 76 et al., 2007; Rogers et al., 2004; Snowden et al., 1989), anterior temporal resection (Drane et al., 77 2008; Lambon Ralph et al., 2009; Schapiro et al., 2013), and herpes viral encephalitis (Gainotti, 78 2018; Noppeney et al., 2007; Rogers et al., 2004); stimulation and evoked response direct 79 neurophysiological explorations; (Abel et al., 2015; Y. Chen et al., 2016; Shimotake et al., 8