Semantic representation in the white matter pathway
Authors: Yuxing Fang, Xiaosha Wang, Suyu Zhong, Luping Song, Zaizhu Han, Gaolang Gong, Yanchao Bi
Journal: PLOS Biology
Accepted: 2018-02-05T00:00:00Z


Object conceptual processing has been localized to distributed cortical regions that represent specific attributes. A challenging question is how object semantic space is formed. We tested a novel framework of representing semantic space in the pattern of white matter (WM) connections by extending the representational similarity analysis (RSA) to structural lesion pattern and behavioral data in 80 brain-damaged patients. For each WM connection, a neural representational dissimilarity matrix (RDM) was computed by first building machine-learning models with the voxel-wise WM lesion patterns as features to predict naming performance of a particular item and then computing the correlation between the predicted naming score and the actual naming score of another item in the testing patients. This correlation was used to build the neural RDM based on the assumption that if the connection pattern contains certain aspects of information shared by the naming processes of these two items, models trained with one item should also predict naming accuracy of the other. Correlating the neural RDM with various cognitive RDMs revealed that neural patterns in several WM connections that connect left occipital/middle temporal regions and anterior temporal regions associated with the object semantic space. Such associations were not attributable to modality-specific attributes (shape, manipulation, color, and motion), to peripheral picture-naming processes (picture visual similarity, phonological similarity), to broad semantic categories, or to the properties of the cortical regions that they connected, which tended to represent multiple modality-specific attributes. That is, the semantic space could be represented through WM connection patterns across cortical regions representing modality-specific attributes.
Author summary: One of the most challenging questions in cognitive neuroscience is how semantic knowledge, for example, that “scissors” and “knives” are related in meaning, can emerge from primary sensory dimensions such as visual forms. It is often assumed that in the human brain, semantics are stored in regions of the brain cortex, where distinct types of modality-specific information are transferred to and bind together. We tested an alternative hypothesis—“representation by connection”—in which higher-order semantic information could be coded by means of connection patterns between cortical regions. Combining data from behavior and brain imaging of 80 patients with brain lesions, we applied machine learning to construct the mapping models between the lesion patterns on axonal tracts (white matter) and item-specific object-naming performances. We found that specific white matter lesions produced deficits in object naming associated with the object’s semantic space, but not relevant to its primary dimension. The naming performances of semantically related objects were better predicted from white matter lesion-pattern models. That is, the higher-order semantic space could be coded in patterns of brain connections by linking cortical areas that do not necessarily contain such information.