Collecting and Predicting Neurocognitive Norms for M andarin C hinese (2023)

Abstract: Language researchers have long assumed that concepts can be represented by sets of semantic features, and have traditionally encountered challenges in identifying a feature set that could be sufficiently general to describe the human conceptual experience in its entirety. In the dataset of English norms presented by Binder et al. (2016), also known as Binder norms, the authors introduced a new set of neurobiologically motivated semantic features in which conceptual primitives were defined in terms of modalities of neural information processing. However, no comparable norms are currently available for other languages. In our work, we built the Mandarin Chinese norm by translating the stimuli used in the original study and developed a comparable collection of human ratings for Mandarin Chinese. We also conducted some experiments on the automatic prediction of the Chinese Binder Norms based on the word embeddings of the corresponding words to assess the feasibility of modeling experiential semantic features via corpus-based representations.

## introduction a longstanding research trend in semantics assumes that the conceptual content of lexical items can be decomposed into semantic features identifying basic meaning components @xcite . such features represent semantic primitives that can be present or absent in the semantic representation of a lexeme, such as boy in example (1). however, this type of view has some critical limitations: first, discrete features are not suitable to address the gradient prototypicality of featureto-concept associations @xcite . second, these feature sets tend to be manually selected, and are generally tailored to a few in vitro examples; thus, they are unable to account for large portions of the lexicon of natural languages @xcite . on one hand, featural representations have the advantage of human interpretability, as they label the dimensions of word meanings explicitly, and provide explanatory factors for their semantic behavior; for example, the similarity between beer and coffee can be explained by assuming that they share the semantic feature of liquid. on the other hand, this type of features is highly subjective, and can only be collected through a time-consuming process of elicitation from human subjects (e.g. @xcite ; @xcite ; @xcite ; @xcite ). an alternative was proposed by @xcite using brain-based semantics based on modalities of neural information processing. after reviewing extensive evidence from studies of human physiology, the authors proposed a dataset of 535 words described in terms of 68 experiential features, each of which was associated with a specific neural processing in the neurobiological literature. the features were categorized according to 14 different domains of experience ( the proposal by binder et al. (2016) should naturally extend to other languages: if the features are genuinely neurobiologically motivated, it should also be possible to use them to describe the essential meaning components of languages other than english. therefore, in our work, we adopted the same design of binder norms: we translated the words in the binder dataset into mandarin chinese, and obtained ratings from human subjects for each of the 68 binder features per word in order to obtain a comparable dataset. moreover, we experimented with regression algorithms to assess the extent to which such norms could be predicted automatically based on the text-derived embeddings of the corresponding words. foot_2 2 related work neurosemantic decoding research, initiated by the seminal work of @xcite , has the aim of creating mappings between different concept representations, typically from a corpus-derived one (such as word embedding) to one derived from human data (such as fmri scans and semantic norms). for example, previous studies used fmri data to learn mapping from the traditional countbased distributional models @xcite , including both count-and prediction-based vectors @xcite , and topic models @xcite ; the same methodology has been used to map word-embedding models onto feature @xcite @xcite and modality norms @xcite to ground the vectors in perceptual data and to make them interpretable. due to the grounding on perceptual experience, the binder features for english have also been used for the same purpose 3 dataset and code for the experiments will be available at the following url: https://github.com/laniqiu/ norming . @xcite @xcite . notice that, differently from property norms @xcite , the collection process is more constrained: the properties of concepts are not freely elicited from human participants; because the binder features are a closed set, the participants were asked to only rate the relevance of a given feature for a given concept. we are not currently aware of any other work that has introduced binder-like norms for languages other than english. the recent work by @xcite introduced a fmri dataset for mandarin chinese, together with a collection of binder ratings for the target words. however, their targets differed from those in the original study by @xcite (a total of 672 words from the synonymy thesaurus of the harbin institute of technology), and the representation was limited to 54 binder features, as some of them were excluded due to high levels of correlation with at least one of the other features. with the aim of providing a comparable and more comprehensive resource to facilitate future experiments on the prediction of crosslingual norms, we opted to retain the original set of target words and features. and 39 adjectives, while another 293 words were added to include more abstract nouns. we adopted the original set of 535 target words and 68 features proposed by @xcite , and the original survey queries that they proposed. we translated them into mandarin chinese using simplified characters. this survey was used to elicit the ratings for the salience of each attribute for each target word, with the same 0-6 likert scale used in the original study (the higher the score, the higher the relevance of a feature when one has to think about the target concept, while 0 corresponds to "feature not applicable to this concept"). the target words and the survey queries were translated by two native speakers of mandarin, who were master's students of linguistics. for features and target words, we adopted their most common and core sense in english to translate into their corresponding chinese. while some words in colloquial uses may have multiple senses, we selected more specific words which were equally frequent to the polysemous ones and to the sense expressed by the english counterparts. we were aware that the concept of "adjective" could sometimes not easily be recognized in chinese, just as the function of words in the -ed form can be ambiguous in english as either adjectival or verbal past participle. when an adjective could be interpreted as other parts of speech categories (pos), we added an adjectival suffix -的 de to such adjectives to avoid such potential confusion. the final version of the survey queries and the target words were manually checked by one of the authors, who is also a native mandarin speaker. the same pos of each word were maintained for the 535 words, and each word was associated with survey questions pertaining to the 68 cognitively motivated features. one target word in the survey, banjo, was replaced for a more culturally relevant musical instrument, 二胡 erhu, while the other words were the same as their english counterparts. as is the case for the binder norms, we adopted a continuous rating design to obtain the attributes for each word. we collected the data on a crowdsourcing platform that is commonly used in china (问卷 星 wenjuanxing), because the rating results might occur along a continuum and could be subjective due to the speakers' personal experiences and backgrounds, thus, a larger sample size was considered to be helpful in overcoming this issue. we obtained 8025 sets of ratings from the crowdsourcing survey; each of the 535 targets obtained 15 sets of rating results covering all 68 features. the demographics and the language backgrounds of the participants were checked before they participated in the survey. each participant received rmb$20 after completing the survey and once their results had passed the survey's attention checks. after completing the survey, we measured the spearman correlation between english and chinese ratings. we found out that the ratings were quite consistent across languages: on average, we obtained a correlation of 0.68 across words and a correlation of 0.59 across features. ## experiments in order to learn to map between word-embedding spaces and our chinese binder features, we trained regression models using three different regressors, namely ridge regression, random forest and multilayer perceptron (mlp) 5 , using the ratings of the 68 features in the dataset as the dependent variables and the dimensions of pretrained word-embedding models as the independent variables. considering that the task requires mapping between word types that are taken out of context, we decided to use static word-embedding mod5 the regression models were implemented using scikitlearn @xcite with standard hyperparameters. the only exception was the mlp, for which we selected the following parameters after a parameter search: hidden layer @xmath0, 10), @xmath1, early @xmath2, max @xmath3. els: we used four different types of embeddings: count-based sparse ppmi vectors @xcite ) that were trained on the chinese wikipedia (ppmi.wiki.zh; @xcite ), skip-gram vectors @xcite that were trained on the chinese wikipedia (sgns.wiki.zh, @xcite ), and fasttext vectors that were trained on the chinese common crawl (fast.cc.zh) or on the chinese wikipedia (fast.wiki.zh) @xcite ). all the embedding models had 300 dimensions as input features for the regressor, except for the sparse ppmi vectors, which had 350k dimensions. in addition, we initialized 300-dimensional random vectors for all the words in the dataset, and used them to train similar regression models as baselines (random). in future, we also plan to test contextualized word embeddings @xcite in the task, although it is worth pointing out that their performance in out-of-context semantic tasks has recently been shown not to differ significantly from that of static models @xcite . following @xcite , we adopted the leaveone-out paradigm for data splitting: for each of the n target words; we extracted one word out and trained a regression model on the other n -1 re- ## results the results in it can also be seen that, while english nouns were predicted much better than other pos, similar correlations were observed for nouns and verbs in chinese (adjectives were the most difficult in both languages).