Automatic Learning of Modality Exclusivity Norms with Crosslingual Word Embeddings (2020)

Abstract: Collecting modality exclusivity norms for lexical items has recently become a common practice in psycholinguistics and cognitive research. However, these norms are available only for a relatively small number of languages and often involve a costly and time-consuming collection of ratings. In this work, we aim at learning a mapping between word embeddings and modality norms. Our experiments focused on crosslingual word embeddings, in order to predict modality association scores by training on a high-resource language and testing on a low-resource one. We ran two experiments, one in a monolingual and the other one in a crosslingual setting. Results show that modality prediction using off-the-shelf crosslingual embeddings indeed has moderate-to-high correlations with human ratings even when regression algorithms are trained on an English resource and tested on a completely unseen language.

## introduction the expression modality exclusivity norms refers to a collection of words and their association scores with sensory modalities. the use of modality norms has been a recent trend in psycholinguistic and cognitive science, based on the observation that words with sensory meanings are associated with certain perceptual regions of the brain @xcite @xcite . words are typically associated with multiple modalities: for example, the word sweet can be used to describe sounds, flavours, looks etc. @xcite @xcite . psycholinguistic studies use modality exclusivity norms to control for this work is licensed under a creative commons attribution 4.0 international license. license details: http: //creativecommons.org/licenses/by/4.0/ . the perceptual strength of lexical items used in their experiments, motivating the publication of datasets in which words are rated according to their association with each of the five senses (see the example in our working hypothesis for this study is that the commonalities in human perceptual cognition would lead to reliable automatic induction of modality exclusivity norms for unseen words (i.e. words without experimental data) in the same language. in addition, the five primary sensory modalities are assumed to be universal, and consequently, the prediction of norms in a new language can be carried out with the same procedure. crosslingual word embeddings provide an ideal model for the prediction of modality exclusivity norms in lowresource languages, as they represent words of multiple languages in a shared feature space. it is thus possible to train a regressor on a resource-rich language, e.g english, and predict modality ratings for words in an unseen language. we experimented with this crosslingual transfer method on italian, dutch and chinese norms. results show that a) crosslingual embeddings perform similarly to or slightly better than monolingual embeddings in a monolingual setting; b) even after training only on english data, the regressor can predict norms in a totally unseen language with moderate-to-high correlations with human judgements. ## related work although word vectors have been a standard for word representations for almost two decades @xcite , they became an essential ingredient for most nlp applications only after the introduction of word embeddings @xcite @xcite . differently from the first generation models using co-occurrence counting and weighting, word embeddings are estimated via neural network training with the objective of maximizing the probability of the contexts of a target word, and they gained popularity due to the availability of efficient and easy-touse tools @xcite . the development of research on crosslingual transfer and the availability of new benchmarks for multilingual nlp has recently led to the introduction of the so-called crosslingual embeddings, vector space models that represent words from multiple languages through some form of mapping from a monolingual to a multilingual space @xcite . a classical study by @xcite learnt a linear projection to transform the space of a source language to the space of a target language by maximizing the similarity between the two spaces. other approaches apply canonical correlation analysis to simultaneously project words from two languages into a shared embedding space where the correlation between projected vectors are maximized @xcite . other works make use of the max-margin method such that, for embeddings projected from a source language, they maximize the margin between the correct translations and other candidates @xcite . for this study, we use the offthe-shelf crosslingual embeddings by @xcite based on fasttext @xcite and trained on wikipedia. 1 despite the success of word embeddings, a common criticism is that they are not grounded in perception, as words are only defined in relation to each other and not to entities and actions in the physical world @xcite @xcite . to address this issue, @xcite used a regression method to map embeddings onto the conceptual properties of the mcrae norms @xcite . a similar approach, using feedforward neural networks for predicting properties, was recently described by @xcite . the work by @xcite goes in the opposite direction: instead of predicting norms from embeddings, they combined pretrained vectors and property vectors to inject conceptual knowledge into a new type of word representations. their fea-ture2vec system showed a strong performance in the predicting norms of unseen words, compared to previous proposals. finally, @xcite proposed a similar mapping technique to exploit semantic feature norms by @xcite to analyze the semantic content of word embeddings in terms of neurobiologically-motivated features. @xcite also experimented with embeddings based on binder features, showing that they can achieve performances comparable to word2vec @xcite and glove @xcite ) on similarity datasets. ## our proposed approach in this work, we used regressors trained on crosslingual word embeddings to predict modality ratings in two different scenarios. in the monolingual scenario, we adopt a 5-fold cross-validation to predict the modality norms of an english dataset @xcite . in the crosslingual scenario, a regressor is trained on a high-resource language, e.g. english, to predict the modality norms of an unseen language. ## conclusions in this paper, we have proposed the first study dedicated to the prediction of modality norms via word embeddings mapping. we experimented with crosslingual embeddings in order to assess the potential for crosslingual transfer. our results first showed that modality norms can be reliably predicted even by purely text-based vectors. this is in accordance with cognitive hypotheses claiming that various aspects of experiential information are redundantly encoded in linguistic expressions (e.g. the symbol interdependency hypothesis) @xcite . moreover, crosslingual vectors turned out to be better performing than their monolingual counterparts. this result is certainly surprising, although it is unclear whether it is due to differences in the training data, or the mapping itself benefits performance by abstracting away from languagespecific patterns to a more 'conceptual' space. even more importantly, given the availability of crosslingual embeddings for a low-resource language, it is possible to train a regressor on a highresource language (e.g. english) and to predict the norms for the low-resource one. in our experiments, we obtained moderate-to-high correlations even in the crosslingual setting. we think this is potentially a very useful application for the research on modality norms. in this first study we used a relatively simple methodology, but several refinements are possible for improving the prediction quality. two possible directions would be, firstly, to exploit the presence of words that are direct translations from english to the other languages to apply retrofitting techniques @xcite @xcite to the crosslingual space, and secondly, to tackle the task by introducing more advanced neural architectures for the representation of words in context, e.g. multilingual transformers @xcite .