Phonological Layers of Meaning: A Computational Exploration of Sound Iconicity
Authors: Andrea Gregor de Varda, Carlo Strapparava (Università di Trento, Fondazione Bruno Kessler, 2020)
Topic: Lexical and Semantic Resources and Analysis

Abstract
The present paper aims to investigate the nature and the extent of cross-linguistic phonosemantic correspondences within a computational framework. An LSTM-based Recurrent Neural Network is trained to associate the phonetic representation of a word, encoded as a sequence of feature vectors, to its corresponding semantic representation in a multilingual vector space. The processing network is tested, without further training, in a language that does not appear in the training set. The performance of the multilingual model is compared with a monolingual upper bound and a randomized baseline. After the quantitative evaluation of its performance, a qualitative analysis is carried out on the network's most effective predictions, showing an inhomogeneous distribution of phonosemantic information in the lexicon, influenced by semantic, syntactic, and pragmatic factors.

Introduction
The idea of a consistent relationship between sound and meaning has held a particular fascination over philosophers and linguists (Plato, 1998). However, in recent times, this charming hypothesis has progressively lost the interest of scholars, especially in the post-Saussurean linguistic tradition, which emphasized the arbitrariness in such relation. The idea that sounds have inherent meanings has recaptured its original attractiveness in the field of cognitive sciences, where the attention has initially focused on the link between sound and shape. A prominent example of these naturally biased mappings came from Köhler’s (1929) finding that, when asked to match two novel shapes with the non-words ‘maluma’ and ‘takete’, English-speaking adults tended to label as ‘maluma’ the curled shape, and as ‘takete’ the sharp one. This germinal study paved the way to several replications and expansions of its findings, that reproduced Köhler’s results in different geocultural contexts (Bremner et al., 2013) and at different developmental stages (Maurer et al., 2006). Since then, different studies have tackled the topic of iconicity in language from a broader perspective, showing that adults can associate visually presented characters (Koriat and Levy, 1977) and auditorily presented words (Berlin, 1995) of a foreign language to their meaning, with an accuracy above chance. Recently, linguistic iconicity has gone from being a marginal – although appealing – matter to being integrated into broader theories of language evolution and acquisition. Indeed, rejecting the assumption of an arbitrary mapping between sound and meaning sensibly reduces the problem space of language emergence, establishing constraints on the consensus of word choice. Furthermore, an iconic relation between a sound and its referent might help with memory consolidation in the process of language acquisition (Sathian and Ramachandran, 2019). Ramachandran and Hubbard (2001) speculate that phenomena as the one reported by Köhler might arise from neural connections among adjacent cortical areas, where the visual features of the referent, the appearance of the speaker’s lips and the kinaesthetic features of the articulation are combined. According to their view, such neural connections would have influenced both the phylogenetic evolution and the ontogenetic development of language. Although the previous findings are consistent with this hypothesis, an alternative explanation must be taken into account: the roots of these correspondences could be grounded in the knowledge of language, that allows children and adults to generalize the regularities in sound-to-meaning mappings from their native language to nonsense and foreign words. Under this rationale, phonosemantic relations would be implicitly learned from general recurrences in already known languages. A crucial aspect of this account lies in the fact that it does not posit any preexisting disposition wired in the human brain, moving the locus of linguistic iconicity from the mind to language itself. A natural question that arises from this perspective is whether linguistic information alone is sufficient to give rise to the phonaesthetic biases presented in the literature. A computational exploration of the phenomenon under scrutiny is a feasible way to approach the subject. The idea that phones have inherent meanings is relatively understudied within the computational framework, and most of the studies addressing the topic have either focused on a single language (Gutiérrez et al., 2016; Sagi and Otis, 2008; Abramova et al., 2013; Monaghan et al., 2014; Tamariz, 2008) or on a small set of concepts on a massively multilingual scale (Blasi et al., 2016; Wichmann et al., 2010). Surprisingly, no study to our knowledge has tackled the topic through a deep learning methodology, and no cross-linguistic investigation has been performed on a lexicon-wide level. The purpose of the present study is two-fold: first, we wish to explore the idea of a cross-linguistic correspondence between the phonetic and the semantic representation of a word on the whole lexicon, without any theory-driven restriction guiding our choice of the lexical items. Then, we aim to examine whether the meaning that is rooted in the sound that words are made of is homogeneously distributed in the lexicon. Ultimately, these two goals converge toward the research question hinted above, namely, whether linguistic information alone could suffice for the extrapolation of the phonosemantic biases reported in the present section. A possible way to answer this question is to assess the ability of a tabula rasa neural network to extend the regularities captured in a set of given languages to a previously unseen one. Although equipped with clear structural priors, neural networks do not conceal biases that resemble those assumed to model the aforementioned phonosemantic correspondences. If a processing network showed the ability to induce cross-linguistic regularities in sound-to-meaning mappings, this would suggest that linguistic data contain a sufficient amount of information to encode for phonosymbolic biases. The present study aims to explore the possibility of a certain degree of cross-linguistic correspondence between sound and meaning that is already encoded in language. A Long Short-Term Memory network (LSTM) is trained on four languages to associate the sequence of sounds that compose a word, encoded as phonetic vectors, to its corresponding semantic representation in a multilingual vector space. Then the processing network is tested, without further training, on a language that does not appear in the set of languages on which the training has been performed. The performance of the multilingual model is compared with the results of (a) a monolingual model, trained and tested on different subsets of a single language’s vocabulary, and (b) a baseline model, where the output vectors in the training are randomly shuffled. After the quantitative evaluation of its performance, a qualitative analysis is carried out on the network’s most effective predictions.

Conclusion
From a methodological standpoint, the reliability of the present results could benefit from the exclusion of lexical borrowings and proper names from the training and the test sets. Excluding etymologically related terms could further improve the reliability of the results, but at the costs of raising the difficulty of assessing the words’ relatedness in different languages, with the subsequent need of a proper metric. Another confound that we wish to address in future research is the role played by morphological factors in aiding the cross-linguistic feature extraction performed by the network. FastText vectors exploit information related to subword character strings, and might therefore encode regularities pertaining to recurrent morphemes in the nonisolating languages in our dataset (Italian, German, French, and Turkish). We acknowledge that the network might have captured the recurrences encoded in the semantic vectors comprising the training set and their relationships with the corresponding phonetic feature vectors; indeed, we believe that this regularities might have played a relevant role in the monolingual condition, where the model might have learnt that morphologically related words (i.e. in this context, words that are similar at the character- and phoneme-level) tend to be associated with close subregions of the semantic space. Nonetheless, we do not see how this information could have altered significantly the performance in the multilingual condition. That said, we leave for future research an assessment of the algorithm’s performance on semantic vectors which lack access to subword-related information, such as word2vec (Mikolov et al., 2013), and in languages with opaque orthography (e.g. English and French) and non-concatenative morphology (e.g. Chinese). As for all the studies that employ artificial neural networks to draw conclusions on human cognition, it is mandatory to clarify some limitations on the extent of the inferences that can legitimately follow the presented results. The finding that a neural network can succeed in a task without the structural priors postulated in the human mind does not necessarily imply that these priors are not actually encoded in the brain: the assumption of a functional equivalence between artificial and biological processes needs to be independently motivated. Moreover, it should be noticed that the participants of the behavioural studies presented in Section 1 were not necessarily polyglots, whereas the promising cross-linguistic performances described in the results have been obtained with a multilingual model. In addition to these intrinsic methodological limitations, an account that does not assume any prior specification for the linguistically encoded phonosymbolic mappings would leave an open question concerning their origin. Hence, the present study does not claim to reject the multi-sensory integration hypothesis presented in the Introduction. Its purpose is simply to show that, in principle, linguistic information alone could suffice for a generalization in sound-to-meaning mappings.