Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings (2021)

Abstract: In contrast to their word- or sentence-level counterparts, character embeddings are still poorly understood. We aim at closing this gap with an in-depth study of English character embeddings. For this, we use resources from research on grapheme–color synesthesia – a neuropsychological phenomenon where letters are associated with colors –, which give us insight into which characters are similar for synesthetes and how characters are organized in color space. Comparing 10 different character embeddings, we ask: How similar are character embeddings to a synesthete’s perception of characters? And how similar are character embeddings extracted from different models? We find that LSTMs agree with humans more than transformers. Comparing across tasks, grapheme-to-phoneme conversion results in the most human-like character embeddings. Finally, ELMo embeddings differ from both humans and other models.

## introduction neural network models have become crucial tools in natural language processing (nlp) and define the state of the art on a large variety of tasks @xcite . however, they are difficult to understand and are often considered "black boxes". 1 this can make their use difficult to defend in many settings, for instance in a legal context, and constitutes a barrier for model improvement. therefore, a lot of research has been dedicated to investigating the information encoded by neural networks. especially word embeddings, contextualized word representations, and language representation models like bert @xcite have been exhaustively studied @xcite . a b c d e f ... character embeddings are used for a large set of tasks, either as a supplement to word-level input, e.g., for part-of-speech tagging by @xcite , or on their own, e.g., for character-level sequence-to-sequence (seq2seq) tasks by @xcite . despite this, they have not yet been explicitly analysed. one reason for this might be that identifying relevant properties to study is more challenging than for their word-level counterparts. however, we argue that, in order to truly shine light into black-box nlp models, it is necessary to understand each and every component of them. in this paper, we perform a detailed study of english character embeddings. our first contribution is a character similarity task ( §3) in analogy to the word-based version: do embeddings agree with humans on which pairs of characters are similar? while e.g., cat and tiger are generally considered more similar than cat and chair, this is not trivial for characters. for our annotations, we exploit a phenomenon called synesthesia. people with synesthesia, synesthetes, share perceptual experiences between two or more senses ( §2). for graphemecolor synesthetes, each letter is associated with a color, which is consistent over a person's life time @xcite , cf. figure 1 . using a dataset of letters from the english alphabet and the associated colors from 4269 synesthetes @xcite , we compute differences in color space ( §3.3) as a proxy for character similarity. as our second contribution, we propose a set of methods to characterize the structure of character embedding matrices ( §4). the methods we propose are (i) a clustering analysis, (ii) computing the clustering coefficient, (iii) measuring betweenness centrality, and (iv) computing cut-distances. our final contribution is a detailed character embedding analysis ( §6). we explore 6 types of embeddings, obtained from 4 different architectures trained on 3 different tasks, as well as 4 embedding matrices from pretrained elmo models @xcite . comparing across models trained on the same task, character embeddings from lstms, which, similar to humans, process input sequentially, correlate more with human similarity scores than embeddings from transformers. comparing across tasks, character embeddings from language modeling show a surprisingly low correlation with human judgments. in contrast, the correlation is highest for grapheme-to-phoneme conversion (g2p). this is in line with findings that colors perceived by synesthetes are influenced by the sound of each letter @xcite . ## synesthesia synesthesia is the perceptual phenomenon that two or more sensory or cognitive pathways are co-activated in the brain by stimulating only one of them. for example, for a person with chromesthesia, a common form of synesthesia, hearing a particular sound evokes the visual perception of colors @xcite . by far the most common form of synesthesia is grapheme-color synesthesia, for which subjects report the perception of colors when seeing numerals or letters (cf. figure 1 ). the neural basis behind the phenomenon is still a much debated topic, but evidence suggests it might be the result of increased effective connectivity between the brain areas involved @xcite . for example, visual area v4a in the ventral pathway, often associated with contextual processing and perception of color, has shown to be part of a complex in the fusiform gyrus exhibiting higher cortical thickness, volume, and surface area in synesthetes @xcite . belonging to this complex and adjacent to the color processing region is the area dedicated to the visual recognition and processing of graphemes, suggesting that a prominence of this type of synesthesia is due to higher region interconnectivity being more probable. additional evidence supports the idea that synesthetic associations often involve the extraction of meaning from a stimulus, suggesting that abstract constructions play a major role in the associations formed @xcite . for cases where language is involved, semantics may in part underlie color representations of words and graphemes in v4. the study of synesthesia presents a unique opportunity to understand the neural basis of cognitive models of language @xcite . here, grapheme-color synesthesia serves as a window to look at how the brain represents individual characters. ## analysis of the distance matrix ciede2000 allows us to obtain pair-wise distances between all letters in the alphabet by computing color differences for the associated colors as perceived by synesthetes. the character similarity task then compares vectors of pair-wise distances. however, we can gain additional insight from the distance matrices, which represent fully connected networks whose nodes are the letters in the alphabet and whose edges are weighted by the pair-wise differences. thus, we further propose four wellestablished methods from network theory (newman, 2018) for the analysis of character embeddings and human difference matrices. clustering analysis. to characterize the global structure of the network, we first propose ward's variance minimization algorithm to identify clusters, i.e., groups of letters that are similar to each other, but far away from other letters. ward's algorithm is part of a family of hierarchical clustering algorithms whose objective function aims at minimizing the variance within clusters @xcite . starting from a forest of clusters (initially single nodes), the algorithm evaluates the ward distance d between a new cluster u made up of clusters s and t, and a third cluster v not used yet, as clustering coefficient. the clustering coefficient provides a way to measure the degree to which nodes in a network cluster together. for a binary network, the local version represents the fraction of the number of pairs of neighbors of a node that are connected, over the total number of pairs of neighbors of said node, and it measures the influence of a node on its immediate neighbors. several generalizations for weighted networks have been proposed @xcite . here, we use the average of the weights for neighbors in the subgraph of a node u the average over all nodes is used as a proxy for the overall level of clustering within the network. betweenness centrality. different concepts of centrality attempt to capture the relative importance of particular nodes in the network. one such concept, betweenness, measures the extent to which a node lies on the shortest path between pairs of nodes @xcite . in a sense, it generalizes the clustering coefficient from a measure of the local influence of a node to immediate neighbors to the whole network. in particular, it accounts for nodes that connect two different clusters while not being a part of either. betweenness centrality is computed as the fraction of all-pairs shortest paths that pass through a particular node u b(u) = s,t n(s, t|u) n(s, t) , ## conclusion in this paper, we performed an in-depth analysis of character embeddings extracted from various character-level models for nlp. we leveraged resources from research on grapheme-color synesthesia -a neuropsychological phenomenon where letters are associated with colors -, to construct a dataset for a character similarity task. we further performed an analysis of networks representing characters as nodes and similarities as edge weights to understand how characters are organized by human synesthetes in comparison to character embeddings. analysing 10 different character embeddings, we found that lstms agreed with humans more than transformer models. comparing different tasks, g2p resulted in embeddings more similar to human character representions than inflection and, by a wide margin, language modeling. elmo embeddings differed from humans and other models in that they exhibited no clear structure.