Signal in Noise: Exploring Meaning Encoded in Random Character Sequences with Character-Aware Language Models (2022)

Abstract: Natural language processing models learn word representations based on the distributional hypothesis, which asserts that word context (e.g., co-occurrence) correlates with meaning. We propose that n-grams composed of random character sequences, or garble, provide a novel context for studying word meaning both within and beyond extant language. In particular, randomly generated character n-grams lack meaning but contain primitive information based on the distribution of characters they contain. By studying the embeddings of a large corpus of garble, extant language, and pseudowords using CharacterBERT, we identify an axis in the model’s high-dimensional embedding space that separates these classes of n-grams. Furthermore, we show that this axis relates to structure within extant language, including word part-of-speech, morphology, and concept concreteness. Thus, in contrast to studies that are mainly limited to extant language, our work reveals that meaning and primitive information are intrinsically linked.

## introduction what primitive information do character sequences contain? modern natural language processing is driven by the distributional hypothesis @xcite , which asserts that the context of a linguistic expression defines its meaning @xcite . because existing words-which represent an extremely small fraction of the space of possible character sequences-appear in context together, the distributional paradigm at this level is limited in its ability to study the meaning of and information encoded by arbitrary character level n-grams (word forms). furthermore, state-of-the-art computational language models operating within the distributional paradigm, such as bert @xcite , are mainly trained on extant words. yet, a plethora of insights into language learning have emerged from inquiries into language beyond extant words, such as the grammatical errors and inference patterns that children exhibit when distinguishing extant words from non-linguistic auditory signals, including emotional expressions, auditory gestures, and other forms of paralinguistic speech @xcite . we therefore propose that character n-grams (i.e., sequences of alphabetic characters) outside the space of extant language can provide new insights into the meaning of words and how they are represented by these models, beyond that captured by word and subword-based distributional semantics alone. we explore this by studying the embeddings of randomly generated character n-grams (referred to as garble), which contain primitive communicative information but are devoid of meaning, using the charac-terbert model @xcite . such randomly generated character n-grams are textual analogues of paralinguistic vocalizations-vocal extra-speech sounds and noises. our analyses contribute to the growing understanding of bertology @xcite ) by identifying a dimension, which we refer to as the information axis, that separates extant and garble n-grams. this finding is supported by a markov model that produces a probabilistic information measure for character n-grams based on their statistical properties. strikingly, this information dimension correlates with properties of extant language; for example, parts of speech separate along the information axis, and word concreteness varies along a roughly orthogonal dimension in our projection of characterbert embedding space. although the information axis we identify separates extant and randomly generated n-grams very effectively, we demonstrate that these classes of n-grams mix into each other in detail, and that pseudowords-i.e., phonologically coherent character n-grams without extant lexical meaning-lie between the two in our characterbert embeddings. this paper is organized as follows. we first discuss concepts from natural language processing, information theory, and linguistics relevant to our study. we then analyse characterbert representations of extant and randomly generated character sequences and how the relation between the two informs the structure of extant language, including morphology, part-of-speech, and word concreteness. finally, we ground our information axis in a predictive markov language model. ## modeling n-grams beyond extant language models in computational linguistics often represent words in a high-dimensional embedding space based on their co-occurrence patterns according to the distributional hypothesis @xcite . embeddings that capture the semantic content of extant words are used for many natural language applications, including document or sentence classification @xcite , information retrieval and search @xcite , language modelling and translation @xcite , language generation @xcite , and more @xcite . in these cases, vector operations performed on word embeddings are used for higher-level tasks such as search or classification. word embeddings have largely concerned themselves with extant language-that is, commonly used words which carry consistent meaning-and thus cannot represent character n-grams outside of this space. the few models that encompass character n-grams, which naturally include n-grams beyond extant words, often use rnns @xcite or encoder-decoder architectures @xcite to represent character-level sequences. in parallel, the ubiquitous use of transformer models has led to studies of their inner representations, weights, and attention mechanism @xcite . most transformer models are trained using extant words and sub-words, largely focusing on their semantics and syntax; however, some recent models operate at the character level, such as characterbert @xcite and charbert @xcite . strikingly, character-level models excel at characterlevel tasks (e.g., spelling correction; @xcite and perform comparably to word-level models at language-modelling tasks @xcite . character-level models are therefore an ideal tool for studying the information and meaning encoded in n-grams beyond the realm of extant language. given that the current state-of-the-art is driven by transformer-based models, throughout our study, we use the characterbert model. char-acterbert is uniquely suited for our study as it uses a charactercnn module @xcite to produce single embeddings for any input token, built as a variant to bert which relies on sub-word tokenization @xcite . ## character-level language models for information analysis as described above, state-of-the-art language models serve as a tool to study meaning as it emerges though the distributional hypothesis paradigm. existing work on the analysis of transformers and bert-based models have explored themes we are interested in, such as semantics @xcite , syntax @xcite , morphology @xcite , and the structure of language @xcite . however, all of this work limits itself to the focus of extant words due to the word and sub-word-based nature of these models. we study the structure of the largely unexplored character n-gram space which includes extant language, pseudowords and garble character n-grams, seen through the representations created by char-acterbert, as follows. to explore how the character n-gram space is structured in the context of character based distributional semantics, we embed 40,000 extant english words, 40,000 randomly generated character n-grams, and 20,000 pseudowords. we choose the 40,000 most used english words that have been annotated for concreteness/abstractness ratings @xcite . randomly generated character n-grams are forced to have a string length distribution that matches the corpus of extant words we analyze. to generate pseudowords, we use a popular pseudoword generator. 2 the characterbert (el @xcite general model has been trained on nearly 40 gb of reddit data using character sequences. we leverage this model to create representations of character ngrams that may not have been seen in the training data. this allows us to use the resulting 512 dimensional embeddings for exploration via visualisation, topology modelling via distances and projections, and classification error analysis. ## structure of extant words along the information axis we use this section to discuss the structure of language across the information axis derived from our low-dimensional umap space. we structure our analysis across this axis as it organises the relative structure of extant words vs. randomly generated character n-grams, while also distinguishing internal structure within the extant word space. ## discussion and conclusion using the characterbert model, we embedded a large corpus of character level n-grams outside of extant language to study how the primitive information they contain relates to the semantic information carried by extant language. the key findings of this paper are: these findings suggest that character-based transformer models are largely able to explore the relation between extant words and randomly generated character strings. in particular, character-level models capture complex structure in the space of words, pseudowords, and randomly generated ngrams. these findings are consistent with work suggesting that character-level and morpheme-aware representations are rich in meaning, even compared to word or sub-word models @xcite @xcite @xcite . our study is limited to extant words in english and randomly generated character n-grams using the english alphabet. given the unique impact of a specific language and alphabet on representation spaces, there is motivation to see whether the relationships we identify generalise to other languages and alphabets. finally, we reiterate that our analysis was limited to the last embedding layer of the characterbert model; future work may focus on weights in earlier layers, including attention mechanisms explored by other bertology studies @xcite . by only analysing the final embedding layer, we study the 'psychology' of such character-level models; in analogy, much may be gained by studying the 'neuroscience' of such models encoded in their attention weights @xcite . our study also has important practical implications for the widespread use of pseudowords as an experimental tool in psycholinguistic research. pseudowords are frequently used as stimuli to observe the psychological and neurocognitive processes underlying the interpretation of novel words @xcite @xcite @xcite . however, the lion's share of this research treats all pseudoword stimuli as equivalent in their novelty, based on prima facie human judgments. by contrast, our method shows that not all pseudowords are created equal. due to various features of character sequences, including morphological structure, some pseudowords encode disproportionately more information according to character-aware language models, and are therefore represented as significantly more similar to extant words, whereas other pseudowords are recognized by these models as random character sequences. this variation is especially striking given that the algorithms used to generate pseudowords are highly constrained and designed to produce morphologically coherent words @xcite ; that some pseudowords are evaluated as random by charac-terbert reveals not only asymmetries in the coherence of pseudowords that may be of psychological relevance, but also assumptions and limitations in terms of which morphological units character-bert and related models recognize as signatures of extant words. our study thus provides a quantitative method for evaluating pseudoword plausibility, without relying on variable human judgments, while also revealing insights into key differences between how humans and contemporary language models evaluate the plausibility of pseudowords. to allow for further explorations and replicability, we release all of our data and code on github foot_3 . our findings reveal new avenues for future work using character-aware embeddings of extant, pseudoword, and garble n-grams, including analyses of nonsense poetry like lewis carroll's "jabberwocky" or of the innovative idiosyncrasies of rap lyricists and graffiti artists. the embeddings we study may also complement philological studies (especially if dynamic analyses are employed), as well as research into novel category formation @xcite . also, language acquisition studies of the distinction between language and noise may benefit from character-level embeddings beyond the realm of extant language @xcite . by investigating a broadened embedding space to include randomly generated n-grams, we found new structures of meaning through the context of meaningless information; further studies may extend our garble-based approach across different media and modes to contribute to more general understandings of human meaning.