Spelling-out is not Straightforward: LLM s’ Capability of Tokenization from Token to Characters (2025)

Abstract: Large language models (LLMs) can spell out tokens character by character with high accuracy, yet they struggle with more complex character-level tasks, such as identifying compositional subcomponents within tokens. In this work, we investigate how LLMs internally represent and utilize character-level information during the spelling-out process. Our analysis reveals that, although spelling out is a simple task for humans, it is not handled in a straightforward manner by LLMs. Specifically, we show that the embedding layer does not fully encode character-level information, particularly beyond the first character. As a result, LLMs rely on intermediate and higher Transformer layers to reconstruct character-level knowledge, where we observe a distinct “breakthrough” in their spelling behavior. We validate this mechanism through three complementary analyses: probing classifiers, identification of knowledge neurons, and inspection of attention weights.

## introduction while large language models (llms) have grown remarkably in recent years, several studies report that they still struggle with fine-grained characterlevel manipulations, such as inserting, deleting, or extracting individual characters within tokens @xcite @xcite . although most llms operate over subword tokens, true mastery of subtoken information is essential for a range of applications, such as morphological inflection @xcite , letter counting @xcite , typoglycemia @xcite , and handling typos @xcite . to improve their reliability in such scenarios, we must understand how llms internally represent and process characters. a paradox emerges from prior work: llms can accurately spell out entire tokens as sequences of characters @xcite , llms spell out tokens by relying directly on embedding-level information for the first character, but gradually shift to using distributed, higher-layer representations for later characters. we found a "breakthrough" layer where character knowledge becomes detectable. while they often fail at simple tasks such as identifying a single character at a fixed position @xcite @xcite . for instance of a token "language," well-pretrained models readily generate "l a n g u a g e" on demand but cannot reliably extract "u" at position five. this discrepancy suggests that, despite lacking explicit access to compositional character knowledge, llms have some mechanism for character-by-character spelling. in this paper, we investigate how and where llms capture and deploy character-level knowledge during spelling-out. we begin by constructing a token-characters dataset from the vocabularies of four representative llms ( §3) and confirming their ability to spell out tokens with few-shot prompts ( §4). probing the embedding layer reveals that it does not encode subtoken characters directly ( §5), but downstream transformer layers gradually recover this information. we identify a clear "breakthrough" layer at which character identities become reliably detectable by our probing classifier ( §6), and observe that the first few characters are handled differently from subsequent ones. finally, through neuron-level analyses ( §7) and attention-weight inspections ( §7.4), we trace how character knowledge is stored and routed, demonstrating that its locus aligns precisely with the breakthrough point. taken together, our study sheds new light on the internal character-level machinery of llms. our main contributions are as follows: codes for our experiments are available at: https://github.com/tathi/token2char . ## related work our research is in line with work focusing on the llms' capability of character-level manipulation. @xcite investigated the llms' capability to identify the appearance of particular characters in a given token using probing classifiers. @xcite directly analyzed the information stored in token embeddings to spell out tokens using additional characterlevel language models. more recently, @xcite introduced a dataset to evaluate this capability from some viewpoints of character-level manipulation, including spelling-out of words or tokens. similarly, @xcite provided a dataset of complicated character-level manipulation tasks. both works conclude that llms have limited capability for complicated character processing. furthermore, these works reported that llms can spell out words in the original order, as reported in @xcite , while they do not have sufficient knowledge of their single characters inside words @xcite @xcite unless finetuning for the models to learn the token internal information directly @xcite . this line of literature motivates us to investigate the llms' internal workings of spelling-out behavior, despite the lack of character-level knowledge. recently, we can see a trend of understanding llms' capability of recognizing character-level information, such as the ability of counting characters @xcite . moreover, beyond the word-to-character spelling out, @xcite investigates their ability to recognize radicals inside chinese characters. our work is also related to the findings of llms' "detokenization" ability in their later layers @xcite , which is an ability to internally merge tokens into words or phrases. in contrast, we focus on the inverse problem: how llms internally "tokenize" tokens into characters. ## llms can spell-out tokens we first assess each llm's ability to spell out single tokens given few-shot examples, following prior work on word-level spelling-out @xcite . the data points named "entire" in figure 2 report the overall token-level spelling-out accuracy of each llm. although direct comparison across models is impossible because each uses its own vocabulary subset, we observe substantial differences: llama3-8b achieves 94.41% accuracy, whereas amber-6.7b reaches only 58.86%. this variation shows the impact of model architecture and pretraining data on character-level capabilities. in figure 2 , we also report accuracy by character position within each token. all models achieve over 94% accuracy on the first character, but accuracy steadily declines for later positions. this result indicates that correctly generating characters further along in the token becomes increasingly difficult. notably, however, every model maintains over 70% accuracy through the fifth character, demonstrating llms' robust mid-token spelling capability. ## embeddings do not know all compositional characters while section 4 demonstrated that llms can spell out single tokens, two key questions remain: where is the character-level knowledge stored in the model, and how is that knowledge utilized during spelling? a natural hypothesis is that token embeddings themselves encode this information @xcite , since the spelling of a token (i.e., its sequence of characters) is inherently contextindependent and embeddings are derived directly from token identities. this section investigates whether the token embedding stores the knowledge of spelling tokens. ## knowledge neuron for spelling-out the breakthrough point observed in §6 implies that large language models (llms) possess the ability to spell out tokens in the layers preceding this point. recent work on interpretability has shown that individual neurons in transformer models may encode factual knowledge or skills, as in knowledge neurons @xcite and skill neurons @xcite . inspired by this line of research, we investigate where the knowledge for spelling out tokens resides within the neurons of each llm. ## attention weight for spelling-out given the nature of the spelling-out task, llms must attend to the target token while generating its characters. the ability to focus attention on the correct token is crucial for accurate spelling-out, and we hypothesize that this ability corresponds to the breakthrough identified in §6. this section further investigates the relationship between spelling-out capability and attention weights to the target token. given a sequence consisting of the target token and its spelling-out from our dataset, we compute the average attention weight from all related elements to the target token. for example, when the input is [few-shot examples] token : t o k e n, where the target token is "token" following a few-shot example, we calculate the average attention weight to "token" from each of the elements: token, :, t, o, k, e, and n. we average these attention weights over 1,000 randomly selected samples and across all attention heads in each layer 4 . to isolate attention to the target token, we remove attention weights to "<@xmath0, inspired by @xcite . figure 8 presents the average attention scores across the 1,000 examples. interestingly, the layer with the highest attention to the target token (red bars) coincides with the breakthrough point (red lines) in three out of four models. this finding suggests that the performance improvement observed in intermediate layers is due to the model's increasing ability to correctly attend to the target token. the distinct result observed in amber-6.7b suggests that performance breakthroughs do not necessarily align with the behavior of attention weights. given the overall lower attention weights assigned to the target token compared to other models, we consider that this model needs to attend to broader contextual information rather than focusing on the single target token, possibly due to a lack of character-level knowledge in its embeddings. ## conclusion this paper investigated the paradox that, although large language models (llms) struggle to recog-4 these tokens are selected from the dataset as those that each llm can correctly spell out in the few-shot setting ( §4). our probing analyses revealed that the token embedding layer does not encode complete characterlevel information, especially beyond the first character. instead, character-level features are dynamically reconstructed in the intermediate and later layers, where we observe a distinct "breakthrough" point. around this stage, models begin to reliably access and utilize character-level knowledge. we further examined the spelling-out behavior through the lens of knowledge neurons and attention patterns, both of which align with the breakthrough layers. crucially, this behavior suggests that spelling out is a learned task (i.e., dependent on identifying the target token and retrieving characterlevel information), rather than a simple extraction of character-level information stored in the embedding layers. as a result, our findings imply that the apparent success of llms in spelling out does not extend to more complex or unfamiliar tasks such as reverse spelling, letter insertion, or character-level reasoning in novel contexts. our findings indicate that current llms are not inherently character-aware; rather, they rely on task-specific heuristics acquired during training or prompting. this suggests that models must explicitly learn how to apply character knowledge to more complex manipulations.