The Impact of Visual Information in C hinese Characters: Evaluating Large Models’ Ability to Recognize and Utilize Radicals (2025)

Abstract: The glyphic writing system of Chinese incorporates information-rich visual features in each character, such as radicals that provide hints about meaning or pronunciation. However, there has been no investigation into whether contemporary Large Language Models (LLMs) and Vision-Language Models (VLMs) can harness these sub-character features in Chinese through prompting. In this study, we establish a benchmark to evaluate LLMs’ and VLMs’ understanding of visual elements in Chinese characters, including radicals, composition structures, strokes, and stroke counts. Our results reveal that models surprisingly exhibit some, but still limited, knowledge of the visual information, regardless of whether images of characters are provided. To incite models’ ability to use radicals, we further experiment with incorporating radicals into the prompts for Chinese language processing (CLP) tasks. We observe consistent improvement in Part-Of-Speech tagging when providing additional information about radicals, suggesting the potential to enhance CLP by integrating sub-character information.

## introduction visual information embedded in chinese characters is important, as most chinese characters convey a meaning equivalent to an entire word in english with a complex glyphic structure. multiple writing strokes form the radicals, foot_1 which often carry information about semantic meaning and pronunciation; the radicals are then visually combined to form chinese characters. when encountering unfamiliar characters, chinese speakers rely on semantic and phonetic hints from radicals, much like how english speakers use sub-words such as prefixes or suffixes, to approximate the meaning and pronunciation of unfamiliar words. for example, the chinese character "花" (meaning "flower"; pronounced as "huā") in figure 1 has "艹" (meaning "herbal") on the top, contributing to its semantic meaning, and "化" (pronounces as "huà") on the bottom, indicating its pronunciation. by utilizing the radical information, one can infer that "花" is related to herbs and has a pronunciation similar to "huà" without prior knowledge of the character. although radicals contain rich information, they have received little attention in digital text processing. contemporary typefaces treat chinese characters, radicals, and strokes as indivisible units, disregarding their compositional relationships. consequently, most language models follow this approach, under-utilizing the rich visual and semantic information embedded in chinese characters. while limited prior works @xcite @xcite have attempted to address this issue by incorporating visual embeddings, such as strokes or font images, into smaller-scale models (i.e., bert or neural mt), there remains a lack of research investigating whether these visual features can be recognized and utilized by models in light of the significant advancements in llms and vlms, especially in inference methods (e.g., prompting). to determine whether pre-trained llms recognize or can acquire the visual knowledge embedded in chinese characters, we establish a benchmark by collecting over 14,000 chinese characters from the chinese, japanese, and korean (cjk) unified ideographs, 3 considering four visual elements: radicals, composition structures, strokes, and stroke counts. as shown in figure 3 , the composition structure refers to the visual arrangement of a character's radicals (e.g., top-to-bottom or left-to-right). stroke composition provides an essential way to represent not typable radicals; as shown in figure 2 , some radicals cannot be typed using standard input methods but can still be accurately depicted through stroke compositions (see more details in appendix d.2). lastly, stroke count offers a measure for chinese characters' visual complexity, similar to word length in english. we conduct experiments on four tasks: structure recognition, radical recognition, stroke count identification, and stroke identification. we evaluated a series of llms and vlms (e.g., gpt-4, gemini-1.5, ernie-4, aya-command, qwen-7b, etc.), and found that all models possess some visual knowledge of chinese characters, even without image inputs; however, it is only to a limited extent. in particular, the models tend to perform well in recognizing the first radical of a chinese character, such as "艹" (herbal) in "花"(flower), but often fail with subsequent ones. we also demonstrate that the pixel-based encoder pixel @xcite has the ability to capture structural information effectively after fine-tuning. as a language model pre-trained only on an english corpus, pixel achieved an f1 score of 84.57, significantly higher than the second-best score of 54.30 achieved by ernie-4 4 and 23.29 by gpt-4 when provided with images of characters, indicating its potential for clp as it naturally captures visual information. we further investigate whether models can utilize radicals to improve performance on understanding tasks (e.g., pos tagging and ner) by prompting them to use radicals when encountering unfamiliar words. our experiments show that radical information yields promising results in downstream tasks, particularly in pos tagging. we observe consistent improvement across models and datasets when the information about radicals is provided. notably, ernie-lite-8k's pos tagging f1 score on gsd @xcite decreases by 2.1 3 the cjk unified ideographs refers to a set of chinese characters used across chinese, japanese, and korean languages to standardize and unify the use of characters. 4 released by chinese company baidu that rivals gpt-4. points when recognizing radicals on its own, but increases by 5.7 points when provided with correct radicals. for name entity recognition (ner), we also observe an improvement in three out of six models. analyzing the cases where incorporating radical degrades the model performance, we see that incorrect answers often occur when the model fails to identify unfamiliar words and bypasses the radical information process, indicating the decrease is likely due to long prompts. when evaluating only sentences where the model detects unfamiliar words, performance on ner generally improves. our work demonstrates that models possess the ability to recognize and utilize radical information only to a certain limit, suggesting that deeper integration, such as additional training on radicals or improvement in the chinese digital system to incorporate radicals, could unlock further potential. chinese character decomposition in computer vision. the task of decomposing chinese characters into constituent components has majorly been studied in the field of computer vision. research within this domain, such as the studies by ma et al. (2021), xia (1994), and liu et al. (2021), has explored analogous challenges. the work by zhang et al. ( 2018 ) employs a methodical approach by categorizing characters into structured types and further decomposing sub-components according to their spatial arrangements-akin to the layered structural analysis that we adopt in this paper. ## evaluation on recognizing visual information of chinese characters to evaluate whether language models contain or can learn the visual information embedded in chinese characters, we establish a benchmark by set-ting up a series of tasks (see examples of each task in figure 3 ) derived from our dataset. ## evaluation on utilizing radicals we evaluated llms on downstream tasks, specifically examining performance differences when models are prompted vs. not prompted to use their knowledge of radicals to infer the meaning of unfamiliar words as illustrated in figure 5 . ## conclusion in this paper, we create a comprehensive benchmark on visual information embedded in chinese characters. our evaluation of the benchmark highlights the suboptimal performance of llms and vlms in handling information below the character level. despite this, our experiments with radical prompting demonstrate that these sub-character features can still be beneficial. the results show consistent improvements in pos tagging when correct radicals are provided, and promising results in ner on sentences containing unfamiliar words. our work highlights the potential of radical knowledge for chinese nlp applications and advocates attention to help models leverage it, including additional training on radicals or improving chinese digital systems to more effectively integrate radical structures.