Challenges in Processing C hinese Texts Across Genres and Eras (2025)

Abstract: Pre-trained Chinese Natural Language Processing (NLP) tools show reduced performance when analyzing poetry compared to prose. This study investigates the discrepancies between tools trained on either Classical or Modern Chinese prose when handling Classical Chinese prose and Classical Chinese poetry. Three experiments reveal error patterns that indicate the weaker performance on Classical Chinese poemsis due to challenges identifying word boundaries. Specifically, tools trained on Classical prose struggle recognizing word boundaries within Classical poetic structures and tools trained on Modern prose have difficulty with word segmentation in both Classical Chinese genres. These findings provide valuable insights into the limitations of current NLP tools for studying Classical Chinese literature.

## introduction the creation of classical chinese treebanks for prose and poetry has enabled training nlp systems across different eras and genres. this study conducts a comparative analysis, examining the performance of nlp tools trained on either classical and modern chinese and either poems or prose. especially, we analyze how nlp systems trained on prose perform when applied to poems, finding that high performance on classical chinese prose (yasuoka, 2019) is reduced on poetry and providing a preliminary explanation why this reduction happens on classical chinese poems. classical chinese, the language of ancient chinese literature, differs significantly from modern chinese in style, vocabulary, and grammar. classical chinese texts are characterized by the absence of spaces and punctuation, appearing as continuous character strings @xcite , presenting a challenge for word segmentation. for example, a classical chinese sentence with 11 characters translates to modern chinese as 29 charac-ters. classical chinese prose is usually expressed in longer, variable-length sentences, whereas classical chinese poetry is usually tightly constrained to five-or seven-character lines. prose also tends to employ more elaborate syntactic structures, in contrast to poetry's concise, rhythmically disciplined forms that often create a more striking aesthetic @xcite . the application of nlp to classical chinese poetry and prose is divided into two categories. first is the archiving and generation of classical poetry and prose. the second is the theme and emotion classification of classical poetry @xcite . since the first step in emotion analysis is text pre-processing, including word segmentation @xcite , the present study can help improve emotion analysis. our findings confirm that the reduced performance in poetry is due to difficulties in identifying word boundaries. these insights can be used to enhance the use of existing nlp tools in the study of classical chinese literature by highlighting opportunities for adapting nlp technologies from other chinese eras and genres. ## data and tools this study analyzed parser performance on two syntactic treebanks of classical chinese: one made up of prose and one for poems. we compared the performance of stanza's modern chinese (traditional) pipeline @xcite ) on both datasets. stanza is an open-source python nlp toolkit supporting 66 human languages. we used stanza instead of the kyoto processor for both datasets for consistency. stanza's modern chinese (traditional) pipeline is able to align the traditional characters used in both modern and classical chinese, avoiding the need to convert characters. this pipeline was originally trained on modern chinese prose. dataset 1: classical chinese poems: classi-cal chinese poetry is connected with particular historical periods, such as the poetry of the tang dynasty. the first dataset used in this study is sourced from the cityu treebank of classical chinese poems @xcite . dataset 1 contains 60 poems written by the esteemed poet du fu (712-770ad), totaling 300 sentences. each poem follows a five-or seven-character fixed-length format. dataset 2: classical chinese prose: this dataset comprises the first 500 sentences from the kyoto treebank @xcite . the kyoto treebank features complete texts of the four books, which are written in prose. sentences generally longer than seven characters, in contrast to the poetry in dataset 1. the kyoto python nlp tool @xcite which is trained and tested on the kyoto treebank of classical chinese prose for tokenization (99.5% accuracy) and pos tagging (90.8% accuracy) remains the first and only python-based nlp tool for classical chinese. since the kyoto tool was trained on the kyoto treebank that makes up dataset 2, we used stanza's modern chinese (traditional) pipeline in our experiments with dataset 2 rather than the the kyoto parser. ## experiment setup to better understand the performance of a tool trained on prose when asked to parser classical po-ems, we conducted three experiments and then analyzed error patterns in two prose parsers applied to classical chinese poems. we evaluate two nlp tools, one for classical chinese and one trained on modern chinese on two datasets: classical chinese poems and classical chinese prose. we analyze the tools' ability to handle word segmentation and pos tagging in the two classical chinese genres. dependency parsing was excluded due to the complexity of manually extracting relations from the cityu treebank of classical chinese poems. our genre-specific analysis compares the performance of both tools on classical poetry and assesses the modern chinese tool on classical prose. we compute accuracy, recall, precision, and f1-score for segmenting one-and two-character words and then identify frequently misclassified pos categories and analyze specific misclassification pairs. experiment 1 examines how the classical chinese tool handles poems in classical chinese (dataset 1). experiment 2 evaluates how a modern chinese tool handles classical chinese prose (dataset 2), serving as a comparative benchmark for experiment 3. experiment 3 evaluates the effectiveness of a modern chinese tool on dataset 1. ## results in this section, we present the experiment results and the main findings. ## conclusion this study explores differences in nlp parsers trained on classical or modern chinese prose in handling prose and poetry from the different eras and analyzes the error patterns to better understand the differences in performance. we aim to contribute to developing a robust nlp tool that accurately distinguishes between these eras and genres. the findings suggest that reduced performance in classical poetry analysis is due to difficulties in identifying word boundaries by a tool trained on a different genre or era of the language. tools trained on classical prose struggle to segment words in poetic structures, while modern parsers struggle with word segmentation in both classical chinese genres. we show how this difficulty affects downstream pos tagging in classical chinese poems. in classical chinese poems, verbs are commonly misclassified by the classical chinese processor, whereas the modern chinese parser commonly misclassifies particles and nouns. future research should look at those common misclassified categories more closely to evaluate whether there are more detailed patterns that may help improve performance on classical chinese poetry. future work should apply large language models (llms) to full-scale datasets and compare their outputs against standard parsers. for example, a classical chinese-specific llm, tonggu @xcite , was recently developed. we prompted the gpt oss 20b language model to perform word segmentation and pos tagging on two classical chinese poems (86 characters in total, 10 sentences). the model's performance fell short of both the kyoto processor and stanza with segmentation accuracy at approximately 42% and pos tagging accuracy achieving only about 1%. our analysis informs the challenges of adapting nlp technologies to various eras and genres in the same language, highlighting the limitations of current tools in studying classical chinese literature, especially poems. these insights could be used to enhance text preprocessing, thereby improving emotion classification and other nlp tasks for classical chinese poetry. moreover, these insights can support many languages that lack sufficient early texts for training parsers, and so using a modern language version is a practical bootstrap solution.