A Comparative Analysis of Word Segmentation, Part-of-Speech Tagging, and Named Entity Recognition for Historical C hinese Sources, 1900-1950 (2025)

Abstract: This paper compares large language models (LLMs) and traditional natural language processing (NLP) tools for performing word segmentation, part-of-speech (POS) tagging, and named entity recognition (NER) on Chinese texts from 1900 to 1950. Historical Chinese documents pose challenges for text analysis due to their logographic script, the absence of natural word boundaries, and significant linguistic changes. Using a sample dataset from the Shanghai Library Republican Journal corpus, traditional tools such as Jieba and spaCy are compared to LLMs, including GPT-4o, Claude 3.5, and the GLM series. The results show that LLMs outperform traditional methods in all metrics, albeit at considerably higher computational costs, highlighting a trade-off between accuracy and efficiency. Additionally, LLMs better handle genre-specific challenges such as poetry and temporal variations (i.e., pre-1920 versus post-1920 texts), demonstrating that their contextual learning capabilities can advance NLP approaches to historical texts by reducing the need for domain-specific training data.

## introduction with the large-scale digitization of historical documents, researchers are increasingly interested in how natural language processing (nlp) methods might be used and adapted to address the unique characteristics of older texts @xcite @xcite . classification models for tasks such as named entity recognition (ner) have improved significantly with the development of neural-based approaches. however, their precision for historical materials still lags behind that of models trained on contemporary texts @xcite . recent applications of language model-based approaches to nlp tasks have shown mixed results for using large language models (llms) such as chatgpt to generate universal ner output @xcite , including for historical documents @xcite . more targeted, domain-specific approaches have also proven effective (polak and morgan, 2024), including classification tasks common in digital humanities research @xcite and in low-resource settings @xcite . the processing of historical chinese documents presents unique challenges for nlp tasks due to the logographic writing system, the absence of natural word boundaries, and the rich morphological structures embedded within individual characters @xcite . previous work on relatively "simple" tasks, such as chinese word segmentation, has evolved through three paradigm shifts: rule-based systems, statistical machine learning models, and llms based on the transformer architecture @xcite . traditional machine learning methods such as jieba and spacy rely on dictionary matching and hidden markov models to identify word boundaries. the dramatic linguistic and logographical transformations that occurred in china during the late nineteenth and twentieth centuries @xcite pose particular challenges for these models, which struggle to handle out-of-vocabulary terms. some researchers have approached this problem by first converting historical sources into standardized simplified chinese before performing nlp tasks @xcite . others have drawn from domainspecific approaches to manually curate datasets from historical sources to improve tasks such as segmentation @xcite . the advent of llms capable of detecting contextual patterns from large corpora presents new opportunities for processing classical and modern chinese texts. although there has been growing interest in bert-based models and the development of domain-specific tools to process historical chinese sources @xcite @xcite , further research is needed to evaluate llms' performance on nlp tasks. this short research paper presents a comparative analysis of machine learning and llm-based tools for word segmentation, part-of-speech (pos) tagging, and ner on a diverse set of sample texts taken from the shanghai library republican journal corpus. 1 this study finds that, for transitional-era chinese texts, llm-based approaches outperform traditional nlp tools on segmentation, pos tagging, and ner tasks. however, these improvements come with notable increases in computational costs, highlighting a trade-off between performance and efficiency. ## methodology to create our ground truth files, we extracted a random sample of passages from a large textual dataset of late-qing and republican periodicals held by the shanghai library. we identified 208 passages spanning the decades 1900 to 1950. these passages include a variety of genres and topics, such as government reports, academic writing, social and political commentary, and literary texts such as short stories and poetry. to assess the ability of existing tools to handle different genres and textual changes over time, our sample included 41 passages identified as poetry, with the the remaining 167 passages distributed across five decades: 1900 (23 passages), 1910 (32 passages), 1920 (40 passages), 1930 (34 passages) and 1940 (38 passages). the passages ranged in length from 6 to 170 characters, with an average of 41.3 characters and a total of 8,610 characters. from this sample, the authors collectively segmented and tagged the passages, with each passage verified by two authors. discrepancies were noted and resolved after further discussion. we selected widely used and reputable tools for chinese segmentation and ner, as well as several popular llms, to evaluate out-of-the-box performance of these tools in our comparative analysis. to evaluate their effectiveness, we generated consistent prompts for each llm and utilized their apis to ensure standardized conditions. the prompts included clear, precise instructions requiring that the results be provided in a structured json format. this enabled a straightforward comparison with our established ground-truth dataset. ## discussion the impact of word segmentation choices on digital humanities (dh) and cultural analytics research, particularly for late 19th century to mid 20th century "transitional" chinese texts, is significant and multifaceted. proper segmentation enhances downstream tasks such as data analysis, pattern recognition, and cross-lingual/temporal studies. it improves the accuracy of frequency analyses, topic modeling, and semantic network analyses, while also making both transitional and classical chinese texts more accessible. while some have argued that word segmentation is becoming less relevant in nlp pipelines @xcite , researchers in the humanities and social science still find it crucial. character-based or sub-character methods (e.g., byte pair encoding) often fall short for dh applications, where accurately representing search keywords and concepts is often prioritized over processing efficiency. proper segmentation enables nuanced identification of linguistic patterns and cultural trends over time, facilitating comparative studies across languages and historical periods. domain-specific models like pkuseg (which has been integrated into spacy as its default tokenizer) offer improvements over generic tools, but to date have failed to curate training data for historical texts @xcite . llms show promise in overcoming these limitations through contextual learning. however, fine-tuning llms or adopting a ## conclusion llms have demonstrated improved performance in handling complex chinese language tasks, consistently outperforming traditional nlp tools across all metrics. llms also showed greater resilience in processing both poetic texts and language spanning multiple decades. these improvements over traditional tools like jieba and spacy highlight the