A Joint Model of Automatic Word Segmentation and Part-Of-Speech Tagging for Ancient Classical Texts Based on Radicals (2023)

Abstract: The digitization of ancient books necessitates the implementation of automatic word segmentation and part-of-speech tagging. However, the existing research on this topic encounters pressing issues, including suboptimal efficiency and precision, which require immediate resolution. This study employs a methodology that combines word segmentation and part-of-speech tagging. It establishes a correlation between fonts and radicals, trains the Radical2Vec radical vector representation model, and integrates it with the SikuRoBERTa word vector representation model. Finally, it connects the BiLSTM-CRF neural network.The study investigates the combination of word segmentation and part-of-speech tagging through an experimental approach using a specific data set. In the evaluation dataset, the F1 score for word segmentation is 95.75%, indicating a high level of accuracy. Similarly, the F1 score for part-of-speech tagging is 91.65%, suggesting a satisfactory performance in this task. This model enhances the efficiency and precision of the processing of ancient books, thereby facilitating the advancement of digitization efforts for ancient books and ensuring the preservation and advancement of ancient book heritage.

## introduction the challenge of automatically segmenting words and assigning part-of-speech tags to ancient chinese text is a crucial area of study within the discipline of natural language processing. the primary objective of this project is to employ computer technology for the precise identification of word boundaries in ancient chinese writings, as well as the exact assignment of appropriate partof-speech labels to these words, including nouns, verbs, conjunctions, and others. by implementing this procedure, the conventional task of manual labelling can be efficiently alleviated, leading to notable enhancements in both labelling efficiency and accuracy. the progress of this technology not only facilitates the processing of ancient chinese texts, but also exerts a significant influence on interconnected disciplines, including literature, history, philology, and digital humanities. the maturation of ancient chinese automatic word segmentation and part-of-speech tagging technologies is occurring with the ongoing advancement of computer technology. nevertheless, the current utilisation of these methodologies is confronted with two distinct obstacles as a result of the numerous distinctive characteristics of ancient chinese. firstly, it is worth noting that there remains potential for enhancing the precision of word segmentation and part-of-speech labelling in the context of ancient chinese. while several technologies currently available have the capacity to partially substitute manual labelling, their level of accuracy falls short of totally replacing manual labelling. consequently, a significant amount of proofreading labour is necessary during the later stages. secondly, there is a need for additional enhancement in the effectiveness of word segmentation and partof-speech tagging in the context of ancient chinese. the prevailing approach involves conducting word segmentation as the initial step, followed by partof-speech tagging. nevertheless, this sequencing will result in diminished processing efficiency and has the potential to propagate errors in word segmentation to the subsequent part-of-speech tagging phase, so exacerbating the influence of these errors and subsequently diminishing overall accuracy. this paper utilizes the word2vec model to incorporate the radical information of chinese characters. it proceeds to train the radical2vector model and combines it with sikuroberta to form the embedding layer. subsequently, the bilstm-crf neural network is connected to conduct an experiment on the integration of word segmentation and part-of-speech tagging in ancient chinese. the utilization of ancient chinese word segmentation and part-of-speech tagging facilitates the exploration of profound insights within ancient texts, thereby advancing the digital advancement and utilization of these texts. furthermore, it contributes to the preservation and progression of ancient literary works. ## related work the co-examination of automatic word segmentation and part-of-speech tagging in the context of ancient chinese is a common area of research. @xcite conducted a study on part-of-speech tagging in ancient chinese using the hidden markov model. they applied this model to analyze "the analects of confucius" and "tao te ching". although the study employed a set of 22 part-of-speech tags, it made significant contributions to the field. in their study, @xcite developed a text segmentation program called yu segmentation program. the researchers focused on ancient books such as "the classic of tea" and employed a model algorithm that utilized tree pruning to achieve efficient text segmentation of these classical texts. the f1 score for word segmentation has been reported to be approximately 86% by min @xcite . a comparative experiment was conducted to evaluate the performance of the conditional random fields (crf) model in the tasks of automatic word segmentation, part-of-speech tagging, and integration of ancient chinese. both features and integrated processing contribute to the enhancement of the f1 value. runhua @xcite proposed a method that utilizes structured annotations to enhance the word segmentation process. in their study, shuiqing @xcite employed the crf model to analyze word categories, phonetics, and probability features. notably, their analysis yielded a remarkable f1 value of 97.47%. according to the study conducted by xiaoyu @xcite , this paper examines the issue of automatic word segmentation in middle ancient chinese by employing a combination of the crf model and a dictionary. it also investigates the impact of inconsistent word segmentation on the results of artificial word segmentation in middle ancient chinese through experimental analysis. additionally, the paper introduces character classification as part of the research methodology. the dictionary information exhibits two notable features. firstly, the word segmentation f1 value achieved a remarkable accuracy rate of over 99% in the closed test. secondly, in the open test, the word segmentation f1 value ranged between 89% and 95%, further highlighting the effectiveness of the dictionary information. ning cheng (2020) employed the word2vec-bilstm-crf model to investigate the amalgamation of part-of-speech tagging for sentence segmentation and part-of-speech analysis in ancient chinese texts. numerous studies have been conducted on the utilization of vector representations of strokes, parts, components, and radicals to facilitate chinese information processing, both in contemporary and ancient contexts. in their study, @xcite introduces a new model called dual-channel word embedding (dwe) that aims to effectively capture both sequential and spatial information of characters. the author argues that this model demonstrates a logical and advantageous approach in representing the morphology of chinese language. in their study, @xcite presents a novel model called the feature subsequence based probability representation model (fsprm) for the purpose of acquiring chinese word embeddings. the model incorporates both morphological and phonetic features, specifically stroke, structure, and pinyin, of chinese characters. by designing a feature subsequence, the model captures a wide range of semantic information pertaining to chinese words. the efficacy of the proposed method is substantiated through a series of comprehensive experiments conducted on various tasks including word analogy, word similarity, text classification, and named entity recognition. the results of these experiments consistently indicate that the proposed method surpasses the performance of the majority of existing state-of-the-art approaches. in the study conducted by @xcite , a novel deep learning technique referred to as "radical embedding" is introduced. the author provides a rationale for this approach by drawing upon principles derived from chinese linguistics. furthermore, the feasibility and usefulness of this technique are assessed through a series of three experiments. in their study, @xcite presents a method for simultaneously embedding chinese words, characters, and subcharacter components at a detailed level. the performance of our model is shown to be superior through evaluation on both word similarity and word analogy tasks. in their study, han (2018) utilized a shared radical level embedding approach to address the task of simplified and traditional chinese word segmen-tation. notably, their method does not require any additional conversion from traditional to simplified chinese. the integration of radical and character embeddings results in a reduction in parameter count, while facilitating the sharing and transfer of semantic knowledge between the two levels. this integration significantly enhances performance. in their recent publication, tang (2021) introduces a pioneering model named moto, which aims to enhance embedding through the incorporation of multiple joint factors. the empirical findings indicate that our moto model attains state-of-the-art performance with an f1-score of 0.8316, representing a 2.11% improvement, when applied to chinese news titles. furthermore, it achieves an accuracy of 96.38 (a 1.24% improvement) on the fudan corpus dataset and 0.9633 (a 3.26% improvement) on the thucnews dataset. among the various research endeavors, the investigation into the utilization of radical vectors stands out as the most prominent. on one hand, this phenomenon can be attributed to the relatively straightforward acquisition of the corresponding relationship data between chinese characters and their radicals. on the other hand, the inclusion of radical vectors has been found to enhance the efficacy of chinese information processing tasks. it is evident that among the aforementioned studies, only one specifically addresses the topic of ancient chinese classical chinese, with a specific focus on automating sentence segmentation tasks. the absence of vector representations for strokes, parts, components, and radicals in ancient chinese information processing has the potential to enhance the morphology of ancient books. this article endeavors to analyze the impacts of research. this study exclusively focuses on the radical vector representation and application of ancient chinese characters, primarily due to limited resources. ## integrated labeling strategy the tasks of chinese automatic word segmentation and part-of-speech tagging are typically performed independently, with the outcome of automatic word segmentation serving as the foundation for part-ofspeech tagging. hence, the general approach in chinese lexical analysis involves the sequential implementation of automatic word segmentation followed by part-of-speech tagging. the concept of integrated tagging can be attributed to shuanhu @xcite , who proposed a combined approach for word segmentation and part-of-speech tagging to address the issue of ambiguous domains in contemporary chinese automatic word segmentation. however, shuanhu bai did not conduct a comprehensive assessment of the practicality of integrated tagging. ng (2004) provide a comprehensive anal-ysis of the viability of integrated tagging in their scholarly work. the authors conducted a comparative analysis of two strategies for chinese word segmentation, namely part-of-speech tagging and integrated tagging, using the maximum entropy model. the findings indicate that the integrated method, which relies on word annotation, demonstrates superior performance. the initial utilization of the integrated tagging method in the domain of ancient chinese can be attributed to the research conducted by min @xcite . the crf model was employed to carry out experiments on word segmentation and part-of-speech tagging for pre-qin chinese. the findings of the study indicated that the integrated strategy was effective. in comparison to the two-step strategy, it demonstrates a notable enhancement in the efficacy of word segmentation and part-of-speech tagging. hence, this study also employs an integrated labeling approach. to achieve integrated labeling, the output label of each word is determined by combining the word's position and its corresponding part of speech. the lexical tagging system for word position information consists of a set of four lexemes: b for begin, i for middle, e for end, and s for a single word. the "basic collection of pre-qin chinese parts of speech tags" prescribes the use of part-of-speech tags. for instance, the tag "v" is employed to indicate verbs, while the tag "n" is used to indicate nouns, among others. the hyphen (-) serves as a connector between the lexeme marker and the part-of-speech marker. an illustration of this can be seen in the token b-v, which represents a word that initiates a verb. ## evaluation in this study, the radical2vector model was selected as the representation model for ancient chinese radicals. the radical vector was combined with the word vector and fed into the same set of bilstm hidden layers. this approach, referred to as the first radical vector input method, was employed. the incorporation of radical vectors enhances the efficacy of part-of-speech tagging; however, its impact remains somewhat constrained. to assess the impact of the model proposed in this research paper, the evaluation metrics employed include accuracy rate (p), recall rate (r), and harmonic mean (f1). the model presented in this study is then compared to the outcomes achieved by participating teams in the open test testa of the first international ancient chinese word segmentation and pos tagging bakeoff @xcite . the findings are juxtaposed, as illustrated in ## discussion innovation can improve ancient chinese word segmentation and part-of-speech labeling. reasons include these. ancient chinese radicals are related with form and meaning. the word segmentation and part-of-speech tagging model captures ancient chinese character structural similarities better by integrating radical information. the resemblance helps the model reliably identify and categorize ancient chinese characters, improving word segmentation precision. radicals also relate to ancient chinese character semantics. radical information helps the model learn radical semantics and apply them to part-of-speech labeling. some radicals associate with nouns, whereas others with verbs or adjectives. semantic information can improve partof-speech labeling. ancient chinese word segmentation and part-of-speech tagging require knowledge of ancient literature and culture. radicals are a vital part of ancient chinese characters. innova-tive information improves the word segmentation and part-of-speech tagging model's understanding of historical manuscripts' lexicon and expressions, improving ancient chinese language processing computational capabilities. lexical analysis is better with integrated tagging. the integrated labeling technique reduces category labels during multi-classification tasks like lexical analysis. this improves lexical analysis. this work uses the four-lexeme tag set for automatic word segmentation and 21 part-of-speech tags for tagging. integrated tagging reduced the training set of "zuo zhuan" to 59 integrated tags. strategy has 84 category labels. this is because ancient chinese auxiliary words (u), quantifiers (q), and concurrent words (j) were single-character terms. these linguistic elements are only combined with the singleword marker (s), not with beginning (b), medial (i), or final (e) markers. the "zuo zhuan" dataset contains terms without three-character words. this applies to prepositions, adverbs, modal particles, and onomatopoeia. this method reduces class labels further by adding in-word (i) tagging. certain characters vary and limit the part-of-speech scope of their words on different lexemes. this limits character consequences. thus, the integrated tagging technique integrates external knowledge and automated processing by leveraging the interrelated and complimentary nature of word segmentation and part-of-speech information. thus, this study labels everything. ## conclusion this study employs deep learning techniques to extract the radical information of chinese characters, thereby achieving the integration of automatic word segmentation and part-of-speech tagging in ancient texts. this study utilizes a dataset comprising over 70,000 chinese characters and their corresponding radicals to establish a correlation between fonts and radicals. additionally, it employs the radical2vector model to train a radical vector representation. an experiment was conducted on the "zuo zhuan" dataset to examine the integration of word segmentation and part-of-speech tagging, utilizing the sikuroberta-radical2vector-bilstm-crf model in conjunction with the original sikuroberta. the model's automatic word segmentation achieved an f1 value of 95.75% on the test set, while the automatic part-of-speech tagging achieved an f1 value of 91.65%. the present