Enhancing A ncient C hinese Understanding with Derived Noisy Syntax Trees (2023)

Abstract: Despite the rapid development of neural-based models, syntax still plays a crucial role in modern natural language processing. However, few studies have incorporated syntactic information into ancient Chinese understanding tasks due to the lack of syntactic annotation. This paper explores the role of syntax in ancient Chinese understanding based on the noisy syntax trees from unsupervised derivation and modern Chinese syntax parsers. On top of that, we propose a novel syntax encoding component – confidence-based syntax encoding network (cSEN) to alleviate the side effects from the existing noise caused by unsupervised syntax derivation and the incompatibility between ancient and modern Chinese. Experiments on two typical ancient Chinese understanding tasks, ancient poetry theme classification and ancient-modern Chinese translation, demonstrate that syntactic information can effectively enhance the understanding of ancient Chinese over strong baselines, and that the proposed cSEN plays an important role in noisy scenarios.

## introduction ancient chinese literature, such as classical poetry, books, and records, is a highly representative and distinctive cultural heritage that is receiving increasing attention from the nlp academia. however, directly applying modern chinese processing methods to ancient texts is not appropriate due to the differences in syntax and semantics between ancient and modern chinese. chinese is one of the oldest written languages in the world, with a history of at least 6,000 years @xcite . over time, the language has undergone many changes, such as the transition from literary to vernacular chinese in the early 20th century @xcite , resulting in a significant gap between ancient and modern chinese. syntactic features has been utilized in a wide range of nlp tasks, including coreference resolution @xcite @xcite , machine reading comprehension @xcite , and machine translation @xcite @xcite . despite the effectiveness of syntax in modern chinese understanding @xcite @xcite , few studies have incorporated syntactic information into ancient chinese processing. most works only take into account explicit features, such as era @xcite and imagery @xcite , ignoring implicit syntactic features. the main reason for this lies in two aspects: (1) the linguistic gap between ancient and modern chinese makes it difficult for supervised modern chinese syntax parsers to correctly parse ancient chinese expressions; (2) training a supervised ancient chinese syntax parser from scratch can be highly costly due to the lack of annotated data. unsupervised syntax parsing or directly employing modern chinese parsers will inevitably cause noise and performance degradation. a unlabeled example and corresponding human annotation on ancient chinese sentence "可怜人似月中孀(it is pitiful like chang'e in the moon)" are shown in figure 1 . to address this challenge, we propose a novel syntax encoding structure -confidencebased syntax encoding network (csen), which alleviates the negative effect of noise by measuring confidence of arcs in syntax graphs. specifically, confidence is calculated by performing biaffine transformation over the sequence representation and the derived syntactic graph adjacency matrix. with this obtained confidence, our model is capable of distinguishing useful syntactic features from noise. moreover, compared with modern chinese, ancient chinese has more concise expressions and thus more compact structures, each token is highly relative to the preceding and following one. considering such linguistic characteristic, we incorporate another graph feature -left-right branch (lrb), which captures local features to further improve ancient chinese understanding. experiments are conducted on two typical ancient chinese understanding tasks, thematic classification of ancient poetry and ancient-modern chinese translation. results show that our model achieves significant improvements over powerful baselines, and our proposed csen can effectively handle the noise in the derived syntax trees. to our best knowledge, our proposed csen is the first solution that makes the syntax practical in ancient chinese processing. the proposed cseb can serve as a backbone for enriching our understanding of ancient texts, offering a scalable and consistent solution for education, research, and broadening the public's access to these significant cultural treasures. overall, the contributions of this paper can be concluded in four folds: ## model in this section, we describe architecture of the proposed csen. we first present a basic gat encoder, then introduce our csen. the overview of csen is shown in figure 2 . ## experiments we evaluate the effectiveness of csen module using two typical ancient chinese understanding tasks: thematic classification of ancient poetry and ancient-modern chinese translation. we build our model by incorporating the csen module into existing solid baselines. for the classification task, we follow the work of @xcite which has a bert-gat-bilstm backbone architecture. and for the translation task, our model is based on @xcite where dependency graphs are incorporated into neural sequence-to-sequence models with a pointer network. ## exploration in this section, we investigate the impact of different csen components and analyze the nature of different syntax parses. first, we conduct ablation studies on csen. results are reported in second, we experiment with different syntax tree combination setups. third, as illustrated in figure figure 3 , we compare our model and baselines over different input lengths. csen performs better in relative longer sentences, according to the results. this supports the hypothesis that syntax helps guide longer sentence understanding as dependency reduces the distance. because of the incompatibility between modern and ancient chinese, unsupervised derivation is more effective than supervised parsing when compared to other syntax parsers. in most cases, csen yeilding better performance due to its stronger denoising capabilities. ## conclusions in this paper, we investigate the role of syntax in improving ancient chinese understanding. due to lack of syntax annotation, syntax trees are obtained by unsupervised derivation and supervised modern chinese parser. to alleviate the negative effect of noise, we propose a confidence-based syntax encoding network (csen). experimental results on two typical ancient chinese understanding tasks show that our model can effectively distinguish informative syntactic information from noise and achieve better performance. the application of our proposed csen can enhance the accessibility of ancient chinese resources by offering a scalable and consistent solution for mining semantic information of ancient chinese texts.