A nchi L m: An Effective Classical-to- M odern C hinese Translation Model Leveraging bpe-drop and S iku R o BERT a (2023)

Abstract: In this paper, we present our submitted model for translating ancient to modern texts, which ranked sixth in the closed track of ancient Chinese in the 2nd International Review of Automatic Analysis of Ancient Chinese (EvaHan). Specifically, we employed two strategies to improve the translation from ancient to modern texts. First, we used bpe-drop to enhance the parallel corpus. Second, we use SikuRoBERTa to simultaneously initialize the translation model’s codec and reconstruct the bpe word list. In our experiments, we compare the baseline model, rdrop, pre-trained model, and parameter initialization methods. The experimental results show that the parameter initialization method in this paper significantly outperforms the baseline model in terms of performance, and its BLEU score reaches 21.75.

## introduction ancient chinese historical texts are not only key parts of chinese civilization but treasures of global culture. understanding these works is challenging for modern people, hence, translation is vital to bridge this gap. traditional manual translation of these texts is time-consuming and challenging. with the advancement of computer science, machine translation (mt) provides a new solution to this problem. among various mt methods, neural machine translation (nmt) is representative. however, applying nmt to ancient text translation faces significant challenges. the main issue is the relatively small corpus for ancient text translation, making it hard for models to learn and capture the complex grammar and rich semantics from limited data. further, as shown in figure 1 , the conversion between modern and ancient chinese is highly complicated. consequently, the results of ancient text translation often fall short of expectations. in recent years, the "pre-training + fine-tuning" paradigm has become a powerful technique in the field of neural machine translation (nmt). researchers have attempted to leverage this paradigm to enhance translation models, such as xlm @xcite , masssong et al. (2019) , mbartliu et al. (2020) , and mrasp2pan et al. (2021) , which have performed excellently in machine translation tasks. meanwhile, there have also been some excellent solutions in the field of ancient text translation. for instance, a transformer model was trained to translate ancient chinese into modern chinese , anchibertliu et al. (2019) pretrained bertdevlin et al. (2018) on ancient texts and then initialized the encoder of the nmt model, or the bert-fusedzhu et al. ( 2020 ) method fused the bert output into every layer of the encoder-decoder using the attention mechanism... existing research provides a series of valuable insights. in this paper, we aim to further advance research in this field by proposing anchilm to deal with specific problems and challenges more effectively. it has the following main features: ## methods in this section, we will detail the methodology of our approach, which consists of three main components: the encoder, the decoder, and the word embedding. each of these components plays a crucial role in our translation model. ## conclusion this paper describes our submission to the 2nd international evaluation of automatic analysis of ancient chinese (evahan) closed track for ancient chinese translation. our ancient textmodern text translation model includes two parts: bpe-drop data enhancement and parameter initialization. future work is to combine parameter initialization and pre-training tasks simultaneously.