BIT - ACT : An A ncient C hinese Translation System Using Data Augmentation (2023)

Abstract: This paper describes a translation model for ancient Chinese to modern Chinese and English for the Evahan 2023 competition, a subtask of the Ancient Language Translation 2023 challenge. During the training of our model, we applied various data augmentation techniques and used SiKu-RoBERTa as part of our model architecture. The results indicate that back translation improves the model’s performance, but double back translation introduces noise and harms the model’s performance. Fine-tuning on the original dataset can be helpful in solving the issue.

## introduction ancient chinese translation is a machine translation task, aiming to translate ancient chinese into modern chinese or english, which is of great significance for the research and understanding of ancient chinese. in the area of ancient chinese translation, the presence of unique features in ancient chinese poses challenges. ancient chinese has its own distinctive features, including a large number of rare characters, characters with different meanings in ancient and modern times. in the aspect of syntax, ancient chinese are very different from modern chinese, including lots of inverted or elliptical sentences, which increases the difficulty in translation. @xcite additionally, due to the limited amount of data, it's necessary to apply data augmentation during training. in our system, we use a transformer @xcite architecture with adjusted parameters, to address the limited data, we applied various data augmentation techniques to generate additional data, achieved improved performance. considering the unique grammatical structures of ancient chinese, we tried a @xcite model pre-trained on ancient chinese, we had analyzed different results from these approaches. ## method in this section, we introduce our data processing and augmentation method. we also describe the architecture of our systems. ## experiments in this section, we delve into the training details and steps, including the utilization of hyperparameters. we also compare and analyze the results obtained from different models. for the model configuration, we used the following set of model configuration as the default. we will point out if there are any modifications. we usd adam @xcite as our optimizer, use the "inverse sqrt lr" scheduler with 4000 warm-up steps. we conducted the training on two titan x gpus, for each model, we trained for a total of 300,000 updates. ## conclusion for the ancient chinese to modern chinese task, we found that back translation can help improve the model's performance with limited data. however, it's important to notice that back translation may introduce noise, and fine-tuning the model after back translation could potentially enhance its performance. when using roberta as embedding, the results were not as expected. one possible reason is that we did not train roberta model on our data separately, leading to mismatch in vocabulary. additionally, the large scale of roberta might cause overfitting, we believe that more data will help address this issue. for the ancient chinese to english task, due to the limited dataset and significant linguistic differences between ancient chinese and endlish, we did not achieve satisfactory results.