# Automated Scoring of Translations with BERT Models: Chinese and English Language Case Study
## 1 Introduction  
In recent years, large language models, powered by advanced deep learning techniques, have revolutionized the field of natural language processing (NLP) and found extensive applications across various domains. These models, often pre-trained on massive datasets, have demonstrated remarkable capabilities in a wide range of NLP tasks, and they can be fine-tuned to improve performance on various downstream tasks, such as sentence similarity [1; 2; 3], translation [4; 5; 6], text classification [7; 8; 9; 10], question answering [11; 12; 13], summarization [14; 15; 16; 17], etc. Additionally, the emergence of Big Data provides opportunities to leverage large amounts of data for building high-quality models and improving the performance of NLP systems.  
This study explores the possibility of developing a system for the automated scoring of translations for large-scale translation contests based on the large language model, the Bidirectional Encoder Representations from Transformers (BERT). The scoring of translations in large-scale translation contests suffers from challenges related to the huge growth in the number of participants. In the 31 st Han Suyin International Translation Contest, the total number of valid Chinese-to-English (C-E) and English-to-Chinese (E-C) translations exceeded 10,000, including 3822 C-E submissions and 6825 E-C submissions. As the assessment of translations is a time-consuming and labor-intensive process, it significantly increases the costs in terms of time, manpower, and money, all of which greatly increase the burden on the organizing committee.  
The automated scoring of translations for the large-scale translation contest, Han Suyin International Translation Contest, focuses on selecting high-quality translations for awards, rather than assigning scores to translations like those assigned in an examination. Therefore, a good approach is expected to satisfactorily predict the ranking of translations on an independent basis according to translation quality and sift out high-qualitytranslations potentially shortlisted for prizes. The selective nature of translation contests makes automated scoring possible. This paper explores the possibility of applying BERT to the automated scoring of translations for the large-scale translation contest with a view to addressing the problems mentioned above. Spearman's rank correlation (\(r\)) between the expected ranking (actual ranking) and the predicted ranking (automated assessment conducted by the system), mean absolute error rate of ranking and maximum absolute error rate of ranking were used for model evaluation. Accuracy, recall and F-measure were used for the reliability analysis.  
This paper has two main contributions. First, it contributes to the field by providing empirical evidence of the viability of automated translation scoring systems based on BERT models focusing specifically on the evaluation of translations in the context of a well-established translation contest, thereby demonstrating real-world applicability and relevance. Second, it establishes effective methods for processing lengthy texts to conform with BERT's 512-token input constraint.  
The remainder of this article is organized as follows. Section 2 shows an overview of previous research related to the topic. Section 3 provides a detailed exposition of the research methodology applied in this study, encompassing the corpus, BERT models, and research procedures. Sections 4 and 5 discuss the research findings in depth and engage in subsequent discussions on BERT in the automated scoring of C-E and E-C translations. The concluding remarks are summarized in Section 6.