# Automated Scoring of Translations with BERT Models: Chinese and English Language Case Study
## 2 Related Work
### Similarity-Based Approaches  
Similarity-based approaches transform the task of assessment into similarity calculations. The assumption behind this method is posed by [PERSON] et al. [21], \"The closer a machine translation is to a professional human translation, the better it is\". To be specific,for a machine translation, the degree of similarity between it and one or more reference human translations determines its translation quality. There are a number of measures to calculate the degree of similarity, all of which are expected to achieve the same goal, which is to generate a numerical metric as the result. In terms of assessment measures, there are three main measures: Levenshtein distance-based, n-gram co-occurrence-based, and vector-based. The proposal of distance measure approach is based on Levenshtein distance calculations [22], a string comparison metric that counts the number of edit operations (insertion, deletion, replacement and swap) required to transform one string into another [23]. As a sensitive measure with which distances between strings (in this case transcriptions of word pronunciations) can be calculated [24], it is an efficient and reliable method for computing string alignments [25]. The number of operations required is inversely proportional to the quality of the translation. In 1992, [PERSON] et al. [26] firstly employed it to assess machine translation quality with one reference human translation so that the assessment process could be done quickly and automatically. Subsequently, [PERSON] et al. [27] and [PERSON] et al. [28] tried to use multiple reference human translations in a dataset. Continuously upgraded by introducing advanced models like bag-of-words [29], the accuracy of scoring has been improved. Various measures for the automated assessment of machine translation quality, such as WER [29], TER [30], CDer [31], etc., are based on Levenshtein distance. The n-gram co-occurrence statistics method is an innovative assessment method for machine translation. The assessment system, developed on the basis of n-grams co-occurrence statistics, is still widely used in various assessment tasks in the field of NLP. The core of this method is that \"a good translation will have a distribution of n-grams similar to other good translations\" [32]. To be specific, the segments in the tested translation text are compared with the corresponding aligned segments in the reference translation texts based on the number of their n-grams. The number of all the compared segments finally enter the final score calculation with various weights and factors. The main representative systems are BLEU [21], NIST [33] and METEOR [34]. For the similarity-based approaches mentioned above, the assessment reliability largely depends on the set of reference human translations. If the computer-generated translation is similar to one or more reference translations, it will be considered a good translation. While translations with a high degree of similarity are generally of high quality and have relatively reliable scoring accuracy, translations with low similarity are not necessarily bad translations, as reference translations cannot encompass all acceptable translations. This means that the matching rate of synonyms may be low. A limited number of reference translations will likely reduce the reliability of the assessment. Moreover, the process requires a significant amount of manual intervention, as invited experts must contribute to building the set of reference human translations. Semantic similarity between two long texts can also be achieved by vectorizing them and then comparing the values of the two vectors generated by large language models. The vectorization of words can technically be achieved by word embedding programs such as Word2 Vec [35; 36]. The linguistic theoretical basis comes from Distributional Hypothesis [37], that is, words that occur in the same contexts tend to have similar meanings. Based on Word2 Vec, Doc2 Vec upgraded semantic similarity calculation to a document level [38].