# Automatic Essay Evaluation Technologies in Chinese Writing--A Systematic Literature Review
## 3 Results
### Scoring Model (RQ3)
#### 3.3.4 Comparative Analysis of Scoring Models  
The performance of scoring models is closely related to the scoring criteria (relevant to the scoring context), the size of the corpora, the selected features, and the methods used to construct scoring models. Table 3 compares the construction details and performance  
Figure 6: Usage of evaluation methods. of scoring models selected in the literature from the above mentioned perspectives. We classify the scoring models in terms of the corpus sources and scoring models used in the literature, as using the same corpus source implies the same scoring criteria, which enhances the comparability of studies when identical corpora are employed. Moreover, considering that the linguistic level of feature representation has a more significant impact on the choice of scoring methods than specific features, the table categorizes the features as shallow and deep semantic attributes and embedding features. To better compare the performance of scoring models and draw relatively accurate conclusions, we selected the best-performing performance of scoring models from the literature as our evaluation results. Furthermore, due to the substantial differences in the scoring ranges across studies, it is not meaningful to compare the values of RMSE and AE. Thus, these evaluation metrics are not presented in the table. It should be noted that there are some limitations in the comparative analysis presented in Table 3. Firstly, the differences in evaluation results can only serve as reference data due to the differences in the employed corpora and evaluation methods. Secondly, some of the studies selected in this article did not adopt the evaluation methods mentioned earlier (e.g., some studies [42; 56] verified the effectiveness of features in scoring by establishing the correspondence between features and essay grades) or did not provide the evaluation results of their models. Therefore, such research findings were not included in the comparative analysis. Overall, the performance of scoring models is influenced by factors such as the corpus, features, and scoring methods. The relationship between the scoring model's performance and the corpus's size is complex. For example, studies on HSK corpora with around 100 samples [18] and over 10,000 samples [35] show similar PCC values. However, it should be noted that some studies with low performance [21; 47] suffer from inadequate samples. DNN models generally need large-scale training corpora. We can see from Table 3 that the relationship between feature selection and the scoring model is not significant except for the embedding features unsuitable for MLR models. Scoring models that utilize multiple hierarchical features tend to achieve a better scoring performance. As corpora often indicate the scoring context of essays, this study analyzed the performance of scoring models under various corpus backgrounds. Since publicly available corpora are used, studies conducted on HSK corpora have relatively high comparability due to the consistency of the content. We can see from Table 3 that both traditional regression/classification models and DNN models can achieve a human-machine scoring correlation of approximately 0.73 in terms of the PCC values of the models. MHK corpora usually come from different exam times or regions, thus having different content but the same scoring criteria. Studies based on MHK corpora have predominantly used traditional scoring models, focusing more on identifying which features and indicators significantly influence essay scoring in MHK exams. MLR methods have demonstrated a higher human-machine scoring correlation in this corpus. In the context of native speakers' scoring, various scoring models mostly achieve accuracies ranging from 84% to 95%, while there are significant variations in human-machine correlation. Notably, scoring models based on deep learning generally yield PCC and QWK values ranging from 0.60 to 0.66 (except for the model in the study of [PERSON], [PERSON] et al.