# Large Language Model-Powered Automated Assessment: A Systematic Review
## 3 Results
### Details of LLMPAA Research
#### 3.2.3 LLM's and Evaluation Metrics

Table 5: Overview of LLMs and evaluation metrics. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**References** & **LLMs** & **Evaluation Metrics** & **Results** \\ \hline
[25] & GPT-3.5 & ICC: 0.81 & GPT-3.5 achieved high consistency with human. \\ \hline
[26] & GPT-3.5 & ICC: 0.56 & GPT-3.5 achieved medium consistency in language criteria. \\ \hline
[27] & BERT, RoBERTa, XLNet, SBERT & ACC: 0.76 (3-class), 0.82 (2-class) & BERT showed the most consistent and highest performance as the recommended model across all datasets; other models were competitive in specific tasks but generally performed lower. \\ \hline
[28] & GPT-4 & r: 0.84 & GPT-4 showed high correlation. \\ \hline
[29] & GPT-4 & P: 0.788 & GPT-4 showed high precision in the SciEntsBank dataset. \\ \hline
[30] & CNN-LSTM, IndoBERT & QWK: 0.465 & IndoBERT achieved a 14.47\% increase in agreement compared to CNN-LSTM. \\ \hline
[31] & BERT & ACC: 0.958 & BERT performed with high accuracy. \\ \hline
[32] & CNN-LSTM-ATT, LSTM-MoT, SkipFlow LSTM, & & \\  & TSLF, Sentence-BERT, & QWK: 0.852 & TSLF model achieved the highest agreement. \\  & CNN-LSTM-ATT, Tran-BERT-MS-ML-R & & \\ \hline
[33] & BERT, GPT-4, OCLL & QWK: 0.819 & GPT-4 achieved the highest agreement. \\ \hline
[34] & GPT-4 & ACC: 0.75 & GPT-4 performed with high accuracy with humans. \\ \hline
[35] & AraBERT & r: 0.88 & AraBERT showed high correlation with humans. \\ \hline
[36] & BERT-1-EN, BERT-1-ES, BERT-2-EN, & & \\  & BERT-1-MU, BERT-2-MU, & r: 0.83 & BERT-1-ES showed the best results. \\  & Skip-Thought, BERT-2-ES & & \\ \hline
[37] & BERT, Word2 vec, WordNet & r: 0.841 & BERT showed the highest result across all datasets. \\ \hline
[38] & GPT-3 text-davinci-003 & CK: 0.682 & GPT-3 achieved agreement with humans. \\ \hline
[39] & BERT, GPT-3 (ada, babbage, curie, davinci), & r: 0.813 & Davinci showed the highest correlation. \\ \hline
[40] & GPT-3.5, GPT-4, Claude 2, PaLM 2 & ICC: 0.927, r: 0.731 & GPT-4 showed the highest consistency. \\ \hline
[41] & GPT-4 & ICC: 0.858 & GPT-4 showed excellent consistency with humans. \\ \hline
[42] & BERT, sentence embedding-LSTM & QWK: 0.766 & The best agreement was achieved with the BERT model. \\ \hline
[43] & Bi-LSTM, Sentence-BERT & QWK: 0.76 & The best agreement was achieved with the Sentence-BERT model. \\ \hline
[44] & ALBERT, BERT & r: 0.950 & ALBERT showed the highest performance. \\ \hline
[45] & GPT-3.5 & - & It showed high agreement with humans. \\ \hline
[46] & Aggregated-BERT + handcrafted + SVR, & & The highest agreement was achieved by Aggregated-BERT + Handcrafted + Skipflow, TSLF-all & \\ \hline
[47] & ChatGLM2-6B, Baichuan-13B, InterLM-7B & QWK: 0.571 & ChatGLM2-6B achieved the highest agreement. \\ \hline \hline \end{tabular}
\end{table} 
Table 5: _Cont._It has been determined that the performance of LLMs is measured using the following metrics: ACC (n = 7), F1-score (n = 2), precision--P (n = 2), and recall--R (n = 1). These metrics are employed to assess the models' ability to predict actual assessment outcomes, which are represented on a scale from 0 to 1. A performance value closer to 1 indicates higher model effectiveness. According to the analysis, one study [31] reported a comparatively high accuracy, with a BERT-based model achieving an ACC of 95.8% in evaluating the written responses of foreign students learning Korean. Furthermore, some studies utilized multiple metrics in addition to ACC. For example, in a study aimed at developing an ASAG system to assess student responses in distance education objectively and quickly, it was found that the BERT model provided a good level of accuracy (ACC = 0.80) and correlation (r = 0.747) [27]. On the other hand, in a study [16] which aimed to create and assess the SPRAG dataset for ASAG related to programming languages, the developed model was reported to achieve high accuracy (ACC: 0.8256; r = 0.69).