# Comparative Analysis of Current Approaches to Quality Estimation for Neural Machine Translation
## 7 Conclusions  
Most recent studies of QE apply data augmentation with finetuning based on state-of-the-art large scale mPLM, such as XLM-R, to obtain a high performance for a WMT shared task. In this study, unlike typical QE research that focused on the competition involving a shared task, we conducted a pure performance comparison between various mPLMs. As a result of the experiments, we confirmed that the XLM-TLM model performed best on both sub-tasks, and that the induced learning of alignment between languages during pre-training had a positive impact. Additionally, we conducted experiments using mBART for the first time, and its additional noising schemes had a positive effect on QE research. Therefore, we confirmed the feasibility of using the mBART model in further QE research. We demonstrated that the order of the input sequence between the source sentence and its MT output can affect the model performance. In the future, we will further investigate data-centric issues that are not model-based [43; 44]. By filtering data based on the HTER score, we will explore which score ranges contribute significantly to the performance of a model and provide a basis for future data-centric research on QE. In addition, we plan to conduct an in-depth study on low resource language QE. We plan to study a methodology that can automatically generate data based on a semi-supervised learning method.  
Conceptualization, C.P.; methodology/software, S.E.; validation, S.E. and H.M.; formal analysis, S.E. and C.P.; investigation, S.E. and H.M.; review and editing, H.M. and J.S.; supervision/project administration, C.P.; funding acquisition, H.L. All authors have read and agreed to the published version of the manuscript.  
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-0-01405) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation) and the MSIT, Korea, under the ICT Creative Consilience program (IITP-2021-2020-0-01819) supervised by the IITP. Additionally, this work was supported by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques).  
Not applicable.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Pearson**} & \multicolumn{4}{c}{**MAE**} & \multicolumn{4}{c}{**RMSE**} \\ \cline{2-10}  & **Max** & **Min** & **Average** & **Avg Diff** & **Min** & **Max** & **Average** & **Min** & **Max** & **Average** \\ \hline XLM-R-base & 0.464 & 0.453 & 0.460 & +0.012 & 0.144 & 0.153 & 0.148 & 0.184 & 0.199 & 0.191 \\ \hline XLM-R-large & 0.523 & 0.501 & 0.509 & +0.011 & 0.140 & 0.144 & 0.142 & 0.178 & 0.188 & 0.183 \\ \hline mBERT & 0.449 & 0.434 & 0.441 & +0.024 & 0.147 & 0.179 & 0.162 & 0.185 & 0.229 & 0.207 \\ \hline mBART & 0.478 & 0.463 & 0.469 & +0.006 & 0.141 & 0.151 & 0.145 & 0.179 & 0.196 & 0.187 \\ \hline XLM-CLM & 0.297 & 0.283 & 0.287 & \(-\)0.011 & 0.159 & 0.162 & 0.160 & 0.197 & 0.205 & 0.199 \\ \hline XLM-MLM & 0.364 & 0.333 & 0.351 & +0.017 & 0.153 & 0.159 & 0.156 & 0.193 & 0.200 & 0.196 \\ \hline XLM-MLM-17 & 0.420 & 0.405 & 0.411 & \(-\)0.004 & 0.154 & 0.218 & 0.172 & 0.190 & 0.273 & 0.217 \\ \hline XLM-MLM-100 & 0.442 & 0.405 & 0.417 & +0.008 & 0.151 & 0.183 & 0.161 & 0.187 & 0.220 & 0.196 \\ \hline XLM-TLM & 0.552 & 0.526 & 0.538 & +0.028 & 0.156 & 0.168 & 0.163 & 0.204 & 0.218 & 0.212 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results of mPLM finetuning with inverted inputs for the test set of WMT20 sub-task 2.