# The Task of Post-Editing Machine Translation for the Low-Resource Language
## 3 Results
### Dataset
However, the basic machine translation metrics can be enhanced by increasing the volume of the corpus and the variety of topics it covers, which is a crucial task for low-resource languages. The metrics show the usefulness and the effectiveness of the developed approach. The developed system was also tested by users. In addition, to assess the usability of the system and the system interface, a System Usability Scale (SUS) survey was conducted. This survey consists of the classic 10 questions. Each question is scored using a Likert scale from 1 to 5 as follows:  
1--Completely disagree  
2--I do not agree  
3--Neutral  
4--Agree  
5--Completely agree  
After the user has answered all questions, the total score is recalculated, taking into account the inversion of scores for some questions. The resulting total score is calculated and can range from 0 to 100. A higher score generally indicates a higher perception of system usability [38; 39; 40; 41; 42; 43]. Based on the survey results, the following results were obtained:  
To evaluate the usability of the developed system, 15 users were recruited for each language. A group of experts with knowledge in the relevant field and language were invited. The group of experts included five employees of the Faculty of Philology of KazNU, [PERSON], five specialists who use machine translation systems in their daily work, and five students. The user was provided with clear instructions regarding the operation of the evaluation criteria system. The experts were tasked not only with assessing the convenience of the system but also with paying attention to the quality of grammatical correctness, conveying meaning, and the structure of sentences and text. To avoid bias, translations were provided to users without indicating their source (professional translation). Figures 8 and 9 show the average of user responses to each question. Figure 10 shows the average score on the SUS scale, where the blue line represents the Kazakh language and the red line represents the Uzbek language. Based on the results of the SUS survey, a value of 71.16 was obtained for the Kazakh language, and a value of 67.6 was obtained for the Uzbek \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Corpus** & **Architecture** & **BLEU** & **WER** & **TER** \\ \hline English–Kazakh translation & BRNN & 0.37 & 0.49 & 0.57 \\ Russian–Kazakh translation & BRNN & 0.25 & 0.56 & 0.25 \\ Post-editing of Kazakh text & Transformer & 0.49 & 0.45 & 0.47 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Quality metrics of the post-editing system for the Kazakh language. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Corpus** & **Architecture** & **BLEU** & **WER** & **TER** \\ \hline English–Uzbek translation & BRNN & 0.26 & 0.54 & 0.59 \\ Post-editing of Uzbek text & Transformer & 0.35 & 0.47 & 0.52 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Quality metrics of the post-editing system for the Uzbek language. During the research, an architecture and a post-editing system for the Kazakh and Uzbek languages were developed. This work serves as a valuable resource for researchers and practitioners in the field of machine translation, especially for those working with low-resource languages like Kazakh. Practical experiments were conducted on various texts: news publications, legislative documents, IT sphere, etc. As shown in the practical results, the quality assessment according to the BLEU metrics improved by 0.12 for the English-Kazakh translation, by 0.24 for the Russian-Kazakh translation, and by 0.09 for the English-Uzbek translation. The developed system was also tested by users. To assess the usability of the system and the system interface, a System Usability Scale (SUS) survey was conducted. The advantage of the developed approach is that, while having a small number of resources and at the same time due to the linguistic properties of the language, it is possible to improve the quality of machine translation for various unrelated groups of languages based on modern machine learning methods.