# The Task of Post-Editing Machine Translation for the Low-Resource Language
## 1 Introduction
### Review of Research Studies
For instance, in the study [20], the model architecture consists of three components: a Seq2 Seq model for transforming the source sentence into the target sentence in forced decoding mode; Seq2 Seq models for gradually generating a sequence of editing operations (action generator), and an RNN (recurrent neural network) for summarizing the sequence of words edited after the editing process, which were obtained as a result of the actions generated so far (interpreter). They demonstrated improvements up to +1 BLEU score and \(-0.7\) TER score for German-English language pairs. In 2019, the application of the transformer model began. In the study [21], a transformer model was employed for the English-German language pair, achieving results that surpassed contemporary technologies due to its simpler architecture, suitable for industrial applications. For post-editing English-Kazakh translations, classical approaches proved to be less effective due to the complexity of the sentences. Therefore, the transformer model was utilized to obtain good results. The use of large language models (LLMs) such as GPT-3, BERT, and others in machine translation provides a number of advantages and opportunities to improve translation quality and system efficiency. Following are some ways these models are used in machine translation:  
Pre-training: Large language models undergo extensive pre-training on huge corpora of text, allowing them to learn large language representations. These representations can then be used in machine translation tasks. Contextual understanding: Models such as BERT are able to understand the context and dependencies between words in a sentence. This allows them to more accurately grasp semantic relationships and correctly interpret phrases in translation. Fine-tuning for machine translation: After preliminary training, models can be further tuned on specific data for the machine translation task. This allows you to take into account the features of the language and the requirements of a specific task. Processing long texts: Larger language models can be better at processing long texts, which can be useful for translating complex and extended sentences. Generating context-aware translations: Transformer-based models are capable of generating context-aware translations, given the previous context. This allows for a more natural conveyance of the meaning and tone of the translated text. Working with multiple languages: Many large language models are trained in multiple languages, allowing them to more efficiently translate text into different languages. This is especially useful in multicultural and multilingual scenarios. Adaptation to specialized fields: Machine translation models based on large language models can be customized and adapted to specialized fields such as medicine, law, or engineering. Supervised and unsupervised learning: Models can be applied to both supervised (using labeled data) and unsupervised (using unsupervised learning techniques for translation) machine translation tasks. The use of large language models in machine translation continues to evolve, and these models provide new opportunities to improve the quality of automatic translation and mitigate some of the problems faced by older models [22]. Large language models typically require extensive data for pre-training. Unfortunately, at the moment, there are no large and diverse text corpora for the Kazakh language that can contribute to the successful application of the model. The Kazakh language has its own unique linguistic features, such as agglutination and the use of the Cyrillic and Latin alphabets. The model must be configured to take these features into account. Training and using large language models require significant computational resources. Having sufficient computing power can be critical. The application of large language models to the Kazakh language requires a careful approach and consideration of various factors.