# Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data
## 2 Related Work  
Researchers have recently come up with a lot of ideas for how to improve NMT for languages with few resources. One way to improve the NMT system, especially for languages with few resources, is to use monolingual data and parallel datasets. In this section, we explore related studies performed for different languages using monolingual datasets as an additional source to improve NMT systems for low-resource languages. [PERSON] et al. [20] proposed utilizing monolingual data via pretrained word embeddings in transformer model-based neural machine translation to tackle the parallel dataset prob lem in bidirectional Tamil-Telugu NMT. The authors used GloVe [21] to pre-train word embedding on the monolingual corpora and used them in the transformer model during the training process. Their model achieved a BLEU score of 4.05 for both Tamil-Telugu and Telugu-Tamil translations, respectively. [PERSON] et al. [22] proposed a new method to generate large synthetic parallel data by leveraging very small monolingual data in a specific domain. They fine-tuned a pre-trained GPT-2 model on small in-domain monolingual data and used the resulting model to generate a large amount of synthetic in-domain monolingual data. Then, they performed a back translation to generate synthetic in-domain parallel data. Their results on the English-German, English-French, and English-Japan pairs and five domains showed improvements in BLEU for all configurations when using synthetic data to train NMT. [PERSON] et al. [23] proposed a multilingual training approach, which can be improved by leveraging monolingual data to create synthetic bilingual corpora using the back translation method to improve low-resource machine translation. Their multilingual learning strategy and synthetic corpora increased the translation quality for language pairs from the Estonian and Finnish geographical regions. [PERSON] et al. [24] utilized monolingual data to improve the performance of the NMT model for the English-German and Turkish-English language pairs. They created synthetic parallel data by translating target language monolingual data into the source language. Then, their initial target-source MT system was trained on the available parallel data. After training, the system translated the monolingual dataset from the source language into the target language to obtain the back-translated data. Next, [PERSON] et al. [24] trained the final source-target NMT system by mixing the back-translated data with the original parallel data. Finally, the researchers achieved significant gains on the WMT-15 English-German task with a +2.8-3.7 BLEU improvement and on the low-resource IWSLT-14 Turkish-English task with a +2.1-3.4 BLEU improvement. [PERSON] et al. [25] proposed self-training sampling with monolingual data uncertainty for NMT to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data. Using the bilingual lexicon generated from the parallel data, they calculated the uncertainty of monolingual sentences. They suggested that emphasizing the learning of uncertain monolingual sentences improves the translation quality of high-uncertainty sentences and benefits the prediction of low-frequency words on the target side. The results of their experiments on large WMT English-German and English-Chinese datasets show that the proposed method improved the performance of NMT. [PERSON] et al. [26] utilized subword segmentation, back translation, and the copied corpus methods to improve the NMT performance of bidirectional Wolf-French translation. When back translation and the copied corpus were used together, the quality of the translation from Wolf to French showed improvement in both directions. [PERSON] [27] proposed the use of the Google Translate application as a reverse MT method to improve the quality of monolingual texts for back translation in the English-Vietnamese language pair.