# Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data
###### Abstract  
Despite the many proposals to solve the neural machine translation (NMT) problem of low-resource languages, it continues to be difficult. The issue becomes even more complicated when few resources cover only a single domain. In this paper, we discuss the applicability of a source-side monolingual dataset of low-resource languages to improve the NMT system for such languages. In our experiments, we used Wolaytta-English translation as a low-resource language. We discuss the use of self-learning and fine-tuning approaches to improve the NMT system for Wolaytta-English translation using both authentic and synthetic datasets. The self-learning approach showed +2.7 and +2.4 BLEU score improvements for Wolaytta-English and English-Wolaytta translations, respectively, over the best-performing baseline model. Further fine-tuning the best-performing self-learning model showed +1.2 and +0.6 BLEU score improvements for Wolaytta-English and English-Wolaytta translations, respectively. We reflect on our contributions and plan for the future of this difficult field of study. Wolaytta-English NMT; English-Wolaytta NMT; low-resource NMT; self-learning; neural machine translation; monolingual data for low-resource languages; low-resource NMT +
Footnote †: journal:  
effective NMT systems. In order to address this problem, researchers have proposed several solutions, including multilingual NMT [11], transfer learning [12], exploiting related languages [13], multimodal NMT [14], data augmentation [15], filtered pseudo-parallel datasets [16], and meta-learning [17]. However, most of the suggested approaches used high-resource languages for their experiments, and to the best of our knowledge, no research has been performed using source-side monolingual data of low-resource languages to improve NMT. Using Wolaytta-English translation as a case study, we propose an efficient method for using such data to improve the NMT performance in low-resource languages. We collected 40 k Wolaytta monolingual data from several sources, such as Wolaytta Fana Radio, Wolaytta Wogeta Radio, and Wolaytta textbooks and used them along with a Wolaytta-English parallel dataset in two experiments: (i) training a model on the available Wolaytta-English parallel dataset (our baseline model) and (ii) training the selected model from the baseline on the combination of the authentic dataset and synthetic dataset with a self-learning and fine-tuning approach following the procedure in [18]. Our objective is to answer the following research questions:  
* Will relying solely on source-side monolingual data improve the NMT performance for a low-resource language? * Will monolingual data from a low-resource language improve the performance of NMT when English is used as the source language? To respond to the above questions and to explore the NMT performance in a low-resource language, we focus on NMT for the Wolaytta-English language pair because this language pair has parallel data only from the religion domain, and this fact also serves as an illustration of how difficult it is to compile a parallel dataset for a low-resource language. Wolaytta is a language spoken in the southern part of Ethiopia, with few or no digital resources [19]; a more detailed description can be found in Section 3. In this paper, we investigate NMT training from a dataset perspective for low-resourced languages. We specifically examine the effects of using low-resource language source-side monolingual data on improving the NMT and the most effective ways to use such data. As a result, the contributions of our work are the following:  
* We thoroughly investigated the usability of a source-side monolingual dataset for low-resource languages to improve NMT. * We found a good way to combine a synthetic parallel dataset with an authentic parallel dataset when training an NMT model for low-resource languages. * We developed a good training method for fine-tuning NMT models that were trained on a mixed set of synthetic and authentic datasets.