Active Learning Approaches to Enhancing Neural Machine Translation (2020)

Abstract: Active learning is an efficient approach for mitigating data dependency when training neural machine translation (NMT) models. In this paper, we explore new training frameworks by incorporating active learning into various techniques such as transfer learning and iterative back-translation (IBT) under a limited human translation budget. We design a word frequency based acquisition function and combine it with a strong uncertainty based method. The combined method steadily outperforms all other acquisition functions in various scenarios. As far as we know, we are the first to do a large-scale study on actively training Transformer for NMT. Specifically, with a human translation budget of only 20% of the original parallel corpus, we manage to surpass Transformer trained on the entire parallel corpus in three language pairs.

## introduction many impressive progresses have been made in neural machine translation (nmt) in the past few years @xcite @xcite . however, the general training procedure requires tremendous amounts of high-quality parallel corpus to achieve a deep model's full potential. the scarcity of the training corpus is a common problem for many language pairs, which might lead to the nmt model's poor performance. however, constructing a parallel corpus is a slow and laborious process. professional human translators and well-trained proofreaders are needed. although several dual learning @xcite and unsupervised learning @xcite @xcite approaches have been successfully used, they are often inferior to the supervised models. in such cases, active learning might be a good choice. the goal of active learning in nmt is to train a well-performing model under a limited human translation budget. we achieve this goal by using some particularly designed acquisition functions to select informative sentences to construct a training corpus. acquisition functions can be categorized into two types: model related and model agnostic. for the former, the methods we use are all based on the idea of uncertainty. for the latter, we devise a word frequency based method which takes linguistic features into consideration. both types of acquisition functions have been proven to be beneficial in active nmt training, especially when they are appropriately combined. data augmentation techniques that consume no human translation budget are worth exploring in active nmt training. if the parallel corpus of a related language pair is available, transfer learning @xcite ) might be a good choice. otherwise, we propose a new training framework that integrates active learning with iterative back-translation (ibt) @xcite . we achieve success in both the settings, especially when active learning bonds with ibt. the main contributions of this work are listed as follows: 1) to the best of our knowledge, we are the first to give a comprehensive study of active learning in nmt under various settings. 2) we propose a word frequency based acquisition function which is model agnostic and effective. this acquisition function can further enhance existing uncertainty based methods, achieving even better results in all settings. 3) we design a new training framework for active iterative back-translation as well as a simple data augmentation technique. with a human translation budget of only 20% of the original parallel corpus, we can achieve better bleu scores than the fully supervised transformer does @xcite . 2 related work active learning as for natural language processing, active learning is well studied in text classification @xcite and named entity recognition @xcite @xcite . @xcite applied attention based acquisition functions for nmt. @xcite introduced reinforcement learning to actively train an nmt model. data selection in nmt although active learning has not been thoroughly studied in nmt, the related data selection problem attracts some attention. van der wees et al. (2017); wang et al. (2018a) deliberately designed weighted sampling methods, which accelerates training and improves performance. wang et al. (2018b); pham et al. (2018) focused on noisy data, coming up with algorithms to filter harmful sentence pairs. wang et al. (2019) simultaneously dealt with domain data selection and clean data selection. fadaee and monz (2018); poncelas et al. (2019); dou et al. (2020) considered domain data selection in back-translation. wang and neubig (2019) proposed a method to select relevant sentences from other languages to bring performance gains in low resource nmt. furthermore, ruiter et al. (2019) tried to extract possible parallel data from bilingual wikipedia. interactive nmt interactive nmt exploits user feedback to help improve translation systems. realworld @xcite or simulated user feedback includes highlighting accurate translation chunks @xcite or correct errors made by machine @xcite . @xcite took the cost of different types of supervision (feedback) into account, which resembles the idea of active learning. ## methodology we give a detailed description of active neural machine translation (nmt) in this section. basic settings and some terminologies are introduced in section 3.1. in section 3.2 and section 3.3, various acquisition functions are presented and explained. section 3.4 deals with combining active learning with transfer learning and iterative backtranslation. figure 1 is an illustration of different training frameworks in nmt. ## conclusion various acquisition functions are conducted on active nmt, active nmt with transfer learning and active iterative back-translation (ibt). our experiment results strongly prove that active learning is beneficial to nmt. our combined method (tedelfy) achieves the best final bleu score in every experiment we do. also, the proposed active ibt++ framework efficiently exploits the selected parallel corpus to further enhance the model accuracy. these techniques may also be useful for unsupervised nmt. active pre-training is worth trying and active ibt has already proven its capability. we leave it for future work to study more acquisition functions in more nmt scenarios.