Dynamic Curriculum Learning for Low-Resource Neural Machine Translation (2020)

Abstract: Large amounts of data has made neural machine translation (NMT) a big success in recent years. But it is still a challenge if we train these models on small-scale corpora. In this case, the way of using data appears to be more important. Here, we investigate the effective use of training data for low-resource NMT. In particular, we propose a dynamic curriculum learning (DCL) method to reorder training samples in training. Unlike previous work, we do not use a static scoring function for reordering. Instead, the order of training samples is dynamically determined in two ways - loss decline and model competence. This eases training by highlighting easy samples that the current model has enough competence to learn. We test our DCL method in a Transformer-based system. Experimental results show that DCL outperforms several strong baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De.

## introduction recently, neural machine translation (nmt) has demonstrated impressive performance improvements and became the de-facto standard @xcite @xcite . however, like other neural methods, nmt is data-hungry. this makes it challenging when we train such a model in low-resource scenarios @xcite . researchers have developed promising approaches to low-resource nmt. among these are data augmentation @xcite @xcite , transfer learning @xcite , and pre-trained models @xcite . but these approaches rely on external data other than bi-text. to date, it is rare to see work on the effective use of bilingual data for low-resource nmt. in general, the way of feeding samples plays an important role in training neural models. a good instance is that it is popular to shuffle the input data for robust training in state-of-the-art systems. more systematic studies on this issue can be found in recent papers @xcite @xcite . for example, @xcite have pointed out that deep neural networks tend to prioritize learning "easy" samples first. this agrees with the idea of curriculum learning @xcite in that an easy-to-hard learning strategy can yield better convergence for training. in nmt, curriculum learning is not new. several research groups have applied it to large-scale translation tasks although few of them discuss the issue in a low-resource setup @xcite @xcite . the first question here is how to define the "difficulty" of a training sample. previous work resorts to functions that produce a difficulty score for each training sample. this score is then used to reorder samples before training. but the methods of this type enforce a static scoring strategy and somehow disagrees with the fact that the sample difficulty might be changing when the model is updated during training. another assumption behind curriculum learning is that the difficulty of a sample should fit the competence of the model we are training. researchers have implicitly modeled this issue by hand-crafted curriculum schedules @xcite or simple functions @xcite , whereas there has no in-depth discussion on it yet. in this paper, we continue the line of research on curriculum learning in low-resource nmt. we propose a dynamic curriculum learning (dcl) method to address the problems discussed above. the novelty of dcl is two-fold. first, we define the difficulty of a sample to be the decline of loss (i.e., negative log-likelihood). in this way, we can measure how hard a sentence can be translated via the real objective used in training. apart from this, the dcl method explicitly estimates the model competence once the model is updated, so that one can select samples that the newly-updated model has enough competence to learn. dcl is general and applicable to any nmt system. in this work, we test it in a transformer-based system on three low-resource mt benchmarks and different sized data selected from the wmt'16 en-de task. experimental results show that our system outperforms the strong baselines and several curriculum learning-based counterparts. however, these approaches rely on a large number of external resources or conditions, e.g., the auxiliary parallel corpus related to the source or target language, or a large amount of monolingual data. @xcite demonstrate the competitive nmt model can be trained with the appropriate hyperparameters in low-resource scenarios without any external resources. this is consistent with our motivation. the difference lies in that they focus on the model settings, and we explore the training strategy which utilizes bilingual data effectively for low-resource nmt. ## problem definition let d train be the training corpus and |d train | be the corpus size. @xmath0, y) is a training sample in d train , where x is the source sentence and y is the target sentence. nmt systems learn a conditional probability p (y|x): our objective is to learn better model parameters by curriculum learning in low-resource nmt. we decompose the whole training process into multiple phases @xmath1, t foot_0 , t 2 , . . . , t k ) 1 . for every phase t, the sub-optimal process can be viewed as: previous work enforces a static scoring strategy to measure sample difficulty and encourage simple functions to estimate the model competence. in this way, the training data at each phase is pre-determined before training. but in fact, sample difficulty and model competence are not independent with the current model parameters. a natural idea is to re-arrange the curriculum once the model is updated, so that we can select training data with appropriate difficulty for current training. we discuss these two questions in-depth in the following section. ## dynamic curriculum learning we propose a dynamic curriculum learning method to reorder training samples in training. we determine the order of training samples dynamically, rather than using a static scoring for reordering. besides, we propose a batching method to reduce gradient noise. ## results with the dynamic model competence, we observe the bleu scores of random (row 4) fluctuates on the baseline model. although the difficulty is measured dynamically during training, the meaningless value is not favorable for curriculum learning. we also measure the difficulty with loss dynamically (row 5), which leads to a slight improvement in the larger datasets. however, it degrades performance in the smaller datasets (th-en, 50k, and 100k). it agrees with analysis in section 4.1 that it is easy to fall into overfitting due to repeated training on some samples with low loss, especially in the extremely scarce datasets. then, we test our proposed difficulty metric of decline. with the simple competence functions (row 6 and row 7), they outperform significantly the strong baselines and previous methods. it demonstrates our proposed metric is of high relevance with sample difficulty for nmt than the other four metrics. however, the sqrt competence function does not perform better than the linear function in all datasets, this is not consistent with the previous conclusion @xcite . the possible reason is that the model competence of some low-resource datasets improves slowly in the early phases. dmc (row 8) avoids the prior hypothesis of the performance change and achieves better or similar performance compared with the above methods. finally, we batch the samples with similar difficulties foot_5 (row 9). while the model is trained with more training steps due to padding, it achieves further improvement over our curriculum learning method in all tasks. this verifies our hypothesis that the gradient of samples with similar difficulty is more stabilized. this is an interesting result and we will explore it in future work. overall, the experimental results show our proposed method achieves better performance compared with strong baselines and several curriculum learning-based counterparts in the low-resource nmt tasks. ## analysis we take en-de 50k dataset which achieves the most performance improvement to analyze our method. although it is sampled from wmt dataset, we think it can demonstrate the advantages of our method obviously. ## conclusion in this paper, we propose a dynamic curriculum learning (dcl) method to explore the effective use of bilingual data for low-resource nmt. we define the difficulty of a training sample by the decline of loss and estimate the model competence self-adaptively based on the performance of the development set. different from previous work, we re-arrange the curriculum once the model is updated, so that the training data with appropriate difficulty is learned by the current model effectively. experimental results show that our method outperforms the strong baselines and several curriculum learning-based counterparts on several low-resource translation tasks. dcl only modifies the training strategy without any external data, which has great practical significance for real low-resource scenarios. with a strong baseline model, we can improve the effectiveness of other semi-supervised methods, such as generating the high quality back-translation data. in the future, we will rethinking the existing training startegies and explore the application of dcl methods to more difficult tasks, such as unsupervised learning and model training with pseudo data.