Exploiting Curriculum Learning in Unsupervised Neural Machine Translation (2021)

Abstract: Back-translation (BT) has become one of the de facto components in unsupervised neural machine translation (UNMT), and it explicitly makes UNMT have translation ability. However, all the pseudo bi-texts generated by BT are treated equally as clean data during optimization without considering the quality diversity, leading to slow convergence and limited translation performance. To address this problem, we propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. Specifically, we first apply crosslingual word embedding to calculate the potential translation difficulty (quality) for the monolingual sentences. Then, the sentences are fed into UNMT from easy to hard batch by batch. Furthermore, considering the quality of sentences/tokens in a particular batch are also diverse, we further adopt the model itself to calculate the fine-grained quality scores, which are served as learning factors to balance the contributions of different parts when computing loss and encourage the UNMT model to focus on pseudo data with higher quality. Experimental results on WMT 14 En-Fr, WMT 14 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks demonstrate that the proposed method achieves consistent improvements with faster convergence speed.

## introduction unsupervised neural machine translation (unmt) @xcite has made significant progress @xcite @xcite in recent years. it consists of three main components: the initialization of the crosslingual pre-trained language model (plm), denoising auto-encoder (ae) @xcite , and i think sentence a is easier because i know the translations of most words in it. id pseudo bi-text and corresponding difficulty scores a sentence 我们 尊重 所有 的 信仰 。 token difficulty 0.31 0.22 0.28 0.16 0.29 0.16 unmt translation we respect all the beliefs . b sentence ( 小 标题 ) 武装 何去何从 token difficulty 0.30 0.25 0.48 0.30 0.36 1.0 unmt translation ( small text ) armed forces 何去何从 back-translation (bt) @xcite . bt generates pseudo bi-texts for training and explicitly enables its translation ability. however, pseudo bi-texts are quite diverse in quality, and the lowquality bi-texts are difficult to learn. equally treating pseudo bi-texts as clean data would negatively influence the convergence process and harm the translation performance @xcite . recently, curriculum learning (cl) @xcite , which aims to help the model learn from easy samples to the hard ones, has shown its effectiveness in speeding up the convergence and improving performance. just as the name implies, the critical point of cl is difficulty criteria. @xcite classify criteria in supervised machine translation into linguistic-inspired criteria @xcite and model-based criteria @xcite @xcite . most of them are designed from the perspective of the source side in the pure parallel corpus. however, pseudo bi-texts produced by bt with monolingual sentences in unmt contain different levels of noise, and low-quality samples with much noise would be difficult for the model to learn appropriately @xcite . in this paper, we propose a cl method to gradually utilize pseudo bi-texts for unmt from easy to hard, helping the model concentrating on the data with high quality from multiple granularities. intuitively, pseudo bi-text with high quality is more accessible and suitable for unmt optimization. accordingly, we will measure the sample difficulty with bi-text quality. first, we apply the unsupervised cross-lingual word embedding @xcite to calculate the quality of bi-texts, which is in turn used to measure the sample quality. then, samples will be fed into unmt from easy to hard batch by batch based on their difficulty. figure 1 illustrates that it is reasonable to bridge the bi-text quality and the sample difficulty. however, the batch-based standard learning procedure is coarse-grained, and the qualities of pseudo bi-texts at sentence/word-level in a particular batch are also different, which should be addressed. to perform such fine-grained learning from easy to difficult, we borrow the idea from selfpaced learning @xcite , which is an adapted cl algorithm. specifically, we first adopt the model to estimate the quality scores of pseudo bi-texts. then, the scores are served as learning factors to balance the contributions of different parts when computing the training loss, encouraging the unmt model to concentrate on the parts with higher quality. in general, the contributions of this paper can be summarized as follows: ## background of unmt the architecture of the current state-of-the-art unmt is the same as supervised nmt model, except that the unmt model simultaneously processes both translation directions. the training procedure comprises three main components: the initialization of cross-lingual plm, denoising autoencoder and back-translation. cross-lingual plm is the auto-encoder that aims to encode the source sentences and target sentences into a shared embedding space. the parameters are used to initialize the encoder and decoder in unmt model before training. denoising auto-encoder is one of the crucial components for unmt. it can improve the model learning ability through reconstructing the original sentences from the sentences with artificial noise, such as random deletion, swapping, or blanking. it is optimized by minimizing the following objective function: back translation is another essential component of unmt, which explicitly ensure the model to have translation ability. first, each batch of monolingual sentences is translated into the other language by unmt model m . then, m applies the pseudo parallel sentences (m l 1 →l 2 (x), x) and (m l 2 →l 1 (y), y) into training. the process is called on-the-fly back translation. the objective function is: in conclusion, the final loss during unmt training can be written as follow: even though strong unmt models have been proposed in recent years, such as xlm @xcite and mass @xcite . the uneven quality of pseudo bi-text is still harmful. first, pseudo bi-texts are produced at each round. the translation performance in the early stages is pretty low and will affect the final results. second, equally treating pseudo bi-texts with uneven quality can bring deviation to the optimization, slowing down the convergence speed and restricting translation performance. ## approach in this section, we introduce the proposed cl method for unmt. as shown in figure 2 , our method consists of two sub-modules that work at different levels: round n …… unmt model monolingual dataset loss the quality of bi-text round 2 round 1 finished？ optimize weight batches from easy to hard back-translation steps batch level sentence / token level figure 2: illustration of our method. batch level cl is shown below the black dash line, which controls the dataloader to prepare batches based on sample difficulty. sentence/token level cl is illustrated above the black dash line, applying unmt model to estimate the quality of pseudo bi-text and weight the training loss. ## analysis 6.1 correlation between the difficulty and improvements even though our methods improve across all the language pairs, it remains a question which part of sentences contribute more to the performance. figure 4 shows the bleu improvements at different difficulty intervals on wmt @xmath0. the difficulty is represented by the definition described in section 3.1.1. we find that our approach outperforms xlm baseline in different difficulty intervals. the easiest sentences (<2%) have significant improvements, which owes to the emphasis on the easy samples during training. in contrast, hard sentences (>70%) have limited performance gains. figure 5 shows the relationship between the improvement of sentence-level bertscore @xcite and the difficulty distribution. the larger points are sparsely distributed on the left side, indicating that simple sentences achieve significant improvements. and the minor points are concentrated in the lower right corner, meaning that complex sentences yield slight performance improvement. this finding is different from related works on supervised nmt @xcite , which prove that curriculum learning is beneficial for complex samples. we suspect the reason lies in the particularity of our method, considering the quality of bi-texts instead of the pure difficulty. therefore, we think our method helps the unmt model mainly strengthen its essential translation ability. ## conclusion in this paper, we propose a multi-granularity cl method to improve unmt. specifically, a novel cross-lingual difficulty definition is first proposed to help unmt learn from easy samples to the hard ones at batch level. then, the qualities of pseudo bi-text at sentence/word-level are estimated by the model itself to regulate the loss function, automatically helping unmt optimize in the appropriate direction. empirical results show that our method outperforms the strong baselines on different lan-guage pairs with faster convergence speed. further analyses confirm that our cl methods at different levels are helpful and complementary with each other, indicating the suitability for unmt. in the future, we will explore its ability on multilingual machine translation and other cross-lingual generation tasks.