Training and Inference Methods for High-Coverage Neural Machine Translation (2020)

Abstract: In this paper, we introduce a system built for the Duolingo Simultaneous Translation And Paraphrase for Language Education (STAPLE) shared task at the 4th Workshop on Neural Generation and Translation (WNGT 2020). We participated in the English-to-Japanese track with a Transformer model pretrained on the JParaCrawl corpus and fine-tuned in two steps on the JESC corpus and then the (smaller) Duolingo training corpus. First, during training, we find it is essential to deliberately expose the model to higher-quality translations more often during training for optimal translation performance. For inference, encouraging a small amount of diversity with Diverse Beam Search to improve translation coverage yielded marginal improvement over regular Beam Search. Finally, using an auxiliary filtering model to filter out unlikely candidates from Beam Search improves performance further. We achieve a weighted F1 score of 27.56% on our own test set, outperforming the STAPLE AWS translations baseline score of 4.31%.

## introduction currently, state of the art machine translation systems generally produce a single output translation. however, human evaluators of translation tasks will often accept multiple translations as correct. we introduce a neural machine translation (nmt) system that generates high-coverage translation sets for a single given prompt in the source language. our system was prepared for the english-to-japanese track foot_0 of the duolingo simultaneous translation and paraphrase for language education (staple) shared task @xcite at the 4th workshop on neural generation and translation (wngt 2020). the shared task datasets consist of english prompts and a weighted set of target language translations for each prompt. the task requires systems to produce translation sets for given english prompts that are evaluated on weighted f1 score, defined in appendix a. we have made our code publicly available. 2 we experimented with models trained and finetuned on the provided duolingo english-japanese prompt-translation data @xcite , the jparacrawl web-crawled corpus @xcite , as well as the japanese-english subtitle corpus (jesc) @xcite . the sizes of each dataset are summarized in our system uses a transformer-based @xcite nmt model and we began with weights pretrained on the large jparacrawl corpus @xcite . section 4 describes in detail how the model was pretrained. our system's nmt model was then obtained by fine-tuning first on the japanese-english subtitle corpus (jesc) @xcite before further fine-tuning on the duolingo training set @xcite . we outline these datasets in more detail in section 2. given the small size of the duolingo data, this multi-step fine-tuning helped the model generalize and outperformed single-step fine-tuning and no fine-tuning. high-coverage translation bitext data is not easy to mine or create, so we expect that in other settings, the size of such available training data will also be small. therefore, it is very likely that adopting a multi-step fine-tuning method may be advantageous more generally. the fine-tuning procedure is described in section 6. outputting the entire beam of candidates from 150-width beam search, scored on per token log likelihood, this two-step fine-tuned system produced the translations that we submitted to the the most effective optimization was using the ground truth weights that indicate variations in translation quality during training. we find that it is essential to deliberately expose the model to higher-quality translations more often during training. otherwise, overexposure to low-quality translations harms the model's translation performance. secondly, diverse beam search with a very small penalty outperformed beam search. however, too much diversity begins to introduce minor semantic shifts that deviate from correct translations. we also explored introducing an auxiliary filtering model for post-processing candidates. our proposed filtering model is able to refine the candidates generated by the nmt model, which improved the system's performance with respect to the weighted f1 score. we share our results in section 7. our best result was a weighted f1 score of 27.56% on our own test set of 200 prompts randomly selected from the training data. ## related work machine translation machine translation (mt) involves finding a target sentence @xmath0, ...y m with the maximum probability conditioned on a source sentence @xmath1, ...x n , i.e argmax y p (y|x). there are various neural approaches to tackle machine translation. these include utilizing recurrent neural networks @xcite , convolutional neural networks @xcite , attention-based models @xcite and transformer networks @xcite . sequence to sequence models deal with the task of mapping an input sequence to an output sequence. these were first introduced by @xcite and typically use an rnn based encoder-decoder architecture, where the encoder outputs a fixed length representation of the input which is fed into the decoder to get a target translation. rnn and lstm based approaches struggle to handle long sequences and long-range dependencies since the encoder network is tasked with encoding all relevant information in a fixedlength hidden state vector. @xcite overcome this by utilizing attention, an alignment model that can attend to important parts of the input during translation. @xcite used the attention mechanism to great effect, observing gains of 5.0 bleu over non-attention based techniques for nmt. the transformer architecture for our experiments, we used the the transformer architecture proposed by @xcite . it is a selfattention based model that produces superior results for machine translation tasks compared to cnn and lstm based models. by stacking multiple layers of multi-head self-attention blocks, they demonstrate that the attention mechanism by itself is very powerful for sequence encoding and decoding. recently, transformer-based models that are pre-trained on large-scale datasets have produced superior performance on various natural language processing (nlp) tasks @xcite @xcite . in section 4 we further describe the transformer architecture and our pretraining procedure. ## pretrained base model as our base model, we used a model pretrained by @xcite on the jparacrawl data using the fairseq framework @xcite . morishita et al. (2019) preprocessed the jparacrawl english and japanese text using sentencepiece @xcite to obtain 32,000-token vocabularies on both the english and japanese sides. ## filtering model apart from the nmt model, we additionally introduce a neural filtering model to post-process the nmt model's candidates. instead of designing a model that will assign a real-value score to each of the candidates, we simplify the task by formulating it as a binary classification problem. namely, the filtering model is trained to classify a given candidate sentence as a valid sample (in the goldstandard list) or an invalid sample. the intuition is that the gold-standard candidate list contains a small number of high-quality sentences (with larger weights) and a large number of lower-quality sentences. thus it is more important to distinguish the hits from misses than high-quality hits from low-quality hits. to construct the dataset for the filtering model, we augmented the duolinguo dataset with the results of nmt model. specifically, we labeled those result sentences that appear in the gold-standard list as true and labeled others as false. as for the model architecture, we encode the source sentence and the candidate sentence separately with a one-layer bidirectional lstm model. the encoding is the concatenation of the hidden vectors in both directions after complete traversal of the sequence, along with a (learned) positional embedding vector. this embedding encodes the position of the candidate sentence in the candidate list generated by the nmt model, which is sorted by descending score order. 4 lastly, we use a multilayer perception (mlp) to classify the concatenated vector. here, s denotes the source sentence, c i denoted the i-th candidate, v i denotes the positional encoding, and p i denotes the predicted likelihood. the filtering model is optimized with binary cross-entropy loss. ## results we conducted our experiments sequentially and generally used the best results so far as a baseline for subsequent experiments. ## conclusions and future work our machine translation system produces highcoverage sets of target language translations from single source language prompts. we used multi-step fine-tuning to train a robust nmt model. this involved first training or finetuning a model on a large bitext dataset, then finetuning on the bitext dataset with high coverage sets of target language translations, which is likely to be small. in our experiments, we find that fine-tuning a pretrained model first on a corpus similar to our intended domain and then fine-tuning further on our smaller in-domain dataset produced the best results. during training, we find that if the ground truth translations come with weights that indicate variations in their quality / likelihood, it is essential to expose the model to higher-quality translations more often during training. one way to do this is to to sample the training data with probabilities commensurate to the ground truth weights. doing so will prevent overexposure to low-quality translations that ultimately harm the model's translation performance. for decoding, we find that beam search scored on per token log likelihood finds very good translation candidates on its own. nevertheless, instead using diverse beam search with a very small penalty improves coverage. we observed a further performance boost from post-processing the translation candidates. this was achieved by training an auxiliary filtering model on the results of the nmt model to prune unlikely candidates as a final step. one idea for future work is to directly optimize the weighted f1 score during training using reinforcement learning. as the weighted f1 score is not a differentiable function, it is impossible to train directly on this metric using maximum likelihood estimation. instead, one may use policy gradients under a reinforcement learning paradigm to do so.