Meta Ensemble for J apanese- C hinese Neural Machine Translation: K yoto- U + ECNU Participation to WAT 2020 (2020)

Abstract: This paper describes the Japanese-Chinese Neural Machine Translation (NMT) system submitted by the joint team of Kyoto University and East China Normal University (Kyoto-U+ECNU) to WAT 2020 (Nakazawa et al.,2020). We participate in APSEC Japanese-Chinese translation task. We revisit several techniques for NMT including various architectures, different data selection and augmentation methods, denoising pre-training, and also some specific tricks for Japanese-Chinese translation. We eventually perform a meta ensemble to combine all of the models into a single model. BLEU results of this meta ensembled model rank the first both on 2 directions of ASPEC Japanese-Chinese translation.

## introduction neural machine translation (nmt) @xcite has led to large improvements in machine translation quality when large parallel corpora are available for training. in this work, we revisit several existing nmt based techniques on aspec japanese-chinese translation task. furthermore, we conduct a meta ensemble to fuse various nmt based systems. the aspects this work feature can be summarized as: although only our team participated in the aspec japanese-chinese translation task this year, bleu results we report on the wat official leader-board rank 1st both on @xmath0. ## aspec japanese-chinese translation task aspec (asian scientific paper excerpt corpus) @xcite was constructed in the japanese-chinese machine translation project from 2006 to 2010 by japan science and technology agency. aspec japanese-chinese (aspec-jc, shown in ## preprocessing and training details for fine-tuning mbart25. for parts of the outof-domain chinese sentences that are in traditional chinese, we transfer traditional chinese characters to simplified chinese ones. 8 we conduct all the experiments by using fairseq foot_8 @xcite , an open source sequenceto-sequence framework implementation. most systems are set to the transformer-big setting except those built up by other architectures or different model capacities. in particular, our model has a 6layer encoder and decoder, a hidden size of 1,024, a feed-forward hidden layer size of 4,096, batch-size of 2,048, dropout rate of 0.3 and 16 attention heads. for lstm, we also use a 6-layer encoder and decoder architecture. for convs2s and lightconv, we use the default settings in fairseq. all the systems are early stopped if bleu does not improve for continuous 50,000 steps. experiments are run on 8 titan x (pascal) gpus except that single v100 gpu is used for mbart25 fine-tuning. we use bleu @xcite for automatic evaluation. ## conclusion in this work, we participated in aspec-jc translation task. we revisited several strong nmt base-lines, tricks for handling training set, and also pre-training + fine-tuning methods for japanese-chinese nmt. furthermore, we conducted a deliberate search of better ensembled models and obtained state-of-the-art translation results on this task. because of the high bleu results on this task, in the future, adapting systems trained on aspec to other domains should be explored and unsupervised machine translation on this task can be focused on.