Goku’s Participation in WAT 2020 (2020)

Abstract: This paper introduces our neural machine translation systems’ participation in the WAT 2020 (team ID: goku20). We participated in the (i) Patent, (ii) Business Scene Dialogue (BSD) document-level translation, (iii) Mixed-domain tasks. Regardless of simplicity, standard Transformer models have been proven to be very effective in many machine translation systems. Recently, some advanced pre-training generative models have been proposed on the basis of encoder-decoder framework. Our main focus of this work is to explore how robust Transformer models perform in translation from sentence-level to document-level, from resource-rich to low-resource languages. Additionally, we also investigated the improvement that fine-tuning on the top of pre-trained transformer-based models can achieve on various tasks.

## introduction this paper introduces our neural machine translation (nmt) systems' participation in the 7th workshop on asian translation (wat-2020) shared translation task @xcite . we participated in the (i) jpo patent, (ii) documentlevel business scene dialogue (bsd) translation, and (iii) mixed-domain tasks. in particular, the document-level translation tasks are newly introduced for wat 2020 as traditional translation tasks such as aspec usually focus on sentence-level translation, whose quality tends to saturation. we built our nmt systems based on the standard transformer @xcite for the jpo patent and mixed-domain tasks. in addition to standard transformer, a pre-training auto-encoder model mbart @xcite has been explored in the jpo patent task. in terms of the documentlevel translation task, we evaluated on the bsd corpus using the hierarchical transformer mod-els @xcite and compared the results with our fine-tuned mbart models, which were initially built to deal with the document-level translation as a downstream task. the nmt systems for the jpo patent task have been trained in a constrained manner, which means no other resources were used except training corpus provided by the shared task organizers, and achieved remarkable performance. on the other hand, we leveraged other data resources when only limited number of data provided for model training. for instance, we included the japanese-english subtitle corpus (jesc) @xcite and myth corpus @xcite as auxiliary training data for the document-level and mixeddomain translation tasks, respectively. our main findings for each task are summarized in the following: ## conclusion we presented our submissions (team id: goku20) to the wat 2020 shared translation tasks in this paper. we trained transformer-based nmt systems across different tasks. we found that additional training datasets from other resources could lead to substantial performance gains on smaller data sets. we also validated the capability of transformers with pre-training in dealing with the sentencelevel and document-level tasks, especially when the data hungry problem appeared. finally, we attempted data augmentation approaches on the lowresource language translation tasks and achieved outperforming experimental results.