The U niversity of T okyo’s Submissions to the WAT 2020 Shared Task (2020)

Abstract: The paper describes the development process of the University of Tokyo’s NMT systems that were submitted to the WAT 2020 Document-level Business Scene Dialogue Translation sub-task. We describe the data processing workflow, NMT system training architectures, and automatic evaluation results. For the WAT 2020 shared task, we submitted 12 systems (both constrained and unconstrained) for English-Japanese and Japanese-English translation directions. The submitted systems were trained using Transformer models and one was a SMT baseline.

## introduction we describe the machine translation (mt) systems submitted to the wat 2020 document-level business scene dialogue translation sub-task developed by the team of the university of tokyo. we chose the identifier of our team to be ut-mrt, which specifies our affiliation (the university of tokyo) and first names @xcite . we participated in both @xmath0. we experimented with mixing and matching several data sets, data processing approaches and training methods. our main findings are: 1) using source side context mainly improves @xmath1, but not always, and mainly degrades or leaves little impact on @xmath2; 2) there are no better data than more data -we see the biggest improvements from using larger training data sets; and 3) optimiser delay @xcite can help a lot -by setting the optimiser delay value to 8 instead of the default 1 increased bleu scores by more than 1.5 in both translation directions. ## data we used multiple dataset combinations to train our models for the shared task. we also filtered some of the larger automatically collected data sets which are usually more noisy and contain duplicates. aside from using only the provided bsd training dataset (bsd 20 @xcite )), we had access to an extended version of the bsd four times the size (bsd 80), as well as two other similar corpora -ami meeting corpus (ami) and a parallel version of ontonotes 5.0 (on) @xcite . we also experimented with using the jparacrawl @xcite dataset, data from wmt 2020 foot_0 (whcih includes jparacrawl, ted talks @xcite , the kyoto free translation task corpus @xcite , japanese-english subtitle corpus @xcite , wikimatrix @xcite and wiki titles v2), and a proprietary document-aligned news dataset gathered from several sources. the full training data statistics are shown in ## model configurations we separate our submissions in 3 main categories by model configuration type -statistical mt (smt) baseline models, nmt models and nmt models with context. the latter category also includes models with tags specifying the domain (or rather training corpus used). ## results we use the sacrebleu foot_2 tool @xcite to evaluate automatic translations and calculate bleu scores @xcite in ## conclusion the paper described the development process of the the university of tokyo's mt systems that were submitted for the wat 2020 document-level business scene dialogue translation sub-task. among other things, we experimented with adding previous context to training data, larger batches and domain specifying tags. while we did find some slight bleu score improvements when training context-aware models, document-aligned data required to train them are still rare and rather small in size. more substantial improvements were gained by simply adding all available sentence-aligned corpora and training regular nmt models. in contrast to our expectation that the contextaware models will be superior at least for the @xmath3, where we saw gains in bleu scores, results from the human evaluation showed otherwise. we believe that a more sophisticated training method may be required to fully take advantage the document-aligned data. we did not perform any back-translation of monolingual business dialogue or similar corpora, nor did we train transformer-big models or perform model distillation. all of these are other popular methods used in similar shared tasks known to improve the final results. our intuition is that such moves would further improve the final outcome by several bleu points, but due to time constraints we chose not to go forward with them. in total, 26 systems were submitted for the english↔japanese language pair and four of them to the human evaluation.