LIT Team’s System Description for J apanese- C hinese Machine Translation Task in IWSLT 2020 (2020)

Abstract: This paper describes the LIT Team’s submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction. Our system is based on the organizers’ baseline system, but we do more works on improving the Transform baseline system by elaborate data pre-processing. We manage to obtain significant improvements, and this paper aims to share some data processing experiences in this translation task. Large-scale back-translation on monolingual corpus is also investigated. In addition, we also try shared and exclusive word embeddings, compare different granularity of tokens like sub-word level. Our Japanese-to-Chinese translation system achieves a performance of BLEU=34.0 and ranks 2nd among all participating systems.

## introduction in recent years, the neural machine translation (nmt) @xcite @xcite has made great progress based on encoder-decoder architecture. we participate in the iwslt 2020 open domain translation task: japanese-to-chinese. this paper describes the nmt systems for the iwslt 2020 japanese-to-chinese machine translation task @xcite . our main efforts are data pre-processing, specifically parallel data filter and sentence alignment. by elaborate data processing, we successfully improve the quality of the training set and thus boost the performance of our translation system. the backtranslation mechanism @xcite is also investigated to extend the training corpus, we translate chinese to japanese to get the japanese-to-chinese training corpus, it is an effective approach to exploit the corresponding monolingual data sets. the transformer model @xcite based on multi-head attention has achieved excellent performance in a variety of neural machine translation tasks in the last three years. this kind of nmt model surpasses the performance of the traditional statistical machine translation and the nmt performs particularly well especially with rich resource corpus. in our system, we adopt bigger transformer architecture, since the performance of the transformer relies on model capacity, ex. the number of dimensions of the feed-forward network. to improve performance, we adopted the relative position attention @xcite . also, we conduct experiments to compare the shared source and target word embeddings and exclusive word embeddings, and whether to adopt the shared embeddings relates to the translation direction. the chinese-to-japanese direction achieves higher scores when adopting shared word embeddings, however, the japanese-to-chinese direction produces opposite results. the paper is structured as follows: section 2 will present a detailed description of our data preprocessing, back-translation is introduced in section 3, the model of our system will be introduced in section 4, the main results of our experiment will be shown in section 5. section 6 will draw a brief conclusion of our work for the iwslt 2020 open domain translation task. ## data and pre-processing on the whole, our system follows the standard transformer-based translation pipeline, and our system implementation is based on the official baseline foot_0 . most of our efforts in this competition are focused on data pre-processing and back-translation. we adopt the same strategies as the official baseline if we don't point out explicitly. corpus original re-filtered part a 2.0m 0.8m part b 19.0m 8.8m part c 161.5m 10.0m part d -2.7m monolingual -200m ## back-translation in recent works, the back-translation mechanism @xcite has been proved as an effective method to improve machine translation systems by utilizing large-scale monolingual corpus. algorithm 1 align bilingual sentences from two documents. require: chinese sentences c 1 , c 2 , • • • , c n ; japanese sentences j 1 , j 2 , • • • , j m ; ensure: aligned sentence pairs set a 1: initialize all auxiliary variables s to zero; 2: for @xmath0: ## model we think the transformer model is a strong model with excellent performance. so, we only take some small tricks on this model. in this section, we describe two different methods to enhance our model performance in this competition. all of them come from previous work @xcite and all of these methods help us to improve the baseline model. in the subsection, we will describe these methods briefly. ## conclusion we participated in the japanese-to-chinese translation direction in the iwslt 2020 open domain translation task. we focus on improving the transformer baseline system by doing elaborate data pre-processing, and we manage to obtain significant improvements. experiments also show that increasing model capacity is beneficial on large training data. finally, our submission of japanese-to-chinese translation achieves the 2nd highest bleu score among all the submissions.