H w T sc SU ’s Submissions on WAT 2022 Shared Task (2022)

Abstract: In this paper we describe our submission to the shared tasks of the 9th Workshop on Asian Translation (WAT 2022) on NICT–SAP under the team name ”HwTscSU”. The tasks involve translation from 5 languages into English and vice-versa in two domains: IT domain and Wikinews domain. The purpose is to determine the feasibility of multilingualism, domain adaptation or document-level knowledge given very little to none clean parallel corpora for training. Our approach for all translation tasks mainly focused on pre-training NMT models on general datasets and fine-tuning them on domain-specific datasets. Due to the small amount of parallel corpora, we collected and cleaned the OPUS dataset including three IT domain corpora, i.e., GNOME, KDE4, and Ubuntu. We then trained Transformer models on the collected dataset and fine-tuned on corresponding dev set. The BLEU scores greatly improved in comparison with other systems. Our submission ranked 1st in all IT-domain tasks and in one out of eight ALT domain tasks.

## introduction explorations on machine translation have come far since the era of neural machine translation (nmt) @xcite . owing to the incorporation of novel structures such as cnn @xcite and transformer @xcite , modern nmt models are able to compete with human translation. however, the performance of neural machine translation is often highly relevant to the size of available datasets. when the training datasets are small in quantity, performances of nmt models are often poor, especially for low-resource languages. considering that it is often helpful, in such low-resource scenarios, to leverage monolingual or bilingual corpora from multiple languages and domains to boost translation quality, we collected a large amount of web-crawled datasets for training models in the task. the workshop on asian translation foot_0 @xcite is an open machine translation evaluation campaign focusing on asian languages. wat gathers and shares the resources and knowledge of asian language translation to understand the problems to be solved for the practical tasks of machine translation technologies among all asian countries. among those tasks, we participated on nict-sap tasks which evaluate hindi/thai/malay/indonesian/vietnamese ↔ english translation in two domains: it domain (software documentation) and wikinews domain (alt). it domain and wikinews are two extremely low-resource domains for machine translation, especially when concerning languages such as hindi, thai, malay, indonesian and vietnamese. often, in these domains, there is no clean bilingual parallel corpus at all (the it domain), or the size of available corpora is extremely small (the wikinews). both two corpora contain a lot of technical terms. moreover, some technical terms are domainspecific and do not exist in general dictionaries. therefore, we focus on domain adaptation for translations of both it and wikinews domains. in this paper, we describe our simple approach involving transformer pre-training and fine-tuning. we first collected and cleaned rich sentence pairs from public dataset. following berling lab @xcite , for both nict-sap it domain and alt domain tasks we first collected public dataset from opus @xcite such as but not limited to gnome, kde4 and ubuntu. then we chose g-transformer @xcite as our model and pre-train the baseline with these datasets. finally, as fine-tuning on domain-specific dataset greatly boosts translation performance in wmt evaluation @xcite , we fine-tuned the pre-trained models on corresponding dev set officially provided for high performance for all sub-tasks. our method obtained the new state-of-the-art results on it-domain tasks. we ranked first place on all nict-sap it domain tasks, especially achieving 10.74 improvement for english to malay. on alt domain tasks, we ranked first in one out of eight sub-tasks. ## our approaches for our submissions we focus on training g-transformer @xcite on opus dataset from scratch and fine-tuning on dev sets. g-transformer is developed on fairseq @xcite for document-level translation, and also supports transformer-based sentence-level translation. ## conclusion in this paper, we have described our translation models to the nict-sap translation tasks on nict-sap track. we first pre-train our models from scratch on the datasets from opus. then we finetune the models on corresponding dev sets. experi-mental results have shown that our model ranked firsts place for nict-sap it domain tasks and achieved good performance for nict-sap alt domain tasks.