SRT ’s Neural Machine Translation System for WMT 22 Biomedical Translation Task (2022)

Abstract: This paper describes the Samsung Research’s Translation system (SRT) submitted to the WMT22 biomedical translation task in two language directions: English to Spanish and Spanish to English. To improve the overall quality, we adopt the deep transformer architecture and employ the back-translation strategy for monolingual corpus. One of the issues in the domain translation is to translate domain-specific terminologies well. To address this issue, we apply the soft-constrained terminology translation based on biomedical terminology dictionaries. In this paper, we provide the performance of our system with WMT20 and WMT21 biomedical testsets. Compared to the best model in WMT20 and WMT21, our system shows equal or better performance. According to the official evaluation results in terms of BLEU scores, our systems get the highest scores in both directions.

## introduction neural machine translation (nmt) has shown rapid growth with an encoder-decoder framework, especially transformer @xcite , in recent years. most of the research focuses on general-purpose translation models since there are a lot of parallel data available. on the other hand, domain-specific translation, which lacks relatively high-quality parallel corpus available, is one of the challenges that need to be solved in the nmt task. to address this issue, there have been several approaches such as finetuning generalpurpose models with in-domain data and utilizing in-domain monolingual corpus through backtranslation @xcite . in the domain translation, one of the issues is the terminology translation. in the case of domainspecific terms, translation results are often poor because they are relatively infrequent. @xcite also mentioned that some domain-specific terms including abbreviations were not translated correctly in previous shared tasks. moreover, when new terms are introduced such as covid-19, it is difficult to obtain the correct translation results as they are not in the training data. to handle this issue, we adopt the soft-constrained terminology translation proposed by @xcite , which provides the terminology constraints of the target language as input to our system with source sentences like a hint. these terminology constraints can be obtained from in-domain dictionaries. in addition, as many domain translation studies, the back-translation strategy @xcite is applied to generate synthetic parallel data from in-domain monolingual corpus. to improve the overall performance of our system, we also employ the deep transformer architecture @xcite and the ensemble strategy @xcite . moreover, to find better translation results, noisy channel modeling @xcite and discriminative reranking @xcite are attempted. our experiment shows that deep transformer and data augmentation by the back-translation strategy improve the overall performance while the performance is not improved with reranking methods. the rest of this paper is organized as follows. section 2 describes the training and test data used in our system, and section 3 explains our systems including deep transformer and soft-constrained terminology translation. section 4 describes the details of our training and experimental results of our system; section 5 presents the official evaluation results. section 6 is the conclusion of our work. ## data in this section, we present general-domain (out-ofdomain) corpus, in-domain corpus, and in-domain terminology dictionaries used as the training data in our system. for training, all data are tokenized by sentencepiece @xcite ; the vocab size is 32k for each lanaugage. the validation and test data are also described in this section. the statistics of our data are listed in ## system overview in this section, we describe our system which is based on transformer architecture @xcite . the training details are described in section 4.1. @xcite have shown that deeper layers could efficiently extract syntactic and semantic information that could improve the overall performance. @xcite also have explored deeper encoders for transformer to improve the translation quality. several teams that participated in the biomedical shared task last year @xcite ) have adopted the deep transformer, especially deeper encoders. in this paper, we also adopt the deep transformer architecture which contains 30 encoder layers and 6 decoder layers based on transformer-big setting @xcite . ## experiments in this section, we present training details and experimental results of our systems. ## official evaluation results the official evaluation results of our submissions (srt) for wmt 2022 biomedical translation task are shown in ## conclusion this paper presents the samsung research's translation system (srt) for the wmt22 biomedical translation shared task in two language directions: english to spanish and spanish to english. we perform experiments with several strategies such as deep transformer, data augmentation, soft-constrained terminology translation, ensembling models, and reranking methods. our experiments show the effectiveness of each strategy. the deep transformer, data augmentation, and ensemble strategies improve effectively the overall performance in the domain translation. moreover, we present that the soft-constrained terminology translation is a reasonable method to achieve good performance in the domain translation. our systems show the best bleu scores in the official evaluation results.