Huawei AARC ’s Submissions to the WMT 21 Biomedical Translation Task: Domain Adaption from a Practical Perspective (2021)

Abstract: This paper describes Huawei Artificial Intelligence Application Research Center’s neural machine translation systems and submissions to the WMT21 biomedical translation shared task. Four of the submissions achieve state-of-the-art BLEU scores based on the official-released automatic evaluation results (EN->FR, EN<->IT and ZH->EN). We perform experiments to unveil the practical insights of the involved domain adaptation techniques, including finetuning order, terminology dictionaries, and ensemble decoding. Issues associated with overfitting and under-translation are also discussed.

## introduction general-purpose machine translation systems have limited capability in addressing domain-specific tasks @xcite , for example, the wmt biomedical translation shared task, due to the low availability for high-quality in-domain data. in our wmt20 submission, various domain adaption technologies @xcite have been applied including practical approaches finetuning on general-purpose models, back-translation @xcite and leveraging in-domain dictionaries @xcite . despite achieving state-of-the-art (sota) bleu scores for most of the submissions, few efforts were put in place to disclose the practical insights associated with these techniques. this year, the artificial intelligence application research center (aarc) participate in the wmt21 biomedical translation task for eight language directions between english and other four languages @xcite . the baseline model is an in-house general-purpose nmt model built upon the transformer-big architecture @xcite . apart from presenting an overview of the proposed biomedical neural * co-first authors. machine translation (nmt) system, we investigate the practical insights of the involved domain adaptation techniques, including finetuning order, terminology dictionaries, and ensemble decoding. issues associated with overfitting to in-domain data and under-translation are also discussed. ## the data in this section we detail the bilingual and monolingual data used in this shared task ( ## the approaches the proposed systems are finetuned using the following methods. all models are trained on one tesla v100 gpu, taking approximately 8-20 hours depending on the volumes of data involved. ## experimental results and analysis the base systems are trained with ood data and finetuned using ind data enhanced with monolingual data to produce the submitted results. we extract the ok-aligned data from the last two years (wmt19 and wmt20) and produce the test data to evaluate the nmt models. the bleu scores are calculated using the mteval script from moses @xcite . results are shown in ## conclusion this paper depicts huawei's neural machine translation systems and submissions to the wmt21 biomedical shared task. we have achieved stateof-the-art bleu scores for four of eight language pairs (@xmath0, en↔it and @xmath1. we also explore practical issues for the involved domain adaptation techniques, including the effects of finetuning order, terminology dictionaries, and ensemble decoding on enhancing the performances of cross-domain nmt. we have discussed issues associated with overfitting and under-translation.