Huawei B abel T ar NMT at WMT 22 Biomedical Translation Task: How We Further Improve Domain-specific NMT (2022)

Abstract: This paper describes Huawei Artificial Intelligence Application Research Center’s neural machine translation system (“BabelTar”). Our submission to the WMT22 biomedical translation shared task covers language directions between English and the other seven languages (French, German, Italian, Spanish, Portuguese, Russian, and Chinese). During the past four years, our participation in this domain-specific track has witnessed a paradigm shift of methodology from a purely data-driven focus to embracing diversified techniques, including pre-trained multilingual NMT models, homograph disambiguation, ensemble learning, and preprocessing methods. We illustrate practical insights and measured performance improvements relating to how we further improve our domain-specific NMT system.

## introduction the existing mainstream neural machine translation (nmt) system is predominantly data-driven. our participation in wmt biomedical tasks traced back from 2019 has witnessed pursuits extending beyond this modality. in our wmt20 and wmt21 submissions, various domain adaption technologies @xcite have been applied including practical approaches finetuning on general-purpose models, back-translation @xcite and leveraging in-domain dictionaries @xcite . despite achieving state-of-the-art (sota) bleu scores for most of our submissions in the last two years, under-translation occurred in the "english ↔ chinese" due to the models' incapability to handle long sentences @xcite . it was rectified by ensembling the affected model with the baseline, resulting in a decrease in bleu scores. in addition, the models trained predominately with the general * corresponding author domain data still face challenges associated with domain adaptation. in this paper, we present practical insights into how we further improve huawei artificial intelligence application research center's neural machine translation system ("babeltar") in domainspecific machine translation. this year, our participation in the wmt22 biomedical translation task covers language directions between "english (en)" and the other seven languages "german (de)", "spanish (es)", "french (fr)", "italian (it)", "portuguese (pt)", "russian (ru)" and "chinese (zh)". more specifically, we adopt in-house general-purposed bilingual nmt models built upon the transformer-big architecture @xcite and a pre-trained multilingual nmt model (m2m100) @xcite with an m2m100-418m configuration as baseline models. finetuned with the in-domain data provided by the organizer, the back-translated monolingual medline data in english dating before july 2018, the in-domain dictionaries enhanced with terminologies, the models can be improved significantly over the last year's submissions, for example, +1.18 bleu on "@xmath0.24 bleu on "@xmath1. leveraging the knowledge learned in addressing the ambiguities caused by homographs, we can further boost +0.65 bleu in the language direction of "@xmath2. by optimizing the sequence length during decoding, we successfully solve the issue of undertranslation in the language pair of "en ↔ zh". ## the data in this section we detail the bilingual and monolingual corpora used in this shared task ( the models used for back-translation are from our last year's shared task @xcite . ## experimental results and analysis as ood data also contribute to the domain-specific nmt @xcite , both ood data and ind data are used to finetune the nmt bilingual and multilingual nmt models. ok-aligned wmt21 test data are used for evaluation in the experiments. the bleu scores are evaluated using the mteval script from moses @xcite with results shown in ## discussion it is the fourth year we have participated in this shared task, and we have made significant progress in our submissions measured against officially released test data from previous years. but the improvements for some language directions are not always accompanied by a consistent uplift of bleu sentence example input the disease duration ranged from 2 weeks to 60 months (median, 4 months), and the affected segment was c all the patients were followed up 3 to 42 months (median, 12 months). wang et al. (2021 for the contest year. the learned nmt models still suffer from "out of distribution" issues many deep learning models have encountered. apart from maintaining the nmt models with a large amount of the latest ind data, we need to design deep learning systems to adapt to changes in distributions @xcite . on another point, we realized that the reference data sometimes do not reflect the ground truth of the translation during our manual evaluation process. it raises a related question about the rationale of using bleu as an exclusive automatic evaluation criterion. although bleu may remain the default metric for evaluating machine translation quality, we strongly suggest the community inves-tigate complementary metrics capable of accommodating good translation results with semantics variations in this shared task. ## conclusion this paper depicts huawei's neural machine translation system ("bebeltar") and the submission to the wmt22 biomedical shared task. the submission consists of fourteen models covering language directions between english and all seven other languages available in this track. we can improve the domain-specific nmt significantly by leveraging a broad range of techniques, which includes pretrained multilingual nmt models, lexicon-based enhancement, homograph disambiguation, ensemble learning, preprocessing and postprocessing, etc. in the meantime, we share practical insights on achieving the measured performance, hoping to contribute to the machine translation community in this shared task. our future work will focus on investigating mechanisms to adapt a domain-specific nmt model to different distributions.