HW - TSC Translation Systems for the WMT 22 Biomedical Translation Task (2022)

Abstract: This paper describes the translation systems trained by Huawei translation services center (HW-TSC) for the WMT22 biomedical translation task in five language pairs: English↔German (en↔de), English↔French (en↔fr), English↔Chinese (en↔zh), English↔Russian (en↔ru) and Spanish→English (es→en). Our primary systems are built on deep Transformer with a large filter size. We also utilize R-Drop, data diversification, forward translation, back translation, data selection, finetuning and ensemble to improve the system performance. According to the official evaluation results in OCELoT or CodaLab, our unconstrained systems in en→de, de→en, en→fr, fr→en, en→zh and es→en (clinical terminology sub-track) get the highest BLEU scores among all submissions for the WMT22 biomedical translation task.

## introduction machine translation (mt) refers to the automatic translation of text from one language to another, and the biomedical translation task aims to evaluate the performance of mt systems in the biomedical domain. in this year's biomedical translation task, our team (hw-tsc) participates in five language pairs, including en↔de, en↔fr, en↔zh, en↔ru and @xmath0. since the size of in-domain (id) data is limited, we first use a large amount of out-of-domain (ood) data to train our baseline neural machine translation (nmt) @xcite @xcite system, which is a deep transformer model @xcite leveraging r-drop @xcite training strategy. we then use the collected id data (except the data from medical database) to further train the nmt model for domain transfer. to better use the limited id training data, we employ data selection to extract id data from ood data, in addition to basic data augmentation strategies including data diversity, forward translation and back translation. finally, we use finetuning @xcite and model ensemble @xcite to further improve model performance in the biomedical domain. this paper is structured as follows: we describe data size and data pre-processing methods in section 2; the model structure and training methods in section 3; final results in section 4; and conclusion in section 5. ## experimental result during the training phase, we use pytorch-based fairseq foot_3 @xcite open-source framework, and use deep transformer model architecture as our benchmark system. each model is trained using 8 gpus with a batch size of 2048. the update frequency is 4 and the learning rate is 5e-4, the label smoothing rate @xcite ) is 0.1, the warm-up steps is 4000, and the dropout is 0.3. adam optimizer (kingma and ba, 2015) with β1=0.9 and β2=0.98 is also used. furthermore, we use reg_label_smoothed_cross_entropy as the loss function and set reg-alpha to 5 when applying r-drop @xcite training strategy. in the evaluation phase, we use marian foot_4 @xcite for decoding and then calculate the sacrebleu foot_5 @xcite on the wmt21 ok-aligned biomedical test set to measure the performance of each model. ## conclusion this paper presents our translation system for the wmt22 en↔de, en↔fr, en↔zh, en↔ru and @xmath1. during the experiment, we use r-drop and id bilingual data finetuning methods to build our id translation system, and then use data diversity, forward translation, back translation and data selection methods to expand the size of training data for training a better system. we also adopt finetuning and ensemble to further improve the system performance. according to the official evaluation results in ocelot or codalab, our submitted systems achieve the highest bleu scores in six language directions of the wmt22 biomedical translation task.