LIMSI @ WMT 2020 (2020)

Abstract: This paper describes LIMSI’s submissions to the translation shared tasks at WMT’20. This year we have focused our efforts on the biomedical translation task, developing a resource-heavy system for the translation of medical abstracts from English into French, using back-translated texts, terminological resources as well as multiple pre-processing pipelines, including pre-trained representations. Systems were also prepared for the robustness task for translating from English into German; for this large-scale task we developed multi-domain, noise-robust, translation systems aim to handle the two test conditions: zero-shot and few-shot domain adaptation.

## introduction this paper describes limsi's submissions to the translation shared tasks at wmt'20. this year we have focused our efforts on the biomedical translation task, developing a resource-heavy system for the translation of medical abstract from english into french, using back-translated texts, terminological resources as well as multiple preprocessing pipelines, including pre-trained representations. systems where also prepared for the robustness task for translating from english into german; for this large-scale task we developed multi-domain, noise-robust, translation systems aim to handle the two test conditions: zero-shot and few-shot domain adaptation. machine translation for the biomedical domain is gaining interest owing to the unequivocal significance of medical scientific texts. the vast majority of these texts are published in english and biomedical mt aims to also make them available in multiple languages. this is a rather challenging task, due to the scope of this domain, and the corresponding large and open vocabulary, including terms and non-lexical forms (for dates, biomedical entities, measures, etc). the quality of the resulting mt output thus varies depending on the amount of biomedical (in-domain) resources available for each target language. we participated in this years wmt'20 biomedical translation evaluation for english to french direction. english-french is a reasonably resourced language pair with respect to biomedical parallel corpora, allowing us to train our neural machine translation (nmt) @xcite with only in-domain corpora and dispense with the processing of large out-of-domain data that exist for this language pair. our main focus for this year's participation was to develop strong baselines by making the best of auxiliary resources: back translation of monolingual data; partial pre-translation of terms; pre-trained multilingual contextual embeddings and ir retrieved in domain corpora. two pre-prossessing pipelines, one using the standard moses tools foot_0 and subword-nmt @xcite and other using huggingface bert api were developed and compared. all systems are based on the transformer architecture @xcite , or and on the related bert-fused transformer model of @xcite . if our baselines were actually strong, we only managed to get relatively small gains from our auxiliary resources, for reasons that by and large remain to be analyzed in depth. our biomedical systems are presented in section 2. we also participated in the robustness translation task, developing a multi-domain, noise-robust and amenable to fast adaptation translation system for the translation direction english-german. our main focus was to study in more depth the adaptor architecture initially introduced in @xcite in a large-scale setting, where multiple heterogeneous corpora of unbalanced size are available for training, and explore ways to make the system robust to spelling noise in the test data. the zero-shot system is a generic system which does not use any adaptation layer; for our few-shot adaptation submission, we did not use the supplementary data provided by the organizers, which turned out to be only mildly relevant for the test condition, but resorted to a data selection strategy. in any case, our submissions are constrained and only use the parallel wmt data for this language pair; they are further described in section 3. ## conclusions in this paper, we have described the development undertaken for this year's participation to wmt shared tasks. taking part to the biomedical track as allowed us to collect and prepare useful resources (monolingual and bilingual corpora, term lists) for this domain, and to explore several pipelines and translation architectures. the general results are overall satisfactory, even though a deeper analysis of the mt is still needed to strengthen our conclusions. they will also help us prepare for next year tasks, where we expect to work on more language pairs. our experiment for the robustness track were less successful: we were not really prepared for the general tone and style that was observed in the zero-shot test set; we also did not understand the general orientation taken for the few-shot adaptation, as it seemed to us that the adaptation data was not really relevant for the only test set that was ever released.