LISN @ WMT 2021 (2021)

Abstract: This paper describes LISN’s submissions to two shared tasks at WMT’21. For the biomedical translation task, we have developed resource-heavy systems for the English-French language pair, using both out-of-domain and in-domain corpora. The target genre for this task (scientific abstracts) corresponds to texts that often have a standardized structure. Our systems attempt to take this structure into account using a hierarchical system of sentence-level tags. Translation systems were also prepared for the News task for the French-German language pair. The challenge was to perform unsupervised adaptation to the target domain (financial news). For this, we explored the potential of retrieval-based strategies, where sentences that are similar to test instances are used to prime the decoder.

## introduction this paper describes lisn's foot_0 submissions to the translation shared tasks at wmt'21, where we took part in two shared tasks. for the biomedical translation tasks, we have developed resource-heavy systems for the english-french language pair, using a diversity of out-of-domain and in-domain corpora, thus continuing the efforts reported in @xcite . like for previous years shared task, the target genre (scientific abstract) corresponds to texts that often have a standardized structure comprising typical subsections of one to five lines. standard subsections report the objective, the method, or the results of the study. our systems for this year attempt to take this structure into account using sentence-level tags, with the hope to capture some of the document structure and the phraseology of the domain into account. these systems are documented in section 2. translation systems were also prepared for the news task for the french-german language pair. the challenge this year was to perform unsupervised adaptation to the target domain (financial news), with no further detail regarding the test data. in particular, the organizers did not release any development data to tune systems. in this setting, we explored the potential of using a retrieval-based strategy, where sentences that are similar to the test instances are used to help the decoding. in this approach, introduced in @xcite and further explored in @xcite , translation is a two-step process: a retrieval phase, which identifies sentences that resemble the source test sentence in parallel corpora. these sentences and their translation are then used to prime the decoder: inserting relevant translations examples in the decoder's context should help to select the right translations, especially for words and terms from the test domain. these systems are described in section 3. ## mt for biomedical texts in this section, we describe our participation to the biomedical task for wmt'21, in which we participated in both english to french and french to english directions. english-french is a reasonably resourced language pair with respect to biomedical parallel corpora, allowing us to train our neural machine translation (nmt) systems @xcite with in-domain corpora as well as large outof-domain data that exists for this language pair. like for last year @xcite , our first goal is to make the best of all the available data, including supplementary in-domain monolingual data. our corpora are described in section 2.1. for this year's participation, we also attempt to take the internal structure of biomedical abstracts into account. many of these abstracts follow what is often refered to as the "imrad format", comprising the following subparts: introduction, out-of-domain 1139 1292 35m for this purpose, we identified in our data all the abstracts that were conforming to this basic structure and worked to make this structure as explicit and standardized as possible. this notably implied to normalize the mains headings, as some variation was observed: for instance, analysis may be replaced with discussion, and additional subparts (objectives, conclusion) are also be observed. to incorporate the standard imrad format we mapped each subheading to the corresponding imrad subpart using a system of tags. details regarding this process are given in section 2.2. all systems are based on the transformer architecture of @xcite . we were able to achieve appreciable gains both from backtranslation and document structure processing. the results are discussed in section 2.4. ## news translation task: de ↔ fr in the 2021 news translation task, we focused on the german-french language pair in which the participants are asked to build mt systems for news in the financial domain. in this section, we discuss details of our approach and the rationale behind it. ## conclusion and outlook in this paper, we have described the systems prepared for this year's participation to wmt shared tasks. for the biomedical track, most of our efforts have been invested in the development of high resource systems, trying to take the structure of medical abstracts into account. in the news task, we have explored ways to perform unsupervised domain adaptation using retrieval based techniques and back-translated data. genci. the first author is funded through a regional grant from the "région ile de france".