The SPECTRANS System Description for the WMT 21 Terminology Task (2021)

Abstract: This paper discusses the WMT 2021 terminology shared task from a “meta” perspective. We present the results of our experiments using the terminology dataset and the OpenNMT (Klein et al., 2017) and JoeyNMT (Kreutzer et al., 2019) toolkits for the language direction English to French. Our experiment 1 compares the predictions of the two toolkits. Experiment 2 uses OpenNMT to fine-tune the model. We report our results for the task with the evaluation script but mostly discuss the linguistic properties of the terminology dataset provided for the task. We provide evidence of the importance of text genres across scores, having replicated the evaluation scripts.

## introduction in our (traditional) sense, terminological databases are the collection of specialised lexical resources that are generally compiled from corpora, in collaboration with experts from the field, then analysed and structured according to the type of information recorded in term records: terms, equivalents, definitions, synonyms, contexts of use, and related terms @xcite . the data thus created are empirical and provide knowledge-based representations of the domain (especially in the case of an ontological approach), underlining conceptual links between terms that can be observed (like meronomy: "x is a part of y") and potentially represented in conceptual graphs. for instance, the artes database @xcite , used at université de paris in masters studies for teaching terminology management to translation students @xcite , adopts such a comprehensive approach to terminology, with specific attention to emerging terminology and complex noun phrases (cnps) @xcite . in recent works combining studies on terminology, specialised translation and corpus linguistics, attention has been drawn to cnps in english which have been demonstrated to cause major difficulties during translation, both human and machine @xcite . moreover, studies have demonstrated an increase of complex compounding in specialised texts in english over the last few decades, with, for instance, an overwhelming use of patterns with adjectival and participial compound pre-modifiers (e.g. receptor-binding activity, electron-dense aggregates) @xcite . for this wmt21 terminology workshop, we focused on the linguistic properties of the terminological dataset provided. we selected what we believe to be the two best models we produced for the en-fr track with two different neural toolkits but we mostly took the opportunity to discuss the addition of terminology to neural machine translation. the rest of the paper is organised as follows: section 2 summarises our approaches to the task, section 3 presents the tools we used and how we used the constrained data. section 4 presents our experiments and the best models we used for the translation challenge. section 5 discusses our results. ## our approaches to the task this section presents our various strategies for the terminology task. ## data and tools used this section presents the datasets used to build our system as well as our replication of the evaluation script to analyse the models we did not have the time to submit. ## conclusion this paper presents the spectrans system description for the wmt21 terminology shared task. we participated in the english-to-french task, using the terminology resources at training time. though english-french is a language-pair with many linguistic resources, we only used the data provided by the organisers. given the novel evaluation of terminology provided for this task, we not only aimed to build a translation system for the competition, but also to provide a critical angle on the task and on its evaluation. for the mt system, we applied a variety of strategies, toolkit comparison, data augmentation and fine-tuning. though we did not experience catastrophic forgetting, our fine-tuning did less well in the terminology metrics, probably because we were not able to update the dictionary. we obtained the best scores for the models we submitted with a model trained with common crawl supplemented with the terminology resource. the translations produced with pretrained models competed in terms of terminology scores, did better for sacrebleu, especially for the translation of pubmed, but proved less robust for the translation of the patient-doctor interactions of the cmu section of the testing data. for the analysis of the terminology, we discussed the role of complex noun phrases and initialisms. our contribution mostly lies in the critical analysis of the terminological input and of the evaluation script. this allowed us to raise the issue of the role of acronyms in the terminology, the importance of complex nps (and the correlative interest of the window overlap accuracy with @xmath0, more likely to capture complex nps than window overlap accuracy when @xmath1.