Facilitating Terminology Translation with Target Lemma Annotations (2021)

Abstract: Most of the recent work on terminology integration in machine translation has assumed that terminology translations are given already inflected in forms that are suitable for the target language sentence. In day-to-day work of professional translators, however, it is seldom the case as translators work with bilingual glossaries where terms are given in their dictionary forms; finding the right target language form is part of the translation process. We argue that the requirement for apriori specified target language forms is unrealistic and impedes the practical applicability of previous work. In this work, we propose to train machine translation systems using a source-side data augmentation method that annotates randomly selected source language words with their target language lemmas. We show that systems trained on such augmented data are readily usable for terminology integration in real-life translation scenarios. Our experiments on terminology translation into the morphologically complex Baltic and Uralic languages show an improvement of up to 7 BLEU points over baseline systems with no means for terminology integration and an average improvement of 4 BLEU points over the previous work. Results of the human evaluation indicate a 47.7% absolute improvement over the previous work in term translation accuracy when translating into Latvian.

## introduction translation into morphologically complex languages involves 1) making a lexical choice for a word in the target language and 2) finding its morphological form that is suitable for the morphosyntactic context of the target sentence. most of the recent work on terminology translation, however, 1 relevant materials and code: https://github. com/tilde-nlp/terminology_translation has assumed that the correct morphological forms are apriori known @xcite @xcite @xcite @xcite . thus previous work has approached terminology translation predominantly as a problem of making sure that the decoder's output contains lexically and morphologically prespecified target language terms. while useful in some cases and some languages, such approaches come short of addressing terminology translation into morphologically complex languages where each word can have many morphological surface forms. for terminology translation to be viable for translation into morphologically complex languages, terminology constraints have to be soft. that is, terminology translation has to account for various natural language phenomena, which cause words to have more than one manifestation of their root morphemes. multiple root morphemes complicate the application of hard constraint methods, such as constrained-decoding @xcite . that is because even after the terminology constraint is striped from the morphemes that encode all grammatical information, the remaining root morphemes still can be too restrictive to be used as hard constraints because, for many words, there can be more than one root morpheme possible. an illustrative example is the consonant mutation in the latvian noun vācietis ("the german") which undergoes the mutation @xmath0, thus yielding two variants of its root morpheme vāciešand vāciet-(bergmanis, 2020). if either of the forms is used as a hard constraint for constrained decoding, the other one is excluded from appearing in the sentence's translation. we propose a necessary modification for the method introduced by @xcite , which allows training neural machine translation (nmt) en src.: faulty engine or in transmission[..] lv trg.: atteice dzinējā vai transmisijas [..] eta: faulty|w engine|s dzinējā|t or|w transmission|s transmisijas|t [..] tla: faulty|w engine|s dzinējs|t or|w transmission|s transmisija|t [..] our results show that the proposed approach not only relaxes the requirement for apriori specified target language forms but also yields substantial improvements over the previous work @xcite when tested on the morphologically complex baltic and uralic languages. ## method: target lemma annotations to train nmt systems that allow applying terminology constraints @xcite prepare training data by amending source language terms with their exact target annotations (eta). to inform the nmt model about the nature of each token (i.e., whether it is a source language term, its target language translation or a regular source language word), the authors use an additional input streamsource-side factors @xcite . their method, however, is limited to cases in which the provided annotation matches the required target form and can be copied verbatim, thus performing poorly in cases where the surface forms of terms in the target language differ from those used to annotate source language sentences @xcite . this constitutes a problem for the method's practical applicability in real-life scenarios. in this work, we propose two changes to the approach of @xcite . first, when preparing training data, instead of using terms found in either iate foot_0 or wiktionary as done by @xcite , we annotate random source language words. this relaxes the requirement for curated bilingual dictionaries for training data preparation. second, rather than providing exactly those target language forms that are used in the target sentence, we use target lemma annotations (tla) instead (see our work is similar to work by @xcite in which authors also aim to achieve copy-andinflect behaviour. however, authors limit their annotations to only those terms for which their base forms differ by no more than two characters from the forms required in the target language sentence. thus wordforms undergoing longer affix change or inflections accompanied by such linguistic phenomena as consonant mutation, consonant gradation or other stem change are never included in training data. ## experimental setup languages and data. as our focus is on morphologically complex languages, in our experiments we translate from english into latvian and lithuanian (baltic branch of the indo-european language family) as well as estonian (finnic branch of the uralic language family). for comparability with the previous work, we also use english-german (germanic branch of the indo-european language family). for all language pairs, we use all data that is available in the tilde data libarary with an exception for english-estonian for which we use data from wmt 2018. the size of the parallel corpora after pre-processing using the tilde mt platform @xcite and filtering tools @xcite is given in to prepare data with tla, we first lemmatise and part-of-speech (pos) tag the target language side of parallel corpora. for lemmatisation and pos tagging, we use pre-trained stanza foot_1 (qi et al., 2020) models. we then use fast align foot_2 @xcite to learn word alignments between the target language lemmas and source language inflected words. we only annotate verbs or nouns. to generate sentences with varying proportions of annotated and unannotated words, we first generate a sentence level annotation threshold uniformly at random from the interval [0.6, 1.0). similarly, for each word in the source language sentence, we generate another number uniformly at random from the interval [0.0, 1.0). if the latter is larger than the sentence level annotation threshold, we annotate the respective word with its target language lemma. we use the original training data and annotated data with a proportion of 1:1. we follow @xcite to prepare eta and replicate their results. for validation during training, we use development sets from the wmt news translation shared tasks. for en-et and en-de, we used the data from wmt 2018, for en-lv -wmt 2017, and for en-lt -wmt 2019. mt model and training. for the most part, we use the default configuration of the transformer @xcite nmt model implementation of the sockeye nmt toolkit @xcite . the exception is the use of source-side factors (sen-nrich and haddow, 2016) with the dimensionality of 8 for systems using inline target lemma annotations. we train all models using early stopping with the patience of 10 based on their development set perplexity @xcite . evaluation methods and data. in previous work, methods were tested on general domain data foot_3 annotated with exact surface forms of generaldomain words from iate and wiktionary. although data constructed in such a way is not only artificial but also gives an oversimplified view on terminology translation, we do use the data from iate to validate our re-implementation of the method from @xcite . other than that, we test on the automotive test suite foot_4 (ats): a data set containing translations of the same 768 sentences in english, estonian, german, latvian, and lithuanian. ats contains about 1.1k term occurrences from a glossary prepared by professional translators. when annotating terms in the source text, we use only the dictionary forms of term translations, since in practical applications having access to the correct inflections (surface forms) is unrealistic. we compare our work with an nmt system without means for terminology integration (baseline) and the previous work by @xcite (eta). although our preliminary experiments with constrained decoding @xcite ) (cd) confirmed the findings by @xcite that strict enforcement of constraints leads to lower-thanbaseline quality, we nevertheless include them for completeness sake. similarly to the previous work, we use two auto we are aware that the automatic evaluation methods are merely an approximation of translation quality. for example, we use lemmatised term exact match accuracy to measure term use in target language translations; however, it does not capture whether the term is inflected correctly. thus human evaluation is in place. we use the en-lv language pair to compare tla against baseline and eta. we use a 100 sentences large randomly selected ats subset that contains 147 terms of the original test suite. we employ four professional translators and latvian native speakers to compare each system's translations according to their overall translation quality and judge individual term translation quality. specifically, given the original sentence and its two translations (in a randomised order), raters are asked to answer "which system's translation is better overall?". raters are also given a list of the terms being evaluated and their reference translations (from the term collection) and are asked to classify translations as either "correct", "wrong lexeme", "wrong inflection", or "other". figure 1 gives an example of the forms presented to raters during the human evaluation of term and overall translation quality. we report inter-annotator agreement using free marginal kappa, κ free @xcite . ## conclusions we proposed tla-a flexible and easy-toimplement method for terminology integration in nmt. using tla does not require access to bilingual terminology resources at system training time as it annotates ordinary words with lemmas of their target language translations. this simplifies data preparation greatly and also relaxes the requirement for apriori specified target language forms during the translation, making our method practically viable for terminology translation in reallife scenarios. results from experiments on three morphologically complex languages demonstrated substantial and systematic improvements over the baseline nmt systems without means for terminology integration and the previous work both in terms of automatic and human evaluation judging term and overall translation quality.