Predicting Anchored Text from Translation Memories for Machine Translation Using Deep Learning Methods (2024)

Abstract: Translation memories (TMs) are the backbone for professional translation tools called computer-aided translation (CAT) tools. In order to perform a translation using a CAT tool, a translator uses the TM to gather translations similar to the desired segment to translate (s’). Many CAT tools offer a fuzzy-match algorithm to locate segments (s) in the TM that are close in distance to s’. After locating two similar segments, the CAT tool will present parallel segments (s, t) that contain one segment in the source language along with its translation in the target language. Additionally, CAT tools contain fuzzy-match repair (FMR) techniques that will automatically use the parallel segments from the TM to create new TM entries containing a modified version of the original with the idea in mind that it will be the translation of s’. Most FMR techniques use machine translation as a way of ‘repairing’ those words that have to be modified. In this article, we show that for a large part of those words which are anchored, we can use other techniques that are based on machine learning approaches such as Word2Vec. BERT, and even ChatGPT. Specifically, we show that for anchored words that follow the continuous bag-of-words (CBOW) paradigm, Word2Vec, BERT, and GPT-4 can be used to achieve similar and, for some cases, better results than neural machine translation for translating anchored words from French to English.

## introduction professional translators use computer-aided translation (cat) tools @xcite to translate text from one language called the source language (sl) to a target language (tl). most cat tools have an option known as fuzzy-match repair (fmr) @xcite @xcite @xcite , which is backed by a parallel translation memory (tm) that contains sentences (called segments) in the sl and tl. each pair, or unit, of parallel segments in the tm is known as a translation unit (tu). a tu contains a source segment (s) along with a target segment (t). when a professional translator attempts to translate a segment in the sl (denoted as s ′ ) a fuzzy-match lookup is performed using a word-based levenshtein dis-tance @xcite between s ′ and s where a 100% match means that the words from s ′ are identical to the words in s. it is often the case that a professional translator uses matches from fmr to only translate a few words (called sub-segments) from the entire segment. in this article, we focus on improving those cases where there exists only one word to translate, known as an anchored word, whose position is in between two words that are already captured. in our studies, the anchored word is a common case that professional translators often use. we experiment with four techniques to translate the anchored word: (1) neural machine translation, (2) a bert-based @xcite implementation, (3) word2vec @xcite and (4) openai gpt-4 prompting @xcite . the prediction of an anchored word has been presented in many contexts and can be considered the main objective of a language model. several models based on attention allow a weight to be assigned to certain words within a context window so that surrounding words that strongly influence the overall context can have a greater impact on the prediction made. this could potentially be used in order to improve predictions made for anchored text by taking longer contexts into account than the surrounding words. we discuss this approach in the context of generative models, where such systems could be harnessed to generate highly accurate predictions. the rest of this article is structured as follows. the next section discusses related work by accentuating the differences between fmr based on mt and anchored-word prediction. section 3 then presents the bert, word2vec, and gpt-4 approaches used for translating anchored words. in section 4, we describe the corpus and configurations used for our experiments whose results are reflected in section 5, followed by concluding remarks in section 6. ## related work for the majority of fmr approaches, mt is used to translate mismatches, regardless if they are anchored words or not. generally, mt techniques for fmr are focused on the decoding process where statistical-based systems @xcite @xcite @xcite or neural-based systems @xcite @xcite @xcite are used in such a manner to "repair" either the mt system or the mismatched sub-segments between s ′ and s. this article is focused on repairing the mismatched subsegments in specific situations where sub-segments of s are common in s ′ with the exception of one word (e.g. @xmath0. previous work @xcite can be considered identical to this article as it uses fmr to first find mismatches between s ′ and s and then translates the missing words with different mt systems. however, their system uses context around all mismatches where we only consider mismatches with anchored words, similar to @xcite . while other techniques @xcite are based on probabilistic mt models or employ different algorithms for aligning s ′ and s, we use a word-based edit distance @xcite that marks the mismatched sub-segments and discards non-anchored words. tezcan and bulté (2022) investigate a wide range of automatic quality estimation (qe) metrics in order to assess what effect integrating fuzzy matches can have on a number of aspects of translation quality, in addition to performing manual mt error analysis. they further evaluate what influence fuzzy matches have on a translation and how further quality improvements can be made by quantitative analyses that focus on the specific characteristics of a retrieved fuzzy match. neural fuzzy repair (nfr) outperforms baselines in all automated evaluation metrics. there was not a discernable difference between nfr and neural machine translation (nmt) error in manual evaluation, but different error profiles emerged in this study, highlighting some of the strengths and weaknesses of each method. namely, nfr produced more errors in the category of "semantically unrelated", whereas the baseline nmt system produced more errors in the categories of "word sense" and "multi-word expression". the nfr system made more accuracy errors, but producing fluent output was its strong suit. meanwhile, in terms of lexical choices, nmt produced more "non-existing/foreign" errors, which was not an issue for nfr. the baseline system performed better on grammar and syntax. our study differs in that it focuses specifically on anchored text and on leveraging the strengths of language models in next word prediction in order to fill in single-word gaps. @xcite attempt to improve cat via the tm using pre-computed word alignments between source and target tus in the tm. when a user is translating s ′ with a fuzzy match score greater than or equal to 60%, the proposed system marks the words that need to change as well as those that must remain the same in order to obtain t ′ . alignments are obtained from giza++ @xcite and take both a statistical and syntactic approach to detecting where changes need to occur. the experiments offer insight into how human decisions to keep/change text during translation can be integrated into fmr. our approach differs in that we specifically locate an-chored text and, following that, continue on to a prediction step, providing the content needed to perform fuzzy match repair in the translation step. i̇rsoy et al. ( 2020 ) compare performance of pretrained word embeddings in use in language models such as bert with continuous bag of words (cbow) embeddings trained with word2vec. the authors claim that, while bert embeddings are useful and effective, they often offer only marginal gains as compared to word2vec embeddings trained using gensim @xcite . the latter are much less computationally expensive to obtain; 768dimensional vectors were trained in one epoch in 1.61 days on a 16-cpu machine. cbow embeddings are trained by using surrounding context to predict a center word. while training via cbow has often shown inferior performance to training via skipgram (sg), this paper shows that with a proper implementation, performance of cbow embeddings can be on par with sg. our work puts the cbow prediction objective to good use, harnessing it to predict anchored text in source language segments. ## methodology neural mt systems have been shown by previous work @xcite to be the state-of-theart for fmr. in this article, we experiment on the one hand with word-based language models that are trained using context around a word, like those that use the continuous bag of words (cbow) model @xcite (word2vec) or masked language modeling @xcite (bert). on the other hand, it is our belief also that generative language modeling techniques may be a good candidate for accomplishing this task. to explore this avenue, we also compare output from these models with predictions obtained from prompting gpt-4 and find it to be competitive with the other methods. an example of a source sentence and the output from each method is provided in ## results in this section, we compare the results obtained from running four approaches for predicting the anchored word: (1) neural machine translation (nmt) (2) word2vec (3) bert and (4) gpt-4. nmt is divided into the two approaches mentioned in section 3.1 (sentence-level and tri-grams). accuracy measurements are performed and reported for all holes 8 . additionally, we report on character matches for each approach after dividing the segments into fuzzy-match thresholds, common practice for fmr work (see @xcite ## conclusion in this article, we have illustrated that via the use of a language model, predicting anchored words performed better in our experiments. the bert model outperforms other approaches including neural machine translation (with two approaches) when measured via character match and tri-gram anchored word coverage. we also demonstrate how generative models might be prompted to aid in predicting anchored text. it is our belief that this work could assist cat tools backed by tms and mt systems.