Augmenting Large Language Model Translators via Translation Memories (2023)

Abstract: Using translation memories (TMs) as prompts is a promising approach to in-context learning of machine translation models. In this work, we take a step towards prompting large language models (LLMs) with TMs and making them better translators. We find that the ability of LLMs to “understand” prompts is indeed helpful for making better use of TMs. Experiments show that the results of a pre-trained LLM translator can be greatly improved by using high-quality TM-based prompts. These results are even comparable to those of the state-of-the-art NMT systems which have access to large-scale in-domain bilingual data and are well tuned on the downstream tasks.

## introduction marrying the world of translation memory (tm) and the world of neural machine translation (nmt) is a challenging but interesting problem in natural language processing (nlp). previous work along this line of research either requires architecture changes of nmt models and/or additional training @xcite @xcite @xcite or constructing translation knowledge base from tm @xcite @xcite . more recently, researchers have been aware of the strength of prompting techniques for oneshot/few-shot machine translation @xcite @xcite . in particular, @xcite investigated oneshot learning methods for nmt by simply viewing tms as prompts. the result of their work is a stronger nmt system that works in the same way as usual but can be prompted when tms are available. interestingly, they found that the ability of nmt models to "understand" prompts plays an method w/o-arch-change w/o-base few-shot zhang et al. (2018) yes he et al. (2021) yes khandelwal et al. (2021) yes reheman et al. (2023) yes yes one-shot tmplm (our) yes yes yes in this work, we take a step forward. we treat large language models (llms) as machine translation systems and prompt them with tms (see we experiment with our method on a gpt-based llm (text-davinci-003 * ). on translation tasks ranging over multiple languages and domains, tm-based prompting improves the llm-based translation system by 20 to 30 bleu points, showing better performance than a well-tuned, largescale, in-domain nmt system on most of the tasks. we also compare different kinds of prompt templates and discuss some interesting issues, such as the role of prompting in treating llms as translators. f (•) : what is the translation of " " from src-lang to tgt-lang? only translation results are required. " from src-lang to tgt-lang is " ", then what is the translation of " " from src-lang to tgt-lang? only translation results are required. ## prompting methods tm is a database that contains the bilingual translation history of professional translators. it is usually used to help the translation of the test sentence by providing similar sentence pairs, which may have translation hints, such as similar sentence patterns, phrases, lexicons, terminologies, or other translation knowledge. either an nmt model or an llm need to dig out those hints and ignore the irrelevant content. this motivates us to investigate prompting llms with tms benefiting from their dazzling ability of "understand" prompts. suppose we have a tm database that retains a collection of pairs of sentences. given a sourcelanguage sentence x, the database returns k most similar sentences x @xmath3, ..., x k tm } along with their corresponding translations y @xmath4, ..., y k tm }. now suppose we have a pretrained translation model (either an nmt model or an llm) that takes x in some way and outputs a translation y, written as we then wish to use this model to generate a new translation y ′ by considering (x tm , y tm ) as instances for reference. this can be written as in this work, we focus on the case in which a powerful generative llm (such as chatgpt) is used to perform translation. the input of trans(•) could be an instruction or question-like text, and so we can design f ref (•) in many different ways. in figure 1 , we present two types of templates: the instruction-style template and the code-style template. these designs come from a consideration of the human instruction tuning and the code training used in developing davinci-003. for a more extensive discussion of template design, see appendix b.2. it is worth emphasizing that, while we restrict ourselves to tm-based prompts in experiments, we can apply this general approach to deal with other knowledge about translation. as a simple example, we can extend (x tm , y tm ) to term or phrase translations. also, when some mt systems are available, we can make use of automatic translations from other systems to define prompts. ## conclusion we have proposed tmplm, an in-context learning method to prompt tms for llms. by incorporat-ing tms into tailored templates, llms with tm-plm outperforms the state-of-the-art nmt models with tm prompting. we have also demonstrated that the ability of language understanding plays an important role in prompting llms with tms.