Leveraging Domain Knowledge at Inference Time for LLM Translation: Retrieval versus Generation (2025)

Abstract: While large language models (LLMs) have been increasingly adopted for machine translation (MT), their performance for specialist domains such as medicine and law remains an open challenge. Prior work has shown that LLMs can be domain-adapted at test-time by retrieving targeted few-shot demonstrations or terminologies for inclusion in the prompt. Meanwhile, for general-purpose LLM MT, recent studies have found some success in generating similarly useful domain knowledge from an LLM itself, prior to translation. Our work studies domain-adapted MT with LLMs through a careful prompting setup, finding that demonstrations consistently outperform terminology, and retrieval consistently outperforms generation. We find that generating demonstrations with weaker models can close the gap with larger model’s zero-shot performance. Given the effectiveness of demonstrations, we perform detailed analyses to understand their value. We find that domain-specificity is particularly important, and that the popular multi-domain benchmark is testing adaptation to a particular writing style more so than to a specific domain.

## introduction large language models (llms) have emerged as the next major paradigm for machine translation (mt), with increasing use in both industrial and academic settings. these models are exciting not only for their strong base (or zero-shot) translation capabilities, but also for their ability to be modified at inference time through alternate prompts @xcite , in-context learning @xcite and the use of intermediate reasoning @xcite . this flexibility is particularly exciting for adapting llms to translate specialist domains, such as legal or medical texts. in the statistical and neural mt eras, domain adaptation would typically take the form of an expensive continued training procedure on in-domain data @xcite . with llms, there is the promise of simple adaptation at inference time. one promising technique is the retrieval of instance-specific demonstrations of translation from a bitext datastore for few-shot in-context learning, which has shown large improvements for domain-adapted mt @xcite , rivaling the performance of specialized nearest-neighbor mt systems @xcite . llms have also been shown to make good use of bilingual terminology dictionaries for lexical translation hints @xcite @xcite . intriguingly, two recent approaches have forgone external resources in favor of querying an llm to generate useful knowledge from its internal memory. first, the maps approach issues llm queries for topics, terminology, and demonstrations based on the source text @xcite . their terminology and demonstrations mirror the knowledge sourced from retrieval steps in earlier work. the idea is that the llm has seen relevant information during pre-training, and would benefit from explicitly surfacing it before translation. second, the stepby-step mt approach queries its llm to translate and discuss idiomatic phrases before performing a complete translation @xcite . however, both these works only consider the general domain. this inspires us to consider the applicability of internal memory approaches to domain adaptation, for which relevant external resources may be more difficult to obtain. in this work, we study the effectiveness of different representations of domain-specific knowledge, in strategies -external retrieval vs. internal generation -and sources -translation demonstrations and bilingual terminology. we consider three domains (law, medical, and koran) from the commonly-1 91 was sind die möglichen gefahren von ibuprofen? what are the possible dangers of ibuprofen? gibt es bedenken hinsichtlich der sicherheit von poulvac flufend h5n3 rg? what potential dangers are associated with poulvac flufend h5n3 rg? ist poulvac flufend h5n3 rg riskant? is poulvac flufend h5n3 rg risky? prompt llm for knowledge gen demos prompts demonstrations terminology translate the given german text into english, following the provided terminology. {source text} {terminology dictionary} translate the given german text into english, following the provided examples. {source text} {few shot mt demos} … prompt llm for translation with terminology knowledge prompt llm for translation with demonstration knowledge what risks are associated with poulvac flufend h5n3 rg? gen terms external global terminology ret terms used multi-domain dataset @xcite , and experiment with two llms (gemini 1.5 and gemma-2). our study addresses three main research questions: rq1. for improving domain-adapted mt, how viable is generation from an llm's parametric memory compared to retrieval from external resources? ## leveraging domain knowledge comparisons between representations of domain knowledge are enabled by our careful prompting setup which decouples the source and strategy, as sketched in figure 1 . bilingual terms, whether retrieved externally or generated by an auxiliary llm call, feed into the same translatewith-terms prompt, and likewise for demonstrations. on sources, retrieval leverages resources such as datastores and dictionaries, while generation elicits information from an llm's own parametric memory. on strategies, demonstrations provide source-target example pairs, whereas terminology focuses on domain-specific lexical items. this section details the integration of these strategies and knowledge sources within our experimental framework for domain adaptation of llm mt. ## experimental setup dataset we experiment with the multi-domain dataset @xcite , using the filtered version provided by @xcite , with 3 domains: law, medical, and koran. multi-domain covers the german-english (de-en) direction, and consists of dev and test sets, with ~2000 entries per domain, as well as a train set with 1m+ entries. ## 94 evaluation we perform zero-shot mt as a baseline, and employ the four settings described in §2 for comparison: retrieved demonstrations, retrieved terminologies, generated demonstrations, and generated terminologies. appendix b lists all prompts used in this work. following @xcite , we use a neural automated metric, comet @xcite . while prior work also considered the lexical metric bleu, we found that it was overly sensitive to minor rephrasing. this is in line with studies that show neural metrics correlate much better with human judgments of llm translation quality @xcite . ## results demonstrations outperform terminology for all models and domains studied, knowledge provided in the form of demonstrations consistently outperforms terminology. for gemma, we see that all settings improve performance, foot_2 but the improvements from demonstrations are markedly larger. the differential is more pronounced for gemini, which starts from a much stronger baseline than gemma. terms, either retrieved or generated, do not provide much of a boost over zero-shot for gemini, while demonstrations result in significant improvements. the takeaway for this finding is that for weaker models, providing domain knowledge from any source or strategy is beneficial. conversely, stronger models do not benefit from domain-specific terminology, but only from more complete demonstrations of the task. ## analysis demonstrations (both retrieved and generated) are by far the most effective domain adaptation strategy we explored, providing a large boost to both llms. in the following sections, we turn to analyses to understand better where the gains are coming from. we begin by analyzing retrieved demonstrations to disentangle contributions from style vs. terminology ( §5.1). then, we investigate the importance of various in-context learning decisions for generating demonstrations ( §5.2). finally, we study how generated domain knowledge can be distilled at test-time from larger to smaller models ( §5.3). ## discussion and conclusion we study the problem of domain adaptation for mt with llms, one which intuitively speaking, should be well addressed by prompting-time adaptation. building upon prior work which injects domainspecific knowledge into prompts, we perform a thorough study into how this knowledge can best be acquired in terms of strategy, demonstrations or terminologies, and sources, retrieval or generation. our main study shows that demonstrations outperform terminology, and knowledge retrieval consistently outperforms generation. furthermore, generation of domain-specific demonstrations can viably improve weaker model's performance, closing the gap with a larger model's zero-shot performance (though comparable to static exemplars). we gain additional insights with our further analyses. notably, we explore the connection between the strategies, characterizing demonstrations as providing both terminology hints and style hints. our 8 decomposition of the contributions of demonstrations finds that the majority of the gains (~65%) come from style over terminology. taken together, our work indicates that for the law, medical and koran domains of the commonlyused multi-domain scenario, large llms need very little terminology help, and the improvements from demonstrations are more so from matching corpus style than from better conveying domain-specific semantics. our work takes a first step in surfacing the domain-specific knowledge of smaller llms through generation, and we look forward to more informed approaches in future work. meanwhile for the largest llms, we recommend as the most promising direction to construct a new mt adaptation scenario that challenges even their broad base of parametric knowledge, perhaps with reference to pretraining cut-off dates.