Exploring In-context Example Generation for Machine Translation (2025)

Abstract: Large language models (LLMs) have demonstrated strong performance across various tasks, leveraging their exceptional in-context learning ability with only a few examples.Accordingly, the selection of optimal in-context examples has been actively studied in the field of machine translation.However, these studies presuppose the presence of a demonstration pool with human-annotated pairs, making them less applicable to low-resource languages where such an assumption is challenging to meet.To overcome this limitation, this paper explores the research direction of in-context example generation for machine translation.Specifically, we propose Demonstration Augmentation for Translation (DAT), a simple yet effective approach that generates example pairs without relying on any external resources.This method builds upon two prior criteria, relevance and diversity, which have been highlighted in previous work as key factors for in-context example selection.Through experiments and analysis on low-resource languages where human-annotated pairs are scarce, we show that DAT achieves superior translation quality compared to the baselines.Furthermore, we investigate the potential of progressively accumulating generated pairs during test time to build and reuse a demonstration pool. Our implementation is publicly available at https://github.com/aiclaudev/DAT.

## introduction the recent emergence of large language models (llms) @xcite and in-context learning (icl) @xcite has shifted the traditional paradigm of building task-specific models trained on large amounts of human-annotated data, which is costly to collect. the strength of icl lies in its versatility, where they achieve outstanding performance across various tasks with just a few task-specific demonstrations @xcite . this reduces the training cost on large datasets and allows rapid adaptation to new domains or problems without extensive fine-tuning. solving various tasks with a single llm provides immense value to users seeking assistance in diverse contexts. llms have also played an increasingly prominent role in the field of machine translation (mt) due to their exceptional linguistic and reasoning capabilities @xcite @xcite @xcite . notably, icl has demonstrated high multilingual translation quality using only a few source-target pairs, driving advancements in retrieving optimal examples. @xcite proposed r-bm25, which initially selects the top bm25 candidates and reranks them using an n-gram recall strategy. @xcite introduced a neural network trained to select pairs based on the multiple features, including semantic similarity and sentence length. these works have demonstrated outstanding performance in high-resource languages, retrieving pairs based on the scores between the given user query and source sentences in the demonstration pool, as shown in figure 1 (a) . however, these works on in-context example selection for mt may face challenges in low-resource languages. this stems from the fact that previous approaches rely on a critical assumption-namely, the availability of a large pool of human-annotated pairs-which may not hold for low-resource languages. for low-resource languages, obtaining a paired corpus for use as demonstrations is challenging due to the limited availability of public datasets and human annotators. this, in turn, poses a barrier that prevents low-resource languages from fully benefiting from the use of in-context examples, as illustrated in figure 1 @xcite . recently, el mekki and abdul-mageed (2025) investigated leveraging llms to generate synthetic parallel data as a way to address this obstacle. however, this approach requires access to the vocabularies of both the source and target languages, as well as unlabeled sentences in the target language. in this paper, we explore a research direction that aims to enable in-context learning for mt without the use of any external resources, instead drawing solely on the capabilities of the llm itself. in pursuit of this goal, we introduce a simple yet effective method, demonstration augmentation for translation (dat), which utilizes the generative and linguistic capabilities of llms. this approach builds upon the intuitive prior criteria of relevance and diversity, which are inspired by previous works analyzing desirable in-context examples for mt @xcite @xcite . to ensure these two criteria, we also utilize maximal marginal relevance @xcite . our experiment focuses on translating from english into low-resource languages-specifically nepali, khmer, pashto, zulu, and swahili-for which the lack of extensive annotated datasets presents a realistic constraint. the results demonstrate the practicability of our easily applicable method in generating pairs that serve as in-context examples, providing valuable clues for user query translation. one more noteworthy point is that we observe a counterintuitive case where utilizing high-quality fixed pairs results in a severe performance degradation compared to the zero-shot approach. we investigate this phenomenon with a focus on the relevance between the source side of the pairs and the user queries. lastly, we explore an extended method that incrementally accumulates the generated pairs and repurposes them through retrieval method such as r-bm25. in summary, our contributions are as follows: ## method overview. we aims to generate pairs for incontext learning in the absence of human-annotated pairs. at test time, when a user provides a query q, llm generates source-target pairs tailored to q and uses them as in-context examples. the overall flow of this method is shown in figure 2 . ## discussions hybrid approach we employed an llm to generate the target-side of in-context examples. while a model fine-tuned for a specific language pair-such as a traditional neural machine translation system-could produce higher-fidelity pairs, our focus is on improving translation quality purely through the inherent capabilities of the llm. we therefore leave the exploration of such approaches as a promising direction for future work. reuse as a training dataset in section 5.4, we explored a setting where the generated pairs are accumulated, allowing for retrieval and reuse. if the demonstration pool becomes large enough, it can serve as a training dataset for developing a translation-specialized model. therefore, leveraging llms to generate translation pairs in lowresource scenarios is a crucial research direction, both for in-context learning and fine-tuning. ## conclusions in this paper, we explore an interesting research direction that leverages only llms to generate source-target pairs. accordingly, we propose a simple yet effective method, dat, which utilizes two priors and mmr to generate in-context examples. experiments proved that dat achieves superior translation quality for low-resource languages compared to baselines, without relying on any humancreated resources. this highlights the potential of our method in scenarios where a demonstration pool is unavailable or fails to provide relevant translation examples. furthermore, we investigated cases where fixed human pairs significantly underperform compared to zero-shot translation and explored the potential of an accumulation setting as a cost-efficient alternative.