Compositional Translation: A Novel LLM -based Approach for Low-resource Machine Translation (2025)

Abstract: The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural language processing tasks. Machine Translation (MT) has been shown to benefit from in-context examples, in particular when they are semantically similar to the sentence to translate. In this paper, we propose a new LLM-based translation paradigm, compositional translation, to replace naive few-shot MT with similarity-based demonstrations. An LLM is used to decompose a sentence into simpler phrases, and then to translate each phrase with the help of retrieved demonstrations. Finally, the LLM is prompted to translate the initial sentence with the help of the self-generated phrase-translation pairs. Our intuition is that this approach should improve translation because these shorter phrases should be intrinsically easier to translate and easier to match with relevant examples. This is especially beneficial in low-resource scenarios, and more generally whenever the selection pool is small or out of domain. We show that compositional translation boosts LLM translation performance on a wide range of popular MT benchmarks, including FLORES200, NTREX 128 and TICO-19. Code and outputs will be made freely available.

## introduction large language models (llms) have demonstrated strong performance across a wide variety of tasks @xcite . they use @xcite to solve problems at inference with the help of a few examples within their context. multiple strategies have been introduced to expand the range of complexity of problems that can be addressed using icl, with a particular emphasis on reasoning tasks @xcite @xcite prompt "we now have 4-month old mice that used to be diabetic," he added. ## methodology we introduce compositional translation (comptra), an llm-based translation paradigm that automatically allows llms to translate any sentence by reasoning on self-generated translation pairs tailored to its content. comptra frames any translation problem as an explicit step-by-step procedure. it consists of three main stages. 1. decomposition. given that the use of related in-context examples helps few-shot mt into lrls, we want to derive pairs as closely related to the source sentence as possible. the aim of decomposition is to create simpler phrases that share words with the source sentence. we achieve this using a divide prompt, which contains examples that demonstrate the decomposition, followed by the source sentence. the examples are from the minwikisplit corpus @xcite , a set of sentences broken down into minimal propositions. it is worth noting that the number of phrases obtained is not a hyperparameter; each sentence is decomposed into the number of phrases that fits its structure. the only hyperparameter is the number of demonstrations per phrase k, which we set to 5 for all phrases. the hypothesis is that llms translate short sentences more effectively than longer ones, especially in languages they marginally encountered during their training. comptra's objective is to propagate this advantage of short entities to bigger ones via a three-step hierarchical approach. models. we use llama-3.1-it (8b, 70b; dubey et al., 2024), gemma-2-it (9b, 27b; gemma team et al., 2024), command-r and -r+ foot_0 (cohere, 2024). evaluation. we mainly evaluate using metricx-23 @xcite and xcomet @xcite in their reference-based versions: xcomet-xxl (which supports the same 100 languages as xlm roberta (conneau et al., 2020)) and metricx-23-xxl (which supports the same 101 languages as mt5 @xcite ). metricx scores range from 0 to 25, with higher scores indicating more translation errors. xcomet scores range from 0 and 1, which we rescale to 0 to 100 (higher scores are better). we also consider n-gram matching metrics via sacrebleu (post, 2018), namely chrf++ foot_1 (popović, 2017) and bleu foot_2 @xcite for transparency reasons. baselines. we compare to zero-shot and 5shot mt with example retrieval via similarity with bm25 @xcite and sonar @xcite . the former retrieves the five most similar examples based on its bm25 score relative to the query, whereas the latter uses the cosine similarity between the query and the candidates in the embedding space. ## conclusion we introduced a simple yet effective strategy, which we refer to as compositional translation to improve the mt capabilities of llms. through experiments on three mt benchmarks covering 15 different low-resource directions, we find that it outperforms the strong few-shot mt baseline with similarity search and several strong, existing strategies. it also enables smaller-scale llms to elicit better translation capabilities in in-domain and outof-domain scenarios. applying compositionality to perform mt will hopefully inspire further work on reasoning-based approaches to mt.