T op XG en: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation (2025)

Abstract: LLMs have been shown to perform well in machine translation (MT) with the use of in-context learning, rivalling supervised models when translating into high-resource languages (HRLs). However, they lag behind when dealing with low-resource language (LRLs). Example selection via similarity search and supervised fine-tuning help. However the improvements they give are limited by the size, quality and diversity of existing parallel datasets. A common technique in low-resource MT is synthetic parallel data creation, the most frequent of which is backtranslation, whereby existing target-side texts are automatically translated into the source language. However, it also relies on the existence of good quality and relevant target-side texts, which are not readily available for many LRLs. In this paper, we present a new approach, TopXGen, which involves using an LLM to automatically generate topic-specific target-side data in the LRL, which can then be backtranslated to produce useful and diverse parallel texts for ICL and fine-tuning. Our intuition is that while LLMs struggle to translate into LRLs, their ability to translate well into HRLs and their multilinguality enable them to generate good quality, natural-sounding target-side texts, which can be translated well into a high-resource source language. We show that TopXGen boosts LLM translation performance during fine-tuning and in-context learning. Our code and outputs will be made freely available.

## introduction the performance of machine translation (mt) models has considerably evolved through the years, with new models increasingly supporting more languages @xcite @xcite . however, performance remains unequal across languages, which themselves vary 1. cross egypt challenge 2. ron williams 3. hafed al-ghwell 4. b. e. devaraj 5. ptolemy xii auletes 6. air japan 7. clock tower beach … die grundlegende verwaltungseinheit in polen wird gemeinde (gmina) genannt. eine stadt ist ebenfalls eine gemeindeallerdings besitzt sie ein stadtrecht. sowohl städte als auch gemeinden werden von einem bürgermeister regiert -dieser wird in den gemeinden vogt (auf polnisch wójt) und in den … la creación de la primera bolsa de valores de varsovia se produjo en el año 1817 y su actividad continuó hasta la segunda guerra mundial. en abril de 1991, tras el fin del control comunista durante la posguerra y la reapertura a una economía de libre mercado, la bolsa abría de nuevo sus puertas. según muchos … the panthers defense gave up just 308 points, ranking sixth in the league, while also leading the nfl in interceptions with 24 and boasting four pro bowl selections. pro bowl defensive tackle kawann short led the team in sacks with 11, while also forcing three fumbles and recovering two … bango da rufi na kogunan kankara na … yau dan wasan da ya fi nuna ƙwazo … you are an helpful and precise polyglot assistant which is highly proficient in multiple languages including hausa. here are a few examples of paragraphs written in different languages that you can create. <@xmath0, informative and insightful texts in hausa within the context of the following topic(s): ptolemy xii auletes. aim to inform, engage and illustrate. moreover, -include relevant words/concepts -write with care (grammatical accuracy) -do not translate the texts provided as examples. here are high-quality sentences written in hausa: <@xmath1. ya hau kujerar mulki tare da 'yar uwarsa, cleopatra vii … 2. ptolemy, mai himma sosai, ya yi aiki don kafa ƙudurin ruwan … 3. … 1. ptolemy xiv philopator, ɗan sarkin ptolemy xii auletes na masar, wani shahararren mutum ne a tarihinta … 2. ya hau kujerar mulki tare da 'yar uwarsa, cleopatra vii … 3. mutuwar ptolemy xiv ta haifar da wani sabon mataki a tarihi na masar. … 4. ptolemy, very diligent, worked greatly in terms of available resources and representation in nlp research @xcite . mt models perform much better in high-resource languages (hrls), such as english, french and german, compared to low-resource languages (lrls) such as hausa, which lack large quantities of highquality parallel data. decoder-based llms can perform mt without relying on parallel data (in a zero-shot fashion). however they lag behind supervised mt baselines when translating into lrls @xcite . in-context learning (icl; @xcite , which involves using a few high-quality demonstrations, has been shown to improve performance, especially when they are similar to the sentence to be translated @xcite , highlighting the importance of parallel datasets even in this setting. a common approach is to synthesize parallel data using forward translation or back-translation @xcite @xcite . forward translation comes with issues such as low translation fidelity and the neglect of cultural nuances in the target language. back-translation (bt; @xcite typically relies on a good quality monolingual corpus in the lrl, which can be difficult to obtain. in this work, we explore the construction of synthetic datasets for mt into lrls by automatically generating text in the lrl target language and then backtranslating into a hrl source language. @xcite first proposed an approach with a similar aim, using an llm to generate in-domain monolingual data, which they then used to perform bt alongside parallel data. they generate their monolingual data using an llm fine-tuned on each of their domain-specific datasets. however, their approach focuses on english and requires domainspecific fine-tuning (and datasets) in addition to relying on parallel data to perform bt, making it impractical for translation into lrls. to address this, we introduce topxgen (figure 1 ), a pipeline that exploits llms' multilinguality and instructionfollowing capabilities. unlike prior work, we generate monolingual data (sentences) beyond english to cover numerous lrls. rather than domainspecific fine-tuning, we directly prompt an llm and guide its generations towards a predefined list of topics in order to encourage diversity in the outputs. the quality of sentences stems from the ability of state-of-the-art multilingual llms to produce coherent text in lrls (enis and hopkins, 2024)even if they struggle to translate accurately into them. we then backtranslate the generated sentences into an hrl (english in this work) using a reverse translation model (a supervised mt system or the same llm). this target-aware generation helps mitigate the cultural loss often observed in lrls with standard forward translation techniques. moreover, translating into a hrl offers a practical advantage, as hrls are generally easier to translate into with high fidelity. to evaluate topxgen, we generate synthetic parallel data between english and ten low-resource languages-basque, hausa, igbo, kinyarwanda, nepali, somali, sundanese, swahili, urdu, and xhosa-using gemma-3-27b-it as the generator and nllb-200-3.3b as the back-translator. we then assess both in-context learning and finetuning setups across small translation models. our experiments show that topxgen consistently outperforms other data generation methods and achieves performance comparable to humantranslated datasets. ## methodology we propose topxgen (figure 1 ), a topic-guided and target-language centric method for automatically generating parallel sentence-level datasets between english and lrls, with the end goal of im-proving mt into lrls via icl or supervised finetuning using the synthetic examples. it consists of two steps: data generation and back-translation. ## ablation studies impact of the generator and the number of topics. in this section, we focus on sundanese and study 2 setups. first, we use gpt-4o-mini-2024-07-18 (openai, 2024) as the generator llm and analyze the performance of small models (llama-2-7b and llama-3-8b) finetuned on 55k sentences. second, we reduce the number of topics from 67,573 to a curated list of 509 elements and analyze how the fine-tuned models perform for both generator(s). as shown in impact of the back-translator. throughout the paper, we used nllb-200-3.3b as the backtranslator. in this section, we study two setups. first, we analyze how the results change when the generator is also used as the back-translator. back-translation is performed with greedy 5-shot with bm25 retrieval @xcite from the flores-200 dev set following @xcite . we fine-tune separate models on hausa and sundanese, and compare their results with using nllb-200-3.3b as the backtranslator in ## conclusion we presented topxgen, a scalable pipeline for constructing synthetic parallel datasets for mt into lrls. our method generates diverse monolingual data directly in a wide range of lrls using topicguided prompting @xcite ; aycock and 16 17 18 bleu( ) english sundanese 14 15 16 17 18 19 20 21 english hausa 1 5 10 15 20 25 5 4.5 4.4 4.3 4.2 metricx24( ) 1 5 10 15 20 25 8 7 6.5 6 5 @xmath2.0 @xmath3.5 @xmath4.0 @xmath5.2 @xmath6.0 @xmath7.5 @xmath8.0 @xmath9.2 bawden, 2024), which we then back-translate into a hrl, building synthetic parallel datasets. our experiments demonstrate that models trained on data generated via topxgen achieve strong performance across both unidirectional and multidirectional scenarios, approaching the generator's performance while requiring only small data volumes and limited gpu resources. furthermore, existing generation strategies such as self-instruct and knn-instruct are compatible with our pipeline but fall short in terms of performance. topxgen enables the creation of synthetic datasets that are comparable to professionally written text for icl and fine-tuning, offering a practical and effective solution for low-resource multilingual mt.