Mufu:  Multilingual Fused Learning for Low-Resource Translation with LLM
Authors: Zheng Wei Lim
Nitish Gupta
Honglin Yu
Trevor Cohn
Venue: ICLR 2025 Poster (2025)
Keywords: translation
low-resource
large language model
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We finetune LLMs for translation by prompting with multilingual context in order to harness their cross-lingual reasoning, and substantially improve the models' performance in low-resource translation.

Abstract: Multilingual large language models (LLMs) are great translators, but this is largely limited to high-resource languages. For many LLMs, translating in and out of low-resource languages remains a challenging task. To maximize data efficiency in this low-resource setting, we introduce Mufu, which includes a selection of automatically generated multilingual candidates and an instruction to correct inaccurate translations in the prompt. Mufu prompts turn a translation task into a postediting one, and seek to harness the LLM’s reasoning capability with auxiliary translation candidates, from which the model is required to assess the input quality, align the semantics cross-lingually, copy from relevant inputs and override instances that are incorrect. Our experiments on En-XX translations over the Flores-200 dataset show LLMs finetuned against Mufu-style prompts are robust to poor quality auxiliary translation candidates, achieving performance superior to NLLB 1.3B distilled model in 64% of low- and very-low-resource language pairs. We then distill these models to reduce inference cost, while maintaining on average 3.1 chrF improvement over finetune-only baseline in low-resource translations.

Published as a conference paper at ICLR 2025
MUFU: MULTILINGUAL FUSED LEARNING FOR Low- RESOURCE TRANSLATION WITH LLM
Zheng Wei Lim Ve Nitish Gupta Honglin Yu Trevor Cohn 3.0
The University of Melbourne Google
ABSTRACT
Multilingual large language models (LL.Ms) are great translators, but this largely limited to high-resource languages. For many LLMs, translating in and out of low- resource languages remains challenging task. To maximize data efficiency in this low-resource setting, we introduce Mufu, which includes selection of automat- ically generated multilingual candidates and an instruction to correct inaccurate translations in the prompt. Mufu prompts turn translation task into a postediting one, and seek to harness the LLM'ss reasoning capability with auxiliary translation candidates, from which the model is required to assess the input quality, align the semantics cross-lingully,, copy from relevant inputs and override instances that are incorrect. Our experiments on En-XX translations over the Flores-200 dataset show LLMs finetuned against Mufu-style prompts are robust to poor quality auxiliary translation candidates, achieving performance superior to NLLB 1.3B distilled model in 64% of low- and very-low-resoure language pairs. We then distill these models to reduce inference cost, while maintaining on average 3.1 chrF improvement over finetune-only baseline in low-resource translations.
INTRODUCTION
The most advanced of large language models (LLM) have demonstrated remarkable competence
in translation-related tasks (Robinson et al. 2023: Hendy et al. 2023 Alves et al.
2024 Koemi
& Federmann 2023 Raunak al. (2023). ) but lag behind in translations involving ower-resource
languages (Robinson et al. 2023] Hendy et al. 2023 Zhu et al. 2024 Lu et al. 2024). compared to
specialized neural machine translation (NMT) systems like NLLB (Costa-jussa al. 2022). This
performance gap is caused primarily by scant pre-training data in these languages (Wei et al. 2023
Yuan et al. 2024 Alves et al. 2024), and is difficult to overcome despite growing efforts to support translations of long-tail languages (Kudugunta et al. 2024 Bapna et al.] 2022 Lu al. 2024).
In this work, we introduce multilingual fused learning (Mufu), which combines multilingual context and a postediting task when translating into lower-resource languages using LLMM Mufu-style prompts (see Table [] top block) include several multilingual translation candidates along with a postediting target, from which model learns "in-context" to translate from languages with which the target language is more closely aligned due to cultural relevance, geographical and genealogical proximity. We rely a larger, more competent multilingual teacher model to generate auxiliary translations in these languages, which help disambiguate inputs and improve cross-lingual semantic alignment in a translation task. Given a task to postedit, LLMs are capable of "translating" better by iteratively improving the fluency and naturalness of the translation candidates (Chen et al. 2023).
The goal to induce LLMs multi-step reasoning akin thain-of-though (CoT) (Wei et al. 2022). as the models are required to assess the input quality, align the candidates cross- cross-lingually, and improve the final translation by drawing from the correct input and overriding incorrect instances. Translating this way can be challenging for small models with limited reasoning capacity. Inspired by Wang al. (2023), we further propose finetuning against Mufu prompts, which allows the models to learn how to best exploit and benefit from the multilingual context.
"Work done during an internship at Google.
We borrow the name from # f (mà fū), secretariat for the imperial Chinese officers dating back to 229 BC (Wikipedia contributors
2024

---

Published as a conference paper at ICLR 2025
The English scutence has boen translated into Malay, Javanese, Sundancse, Indonesian, Minangkabau and Achinese. These
translations may contain errors. Correct the translation from English to Achinese.
English: The proposed amendment already passed both houses in 2011.
Automatic Malay: Pindaan yang dicudangkan telah diluluskan uleh kedus dewan pada tahun 2011.
Automatic Javanese: Amandemen sing diusulake wis ditampa dening lom omah ing taun 2011.
Automatic Sundanese: Amandemen anu diusulkeun parantos lulus duanana imah dina 2011.
Automatic Indonesian: Amandomen yang diusulkan sudah disahkan olch kedua majelis pada tabsan 2011.
Automatic Minangkabu: Amandemen nun diusulkan alah disetujui dewan legislatil pado laun 2011.
Automatic Achinese: Amandemen nyang grupeugah nyan geupengos hak keu-2 2 bak thön 2011.
Corrected Achinese:
Reference: Amandemen nyang genusong ka geuteurimeeng lē handuwa majeulis bak thôn 201 1.
Baseline instruction: Translate from English Achinese.
Table 1: Prompt template for mufu5 (top block) with Achinese example, which includes an instruction (line 0), an input (line 1, blue), five multilingual candidates (lines 2-6, orange) and postediting target (line 7, red). For baseline we omit lines 2-7, replacing Corrected Achinese with Achinese and the initial instruction with the baseline instruction in purple. In postedi ting, we remove auxiliary languages (teal) in the instruction along with the multilingual candidates, retaining only the postediting target.
We show that the best Mufu model, finetuned only with hundreds of parallel examples in each language pair, is competitive against the teacher model and the benchmark NLLB 1.3B distilled model, scoring average 2.7 higher chrF on FLORES-200 devtest and 0.7 NTREX test sets in En-XX translations F Importantly, Mufu works well on a range of pre-trained models including PaLM2 and Gemma, despite limited data and the fact that Gemma models are English- models that have not been trained for multilingual capabilities (Anil et al.) 2023 Gemma Team et al.) 2024 Our experiments further demonstrate knowledge distillation Mufu models be effective in reducing the inference cost, while maintaining competitive ad advantage against benchmark.
2
MULTHLINGUAL FUSED LEARNING
2.1 COMBINING TWO LEARNING PARADIGMS
Few-shot in-context learning (ICL) is incredibly effective for eliciting translations from an LLM (Winata et al., 2021 Lin et al. 2022). but is usually less performant than more compute- and data- intensive finetuned models Zhang et al. 2023b Vilar et 2023 Xu et al. 2024 Lu al. 2024). On one hand, ICL improves translations of LLMs by allowing for informative contexts that induce reasoning processes in the model, and prompt the model reach a latent feature space that is otherwise difficult to access with shorter input (Wei et al. [0022] Wang et al., 2023 Vilar et al.) 2023 Puduppully et al.) [203] Zhuu et al., 2024) Zhang et al. 2023a). On the other hand, LLMs produce higher quality final predictions with parameter tuning. Motivated by /aan et al. (002), our work combines the strengths of both learning paradigms by finetuning LLMs with reference output against multilingual prompts, and substantially improves the overall quality of LLMs' translations over finetuned-only models, under a low-data condition.
2.2
MAXIMIZING DATA EFFICIENCY MULTILINGUAL AUXILIARY TRANSLATIONS
Beyond providing few-shot examplars in a translation prompt, we incorporate translations in other languages as auxiliary information to the task. Learning to translate this way facilitates semantic alignment beyond the lexical level, by allowing the encoding of rich knowledge network embedded in the multilingual translations. This multilingual context includes a draft translation in the target language, thus turning the difficult task of translating from scratch into postediting task. Taken together, this approach can be considered similar to CoT rationales, as we expect LLM to be able to disambiguate words and align across multilingual context, copy from high- high-quality inputs and to disregard instances that are less informative or are of poor quality. Unlike typical CoT, however, Mufu models do not predict the chain of thought and is instead provided rich context for intermediate reasoning in translation.
"Based on the performance of Pal M2 XXS-NTI. (mufu20), further details in Section 3.3
2

---

labels to the language tags in the auxiliary translations instead Published as a conference paper at ICLR 2025
The English sentence has been transiated into Malay, Sundanose, Javanese, indonesian, Mhanglabeu and Achinese, Thede translations may contain
translation Engiish to
Automsted Malay.
Sishat prompting
Automated Sundanese:
Student model
Teacher model
Automated Javanese:
Automated indonesian:
Automated Wnnaankahhuu
Automated Achinese:
Corrected Achinese translation
Corrected
Figure 1: Mufu involves two iterations. First, teacher model generates of multilingual auxiliary translations and a postediting target. These translations then become part of the input during the second iteration, where the student model learns in-context to produce the corrected target translation. We then finetune the student model against target references.
In practice, obtain and incorporate the auxiliary translations and postediting target in context, Mufu requires two iterations. During the first iteration, a teacher model is required to generate the intermediary translations. These translations are later included as part of the input for student model, which learns in-contex. to correct the target translation in the second iteration We illustrate an example of this process in Figure [ where the teacher model first translates the same input from English to auxiliary translations in Malay, Sundanese, Javanese, Indonesian, Minangkabau and Achinese (the target )anguage) These outputs are then added as part of the in-context prompt for the student model, along with an instruction to correct the target translation.
3 EXPERIMENTS
3.1 DATA AND EVALUATION
As low-data setup, we train and validate on the FLORES-200 dev split (Costa-jussà et al. 2022). which differs from the usual practice of reserving the split entirely for validation Out of 997 source sentences in the split, we randomly sampled 787 sentences as the train set, T00 sentences the validation data, and another 100 sentences to perform initial prompt selection. We reserve the remaining ten source sentences, from which we sample five-shot exemplars used in generating auxiliary translations the first iteration. Each of the source sentences is paired with translations in 203 languages, from which we finetune the student models to translate from English into subset of 201 target languages R Some languages use more than one writing systems for example, Achinese can be written in Latin and Arabic scripts; we treat translations into different scripts as individual language pairs.
We evaluate our approach using chrF, character overlap statistic (Popović) 2015). The finetuned models are tested on FLORES-200 devtest split for the ideal in-domain setting where train and test conditions are closely matched. The source sentences of FLORES-200 are sampled from Wikipedia-to assess our finetuned models out of domain, we use NTREX (Federmann et al, 2022), which comprises translations of English news data, which evaluate 112 languages, the subset of languages also found in FLORES-2001
3.2 PROMPT STYLE AND AUXILIARY LANGUAGES
We a variety of prompts with one-shot prompting and choose instruction that list all auxiliary languages (c.g.. from English to Malay, Sundanese, Javanese, ...) over an instruction for the model infer these languages from the prompt (c.g.. from English to several languages as specified). We also prepend Automatic/Correcaa
*The student may be the same model as the teacher this setup.
*See Section 3.2 for details on how the intermediate languages are chosen.
5 As described Costa-jussà et al. (2022)
oThe two languages omitied are Akan and Twi.
7The languages from FL.RRFS-200 not supported in NTREX are shown as dashed entries in Table
(Appendix A.5)

---

Published as a conference paper at ICLR 2025
of Candidate/Reference pair. We show in Table an example template of a Mufu instruction, in contrast with the baseline setup where we provide only an instruction to translate in the prompt, without any multilingual context postediting target. Further details prompt selection can be found in Appendix |..
To select the most relevant auxiliary languages in Mufu, we werely rely language data from URIEL (Littell et a 2017) to select the closest languages by geological and genetic distance (equally weighted) for each target language, and arrange them by the farthest closest in the prompt. Several languages are not included in the
repositor, in which case we sampled their auxiliary languages randomly For the full list of auxiliary languages used in Mufu prompts, see Appendix [A.2]
We finetune with Mufu prompt over a varying number of auxiliary translations: postedi ting (mufu0) contains only a postediting target and does not include any multilingual context; mufuN incorporates N E (5, 10, 20} auxiliary multilingual translations in addition to postediting target.
3.3 MODELS
The teacher model, PaLM2 S (also known Bison), has shown excellent multilingual and translation capability (Anil at 2023). but there remains a significant performance gap between higher-resoune report the teacher performance in Section 4 and show the gap can be largely reduced by the student models through Mufu. During the first iteration, the teacher model generates auxiliary translations for each instance with 5-shot prompting. For all prompt setups described in the previous section, we perform supervised finetuning jointly over 201 languages for En-XX translation over a range of student models: P2 XXS (Gecko), P2 XS (Otter), Gemma 2B-IT and Gemma 7B-IT; given the same auxiliary translations generated previously.
and lower-resource languages-
When comparing the performance across student models, it is worth noting that PaLM2 are mul- tilingual LLMs with superior initial translation capacity compared to Gemma models, which have not received any specialized training on multilingual tasks (Gemma Team et al. 2024). We also further pre-train PaL.2 XXS, the smallest model from PaL.2 family, on a corpora derived from the Sext-Thousand-Lagguae (NTL) effort, which comprise monolingual and parallel sentences in 1000+ languages (Caswell et al. 2020 Bapna al. 2022). We refer to this version of the model as PaLM2 N...
4 RESULTS
We evaluate primarily using chrF rather than BLEU (Papineni et al. 2002), which heavily relies on tokenization that is underdeveloped for many low-resource languages Table[2 Ishoos shows the mean chrF across 201 En-XX language pairs of all teacher, student and benchmark models; and Win%, the percentage of language pairs where the model outperforms a benchmark. NLLB models only support 198 of these language pairs- to facilitate comparison, we therefore report also the average chrF and win percentages over just these languages 10
When tested with in-domain FLORES devtest data, Mufu finetuned models gain substantially over their baselines. Turning translation task to postediting one is advantageous to the output quality, and we see further improvements with multilingual context Mufu prompts. Mufu models also show superior performance compared to the teacher, with PaLM2 XXS-NTL exceeding teacher performance in 54.2% translation pairs respectively. The exception is regular PaLM2 XXS, which score better than the baseline but underperforms compared to the teacher and the smaller NLLB model, presumably due to its limited capacity.
In theory, it is possible for the student be at least as good as the teacher through word-for word-forwordd copying from the postediting target. However, some Mufu translations are worse than the teacher.
'The languages found in URIEL include Latgalian, Swahili, Kongo, Kanuri, Kanuri in Arabic script, Silesian, Pashto, Oromo, Guarani, Kabuverdianu, Tumbuka, Kimbundu, Filipino. Friulian, Dinka, Mongolian, Azerbaijani, Fulfulde, South Levantine Arabic, Uzbek, Sardinian, Limburgan, Persian, Tamazight, Crimean Tatar in Latin script, Dzongkha, Lombard and Dari. Nonetheless, we report the corresponding results in BLEU scores in Appendix A.4 which largely corrob- orate our main findings.
The languages not supported by NLLB are Minangkahau in Arabic script, Arabic Latin script and Santali.
4

---

Published as a conference paper at ICLR 2025
FLORES- devtest
NTREX
chfF (n=201)
chfF (u=198)
Win%
Wink vs.
Wink vs.
chrF (a=112)
Wink teacher
Winth vs.
NLLB
NLLB
Teacher
1.38
54B
1.38
PaLM2 (teacher)
43.3
43.7
58.1
43.2
73.2
NLLB 133 distilled
46.0
41.3
4.0
48.1
26.8
NLLB 54B
48.9
56.2
96.0
baseline
39.2
39.4
32.8
11.6
8.0
36.3
8.9
0.9
postedit
mufa5
47.1
47.3
46.8
57.1
24.6
46.5
17.0
21.4
muful0
48.0
48.3
52.2
75.3
32.7
47.7
17.0
35.7
M2 XXS
mufu20
48.4
48.7
54.2
76.8
39.7
48.8
20.5
61.6
-NTL
mufuShul
42.9
43.1
muluStr
44.4
44.6
42.3
33.8
19.1
43.0
11.6
7.1
distilled
45.1
45.5
42.8
35.4
17.1
49.0
45.5
48.2
baseline
35.8
35.9
26.9
7.6
5.5
34.2
1.8
postedit
41.7
42.0
28.9
22.2
9.0
43.4
6.2
8.9
PaLM2 XXS
mufuS
mufulo
41.0
41.1
30,8
14.1
9.0
40.2
8.0
4.5
mufu20
41.1
41.2
14.1
9.5
baseline
31.7
31.9
21.9
2.5
1.0
31.3
postedit
43.8
44.1
36.8
28.3
43.3
10.7
PaLM2 XS
mufuS
44.5
44.6
40,8
33.8
17.6
43.6
8.9
11.6
mufu20
44.7
44,8
43.3
36.9
19.1
43.8
9.8
13.4
baseline mufu20
32.9
33.0
27.4
4.5
2,5
30.7
S
47.0
47.1
51.2
58.6
27.6
45.6
17.9
26.8
mufa20lora
47.2
47.5
99.0
72.2
59.8
50.1
91.1
83.9
baseline
34.4
34.4
28.9
9.1
4.0
29.2
6.2
0.9
postodit
44.1
44.3
32.8
16.1
Gemma 2B
mufu5
45.1
45.3
37.8
49.5
22.1
43.2
9.8
mufal0
45.4
45.5
39.3
47.0
21.1
43.3
9.8
10.7
mufu20
45.6
22.6
43.6
13.4
33,3
35.
postedit
46.3
46.5
41.8
$4.0
24.6
43.2
9,8
12.5
mulu5
47.2
47.3
49.3
60.6
27.6
43.4
11.6
mufu10
47.2
47.3
49.3
61.6
27.1
43.2
14.3
Gemma 7B
mufu20
47.6
47.7
51.7
63.6
29.6
43.6
11.6
17.9
mufuShrl
46.4
46.6
42,8
$2.0
26.1
43.2
10.7
13.4
mufuStr
mufu20+5hrf Shrl
47.7
47.8
51.2
66.7
30.7
44.1
12.5
17.9
distilled
Table 2: Mean chrF scores and win percentages against PaLM2 as teacher model for 201 En-XX language pairs; NLLB 1.3B distilled model and NLLB 54B MoE model for 198 language pairs. Bold values are the best chrF scores in a given model class. Red values are win rates above 50%. Mufu/5, 10, 20) indicate the number of non-target multilingual candidates in the prompt. We also report the distillation performance of PaLM2 XXS-NTL and Gemma 7B finetuned with mufu20.
We attribute this phenomenon to the limited amount of supervision in each language pair and autoregressive modeling objective with gold- standada translation strategy known to be inferior to distilling from model outputs (Kim & Rush 2016 Wang et al. 2021 Finkelstein & Freitag 2023). Mufu effective for under-resourced languages with low-quality postediting candidates. However, improving high-quality translations in high-resource languages is harder and requires the student model to also learn the subtle differences between model- and -eemanngenerated output (Sizov al.)
2024 Zhang et al.] 2024 Koemi al.) 2024). It is also possible that the teacher model surpasses humans for some translations in high-resource languages in which case, learning from the human translations could be detrimental.
Compared to NLLB 1.3B distilled, PaLM2 XXS-NTI finetuned with mufu20 translates better in nearly 77% language pairs. The best Mufu models also outperform NLLBB 54B MoE in up nearly

---

Published as a conference paper at ICLR 2025
(a) Very low resource
(b) Low resource
PuLe Km
PREM
25 25
PALM25 Genmaa 23 Gemmaa
Medium resource
(d) High resource
PAL PHLM
XXX PAIM PAIM2 Gemma 23
PAEM
poctedt
HLLD
Figure 2: Mean chrF across languages of the same resource level. Mufu outperforms the baseline consistently, and improves upon translations by the teacher model in low and very-low resource languages. Mufu is also competitive against NLLB 1.3B distilled translating into very low resource languages, and consistently outperforms the latter in low, medium and high resource setting. Note that the scales of y-axes are different for the top and bottom rows. Error bars shown are 95% confidence intervals across the language pairs.
40% of the translation pairs, despite being an order of magnitude smaller than the benchmark model. The result thus suggests the potential advantage using higher-quality multilingual candidates produced by NLLB for Mufu
While we expect decline performance due to distribution shift when translating out-of-domain sentences of NTREX, Mufu models hold up well in comparison with the baseline. Most Mufu models no longer outperform the teacher model and NLLLB 1.3B distilled, but PaIM2 XXS-NTL. with mufu20 maintains advantage over the NLLB model, scoring higher average with 48.8 chrF, and is better 62% language pairs.
The full results of PaLM2 XXS-NTL (mufu20) and Gemma 7B (mufu20) are reported in Ap- pendix A.5 To generalize the performance of Mufu beyond PaLM2 and Gemma models, we additionally report the translation results of finetuned BLOOMZ IB7 (Muennighoff et al. 2003) in Appendix A.6 which show significant improvement over baseline and postedit only conditions.
4.1 PERFORMANCE IN LOW RESSURCUCCE LANGUAGES
Figure shows the mean chrF of Mufu models in four language categories: very-low resource (n (8). low resource (n 45), medium resource (n 68) and high resource (n = 17) languages Again, we compare against the teacher and NL.BB 1.3B distilled models, indicated by the red and black dashed lines respectively.
We are most interested the very-low-resource languages, where we observe all Mufu models obtain substantial gains over the teacher model. This shows Mufu is capable of overcoming noisy auxiliary candidates, since most low-resource target languages are in proximity with other low- resource languages, as included in the prompt. The best Mufu models also competitive against NLLB 1.3B distilled, and maintain these advantages in low-, medium- and high-resource settings.
"We also extract translations from PaLM2 XXS-NTL by five-shot prompting (without any parameter updates), and find the translation quality to he worse than baseline finetuning, supporting Zhang et al, (2023b) 12ehee resource levels of each language were based on our subjective judgements on the accessibility of data and the competency of current translation systems and from English. We report the resource levels of the languages in Appendix A.5 A.5 Table g

---

Published as a conference paper at ICLR 2025
FLORES- 200 devtest
NTREX
NLLB
NLLB
NLLB
teacher
teacher
1.38
baseline
56.9
16.8
11.4
32.3
0.0
postedit
13.2
35.5
PaLM2 XXS
mufu5
78.4
56.6
28.9
54.8
19.4
-TTL
muful0
85.3
65.5
35.1
54.8
12.9
mufu20
85.3
63.7
36.0
64.5
38.7
distiled
73.3
40.7
21.1
77.4
41.9
baseline
57.8
23.9
14.9
25.8
postedit
71.6
42.5
27.2
25.8
6.5
Gemma 7B
mulus
81.0
50.4
30.7
25.8
6.5
muful0
81.9
51.3
29.8
22.6
6.5
mufu20
84.5
53.1
33.3
29.0
6.5
distilled
71.6
33.6
26.3
61.3
25.8
Table 3:
Win percentages measured over the 113 low and very-low resource languages for models shown in rows against, as columns, the teacher model, NLLB 1.3B distilled and NLLB 54B MoE. Win rates above 50% are in red.
In medium- and high-resource languages, Mufu models improve the most relative the baseline, but fall short compared to the teacher model.
The win percentages of the best Mufu models, PaLM2 XXS-NTL and Gemma 7B, against the teacher model and NLLB models in low and very-low resource languages are reported in Table which largely corroborate the results in Figure Mufu models outperform the teacher in 78-85% of these languages on FLORES devtest and up to 64.5% on NTREX. Among the Mufu models, PaLM2 XXS-NTL is the most consistent, outscoring NLLB 1.3B in 64% and 39% languages. It is also impressive that the Mufu model beats NLLB 54B MoE in more than one third of the languages on FLORES devtest, given the substantial difference in training and capacity.
4.2 CROSS-LINGUAL, ALIGNMENT WITH ATTENTION AND THE EFFECT OF AUXILIARY TRANSLATIONS IN CLOSELY RELATED LANGUAGES
We present cross-lingual attention alignment of the finetuned models across Mufu input mecha- nistic explanation of the improvement in translation performance. Table compares the translations by Gemma 2B finetuned with mufu5 prompt and the baseline prompt. Tenth is translated as Keupulôh by mufu5, which is close in form to the reference (kesiploh) and is untranslated in the postediting target and skipped entirely by the baseline model. The top block highlights parts of the input attended by the mufu-finetuned model, immediately before the production of Keupulöh, indicating transfer of the form from these auxiliary translations. The model also fixates Achinese, the target language in this example.
Beyond outright copying, Mufu models are also capable of transliterating and translating from attention-aligned input that are dissimilar in form. Transliteration from Latin to Arabic script is observed in Achinese-an example where the model transliterates Jamaika into the correct Arabic form باماييك a word unseen in the postediting target and the baseline translation, is shown in Table in Appendix A.7 whereas the translation of minimum Mizo involves attention to Bengali, which differs from Mizo in form and script, as shown in Table 11
We provide quantitative evidence Figure showing the sum mean multi-head attention of all layers directed to different parts of mufu5 inputs from the generated candidate (normalized by length), across validation examples Indonesian auxiliary input is the most useful when translating into Achinese in both Latin and Arabic script; Myanmar receives the most attention relative to the other auxiliary inputs during the translation into Mizo; auxiliary translation in Rundi is helpful to the translations into Kinyarwanda, as Zulu is to Swati some of these auxiliary translations receive comparable attention to the English source during the process.
sample of language pairs. Apart from the postediting target,

---

Published as a conference paper at ICLR 2025
The English sentence has been translated into Mally, Sundanese, Javancse, Indonesion, Minangkabau and Achinese. These translations may contain errors, the translation from English to Aclenese
English: In ambush of Bardia, the British captured the Italian Tenth Army's 's Engineer-in Ch-Ceeef, General Lastucci.
Automatic Malay: Dalam satu serangan hendap di timur Bardia, British berjaya menangkap Ketua Jurutera Tentera Itali, Jeneral Lastuceij
Automatic Javamese: Ing serangan ing sisih wetan Bardia, Inggris nyckel Insinyur ing Kepala Tentara Itatia Sepu Cit
Jenderal
Lastucei|
Automatic Sundanese: Dina hiji towak di wétan Bardia, Inggris néwak Insinyur-in-Chif Tentara Italia, Jenderal Lastuoci.
Automatic Indonesion: Dalam sebuah penyergapan di sebelah timur Bardia, Ingaris menangkap Insinyur-in-Chice Angkatan
Darut Italis Kesepalait Jenderal Lastueci.
Automatic Minangkahar: Dalam suatu penyergopan di timur Bardia, Inggris manawan Insinyur Kapalo dari Tentarn Italia kn- 10,
Jenderal Lastucci
Automatic Achinese: Bak sèngkeiji bak timu Bardin, ureueng Inggeris gelipeunan ureueng Italia Tenth Army's Engineer-in-Chief
General Lastucci
Corrected Acl mnsei sctillop senyeurch sebelah timii Bardia Ingerch neulsapol roh Insinyur- Angkatan Darek
Talia
mufu5
Lam scuhap senyeurch di sebelah time Bardin, Ingerch neukapol roh Insinyur-in- in-hiff Angkatan Darek Italia
Keupulóh, Jeneral Lastucci.
baseline
Bak saboh scmbuh kira-kira Bardin, Ureueng Inggreh ipcumeunangan Enreng Italia Jumat Pkat Teuntn-dalam- Cahya, Jendral Museh Lokka.
reference
Lam penyerangan di timu Bardia, ureueng Ingerch geudrop pangulés insinyur angkatan darat kesiploh Italia, Jenderal Lastucci.
Table 4: Translations from English to Achinese. The word Tenth in English is untranslated in the postediting target and baseline, but is translated into Keupulóh (cf. kesiploh in reference) by Gemma 2B finetuned with mufu5 prompt. The highlighted text shows the aligned attention across mufu5 prompt right before the production of Keupulōh, indicating form transfer from the multilingual input (Sepuluh in Javanese, Kesepuluh in Indonesian, ka-10 Minangkabau). Note that the attention presented here is the mean value across multiple heads and layers. Tokens with aggregated attention values under ,01, .6,6 13, .2 are colored white, light gray, dark gray and black respectively.
Achinese (Arabic script)
Achinese
Mizo
Kinyarwanda
Swati
instruction
instrctionn
Instruction
Instrction
Insbuction
Input
Input-
Input
Maley
Maleyy
Tibetan
Chichena
Sesotho
Sundanme
Sundanese
Kechin
Kambe
Xhost
javanese
Indonmien
Indonesian
Myermer
Lugands
Sepedl
Miaankaabuu
Minangkabau
Rundi
postediting
postedting
tirget
targes
aaget
target
target
0.25 5.18 6.5
5.1
0.1
Figure 3: Sum of self-attention from the tokens of generated candidate (e.g.. "Corrected Achinese: ...") to the instruction, input, auxiliary translations and the postediting target. Some auxiliary translations receive more attention than the input (c.g.. Indonesian vs. input in translating into Achinese in Latin and Arabic scripts; Zulu vs. input in translating into Swati). Note that significant portion of attention placed the generated sequence itself, which is omitted from the plot.
4.3 ABLATION
Mufu iteratively improves translations where teacher and student are the same model. report the results of finetuned PaLM2 S (baseline and mufu20) in Table 2 and Figure 2o demonstrate the efficacy of Mufu in setups where the student and teacher are the same model.
Mufu mitigates overfitting. PaLM2 XS and PaLM2 S finetuned with the baseline method overfit and perform worse than PaL.2 XXS (Table [3 Mufu is largely resistant to the problem, showing consistent improvement with increasing model size. To further reduce overfitting, we experiment with LoRA finetuning (r = 16) (Hu et (022) on PaLM2 S with mufu20 (mufu20lora), This setup
1t is possible that the models overfit to translations in high- nigh-resource languages, but not in low-resource languages. Thus, reasonable approach would he to terminate hiighresource--anguag training early (i.c., as form of curriculum learning). We leave this experiment future work.
8

---

Published as a conference paper at ICLR 2025
pushes the model's win rates to 99% and 91% in FLORES-200 test and NTREX; and leads to better performance than NLLB 54B in nearly 60% translation directions (Table ( Figure 2a however, reveals that mufu20 with LoRA, while being highly resistant to overfitting with few parameter updates, is less effective than full finetuning on sery-low-resoucee languages. The result is presumably related to recent findings that LoRA with low-rank perturbation underperforms compared to full finetuning in newly acquired skills fower-resource languages), but forgets less of the prior knowledge gained during pre-training (higher-resource languages) (Biderman et al. 2024).
Mufu works best with closely related auxiliary languages. To test if Mufu is still effective without these careful selection of auxiliary languages, we additionally finetune PaLM2 XXS-NTL and Gemma 7B with mufu5 prompt consisting of only five high-resouree languages chosen to simulate colonial influence: Dutch, Russian, French, Chinese and Spanish; and report the result Table 2 (mufu5hr1)) [ While having less relevant multilingual context is better than having no context at all, the improvement far below the model's upper threshold of translation capacity that we observe in the other Mufu variants. Adding these languages to mufu20 (mufu20+55r1, Table 2 improves Gemma 7B's s translations but undermines the performance of PaLM 2 XXS-NTL, as the model is distracted from highly informative candidates in relevant languages 15
Mufu's performance is predominantly driven by multilingual candidates. In mufu5tr, we remove the postediting target and instruct the model (PaLM2 XXS-XXXIII and Gemma 7B) to translate given the other auxiliary candidates. Table 2 shows mufu5tr to be better than the postediting task alone, but combining both conditions (mufu5) yields the best performance.
Distilling Mufu models reduces inference cost and retains accuracy gains. Translating with Mufu admittedly incurs a high inference cost given the need generate auxiliary translations. Thus, we propose distilling Mufu models with the best performance in low-resource languages to reduce the cost to the bascline level (Kim & Rush, 2016). For distillation data, we use the 6193 English sentences from NLLB seed data (Costa-jussà et al. 2022), and sample 6000 English sentences from past WMT General Tasks test sets (2009-2018) that are not found in NTREX [T We use the simple sequence knowledge distillation method from Kim & Rush (2016). which involves supervised fine-tuning of the student model against eaacher-predicter sequences.
We choose to distill PaLM2 XXS NTL and Gemma 7B finetuned with mufu20 for their strong performance in low resource languages. The distilled models are competitive against baseline and the teacher model across all languages (Ta as well as in low-resource languages (Figure Table 3). Given the mixture of domains in the distillation data, it is not surprising to see the distilled model outperforming the initial model in NTREX, in spite of the latter having never been exposed to gold translation output from the news domain. This signals strong potential to improve out-of-distribution performance of other Mufu models without additional parallel data source.
4.4 FAILURE CASES
Although translation quality improves in most languages pairs, there are a few cases where Mufu underperforms the baseline. One reason the use of randomly sampled auxiliary languages for some target languages (S.ctioon.22... In practice, however, only four out of these 28 target languages has auxiliary languages that diverge sufficiently from the target languages and hurt the translation performance consistently[7] Another major cause is the inclusion of auxiliary inputs of extremely poor quality- with three or more bad auxiliary translations, the input becomes more of a distraction than providing informative context. We provide an example of such input in Appendix A.8
5 RELATED WORK
ICL for translation. Vilar et al. (2023). Zhang et al. (202a)) and Zhu et al. (2024) find exemplar quality plays a more important role than semantic relevance in prompting for good translations.
where the target language is one of these languages, we replace the auxiliary input with translation candidate in Arabic.
15For target languages with high nigh-resource languages also appearing in the related auxiliary languages, we include additional related languages such that there are 25 distinct auxiliary candidates in total in the context. wwwrreesaarrc/forreesseeeeee wwwoore//llo/miiiii. w Mhttps://gi thub. cww. cwm/faceeeeeccccooo http//accbookeeearccheeo
seed/README .dd. if The languages are Kanur Arable script, Fultuide, Tamazight and Kimbundu.

---

Published as a conference paper at ICLR 2025
Few-shot ICL is however less effective in translating out of English than into English, contributing to the huge performance gap between low-resource and high-resource languages (Robinson et al,
2023 Zhu et al.; 2024). Ghazvininejad al. (202)) improve LLM's translation of rare words by providing multiple word-word hints derived from bilingual dictionaries. Mufu does not require billiguaa dictionaries, which can be hard to obtain for very-low-resourge languages; and has shown
remarkable improvement over baselines when translating into low-resource languages, which are among the harder translation directions.
Multilingual CoT reasoning for translation. LLMs are capable of chain-of-thought reasoning with multilingual prompts (Shi et al., [023 Chai et al. 2024). Zhu et al. (2024) find cross-lingual translation exemplars to improve translations from lower-resource languages to English. [Puduppulyy et al. (023) iteratively combines chunks of zero-shot translated input, assuming monotonicity between the source and target languages. [He et al. (2024) translate with LLM using synthetic keyword pairs, input topics and semantically related exemplars extracted from the same model, but rely on quality estimators to select the final predictions.
Low-resource translation with LLM. Low-resource languages are notoriously difficult for LLMs. Claude Opus, an LLM nearly three orders of magnitude larger than Mufu models (Anthropic) 2024).
outscores NLLB 54B in only 33% pairs of languages in the En-XX directions (Enis & Hopkins
2024). This is in spite of the fact that the model showing signs of contamination from FLORES 200
(Enis & Hopkins 2024). A growing body of work has nonetheless shown progress in the effort to
reduce the transl ation performance gap across language pairs, as well that between LLMs and
supervised NMT models (Tanzer et al. 2024 Zhu et al. 2024| Bansal et al., 2024 Lu et al. 2024
human in translations of unseen low-resource languages, when given the same language material (Tanzer ct al., 2024] Reid et al. 2024). Bansal et (2024) augments an LLM with smaller LLM of higher expertise in multiinguuality to improve low-resource XX-En translation, adding only small set of trainable parameters. [u et al.] (024) extend the vocabulary of LLLMMa models (Touvron et al., 2023 [Dubey et al. (2024) and continually pre-train the models with large-scale monolingual, parallel and synthetic data involving 102 languages. The pretrained models are superior to M2M-100 (Fan) cet al. (021)) in En-XX translations, but are nevertheless outmatched by NLLB 1.3B, which is more advanced than M2M-100.
Enis & Hopkins 2024 Bapna et al. 2022 Hendy al. 2023). LLMs are comparable
6 DISCUSSION
We present Mufu in this work, a method that maximizes data efficiency in low-resource translations with multilingual ICL and finetuning. Our analysis on cross-altention behaviour in Mufu-finetuned models provides evidence that the method extends LLM's capability in multilingual reasoning. That is, given any Mufu-style prompt. the finetuned models are capable of discerning input quality from multilingual candidates, aligning the input semantics across languages beyond orthographic similar- ity, and improving the candidate translation drawing only from informative context. Mufu models are stronger than the teacher model in low-resource languages and achieve consistent improvement over baseline finetuned models.
Mufu showcases a practical application of multilingual CoT to serve under-resourced languages, but the method carries two limitations. First, while it is largely robust against imperfect multilingual candidates, there seems to be minimum quality threshold under which Mufu translates worse than the baseline. would be, however, possible to extract higher-quality auxiliary translations from a stronger teacher (e.g., NLLB 54B), or to perform simple automated checks (c.g., for repetitions) to remove poor auxiliary candidates, to ensure the usefulness of the multilingual context. Second, relative to NMT models, Mufu incurs additional latency for improved accuracy, e.g., mufu5 improves the baseline model by 25.4% on average with six times more inference cost. The tradeoff is also evident in knowledge distillation on Mufu models with limited performance gains despite minimal latency. Thus is up to the practitioners to train using a more comprehensive data set, or to consider the acceptable tradeoff in their cases. There are nevertheless alternative LLM distillation methods that learn from model- model-generated generated text with substantial gains in generalization performance (Finkelstein
& Freitag| 2023
Agarwal al. 2024
Gu et al. 2024
Wang al., 2024).
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Special thanks to Isaac Caswell, Colin Cherry, Aditi Chaudhary, Grace Chung, Xin Yu and Daniel Formosa for their helpful feedback drafts of the paper and research support.
REFERENCES
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. In The Twelfth International Conference on Learning Representattons, 2024.
Duarte M Alves, José Pombal, Nuno M Guerreiro, Pedro Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, et al. Tower: An open multilingual large language model for tramslation-related tasks. arXiv preprint arXiv:2402.17733, 2024.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint 2rXS: 10403, 2023.
Anthropic. Introducing the next generation of claude, Mar 2024. URL wwR/https:/ww..antrroo.. Comm/news/llaude-3fFaa
Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Sriram Ganapathy, Abhishek Bapna, Prateck Jain, and Partha Talukdar. LLM augmented LLMs: Expanding capabilities through composition. In The Twelfth International Conference on Learning Representations, 2024.
Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat, Daan van Esch, Aditya Siddhant, Meng- meng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, al. Building machine translation systems for the next thousand languages. arXiv preprint arXiv:2255.038 2022.
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA learns less and forgets less. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https:://openrreien Featured Certification.
Isaac Caswell, Theresa Breiner, Daan van Esch, and Ankur Bapna. Language ID in the wild: Unexpected challenges on the path to a thousand languugge web text corpus. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 6588-6608, 2020.
Linzheng Chai, Jian Yang, Tao Sun, Hongcheng Guo, Jiaheng Liu, Bing Wang, Xiannian Liang, Jiaqi Bai. Tongliang Li, Qiyao Peng, et al. xcot: Cross-lingual instruction tuning for cross-lingual Chain-of-thoughe reasoning. arXiv preprint arXiv:2401.07037, 2024.
Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. Iterative translation refinement with large language models. arXiv preprint 1rXiv:2306.03850 2023.
Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centere machine translation. arXiv preprint arXiv:2207.04672 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXh::240..21783, 2024.
Maxim Enis and Mark Hopkins. From LLM to NMT: Advancing low-resource machine translation with claude. arXiv preprint arXiv:2404. 138133 2024.
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed Ki-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. Beyond English- centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1-48, 2021.
11

---

Published as a conference paper at ICLR 2025
Christian Federmann, Tom Koemi, and Ying Xin. NTREX-128 news test references for MT eval- uation of 128 languages. In Proceedings of the First Workshop on Scaling Up Mulrilingual Evaluation, pp. 21-24, 2022.
Mara Finkelstein and Markus Freitag. Mbr and qe finetuning: Training-time distillation of best and most expensive decoding methods. In The Twelfth International Conference Learning Representations, 2023.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et Gemma: Open models based Gemini research and technology. arXiv preprint arXhh:2403.08295 2024.
Marjan Ghazvininejad, Hila Gonen, and Luke Zeutlemoyer. Dietionary-based phrase--evell prompting of large language models for machine translation. arXiv preprint arXiv:2302.07835, 2023.
Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations, 2024.
Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang. Rui Wang, Zhaopeng Tu, Shumiing Shi, and Xing Wang. Exploring human-like translation strategy with large language models. Transactions of the Association for Computational Linguistics, 22:229-246, 2024.
Amr Hendy, Mohamed Abdetrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Mat- sushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. How good gpt models at machine translation? comprehensive evaluation. arXiv preprint arXiv:2302.09210 2023.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https:////penneeieee..nett iowpeeeeeiiee..ettooomimiii InZeVKeFYyf9
Yoon Kim and Alexander M Rush. Sequence-level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1317-1327,
Tom Koemi and Christian Federmann. GEMBA-MQM: Detecting translation quality error spans with GPT-4. In Philipp Koehn, Barry Haddow, Tom Koemi, and Christof Monz (eds.), Proceedings of the Eighth Conference on Machine Translation, pp. 768-775, Singapore, December 2023.
Association for Computational Linguistics. doi: 1/28633//12022..mm //aclanthology.org/
URL https:
wmt-1.64
Tom Koemi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dyorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewiez, et al. Findings of the WMT24 general machine translation shared task: the LLM era is here but MT is not solved yet. In Proceedings of the Ninth Conference on Machine Translation, pp. 1 1-46, 2024.
Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. Madlad-400: A multilingual and document-level large audited dataset. Advances in Neural Information Processing Systems, 36, 2024.
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, at. Few-shot learning with multilingual generative language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9019-9052, 2022.
Patrick Littell, David R Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 8-14, 2017.
Yinquan Lu, Wenhao Zhu. Lei Li, Yu Qiao, and Fei Yuan. LLaMAX: Scaling linguistic hori- zons of LLM by enhancing translation capabilities beyond 100 languages. arXiv preprint a0Xiv:2407.05975. 2024.
12

---

Published as a conference paper at ICLR 2025
Niklas Muennighoff, Thomas Wang. Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers), pp. 15991-16111, 2023.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: A method for automatic evaluation of machine translation. In Annual Meeting of the Association for Computational Linguistics, 2002.
Maja Popović, chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the tenth workshop statistical machine translation, pp. 392-395, 2015,
Ratish Puduppully, Anoop Kunchukuttan, Raj Dabre, Aiti Aw, and Nancy Chen. Decomt: Decom- posed prompting for machine translation between related languages using large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
pp.
Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Awadalla, and Arul Menezes. Leveraging GPT-4 for automatic translation post-editing. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 12009-12024, Singapore, December 2023. Association for Computational Linguisties. doi: 10. 12.118666//22233 org.2/oo findings wwandingg-eemlp..... 804
findings-emnlp.80 804. URL
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024.
Nathaniel Robinson, Perez Ogayo, David R Mortensen, and Graham Neubig. ChatGPT MT: Com- petitive for high-(but not low-) resource languages. Proceedings of the Eighth Conference on Machine Translation, pp. 392-418, 2023.
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. Language models are mul- tilingual chain-of-thought reasoners. In The Eleventh International Conference Learning Representations, 2023.
Fedor Sizov, Cristina España-Boner Josef van Genabith, Roy Xie, and Koel Dutta Chowdhury. Analysing translation artifacts: A comparative study of LLMs, NMTs, and human translations. In Proceedings of the Ninth Conference on Machine Translation, pp. 1 1183-1199, 2024.
Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi A benehmark for learning to translate a new language from one grammar book. In The Twelfth International Conference on Learning Representations, 2024.
Hugo Touvron. Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint w:Xiv:2307.09288. 2023.
David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. Prompting PaLM for translation: Assessing strategies and performance. In The 61sl Annual Meeting Of The Association For Computational Linguistics, 2023.
Fusheng Wang, Jianhao Yan, Fandong Meng, and Jie Zhou. Selective knowledge distillation for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( Volume 1: Long Papers), pp.
2021.
Jun Wang, Eleftheria Briakou, Hamid Dadkhahi, Rishabh Agarwal, Colin Cherry, and Trevor Cohn. Don't throw away data: Better sequence knowledge distillation. arXiv preprint arXiv:2407.10455 2024.
Xinyi Wang, John Wieting, and Jonathan H Clark. Fiat: Fusing learning paradigms with instruction- accelerated tuning. arXiv preprint 230.:2330 2023.
13

---

Published as a conference paper at ICLR 2025
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 01224424-248885 2022.
Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, et al. Polylm: An open source polyglot large language model. arXiv preprint wrXiv:2307.00116 2023.
Wikipedia contributors. Mufu Wikipedia, the free encyclopedia. https://en wi wikipedia. 2024. [Online; accessed 12-September-
- 2024].
Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. Language models are few-shot multilingual learners. arXiv preprint 2Xix:2009.07684, 2021.
Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in ma- chine translation: Boosting translation performance of large language models. In The Twelfth International Conference on Learning Representations, 2024.
Fei Yuan, Shuai Yuan, Zhiyong Wu, and Lei Li. How vocabulary sharing facilitates multilingual- ism in LLaMA? In Findings of the Association for Computational Linguistics ACL 2024, pp- 12111-12130, 2024.
Biao Zhang, Barry Haddow, and Alexandra Birch. Prompting large language model for machine translation: A case study. In Proceedings of the 40th International Conference Machine Learning, pp. 41092-411 41110, 2023a.
Ran Zhang, Wei Zhao, and Steffen Eger. How good are llms for literary translation, really? literary translation evaluation with humans and llms. arXiv preprint arXiv:2410. 18697, 2024.
Xuan Zhang, Navid Rajabi, Kevin Duh, and Philipp Koehn. Machine translation with large language models: Prompting, few-shot learning, and fine-tuning with QLoRA. Proceedings of the Eighth Conference on Machine Translation, pp. 468 481, 2023b.
Wenhao Zhu, Hongyi Liu, Qingxiu Dong. Jingjing Xu, Shujian Huang. Lingpeng Kong, Jiajun Chen, and Lei Li. Multilingual machine translation with large language models: Empirical results and analysis. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2024.
A APPENDIX
A.I PROMPT SELECTION
Prior to conducting experiments reported in the main text, we tested several versions of Mufu prompt on 100 sentences from FLORES-200 dev split reserved for prompt selection (see Section 3.1). We focused on a handful of target languages in the preliminary experiment: Achinese, Balinese, Buginese, Banjar and Minangkabau; using fixed set of auxiliary languages: Indonesian, Malay, Javanese, Sundanese and Arabic. Auxiliary candidates for prompt selection were first generated by PaLM2 via one-shot prompting:
Translate from English to <target language>.
Maybe one day, your great grandchildren will be standing atop an alien world wondering about their ancient ancestors?
English:
<target language>: <reference translation>
English: <input>
<target language>:
14

---

Published as a conference paper at ICLR 2025
We then evaluated different versions of the prompt using the same model during the second iteration, where auxiliary candidates were included in the instruction similar to the template shown in Table in the main text. We swapped out listed languages in the instruction with *... several languages as specified", and discovered it to be sub-par compared to the original prompt. We also experimented with prepending "Candidate/Reference" found the latter to yield superior performance. Note that these preliminary experiments on prompt variation do not involve finetuning, and we arrive at a final prompt template based on results derived entirely from zero-shot prompting.
to the language tags in place of "Automatic/Correctee and
A.2 AUXILIARY LANGUAGES
Table 5 lists the custom set of auxiliary languages for each target language included in Mufu-style prompt. The languages are selected based on URIEL repository as described in Section 3.2 and are arranged from farthest to closest. Target languages assigned with random auxiliary languages are marked with t.
Target langunge
Auxiliary langnages
Achinese
Buginese, Samoun, Shan, Vietnamese, Malagasy, llocano, Myanmar (Burmese), Fijian, Maori, Sinhala, Lao, Khmer, Thai, Ralinese, Banjar, Jalay,Javannss,, Sundanese, Indonesion, Miningkahmu
Achinese Arabic script
Buginese, Samoun, Shan, Vietnamese, Malagasy, Blocano, Myammar (Burmese), Fijian, Maori, Sinhala, Lao, Khmer, Thai, Balinese, Banjar, Malay, Javanese, Sundanese, Indonesion, Minungkahmu
Afrikaans
Bemba (Zambia), Danish, Xhosa, Swedish, Sesotho, Norwegian, Chichewa, Faroese, Icelandic, Tswana, Shona, Yiddish, Swati, Tok Pisin, Laxembourgish, Zulu, Sepedi, German, Tsonga, Dutch
Alhanian
Slovenian, French, Finnish, Romanian, Sicifian, Fwe, Basque, Italian, Craatia, Rengali, South Azerbaijani, Serbian, Hungarian, Egyptian Arabic, Bosnian, Ambaric, Macedonian, German, Greek, Bulgarian
Amharic
Sango, Hausa, Kinyarwanda, Rundi, Lua, Kamba (Kenya), Tunisian Arabic, Luganda, Kikuyu, Maltese, Nuer, North Levantine Arable, Mesopotamian Arabic, Najdi Arabic, Arabic, Hebrew, Egyptian Arabie, Somali, Tz'izii- Adeni Arabic, Tigrinya
Arahic
Bulgarian, Turkish, Tamasheq, Somali, Hausa, Armenian, Georgian, Tunisian Arabic, Kurdish (Kurmanji), Amharic, Sorani Kurdish. Maltese, South Azerbaijani, Tigrinya, Ta la'izzz-Adeni Arabic, Hebrew, Egyptian Arabic, North Levantine Arabic, Najdi Arabic, Mesopotamian Arabic
Arabic in Latin script
Bulgarian, Turkish, Tamashoq, Somali, Hausa, Armenian, Georgian, Tunisian Arabic, Kurdish (Kurmanjj), Amharic, Sorani Kurdish. Maltese, South Azerbaijani, Tigrinya, Th Adizzzi-deeii Arabic, Hebrew, Egyptian Arabic, North Levantine Arabic, Najdi Arabic, Mesopotamian Arabic
Armenian
Romanian, Lithuanian, Turkmen. Kashmiri. Najdi Arabic, Ioclandic, Turkish, Hindi, North Levantine Arabic, Irish, French, Mesopotamian Arabic, South Azerbijani, German, Sorani Kurdish, Kurdish (Kurmunji), Georgian, Bengali, Greek, Bulgarian
Assamese
Marathi, Myanmar (Burmese), Sanskrit, Gujarati, Kachin, Sinhola, Mizo, Santali, Kashmiri, Bhojpuri, Tibetan, Magahi, Meiteilon (Manipuri). Awadhi, Punjabi, Hindi, Nepali, Maithili, Odia (Oriya), Bengali
Asturian
Laxemhhargish,, Romanian, German, Sicilian, Kabyle, Welsh, Irish, Haitian Creole, Esperantoo Italian, Venetian, Papiamenio, Busque, Ligurian, Occitun, French, Catalan, Spunish, Galician, Portuguese
Awadhi
Meiteilon (Manipuri), Sindhi, Marathi, Tibetan, Sinhula, Sanskrit, Santali, Urdu, Assamese, Gujarati, Magahi, Kashmiri, Odia (Oryya), Bhojpari, Bengali, Manhili, Punjabi, Chhattisgarhi, Nepali, Hindi
Ayacucho Quechua
Kabiyé, Finnish, Tamashey, Basque, Mossi, Greek, Dyula, German, Bambara, Wolof, Bulgarian, Yiddish, Rengali, Haitian Creale, South Azerbaijani, Papiamento, Egyptian Arabic, Aymana, Amharic, Ewe
Aymara
Finnish, Hausa, Tamashey, Basque, Grek,, Dyula, German, Bambara, Wolof, Bulgarian, Yiddish, Bengali, Haitian Creole, South Azerhaijani, Papiamento, Egyptian Arahic, Amharic, Ayacucho Quechua, Fwe
Azerbuijani
Buginese, Cebuano, Chokwe, Icelandie, Fulfulde, Wolof, Norwegian. Lubu-Lalun, Malayalam, Uyghur, Se- rani Kurdish, Bambara, Myanmar (Burmese), Mandarin Chinese, Kabyle, Undu, Tamazight, Zulu, German, Laxembourgish
Balinese
Vietnamesse, Thai, Lao, Samoan, Khmer, Malagasy, Fijian, Maori, Pangasinan, Waray (Philippines), Hlccano Cehuano, Buginese, Minangkahum, Malay, Javanese, Banjar, Achinese, Sundunese, Indonesian
Bambura
Kabyle, Finnish, Igbo, Basque, Greek, Yoruba, Fon, German, Bulgarian, Bengali, Kabiye, Mossi, South Azer- baijani, Egyptian Arabic, Tamasheq, Amharic, Wolof, Hawsa, Ewe, Dyula
Banjar
Thai, Vietnamese, Samoan, Lao, Malagasy, Khmer, Fijian, Maori, Pangasinan, Wamyy (Philippinees), Hlccano
Cebuano, Minangkahau, Achinese, Buginese, Sundanese, Javanese, Balinese, Indonesian, Malay
Banjar in Arabic script
Thai, Viernamese, Samoan, Lao, Malagasy, Khmer, Fijian, Maori, Pangasinan, Wamy (Philippines), Hlocano,
Cebuano, Minangkabau, Achinese, Buginese, Sundanese, Javanese, Balinese, Indonesian, Malay
Bashkir
Lithuanian, Latvian, German, Belamusian, Bulgarian, Rengali, Finnish, Tgyptian Arabic, Amharic, Tajik, Geor- glan, Armenian, Turkish, Russian, Uyghur, Turkmen, South Azerhaijani, Kyrghyz, Kazakh, Tatar
Basque
Luxembourgish, Finnish, Ligurian, Esperanto, Ewe, Greek, Occitan, Irish, Galician, Bulgarian, Bengali, Catalan, South Azerbaijani, Asturian, Spanish, Egyptian Arabic, German, Portuguese, Amharic, French
15

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Belarusian
Greek, Danish, Macedonian. Swedish, Hungarian, Bosaian, Romanian, German, Estonian. Slovenian, Russian, Croatian, Serbian, Bulgarian, Slovak, Latvian, Lithusniam, Czech, Polish, L'krainian
Bemba (Zambia)
Xhosa, Umbundu, Lingala, Scsotho, Swati, Afrikaans, Luo, Tswana, Tsonga, Chokwe, Loba-Lulua. Sepedi, Zalu, Kamba (Kenya), Rundi, Kikuyu, Laganda, Shona, Kinyarwanda, Chichewa
Bengali
Chhattisgarhi Marathi, Gujarati, Myanmar (Burmese), Sunskrit, TThhetan, Sinhala, Punjahi, Meiteilon ( (Manipuri), Bhojpuri, Kashmiri, Mizo, Magahi, Sintali, Awadhi, Hindi, Nepali, Maithili, Odia (Oriys), Assamese
Bhojpuri
Sindhi, Mizo, Meiteilon (Manipuri), Gujarati, Marathi, Sunskrit, Sinhula, Chhattiggrhii, Tibetan, Santali, Kush- miri, Punjahi, Assamese, Odia (Oriya), Hindi, Awadhi, Rengali, Nepali, Magahi, Maithili
Bosnian
Sicilian, Romanian, Lithuanian, Belarusian, Hungarian, German, Venetian, Polish, Russan, Italian, Greek, Ukrainian, Czech, Slovak, Albunian, Bulgarian, Slovenian, Macedonian, Serhian, Croatian
Buginese
Thai, Vietnamese, Lao, Khmer, Samoan, Malagasy, Minangkabun, Fijan, Maori, Pangasinan. Waray (Phillip-
pines), Malay, Achinese, Bunjar, Blocano, Cebuano, Balinese, Indonesian, Sundanese, Javanese
Bulgarian
Latvian, Venetian. German, Lithuanian. Turkish. Hungarian, Belarusian, Russian, Albanian, Remanian, Polish. Czech, Slovak, Greek, Slovenian, Ukrainian, Bosnian, Croatian, Serbian, Macedonian
Cantonese
German, Bulgarian, Warry (Philippines), Thai, South Azerhaijani, Egyptian Arabic, Amharic, Sham, Ben- gall, Khmer, Lao, llocano, Tibetan, Pangasinan, Vietnamese, Mizo, Melteilon (Manipuri), Kachin, Myanmar (Burmese), Mandarin Chinese
Catalon
Rulgarian, Rengali, Romantan, Laxembourgish, German, Esperanto, Haitian Creole, Papiamento, Kabyle, Sieil- ian, Basque, Venetian, Italian, Galician, French, Asturian, Ligurian, Portuguese, Occitan, Spanish
Cahuano
Lao, Vietnamese, Sumran, Khmnr, Minangkahan, Malagasy, Cantrnese, Fijian, Maori, Malay, Achinese, Indone- slan, Banjar, Balinese, Sundanese, Pangasinan, Buginese, Javanese, Ilocano, Warry (Philippines)
Chhattiggurhi
Mizo, Karmada, Santali, Sinhala, Punjabi, Kashniri, Assamese, Urdu, Telugu, Bhojpuri, Maithili, Magahi, Gujarati, Nepali, Marathi, Bengali, Sanskrit, Odia (Oriya), Awadhi, Hindi
Chichewa
Lingaaa, Malagasy, Los, Xhosa, Labe Lulua, Sesotha, Chokwe, Afrikaans, Luganda, Tswana, Kumba (Kenya). Kikuyu, Rundi, Swari Tsonga, Remha (7amhia), Kinyarwanda, Sepedi, Shona, 7ulu
Chokwe
Scsotho, Sango, Swati, Luo. Tsonga, Afriknans. Kamba (Konya), Tswann Sepedi, Kikuyu, Bemba (Zambia), Zulu, Rundi, Luganda, Shona, Kinyarwanda. Chichewa, Lingala, Luba-Lulun, Umbundu
Crimean Tatar in Latin
Sanskrit, Fulfulde, Tamil, South Levantine Arabic, Sundanese, Limburgan, Azerbaijani, Guarani, Latvian, Kikuyu, Kinyarwanda, Irish, Tatar, Egyptan Arabic, Lingala, Hausa, Friulian, Maori, Tamazight, Oromo
script"
Cmaaian
Latvian, Romanian, Lithuanian, Greek, Belarusion, Alhanian, Iralion, Russian, Venetion, Polish, Hungarian,
German, Ukrainian, Bulgarian, Czoch, Macedonian, Slovak, Scrbian, Slovenian, Bosnian
Crech
Italian, Romanian, Dutch, Macedonian, Danish, Laxenbourgish, Lithnanian, Venetian, Hungarian, Belarusian, Russian, Bosnian, Bulgarian, Serbian, Ukrainian, German, Creatian, Slevenlan, Polish, Slovak
Danish
Groek. Scottish Gaelic, Bulgarian, Bengali, Norwegian Nynorsk. Yiddish, Lithuanian. Tok Pisin, Esporanto. Afrikaans, Polish, Czech, Faroese, French, Ioelandic, Luxembourgish, German, Dutch, Swedish, Norwegian
Dari'
Japanese, Aymura, Pangusinan, Maltese, llocano, Turkmen, Farosse, Oroma, Igbo, Yoruba, South Levantine Arabic, Guarani, Kikuyu, Ayacucho Quechua, Lan, Balinese, Latvian, Fijian, Belarusian, Kahuverdiama
Dinkat
Umbundu, Uyghur, Arabie, Fijian, Catalan, Sorani Kurdish. Mandarin Chinese, Bulgarian, Bengali, Japanese, Ilocano, Spanish, Korean, Balinese, Kabuverdiamu, Achinese, Tsonga, Macedoniam, Friulian, Polish
Dutch
Bulgarian, Bengali, Ligurian, Occitan, Swedish, Scottish Gaelic, Faroese, Czech, Ioelandic, Welsh, Yiddish. Tok Pisin, Irish, Esperanto, Afrikumns, Norwegiun, French, Danish, German Luxembourgish
Dyula
Finnish, Sango, Basque, Greek, Wolof, Igho, German, Yombh, Fon, Bulgurian, Bengali, South Azerbuijani, Egyptian Arabic, Hausa, Kabiyé. Amharic, Tamashey, Mussi, Ewe, Bambara
Dzongkha
Cantonese, Kashmiri, Fon. Aymara, Ayscucho Qucehua, Albanian, Swati, Lingala, Tal -zizzzz-Adeei Arabic, South Levantine Arabic, Georgian. Italian, Norwegian Nynorsk, Crimean Tatar in Latin script. Kannada, Maltese, Fijia,, Welsh, Shona, Igbo
Egyptian Arabic
Somali, South Azerbaijani. Albanian, Hausa, Kurdish (Kurmanji), Tigrinya, Amharic, Sorani Kurdish, Macedo- nian, Ta' izzi- Adeni Arabic, Bulgarian, Greek, Tunisian Arabic, Turkish, Maltese, Najdi Arabic, Arabic, Hebrew, Mesopotamian Arabic, North Levantine Arabic
Esperanto
Venetian, Fiunish, Catalan, Danish, Ewe, Greek, Ligurian, Occitan, Bulgarian. Welsh, Bengali, Dutch South Azerbaijani, Luxembourgish, Egyptian Arabic, Irish, Amharic, Basque, German, French
Estonian
Fwe, Basque, Czech, Greek, Danish, Polish, Norwegian Nynoesk, Rulgarian, Rengali, Relarusian, South Aver-
baijani, Norwegian, Egyptian Arabic, Lithuanian, Amharic, Larvion, Swedish, German, Hungarian, Finnish
Fwwe
Umhundu, Kamha (Kenya), Wolof, Luganda, Kinyarwanda, Bambara, Hausa, Tamasheq, Luba.Laloa, Kikuyu, Dyula, Chichewa, Zalu, Sango, Lingala, Mossi, Kabiyě, Yoriba, Igho, Fon
Farcese
Greek, Estonian, Bulgarian, Bengali, Esperanto, Yiddish, Tok Pisin, French, Welsh, Afrikauns, Norwegian Nynoesk, German, Scottish Gaelic, I Tnxemhourgish,, Irish, Durch Swedish, Danish, Norwegian, Icelandic
Fijiin
Cantonese, Korean, Japanese, Minangkabau, Malagasy, Malay, Achinese, Pisin, Pangusinan, Banjar, Indone- sian, Waray (Philippines), Sundanese, Javanese, Balinese, Buginese, Dlcaano Cebuano, Samoun, Maori
16

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Filipino
Magahi, Sopedi, Labe-Lolua, Czoch, Khmer, Iswana, Tamazight, Lithuanian, Lingala, Aymara, Swahili, Tajik, Chichewa, Venetian, Swedish, Ewe, North Levantine Arabic, Finnish, Fon, Mandarin Chinese
Finnish
Tatar, Basque, Faroese, Groek, Polish, Danish, Belarusian, Bulgarian, Bengali, Lithuanion, South Azerbaljani, Norwegian, Egypiian Arabic, Larvian, German, Norwegian Nynorsk, Amharic, Swedish, Hungarian, Estonian
Fon
Umbundu, Kamba (Kenya), Wolof, Luganda, Kinyarwanda, Bambara, Tamashog, Hausa, Lube-Lalon, Kikuyu, Dyula, Chichewa, Zalu, Sango, Lingala, Mossi, Kabiyč, Jgbo, Yoruba, Ewe
Fronch
Romanian, Steillan, Irish, Puplamento, Italian, Welsh, Basque, German, Dutch, Galiccan, Laxembourgish, Haltian Croole, Espcranto, Asturian, Spanish, Vonctian, Portuguese, Catalan, Occitan, Ligurin
Friulian
Spanish, Chichewa, Italion, Chhatrisgurhi, Mossi, Uyghur, Macedonian, Slovak, Odia (Origa), French, Haitian Crcole, Sorani Kurdish, Tok Plsin, Indonesian, Latgalion, Nepali, Icclandic, Samoan, Ayacucho Quechun, Dari
Fulfulle
Saniali, Catalan, Ta'izzi Ti'izzi-Adeni Adeni Arabic, Esperanto, Basque, Mamdarin Chinese, Arabic in Latin scripi, Balinese, Myanmar (Barmese), Kachin, Xhosa, Alhanian, Meireilon (Manipuri), Italian, Dari, Drongkha, Norwegian, Pangasinan, Assamese, Swati
Galician
Laxemboorgish, Romanian, German, Sicilian, Kabyle, Haitian Creale, Esperanto, Welsh, Irish, Italian, Venetian, Basque, Papiamento, Ligurian, Occitan, French, Catalan, Spanish, Asturian, Portuguese
Georgian
Finnish, Arabic, Turkmen, Ewe, Turkish, Rasque, Najdi Arahic, North Levantine Arabic, German, Mesopolamian Arabie, Hebrew, Sorani Kurdish, Bengali, Greek, Kurdish (KurmanjO, Amharic, Bulgarian, Armenian, Egyptian Arabic, South Azerbaijani
German
Italian, French, Swedish, Ligurian, Hungarian, Norwegion, Farnese, Polish, Creatian, Icelandic, Yiddish, Tok
Pisin, Slovak, Venetian, Danish, Slovenian, Afrikaons, Czoch, Dutch, Luxembourgish
Greek
Punjabi, Turkish, Lathmanian, Creatian, Kushmiri, Hungarian, Icelandic, Ukrainian, Armenian, Bosnian, Hindi, Irish, French, Albanian, Serbian, Macedonian, German, Bengali, Romanian, Bulgarian
Guarani!
Malayalam, Lingala, Ukrainian, Aymura, Galician, Lubu Luku,, Zulu, Bashkir, Sepedi, Chhattisgarhi, Arabic, Tok Pisin, Thai, Tigrinya, Japanese, Arabic in Latin script, Mizo, Najdl Arahic, Malay, Egyptian Arabie
Gujarati
Malayalam, Assamese, Magahi, Kanmada, Sinbala, Telugu. Bhujpuri, Odia (Oriya). Maithili, Bengali, Nepuli, Sanskrit, Marathi, Kashmiri, Sindhi, Chhattisgari, Punjahi, Awadhi, Urdu, Hindi
Haltian Creole
Steilian, Scottish Gaelic, Italian, Bambara, Occitan, Icelandic, Catalan, Wolof, Trish, Aymara, Ayacucho Quechun, Ligurian, Venctian, Yiddish, Spanish, French, Asturian, Fortuguese, Galicion, Papiamento
Hausa
Bamhara, Lingala, Arabic, Dyula, Mesogotamian Arabie, Sango, North Levantine Arabic, Mossi, Somali, Ewe, Kabiyč, Fon, Hebrew, Igho, Egyptian Arabic, Maltese, Amharic, Yoruba, Tunisian Arabic, Tamashoq
Hehrew
Somali, Hausn, Georgian. Greek, Amharic, Armenian, Balgarian, Kurdish (Kurmanjii, Ta'iniiAAdei Am- bic, Sorani Kurdish. Turkish, South Azerboijani, Tigrinya, Tunisian Arabic, Maltese, Najdi Arable, Arabic, Mesopocamian Arabic, North Levantine Arabic, Egyptian Arabic
Hindi
Kannada, Suntali, Telugu, Assamese, Sinhala, Magahi, Odia (Oriya), Sindhi, Bhojpuri, Maithili, Rengali, Kash- miri, Marathi, Nopali, Sanskrit, Urdu, Chhattisgarhi, Punjabi, Gujarati, Awadhi
Hungarian
Rasque, Polish, Venetian, Czech, Bosnian, Ukrainian, Bengali, Slovenian, South Azerhaijani, Egyptian Arabic, Romanian, Ambaric, Grook, Serbian, Croatian, Estonian, Finnish, Slovak, German, Bulgarian
Icelandic
Greek. Bulgarian, Finnish. Bengali, Esperanto, Yiddish, Tok Pisin. Afrikaans, Welsh, French, Nerwegian Nynorsk, Luxembourgish, German, Scotiish Gaelic, Irish, Dutch, Swedish, Danish, Norwegian, Farosse
Igbo
Tamashoq, Sepedi, Dyula. Umbundu, Sango, Rundi, Kamba (Kenya), Mossi, Kabiyč, Hausa, Kikuya, Luganda, Ewe, Fon, Kinyarwanda, Chichewa, Zulu, Laba-Lulua. Lingala, Yoruba
llocano
Thai, Samoan, Malagasy, Balinese, Khmor, Lao, Fijian, Malay, Achinese, Indonesian, Vietnamese, Cantoneac, Sundanese, Maori, Banjar, Buginese, Waray (Philippines), Cebuano, Javanese, Pangasinan
Indonesian
Vietnamese, Samoan, Lao, Thai, Malagosy, Khmer, Fijian, Pangasinan, Maori, Waray (Philippinas), Ilocano, Cebuano, Buginese, Achinese, Javanese, Minangkabon, Malay, Banjar, Balinese, Sundanese
Irish
Galician, Kushmiri, Asturian, Baqque, Armenian, Hindi, Dunish, Luxembourgish. Greek, Faroese, Portuguese,
Dutch, Bulgarion, Esperanto, Ioelandic, Bengali, German, French, Welsh, Scottish Gaelic
Italian
Papiamento, Serbian, Hungarian, Romanian, Asturian, Haitian Croole, Albunian, Galician, Crostian, Spanish, German, Busnian, Portuguese, Slovenian, Frrench, Catalan, Occitan, Ligurian, Venetian, Sicilian
Japanese
Finnish. Kachin, Ewe, Lao, Vietnamese, Basque, Greek, Cebuano, Waray (Philippines), German, Pangasinan, Bulgarian, Bengali, llucano, Cantonese, South Averbuijani, Mandarin Chinese, Egyytiin Arabie, Amburic, Korean
Javanese
Vietnamese, Thai, Lao, Samoun, Khmnrr Malagasy, Pangusiman, Waray (Philippines), Hijian, Maori, Minangk- ahau, Cehuano, Dlcaaoo Malay, Banjar, Balinese, Buginese, Achinese, Indonesian, Sundanese
Kabiyè
Umbundu, Kamba (Kenya), Luganda, Kinyarwanda, Wolof, Bambara, Kikuyu, Hausa, Luba-Lulus, Tamasheq. Chichewa, Dyula, Zulu, Lingala, Sango, Igho, Yoruba, Fon, Ewe, Mussi
Kabuverdiano +
Albanian, Achinese in Arabic script. Venetian, Malagasy, Najdi Arabie, Fulfulde, Marathi, Tamil. Xhosa, Sicilian, Slovak, Bashkir, Itllian, Irish, Georgian, Samoan, Achinese, Fijian Magahi, Tigrinya
17

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Kabylo
Asturian, Arabic, Italion. Portuguese, Mesopotamian Arabic, North Levantine Arabic, Somali, Basque, Ligurann, Occitan, Hebrew, Hausa, Sieilian, Spunish, Egyptian Arabic, Amburic, Catalan, Tamashey. Mallese, Tunisian
Arabic
Kachin
Nepali. German, Vietnamese, Magahi, Bulgarian. Odia (Oriya), Maithili, South Azerbaijani, Santali, Egyptian Arabic, Amburic, Mandarin Chinese, Assamese, Shan, Cantonese, Bengali, Tibetan, Mizo, Myanmar (Burmese), Meiteilon (Manipuri)
Kambu (Kenya)
Chokwe, Tonga, Tswana, Swati, Lingala, Amburic, Seaohha Sepedi. Nuer, Somali, Luba Lulun, Zulu, Luo, Shona, Remha (Zambia), Chichewa, Rundi, Kinyarwanda, Luginda, Kikuyu
Kannada
Ewe, Sindhi, Basque, Greek, Odia (Oriya), German, Gejaaa, Bulgarian, Chhattisgurhi, South Azerbuijani, Hindi, Sinhala, Bengali, Sanskrit, Egyptiia Arabic, Amharic, Marathi, Tamil, Telugu, Malayalam
Kanuri
Tsonga, Tunisian Arabic, Norwegian Nynorsk, Khmer, Dutch, Urdu, Macedonian, Lingala, Ewe, Fijian, Dinka, Odia (Oriya), Faroese, Marathi, Belarasian, Wolof, Tigrinya, Banjar in Arabic scripi, Mesopulariian Arabic, Estonian
Kanuri in Arabic script
Urdu, Uzbek. Persian, Odia (Oriya), Tonga, Kashmiri, Irish, Achinese, Maori, Duri, North Levantine Arabic, Slovak, Lingala, Kikuyu, Banjar in Arabic script, Banjar, Mandurin Chinese, Telugu, Kyrghyz, Hlccano
Kashmiri
Uyghur, Assamese, Kazakh, Odia OOriya), Bhojpuri, Sinhala, Maithili, Urdu, Kyrghyz, Bengali, Gujarati, Tijik, Awadhi, Nepali, Hindi, Sinthi, Punjahi
Marathi, Magahi, Sanskrit, Uyghur,
Kashmiri in Devanagari script
Marathi, Magahi, Sanskrit, Uyghur, Assamese, Kazakh, (OriaO) Bhojpuri, Sinhala, Maithili, Urdu, Kyrghyz, Bengali, Gujaruti, Tijik, Awadhi, Nepali, Hindi, Sinthi, Punjabi
Kazakh
Kurdish (Kurmanji), Sindhi, German, Armenian, Bulgarian, Georgian, Dengali, Egypttia Arabic, Ambaric,
Punjabi, Russian, Turkish, Kashmiri, Tajik, Tatar, South Azerbaijani. Uyghur. Tarkmen, Bashkir, Kyrghyz
Khmer
Kachin, Javamese, Basque, Myanmar (Burmese). Greek, German, Malay, Cantonese, Bulgurian, Minanglabau, Shan, South Azerhuijani, Achinese, Fgyptian Arbiic, Amharic, Rengali, That, Santali, Lan, Viernamese
Kikuyu
Chokwe, Tsonga, Towan, Swati, Sesotho, Amburic, Lingala, Somali, Sepedi, Lubu-Lalum, Nuer, Zulu, Shoma Lao, Bemha (Zamhia), Chichewa, Rundi, Kinyarwanda, Luganda, Kamha (Kenya)
Kimbundu
Irish. Chhattisgathi, Swahili. Nepali. Kongo, Pashto, Tunisian Arabic, Norwegian Nynorsk, Uzbek, Xhosa, Bemba (Zambia), Towana, Kashmiri in Devanugari scripi, South Azerbaijani, Kazukh, Azerbuijani, Kinyarwanda, Javanese, Morrocan Arabie, Latvian
Kinyarwandu
Unbundu, Bonga, Tswana, Sango, Swati, Sesotho, Nuer, Sepedi, Chokwe, Zulu, Luo, Lingala, Lubu-Laiee Lulun, Shona, Bemba (Zambia), Chichewa, Kamha (Kenya), Kikuyu, Luganda, Rundi
Kongo*
Bosnian, Serbian, Kashmiri, Kyrghyz, Arabic, Waray (Philippines), Amharic, Dutch, Tamazighht Marathi, Luba- Lulua, Umbundo, Mesopotamian Arabic, Samoan, Najdi Arabic, Achinese, Zalu, Tsonga, Indonesion, Balinese
Korean
Finnish. Lao, Ewe, Cebuano. Basque, Kachin, Greek, Vietnamese, Waray (Philippincs), German, Pangasinan, Bulgarian, llocano, Cantonese, South Azerbaijani, Bengali, Mandarin Chinese. ligytiinn Arabic, Amburic, Japanese
Kurdish (Kurmanji)
Awadhi, Turkmen, Assamese, Hebrew, Odia (Oriya). Nepali, North Levantine Arabic, Arabic, Sinbala, Nujdi Arabic, Punjahi, Kashmiri, Armenian, Georgian, Hindi, Bengali, Mesopotamian Arabic, South Azerbujjani, Tajik, Somai Kundish
Kyrghyz
German, Hindi, Nepali, Bulgarian, Sindhi, Bengali, Egyptian Arabic, Ambaric, Awadhi, Punjabi, Russian, Turkish, South Azerhaijani, Kashmiri, Tajik, Tatar, Turkmen, Rashkir, Uyghur, Kazakh
Lao
Achinese, Ewe, Busque, Minungkabau, Meiteilon (Manipuri), Greek, Mizo, German, Kachin, Bulgarian, Myan- mar (Burmese), Cantonese, South Azerhuijani, Egyptian Araic,, Amharic, Khmer, Vietnamese, Bengali, Shan, Thai
Laigalian!
Indomesian, Kinyarwanda, Nuer, Telugu, Finmish, Polish, Bulinese, Arabic in Latin script, Turkish, Sesotho, Cehuano, Tronga, Kamba (Kenya), Awadhi, Magahi, Hungarian, Achinese, Tunisian Arabic, Malayslam, Occitan
Latvian
Macedonian, Norwegian Nynorsk, Bosnian. German, Danish, Norwegian, Finnish, Serbian, Swedish, Slovak, Russian, Slovenian, Croatian, Estonian, Bulgarian, Ukrainian, Czech, Belarusian, Polish, Lithuanian
Ligurian
Romanian, Bosnian, Basque, Croatian, Papiamento, Esperanto, Asturian, Luxembougishh Galician, Sicilian, Slovenian, Portuguese, Iuitian Creule, German, Spunish, Italian, Catalan, Occitan, French, Venetian
Limburgan!
South Azerbaijani, Merrocan Arabic, Albanian, Tok Pisin, Sinbala, Assamese, Sundanese, Khmer, llocano. Georgian, Somali, Sorani Kurdish, Tatar, Kabuverdianu, Irish, Romanian, Turkish, Latgalian, Kongo, Telugu
Lingala
Sesotho, Yoruba, Luo, Shona, Sepedi, Hausa, Bemba (Zambia), Nucr, Igbo, Zala, Chichewa, Kamba (Kenya), Sango, Kikuyu, Rundi, Luganda, Umbundu, Kinyarwanda, Chokwe, Luba-Lulua
Lithuanian
Macedonian, Norwegian, Romanian, Bosnlan, Hungarian, Danish, German, Swedish, Serbian, Russian, Estonian, Slovenian, Croatian, Bulgarian. Slovak, Ukrainian, Belarusian. Czech, Polish, Latvian
Lombard!
Sanskrit, Tajik, Bashkir, Myanmar (Burmese), Armeninn, Spanish, Sepedi, Kyrghyz, Uyghur, Xhosa, Dzongkha, Lithoanian, Kamba (Kenya), Urdo, Bocano, Haltian Croole, Mattblli, Bhojpuri, Indonesian, Dutch
Laba-Lulua
Swati, Nuer, Tsonga, Sesotho, Tswana, Lao, Sepedi, Sango, Zalu, Shoma, Bemha (Zambia), Kamba (Kenya), Kikuyu, Rundi, Chichewa, Luganda, Kinyarwanda, Umbundu, Chokwe, Lingala
18

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Luganda
Tsonga, Tssaan, Sango, Amharic, Swati, Chokwe, Sesotho, Sepedi. Nuer, Zulu, Lingala, Shona, Luo, Luba- Lulu,, Bemba (Zambia). Chichewa, Kamba (Kenya), Kikuyu, Kinyarwanda, Rumdi
Luo
Chichewa, Finnish, Ews, Labe-Lulua, Basque, Somali, Groek, Bemba (Zambia). German, Bulgarian, Bengali, Kinyarwanda, Kamba (Kenya), South Azerbajaani Rundi, Egyptian Arabic, Luganda, Kikuyu, Ambaric, Nuer
Laxembourgish
Bulgarian, Welsh, Bengali, Slovenian, Venetian, Swedish, Czech, Norwegian, Farosse Ligurian, Icelandic, Occitan, Yiddish, Tok Pisin, Afriksans, Esperanto, French, Danish, Dutch, German
Macedonian
Latvian, German, Sicilian, Lithuanian, Italian, Belarusian, Hungarian, Romanian, Polish, Russian, Czech, Slovak, Alhamian, Ukrainian, Greek, Slovenian, Bosnian, Croatian, Serhian, Bulgarian
Magahi
Myanmur (Burmeeee) Meiteilon (Manipuri), Gujarati. Mizo, Marathi, Samskit, Tibetan, Sinhula, Chhattsgarhi, Kashmiri, Santali, Punjahi, Assamese, Hindi, Awadhi, Odia (Oriya), Nepali, Bengali, Bhejpuri, Maithili
Maithili
Myanmar (Burmese), Marathi. Gujarati, Mizo, Meiteilon (Manipuri). Sanskrit, Chhattisgurhi, Sinhala, Tibetan. Kashmiri, Santali, Punjahi, Odia (Oriya), Assamese, liiddi, Awadhi, Nepali, Bengali, Bhujpuri, Magahi
Malagasy
Sesocho, Buginese, Balinese, Kamba (Kenyal, Bemba (Zambia). Samoan, Indonesian, Sepedi, Shena, Ilocano. Aftikaans, Fijian, Swati, Achinese, Zulu, Sundanese, Tsonga, Maori, Chichewa, Javanese
Malay
Vietnamese, Thai, Lao, Samoan, Malagary, Fijian, Maori, Khmer, Pangasinan, Waray (Philippines), Hlccano Cebuano, Minongkabuu, Achinese, Buginese, Sundanese, Balinese, Javanese, Indonesian, Banjar
Malayalam
Ewe, Basque, Magahi, Greek, Odia (Oriya), German, Gujanti, Bulgarian, Chhattisgarhi, Sanskrit, South Azer- baijani, Hindi, Marathi, Egyptian Arabic, Amharic, Rengali, Sinhala, Telugu, Kannada, Tamil
Maliese
Occitan, Tigrinya, Ligurian, Amburic, Venetian, Craatian, Hehrew, Macedonian, Najdi Arabic, Ta'izzi Ademi Arabic, Besnian, Arabic, Italian, Mesopetamian Arahic, North Levantine Arbbic, Alhanian, Egyptian Arabic, Sicilian, Kabyle, Tunisian Arabic
Mandarin Chinese
Pangusinan, Greek, Japanese. German, Bulgarian, South Azerhaijani, Lan, Egyptian Arabic, Assamese, Amharic, Shan, Vietnamese, Korean, Bengali, Mizo, Myanmar (Burmese), Tibetan, Metteilon (Manipar)), Kachin, Can- tonese
Maori
Amharic, Khmerr, Japanese, Minangkahuu, Malugasy, Malay, Pangnsinan, Tok Pisin, Waray (Philippines), Banjar, Achinese, llocano, Cobuano, Javanese, Buginese, Indonesian, Sundanese, Balinese, Samean, Fijian
Marathi
Bhojpuri, Urdu. Assamese, Maithili, Tamil, Magahi, Nopali, Kannada, Kashmiri, Sindhi, Telugu, Awadhi, Chhattisgarhi, Bengali, Punjabi, Sinhala, Odia (Oriya), Gujarari, Sanskrit, Hindi
Mettcilon (Manipuri)
Nepali, Bhojpuri. German, Magahi, Bulgarian, Maithili, Odia (Oriys), South Azerbaijani, Shan, Egyptian Arabic, Amharic, Santali, Mantarin Chinese, Cantonese, Assamese, Bengali, Tibetan, Kachin, Myunmar (Burmese). Mizo
Mesopotamian Arabic
Turkmen, Greek, Hausa, Turkish, Bulgarian, Amharic, Armenian, Georgian, Wizzi-ddeni Arabic, Tigrinya, Tunisian Arahic, Kurdish (Kurmanji), Maltese, Sorani Kurdish, South Azerhaijani, Hebree, Egyptian Arabic, Arabic, Najdi Arabic, North Levantine Arabic
Minangkabou
Buginese, Samoan. Vietnamese, Malagasy, Shun, llccano, Fijiun, Mauri, Myanmar (Burmese), Sinhula, Balinese, Lao, Khmer, Thai, Banjar, Indomesian, Malay, Javanese, Sundanese, Achinese
Minangkabau in Arabic
Buginese, Samean. Vietnamese, Malagasy, Shan, Hocano. Fijian, Mauri, Myanmar (Burmese), Siahala, Balinese, Lao, Khmer, Thai, Banjar, Indonesian, Malay, Javanese, Sundanese, Achinese
script
Mizo
Nepali, Bhojpuri, German, Magahi, Bulgarian, Maithili, South Azerbaijani, Odia (Oriya), Egyptian Ara- bic, Amburic, Shan, Mandarin Chinese, Santali, Assamese, Cantonese, Tibelan, Bengali, Kachin, Myanmur (Burmese), Meiteilon (Maniguri)
Mongolian" 1
Chiuttisgarhi, Welsh, Kachin, Norwegian, Marathi, Punjabi, Catalan, Kabiyě, Magahi, Tibetan, Umbundua, Faroese, Cantonese, Armenian, Russian, Doongkha, Georgian, Turkmen, Egyptian Arahic, Shan
Morrocan Arabic
Bulgarian, Turkish. Tamasheq. Somali. Hausa, Armenian, Georgian, Tunisian Arabic, Kurdish (Kurmanji), Amburic, Sorani Kurdish, Maltese, South Azerbuijani, Tigrinys, Ta Leiizii-an Adeni Arabic, Hebrew, Egypian Arabic, North Levantine Arabic, Najdi Arabic, Mesopotamian Arabic
Mossi
Kinyarwanda. Tunisian Arabic, Kamba (Kenya), Luganda. Wolef, Lubu-Lulun, Kikuyu, Hausa, Bambara. Chichewa, Zulu, Tamasheq, Dyula, Lingsla, Igho, Yoruba, Sango, Fon, Ewe, Kabiyè
Myanmar (Burmese)
Greek, Thai, German, Magahi, Bulgarian, Maithlli, South Azerbaijani, Santali, Egyptian Ambbc,, Amharic, Odia (Oriya), Mandarin Chinese, Assamese, Shan, Cantonese, Bengali, Tibetan, Kachin, Meiteilon (Manipuri), Mizo
Najdi Arahic
Tamasheq, Bulgarian, Somali, Turkish, Haus, Armenian, Georgian, Kurdish (Kurmanji), Tunisian Arabic, Amharic, Sorani Kurdish, Maltese, South AzerbalJani, Tigrinya, Ta izzi-Adoii Arabic, Hebrow, Egyptian Arabic, North I Lvantine Arabic, Arabic, Mesogotamian Arabic
Nepali
Mizo, Sinchi, Mettellon (Manipuri), Gujarari, Marari,, Chhattisgurhi, Sunskrit, Tibetan, Santali, Sinhala, Kashmiri, Punjabi, Bhojpuri, Odia (Oriya). Assamose, Maithili, Hindi, Awadhi, Bongali
Sinhal, Magahi,
North Levantine Arabic
Somali, Hausa, Georgian, Greek, Tigrinya, Amharie, Armenian, Bulgarian, Kurdish (Kurmanji), -zzziii-
Arable, Sorani Kurdish. South Azerbaijani, Turkish, Tunisian Arabic, Maltese, Arabic, Najdi Arabic, Hobrow Mesopotamian Arabic, Egyptian Arabic
Norwegian
Irish, Bulgarian, Polish, Bengali, Scottish Gaelic, Yiddish, Finnish, Tok Pisin, Lithaanian, Afrikaans, Latvian, Estonian, Luxembourgish, German, Norwegian Nynersk, Ieelandic, Dutch, Faroese, Danish, Swedish
19

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Norwegian Nynorsk
Dusch, Ewe. Litbuanian, Basque, Irish, Scottish Gaelic, Greek, Latvian, Faroese, Danish, Bulgarian, German, Bengali, South Azerbuijani, Estonian, Egyptiin Arabic, Swedish, Amburic, Norwegian, Finnish
Nocr
Finnish. Ta izzl-Adeni Arabic, Somali, Ewc, Basque, Sango, Greek, Kamba (Keaya), German, Kinyarwanda,
Rundi, Bulgurian, Bengali, Tigrinya, South Azerbajjani Kikuyu, Luganda, Egyptian Arabic, Amburic, Luo
Occitan
Kabyle, Romamian, Croatian, Slovenian, Sicilian, Basque, Haitian Creole, Esperanio, Luxernbourgish, Papia.
mento, Asturian, German, Galician. Portuguese, Italian. Venetian, Spanish, French, Ligurian, Catalan
Odia (Oriya)
Tibelan, Myammur (Burmese), Meiteilon (Manipuri), Gujarati, Marathi, Mizo, Sanskrit, Sinhala, Punjabi, Kush- miri, Santali, Chhattisgarhi, Rhojpari, Awadhi, Hindi, Magahi, Nepali, Maithili, Assamese, Bengali
Oromo
Filipino, Shan, Tunisian Arabic. Tibetan. Mongolian, South Levantine Arabic, Crimean Tatar Latin script. Kongo, Loba-Lulua, Silesian, Lingala, Ligurian, Kinyarwanda, Meitollon (Manipuri), Latvian, Lao, Turkmen, Egyptian Arabic, Maori, Maithili
Pangasimun
Thai, Samsun, Malagasy, Balinese, Khuner, Indonesian, Lao, Fijiun, Achinese, Vielnamese, Malay, Caniomese, Sundanese, Maori, Banjar, Buginese, Waray (Philippinsss, Cobuano, Javanese, llocano
Papiamento
Dyula, Sieilian, Italion, Ligurian, Venetian, Ioelandic, Irish, Bambura, Occilan, French, Wolof, Catalan, Aymara, Yidlish, Ayacucho Quechua, Spanish, Asturian, Portuguese, Haitian Creole, Galician
Pashto
Kongo, Malagasy, Kabiye, Galician, Belarusian, Sinhala, Mossi, Korean, Sorani Kurdish. Friulian, Tatar, Tunisian Arable, North Levantine Arabie, Japanese. Luba-Lulua, Malay, Xhosa, Swati, Sanskrit, Mandarin Chinese
Persian
Luxembourgish, Wolof, Ukrainian, Bengali, Sesotbo, Spunish, Tamashey in Tifinugh script, Scottish Gaelic, Tamazight, Telugu, Marathi, Laba-Tulua, Sundanese, Ruginese, Italian, Ligurian, Kushmiri in Devanagari script, Nuer, Chichewa, Silesian
Polish
Swedish, Bengali, Venetian, Macedoonin, Romanian, Danish, Bosnian, Hungarian, Russian, Bulgarian, Latvian, German, Serbian, Croatian, Lithuanian, Slevenian, Ukminian, Belarusion, Slovak, Czech
Portuguesc
Romanian, Luxembourgihh, German, Welsh, Sicilian, Irish, Kabyle, Haltian Croole. Esperanto, Italian, Venctian, Papiamento, Basque, Ligurian, Occitan, French, Catalan, Spunish, Asturian, Galician
Punjabl
Kyrghyz, Santali, Tajik, Chhattisgarhi, Assamese, Marathi, Sinhala, Odia (Oriya), Magahi, Sanskrit, Bongali, Bhojpuri, Urdu, Maithili, Gujarati, Awadhi, Nepali, Hindi, Sindhi, Kashmiri
Romanian
Albonian, Bosnian, Sieilaa, Croatian, Occitan, Macedonian. Haitian Creole, Slovak, Galician, Hungarian, Venetian, Italian, Catalan, Spanish, Portuguese, Serbian, Ukrainian, Greek, French, Bulgarian
Rundi
Xhosa, Sango, Tronga, Tswana, Swati, Sesotho, Sepedi, Nuer, Chokwe, Zulu, Lingala, Luo, Luba-Lalua, Shona, Bemba (Zambia), Chichewa, Kamba (Kenya), Kikuyu, Luganda, Kinyarwanda
Russian
Georgian, Bosnian, Latvian, Armenian. Serbian, Kashmiri. Slovenian, Turkmen, Polish, Tajik, Tatar, Croatian,
Kazakh, Czech, Kyrghyz, Bulgarian. Uyghur, Ukrainian, Bashkir, Belarusian
Samoan
Cantonese, Korean, Minangkabsu, Malagasy, Malay, Japanese, Achinese, Banjar, Tok Pisin. Pangasinan, Indone- sian, Sundanese, Waray (Philippines) Javanese, Balinese, Buginese, Ilocano, Cebuano, Maori, Fijian
Songo
Sepedi, Chokwe, Lue, Kamba (Kenya), Chichewa, Zulu. Rundi, Nucr, Mossi, Hausa, Fon, Kikuyu, Luba-Lalua, Luganda, Kinyarwanda. Eww, Yorubu, Kabiyé, Lingula, Igho
Sonskrit
Tamil, Urdu, Assamese, Bhojpuri. Maichili, Kannada, Magahi, Sinhaln, Kashmiri, Sindbi, Tolugu, Nepali, Bengali, Chhattisgarhi, Punjabi, Odia (Oriya), Awadhi, Gujarati, Marathi, Hindi
Sontali
Awadhi, Mettellon (Manipuri), Basque, Greek, Mizo, German, Tibetan, Nepali, Bulgarion, Odia (Oriya), South Azerbaijani, Assamese, Bbojpuri, Egyptian Arabic, Ambaric, Magahi, Vietnameee, Maithili, Khmer, Bengali
Sardinian 1
Creole,Chichewa, Chichewa, Armenian, Occitan, Tumbukn, Gujarati, Bemba (Zam- bia), Umbundu, Mizo, Mesopotamian Arabic, Tunisian Arabic, Shan, Punjabi, Maltese Catalan, Kabiyč, Lux- embourgish
Friulian, Kashmiri, Assamese, Haitian Creole,
Scottish Gnelic
Lithuanian, Swedish, Luxembourissh, Kashmiri, Norwegian Nynoesk, Armenian, Hindi, Esperanto, Greek, Norwegian, Danish, Dutch, Bulgarian, Faroese, Bengali, German, Ioelandic, French, Welsh, Irish
Sepedi
Lingala, Malagasy, Umbundu, Luba- Luba-Lulua Luganda, Chokwe, Rundi, Kamba (Kenya), Kikuyu, Kinyarwanda. Afrikaans, Bemba (Zambia), Shona, Chichewa, Xhosa, Tooga, Tswana, Sccotbo, Swati, Zulu
Serbian
Latvian, Italian, Venetian, Lithuanian, German, Belarusian, Russian, Albanian, Hungarian, Polish, Romanian, Czech, Slovak, Greek, Ukrainian, Slovenian, Croatian, Bosnian, Macedonian, Bulgarian
Sesotho
Lingala, Malagasy, Umbundu, Luba-Lalua. Chokwe, Luganda, Rundi, Kamba (Keuya), Afriksans, Kikuyu, Kinyarwanda, Bemba (Zambia), Shoma, Chichewa, Toonga, Xhosa, Tswana, Zulu, Swati, Sepedi
Shan
Finnish, Santali, Ewe, Basque, Tibetan, Greek, Victameee, German, Bolgarian, South Azerbaijani, Assamese, Meiteilon (Manipuri), Ligyptian Arabic, Amharic, Kachin, Myanmar (Burmese), Bengali, Lao, Thii
Shona
Luo, Lingala, Umbundu, Laba-Lulua, Chokwe, Xhosa, Rundi, Luganda, Kamba (Kenya), Afrikaans, Kikuyu, Kinyarwanda, Sesotho, Swati, liwana, Bemba (Zambia), Isonga, Sepedi, Zulu, Chichewa
Sicilian
Greek, Asturian, Bulgarian, Croatian, Kabyle, Haitian Creole, Mucedomian, Galician, Bosmiam, Spanish, Albu- nian, Portuguese, French, Tunssia Arabic, Maltese, Ligurian, Occitan, Catalan, Venetian, Italian
20

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Silesian'
Chhattisgarhi, Scottish Gaelie, Morrocan Arabic, Banjar Arabic script, Haitian Croole, Japanese, Koago, Docano, Aymarn, Venetian, Telugu, Guarani, Latgalion, Hungarian, Tigrinya, South Azerbaijani, Sardinian, Gujarati, Luo, Sanskrit
Sindhi
Turkmen, Magahi, Telugu, Shojpuri, Assamese, Chhattisgarhi, Maithili, Tajik, Odia (Oriya), Sinhala, Benguli,
Nepali, Marathi, Sanskrit, Awadhi, Urdu, Gujarati, Hindi, Kashmiri, Punjahi
Sinbala
Achinese, Minangkabau, Maithili, Magahi, Awadhi, Assumese, Nepali, Telugu. Punjabi, Kannada, Kashmiri, Chhattisgarhi, Malayalam, Tamil, Gujarati, Sunskrit, Marathi, Bengali, Odia (Oriya), Himdi
Slovak
Italian, Bengali, Greek, Latvion, Venctian, Romanian, Lithuanian, Macedonian, Russian, Hungarian, German, Belarusian, Bosnian, Serbian, Bulgarian, Ukrainian, Slovenian, Croatian, Polish, Czech
Slovenian
Latvian, Lithuanian, Albanian, Occitan, Belarusian, Ligurian, Russian, Italian, Ukrainian, Hungarian, Macedo- nian, Venetian, Bulgarian. Polish, German, Slovak, Czech, Serbian, Bosnian, Croation
Somali
Bemba (Zambia), Kinyarwanda, Rundi, Tunisian Arabic, Luganda, Mesopotamian Arabic, Tamushey. Nuer, North Levantine Arabic, Luo. Maltese, Hebrew, Kikuyu, Kamba (Kenya), Hausa, Egyption Arabic, Arabic, Tigrinya. Ad'zziiAAeeni Arabic, Ambaric
Sorani Kurdish
Awadhi, Turkmen, Assamese, Hebrew, Odia (Oriya). Nepali, North Levantine Arabic, Arahic, Sinhala, Najdi Arabic, Punjabl, Kashmiri, Armenian, Georgian, Hindi, Bengali, Mesopotamion Arabic, South Azerbaijant, Tajik, Kurdish (Kurmanji)
South Azerbaijani
North Levantine Arabic, Hehrew, Greek, Amharic, Rulgarian, Ambic, Uyghur. Najdi Arabic, Mesopotamian Arabic, Armenian, Georgian, Sorani Kurdish, Kyrghyz, Egyptian Arable, Kurdish (Kurmanji), Tatar, Bashkir, Kazakh, Turkish. Turkmen
South Levantine Arabic
Kamba (Keaya), llocano. Dutch, Bemba (Zambia). Mossi, Norwegian, Somai Kurdish, Cebuano. Kyrghyz, Bambara, Turkish, Meiteilon (Manipuri), Kanmada, Samoan, Spunish, Sesotho, Crimean Tatar Latin scripi. Tsonga, Tamil, Bosnian
Spanish
Bengali, Romanian, German, Tunisian Arabic, Laxemboouriish, Experanto, Haitian Creole, Sicilian, Kabyle, Pupiamento, Venetian, Basque, Italian, Ligurian, French, Occitan, Catalan, Galician, Asturian, Portuguese
Sundanese
Vietnamese. Samoan, Lao, Malagasy, Thai, Pangasinan, Warry (Philippiness, Khmer, Fijian, Maori, Ilocano, Cebuano. Buginese. Minangkabau, Malay. Banjar, Javanese, Achinese, Balinese, Indonesian
Swahili"
Danish, Balinese, Thai, Irish, Yorbba, Arabic Latin script, Russian, Yiddish, Bosnian, Tumbuka. Waray (Philippines), Arabic, Malagasy, Korean, Pottuguese, Occitan, Sundanese, Indonesiun, Galiciun, Basque
Swati
Lingola, Malagasy, Umbundu, Luba-Lahm, Chokwe, Laganda, Rundi, Kamba (Kenya), Kikuyu, Kinyarwanda, Afrikaons, Bemba (Zambia), Shonn, Chichewa, Tswana, Sesotho, Xhosn, Taonga, Sepedi, Zula
Swedish
Bulgarian, Bengali, Czech, Polish, Yiddish, Belarusian, Tok Pisin, Finnish, Afrikaans, Lusemhourgish, Latvian, Norwegian Nynoesk, I Lithuanian, Icelandic, Estonian, Dutch German, Faroese, Danish, Norwegian
-iiezi-Adenn Arabic
Rundi, South Azerbujani, Tamashey, Hausa, Luganda, Lus, Kamba (Kenya), Kikuyu, Tunisian Arabic, Nuet, Maltese, North Levantine Arabic, Hehrew, Mesopotamian Arabic, Egyptian Arabic, Somali, Tigrinya, Arabic, Amharic, Najdi Arabic
Taiwanese Mandarin in Pangasinam, Greek, Japunese, German, Bulgarian, South Azerhaijani, Lan, Egyptian Anabic, Assamese, Amharic, Traditional script
Shan, Vietnamese, Korean, Bengali, Mizo, Myanmar (Burmese), Tihetan, Mesteilon (Manipuri) Kachin, Can- tonese
Tajik
South Azerhuijani, Assamese, Russian, Odia (Oriya), Sinhala, Uyghur. Gujarati, Turkmen, Urdu, Rengali, Sindhi, Sindli,, Kyrghyz, Awadhi, Nepali, Kazakh, Sorani Kurdish, Kurdish (Kurmanji), Hindi, Punjabl, Kashmiri
Tamasheq
Arabic, Igbo, Wolof, Mesopotamian Arabic, North Levantine Arabic, Somali, Yoruba, Ewe, Fon. Hebrew, Bambarn, Egyptian Arabie, Kabiyč, Amharic, Dyula, Mossi, Maltese, Tunisian Arabic, Kabyle, Hausa
Tamashoq in Titinagh
Arabic, Igbo, Wolof, Mesopotamian Arabic, North Levantine Arabic, Somali, Yoruba, Eue, Fon, Hebrew, Bambara, Egyptian Arabic, Kabiyè. Amharic, Dyula, Mossi, Maltss, Tunisian Arabic, Kabyle, Hausa
script
Tamazighe!
Estonian, Somali, Afrikaans, Kabyle, Samean, Punjabi, Indonesian, Buginese, Egyptian Arabic, Ieelandic, Magahi, Belarusian, Noewegian Nynorsk, Sango, Persian, Orema, Tumbuka, Norwegian, Umbundu, Kushmiri Devanagari script
Tamil
Ewe, Basque, Magahi, Greek, Gujarati, German, Odia (Oriya), Bulgarian, Chhattisgarhii Hindi, Sanskrit, South Azerbaijani, Marathi, Fgyptian Arabie, Amharic, Sinhala, Bengali, Telugu, Kannada, Malayalam
Tatar
Ukrainion, Bongali, Bulgarian, Georgian, Egyptian Arabic, Amharic, Armenian, Uyghur, Estonian, Lithuanian, Latvian, Belarusian, Finnish, Turkmen, Kyrghyz, Russian, Turkish, South Azerbaijani, Kazakh, Bashkir
Telugu
Ewe, Basque, Magahi, Greek, Gujarati, Odia (Oriya), German, Sinhala, Bulgarion, South Azerbaijani, Egyptian Arabic, Chhattisgarhi, Amharic, Hindi, Sanskrit, Bengali, Marathi, Malayalam. Tamil, Kannada
Thai
Cantonese, Ewe, Mettellon (Manipari), Basque, Greek, Kachin, Mizo, German, Achinese, Bulgarian, Minangk- abou. South Azerbaijani. Myanmar (Burnese), Egyptian Arabic, Amharic, Vietnamese, Khmer, Bengali, Shan, Lao
Tibetan
Odin (Oriya), German, Bulgarian, Awadhi, South Azerbaljani, Egyptian Arabic, Magahi, Amharic, Bhojpur), Mandarin Chinese, Nepali, Maithili, Santali, Cantonese, Assamese, Kachin, Myanmar (Burmese), Bengali, Mizo. Meiteilon (Manipuri)
21

---

Published as a conference paper at ICLR 2025
Target language
Auxillary languages
Tigrinya
South Azerbaijani, Rundi, Tamasheg, Kamba (Kenya), Hausa, Luo. Luganda, Kikuyu, Tunisian Arabic, Maltese, Nuer, North Levantine Arabic, Mesopotamian Arabic, Somali, Najdi Arabic, Egyptian Arabic, Arabic, Hebrew, Ta Ta'izzi-deen Adeii Arabic, Amharic
Tok Pisin
Indonesian, Danish, Pangusinan, Swedish, Norwegian, llocano, Malay, Faroese, Icelandic, Banjar, Luxembour- gish, Balinese, Yiddlih,, Fijian, Dutch, Waray (Philippines), Afrikaans, Buginese, German, Cebuano
Tsonga
Lingala, Malagasy, Umbundu, Luba-Lulua. Chokwe, Luganda, Rundi, Kamba (Kenya), Kikuyu, Kinyarwanda, Bemba (Zambia), Afrikamns, Shona, Chichewa, Sesotho, Xhosa, Tsaan, Sepedi, Swali, Zulu
Tswana
Lingala, Malagasy, Umbundu, Kamha (Kenya), Laha-Tulua, Kikuyn, Chokwe, Rundi, Luganda, Kinyarwanda, Bembu (Zambia), Alrikamns, Shona, Chichewa, Xhosa, Tronga, Swati, Sesotho, Zalu, Sepedi
Tumbuka
Papiamento, Odia (Oriya), Irish, Achinese in Arabic script, Kachin, Farosse, Cantonese, Ligurian, Banjar in Arabic script. Kimbundu, Bengali, Metciion (Manipuri). Fijian. Chokwe, Nuer, Morrocan Arabic, Hebrew, Mungelian, Afrikauns, Tswana
Tunisian Arabic
Hausa, Bosnian. Tigrinya. Amharic, Albanian, Hebrew, Spanish, Najdi Arabic, Occitan, Ta'izzi-Adeni Arabic, Ligurian, Italian Arabic, Mesopoiamian Arabic, Catalan, North Levantine Arabic, Sicilian, Egyptian Arabic, Kabyle, Maltese
Turkish
Alhanian, Georgian, Bengali, Armenian, Amharic, Serbian, Romanian, Uyghur, Turkmen, Tatur, Kazakh, Mace- donian, Hehrew, Bashkir, Kyrghyz, North Levantine Arabic, South Azerbaijani, Greek, Egyplian Arahic, Bulgar-
ian
Turkmen
German, Urdu, Bulgarian, Bengali, Mesopotamian Arabic, Egyptian Arabic, Kashmiri, Amharie, Armenian, Sorani Kurdish, Georgian, Tatar, Kurdish (Kurmanji), Turkish, Uyghur. Tajik. Bashkär, Kynghyz. Kazakh, South Azerbaijani
Ukrainian
French. Albanian, Bengali, Latvian, German, Lithuanian, Greek, Hungarian, Bosnian, Macedonian, Romanian, Russian, Slevenian, Croatian, Czech, Slovak, Polish, Serbian, Belarusian, Bulgarian
Umbundu
Sango, Swari, Sesotho, Kamba (Kenya), Igbo, Tsonga, Afrikaans, Kikuyu, Luganda, Rundi, Bemba (Zambla), Sepedi, Iswana, Kinyarwanda, Zulu. Shona, Chichewa, Lingala, Laba-Lulua Chokwe
Urdu
Magahi, Assamese, Odia (Oriya), Telugu, Bhojpuri, Maithili, Turkmen, Sinhala, Tajik, Bengali, Nepall, Marathi, Sanskrit, Chbastisgarhi, Sindhi, Awadhi, Kashmiri. Punjabi. Gujarati, Hindi
Uyghur
German, Bhoipari, Bulgarian, Tibetan, Egyptian Arabie, Amharic, Awadhi, Bengali, Mepali, Tatar, Punjabi, Russian, Turkish, Kashmiri, Tajik, South Azerbaijani, Bashkir, Turkmen, Kazakh, Kyrghyz
Uzhek
Rhojpuri, Hebrew, Fijian, Romanian, French, Tumbuka, Spanish, Irish, Ranjarin in Ambic script, Sundanese, Swati, Thai, Lao, Maori, Bulgarian, Finnish, Tamashoq in Tifinagh script, Slovak, Ayacucho Quechua, Danish
Venetian
Slovak, Papiamento, Hungarian, Romanian, Sicilian, Asturian, Czech, Galician, Bosnian, Spunish, Portuguese, Croatian, Haltian Creole. Slevenian, French, Catalan, German, Occitan, Italian, Ligurian
Vietnamese
Ewe, Meitetlon (Manipuri), Basque, Greok, llecano, Pangasinan, German, Myanmar (Burmase). Bulgarian, Kachin, South Azerhuijani, Shan, Cantonese, Fgyptian Ambic, Thai, Amharic, Lao, Santali, Bengali, Khmer
Waray (Phillipinos)
Minangkaban, Lao, Samoan, Khmer, Vietnamese, Malagasy, Cantonase, Fijian, Achinese, Maori, Balinese, Malay, Indenesian, Banjar, Sundanese, Pangasinan, Buginese, Javanese, llocano, Cebuano
Welsh
Lithuanian, Galician, Kashmiri, Icelandic, Armenian, Asturian, Basque, Hindi, Danish, Luxembourgish, Greek, Dutch, Bulgarion. Portuguese, Esperanto, Bengali, German, French, Scottish Gaelic, Irish
Wolof
Kamba (Kenya), Spanish, Galician, Sango, Kahyle, Luganda, Lingala, Hausa, Kikuyn, Chichewa, Tamasheq. Zulu, Dyula, Bambara, Mossi, Foo. Yorbba, Igho, Kabiyč, Ewe
Xhosa
Lingala, Malagasy. Umbundu. Luba-Lalua. Chokwe, Luganda, Rundi, Kamba (Kenya), Afrikaans, Kikuyu, Kinyarwanda, Bemba (Zambia), Shona, Chichewa, Tswana, Tsongn, Sopedi, Sesotho, Zulu, Swati
Yiddish
Bengali, Portuguese, Swedish, Asturian, Galician, Welsh, Danish, Scottish Gaelic, French, Tok Pisin, Luxem- bourgish, Papiamento,, Afrikamss, Norwegian, Hailian Creole, German, Irish, Dutch, Faroese, Icelandic
Yoruba
Sango, Rundi, Umbundu, Bambera, Tamashoq, Kamba (Keaya), Dyula, Hausa, Luganda, Mossi, Kinyarwanda, Kikuyu, Chichewa, Kabiye, Luba-Lalua, Zulu, Ewe, Fon. Lingala, Igbo
Zulu
Lingala, Malagasy, Umbundo, Luba-Lulua, Cholwe, Luganda. Rundi, Kamba (Kenyn), Kikuyu, Kinyarwanda, Attikaans, Bemba (Zambia), Shona, Chichewa, Iswana, Sesotho, Xhosa, Tsonga. Sopedi, Swati
Table 5: Auxiliary languages sorted from furthest closest, based on genealogical and geographic distance documented in URIEL repository (Littell et al. 2017). Languages marked with 't' are not included in the database, for which we sample the auxiliary languages in random. Languages without script notation are in the dominant script-Achinese in Latin script, Hindi in Devanagari script, etc.
22

---

Published as a conference paper at ICLR 2025
FLORES- devtest
NTREX
BLEU f (n=201)
BLEU 1 (s=198)
Win% Teacher
Wink vs.
Wink vs.
BLEU 1 (a=112)
Wink
Winth vs.
NLLB
NLLB 54B
teacher
1.38
1.38
PaLM2 (tcacher)
NLLB 133 distilled
16.9
40.8
7.0
18.7
24.1
NLLB 54B
19.4
55.2
92.9
baseline 11.8
11.9
35.3
18.2
12.6
9.2
10.7
6.2
mufa5
18.7
18.9
53.2
63.1
32.7
17.5
21.4
25.9
muful0
19.8
20.0
64.7
80.8
46.2
18.9
25.0
45.5
M2 XXS
mufu20
20.2
20.5
66.2
83.8
52.8
20.1
31.2
61.6
-NTL
mufuShul
muluStr 16.2
16.3
45.3
40.9
27.1
14.5
17.0
12.5
distilled
17.2
17.4
50.2
44.9
28.1
20.2
53.6
54.5
baseline
9.7
9.8
28.9
10.6
10.1
6.2
6.2
mufaO
14.9
15,1
32.8
27.8
18.1
15.1
12.5
15.2
PaLM2 XXS
mufuS
25.3
14.5
muful0
13.4
13.6
34,3
18.2
14.6
11.9
8.0
7.1
mufu20
baseline
2.9
2.9
8.0
3.5
2.5
2,7
1.8
2.7
mufa0
41.3
32.8
20.1
13.8
12.5
12.5
PaLM2 XS
mufuS
16.1
16.3
44.3
36.4
22.1
14.2
12.5
13.4
mufu20
16.1
16.3
45.3
34,8
21.1
14.1
11.6
13.4
baseline mufu20
3.9
3.9
15.4
4.0
3,0
1.8
S
18.5
18.6
56.7
59.1
31.2
16.1
18.8
25.9
mufa20lora
20.5
20.7
98.0
73.2
63.3
21.7
81.2
83.9
baseline
9.5
9.5
33.8
14.6
12.6
6.9
12.5
8.0
mufai
16.9
40.3
43.4
25.1
16.1
16.1
Gemma 2B
mufu5
17.4
17.6
45.8
51.0
28.6
15.3
17.0
17.9
mufal0
17.6
17.7
46.3
56.1
30.2
15.3
19.6
19.6
mufu20
17.7
46.8
53.0
15.5
20.5
mufa0 18.6
18.8
50.7
59.1
32.7
15.4
16.1
22.3
mulu5 19.1
19.3
56.2
67.2
36.2
18.8
20.5
mufu10
19.3
19.4
58.2
67.7
37.2
15.4
17.9
20.5
Gemma 7B
mufu20 19.6
19.7
59.2
71.2
39.7
15.7
19.6
23.2
mufuShrl 18.6
18.8
51.7
61.6
33.7
15.2
17.9
23.2
mufuStr
mufu20+55h Shrl
19.6
19.7
58.2
72.2
40.7
15.7
19.6
19.6
Table Mean BLEU scores, analogous to chrF scores reported in Table scores in given model class, Red values are win rates above 50%.
Bold values are the best
A.3 EXPERIMENTAL DETAILS
We perform full parameter updates for 25 epochs across all models, and select the final checkpoints with the best chrF scores for very-low- and low-resource languages over the validation split, which is partitioned from FLORES-22 200 devtest as described in Section[.1 All Gemma models are finetuned learning rate of 1e-5. We the initial learning rate to le-4 for PaLM2 models. When the models fail to converge, we reduce the rate to le-5 in the reruns. During evaluation, we greedily decode from the finetuned models and compute chrF based on the generated sequence and reference translation.
A.4 BLEU SCORES
We report mean BLEU and overall win rates against benchmarks in Table which is analogous to Table 2 in the main text. Figure 4 and Table 7 report Mufu's performance very-low- and low-resource languages, and analogous Figure and Table 3 respectively
23

---

Published as a conference paper at ICLR 2025
(a) Very low resource
(b) Low resource
PUL KT
XXS
X5
PAL22 Genmaa 23 Genma
(c) Medium resource
(d) High resource
PAL 235
K.T.
PREM Xxs 2 23
PA Camma 23 Gammaa 70
pastedt
cooked
Figure 4: Mean BLEU across languages of the same resource level, analogous to Figure 2 Note that the scales of y-axes different for the top and bottom rows. Error bars shown are 95% confidence intervals across the language pairs.
FLORES-200 devtest
NTREX
NLLLB
NLLB
NLLB
teacher
teacher
1.3B
54B
1.3B
baseline
61.2
24.8
14.9
35.5
6.5
66.4
PaLM2 XXS
mufuS
81.0
64.6
39.5
54.8
25.8
mufu20
92.2
75.2
50.9
64,5
41.9
distille
80.2
54.9
33.3
83,9
48.4
baseline
64.7
35.4
20.2
32.3
9.7
mufut)
77.6
50.4
37.7
35,5
19.4
Gemma
9.7
muful0
85.3
59.3
42.1
35.5
9.7
mufu20
87.9
64.6
44.7
38.7
12.9
distilled
76.7
47.8
30.7
58.1
45.2
Table 7: Win percentages by BLEU scores, analogous to Table measured over the 113 low and very-low resource languages for models shown rows against, columns, the teacher model, NLLB 1.3B distilled and NLLB 54B MoE. Win rates above 50% are red.
A.5
MUFU RESULTS BY LANGUAGE PAIRS
The full results (chrF) by language pairs for PaLM2 XXS-NTL and Gemma 7B finetuned mufu20 reported in T The models are mostly better than the teacher and NLLB 1.3B distilled when translating into languages classified very-low- or low-resource.
FLORES-200 200 deviest
NTREX
Pa
Pa1.M2
PaLM2 (teacher)
NLLB distilled
XXS
Gemma
PaLM2 S (leacher)
NLLB 1.3B
XXS
Gemma 7B (mufu20)
targot
resouroe
pL NTL (mufu20)
(mufu20)
distitled
pt. NIL
(mufuu20)
Achinese
VI.
31.8
40.7
47.6
46.7
Achinese in Arabic script
VI.
5.9
18.0
27.1
36.6
Afrikians
70.7
65.0
70.2
70.1
70.7
68.7
68.4
62.5
Albunian
Ambaric
L
41.3
37.0
39.6
35.9
26.4
26.6
25.4
21.4
24

---

Published as a conference paper at ICLR 2025
FLORES-200 devtest
NTREX
PaLM2
PaLM2
PaLM2 (leacher)
NLLB 1,3B
Gomma 7B
PaLM2 $ (leacber)
NLLB 1,3B
XXS
Gemma 7B (mulu20)
larget
resource
distilled
PL NIL
(mutu20)
distilled
NIL
(mufu20)
(mufo20)
Arabic
60.7
59.8
51.6
53.2
49.2
Arablc Latin script
VL
27.8
33.5
44.1
Armenian
M
58.7
52.5
56.8
56.7
53.5
50.2
51.5
47.7
Assamese
41.6
37.9
42.5
40.6
Awadhi
VL
50.8
49.3
56.0
59.5
Ayacucho Quechua
VL
23.6
28.0
38.3
32.8
Aymarn
Azerhaijani
M
47.8
45.0
46.0
43.8
49.0
48.2
46.0
41.6
Balinese
VI
40.5
48.3
53.8
51.2
Bambara
VL
10.6
32.1
31.9
28.8
Banjar
Banjar in Arabic script
VI.
14,5
17.5
30.3
36.6
Bashkir
47.4
48.3
51.7
50.0
39.6
42.0
42.9
39.3
Basque
Belarusian
M
45.7
43.2
43.8
44.5
54.4
54.5
50.0
45.6
Bemba (Zambia)
VL
35.8
37.9
42.0
39.0
37.1
40.9
41.2
36.9
Rengali
50.9
Bhojpuri
VI.
41.1
43.7
45.0
41.9
Bosnian
M
62.6
58.6
61.5
61.4
58.5
56.8
57.2
53.8
Buginese
VL
20.5
37.2
37.8
34.2
Bulgarian
68.3
57.6
Cantenese
M
40.1
18.0
38.3
31.7
26.2
18.1
24.9
22.1
Catalan
67.2
63.8
66.3
65.1
62.9
61.0
61.6
$2.0
Cehuano
Chhattisguri
VL
50.6
55.8
57.6
58.8
Chichewa
M
49.2
48.3
48.7
44.8
52.2
$1.0
50.5
44.8
Chokwe
9.2
25.7
17.8
27.2
Crimean Tatar in Latin
38.0
47.3
39.2
42.1
script
Czech
II 60.3
56.1
58.8
55.6
58.9
55.9
56.4
52.3
Dunish
II 71.1
65.0
69.3
69.2
64.1
60.5
63.0
63.1
Dari
54,9
53.2
54.3
49.3
44.3
42.6
43.7
36.4
23.8
Dutch
H
59.7
56.3
58.3
55.1
63.7
60.7
61.1
53.3
Dyula
VI.
8.0
18.4
21.3
Dxongkha
37.6
Egyptian Arabie
VL
49.1
47.9
51.2
48.3
Esperanto
M
63.4
62.7
62.8
63.6
Estonian
62.4
54.5
59.8
59.0
59.3
54.6
56.8
49.9
Ewe
VL
8.0
33.8
29.6
38.7
33.5
26.3
Faruese
L
46.0
45.8
49.6
48.1
48.7
50.5
51.9
44.8
Fijian
28.4
46.2
46.0
41.0
29.7
49.4
50.7
38.5
Filipino
64.0
63.4
61.5
53.0
Fiunish
H
61.1
53.8
58.1
$7.0
56.3
50.0
54.1
50.2
Fon
VL
4.2
20.0
20.1
18.0
French
73.1
69.7
Priutian
VL
49.2
57.1
56.5
54.2
Fulfulde
VL
5.7
23.8
21.8
24.1
6.0
27.6
22.0
22.0
Galician
62.5
62.6
61.8
63.7
62.6
62.6
59.1
Georgian
M
$4.1
48.4
52.2
52.2
49,8
45.5
47.2
44.4
25

---

Published as a conference paper at ICLR 2025
FLORES-200 devtest
NTREX
PaLM2
PaLM2
PaLM2 (leacher)
NLLB 1,3B
Gomma 7B
PaLM2 $ (leacber)
NLLB 1,3B
XXS
Gemma 7B (mulu20)
larget
resource
distilled
PL NIL
(mutu20)
distilled
NIL
(mafu20)
(mufo20)
German
67.1
61.5
62.1
60.8
Greek
M
54.4
52.3
53.7
54.3
59.4
57.6
49.7
Guarani
VI.
24.3
39.1
38.8
34,5
Gujarati
53.6
53.5
54.4
53.4
48.4
49.3
48.0
44.6
Нausa
L
52.9
51.8
51.5
49.8
54.1
54.1
51.9
45.5
Hebrew
M
61.6
57.0
59.5
58.0
54.2
51.5
51.7
47.1
59.7
51.0
Hungarian
M
57.8
53.5
56.2
55.9
49.7
46.2
47.7
42.0
Ieelandic
M
52.8
47.9
50.6
49.7
54.1
50.2
52.0
47.3
Igbo
L
42.4
41.8
41.3
38.8
47.6
48.0
45.2
37.3
Indonesian
M
72.3
69.0
71.4
70.6
67.4
65.0
66.5
62.9
Trish
58.7
53.8
56.2
58.4
55.0
51.7
52.2
48.9
Italian
Japanese
II
46.6
30.0
44.0
38.8
37.9
27.7
34,9
28.0
Javanese
L
$7.0
56.0
56.9
52.6
Kahiyè
Kabuvendiamu
VI
43.2
44.7
47.8
58.3
Kabyle
VL
15.2
32.1
32.7
31.4
Kachin
VL
14.0
37.5
39.9
35.9
(Kenya)
Kannada
M
56.0
55.2
54.8
54.9
52.2
53.0
50.8
44.1
Kanuri
VI.
10.6
25.2
27.2
24.7
script
19.4
Kashmiri
VL
16.9
37.1
36.6
34.3
Kashmiri in Devanagari
VL
13.6
18.7
26.6
29.2
script
Kazakh
M
58.1
50.1
56.9
57.1
48.9
45.2
48.4
43.7
Khmme
M
46.5
37.9
45.5
43.8
50,5
49.0
48.0
44.1
Kikuyu
35.5
Kimbundu
VI
13.6
28.5
31.2
35.1
Kinyarwanda
L
26.3
48.6
45.2
38.0
27.9
47.9
43.4
33.8
Kongo
VL
21.3
46.9
48.8
41.0
36.5
33.5
Kurdish (Kurmanji)
M
40.5
39.1
40.7
38.6
39.2
39.2
38.3
34.1
Kyrghyz
L
47.6
44.6
47.5
45.2
43.6
43.4
43.6
39.1
52.3
39.6
Latgalian
VL
31.6
48.1
50.5
46.9
Latvion
M
60.4
50.3
$8.0
$7.0
$2.8
45.9
50.5
49.4
Ligurian
45.2
48.5
54.2
Limburgan
VL
49.7
48.4
48.4
Lingala
L
27.1
49.6
49.7
45.8
Lithuanian
60.0
53.2
57.5
56.5
55.1
50.6
52.4
50.5
Lombard
36.3
38.9
40.6
Luba-Lulua
VL
15.0
37.5
38.3
31.9
Luganda
L
20.5
40.8
38.7
31.7
Luxembourgish
M
59.1
55.2
58.5
59.6
53.4
$2.5
51.2
49.4
Macedonian
M
65.0
60.3
63.1
62.5
62.7
60.2
60.6
59.8
Magahi
55.2
58.1
63.0
Maithili
L
50.8
48.9
58.7
61.5
26

---

Published as a conference paper at ICLR 2025
FLORES-200 devtest
NTREX
PaLM2
PaLM2
PaLM2 (leacher)
NLLB 1,3B
Gomma 7B
PaLM2 $
NLLB 1,3B
XXS
Gemma 7B (mulu20)
larget
resource
distilled
PL NIL
(mutu20)
(leacber)
distilled
NIL
(mafu20)
(mufo20)
Malagasy
57.6
52.7
52.1
49.8
43.4
Malay
70.2
66.7
66.8
66.2
63.6
65.1
65.6
Malayalam
M
58.1
50.4
55.8
55.5
49.6
44.2
47.7
45.9
Maltese
71.2
66.0
68.9
69.5
66.9
62.2
64.1
61.1
Chinese
Maori
48.2
47.4
48.8
48.7
51.8
49.5
50.9
45.0
Marathi
M
52.2
47.6
50.7
52.1
47.7
45.5
46.2
45.8
(Manipur))
39.2
Mesopotamian Arabic
52.2
48.4
53.6
53.4
Minangkabau
VL
51.1
52.0
57.4
55.0
Minangkabau Arabic
VL
16.8
34,8
44.8
Mizo
VL
19.7
38.0
38.2
33.9
Mongolian
51.4
49.4
40.2
44.5
36.1
Morrocan Arabic
42.7
40.7
43.4
42.2
Mossi
VI.
3.7
23.5
11.9
22.6
Myanmar (Burmese)
51.7
37.8
50.4
49.1
18.0
17.8
17.6
17.4
Najdi
Nepali
M
58.4
50.4
57.2
56.9
47.4
44.1
46.0
42.9
North Levantine Arabic
L
$2.6
49.3
57.8
59.9
Norwegian
Norwegian Nynorsk
M
61.4
53.6
61.6
63.2
60.3
53.8
60.4
51.9
Nuer
VL
6.9
28.7
28.3
26.1
Occilan
63.1
61.2
65.6
65.7
Odia (Oriya)
Orooo
VI.
17.1
39.1
40.0
30.4
17.2
35.4
33.6
26.9
Pangasinan
VI
31.3
48.5
48.3
40.7
Papiamento
59.4
Pashto
36.3
38.8
35.3
33.1
33.2
36.3
33.2
27.5
Persian
M
56.3
49.6
55.5
53.7
49.8
43.8
48.6
44.8
Palish
Partuguese
II
72.3
68.6
71.4
69.3
65.8
63.4
64.9
56.8
Punjabi
M
48.0
48.9
48.6
50.3
44.1
48.9
45.7
46.6
Romanian
65.9
60.5
64.9
63.0
60.3
55.4
58.8
54.3
Rundi
Russian
H
60.5
55.8
59.1
55.6
56.2
34.7
54.8
40.2
Samsun
L
53.1
48.6
55.2
51.5
54.6
53.1
52.7
43.7
Sango
12.1
Sanskrit
33.2
28.3
36.2
34.7
Santali
VL
11.4
16.8
37.7
Sardinian
VL
53.1
56.9
56.7
56.6
Scottish Gaclic
54.4
53.4
50.4
Sepedi
L
37.6
$1.1
54.7
48.7
35,2
37.4
35.1
31.7
Serbian
61.2
57.6
60.0
61.2
46.2
44.5
44.9
51.0
Sesotho
54.5
$4.0
Shan
VL
2.9
39.3
33.5
34.3
Shona
M
47.1
47.8
45.9
41.1
48.2
50.1
47.1
39.7
46.7
Sijesian
42.2
51.6
41.5
48.5
Sindhi
L
45.7
48.1
49.5
49.8
37.8
39.8
39.4
31,2
53.4
45.1
50.4
51.5
50.4
47.7
45.5
Slovak
M
62.0
60.5
$9.0
60.0
57.4
50.2
27

---

Published as a conference paper at ICLR 2025
FLORES-200 devtest
NTREX
PaLM2
PaLM2
PaLM2 (leacher)
NLLB 1,3B
Gomma 7B
PaLM2 $ (leacber)
NLLB 1,3B
XXS
Gemma 7B (mulu20)
larget
resource
distilled
PL NIL
(mutu20)
distilled
NIL
(mafu20)
(mufo20)
Slovenian
58.9
54.2
54.7
53.6
55.4
54.2
Somali
46.6
45.5
42.8
51.7
49.1
40.6
Sorani Kundish
L
44.3
48.7
45.0
44.5
41.5
45.3
41.1
34.6
South Azerbaijani
VI
28.1
26.7
35.7
32.7
Spanish
II
57.2
55.2
57.1
50.4
64.9
64.1
62.7
52.3
Sundanese
L
54.5
48.6
53.6
52.2
Swati
VI.
39.6
47.0
46.4
40.6
41.0
50.2
47.4
37.6
Swedish
II
70.6
64.8
69.3
69.8
67.0
64.1
65.8
59.1
Taiizii Ademi Arabic
VL
51.8
48.5
53.4
55.0
Taaiaaeee Mandarin
Traditional script
34.8
13.7
33.2
29.8
27.0
11.3
24.7
16.2
Tajik
52.3
49.2
43.9
42.3
39.8
Tamashoq
VL
4.3
23.7
17.7
24.8
Tamasheq in
Tifinagh
VL
6.8
17.5
27.2
script
Tamazight
VL
8.4
30.4
24.3
32.2
Tamil
M
59.5
56.6
57.6
58.7
48.8
48.3
47.7
47,8
Tatar
48.6
48.1
50.9
49.3
45.7
48.4
49.1
42.9
Telugu
45.5
Thai
H
$7.9
43.6
56.9
55.7
52.7
43.8
51.7
44.0
Thhetan
L
32.4
34.7
39.0
36.7
28.9
33.9
36.0
30.5
Tigrinya
23.3
Tak Pisin
L
41.5
41.7
54.2
54.3
Tsonga
L
19.4
$1.8
49.2
40.7
Tswana
L
37,9
49.3
48.3
41.2
39.8
54.5
48.2
38.3
Tumbaka
VI
24.3
36.3
39.9
34,9
Tunisian Arabic
VI.
45.0
40.8
47.5
48.2
Turkish
63.4
58.2
61.9
60.8
54.3
51.9
53.4
49.5
Ukrainian
M
60.8
54.5
58.9
58.6
54.7
51.5
52.7
52.6
Umbandu
VI.
9.8
28.0
24.2
32.0
Uyghur
L.
38.6
46.4
44.0
41.0
32.4
39.9
37.9
30.8
Uzbek
M
59.7
$4.1
58.7
57.1
46.8
45.8
46.0
41.6
Vemetian
L
49.3
50.1
54.2
53.7
61.4
60.2
Waray (Philippines)
VI.
$5.0
56.2
64.1
62.1
Welsh
73.1
63.9
70.2
72.3
62.2
$7.9
60.1
55.8
Wolor
Xhesa
51.7
52.7
50.0
47.7
48.7
49.2
48.0
43.6
Yiddish
L
$2.3
38.6
52.5
56,7
Yoruba
25.7
25.7
26.1
19.0
17.9
18.4
12.5
Zulu
M
55.9
56.7
54.6
53.9
55.5
56.8
53.9
48.6
Table ChrF by 201 language pairs in FLORES-200. VI., I., M and refer to very-low-, low-, medium- and high-resource languages respectively. Bold values are higher than both the teacher model (PaLM2 S) and NLLB 1.3B.
28

---

Published as a conference paper at ICLR 2025
FLORES devtest
NTREX
chfF (m=201)
chfF (u=198)
Win% Teacher
Wink vs.
Wink vs.
chrF f (a=112)
Wink
Winth vs. NLLB
NLLB
NLLB
teacher
1.3B
54B
1.38
baseline
23.6
All language pairs
postedit
38.6
38.7
23.4
10.6
1.5
36.8
mufa5
40.6
40.7
24.9
14.1
3.5
38.5
muful0
41.0
41.1
25,4
15.2
38.9
mufu20
38.8
38.9
24.4
12.6
3.0
37.1
6.2
1.8
baseline
28.2
28.2
37.9
3,5
0,9
24.0
19.4
0.0
Low-resource language
postodit
33.1
33.2
40.5
6.2
30.8
19.4
mufa5
35.3
35.4
43.1
8.0
31.8
22.6
pairs
muful0
35.8
35,9
44.0
9.7
4.4
32.2
25.8
mufu20
Table 9: Mean chrF of BLOOMZ 1B7 finetuned on Mufu, which is analogous Table 2 in the main text. Bold values are the highest chrF scores. Mufu models consistently translate better than baseline and postedit-only.
A.6 MUFU WITH BLOOMZ
Using the same Mufu prompts, we finetune BLOOMZ 1B7 and report the mean chrl across language pairs Table 18 The results corroborate our key findings in the main text. that Mufu-finetuna mod- els are consistently ahead of baseline and postedit-only and achieve the most competitve performance against the teacher low-resource languages.
A.7
MUFU SELF-ATTENTION
Tables 10 and IT analogous to Table 4 where the attention weights placed over the input by Gemma 2B (mufu5) are highlighted. The examples demonstrate that Mufu models are capable of overriding the postediting target accurately based semantic alignment across languages beyond orthographic mapping.
A.8 FAILURE EXAMPLE: BAD AUXILIARY INPUT
We identified few failure cases Section and attribute them partially to poor auxiliary candidates in Mufu input. For example,
English: Bird flu, or more formally avian influenza, can infect both birds and mammals. Automatic Luganda: Enfuba y'enyonyi, obu awamu ey' 'eenfuha d'eyynny, ey e'eyonyy een- solo eziyitibwa ennyama.
Automatic Kinyarwanda: Ibirori byamahoro, cyangwa uko byatangiye ibinyamurenge, by- alera indwara mu nyamaswa n' l'ibiddi binyabutabire.
Automatic Umbundu: "Oisiku Isika, tsiku tsiku, tsiku tsiku, tsiku tsiku, tsiku tsiku
Automatic Chokwe: Flu ndege, nhi cindji cindji cindji cindji cindji cindji cindji Automatic Luba-Lulua: Bu tshisuku tshia nsuku, bu tshisuku tshia nsuku tshia nsuku Automatic Lingala: Nzela ya nzoto, to ndenge ya ndenge ya nzoto ya nzoto, ezalaki kozala na nzoto mpe na ndenge ya ndenge nzota.
Note that Mufu models produce overall worse translations in Lingala than baseline, except for PaLM2 XXS-NTL (Table g) and PaLM2 XS.
#BI.OOMZ 1B7 model card, hattsss//angiiggaaeeeo/hhhggagga 1b7
29

---

Published as a conference paper at ICLR 2025
The English sentence has been translated into Malay, Sundanese, Javanese, Indonesiam, Minangkahuu and Achinese in Arabie script. These translations may contain errors. Correct the translation from English Achiaese Arabic script.
Imagine. if you will, Mancunian, Bostonian, Jamascan and Sydneysider sitting around table having dinner restaurant in Toronto.
English:
Bayangkan, jika anda maho, scorang Mancunian, Bostonian, Jamaican dan Sydncysider duduk sekellling meja makan di sebuah restoran di Foronto.
Automatic Malay:
Automatic Javanese: Mbayangno, yen sampeyan bakal, Mancunian, Bostonian, Jamaika Sydneysider lungguh ngubengi meja mangan nedha bengi restoran ing Toronto.
Automatic Sundanese: Bayangkeun, upami anjeun badé, aya Mancunian, Bostonian, Jamaika saleng Sydncysider aou calik di sabudeureun mája wang di réstoran di Toronto.
Automatic Indonesian: Rayangkan, jika Anda mau, seorang Mancunian, Bostonian, Jamaika Sydneysider duduk sekitar meja makan di schuah restoran di Toronto.
Automaticc Minangkabuu: Bayangkam, apubilo indak salah, urang Mancumian, Bostoniaa, Jamaiika jo Sydneysider duduak di sakitar meja makan di restoran di Toranta.
Automatic Achinese Arabic script: غعكي Stمt غعيي كفكوي افا ااكيي اطايي قطكي مlا مiu Lea o Seud
Corrected Ac
Arabie scripit باا
G Spha
reference
ق agglu c4 alla agelid basa مااود de كدوق ملر 3ن [5/laly Aeتونل Lالgنن مناييي aay geloo تاايرو ووووني a aggelaw
Nvykyr, nur Jadi, sydry mnjion, bustwny, juyy,, sydnysydr dug Bik dhwh myj vjoh B Miam Rik shouk smjjch D nerwonw.
mufu5
agula da ماك laya سامكك di Eaiz رزاي مديي اااlaعe الع Laios Eusl سنيt شاكك مقبد Le
slarsi de slسسsmel
Vry, iw nkh, sydry owr mujwnyy bokawsy, joyk soyrydly tre jow slykr nay mkn Bik sbwh ryustwrn Bik teruow:
baseline
sags egels cada la a bangogw Eual of gلاهls caanese Evol lossle Lssl eagha wie angso seho I5 تورططنس
Vykyr, mw drynh Jadu, owr mayn, owr busstin, awr owr sydays jk D kaj uwve ujwh Bik Rmah nerwins.
Table 10: Translations from English to Achinese in Arabic script and their romanized form by mufu5 and the baseline Gemma 2B models. جاممييكا is correctly transliterated from Jamaika in mufu5, which is attended by the model during its production and is absent in both the postediting target and the baseline translation. Tokens with aggregated attention values under .02, .0,,.1, 14, .24 are highlighted in white, light gray, dark gray and black respectively.
30

---

Published as a conference paper at ICLR 2025
The English sentence has been translated into Tibetan, Kachin, Bengali, Myanmar (Burmese), Meltellon (Manipuri)
translations may contain erross, Correct translation English Mizn.
English: minimum, you need [ootweel with suitable soles Summer shoes are usually very slippery on ice and snow, even some winter boots are deficient)
Automatic Tibetan: awwertoyy Biyccors oao Sesees Ser Geney www.figgggggggggggooooooooo wtwp://wwweeeeeeeeeeeeeoo Re $1 wewpeiiggggsssseeee
Automatic Bengali: waaee contita (ugg oftage अारा e कोययक सायरर गगक काक @ अिा.r 34F य2र Misees,, qwalf Ray Ilosidls goo wergel
Automatic Kachin: Hiuu sha sha sha sha sha sha sha sha sha shasha
sha sha sha sha sha sha sha sha sha
Automatic Myanmar (Burmese): megoii
saoorpappsloleap 8goup: ecsoodlagghi esepaserós oppooppa sgasc,
poogeeppseguSsie soqel can Sappasoppppp por all pooosellaa
Automnaiic Meiteilon (Manipwr):: लहाारर लяानाा, रलाममाी दनरारर लीाश उनरारर टяाशा लनरनरर сरोर दनापरर लीोता Zerars сйt टिायरर câtta टहारकर eяttu žनापरर eйtता žeाars
dlnng tlang, rawa tlang tlang chuan, kan dlang tlang rawn tlang tlang chuan, kan dang tlang rawn clang dling chuan, kan tlang tlang a rawn tlang tlang chuan!
MIZE tlang tlang
Corrected Mizo: Thil buaithlak
reference
A berah. pheikhowk bun chuan kepbah siam bik a mamawh aga. Nipui laia bun thin pheikhawkte bi chu vurah nal ilangpui thlasik laia bun thin pheikhowk thenkhatte pawh hi la tawk to cheu mi.
mufu5
Thil buaithlak berah veivah tak te pawh chiang lo nl.
botu ropui
po a ngai a, a luahna ring gyhooddw yd tak tlem lch
baseline
A tlem chean, foot hreuh tak hi thil rih chiing
sawh chuan hruuh zuah leh liah hi che.
Table 11: Translations from English to Mizo by mufu5 and the baseline Gemma 2B. The mufu5 model generates berah (the most) for the source word "minimum" as it attends to multiple auxiliary translations-some of which of low quality. The corresponding translations of the word in Bengali (बत(लल and Myanmar (30$2::ad:) are partially attended to by the model. Tokens with aggregated attention values under 001, 03, 10, .18 are highlighted white, light gray, dark gray and black respectively.
31


Reviews:
[{"review_id": "FZVEb6rgDR", "reviewer": "Reviewer_84wT", "summary": "This paper introduces \"Mufu\"  , which is a method for low-resource language translation using a multilingual fused learning approach, specifically targeting large language models (LLMs).\nThe Mufu method, which aims to address the challenge that large language models (LLMs) perform well in translating high-resource languages but still struggle with low-resource languages. The Mufu prompting approach turns the translation task into a post-editing task, leveraging the reasoning capabilities of LLMs with auxiliary translation candidates, requiring the model to assess input quality, align semantics cross-lingually, copy from relevant inputs, and override incorrect instances. Experiments show that LLMs fine-tuned with Mufu-style prompts achieve better performance than the NLLB 1.3B distilled model in 64% of low- and very-low-resource language pairs on the Flores-200 dataset.", "questions": "1、 more diverse set of low-resource languages in the experimental dataset will be helpful\n2、  the impact of various auxiliary languages can be deeply analyzed\n3、 prompt analyzation can be improved", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Interesting research, Introduces Mufu, a novel approach leveraging multilingual context and post-editing for low-resource language translation.\n2. Employs automatically generated candidates and instructions to correct translations, enhancing LLM's reasoning capability.\n3. Demonstrates robustness against poor-quality auxiliary translations, outperforming specialized NMT systems in many low-resource pairs.\n4. Proposes a hybrid learning paradigm, combining in-context learning and finetuning for improved translation quality.\n5. Implements knowledge distillation to reduce inference costs while maintaining performance gains in low-resource translations.", "weaknesses": "1. Experiment Method Optimization， Consider incorporating a more diverse set of low-resource languages in the experimental dataset to better generalize the findings and evaluate the model's performance across a wider linguistic spectrum.\n\n2. Experiment Conclusion Enhancement， Suggest conducting ablation studies to isolate the specific contributions of different components of Mufu, such as the impact of various auxiliary languages, to fine-tune the approach and maximize translation accuracy.\n\n3. 5-shot Prompting Improvement， Explore the use of meta-learning strategies in 5-shot prompting to enhance the model's ability to quickly adapt to new translation tasks with limited examples, potentially improving the efficiency of the learning process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jjsS6R3aXd", "reviewer": "Reviewer_Xmqh", "summary": "This paper tackles low-resource translation quality improvement in LLM models. To maximize data eﬃciency in the low-resource setting, the authors introduce a new approach called Mufu, including automatic selection of multilingual translation candidates and an instruction tuning to correct inaccurate translations via the prompt. Experimental results on Flores-200 dataset for English-XX directions show robustness and achieves better performance against NLLB 1.3B distilled model in 64% of low- and very-low resource language pairs.", "questions": "Please see the weaknesses for the questions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- experimental results show some effectiveness of the proposed approach\n- the idea of leveraging multilinguality via the prompt sounds technically good", "weaknesses": "-  unclear about the experimental results; how to decide the best prompt template for mufu; any impacts of language combination used in the prompt template - for example, have you ever tried adding high-resource language translation pairs during training to enhance multilingual training  with high and low-resource language pairs?\n-  results are not convincing enough, maybe due to low-resource setting with limited improvement in ChrF. Can you report other metrics such as sacreBLEU scores? Have you tried finetuning LLM with low-resource monolingual data so that the LLM can more effectively enhance Mufu.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iqbYhV2PPZ", "reviewer": "Reviewer_uix1", "summary": "This paper introduces Mufu, which turns translation into post-editing task by providing auxiliary translations and target translation from teacher model. The student model learns in-context to produce the correct target translation and is then fine-tuned against references. Languages for auxiliary translations are chosen from URIEL and they evaluate using PaLM S family models along with Gemma 2B,7B on FLORES 200 (iid), and NTREX (ood). The paper contains thorough ablation studies as well as cross lingual attention alignment which helps understanding or interpreting how model is learning through in-context.", "questions": "- Were there any accidental translations in a different language for Mufu{5,10,20}?\n- What exactly is Win% vs teacher? For instance, for NLLB 1.3B distilled, its chrF is 46.0 whereas that of teacher is 43.7, still its win% is 41.3? It means NLLB 1.3B was less than 50% correct when compared to teacher model still its chrF score is higher? Another example, Win% vs teacher is 56.2 for NLLB 54B MoE (48.9 chrF) whereas for mufulora20 with PaLM2 S it is 99% with chrF less than NLLB 54B MoE on FLORES 200. It will be great if authors can formalise what is Win% vs teacher.\n- Can the authors explain In theory… model outputs (line 207-211)?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is very clearly written and easy to follow.\n- They combine 2 interesting learning paradigms: ICL and parameter tuning and their core focus is on very-low and low resource language which I really liked. \n- They perform evaluation on NTREX which is important for ood evaluation.\n- The experiments performed by authors are quite extensive. I especially liked mufu5hrl, mufu5tr, distilled, and lora which corroborate their approach of selecting 5,10,20 related languages from URIEL.\n- Quantitative evidence provided in Figure 3 is quite helpful in knowing how language transfer is taking place. Moreover, the attention pattern further helps in understanding how attention pattern is making mufu models perform better.", "weaknesses": "- No model sizes available for PaLM2 family of models. I’m not sure how to compare them with Gemma or NLLB.\n- If I were to just compare on the basis of chrF score, only PaLM2 XXS -NLT and PaLM2 S are able to beat NLLB 1.3B distilled model in both FLORES 200 and NTREX (and Gemma 7B on FLORES 200). Rest all are inferior to NLLB 1.3B distilled. One suggestion for authors in this case will be to add `Latency` column for all models (higher for mufu and lower for distilled models) to show the trade off between accuracy and latency which will help readers understand how competitive other models are.\n- The authors have mentioned this but finetuning an LLM (or even NLLB with 1B+ param) with just 787 sentences and in-context learning will definitely lead to overfitting which is evident by the fact that mufu20lora performed better than full finetuning. I wonder if that is the case for other models too? \n- It’s great they used Gemma 2, an open weight model but I’m slightly disappointed that majority of their experiments use PaLM2 models which are not public like Gemma 2. \n- Two iteration process (teacher model followed by student model) is quite expensive. The authors have mentioned that distillation helps to alleviate the problem but it only worked for NTREX in PaLM2 XXS - NTL (not for Gemma 7B), performance on FLORES 200 for both distilled models is lower than NLLB 1.3B. \n- The authors experiment with one learning paradigm i.e., in-context learning for LLMs for distillation. Did they try distillation from model outputs (not the one fine-tuned with mufu20)? How much better or worse is in-context learning compared to vanilla distillation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)