DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models
Authors: Niyati Bafna, Emily Chang, Nathaniel Romney Robinson, David R Mortensen, Kenton Murray, David Yarowsky, Hale Sirin
Accepted at: acl25

DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models

Niyati Bafna1, Emily Chang2, Nathaniel R. Robinson1, David R. Mortensen3, Kenton Murray1, David Yarowsky1, and Hale Sirin1
1Johns Hopkins University, Center for Language and Speech Processing; 2University of Virginia; 3Language Technologies Institute, Carnegie Mellon University {nbafna1,nrobin38}@jhu.edu, echang22911@gmail.com

Abstract

Most of the world’s languages and dialects are low-resource, and lack support in mainstream machine translation (MT) models. However, many of them have a closely-related high-resource language (HRL) neighbor, and differ in linguistically regular ways from it. This underscores the importance of model robustness to dialectal variation and cross-lingual generalization to the HRL dialect continuum. We present DialUp, consisting of a training-time technique for adapting a pretrained model to dialectal data (M→D), and an inference-time intervention adapting dialectal data to the model expertise (D→M). M→D induces model robustness to potentially unseen and unknown dialects by exposure to synthetic data exemplifying linguistic mechanisms of dialectal variation, whereas D→M treats dialectal divergence for known target dialects. These methods show considerable performance gains for several dialects from four language families, and modest gains for two other language families. We also conduct feature and error analyses, which show that language varieties with low baseline MT performance are more likely to benefit from these approaches.1

1 Introduction

Recent years have seen advancement in MT quality and language coverage, with models such as M2M100 (Fan et al., 2021) and NLLB (Team et al., 2022) covering 100 and 200 languages respectively, as well as multi-purpose generative language models such as the GPT, BLOOM and Llama (Muenninghoff et al., 2023; Dubey et al., 2024) series expanding to dozens of languages. These models have displayed cross-lingual generalization capabilities for some unseen or low-resource languages, but they tend to suffer significant performance degradation for most (Jiao et al., 2023; Robinson et al., 2023; Ziem et al., 2023; Cahyawijaya et al., 2024; Joshi et al., 2024).

The roughly 7000 languages in the world can be grouped into a few hundred families. Closely-related languages (CRLs) and dialects within a family largely exhibit continuous and structured differences along phonological, morphological, and lexical dimensions, rather than being discrete monoliths (Hovy and Purschke, 2018; Bergman and Diab, 2022). Not only is it unfeasible to collect training data for every lect, but language is fluid and dynamic across and between dialect boundaries, calling for principled general approaches to dialectal variation.

Many language families have at least one HRL member that is supported by state-of-the-art models. We seek to expand models proficient in HRLs to their lower-resource CRLs, by inducing robustness to unseen varieties across language continua. We do this with DialUp: adapting models from an HRL to its language relatives, and adapting those relatives’ data inversely towards the HRL model, as in Figure 1.

We first aim to adapt the model to the language continuum, or model-to-data (M→D) adaptation

Figure 1: Two paradigms for robustness to dialects on a continuum of distances from an HRL. DialUp involves M→D: training the model on artificial dialectal variation, and D→M: bringing dialectal data closer to model expectations (HRL-like input) at inference.

1https://github.com/niyatibafna/dialup/
<table>
  <tr>
    <th>Turkic</th>
    <th>today</th>
    <th>dayna (tur)</th>
    <th>dayna (taz)</th>
    <th>dayna (Tuk)</th>
    <th>dayna (chm)</th>
  </tr>
  <tr>
    <td>Turkic [today]</td>
    <td>jodi a [hat]</td>
    <td>jodi a [hat]</td>
    <td>jordr [hat]</td>
    <td>osordi (crs)</td>
    <td>sordi (mle)</td>
  </tr>
</table>

Table 1: Cognates differ in predictable ways, via sound change patterns, vowel changes, and new suffixal paradigms.

(yellow arrows in Figure 1). The goal is robustness to unseen CRLs (both known CRLs absent in pre-training data and yet undocumented CRL varieties), using only HRL bitext. We do this by simulating dialectal variation patterns over HRL fine-tuning text, teaching the model to generalise to realistic variants such as actual CRL inputs.

Next, we aim to adapt data to the HRL model, or data-to-model (D→M) adaptation (red arrows in Figure 1), in the case that a HRL-CRL bilingual lexicon is available or can be induced. In this circumstance, we pull CRL data towards the model distribution at inference time by interchanging words for their HRL counterparts.

Note that cross-lingual generalization can be viewed as a train-test data mismatch problem: we want our models to work on dialectal data which differs in various ways from the training data of the model (HRL data). Our two approaches above then represent two broad paradigms for this general problem in machine learning (see Figure 1). Model-to-data adaptation has roots in existing cross-lingual transfer approaches, which approximate CRLs using an HRL, relying on linguistic similarities for transfer; however, this does not train the model to handle dialectal divergences. M→D innovates on this by synthetically approximating such divergences. By approximating language continua, we may avoid pressing questions raised by discrete language paradigms, e.g. which transfer language to use and how much training data to collect (Dalmia et al., 2018). D→M adaptation has roots in prior approaches to approximate a model’s expected domain in data (Nie et al., 2023).

These two approaches have different advantages. M→D does not require any CRL resources or data and does not require partitioning the continuum into discrete dialects. D→M is train-free and can be directly applied even to closed-source models. These approaches can also be used in tandem, by applying D→M at inference time to a model adapted via M→D. We expect these methods to be most beneficial for unseen CRLs and CRLs that have high linguistic overlap with the HRL, i.e. languages that depend on existing HRL representations in the pre-trained model. In sum, we contribute:

• DialUp, a principled and inexpensive method to induce robustness over language family continua via M→D and D→M adaptation.

• Evaluations of DialUp’s benefits for X→eng MT with two models for 49 CRLs across six language (sub)families.

• Consistent gains via M→D for low-baseline varieties across 4/6 language families (up to mean +11.4 BLEU for Romance languages)

• Gains from D→M adaptation over baselines for certain families and CRLs (yielding up to mean +12 BLEU for Indic dialects), showing for the first time that adapting dialectal function words is more beneficial than adapting content words

• Evidence that M→D and D→M combine advantageously, and provide a recipe for increasing the flexibility of existing MT models to general dialectal variation.

2 DialUp

2.1 Model-to-data (M→D)

In this paradigm, given an HRL-proficient model and bitext in that HRL, we adapt the model to unseen CRLs. We generate varied artificial CRLs of the HRL by simulating mechanisms of dialectal variation over the HRL bitext, and fine-tune on the resulting synthetic data to induce model robustness to dialectal variation.

Artificial language generation We employ the approach of Bafna et al. (2024b) to simulate language variation along phonological, morphological, and lexical (function word and non-cognate) content word dimensions. Each constituent process operates on the relevant units of the inputs: phonemes, suffixes, and function and content lexemes, and “noises” a given unit with probability \( \theta^n \), \( n \in \{p,m,f,c\} \), replacing all instances of that unit with a plausible alternative in the constructed language (i.e. phones in the original unit randomly replaced with alternate phones of high phonetic proximity, or suffixes swapped with similar-sounding alternatives).\footnote{We provide further details on each noiser in Appendix B.} \( \theta^n \) serves as a proxy for linguistic distance in each of these dimensions: increase or "dialing up" of \( \theta^n \) results in more
<table>
  <tr>
    <th>Using HRL text (top) to generate artificial dialect text (bottom) via noising</th>
    <th></th>
  </tr>
  <tr>
    <td>Indic</td>
    <td>विश्राम , तीन अधिकतम एक्सप्रेस हैं , जिन पर मैं अधिकतम वचन और भागीदारी का आग्रह करना चाहता हूँ।<br>विश्राम , तीन अधिकतम एक्सप्रेस हो जिन पर मैं अधिकतम वचन और भागीदारी गा आग्रह करना चाहते हूँ।</td>
  </tr>
  <tr>
    <td>Austronesian</td>
    <td>Beberapa proyek akan melalui langkah 2, 3 dan 4 beberapa kali.<br>Veverofu broyek aqen mevalhi langkhi 2, 3 dun veverofu kali</td>
  </tr>
  <tr>
    <td>Romance</td>
    <td>Paul accetta con riluttanza, sapendo ciò che sta per accadere.<br>Paul acceddi gen riluttanza, sapendo ciò che zta ber accadere.</td>
  </tr>
  <tr>
    <td>Turkic</td>
    <td>Şükür sadakaşı olark da bilinir.<br>Şükür jadakası ovariq ta bilinir.</td>
  </tr>
  <tr>
    <td>French-Creole</td>
    <td>Kijan nou ka montre gratitud nou pou sa Sovè à fe pon nou?<br>Kijan ngou ka mondre gratitud ngou pou sa Sovè ô fe pon ngou?</td>
  </tr>
</table>

Table 2: Linguistically-motivated noisers mimic function word, suffixal, and regular sound changes.

divergent artificial varieties. Hence, artificial languages exhibit various kinds of noise, at respective \( \theta^n \)-dials. (See Table 2 for examples of generated artificial language text.) Each artificial language is defined as the map of changes made from the HRL. We present two alternative ways to distribute these artificial languages: M→D-shell generates them on a single hypersphere at a fixed \( \theta^{p,m,f,c} \)-radius from the HRL,\footnote{We use \( \theta^{p,m,f,c}_{0.05,0.5,0.5,0.001} \), as an average CRL-HRL distance given \( \theta^n \) posteriors over several CRL-HRL pairs (Bafna et al., 2024b). See § E.1 for more details.} while M→D-cloud generates them on multiple hyperspheres around the HRL, populating the hypothesized dialect continuum.\footnote{We use \( K = 10 \) hyperspheres at uniform intervals in each dimension from the HRL and \( \theta^n_i = \theta^n_{max} - \frac{i}{K} \). Each hypersphere is therefore used for noising \( 1/K^{th} \) of the total data. We set \( \theta_{max} \) as \( \theta^{p,m,f,c}_{0.07,0.8,0.8,0.001} \) as reasonable maximum CRL-HRL distances in each dimension. See § E.1 for more details and tuning experiments for both variants.} (Figure 1 depicts M→D-cloud; M→D-shell would show all yellow arrows extending to the same blue band.)

2.2 Data-to-model (D→M)

In this paradigm, given CRL-HRL bilingual lexicons, we swap divergent parts of CRL input with known HRL equivalents to pull data towards the model’s proficiency distribution. Languages from the same family are generally syntactically similar or monotonic (Posner and Sala, 2024); hence this switching should largely maintain comprehensible grammatical structure.

We explore three D→M settings: func, cont, and all; in which we swap out only function words,\footnote{determiners, adpositions, auxiliaries, conjunctions, pronouns, and determiners} only content words,\footnote{all other word classes} and both, respectively. Function and content word classes differ both in their role in language, with the former crucial for grammaticality and coherence, and the latter providing semantics, as well as in the extent to which they are affected by language change: function words diverge quickly and often opaquely across dialects as compared to less frequent content words (Ellis, 2008). It is therefore relevant to isolate the effects of D→M on these word classes.

Note that isolating these settings requires function word identification in CRL input. We achieve this by collecting HRL function words from the Universal Dependencies corpus (Nivre et al., 2016) and annotation projection using the HRL-CRL lexicons. Any word that is not identified as a function word is considered a content word.

2.3 M↔D and baselines

We also combine M→D (~cloud fine-tuning) with D→M (inference-time adaptation) into M↔D.

We compare all proposed approaches to three baselines: (1) evaluating the model on CRL→eng MT without any adaptation (off-the-shelf), (2) evaluating on CRL→eng after fine-tuning on ordinary HRL→eng bitext without any simulated linguistic variation (fthr1), and (3) fine-tuning and evaluation after augmenting the HRL→eng bitext with completely random (i.e. not linguistically motivated or plausible) variations (randaug). For the final baseline, we randomly swap out characters in the HRL text with uniformly sampled target same-script characters, and words with different words from the source language vocabulary at probability (\( \theta^{rc}, \theta^{rw} \)). This is in similar to prior work (Belinkov and Bisk, 2018; Heigold et al., 2018) that introduces random character and word perturbations in the HRL as a method of data augmentation. We implement shell and cloud variants of this baseline analogously to M→D, maintain analogous parametrization for a fair comparison\footnote{See § E.1 for details.}, and only report the better of the two.
Figure 2: BLEU point improvement of the best DialUp method (M→D, D→M, or M↔D) over the best baseline (off-the-shelf, fthr1, or randaug). Languages are ordered by their M2M off-the-shelf performance.

3 Experimental Setup

We work with X→eng MT for 49 languages in six (sub)families.

Languages and Datasets  We include the following language groups and designate one HRL in each: Austronesian (with HRL Indonesian), Indic (Hindi), Turkic (Turkish), Romance (Italian), Arabic (Standard Arabic), and French-related Creole languages (Haitian). (See Table 6 in the Appendix for a full list.) We use Wikitext bitext (Schwenk et al., 2021) in the HRLs as training data; we do not use any CRL bitext. All compared methods use the same amount of total HRL bitext (\(100K\) sentences). We included CRLs per language family according to availability of evaluation data in the FloRes-200 (Team et al., 2022) dataset, maintaining a single script within each family. We use Kreyòl-MT evaluation sets for Creole CRLs (absent in FloRes) (Robinson et al., 2024). Our set of CRLs includes languages on a spectrum of relatedness to each HRL, as well as variety of resource levels. Some of the CRLs we include, such as French, are high-resource themselves, while most are low-resource. The Turkic, Austronesian, and Arabic languages we included vary widely in HRL proximity; some have a high degree of mutual intelligibility with the HRL (Azerbaijani, Malay, Saudi Arabic), and some are quite distant (Uzbek, Tagalog, Moroccan Arabic) (Hammarström et al., 2024; Nouri and Yangarber, 2014).

Bilingual lexicons  For each CRL-HRL pair, we use a combination of PanLex (Kamholz et al., 2014), and Swadesh lexicons.8 We also added Art Dieli’s Sicilian-Italian dictionary (Dieli, 2011) and Indic dialect lexicons from Bafna et al. (2024a). However, most of these have little coverage of function words. We therefore additionally used (function word-inclusive) lexicons obtained from performing statistical alignment (Dyer et al., 2013) on FloRes dev data.9 The Creole CRLs are not included in any of the above lexicon datasets; we used statistically aligned lexicons from JHU Bible translations (McCarthy et al., 2020), available for acf, crs, and mfe. We therefore only report D→M and M↔D results for these three Creole CRLs.

Models  We employ two MT models in our experiments: one multilingual supervised seq2seq model, M2M-100-1.2B (Fan et al., 2020); and one

8https://en.wiktionary.org/wiki/Category:Swadesh_lists_by_language
9See Appendix D for our survey of available lexicons, details and statistics of used lexicons. We also report the type coverage of CRL text for each lexicon, and note that this is generally low for PanLex and Swadesh.
multilingual instruction-tuned LLM, Aya-23-8B (Aryabumi et al., 2024). We selected these models because they are highly multilingual (supporting all our selected HRLs) but lack support for enough of our CRLs to warrant legitimate evaluation of adaptation to unseen languages. M2M only supports 15 of our 49 selected CRLs, while Aya-23 supports only 4. We selected one seq2seq model and one LLM in order to evaluate our methods in these two settings. We curated translation instructions from our bitexts to fine-tune Aya-23. (See § C.1 for details.) We use LoRA (Hu et al., 2021) for a single epoch for all fine-tuning processes.

4 Results

Results across all languages are in Figure 2. Our approaches give gains across the board, with varied trends by language family. Mean gains of low-performing languages (< 25 BLEU for off-the-shelf) per method and language family are in Table 3. See Appendix K for detailed results and COMET (Rei et al., 2020) scores. We found that these show trends consistent with BLEU.

We observe that for both models, languages with poorer baseline scores, which tend to be the low-performing varieties we aim to assist in this work, benefit more from DialUp, while languages with better baselines show small or negative gains. This trend is visible in Figure 2, where languages are ordered by their off-the-shelf BLEU score, and especially pronounced for Romance languages. We also see that M2M typically benefits more. This may be because Aya-23 was likely exposed to some CRLs in pre-training, despite not being trained explicitly on them, given the heterogeneity and poor documentation of LLM pre-training sets. Two exceptions to this trend are Javanese (jav) and Sundanese (sun) (gaining +13.3 BLEU with Aya-23, and little with M2M), which Aya-23 translates poorly off-the-shelf, and which M2M explicitly supports in pre-training. Proximity to the HRL also appears to play a role: CRLs distant from their HRL, such as Samoan and Tagalog, as well as those extremely close to the HRL, such as Malay (zsm), Azerbaijani (aze), and Saudi Arabic (ars), appear to benefit little from DialUp.

M→D gives mean gains over best baselines for all language families, demonstrating the efficacy of linguistically-motivated (as opposed to random) synthetic data augmentation. Table 4 shows examples of improvements on cognate words, including new inflections and function words. We often see that while off-the-shelf models transliterated them or left them as are instead of translating, M→D-tuned models decode them correctly.10 D→M introduces gains for many low-performing varieties—up to +11.4 BLEU for Aya-23 on Sundanese and +13.7 BLEU for M2M on Chhattisgarhi (hne)—where replacing dialectal words with HRL equivalents is helpful. However, it is also damaging for several, especially high-performing CRLs like French (fra), Spanish (spa), and Tagalog (tgl). Conceivably, if the base model is already proficient in a CRL, D→M code-switching introduces counterproductive unnaturalness. Since D→M is an inference-time method, practitioners may activate or deactivate it according to language needs. See Table 5 for examples of D→M-func input and output, demonstrating the brittleness of baseline models on dialectal function words, as well as the impact of treating them.

![Boxplots showing BLEU score improvements over the best baseline with M2M for three language families.](page_1016_1342_377_246.png)

Figure 3: BLEU score improvements over the best baseline with M2M for three language families. ↑ and ↓: # CRLs with positive/negative gains. M→D gives more consistent positive gains.

So which paradigm performs better? That depends on the model and the language. See Figure 3 for a distribution of score improvements for two families, and number of wins over the best baseline. Most families behave similarly to Austronesian and Romance (see Appendix H for similar plots for all languages/models). In general M→D gives lower-variance gains, with wins over the baseline for most CRLs. D→M often shows similar or higher mean gains but with higher variance, and consistently fewer wins. Indic languages show a different distribution, with a unanimous preference for D→M. Notably, M→D generally performs best across languages. (See Table 3.) Holistically, results suggest that M→D provides gains with low-risk consistency, and that D→M provides significant benefits for a select set of CRLs.

We note that D→M-func consistently outper-

10See full-length examples in Appendix G.
<table>
  <tr>
    <th rowspan="2">M2M</th>
    <th colspan="6">AUS (9)</th>
    <th colspan="6">ARA (7)</th>
    <th colspan="6">ROM (6)</th>
    <th colspan="6">TUR (4)</th>
    <th colspan="6">IND (4)</th>
    <th colspan="6">CRE (8)</th>
    <th colspan="6">Mean</th>
  </tr>
  <tr>
    <th>AUS</th><th>ARA</th><th>ROM</th><th>TUR</th><th>IND</th><th>CRE</th><th>Mean</th>
    <th>AUS</th><th>ARA</th><th>ROM</th><th>TUR</th><th>IND</th><th>CRE</th><th>Mean</th>
    <th>AUS</th><th>ARA</th><th>ROM</th><th>TUR</th><th>IND</th><th>CRE</th><th>Mean</th>
    <th>AUS</th><th>ARA</th><th>ROM</th><th>TUR</th><th>IND</th><th>CRE</th><th>Mean</th>
    <th>AUS</th><th>ARA</th><th>ROM</th><th>TUR</th><th>IND</th><th>CRE</th><th>Mean</th>
  </tr>
  <tr>
    <th>Baselines</th>
    <td>off-the-shelf</td><td>10.8</td><td>20.2</td><td>9.6</td><td>5.0</td><td>12.6</td><td>5.9</td><td>10.7</td><td>7.6</td><td>23.5</td><td>17.1</td><td>7.9</td><td>20.4</td><td>7.3</td><td>14.0</td>
    <td>fthrln</td><td>+0.3</td><td>+0.3</td><td>+0.5</td><td>-0.1</td><td>+0.4</td><td>+0.9</td><td>+0.4</td><td>-0.6</td><td>-1.8</td><td>+1.4</td><td>+1.2</td><td>-1.8</td><td>+2.3</td><td>+0.1</td>
    <td>ftrendaug</td><td>+0.2</td><td>+0.2</td><td>+0.6</td><td>+0.2</td><td>+0.4</td><td>+0.9</td><td>+0.4</td><td>-0.4</td><td>-2.3</td><td>+1.9</td><td>+1.1</td><td>-1.8</td><td>+2.1</td><td>+0.1</td>
  </tr>
  <tr>
    <th>M→D</th>
    <td>-shell</td><td>+1.9</td><td>+1.0</td><td>+1.1</td><td>+2.5</td><td>+3.0</td><td>+2.7</td><td>+4.1</td><td>+2.1</td><td>+0.4</td><td>+7.9</td><td>+2.9</td><td>+2.8</td><td>+3.0</td><td>+3.2</td>
    <td>-cloud</td><td>+1.3</td><td>+0.9</td><td>+0.1</td><td>+1.6</td><td>+3.4</td><td>+2.7</td><td>+3.5</td><td>+2.2</td><td>+0.1</td><td>+7.7</td><td>+3.2</td><td>+2.7</td><td>+3.5</td><td>+3.2</td>
  </tr>
  <tr>
    <th>D→M</th>
    <td>-cont</td><td>-0.4</td><td>-2.2</td><td>+1.6</td><td>+1.4</td><td>+0.3</td><td>+2.5</td><td>+0.5</td><td>+0.8</td><td>-6.2</td><td>-1.0</td><td>-0.6</td><td>-1.1</td><td>-3.8</td><td>-1.9</td>
    <td>-func</td><td>-0.1</td><td>+0.8</td><td>+8.0</td><td>+1.0</td><td>+12.0</td><td>+3.7</td><td>+4.3</td><td>+2.4</td><td>+1.8</td><td>+4.7</td><td>+0.3</td><td>+5.6</td><td>+3.3</td><td>+1.3</td>
    <td>-all</td><td>+0.0</td><td>-1.4</td><td>+9.8</td><td>+3.0</td><td>+11.4</td><td>+3.6</td><td>+2.7</td><td>+2.7</td><td>-6.9</td><td>+3.1</td><td>+0.0</td><td>+4.4</td><td>+3.5</td><td>+0.2</td>
  </tr>
  <tr>
    <th>M→D</th>
    <td>-cloud-cont</td><td>+1.4</td><td>-1.5</td><td>+9.0</td><td>+2.6</td><td>+5.1</td><td>+3.7</td><td>+3.4</td><td>+3.3</td><td>-4.3</td><td>+5.3</td><td>+1.8</td><td>+1.6</td><td>+2.6</td><td>+1.7</td>
    <td>-cloud-func</td><td>+1.1</td><td>+1.3</td><td>+14.1</td><td>+2.3</td><td>+13.0</td><td>+4.6</td><td>+6.1</td><td>+3.8</td><td>-0.8</td><td>+9.8</td><td>+3.1</td><td>+6.0</td><td>+2.8</td><td>+4.1</td>
    <td>-cloud-all</td><td>+0.9</td><td>-1.2</td><td>+13.1</td><td>+3.9</td><td>+12.2</td><td>+6.1</td><td>+5.8</td><td>+5.1</td><td>-4.5</td><td>+7.4</td><td>+2.1</td><td>+4.6</td><td>+2.5</td><td>+2.9</td>
  </tr>
</table>

Table 3: BLEU score performance gains relative to off-the-shelf, for low-performing CRLs (off-the-shelf score < 25), averaged by language family. # of such CRLs per family provided in parentheses. The overall best score is bolded and the best score in each paradigm is underlined. Language families are abbreviated with first three letters (e.g. Indic (IND)). M→D and D→M both outperform best baselines; M→D is generally the best.

<table>
  <tr>
    <th>lang</th>
    <th>Input CRL phrase</th>
    <th>HRL equivalent</th>
    <th>Baseline (CRL eng)</th>
    <th>D M (CRL eng)</th>
  </tr>
  <tr>
    <td>bho</td>
    <td>अध्ययन करने पर</td>
    <td>(hin) अध्ययन करने पर</td>
    <td>the study on Kila revealed</td>
    <td>studies have shown</td>
  </tr>
  <tr>
    <td>lij</td>
    <td>da reduc a mortalită do Ebola</td>
    <td>(ita) da ridurre la mortalità dell’Ebola</td>
    <td>da reduc a mortalită do Ebola</td>
    <td>to reduce Ebola mortality</td>
  </tr>
  <tr>
    <td>scr</td>
    <td>alqamah</td>
    <td>(tur) ekspert</td>
    <td></td>
    <td>expert</td>
  </tr>
</table>

Table 4: M→D improves M2M on input with cognate processes such as function words, new suffixes, and phonological/orthographic change. The baseline often transliterates the input or leaves it as is.

<table>
  <tr>
    <th>lang</th>
    <th>Input CRL Phrase</th>
    <th>D M modification</th>
    <th>Baseline (CRL eng)</th>
    <th>D M (CRL eng)</th>
  </tr>
  <tr>
    <td>mag</td>
    <td>जोड़ी एक मुद्रादान</td>
    <td>साथ एक मुद्रादान</td>
    <td>Jure Ego Prosecution</td>
    <td>a litigation with</td>
  </tr>
  <tr>
    <td>scn</td>
    <td>ntà cademia svìdisi</td>
    <td>nella cademia svìdisi</td>
    <td>ntà cademia svìdisi</td>
    <td>the Swedish Academy</td>
  </tr>
  <tr>
    <td>jav</td>
    <td>kaya sawetara ahli liyane</td>
    <td>seperi beberapa ahli liyane</td>
    <td>rich a handful of other experts</td>
    <td>the some other experts</td>
  </tr>
</table>

Table 5: D→M improves M2M output by replacing function words in the input (underlined in output) with HRL equivalents.

forms D→M-cont, and is often better or close to D→M-all. This highlights the importance of treating dialectal function words in enabling model comprehension, and the utility of collecting CRL-HRL function word maps for low-resource languages. This insight is particularly convenient given that function words form a small closed class and are much easier to collect comprehensive lexicons for than open-class content words; they are also more likely to be accurately aligned with statistical alignment than relatively rarer content words.

In fact, exchanging content words is largely unhelpful across the board, including from large high-quality lexicons like Art Dieli for Sicilian (scn). We attribute this to the higher degree of context dependence in handling content words; synonymy, word sense variation, naturalness, and lexicon noise likely contribute to this problem.

See Appendix E for our additional M→D experiments in using multiple HRLs per family, training with more data or for epochs, different source datasets, and more aggressive noising. These mildly improve or maintain current trends. We also show that switching CRL words directly into English instead of the HRL for D→M degrades performance, presumably due to unnaturalness of CRL-eng (as compared to CRL-HRL)code-mixing.

5 Discussion

When does M→D help? To test our hypotheses that baseline performance and proximity to the HRL interact meaningfully with score improvement, and to understand which circumstances render DialUp more or less effective, we analyze how language features correlate with gains. We computed Spearman’s \( \rho \) coefficients between BLEU improvement from M→D over the off-the-shelf baseline and features indicating both baseline CRL support (off-the-shelf BLEU score, whether the model explicitly supports the CRL, and number of CRL Wikipedia pages\(^{11}\)) and HRL-CRL relatedness (character F-score (Popović, 2015) between HRL and CRL FloRes dev sets and average token-count ratio in CRL dev lines to HRL dev lines when using the model’s HRL tokenizer\(^{12}\)). For

\( ^{11} \)per Robinson et al. (2023)
\( ^{12} \)In the case of the Creole CRLs, which lack FloRes dev sets, we used JHU Bible translations, available for acf, crs, and mfe.
Figure 4: Decision trees indicate that the languages benefiting most from adaptation are low-baseline languages with less than 1.75 times HRL token fertility for Aya, and low-baseline languages with more than 24.2 chrF proximity to the HRL.

both Aya and M2M the only such feature that correlated significantly (p<0.01) was off-the-shelf BLEU, with \( \rho = -0.52 \) and \( -0.45 \), respectively—indicating a moderate-to-strong negative correlation between baseline performance and improvement from M→D, as hypothesized in § 4.

Further analysis also suggests that baseline score is more predictive of M→D success than other features; this is confirmed by random forests fitted over the same features for both models (see decision trees in Figure 4), as well as feature weights from a linear regressor. (See Table 35.)

Measurements of HRL-CRL relatedness had the second-highest forest feature importance for both models. Though our measurements of HRL-CRL relatedness do not correlate with BLEU score improvement, we suspect there may be a non-linear relationship between dev-set character-F score and method effectiveness. When plotted in Figure 6, the points suggest the contour of a downward-facing parabola, indicating that M→D is most effective for CRLs that are not too close but not too far from the HRL, as we hypothesized. This has an intuitive interpretation: CRLs that are too close may either already be well-performing, or may not display enough dialectal divergence to benefit from DialUp, whereas CRLs that are too far may have a higher amount of non-cognate divergences from the HRL that DialUp does not help with. However, note that language family is possibly confounded with character-F score, with only Austronesian CRLs covering a wide range on our observed contour; it would be difficult to disentangle the effect of language family without many more languages.

Setting \( \theta^{p,m,f}\)-dials Although M→D uses phonological, morphological, and function word noising to model cognate processes in CRLs, the latter two are specialized versions of the phonological noiser, applying it at a fixed high dial to HRL suffixes/function words (Appendix B). We show that this 3-dimensional noiser can be approximated solely by the phonological noiser at a suitable \( \theta^p \)-dial. Setting all other \( \theta^n = 0 \), we sweep over \( \theta^p \) for Indic (Figure 5). We observe that with an optimal \( \theta^p \), the phonological noiser achieves competitive gains as M→D-cloud, which uses all noisers. Our default M→D \( \theta \)-dials are eminently reasonable, yielding the best possible performance achievable by the above procedure; however, these defaults may be more suited to particular language families than others. Given that tuning \( \theta^{p,m,f} \) in combination is intractable, we therefore recommend simply tuning \( \theta^p \) for a given language family as a good proxy, along with trying our defaults. The above results indicate that model gains from M→D largely arise from learning to decode sound changes in dialects, regardless of their placement in words.

M→D error analysis Arabic varieties gain less than other groups from M→D and M←D. This may indicate that our M→D method does not approximate the Arabic linguistic variation well. Arabic varieties do exhibit phonological differences (which M→D simulates). However, these are often not expressed in the orthography, making lexical, syntactic, and register differences more prominent differentiators in text. We experimented simulating lexical choice variation in CRLs using HRL

Figure 5: Gains in BLEU points for different values of \( \theta^p \) (1-dimensional noiser) for Indic languages, with dotted lines showing the performance of M→D-cloud, using the the 3-dimensional noiser \( \theta^{p,m,f} \) with default parameters. Tuning only \( \theta^p \) for Indic is competitive with M→D-cloud.
WordNets as part of our artificial language generation method for 5 language families; however, this did not give us significant gains (Appendix E). DialUp in its current form may not be well suited for Arabic. Additionally, we note that FloRes sets are extremely close to standard Arabic, with a mean dev "dialectness" score (Keleg et al., 2023) of only 28%.13 This may also limit the scope of improvements from DialUp methods on this test set.

French-related Creole languages, which also had low baseline scores and small M→D gains, may have also suffered from genre mismatch (given their heterogeneous test sets (Robinson et al., 2024a)). We initially hypothesized that their poor performance may be due to the HRL’s (Haitian) poor baseline performance with Aya-23. We attempted curriculum training with fine-tuning first on Haitian bitext and then with M→D; however this did not materially help. It is still possible that given Haitian’s low-resource nature (and its extremely low-resource CRLs), training token frequencies are too low to risk obfuscating them with DialUp noise. We hope future work will find more concrete conclusions on this trend.

D→M error analysis We investigate why D→M–func degrades the performance of Romance and Austronesian HRLs, while benefiting several CRLs in these families and Indic. We perform a manual evaluation of 100 words from our function word lexicons for 4 languages from these families (Table 9)14 to characterize the extent of noise in the D→M–func pipeline: a CRL word may be misidentified as a function word (mean function word identification accuracy: 83.6%), or it may be mistranslated (mean function word translation accuracy: 56.5%, general translation accuracy: 59.7%). The introduced noise is naturally particularly damaging when the baseline translation is good (as in for many high-resource CRLs). In fact, we find that D→M swaps hurt for high-resource CRLs even with ideal-case accurate swaps from clean lexicons. On the other hand, for a number of CRLs such as Maithili (mai), we observe that the benefits of making key function words swaps outweighs the negative impact of this noise; we find that even noisy automatically collected lexicons contribute to D→M

13Comparatively, MADAR-26 dev sets (Bouamor et al., 2018) for the same varieties achieve 59%. We initially attempted to use this dataset, but the genre mismatch with train data rendered both baseline and challenger scores too low.
14See Appendix I for details.

Figure 6: CRL-HRL proximity, measured as character F-score between dev sets, plotted against BLEU increase from M→D, suggests a peak from 20-30 chrF. Arabic varieties are outliers because the FloRes dialectal Arabic sets are unusually close to standard Arabic.

improvements in some languages (e.g. excluding relatively noisy Bafna et al. (2024a) lexicons results in a drop of 9 BLEU for Maithili with M2M). D→M benefits also depend on the model and HRL: in the case of Haitian, D→M performs well with M2M (yielding +7 BLEU for acf), but degrades baseline performance for Aya-23. This can be explained by the fact that Aya-23 has <10 BLEU performance on Haitian itself, making it unhelpful to switch other CRL words into Haitian; on the other hand, M2M baseline is strong for Haitian but weak for related CRLs, rendering D→M a useful strategy. We therefore recommend D→M for low-baseline CRLs, with a model that performs well on the language family HRL.

Note that D→M shows small gains for Arabic and Turkic CRLs. This is likely caused by a scarcity of word exchanges, which D→M relies on for increasing CRL input comprehensibility. In fact, D→M–func results in only 16.3% and 15% of words being swapped for Arabic and Turkic, respectively, compared to 43.1% for Austronesian, 38.5% for Romance, and 33.4% for Indic. (See Appendix F). This is potentially a result of Arabic and Turkic languages’ comparatively complex morphology, in addition to above noted issues with Arabic test sets. Grammatical morphemes in these languages are frequently affixes, clitics, or attached to affixes and seldom occur on their own; these will be missed by our whitespace-reliant word-switching technique. Future work can investigate integrating morphological analysis technologies when available into our methods.

6 Prior Work

DialUp has theoretical bases in a large body of prior work of noise pattern induction for robustness. Similar to M→D previous researchers have developed methods of injecting noise into training data, including orthographic variants or typos (Belinkov
and Bisk, 2018; Heigold et al., 2018), grammatical errors (Anastasopoulos et al., 2019), learned noise types (Brahma et al., 2023), and bilingual lexicon-induced code-mixing (Jones et al., 2023; Xia et al., 2019). These approaches require monolingual data or bilingual lexicons for from-scratch training. M→D innovates by inducing general dialectal robustness in a pre-trained model to unseen dialects without knowledge or resources in target varieties. D→M also draws from prior work, namely input processing techniques for LLM in-context learning via CRL-English lexicons, morphological analyzers, and grammar references (Zhang et al., 2024a; Ghazvininejad et al., 2023; Zhang et al., 2024b; Tanzer et al., 2024; Dimakis et al., 2024). D→M is likewise an inference-time intervention, but it is applicable to both traditional MT models and LLMs. Notably, our exploration of D→M is the first to validate the insight that model comprehensibility of often-ignored function words is beneficial, and to show considerable gains using small, automatically collected lexicons.

7 Conclusion

We present DialUp, a general recipe for expanding pretrained model coverage from its high-resource training languages to their dialect continua. This involves a finetuning-based technique involving artificial dialect generation using only HRL data, as well as an inference-time technique for rendering dialectal input more comprehensible to the model. DialUp shows large gains on several low-resource languages, and introduces a promising paradigm for making NLP flexible to potentially unseen and undocumented dialectal variation.

Limitations

Language-family-dependent gains The benefits of M→D and D→M clearly vary by language family: Indic and Romance benefit much more than other families, and Arabic in particular benefits minimally. M→D relies on approximating realistic synthetic dialects of the HRL by simulating various mechanisms of language variation. While these are broadly similar across language families and can be grouped into phonological, morphological, and lexical processes, which our artificial language generation method seeks to model, each language family naturally varies in typology, intra-family linguistic diversity, and the kinds of relationship that its CRLs have with each other and with the HRL.

For example, certain variants may differ from the HRL in their extent of lexical influence from a third (colonial) language, as in the cases of some Arabic and Turkic languages. Certain typologies or scripts may also be more or less amenable to our methods; e.g. we observe that suffix stacking in Turkic languages is challenging for the morphological noiser, which performs suffix identification in a manner most suited for fusional or morphologically non-complex languages (Ismayilzada et al., 2024). Finally, some language families, such as Arabic, may exhibit syntactic variation, which is not modeled at all by our current method. Our current artificial language generation method does not cater to these family-specific characteristics, which may contribute to its poorer performance on these families. We leave it to future work to refine the current catch-all technique of artificial language generation for specific families.

Different-script CRLs M→D does not support artificial language generation across scripts, i.e. it cannot perturb an HRL in one script into an artificial language in a different script. This is relevant for language families with more than one script: for example, several Turkic low-resource languages use the Cyrillic script, and cannot be supported with our current methods. Future work can look into transliteration-based approaches to handle this issue.

Models We provide results on one representative model of the traditional NMT as well as LLM paradigm, choosing an appropriate model with respect to its training languages given our goal of handling unseen and unknown target dialects, and available evaluation data. Naturally, our method should be tested on a variety of models from each paradigm to gauge its usefulness as a general data augmentation technique for achieving dialectal robustness.

Availability of bilingual lexicons Our best-performing D→M-func approach depends on the availability of CRL-HRL function word mappings. These are much easier to collect than bilingual lexicons given that they are a small class of closed class words, and we show that using even noisy alignments from a small amount of bitext is effective for this purpose: however, this may still be unfeasible for particularly low-resource languages.

Long way to go While our methods show considerable improvements on cognates for CRLs (ex-
amples in Table 4), and improved scores using automated MT evaluation metrics such as BLEU and COMET, the improved translation may still not be usable or coherent. We provide examples of this in Table 7.

Ethics Statement

Our work seeks to expand the benefits of mainstream NLP (specifically, machine translation) in standard high-resource varieties of languages to their diverse, dialectal, colloquial, and/or low-resource neighbours. Our work introduces general techniques towards this goal; however, any particular language community may have specific and different needs, potentially not served by mainstream NLP, which our work does not take into account (Bird, 2020). While we attempt to use minimal resources in low-resource languages to achieve our goal in order to make our methods resource-light and widely applicable, using only HRL data in training risks reinforcing a culturally mainstream view of the world in our models. This may be particularly problematic for continua where dialect-speaking communities diverge in political or other views from communities speaking the standard variety.

References

Rached Ben Abdelkader, Abdeljelil Ayed, and Aziz Naouar. 1977. Peace corps english-tunisian arabic dictionary.

Sotiris Anagnostidis and Jannis Bulian. 2024. How susceptible are llms to influence in prompts? Preprint, arXiv:2408.11865.

Antonios Anastasopoulos, Alison Lui, Toan Q. Nguyen, and David Chiang. 2019. Neural machine translation of text from non-native speakers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3070–3080, Minneapolis, Minnesota. Association for Computational Linguistics.

Viraat Aryabumi, John Dang, Dwarak Talpururu, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, Acyr Locatelli, Julia Kreutzer, Nick Frosst, Phil Blunsom, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. Aya 23: Open weight releases to further multilingual progress. Preprint, arXiv:2405.15032.

Niyati Bafna, Cristina España-Bonet, Josef van Genabith, Benoît Sagot, and Rachel Bawden. 2024a. When your cousin has the right connections: Unsupervised bilingual lexicon induction for related data-imbalanced languages. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17544–17556, Torino, Italia. ELRA and ICCL.

Niyati Bafna, Kenton Murray, and David Yarowsky. 2024b. Evaluating large language models along dimensions of language variation: A systematik investigation uv cross-lingual generalization. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18742–18762, Miami, Florida, USA. Association for Computational Linguistics.

Özge Bakay, Özlem Ergelen, Elif Sarmış, Selin Yildirim, Bilge Nas Arcan, Atilla Kocabalcıoğlu, Merve Özçelik, Ezgi Sanyiar, Oğuzhan Kuyrukuç, Begüm Avar, and Olcay Taner Yildiz. 2021. Turkish WordNet KeNet. In Proceedings of the 11th Global Wordnet Conference, pages 166–174, University of South Africa (UNISA). Global Wordnet Association.

Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and Natural Noise Both Break Neural Machine Translation. Preprint, arxiv:1711.02173.

Martin Benjamin. 2019. Teach you backwards: An in-depth study of google translate for 108 languages. Online.

A. Bergman and Mona Diab. 2022. Towards responsible natural language annotation for the varieties of Arabic. In Findings of the Association for Computational Linguistics: ACL 2022, pages 364–371, Dublin, Ireland. Association for Computational Linguistics.

Pushpak Bhattacharyya. 2010. IndoWordNet. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA).

Steven Bird. 2020. Decolonising speech and language technology. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3504–3519, Barcelona, Spain (Online). International Committee on Computational Linguistics.

Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Ofazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).

Maharaj Brahma, Kaushal Maurya, and Maunendra Desarkar. 2023. SelectNoise: Unsupervised noise injection to enable zero-shot machine translation for extremely low-resource languages. In Findings of the
Association for Computational Linguistics: EMNLP 2023, pages 1615–1629, Singapore. Association for Computational Linguistics.

Samuel Cahyawijaya, Holy Lovenia, and Pascale Fung. 2024. LLMs are few-shot in-context low-resource language learners. arXiv preprint arXiv:2403.16512.

Siddharth Dalmia, Ramon Sanabria, Florian Metze, and Alan W. Black. 2018. Sequence-based multilingual low resource speech recognition. CoRR, abs/1802.07420.

Art Dieli. 2011. The sicilian language. Online.

Antonios Dimakis, Stella Markantonatou, and Antonios Anastasopoulos. 2024. Dictionary-aided translation for handling multi-word expressions in low-resource languages. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2588–2595, Bangkok, Thailand. Association for Computational Linguistics.

Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023. Towards leaving no Indic language behind: Building monolingual corpora, benchmark and models for Indic languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12402–12426, Toronto, Canada. Association for Computational Linguistics.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Chris Dyer, Victor Chahuneau, and Noah A Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 644–648.

Anas Assadek El Ouamari. 2024. Moroccan darija arabic translator.

Nick C Ellis. 2008. The dynamics of second language emergence: Cycles of language use, language change, and language acquisition. The modern language journal, 92(2):232–249.

Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Eduard Grave, Michael Auli, and Armand Joulin. 2020. Beyond english-centric multilingual machine translation. Preprint, arXiv:2010.11125.

Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48.

Jonathan Garrett, Greg Lastowka, Akgul Muhammetmuradova, Jahan Myradova, Kimberly Naahielua, Meena Pallipamu, Muhammed Rustamov, Meretgul Sharipova, and Aynabat Yaylymova. 1996. Turkmen-english dictionary.

Ronald Gatty. 2009. Fijian-english dictionary with notes on fijian culture and natural history.

Marjan Ghazvininejad, Hila Gonen, and Luke Zettlemoyer. 2023. Dictionary-based phrase-level prompting of large language models for machine translation. arXiv preprint arXiv:2302.07856.

Mike Green. 2020. Egyptian arabic dictionary.

Harald Hammarström, Robert Forkel, Martin Haspelmath, and Sebastian Bank. 2024. Glottolog. Available online at http://glottolog.org. Accessed on 2024-12-14.

Georg Heigold, Stalin Varanasi, Günter Neumann, and Josef van Genabith. 2018. How Robust Are Character-Based Word Embeddings in Tagging and MT Against Word Scrambling or Random Noise? In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 68–80, Boston, MA. Association for Machine Translation in the Americas.

Dirk Hovy and Christoph Puschke. 2018. Capturing regional variation with distributed place representations and geographic retrofitting. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4383–4394, Brussels, Belgium. Association for Computational Linguistics.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.

Mete Ismayilzada, Defne Cirici, Jonne Sälevä, Hale Sirin, Abdullatif Köksal, Bhuwan Dhingra, Antoine Bosselut, Lonneke van der Plas, and Duygu Ataman. 2024. Evaluating morphological compositional generalization in large language models. Preprint, arXiv:2410.12656.

Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. 2023. Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine. Preprint, arxiv:2301.08745.

Alexander Jones, Isaac Caswell, Orhan Firat, and Ishank Saxena. 2023. GATITOS: Using a new multilingual lexicon for low-resource machine translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 371–405, Singapore. Association for Computational Linguistics.
Aditya Joshi, Raj Dabre, Diptesh Kanojia, Zhuang Li, Haolan Zhan, Gholamreza Haffari, and Doris Dippold. 2024. Natural language processing for dialects of a language: A survey. arXiv preprint arXiv:2401.05632.

David Kamholz, Jonathan Pool, and Susan Colowick. 2014. PanLex: Building a resource for panlingual lexical translation. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3145–3150, Reykjavik, Iceland. European Language Resources Association (ELRA).

Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. ALDi: Quantifying the Arabic level of dialectness of text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10597–10611, Singapore. Association for Computational Linguistics.

Dylan Kobayashi, Jason Leigh, John Maher, and James Metz. 2016. Samoan / english dictionary. Department of Indo-Pacific Languages and Literature, and Department of Information & Computer Sciences, University of Hawai‘i at Manoa.

Arya D. McCarthy, Dylan Wicks, Dylan Lewis, Aaron Mueller, Winston Wu, Oliver Adams, Garrett Nicolai-Matt Post, and David Yarowsky. 2020. The Johns Hopkins University Bible corpus: 1600+ tongues for typological exploration. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2884–2892, Marseille, France. European Language Resources Association.

John Moorfield. 2024. Te aka māori dictionary.

Niklas Muenninghoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. 2023. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15991–16111.

Ercong Nie, Sheng Liang, Helmut Schmid, and Hinrich Schütze. 2023. Cross-lingual retrieval augmented prompt for low-resource languages. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8320–8340, Toronto, Canada. Association for Computational Linguistics.

Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampi Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1659–1666.

Nurril Hirfana Bte Mohamed Noor, Suerya Sapuan, and Francis Bond. 2011. Creating the open Wordnet Bahasa. In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, pages 255–264, Singapore. Institute of Digital Enhancement of Cognitive Processing, Waseda University.

Javad Nouri and Roman Yangarber. 2014. Measuring language closeness by modeling regularity. In Proceedings of the EMNLP’2014 Workshop on Language Technology for Closely Related Languages and Language Variants, pages 56–65, Doha, Qatar. Association for Computational Linguistics.

Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.

Rebecca Posner and Marius Sala. 2024. Linguistic characteristics of the romance languages.

Yasser Regragui, Lahsen Abouenour, Fettoum Krieche, Karim Bouzoubaa, and Paolo Rosso. 2016. Arabic WordNet: New content and new applications. In Proceedings of the 8th Global WordNet Conference (GWC), pages 333–341, Bucharest, Romania. Global Wordnet Association.

Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. Preprint, arXiv:2009.09025.

Ayla Rigouts Terryh and Miryam de Lhoneux. 2024. Exploratory study on the impact of English bias of generative large language models in Dutch and French. In Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024, pages 12–27, Torino, Italia. ELRA and ICCL.

Nathaniel Robinson, Raj Dabre, Ammon Shurtz, Rasul Dent, Onenamiyi Onesi, Claire Monroc, Loïc Grobol, Hasan Muhammad, Ashi Garg, Naome Etori, Vijay Murari Tiyyala, Olanrewaju Samuel, Matthew Stutzman, Bismarck Odoo, Sanjeev Khudanpur, Stephen Richardson, and Kenton Murray. 2024. Kreyol-MT: Building MT for Latin American, Caribbean and colonial African creole languages. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3083–3110, Mexico City, Mexico. Association for Computational Linguistics.

Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. ChatGPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Association for Computational Linguistics.

Adriana Roventini, Antonietta Alonge, Nicoletta Calzolari, Bernardo Magnini, and Francesca Bertagna. 2000. ItalWordNet: a large semantic database for
Italian. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00), Athens, Greece. European Language Resources Association (ELRA).

Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2021. WikiMatrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1351–1361, Online. Association for Computational Linguistics.

Morris Swadesh. 2015. Swadesh lists by language. Online.

Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. 2024. A Benchmark for Learning to Translate a New Language from One Grammar Book. Preprint, arxiv:2309.16575.

NLLB Team, Marta R. Costa-jussá, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hefernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejía Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Rogers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation. Preprint, arXiv:2207.04672.

Mengzhou Xia, Xiang Kong, Antonios Anastasopoulos, and Graham Neubig. 2019. Generalized data augmentation for low-resource translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5786–5796.

Chen Zhang, Xiao Liu, Jiaheng Lin, and Yansong Feng. 2024a. Teaching Large Language Models an Unseen Language on the Fly. Preprint, arxiv:2402.19167.

Kexun Zhang, Yee Man Choi, Zhenqiao Song, Taiqi He, William Yang Wang, and Lei Li. 2024b. Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions. Preprint, arxiv:2402.18025.

Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. Multi-VALUE: A framework for cross-dialectal English NLP. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Toronto, Canada. Association for Computational Linguistics.
A  Languages

See Table 6.

B  Details of artificial language generation

In § 2, we introduce the artificial language generation method for M→D, which consists of phonological, morphological, function word, and content word “noising”, or corruption, taken from Bafna et al. (2024b).

• The phonological noiser changes phonemes in a given left and right phonological context to phonetically nearby targets (e.g. t→d, which differ only in voicing). We use script-to-IPA mappings and vice versa to first convert our input into IPA. We define plausible target phoneme sets for each phoneme based on its phonetic characteristics. The noised phoneme is then converted back into the input script.

• The morphological noiser targets suffixes in the HRL. Suffixes are identified heuristically, collected by frequency over an HRL corpus. A given suffix is noised by applying a fixed high amount of phonological noise over it (\( \theta^p = 0.8 \)), and changed globally over all occurrences to its noised version.

• Similarly, the function word noiser targets function words in the HRL. Function word identification is done in the following way: we curate a set of POS tags for closed-class words, including determiners, pronouns, conjunctions, auxiliaries, and adpositions. We then use the tagged Universal Dependencies corpus (Nivre et al., 2016) to identify all words in a given language whose most frequent tag is one of these. Since function words are a relatively small set, this procedure should yield coverage even over a small corpus. A given function word is noised by applying a fixed high amount of phonological noise over it (\( \theta^p = 0.8 \)), and changed globally over all occurrences to its noised version.

• The non-cognate content word noiser generates non-words using an HRL character ngram model as replacements for HRL content words. Note that this is the only noiser that models non-cognate processes.

C  Experimental details

C.1  Aya prompt

LLMs are known to be sensitive to the prompt used for the task (Anagnostidis and Bulian, 2024).

We evaluated Aya-23 off-the-shelf on the following prompts, varying in their specification of the source language:

1. Translate into English:
   <source_text>

2. Translate from a dialect of <hrl> into English:
   <source_text>

3. Translate from <flores_code_of_lrl> into English:
   <source_text>

The first two prompts are chosen in accordance with our goal of translating potentially unknown dialects of the HRL. While prompts 2 and 3 generally give improvements on the baseline for known target dialects (+1.3, +2.5, +1.3, −0.5 mean BLEU improvements for Arabic, Romance, Turkic and Indic respectively) the best performing prompt differs by language. For the purpose of our study, using prompts 2 and 3 is non-ideal, since it introduces the confound of degradation stemming from a certain target being referred to as a dialect of another language, or model familiarity with the particular dialect name or code. This is relevant because our evaluation languages range over degrees of resourcedness and presence in Aya-23, as well as relatedness from the corresponding HRL. Further, note that also use our chosen prompt during instruction finetuning over noisy text for consistency, and it is unclear what language name or code we should use for the noised text for best transfer to all target dialects in this setting. In order to avoid the above confounds and for simplicity, we use the first prompt for all approaches and languages, both during training and evaluation.

C.2  M2M tokenizer

M2M requires specification of source and target language tokens; the source language token is appended to the end of the input text during tokenization, and the target language token is provided to the decoder as its first token to specify the output language. We use CRL language tokens when they
<table>
  <tr>
    <th>Language</th>
    <th>Family</th>
    <th>Subfamily</th>
    <th>ISO code</th>
    <th>PlaRes/NLLB</th>
    <th>In M2M?</th>
    <th>In Aya-23?</th>
    <th>Resource Level</th>
  </tr>
  <tr>
    <td>Indonesian</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>ind</td>
    <td>ind_Latn</td>
    <td>-</td>
    <td>-</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Javanese</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>jav</td>
    <td>jav_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><200K</td>
  </tr>
  <tr>
    <td>Sundanese</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>sun</td>
    <td>sun_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><100K</td>
  </tr>
  <tr>
    <td>Samoan</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Polynesian</td>
    <td>smo</td>
    <td>smo_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><10K</td>
  </tr>
  <tr>
    <td>Tagalog</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>mri</td>
    <td>mri_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Cebuano</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>cbr</td>
    <td>cbr_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><12M</td>
  </tr>
  <tr>
    <td>Standard Malay</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian</td>
    <td>zsm</td>
    <td>zsm_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><2M</td>
  </tr>
  <tr>
    <td>Tagalog</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Philippine</td>
    <td>tgl</td>
    <td>tgl_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><500K</td>
  </tr>
  <tr>
    <td>Ilocano</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Philippine</td>
    <td>iio</td>
    <td>iio_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><100K</td>
  </tr>
  <tr>
    <td>Pangasinan</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Philippine</td>
    <td>paj</td>
    <td>paj_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><10K</td>
  </tr>
  <tr>
    <td>Plateau Malagasy</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Western Indonesian</td>
    <td>plt</td>
    <td>plt_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><500K</td>
  </tr>
  <tr>
    <td>Pangasian</td>
    <td>Austronesian</td>
    <td>Malayo-Polynesian, Philippine</td>
    <td>pag</td>
    <td>pag_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><10K</td>
  </tr>
  <tr>
    <td>Arabic (MSA)</td>
    <td>Arabic</td>
    <td>Modern Standard Arabic</td>
    <td>arb</td>
    <td>arb_Arab</td>
    <td>ar</td>
    <td>ara</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Mesopotamian Arabic</td>
    <td>Arabic</td>
    <td>Mesopotamian Arabic</td>
    <td>acm</td>
    <td>acm_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1M</td>
  </tr>
  <tr>
    <td>Al-Izz-Adebi Arabic</td>
    <td>Arabic</td>
    <td>Saudi Yemeni</td>
    <td>aqy</td>
    <td>aqy_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Tunisian Arabic</td>
    <td>Arabic</td>
    <td>Maghrebi</td>
    <td>aeb</td>
    <td>aeb_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>South Levantine Arabic</td>
    <td>Arabic</td>
    <td>Levantine Arabic</td>
    <td>aip</td>
    <td>aip_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>North Levantine Arabic</td>
    <td>Arabic</td>
    <td>Levantine Arabic</td>
    <td>apc</td>
    <td>apc_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Wadi Arabic</td>
    <td>Arabic</td>
    <td>Peninsular Arabic</td>
    <td>ars</td>
    <td>ars_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Maghrebi Arabic</td>
    <td>Arabic</td>
    <td>Maghrebi</td>
    <td>ary</td>
    <td>ary_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><100K</td>
  </tr>
  <tr>
    <td>Egyptian Arabic</td>
    <td>Arabic</td>
    <td>Egyptian Arabic</td>
    <td>arz</td>
    <td>arz_Arab</td>
    <td>-</td>
    <td>-</td>
    <td><3M</td>
  </tr>
  <tr>
    <td>Italian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>ita</td>
    <td>ita_Latin</td>
    <td>it</td>
    <td>ita</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Spanish</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>spa</td>
    <td>spa_Latin</td>
    <td>es</td>
    <td>spa</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>French</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>fra</td>
    <td>fra_Latin</td>
    <td>fr</td>
    <td>fra</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Portuguese</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>por</td>
    <td>por_Latin</td>
    <td>pt</td>
    <td>por</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Romanian</td>
    <td>Romance</td>
    <td>Eastern Romance</td>
    <td>ron</td>
    <td>ron_Latin</td>
    <td>ro</td>
    <td>ron</td>
    <td><3M</td>
  </tr>
  <tr>
    <td>Lombard</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>lmo</td>
    <td>lmo_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><200K</td>
  </tr>
  <tr>
    <td>Asturian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>ast</td>
    <td>ast_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Venetian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>vec</td>
    <td>vec_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><200K</td>
  </tr>
  <tr>
    <td>Catalan</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>cat</td>
    <td>cat_Latin</td>
    <td>ca</td>
    <td>-</td>
    <td><2M</td>
  </tr>
  <tr>
    <td>Sicilian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>sen</td>
    <td>sen_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><100K</td>
  </tr>
  <tr>
    <td>Sardinian</td>
    <td>Romance</td>
    <td>Southern Romance</td>
    <td>srd</td>
    <td>srd_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Friulian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>fur</td>
    <td>fur_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><10K</td>
  </tr>
  <tr>
    <td>Ligurian</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>lij</td>
    <td>lij_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Occitan</td>
    <td>Romance</td>
    <td>Italo-Western</td>
    <td>oci</td>
    <td>oci_Latin</td>
    <td>oc</td>
    <td>-</td>
    <td><200K</td>
  </tr>
  <tr>
    <td>Turkish</td>
    <td>Turkic</td>
    <td>Oghuz</td>
    <td>tur</td>
    <td>tur_Latin</td>
    <td>tr</td>
    <td>tur</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Azeri</td>
    <td>Turkic</td>
    <td>Oghuz</td>
    <td>azj</td>
    <td>azj_Latin</td>
    <td>az</td>
    <td>-</td>
    <td><1M</td>
  </tr>
  <tr>
    <td>Crimean Tatar</td>
    <td>Turkic</td>
    <td>Kipchak</td>
    <td>crh</td>
    <td>crh_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Turkmen</td>
    <td>Turkic</td>
    <td>Oghuz</td>
    <td>tuk</td>
    <td>tuk_Latin</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Uzbek</td>
    <td>Turkic</td>
    <td>Kartik</td>
    <td>uzb</td>
    <td>uzn_Latin</td>
    <td>uz</td>
    <td>-</td>
    <td><1M</td>
  </tr>
  <tr>
    <td>Hindi</td>
    <td>Indic</td>
    <td>Western</td>
    <td>hin</td>
    <td>hin_Deva</td>
    <td>-</td>
    <td>-</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Bengali</td>
    <td>Indic</td>
    <td>Eastern</td>
    <td>bno</td>
    <td>bno_Deva</td>
    <td>-</td>
    <td>-</td>
    <td>CRL</td>
  </tr>
  <tr>
    <td>Bhojpuri</td>
    <td>Indic</td>
    <td>Eastern</td>
    <td>bho</td>
    <td>bho_Deva</td>
    <td>-</td>
    <td>-</td>
    <td><100K</td>
  </tr>
  <tr>
    <td>Chhattisgarhi</td>
    <td>Indic</td>
    <td>Eastern</td>
    <td>hne</td>
    <td>hne_Deva</td>
    <td>-</td>
    <td>-</td>
    <td><2M</td>
  </tr>
  <tr>
    <td>Magahi</td>
    <td>Indic</td>
    <td>Eastern</td>
    <td>mag</td>
    <td>mag_Deva</td>
    <td>-</td>
    <td>-</td>
    <td><2M</td>
  </tr>
  <tr>
    <td>Maithili</td>
    <td>Indic</td>
    <td>Eastern</td>
    <td>mai</td>
    <td>mai_Deva</td>
    <td>-</td>
    <td>-</td>
    <td><50K</td>
  </tr>
  <tr>
    <td>Haitian Creole</td>
    <td>French-related Creole</td>
    <td>Creole</td>
    <td>hat</td>
    <td>hat_Latin</td>
    <td>ht</td>
    <td>hat</td>
    <td>HRL</td>
  </tr>
  <tr>
    <td>Guadeloupean Creole</td>
    <td>French-related Creole</td>
    <td>Antillean Creole</td>
    <td>gcl</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Martinican Creole</td>
    <td>French-related Creole</td>
    <td>Antillean Creole</td>
    <td>gcf</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Saint Lucia Patois</td>
    <td>French-related Creole</td>
    <td>Antillean Creole</td>
    <td>acf</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>French Guinean Creole</td>
    <td>French-related Creole</td>
    <td>French Guinean Creole</td>
    <td>gor</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Louisiana Creole</td>
    <td>French-related Creole</td>
    <td>Louisiana Creole</td>
    <td>lou</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Mauritian Creole</td>
    <td>French-related Creole</td>
    <td>Bourbonnais Creoles</td>
    <td>mfe</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Réunion Creole</td>
    <td>French-related Creole</td>
    <td>Bourbonnais Creoles</td>
    <td>lmo</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
  <tr>
    <td>Seychellois</td>
    <td>French-related Creole</td>
    <td>Bourbonnais Creoles</td>
    <td>crs</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><1K</td>
  </tr>
</table>

Table 6: Languages studied organized by language family. We also report the languages supported by the models used, as well resource-level. We report the number of Wikipedia articles for a language as a proxy for its resource level.

are supported by M2M, and the HRL language token otherwise. Note that the tokenizer itself is shared across all languages, as well as encoder parameters. While decoder parameters have language-family-specific components depending on the target language, this is irrelevant for us as we always have English as our target language.

D Bilingual Lexicon Collection for D→M

Our D→M approach uses CRL-HRL bilingual lexicons for an inference-time intervention. Ideally, we would like the lexicons to cover fully inflected words and all parts-of-speech.

D.1 Survey of lexicons, and challenges in collection

See a list of the language-pair specific bilingual lexicons we considered in Table 29: as shown here, many lexicons list lemmas rather than inflected words, rendering them unusable for our purpose. This is naturally particularly a problem for morphologically rich languages.

APIs and web apps Many available sources for bilingual lexicons are APIs and web apps. These include but are not limited to Google Translate 15, Microsoft Bing 16, ModernMT API 17, Alibaba

15 https://translate.google.com/
16 https://www.bing.com/translator
17 https://www.modernmt.com/translate
Translate 18, tradukka 19, freelang 20, From-to.io 21, iTranslate 22, and Glosbe 23. However:

• Querying is often problematic. APIs are often blocked by some paywall and mining the databases of web apps may be considered unethical or illegal. This is the case for The Living Arabic Project 24, a database of endangered Arabic dialects.

• The quality of such resources can be poor. A study from the Kamusi Project found that Google Translate performed poorly on some of the study’s languages, namely Plateau Malagasy, Ilocano, Samoan, and Maori (Benjamin, 2019). Glosbe is a crowdsourced database, meaning the translations may not be accurate.

Programmatic readability Some lexicons that exist as a PDF may have text that is difficult to extract. For example, the Peace Corps Tunisian Arabic Dictionary (Abdelkader et al., 1977) is grainy, making it difficult to perform OCR.

English-centric resources While we could find many lexicons that translated from a low/mid-resource language into English, we had trouble finding lexicons that translated from a low/mid-resource language into a related non-English resource language (see Table 29). This underscores the existing bias toward English-centric resources and technologies in natural language processing (Rigouts Terryn and de Lhoneux, 2024).

D.2 Lexicon sizes
See Tables 30, 31, and 32 for lexicon sizes of the lexicons we aggregated and used for each CRL-HRL pair, with a breakdown by function and content word counts. In these tables, we also report the word type coverage, a measure of how extensively the lexicon documents a LRL. Coverage is calculated as the percentage of unique words in the FloRes dev set/JHU Bible test set that are documented in the associated LRL lexicon.

Note that we only use publicly available data resources.

18https://translate.alibaba.com/
19https://tradukka.com/
20https://www.freelang.net/
21https://from-to.io/
22https://itranslate.com/
23https://glosbe.com/
24https://www.livingarabic.com/

E Variant approaches and hyperparameter exploration

E.1 Choosing noising dials
The search space for \( \theta^{p,m,f,c} \), i.e., the noising dials used to generate artificial dialectal data, is large. We conduct some small-scale experiments to understand the impact of this choice on the M→D approaches.

Default choices for \( \theta^{p,m,f,c} \) We use the noiser framework as set up in Bafna et al. (2024b). Within the Bayesian generative framework of these noisers, it is possible to compute the posteriors (MLE estimates) of \( \theta^{p} \) independently for each noiser, given a real CRL-HRL pair. Bafna et al. (2024b) compute and provide these posteriors over a number of CRL-HRL pairs for a few different language families, that include languages from four of the six language families that we work with. Our default choices for these parameters depend on the observed range of these \( \theta \)-posteriors over 18 real language pairs. For M→D-shell, we take a simple average of these posteriors to give us a single \( \theta^{p,m,f,c}\)-radius, which serves as a reasonable default distance between a random CRL and HRL. For M→D-cloud, given that we generate artificial languages on several hyperspheres up to a maximum \( \theta^{p,m,f,c}\)-radius, we take maximums over the observed posteriors instead in each dimension, discarding some clear outliers. While this yields reasonable defaults for the phonological, morphological, and function word noisers (which all model cognate processes), we find that this is not ideal for the content word noiser, as discussed next.

Setting \( \theta^{c} \) Our initial experiments indicate that setting \( \theta^{c} \), i.e. the content word noiser dial in accordance with the procedure for calculating other noiser dials is suboptimal, and much lower \( \theta^{c} \)'s are better. This makes sense: the content word noiser is the only noiser that models non-cognate processes, and introduces non-words with no connection to the source word into the noised text. This is naturally not helpful, since the introduced non-words have no systematic relationships with the target dialects, and there is no way for the model to generalize what it observes. Therefore, we set \( \theta^{c} = 0.001 \), i.e., very low, and largely rely on and discuss the other three noisers for this work (as in § 5).

Effect of hyperparameters on M→D-cloud We tried more aggressive noising for Indic, Turkic, and
Haitian, using \( \theta_{0.2,0.8,0.8,0.001}^{p,m,f,e} \) as the max-radius for M→D–cloud; i.e. significantly increasing \( \theta^p \) and \( \theta^m \) (the \( \theta^f \) default is already high at 0.8). This only gave minor variations on existing results for all language families and models. This is not very surprising: since M→D–cloud samples from many radii through the hypersphere up to the max-radius, it is relatively less sensitive to the choice of the max-radius \( \theta^{p,m,f,e} \).

Effect of hyperparameters on M→D–shell On the other hand, M→D–shell only samples from the shell defined by the radius, and therefore might be more sensitive to this choice. We observe that it performs slightly better or worse than M→D–cloud (see Table 19 and Table 20) depending on the language and model. Given a specific target language family, it may make sense to conduct a hyperparameter search for \( \theta \) for optimal M→D–shell performance. Our small-scale experiments with Indic with a few different \( \theta \)'s indicate only minor improvements from this search over our default parameters; however, this may vary by language family and model. Given the intractability of a systematic hyperparameter search over 3-dimensional \( \theta \)-radius (excluding the non-cognate lexical noiser, i.e. \( \theta^{p,m,f} \)), we instead show that it is possible to approximate all cognate processes solely with \( \theta^p \); we discuss this in § 5.

Number of radii \( N \) for M→D–cloud For the approach M→D–cloud, we use \( N \) radii at uniform intervals from 0 to max-radius \( \theta \); thus, \( N \) defines the density of the cloud given a max-radius. Of course, the total amount of data used remains constant regardless of \( N \) and \( \theta \). We compute results for Indic for the choice of number of radii \( K \in \{5, 10, 20\} \) and observe very minor differences. We use \( N = 10 \) throughout the paper. Note that since \( \theta \) is a probability, \( N = 10 \) is already enough to approximate an effectively continuous cloud up to the max-radius. While there is no cost to increasing \( N \), we observe no gains from doing so beyond a certain point.

Parametrizing randaug–shell and –cloud We can consider randaug to consist of a character-level and a word-level noiser. The difference from our noisers lies only in the target generation: while our noisers choose linguistically plausible targets as described in § 2, the randaug noisers choose random equivalents in accordance with previous literature. The character-level noiser is analogous to the M→D phonological noiser which affects characters, and the word-level noiser with the content word noiser which affects lexemes. For a fair comparison and in order to test the importance of linguistically-motivated target generation, we therefore set the dials for these noisers to be the same as the default dials for their analogous noisers, both for -shell and –cloud variants. We use \( \theta_{rc,wr}^{0.05,0.001} \) for randaug–shell and \( \theta_{rc,wr}^{0.07,0.001} \) as the max-radius for randaug–cloud. For the latter, we use \( K = 10 \) hyperspheres, consistent with M→D–cloud. Small tuning experiments over the Indic family yield minor variations in the baseline results obtained with these parameters. Note that the morphological and function word noisers do not have random baseline equivalents in the literature: these are inherently linguistically-motivated and inspired by patterns of dialectal variation.

E.2 Multiple HRLs
A language family may have more than one HRL, and the LRLs in the family may be closer to any of them. We experiment with using two HRLs as sources for noising, additionally using French for the Romance family, Uzbek for Turkic, and Tunisian for the Arabic family, along with the primary HRLs. We still maintain the total amount of finetuning data, but split it uniformly between our source HRLs. We choose Uzbek since it differs from Turkish in having a high amount of Russian-influenced and Persian-influenced vocabulary, similar to crh_Latn. Similarly, we choose Tunisian as a representative of the Maghrebi Arabic dialects, hoping to help with Moroccan (ary_Arab) which also belongs to this subfamily. See Table 21 for the results for Aya-23. We see that this helps for some Romance dialects that share high similarities with French and Spanish (such as oci_Latn and glg_Latn). Similarly, for Arabic, this gives a considerable performance boost for Moroccan Arabic. However, we don’t observe any improvements for the Turkic family except Uzbek itself.

E.3 Different datasets
We experiment with different choices of base dataset for noising for Indic: namely, indiccorp (Doddapaneni et al., 2023) and the NLLB dataset (Team et al., 2022). We observed that the choice of dataset matters for the absolute performance of the finetuned model (fthr1): specifically, finetuning with indiccorp actually hurts the baseline performance of both models. However, the gains from
the noising method over this finetuned baseline are consistent across these three datasets, for the Indic family. We present results on indiccorp in Table 22); those on the NLLB dataset show similar trends (see Table 23).

Training hyperparameters  We observe that increasing either the amount of data used or number of epochs worsens ordinary HRL fine-tuning results slightly for CRLs, while M→D approaches show small improvements for some languages. We fix these hyperparameters for all experiments (i.e. 100K sentences of bitext, a single epoch) for ease of comparison.

E.4 Using CRL-Eng swapping

We experiment with switching CRL words directly into English instead of the HRL: as expected, this degrades performance considerably since the model now receives unnatural CRL-English code-mixed input. See Table 39 and Table 38 for results.

E.5 Enhancing noisers to model lexical variation: Semantic noise

We would like to model different patterns of lexical usage among closely-related languages. For example, while the word “book” is “pustak” in Marathi and “kitab” in Hindi by common usage, “pustak” is also an entirely comprehensible Hindi word. We would like our noisers therefore to expose the model during finetuning to various lexical variants of a given concept as they might be realized in different dialects of a language family. We use WordNets in all our HRls: IndoWordNet for Hindi (Bhattacharyya, 2010), TurkishWordNet (Bakay et al., 2021), Arabic WordNet (AWN v2) (Regragui et al., 2016), WordNet Bahasa (Noor et al., 2011), and Italian Wordnet (Roventini et al., 2000) to “noise” a given word with some probability \( \theta^s \) to its (randomly chosen) synonym. Note that since these wordnets contain lemmas, whereas we typically encounter inflected forms in text, we first lemmatize the word to be noised, perform semantic noising, and then re-append its original inflections, to roughly maintain grammaticality. This process may introduce incidental noise at inflection boundaries, but is tolerable given our general goal of perturbing the data.

This noised and inflected synonym then undergoes the other noising processes as per usual, modeling the intuition that the diverging lexeme would still bear the effects of general phonological and morphological change in a given dialect.

We set \( \theta^s \in \{0.1, 0, 3\} \), keeping other noising parameters constant; however, we do not see that this helps much. See language family means for M2M in Table 24.

F Function word identification for D→M

HRL  M→D requires function word identification in the HRL, for the function word lexical noiser. We use the same procedure for function word identification in the HRL as used by the lexical noiser in differential between functional and content words, consistent with Bafna et al. (2024b), summarized in Appendix B, to create a list of HRL function words.

CRLs  The D→M-func approach only affects function words in the CRL input, and therefore requires function word identification for CRL text. Given that we have a set of HRL function words and HRL-CRL bilingual lexicons, we can identify function words in the CRL using projection of POS tags from the HRL.

Comparing against natural distribution  The impact of our best-performing variant D→M-func depends on the identification of function words in the CRL input as described above. We would like to evaluate the coverage of this method; however, we lack annotated data for the CRLs for this evaluation. Instead, assuming that language families share general distributive properties of function words, we compare the percentage of identified function words in our CRL input against the natural percentage of function words in the HRL (as a representative of the language family) in the UD corpus. See Table 36. We observe that D→M affects an expected percentage of words for Romance, Turkic, and Indic, but there is over-identification for Austronesian and under-identification for Arabic, possibly due to noisier automatic alignments for these families.

See Table 37 for a summary of the number of (1) content, (2) functional, and (3) content and functional combined (represented as “all” in the table) words switched out in each D→M approach.

G More examples

See examples of M→D and D→M outputs in Table 7 and Table 8.
Input | ई छेत्र में एक एंटीबॉडी कॉक्टेल , जेसे नाम ZMapp बा , शुल में अच्छा नतीजा देखियाँ , लेकिन अध्ययन कहाँ पर पता चलने कि इस विश्लेषण में होने वाला मौत रोके खातिर आरोग्य फायदेमंद नहीं जतना फर्क बराबर बनाया गया था .
Reference | One antibody cocktail, ZMapp, initially showed promise in the field, but formal studies indicated it had less benefit than sought in prevention death.
Baseline | The egocentric body cocktail, Jake named ZMapp Ba, initially had a good outcome, but the studies in Kila revealed that the deaths caused by HIV/AIDS in order to prevent HIV/AIDS remained beneficial to the awareness of HIV/AIDS.
Output | The antibody cocktail, ZMapp, was initially a good result, but studies have shown that it has been a very beneficial outcome in preventing deaths from Ebola.

<table>
  <tr>
    <th>Input</th>
    <td>Ind ol test PALM, ZMapp a l’è stait drovad coma contrôll, ‘l saria a dir qe a i sienziats a i ’l han drovad coma basament e a i hann comparad a i olter tré trataments a quest qi.</td>
  </tr>
  <tr>
    <th>Reference</th>
    <td>In the PALM trial, ZMapp served as a control, meaning scientists used it as a baseline and compared the three other treatments to it.</td>
  </tr>
  <tr>
    <th>Baseline</th>
    <td>In the test PALM, ZMapp a l’è stait drovad coma contrôll, ‘l would be to say that to the sienziats to the l’han drovad coma basament and to the hann compared to the olter tré treatments to this qi.</td>
  </tr>
  <tr>
    <th>M→D</th>
    <td>In the PALM test, ZMapp’s standing as a control would be to say that the subjects were found as a base and compared to the other treatments to this qi.</td>
  </tr>
  <tr>
    <th>Output</th>
    <td></td>
  </tr>
</table>

Input | Nen Lunes, si Sara Danius, say permanentin sekretaryo na Nobel Committee for Literature ed Swedish Academy et inyanunsio tod publiko diad sakey a programa ed Sveriges Radio diad Sweden, ya aga direktan nakontak na komite si Bob Dylan diad impanalo tod 2016 Nobel Prize in Literatura, kaman agla ra nansagpot ya sikatoy kontaken lamet.
Reference | On Monday, Sara Danius, permanent secretary of the Nobel Committee for Literature at the Swedish Academy, publicly announced during a radio program on Sveriges Radio in Sweden the committee, unable to reach Bob Dylan directly about winning the 2016 Nobel Prize in Literature, had abandoned its efforts to reach him.
Baseline | Nen Lunes, si Sara Danius, say permanentin sekretaryo na Nobel Committee for Literature ed Swedish Academy et inyanunsio tod publiko diad sakey a programa ed Sveriges Radio diad Sweden, ya aga direktor nakontakta na committee si Bob Dylan diad impanalo tod 2016 Nobel Prize in Literature, kaman agla ra nansagpot ya sikatoy kontaken lamet.
M→D | On Monday, Sara Danius was the permanent secretary of the Nobel Committee for Literature and the Swedish Academy and the public in Sweden’s Sveriges Radio program, and the director of Bob Dylan’s committee for the 2016 Nobel Prize in Literature.

<table>
  <tr>
    <th>Input</th>
    <td>Lunedi, Sara Danius, segretaria permanenti du cumitatu Nobel pà littiratura ntà cademia Lunedi, Sara Danius, segretaria permanenti del cumitatu Nobel per littiratura nella cademia svidisì, dissi pubblicamenti nna nu programma radiu di Radio Sveriges nna Svezia ca lu svidisì, ha pubblicamenti in un programma radiu di Radio Sveriges in Svezia che il cumitatu, nun putennu rintracciari direttamente Bob Dylan e nfirmàriu ca avia vinciu tu cumitatu, non putennu rintracciari direttamente Bob Dylan e nfirmàriu che aveva vinciu tu premiu Nobel pà littiratura 2016, smisi di pruvari a cuntattàrilu.
premiu Nobel per littiratura 2016, smisi di cercare a cuntattàrilu.
</td>
  </tr>
  <tr>
    <th>Reference</th>
    <td>On Monday, Sara Danius, permanent secretary of the Nobel Committee for Literature at the Swedish Academy, publicly announced during a radio program on Sveriges Radio in Sweden the committee, unable to reach Bob Dylan directly about winning the 2016 Nobel Prize in Literature, had abandoned its efforts to reach him.</td>
  </tr>
  <tr>
    <th>Original Output</th>
    <td>On Monday, Sara Danius, permanent secretary due cumitatu Nobel pà littiratura ntà cademia svidisì, said publications nna nu programma radiu di Radio Sveriges nna Sweden ca lu cumitatu, nun putennu rintracciari direttamente Bob Dylan e nfirmàriu ca avia vinciu tu premiu Nobel pà littiratura 2016, cessau pruvari a cuntattàrilu.
On Mondad, Sara Danius, permanent secretary of the Nobel laureate for literature at the Swedish Academy, has published in a radio program of Radio Sveriges in Sweden that the cumitatu, could not trace Bob Dylan and nfirmàriu who had won the Nobel prize for literature 2016, stopped looking at cuntattàrilu.
</td>
  </tr>
  <tr>
    <th>D→M Output</th>
    <td>On Monday, Sara Danius, permanent secretary of the Nobel Committee for Literature at the Swedish Academy, publicly announced during a radio program on Sveriges Radio in Sweden that the committee, unable to reach Bob Dylan directly about winning the 2016 Nobel Prize in Literature, had abandoned its efforts to reach him.
On Monday, Sara Danius, permanent secretary of the Nobel Committee for Literature at the Swedish Academy, has published in a radio program of Radio Sveriges in Sweden that the committee, could not trace Bob Dylan and nfirmàriu who had won the Nobel prize for literature 2016, stopped looking at cuntattàrilu.
</td>
  </tr>
</table>

<table>
  <tr>
    <th>Input</th>
    <td>हनका समक अनुसारित रथा , गेड साग खेलबाक कोशल आ बेस नोक टीम प्रयास हनका सम के आन से अलग टीम बनावत उन्हें के अनुसारित रथा , गेड साग खेलबाक कोशल और ने नोक टीम प्रयास उन्हें के को आन से अलग टीम बनावत छल आ ई समु भ नेत छल जे ई हावान वना टीम अधिक। था और वह समु भ नेता था जो यह हावान वना टीम है।</td>
  </tr>
  <tr>
    <th>Reference</th>
    <td>Their disciplined defence, ball handling skills and excellent team work made them stand out and it was clear that this was the team to beat.</td>
  </tr>
  <tr>
    <th>Original Output</th>
    <td>Jhxfyi/jhjhjhhkhdhtdfnhjihb hydhhn hey nah gun hvbyj hvbyj him bun jvyctsgnhh tcj her n f u him g j t him ubufybhknugyfybj hvbyj much hhi b mhyvghtdj gf</td>
  </tr>
  <tr>
    <th>D→M Output</th>
    <td>They had disciplined defense, backbone skills with the ball and made a nick team effort to make them a different team from K'u'lu and it was clear that this was a bad team.</td>
  </tr>
</table>
H  Wins and variance for approach paradigms

See Figure 7 and Figure 8 for a comparison of win rates over baseline and improvement distributions for different paradigms for M2M and Aya-23 respectively, for all language families.

I  Manual Evaluation of Function Word Lexicons

<table>
  <tr>
    <th>lang</th>
    <th>FW acc. (%)</th>
    <th>FW Id. (%)</th>
    <th>Gen. acc. (%)</th>
  </tr>
  <tr>
    <td>fra_ita</td>
    <td>67.6</td>
    <td>67.0</td>
    <td>60.4</td>
  </tr>
  <tr>
    <td>bho_hin</td>
    <td>97.9</td>
    <td>51.6</td>
    <td>74.7</td>
  </tr>
  <tr>
    <td>arz_arb</td>
    <td>84.0</td>
    <td>32.9</td>
    <td>38.2</td>
  </tr>
  <tr>
    <td>azj_tur</td>
    <td>84.9</td>
    <td>74.5</td>
    <td>65.3</td>
  </tr>
  <tr>
    <td>Mean</td>
    <td>83.6</td>
    <td>56.5</td>
    <td>59.7</td>
  </tr>
</table>

Table 9: Function word (FW) accuracy (%), function word identification accuracy (FW Id, %), general translation accuracy (%) in function word lexicons

The evaluation was conducted by three of our authors, fluent in Turkish (tur), Hindi (hin), Arabic (arb), Egyptian (ary), and French (fra), with a working understanding of the respective CRLs and Italian (ita). See Table 9.

J  Training and evaluation details

J.1 Compute

Our noising approaches and baselines require fine-tuning: we had 2 models x 8 approaches x 6 HRLs = 96 finetuning runs. Each experiment required about 3 hours on a single A100, totaling 288 GPU hours.

We conducted evaluation for 54 languages (49 CRLs + 6 HRLs) x 15 total approaches = 810 evaluation runs. Each evaluation took 30 minutes on a single GTX 1080 machine, totaling 405 GPU hours.

These calculations do not include development and initial experiments, and ablation studies.

J.2 Evaluation

We tested our models on the devtest splits of the FloRes-200 dataset, using BLEU (Hugging-Face evaluate wrapper: mixed case, tokenization with sacrebleu 13a) as well as COMET (Unbabel/wmt22-comet-da).

K  Results by language and approach

We summarize our results in the following.

• Table 10 and Table 11 detail the average BLEU and COMET scores respectively for each approach within the M→D, D→M, and M↔D paradigms for each language family. Table 12 provides COMET score means for low-baseline CRLs (<25 BLEU) per language family.

• Table 13, Table 14, Table 15, and Table 16 present the BLEU and COMET scores of best-performing variant each of the baseline, M→D, D→M, and M↔D approaches, for M2M and Aya-23. Note that the best approach is chosen on a family-wide basis, in order to recommend a general strategy for a given language family, by comparing means over the language family for each variant, and may not reflect the best scores for an individual language in that paradigm (similarly for the best baseline). This is different from how best variants are chosen for Figure 2, Figure 3, Figure 7, and Figure 8, which consider best performing paradigms variants on a per-language basis.

The following tables provide a more detailed view of the performance of each paradigm.

• M→D: Table 17, Table 18, Table 19, and Table 20 detail the BLEU and COMET performance for each language in our language family families using each approach in the M→D paradigm. Table 7 provides a qualitative example of how translations improved following the M→D paradigm.

• D→M: Table 25, Table 26, Table 27, and Table 28 detail the BLEU and COMET performance for each language in our language family families using each approach in the D→M paradigm. Table 8 provides a qualitative example of how translations improved following the D→M paradigm.

• M↔D: Table 40, Table 41, Table 42, and Table 43 detail the BLEU and COMET performance for each language in our language family families using each approach in the M↔D paradigm.

L  Use of AI assistants

We used GitHub copilot for coding assistance. No AI assistants were used for any writing purposes.
Figure 7: BLEU score improvements over the best baseline with M2M for all language families. \( \uparrow \) and \( \downarrow \): # CRLs with positive/negative gains. M→D gives more consistent positive gains.

![Boxplots showing BLEU score improvements for AUS, ARA, ROM, TUR, IND, CRE with M→D, D→M, M↔D for each](page_184_153_1080_496.png)

Figure 8: BLEU score improvements over the best baseline with Aya-23 for all language families. \( \uparrow \) and \( \downarrow \): # CRLs with positive/negative gains. M→D gives more consistent positive gains.

![Boxplots showing BLEU score improvements for AUS, ARA, ROM, TUR, IND, CRE with M→D, D→M, M↔D for each](page_184_728_1080_496.png)
<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="8">M2M</th>
    <th colspan="8">Aya-23</th>
  </tr>
  <tr>
    <th>AUS (9)</th><th>ARA (7)</th><th>ROM (6)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (8)</th><th>Mean</th>
    <th>AUS (9)</th><th>ARA (1)</th><th>ROM (5)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (9)</th><th>Mean</th>
  </tr>
  <tr>
    <td>Baselines</td>
    <td>off-the-shelf</td><td>14.9</td><td>21.2</td><td>26.6</td><td>5.0</td><td>12.6</td><td>5.9</td><td>14.3</td><td>11.9</td><td>31.5</td><td>29.5</td><td>7.9</td><td>20.4</td><td>7.3</td><td>18.1</td>
  </tr>
  <tr>
    <td></td>
    <td>fthr1</td><td>+0.3</td><td>+0.3</td><td>+0.4</td><td>-0.1</td><td>+0.4</td><td>+0.9</td><td>+0.4</td><td>-1.0</td><td>-0.7</td><td>+1.2</td><td>-1.8</td><td>+2.2</td><td>-0.0</td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>randaug</td><td>+0.2</td><td>+0.2</td><td>+0.2</td><td>+0.2</td><td>+0.2</td><td>+0.5</td><td>+0.5</td><td>+0.5</td><td>+0.2</td><td>+1.1</td><td>+1.8</td><td>+2.1</td><td>+0.0</td><td></td>
  </tr>
  <tr>
    <td>M→O</td>
    <td>-shell</td><td>+1.7</td><td>+1.0</td><td>+5.1</td><td>+2.5</td><td>+5.0</td><td>+2.7</td><td>+3.0</td><td>+1.1</td><td>-0.2</td><td>+2.4</td><td>+2.9</td><td>+2.8</td><td>+3.0</td><td>+2.0</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud</td><td>+1.1</td><td>+0.7</td><td>+3.9</td><td>+1.8</td><td>+5.4</td><td>+2.7</td><td>+2.6</td><td>+1.2</td><td>+0.0</td><td>+2.5</td><td>+3.2</td><td>+2.7</td><td>+3.5</td><td>+2.2</td>
  </tr>
  <tr>
    <td>D→M</td>
    <td>-cont</td><td>-1.5</td><td>-2.0</td><td>-7.1</td><td>+1.4</td><td>+0.3</td><td>+2.5</td><td>-1.1</td><td>-0.1</td><td>-4.3</td><td>-7.6</td><td>-0.6</td><td>-1.1</td><td>-3.8</td><td>-2.9</td>
  </tr>
  <tr>
    <td></td>
    <td>-func</td><td>+1.2</td><td>+0.2</td><td>+2.8</td><td>+1.0</td><td>+1.0</td><td>+3.7</td><td>+2.2</td><td>+0.7</td><td>+1.1</td><td>+1.4</td><td>+0.6</td><td>+5.6</td><td>+2.7</td><td>+0.1</td>
  </tr>
  <tr>
    <td></td>
    <td>-all</td><td>+2.1</td><td>+1.2</td><td>+7.0</td><td>+1.0</td><td>+1.0</td><td>+5.6</td><td>+1.2</td><td>+1.5</td><td>+3.0</td><td>+8.0</td><td>+4.4</td><td>+7.5</td><td>+1.8</td><td></td>
  </tr>
  <tr>
    <td>M→D</td>
    <td>-cloud-cont</td><td>+0.1</td><td>-1.4</td><td>-2.7</td><td>+2.6</td><td>+5.1</td><td>+3.7</td><td>+1.2</td><td>+1.6</td><td>+3.6</td><td>+4.0</td><td>+1.8</td><td>+1.6</td><td>+2.6</td><td>+0.0</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-func</td><td>+0.3</td><td>+1.1</td><td>+2.3</td><td>+3.0</td><td>+4.6</td><td>+3.8</td><td>+1.9</td><td>+0.8</td><td>+0.5</td><td>+2.1</td><td>+6.0</td><td>+2.8</td><td>+2.2</td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-all</td><td>+1.3</td><td>+1.1</td><td>+3.3</td><td>+3.9</td><td>+12.2</td><td>+6.1</td><td>+2.6</td><td>+2.8</td><td>+4.0</td><td>+4.9</td><td>+2.1</td><td>+4.6</td><td>+2.5</td><td>+0.5</td>
  </tr>
</table>

Table 10: Comparison of language family BLEU means by model. Performance gains/loss are relative to off-the-shelf. The overall best score is bolded and the best score in each paradigm is underlined.

<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="8">M2M</th>
    <th colspan="8">Aya-23</th>
  </tr>
  <tr>
    <th>AUS (9)</th><th>ARA (7)</th><th>ROM (6)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (8)</th><th>Mean</th>
    <th>AUS (9)</th><th>ARB (1)</th><th>ROM (5)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (9)</th><th>Mean</th>
  </tr>
  <tr>
    <td>Baselines</td>
    <td>off-the-shelf</td><td>52.9</td><td>72.6</td><td>67.9</td><td>47.6</td><td>64.2</td><td>47.6</td><td>58.8</td><td>55.2</td><td>81.7</td><td>74.0</td><td>60.7</td><td>76.3</td><td>47.6</td><td>66.0</td>
  </tr>
  <tr>
    <td></td>
    <td>fthr1</td><td>+0.7</td><td>-0.1</td><td>-0.6</td><td>-0.6</td><td>+0.0</td><td>-0.3</td><td>-0.1</td><td>-2.0</td><td>+0.2</td><td>+0.9</td><td>+1.3</td><td>+1.7</td><td>+6.2</td><td>+1.3</td>
  </tr>
  <tr>
    <td></td>
    <td>randaug-shell</td><td>+0.1</td><td>-0.5</td><td>-0.2</td><td>-0.3</td><td>-0.0</td><td>-0.3</td><td>-0.2</td><td>-1.6</td><td>+0.3</td><td>+1.5</td><td>+4.3</td><td>+1.8</td><td>+5.7</td><td>+1.4</td>
  </tr>
  <tr>
    <td>M→O</td>
    <td>-shell</td><td>+1.3</td><td>+1.3</td><td>+8.3</td><td>+8.0</td><td>+8.0</td><td>+5.6</td><td>+5.6</td><td>+3.8</td><td>+1.0</td><td>+5.1</td><td>+8.3</td><td>+2.8</td><td>+7.5</td><td>+4.8</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud</td><td>+2.9</td><td>+4.0</td><td>+7.0</td><td>+4.4</td><td>+8.7</td><td>+3.5</td><td>+4.5</td><td>+4.1</td><td>+1.1</td><td>+5.2</td><td>+8.2</td><td>+3.0</td><td>+7.8</td><td>+4.9</td>
  </tr>
  <tr>
    <td>D→M</td>
    <td>-cont</td><td>+0.3</td><td>-2.7</td><td>-8.7</td><td>+5.2</td><td>+0.8</td><td>+1.2</td><td>+0.5</td><td>-4.1</td><td>-8.6</td><td>-3.7</td><td>-1.3</td><td>-1.3</td><td>-0.2</td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>-func</td><td>-0.6</td><td>+0.5</td><td>+2.6</td><td>+2.9</td><td>+10.8</td><td>+4.7</td><td>+1.7</td><td>+0.7</td><td>+0.9</td><td>+0.1</td><td>+2.1</td><td>+2.1</td><td>+2.6</td><td>+0.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-all</td><td>+1.3</td><td>+2.3</td><td>+1.1</td><td>+2.9</td><td>+10.8</td><td>+5.1</td><td>+3.5</td><td>+3.0</td><td>+0.0</td><td>+1.4</td><td>+1.7</td><td>+1.4</td><td>+1.7</td><td>+0.3</td>
  </tr>
  <tr>
    <td>M→D</td>
    <td>-cloud-cont</td><td>+4.7</td><td>-1.9</td><td>+1.0</td><td>+8.3</td><td>+7.8</td><td>+5.8</td><td>+5.7</td><td>-1.7</td><td>-1.0</td><td>+5.9</td><td>+1.6</td><td>+5.8</td><td>+5.8</td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-func</td><td>+2.6</td><td>+1.0</td><td>+5.9</td><td>+6.5</td><td>+12.8</td><td>+8.0</td><td>+3.0</td><td>+0.5</td><td>+8.0</td><td>+8.0</td><td>+4.1</td><td>+4.1</td><td>+4.2</td><td>+2.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-all</td><td>+3.5</td><td>+1.8</td><td>+2.6</td><td>+9.8</td><td>+11.4</td><td>+6.3</td><td>+4.8</td><td>-2.3</td><td>-2.4</td><td>+5.6</td><td>+2.8</td><td>+5.4</td><td>+5.4</td><td></td>
  </tr>
</table>

Table 11: Comparison of language family COMET means by model. Performance gains/losses are relative to off-the-shelf. The overall best score is bolded and best score in each paradigm is underlined.

<table>
  <tr>
    <th rowspan="2"></th>
    <th colspan="8">M2M</th>
    <th colspan="8">Aya-23</th>
  </tr>
  <tr>
    <th>AUS (9)</th><th>ARA (7)</th><th>ROM (6)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (8)</th><th>Mean</th>
    <th>AUS (9)</th><th>ARB (1)</th><th>ROM (5)</th><th>TUR (4)</th><th>IND (4)</th><th>CRE (9)</th><th>Mean</th>
  </tr>
  <tr>
    <td>Baselines</td>
    <td>off-the-shelf</td><td>46.9</td><td>71.5</td><td>44.7</td><td>47.6</td><td>64.2</td><td>47.6</td><td>53.8</td><td>49.5</td><td>74.0</td><td>58.4</td><td>60.7</td><td>76.3</td><td>47.6</td><td>61.1</td>
  </tr>
  <tr>
    <td></td>
    <td>fthr1</td><td>+0.7</td><td>-0.1</td><td>-1.3</td><td>-0.6</td><td>+0.0</td><td>-0.3</td><td>-0.2</td><td>-2.0</td><td>+0.2</td><td>+2.8</td><td>+1.3</td><td>+1.7</td><td>+6.2</td><td>+1.6</td>
  </tr>
  <tr>
    <td></td>
    <td>randaug-shell</td><td>+0.2</td><td>-0.5</td><td>-0.3</td><td>-0.3</td><td>-0.3</td><td>-0.2</td><td>-1.6</td><td>+0.6</td><td>+4.0</td><td>+4.3</td><td>+1.8</td><td>+5.7</td><td>+1.9</td><td></td>
  </tr>
  <tr>
    <td>M→O</td>
    <td>-shell</td><td>+5.1</td><td>+1.5</td><td>+19.4</td><td>+8.0</td><td>+8.0</td><td>+5.6</td><td>+7.6</td><td>+5.3</td><td>+2.7</td><td>+12.7</td><td>+8.3</td><td>+2.8</td><td>+7.5</td><td>+6.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud</td><td>+3.4</td><td>+1.0</td><td>+16.5</td><td>+4.4</td><td>+8.7</td><td>+3.5</td><td>+6.2</td><td>+5.5</td><td>+2.9</td><td>+12.7</td><td>+8.2</td><td>+3.0</td><td>+7.8</td><td>+6.7</td>
  </tr>
  <tr>
    <td>D→M</td>
    <td>-cont</td><td>+2.0</td><td>-3.1</td><td>+2.8</td><td>+5.2</td><td>+0.8</td><td>+2.8</td><td>+1.9</td><td>-6.5</td><td>-2.1</td><td>-3.7</td><td>-1.3</td><td>-0.4</td><td></td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>-func</td><td>+1.0</td><td>+0.6</td><td>+9.0</td><td>+2.9</td><td>+10.8</td><td>+6.1</td><td>+0.1</td><td>+1.7</td><td>+4.5</td><td>+0.1</td><td>+2.1</td><td>+2.1</td><td>+2.6</td><td>+0.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-all</td><td>+4.4</td><td>+2.0</td><td>+11.5</td><td>+5.6</td><td>+10.8</td><td>+5.1</td><td>+3.5</td><td>+3.0</td><td>+0.0</td><td>+1.4</td><td>+1.7</td><td>+1.4</td><td>+1.7</td><td>+0.3</td>
  </tr>
  <tr>
    <td>M→D</td>
    <td>-cloud-cont</td><td>+7.2</td><td>+2.2</td><td>+13.5</td><td>+8.3</td><td>+7.8</td><td>+8.3</td><td>+8.4</td><td>+1.7</td><td>+0.2</td><td>+5.9</td><td>+1.6</td><td>+5.8</td><td>+5.8</td><td></td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-func</td><td>+4.4</td><td>+1.2</td><td>+10.8</td><td>+6.5</td><td>+12.8</td><td>+8.0</td><td>+3.0</td><td>+0.5</td><td>+8.0</td><td>+8.0</td><td>+4.1</td><td>+4.1</td><td>+4.2</td><td>+2.6</td>
  </tr>
  <tr>
    <td></td>
    <td>-cloud-all</td><td>+6.6</td><td>+2.0</td><td>+17.3</td><td>+9.8</td><td>+11.4</td><td>+6.3</td><td>+4.8</td><td>-2.3</td><td>-2.4</td><td>+5.6</td><td>+2.8</td><td>+5.4</td><td>+5.4</td><td></td>
  </tr>
</table>

Table 12: Comparison of language family COMET means by model where baseline < 25. Performance gains/loss are relative to off-the-shelf. The overall best score for each language is bolded and the best score in each paradigm is underlined. Numbers in parenthesis represent the number of languages within the model and language family whose baseline off-the-shelf BLEU score was less than 25.
<table>
  <tr>
    <th>Language</th>
    <th>Best Baseline</th>
    <th>Best M→D</th>
    <th>Best D→M</th>
    <th>Best M↔D</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latn (Indonesian)</td>
    <td>38.9</td>
    <td>+0.2</td>
    <td>-0.2</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>jav_Latn (Javanese)</td>
    <td>27.8</td>
    <td>+0.7</td>
    <td>-0.2</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>sun_Latn (Sundanese)</td>
    <td>23.5</td>
    <td>+0.8</td>
    <td>+0.4</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <td>sinc_Latn (Samarai)</td>
    <td>2.4</td>
    <td>-0.2</td>
    <td>+2.0</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>mti_Latn (Malay)</td>
    <td>25.5</td>
    <td>+0.4</td>
    <td>+1.6</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>ceb_Latn (Cebuano)</td>
    <td>22.6</td>
    <td>+1.1</td>
    <td>-3.1</td>
    <td>-2.3</td>
  </tr>
  <tr>
    <td>zsm_Latn (Standard Malay)</td>
    <td>39.0</td>
    <td>+0.0</td>
    <td>-1.8</td>
    <td>-2.5</td>
  </tr>
  <tr>
    <td>tgl_Latn (Tagalog)</td>
    <td>27.8</td>
    <td>+0.7</td>
    <td>+0.6</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>ilo_Latn (Ilocano)</td>
    <td>14.3</td>
    <td>+1.5</td>
    <td>-0.4</td>
    <td>+1.4</td>
  </tr>
  <tr>
    <td>fj_Latn (Fijian)</td>
    <td>2.5</td>
    <td>+0.5</td>
    <td>+1.7</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>plk_Latn (Malagasy Malagasy)</td>
    <td>25.2</td>
    <td>+5.1</td>
    <td>+3.8</td>
    <td>+8.2</td>
  </tr>
  <tr>
    <td>pag_Latn (Pangasinan)</td>
    <td>6.7</td>
    <td>+4.0</td>
    <td>+5.2</td>
    <td>+7.2</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>15.2</td>
    <td>+1.3</td>
    <td>+0.3</td>
    <td>+1.2</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>11.1</td>
    <td>+1.5</td>
    <td>+1.3</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>29.9</td>
    <td>-0.1</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>22.9</td>
    <td>+0.4</td>
    <td>+0.5</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>aqc_Arab (Tif'izzi-Adenis Arabic)</td>
    <td>25.2</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>arb_Arab (Libyan Arabic)</td>
    <td>16.0</td>
    <td>+0.7</td>
    <td>+0.3</td>
    <td>+1.7</td>
  </tr>
  <tr>
    <td>ajj_Arab (South Levantine Arabic)</td>
    <td>24.6</td>
    <td>+0.5</td>
    <td>-0.9</td>
    <td>-1.2</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>21.7</td>
    <td>-0.1</td>
    <td>+1.0</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>20.7</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>13.1</td>
    <td>+1.9</td>
    <td>+0.8</td>
    <td>+2.4</td>
  </tr>
  <tr>
    <td>azr_Arab (Egyptian Arabic)</td>
    <td>18.3</td>
    <td>+1.0</td>
    <td>+0.6</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>21.6</td>
    <td>+0.6</td>
    <td>+0.4</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.5</td>
    <td>+0.7</td>
    <td>+0.5</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latn (Italian)</td>
    <td>31.7</td>
    <td>+0.0</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>spa_Latn (Spanish)</td>
    <td>27.1</td>
    <td>+0.2</td>
    <td>-7.5</td>
    <td>-4.2</td>
  </tr>
  <tr>
    <td>fra_Latn (French)</td>
    <td>41.6</td>
    <td>+0.2</td>
    <td>+1.3</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <td>por_Latn (Portuguese)</td>
    <td>46.0</td>
    <td>+0.1</td>
    <td>-15.1</td>
    <td>-8.8</td>
  </tr>
  <tr>
    <td>ron_Latn (Romanian)</td>
    <td>31.5</td>
    <td>-0.3</td>
    <td>-16.3</td>
    <td>-11.8</td>
  </tr>
  <tr>
    <td>glg_Latn (Galician)</td>
    <td>35.2</td>
    <td>+0.0</td>
    <td>-11.8</td>
    <td>-6.5</td>
  </tr>
  <tr>
    <td>cat_Latn (Catalan)</td>
    <td>32.9</td>
    <td>+0.0</td>
    <td>-10.0</td>
    <td>-5.8</td>
  </tr>
  <tr>
    <td>oci_Latn (Occitan)</td>
    <td>31.7</td>
    <td>+0.1</td>
    <td>-5.2</td>
    <td>-4.7</td>
  </tr>
  <tr>
    <td>ast_Latn (Asturian)</td>
    <td>24.7</td>
    <td>+0.7</td>
    <td>-8.0</td>
    <td>-6.1</td>
  </tr>
  <tr>
    <td>lim_Latn (Limburgish)</td>
    <td>10.1</td>
    <td>+1.6</td>
    <td>+1.2</td>
    <td>+2.4</td>
  </tr>
  <tr>
    <td>vec_Latn (Venetian)</td>
    <td>17.7</td>
    <td>+1.0</td>
    <td>+7.3</td>
    <td>+10.3</td>
  </tr>
  <tr>
    <td>scd_Latn (Sicilian)</td>
    <td>9.4</td>
    <td>+0.9</td>
    <td>+1.9</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>std_Latn (Sardinian)</td>
    <td>5.8</td>
    <td>+1.5</td>
    <td>+1.3</td>
    <td>+1.8</td>
  </tr>
  <tr>
    <td>fur_Latn (Friulian)</td>
    <td>8.6</td>
    <td>+1.9</td>
    <td>+1.0</td>
    <td>+1.4</td>
  </tr>
  <tr>
    <td>lg_Latn (Ligurian)</td>
    <td>10.9</td>
    <td>+1.1</td>
    <td>+1.0</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>27.1</td>
    <td>+0.6</td>
    <td>-2.4</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.4</td>
    <td>+0.7</td>
    <td>+0.4</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latn (Turkish)</td>
    <td>33.3</td>
    <td>-0.6</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>uzb_Latn (Northern Uzbek)</td>
    <td>1.9</td>
    <td>+0.4</td>
    <td>+4.7</td>
    <td>+5.1</td>
  </tr>
  <tr>
    <td>tuk_Latn (Turkmen)</td>
    <td>2.4</td>
    <td>+1.4</td>
    <td>+4.6</td>
    <td>+5.0</td>
  </tr>
  <tr>
    <td>azg_Latn (North Azerbaijani)</td>
    <td>7.6</td>
    <td>+4.0</td>
    <td>-0.1</td>
    <td>+3.2</td>
  </tr>
  <tr>
    <td>cft_Latn (Crimean Tatar)</td>
    <td>8.9</td>
    <td>+2.2</td>
    <td>+2.1</td>
    <td>+3.0</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>5.2</td>
    <td>+2.3</td>
    <td>+2.8</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>5.2</td>
    <td>+2.3</td>
    <td>+2.8</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>33.4</td>
    <td>-0.4</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>tne_Deva (Telugu)</td>
    <td>15.5</td>
    <td>+5.0</td>
    <td>+1.6</td>
    <td>+6.6</td>
  </tr>
  <tr>
    <td>bis_Deva (Bengali)</td>
    <td>10.6</td>
    <td>+3.3</td>
    <td>+6.6</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>18.0</td>
    <td>+4.5</td>
    <td>+12.0</td>
    <td>+13.5</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>8.1</td>
    <td>+7.1</td>
    <td>+16.2</td>
    <td>+16.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>13.0</td>
    <td>+5.0</td>
    <td>+12.0</td>
    <td>+12.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>13.0</td>
    <td>+5.0</td>
    <td>+12.0</td>
    <td>+12.8</td>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat_Haitian</td>
    <td>40.6</td>
    <td>+0.3</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>gcf_Guadeloupean</td>
    <td>5.7</td>
    <td>+1.7</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>marf_Haitian</td>
    <td>4.7</td>
    <td>-0.1</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>afc_Saint Lucia Patois</td>
    <td>2.8</td>
    <td>+1.3</td>
    <td>+7.1</td>
    <td>+8.4</td>
  </tr>
  <tr>
    <td>ger_French Guianese Creole</td>
    <td>2.5</td>
    <td>+2.8</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>lou_Louisiana Creole</td>
    <td>3.7</td>
    <td>+2.1</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>maf_Mauritian</td>
    <td>2.5</td>
    <td>+1.5</td>
    <td>+3.1</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>rcf_Réunion Creole</td>
    <td>16.3</td>
    <td>+1.1</td>
    <td>+3.9</td>
    <td>+4.3</td>
  </tr>
  <tr>
    <td>cis_Seychellois</td>
    <td>11.4</td>
    <td>+2.5</td>
    <td>+3.9</td>
    <td>+4.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>7.0</td>
    <td>+1.7</td>
    <td>+4.7</td>
    <td>+5.4</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>7.0</td>
    <td>+1.7</td>
    <td>+4.7</td>
    <td>+5.4</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>17.3</td>
    <td>+2.6</td>
    <td>+2.2</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>11.5</td>
    <td>+1.5</td>
    <td>+1.1</td>
    <td>+2.2</td>
  </tr>
</table>

Table 13: M2M BLEU for best performing M→D, D→M, M↔D by language. Performance gains/loss are relative to Best Baseline.
<table>
  <tr>
    <th>Language</th>
    <th>Best Baseline</th>
    <th>Best M→D</th>
    <th>Best D→M</th>
    <th>Best M←D</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latn (Indonesian)</td>
    <td>41.5</td>
    <td>-1.0</td>
    <td>+9.6</td>
    <td>+11.2</td>
  </tr>
  <tr>
    <td>jav_Latn (Javanese)</td>
    <td>37.2</td>
    <td>+4.9</td>
    <td>+10.2</td>
    <td>+12.1</td>
  </tr>
  <tr>
    <td>sun_Latn (Sundanese)</td>
    <td>12.2</td>
    <td>+5.4</td>
    <td>+1.1</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>sinc_Latn (Samarai)</td>
    <td>3.2</td>
    <td>+1.1</td>
    <td>+1.1</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>mti_Latn (Malay)</td>
    <td>4.8</td>
    <td>+0.1</td>
    <td>+0.4</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>ceb_Latn (Cebuano)</td>
    <td>14.0</td>
    <td>-0.6</td>
    <td>-0.0</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>zsm_Latn (Standard Malay)</td>
    <td>56.6</td>
    <td>+0.4</td>
    <td>-1.3</td>
    <td>-0.6</td>
  </tr>
  <tr>
    <td>tgl_Latn (Tagalog)</td>
    <td>26.6</td>
    <td>+2.7</td>
    <td>+2.8</td>
    <td>+8.8</td>
  </tr>
  <tr>
    <td>ilo_Latn (Ilocano)</td>
    <td>7.5</td>
    <td>+1.5</td>
    <td>+2.4</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>fij_Latn (Fijian)</td>
    <td>3.3</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>plk_Latn (Malagasy Malagasy)</td>
    <td>23.8</td>
    <td>+1.0</td>
    <td>+1.9</td>
    <td>+3.6</td>
  </tr>
  <tr>
    <td>pag_Latn (Pangasinan)</td>
    <td>8.2</td>
    <td>+3.5</td>
    <td>+4.2</td>
    <td>+5.7</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>12.2</td>
    <td>+1.0</td>
    <td>+2.4</td>
    <td>+3.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>8.0</td>
    <td>+1.9</td>
    <td>+3.6</td>
    <td>+3.8</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>38.2</td>
    <td>-0.8</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>32.4</td>
    <td>+0.0</td>
    <td>-1.0</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>aqc_Arab (Tif'izzi-Adenis Arabic)</td>
    <td>34.1</td>
    <td>-0.1</td>
    <td>-0.2</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>arb_Arab (Egyptian Arabic)</td>
    <td>35.0</td>
    <td>+0.5</td>
    <td>+0.4</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>ajj_Arab (South Levantine Arabic)</td>
    <td>37.4</td>
    <td>+0.0</td>
    <td>-4.0</td>
    <td>-4.7</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>33.2</td>
    <td>-0.2</td>
    <td>-1.2</td>
    <td>-1.8</td>
  </tr>
  <tr>
    <td>arc_Arab (Iraqi Arabic)</td>
    <td>37.8</td>
    <td>+0.7</td>
    <td>+0.9</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>23.5</td>
    <td>+0.4</td>
    <td>-1.8</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>azr_Arab (Egyptian Arabic)</td>
    <td>29.2</td>
    <td>+0.1</td>
    <td>-1.7</td>
    <td>-1.0</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>31.8</td>
    <td>-0.3</td>
    <td>-1.3</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>23.5</td>
    <td>+0.4</td>
    <td>-1.8</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latn (Italian)</td>
    <td>33.8</td>
    <td>-0.7</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latn (Spanish)</td>
    <td>31.0</td>
    <td>-0.7</td>
    <td>-6.6</td>
    <td>-3.6</td>
  </tr>
  <tr>
    <td>fra_Latn (French)</td>
    <td>24.1</td>
    <td>-3.3</td>
    <td>-9.8</td>
    <td>-9.7</td>
  </tr>
  <tr>
    <td>por_Latn (Portuguese)</td>
    <td>26.9</td>
    <td>-3.2</td>
    <td>-10.7</td>
    <td>-9.1</td>
  </tr>
  <tr>
    <td>ron_Latn (Romanian)</td>
    <td>24.0</td>
    <td>-3.8</td>
    <td>-10.2</td>
    <td>-11.7</td>
  </tr>
  <tr>
    <td>glg_Latn (Galician)</td>
    <td>25.2</td>
    <td>-0.2</td>
    <td>-5.7</td>
    <td>-4.5</td>
  </tr>
  <tr>
    <td>cst_Latn (Catalan)</td>
    <td>30.5</td>
    <td>-1.5</td>
    <td>-5.7</td>
    <td>-5.5</td>
  </tr>
  <tr>
    <td>oc1_Latn (Occitan)</td>
    <td>33.1</td>
    <td>+0.9</td>
    <td>-2.2</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>ast_Latn (Asturian)</td>
    <td>18.8</td>
    <td>+0.1</td>
    <td>+1.1</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>linn_Latn (Ladin)</td>
    <td>19.8</td>
    <td>+0.2</td>
    <td>+0.9</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>vec_Latn (Venetian)</td>
    <td>30.2</td>
    <td>+3.2</td>
    <td>-0.8</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>scd_Latn (Sicilian)</td>
    <td>17.7</td>
    <td>+0.7</td>
    <td>+0.7</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>std_Latn (Sardinian)</td>
    <td>16.6</td>
    <td>+6.7</td>
    <td>+5.9</td>
    <td>+9.8</td>
  </tr>
  <tr>
    <td>fur_Latn (Friulian)</td>
    <td>20.3</td>
    <td>+5.1</td>
    <td>+1.9</td>
    <td>+4.5</td>
  </tr>
  <tr>
    <td>lg_Latn (Ligurian)</td>
    <td>20.7</td>
    <td>+6.7</td>
    <td>+2.8</td>
    <td>+5.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>30.6</td>
    <td>+1.5</td>
    <td>-2.8</td>
    <td>-0.5</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>18.0</td>
    <td>+0.7</td>
    <td>+2.6</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latn (Turkish)</td>
    <td>31.7</td>
    <td>-1.7</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzb_Latn (Northern Uzbek)</td>
    <td>4.8</td>
    <td>+2.3</td>
    <td>+1.1</td>
    <td>+1.2</td>
  </tr>
  <tr>
    <td>tuk_Latn (Turkmen)</td>
    <td>8.4</td>
    <td>+1.5</td>
    <td>+0.3</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>azg_Latn (North Azerbaijani)</td>
    <td>10.5</td>
    <td>+1.6</td>
    <td>-1.0</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>ctf_Latn (Crimean Tatar)</td>
    <td>13.3</td>
    <td>+1.9</td>
    <td>-1.4</td>
    <td>+1.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>9.3</td>
    <td>+1.8</td>
    <td>-0.4</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>9.3</td>
    <td>+1.8</td>
    <td>-0.4</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>34.8</td>
    <td>-1.7</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>tne_Deva (Telugu)</td>
    <td>23.3</td>
    <td>+2.2</td>
    <td>+6.0</td>
    <td>+6.3</td>
  </tr>
  <tr>
    <td>bhs_Deva (Bhojpuri)</td>
    <td>16.2</td>
    <td>+2.4</td>
    <td>+3.2</td>
    <td>+4.2</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>24.9</td>
    <td>+2.5</td>
    <td>+5.3</td>
    <td>+5.8</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>17.2</td>
    <td>+4.2</td>
    <td>+7.6</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>20.4</td>
    <td>+2.8</td>
    <td>+5.6</td>
    <td>+6.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.4</td>
    <td>+2.8</td>
    <td>+5.6</td>
    <td>+6.0</td>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat_Haitian</td>
    <td>29.7</td>
    <td>+1.6</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>gcf_Guadeloupean</td>
    <td>7.0</td>
    <td>+3.8</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>marf159 (Martinican)</td>
    <td>5.8</td>
    <td>+2.7</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>afc (Saint Lucia Patois)</td>
    <td>5.6</td>
    <td>+0.5</td>
    <td>-2.6</td>
    <td>+3.9</td>
  </tr>
  <tr>
    <td>ger (French Guianese Creole)</td>
    <td>4.7</td>
    <td>+3.1</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>lou (Louisiana Creole)</td>
    <td>12.1</td>
    <td>-0.1</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>mfu_Mauritian</td>
    <td>7.6</td>
    <td>+0.6</td>
    <td>-5.3</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>rcf (Réunion Creole)</td>
    <td>20.9</td>
    <td>-2.7</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>cis (Seychellois)</td>
    <td>15.0</td>
    <td>+0.2</td>
    <td>-8.7</td>
    <td>+8.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>10.0</td>
    <td>+0.9</td>
    <td>-5.8</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>10.0</td>
    <td>+0.9</td>
    <td>-5.8</td>
    <td>+3.8</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>20.7</td>
    <td>+1.2</td>
    <td>+0.8</td>
    <td>+2.4</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>12.5</td>
    <td>+2.4</td>
    <td>+2.3</td>
    <td>+2.8</td>
  </tr>
</table>

Table 14: Aya-23 BLEU for best performing M→D, D→M, M←D by language. Performance gains/loss are relative to Best Baseline.
<table>
  <tr>
    <th>Language</th>
    <th>Best Baseline</th>
    <th>Best M→D</th>
    <th>Best D→M</th>
    <th>Best M→D</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latn (Indonesian)</td>
    <td>87.3 +0.4</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Ipi, Latn (Ibanese)</td>
    <td>72.9 +1.2 -1.5 -0.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>sun_Latn (Sundanese)</td>
    <td>77.3 -1.3 -3.4 -2.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>swe, Latn (Swedish)</td>
    <td>77.6 +4.5 +1.5 +2.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>jv_nested (javanese Malay)</td>
    <td>64.7 +4.4 -10.3 -2.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>cbe_Latn (Cebuano)</td>
    <td>66.5 +6.0 -8.7 -1.0</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>tgl_Latn (Tagalog)</td>
    <td>73.8 +4.0 -18.6 -6.6</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>loc_Latn (Bisuano)</td>
    <td>70.3 +2.0 +2.2 +1.8</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>plh_Latn (Platner Malagasy)</td>
    <td>67.8 +5.0 -5.5 +4.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>lmt_Latn (Maite Toba)</td>
    <td>69.5 -2.0 =-2.5 -=0.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>71.6 +3.6 +5.0 +4.1</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
  </tr>
  <tr>
    <th>Baseline + 25 Average</th>
    <th>47.7 +3.3 +4.3 +4.6</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <th></th><th>aar Arab (Modern Standard Arabic)</th><th></th><th></th><th></th>
  </tr>
  <tr>
    <td>าร์, Arab (McGilvray’i Amiri Arabic)</td>
    <td>89.6 +0.0</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>aec, Arab (Zifizrd ADMilate Arabic)</td>
    <td>77.6 -0.3 +0.0 +0.0</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>aqd, Arab (Tissani Arabic)</td>
    <td>68.8 +1.5 +4.8 +3.8</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>aqd-Arab (South Levantine Arabic)</td>
    <td>72.8 +1.2 +0.7 +0.6</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>af, Arab (Tahirit Arabic)</td>
    <td>60.1 +3.9 4.7 3.9</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ary, Arab (Moroccan Arabic)</td>
    <td>68.8 +0.3 +4.2 +3.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>zhq-Arab (Egyptian Arabic)</td>
    <td>74.1 -5.7 +0.6 +3.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>72.6 +1.1 +0.5 +1.4</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>Baseline + 25 Average</th>
    <th>79.5 +7.1 +1.4 +1.8</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <th></th><th>ita, Latn (Italian)</th><th></th><th></th><th></th>
  </tr>
  <tr>
    <td>spi, Latn (Spanish)</td>
    <td>85.2 -0.2 -8.1 -4.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>fra, Latn (French)</td>
    <td>87.8 -0.2 -0.5 -0.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>por, Latn (Portuguese)</td>
    <td>88.1 -0.1 -10.2 -4.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ron, Latn (Romanian)</td>
    <td>86.9 +3.7 -19.9 -8.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>cat, Latn (Catalan)</td>
    <td>82.7 +0.7 1.4 1.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>nl_Latn (Flemish)</td>
    <td>80.1 -0.3 -0.5 -0.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>sll_Latn (Albanian)</td>
    <td>70.1 +4.3 -7.3 -1.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>litl_Latn (Lithuanian)</td>
    <td>74.1 +4.3 4.7 5.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>scn, Latn (Sicilian)</td>
    <td>64.0 +17.0 48.6 +14.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>cs, Latn (Czech)</td>
    <td>64.1 +5.1 +1.5 +2.0</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>csu_Iowiaiz (Iowian Czech)</td>
    <td>70.9 +4.1 -6.4 -4.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>fur, Latn (Friulian)</td>
    <td>61.4 +14.9 +8.4 +20.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>teaz_Latn (Jonian)</td>
    <td>66.4 +7.8 -5.4 +12.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>67.9 +8.3 -1.2 +5.9</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>Baseline + 25 Average</th>
    <th>82.4 +10.9 +4.9 +5.9</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <th></th><th>tur, Latn (Turkish)</th><th></th><th></th><th></th>
  </tr>
  <tr>
    <td>urz_Latn (Northern Uyghur)</td>
    <td>88.4 -0.3 -18.5 -11.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>uaz_Latn (North Azeri)</td>
    <td>66.8 -9.4 +1.1 +14.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>lcr_Latn (Crimean Tatar)</td>
    <td>87.7 +7.1 +5.2 +4.3</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>64.8 ±4.8 ±7.8 ±9.2</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>Baseline + 25 Average</th>
    <th>63.6 +6.2 +8.6 +3.2</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th colspan="5">Indo-Liradaan Language Family</th>
  </tr>
  <tr>
    <th></th><th>hin_Deva (Hindi)</th><th></th><th></th><th></th>
  </tr>
  <tr>
    <td>kbr_Gujr (Gujarati)</td>
    <td>65.2 -4) ± 1±.1 9±2 ++</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>bdv_Deva (Bangla)</td>
    <td>70.7 ±.4+ 1±2 +0 ±.9</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>hmz, Deva (Marathi)</td>
    <td>79.6 6±.9 +8±9 -±1.1 -1±1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>st, Deva (Maldivhi)</td>
    <td>61.1 ±2 2.1 +1.5 2.1 2±2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>74.3 ±6.9 ±7.8 ±1.0 ±6.2 ±2.4</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>French-Creole Language Family</th><th></th><th></th><th></th><th></th>
  </tr>
  <tr>
    <th></th><th>fr (French)</th><th></th><th></th><th></th>
  </tr>
  <tr>
    <td>gf (Girlinguegal)</td>
    <td>72.4 -2.1 +1.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>tisa259 (Matinand 공간가)</td>
    <td>70.7 +1.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>uft (Saint Lucia Patois)</td>
    <td>74.7 -5.9</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>geo (French Caribbean Creole)</td>
    <td>69.9 4.7 4.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>hct (Louisiana Creole)</td>
    <td>43.0 +6.0</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ntd (Martinique)</td>
    <td>40.3 +3.9</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>tct (Creole usage)</td>
    <td>53.1 -3.8</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>Average</th>
    <th>47.6 ±8.5 +5.0 +5.4 +12.8</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th>Baseline + 25 Average</th>
    <th>47.6 +3.1 ±8.8 ±4.9 +5.4</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <th colspan="5">General</th>
  </tr>
  <tr>
    <th></th><th>General</th><th>Best</th><th>W→D</th><th>D→M</th>
  </tr>
  <tr>
    <td></td>
    <td>16.2 5 5 ±4.9 ±2.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <th>General Baseline + 25 Average</th>
    <th>53.1 +3.4 4.9 5.6</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <table>
    <tr>
      <th>Language</th>
      <th>Best Baseline</th>
      <th>Best M→D</th>
      <th>Best D→M</th>
      <th>Best M→D</th>
    </tr>
    <tr>
      <th colspan="5">Austronesian Language Family</th>
    </tr>
    <tr>
      <td>ind_Latn (Indonesian)</td>
      <td>88.5 -0.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>pra, Latn (Javanese)</td>
      <td>75.0 -6.2 =1.3 +6.5</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>su, Latn (Sundanese)</td>
      <td>63.7 -9.0 +1.7 +8-9</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>sims, Latn (Singosari)</td>
      <td>76.3 -4.6 +5.9 +14.4</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>tnz, Latn (Tanji)</td>
      <td>40.2 -2.7 +3.8 +9.9</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>cbl, Latn (Chamfas)</td>
      <td>61.1 -0.3 -2.3 -2.8</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>cslb22 (Cimarae Malay)</td>
      <td>67.5 -5.0 +2.0 +3.2</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>tbg, Latn (Tagalog)</td>
      <td>76.4 ±5.4 7.3 -9.4</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>ios, Latn (Bisuano)</td>
      <td>82.2 ±7.4 2.1 +8.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>plh, Latn (Platner Malagasy)</td>
      <td>64.1 +3.7 +5.5 +4.8</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>pin, Latn (Yayson Igab)</td>
      <td>63.3 +15.8 +4.3 +4.3</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>55.2 -4.1 ±0.5 ±5.7</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>Baseline + 25 Average</th>
      <th>49.5 ±5.5 ±1.9 ±4.8</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th colspan="5">Arabic Language Family</th>
    </tr>
    <tr>
      <td>achts_Arab (Modern Standard Arabic)</td>
      <td>86.5 -0.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>aces_Arab (Rzed-Menut Arhae)</td>
      <td>83.6 -0.7 -0.6 +23.5</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>acds_Arab (Shimz-Idtron Arabic)</td>
      <td>84.7 -0.4 -0.5 +1.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>aqrr_AB (Tomar Arabic)</td>
      <td>78.8 -0.4 -6.1 +4.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>ayeq_Arab (Mooonime Arabic)</td>
      <td>85.4 +5.0 -6.1 +3.0 -0.6</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>apnb_Arab (North Levantine Arabic)</td>
      <td>82.7 -0.8 -1.0 -0.8</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>shlr_Arab (Harsian Arabic)</td>
      <td>86.2 +2.1 -3.7 -8.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>aryd, Arab MorroBanum and Arabic</td>
      <td>72.6 +2.3 -4.1 +1.5 -3.8</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>82.0 ±4.0 ±1.0 ±4.1</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>Baseline + 25 Average</th>
      <th>76.6 -3.1 ±1.0 ±4.1</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th colspan="5">Romance Language Family</th>
    </tr>
    <tr>
      <td>ita, Latn (Italian)</td>
      <td>87.2 -0.2 -</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>ani, Latn (Spanish)</td>
      <td>86.1 ±-5.2 ±-8.0 -</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>nfa, Latn (Portuguese)</td>
      <td>87.9 ±-4.5 ±-5.8 ±-2.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>rlg, Latn (Romanian)</td>
      <td>85.9 -14.3 -4.5 ±22.6</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>clt, Latn (Catalan)</td>
      <td>82.3 ±-4.7 ±-4.3 -±1.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>rlg, Latn (Ladinian)</td>
      <td>79.4 ±-2.3 ±-2.8 ±-4.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>_lus (Luhanian)</td>
      <td>84.5 -6.0 -8.7 -±1.4</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>len, Latn (Lombardian)</td>
      <td>74.4 -2.7 -1.8 -±8.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>scl, Latn (Sicilian)</td>
      <td>62.5 ±-5.3 ±-1.5 ±-3.4</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>75.5 ±3.7 ±2.4 ±4.2</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>Baseline + 25 Average</th>
      <th>67.4 ±8.0 ±8.7 ±0.0</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th colspan="5">Turkic Language Family</th>
    </tr>
    <tr>
      <td>tur, Latn (Turkish)</td>
      <td>87.3 -0.1 -</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>urz_Latn (Northern Uychur)</td>
      <td>58.0 -22.1 +6.7 22.0</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>uz77_nfall652 (North Abuzerj)</td>
      <td>89.9 ±4.3 ±4.9 ±5.6</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>uc742 (Crimean Tatar)</td>
      <td>81.3 ±2.5 ±-1.9 ±2.3</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>85.8 ±4.1 ±2.9 ±8.1 ±12.3</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>Baseline + 25 Average</th>
      <th>65.0 ±6.8 ±0.0 ±4.3 ±2.5</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th colspan="5">Indo-Liradaan Language Family</th>
    </tr>
    <tr>
      <td>hin_Deva (Hindi)</td>
      <td>88.3 -0.1 -</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>kbr_Gujr (Gurringgil)</td>
      <td>82.7 ±2.4 ±1.2 ±5.7 ±4.8</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>pjsi_Gujr (Gujr Hirza)</td>
      <td>75.2 ±1.2 ±4.0 ±3.0 ±4.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>mad_Guajr (mguir)</td>
      <td>78.9 ±2.9 ±0.0 ±4.7 ±4.9</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>mandi_Milli2 (Millikil)</td>
      <td>78.4 ±2.9 ±1.6 ±2.4 ±6.5</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>76.5 ±3.0 ±3.3 ±4.1</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>French-Creole Language Family</th><th></th><th></th><th></th><th></th>
    </tr>
    <tr>
      <td>fht (French)</td>
      <td>77.4 ±2.5 ±3.2 ±7.4 ±0.0 ±4.0 ±8.1</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>gec_Gillespiralin</td>
      <td>84.8 ±2 1.8 ±-1.8 ±-9.0 ±-2.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>mar259 (Maritianian)</td>
      <td>46.1 ±3.3</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>acf_SaintLucian_Patoris</td>
      <td>54.1 ±4.9</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>gtoFritsw_CreebleGraphCreollc</td>
      <td>67.4 ±2.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>spji_Brama_Gib...</td>
      <td>55.8 ±0.0 ±3.7 ±-4.9 ±-2.7</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <td>mfi_MauiHonu</td>
      <td>54.6 ±14</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>Average</th>
      <th>53.6 ±4.7</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    <tr>
      <th>General</th><th></th><th></th><th></th><th></th>
    </tr>
    <tr>
      <td>General</td>
      <td>16.2 ±5.0 ±4.9 ±5.6</td>
      <td></td>
      <td></td>
      <td></td>
    </tr>
    <tr>
      <th>General Baseline + 25 Average</th>
      <th>53.1 ±4.2 ±4.8 ±9.0</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
    </table>
Table 15: m2mLL’s COMET for best performing M→D, D→M, Mℵ→D by language. Performance gains/loss are relative to Best Baseline.

Table 16: aya-23-3b’s COMET for best performing M→D, D→M, Mℵ→D by language. Performance gains/loss are relative to Best Baseline.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>theLo</th>
    <th>forwarding</th>
    <th>forwarding+clued</th>
    <th>-shall</th>
    <th>-clued</th>
  </tr>
  <tr>
    <th colspan="7">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>58.9</td>
    <td>-0.1</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>+0.5</td>
    <td>-0.7</td>
  </tr>
  <tr>
    <td>jat_Latin (Javanese)</td>
    <td>22.9</td>
    <td>+0.4</td>
    <td>+0.0</td>
    <td>-0.1</td>
    <td>+1.1</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>22.9</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+1.4</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>ans_Latin (Ansarese)</td>
    <td>2.3</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.4</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>msi_Latin (Mentawai)</td>
    <td>22.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.6</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>zam_Latin (Standard Malay)</td>
    <td>38.7</td>
    <td>+0.3</td>
    <td>-0.3</td>
    <td>-0.2</td>
    <td>+0.3</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>27.8</td>
    <td>+0.5</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+1.2</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>ik_Latin (Iban)</td>
    <td>13.4</td>
    <td>+0.8</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+2.4</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>fij_Latin (Fijian)</td>
    <td>2.3</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.6</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>plg_Latin (Malagasy)</td>
    <td>2.5</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>6.4</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+1.3</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>14.9</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+1.7</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>10.8</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+1.5</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <th colspan="7">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>29.6</td>
    <td>+0.3</td>
    <td>-0.1</td>
    <td>-0.2</td>
    <td>+0.2</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>22.4</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.8</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>aqj_Arab (Tirizi-Adeni Arabic)</td>
    <td>24.7</td>
    <td>+0.5</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+0.5</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>arh_Arab (Tunisian Arabic)</td>
    <td>17.7</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.8</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>24.5</td>
    <td>+0.1</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+0.6</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>arq_Arab (Iraqi Arabic)</td>
    <td>27.0</td>
    <td>+0.5</td>
    <td>+0.4</td>
    <td>+0.5</td>
    <td>+1.0</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>11.9</td>
    <td>+1.0</td>
    <td>+1.0</td>
    <td>+1.1</td>
    <td>+3.7</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>18.2</td>
    <td>+0.1</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+1.1</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>21.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+1.0</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+1.0</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <th colspan="7">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>30.8</td>
    <td>+0.9</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.9</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>26.8</td>
    <td>+0.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>+0.5</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>41.8</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+1.0</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>45.8</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.7</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>40.8</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>glb_Latin (Galician)</td>
    <td>27.5</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>42.7</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+0.0</td>
    <td>+0.2</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>oc_Latin (Occitan)</td>
    <td>44.6</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.1</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>lan_Latin (Ladin)</td>
    <td>34.3</td>
    <td>+0.4</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+1.1</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>lmo_Latin (Lombard)</td>
    <td>9.6</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+1.1</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <td>vev_Latin (Venetian)</td>
    <td>17.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.7</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>sca_Latin (Sicilian)</td>
    <td>7.3</td>
    <td>+1.2</td>
    <td>+1.2</td>
    <td>+1.2</td>
    <td>+1.8</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>sed_Latin (Sardinian)</td>
    <td>5.3</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+1.0</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>lan_Latin (Ligurian)</td>
    <td>8.6</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+1.0</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>lj_Latin (Ligurian)</td>
    <td>9.8</td>
    <td>+1.1</td>
    <td>+0.8</td>
    <td>+0.7</td>
    <td>+1.2</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>26.6</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>9.6</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+1.1</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <th colspan="7">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>33.3</td>
    <td>-0.4</td>
    <td>-0.5</td>
    <td>-0.4</td>
    <td>-0.6</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>uaz_Latin (Northern Uzbek)</td>
    <td>1.6</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.3</td>
    <td>+0.7</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>5.7</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.4</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>ash_Latin (Ashgabati)</td>
    <td>5.5</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.4</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>ctb_Latin (Crimean Tatar)</td>
    <td>8.5</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>5.0</td>
    <td>-0.1</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>5.0</td>
    <td>-0.1</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.5</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <th colspan="7">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>33.4</td>
    <td>+0.5</td>
    <td>-0.6</td>
    <td>-0.6</td>
    <td>+0.4</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>bne_Deva (Chhattisgarhi)</td>
    <td>15.2</td>
    <td>+0.2</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+4.8</td>
    <td>+5.3</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>10.1</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+3.3</td>
    <td>+3.6</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>17.2</td>
    <td>+0.7</td>
    <td>+0.7</td>
    <td>+0.7</td>
    <td>+2.0</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>mad_Deva (Marathi)</td>
    <td>7.7</td>
    <td>+0.4</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+6.9</td>
    <td>+7.5</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>12.6</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+5.0</td>
    <td>+5.4</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>12.6</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+5.0</td>
    <td>+5.4</td>
  </tr>
  <tr>
    <th colspan="7">Creole Language Family</th>
  </tr>
  <tr>
    <td>ht_Latin (Haitian)</td>
    <td>59.7</td>
    <td>+0.9</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+0.5</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>gof (Guadeloupean)</td>
    <td>4.4</td>
    <td>+1.2</td>
    <td>+1.2</td>
    <td>+1.2</td>
    <td>+2.8</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>mart1259 (Martinican)</td>
    <td>0.0</td>
    <td>+4.2</td>
    <td>+4.2</td>
    <td>+4.2</td>
    <td>+3.6</td>
    <td>+3.7</td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>3.7</td>
    <td>+1.0</td>
    <td>+1.0</td>
    <td>+1.0</td>
    <td>+2.0</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>gep (Benedictine Creole)</td>
    <td>2.5</td>
    <td>+0.8</td>
    <td>+0.8</td>
    <td>+0.8</td>
    <td>+2.5</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>lou (Louisiana Creole)</td>
    <td>5.3</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+1.3</td>
    <td>+1.4</td>
  </tr>
  <tr>
    <td>mao (Mauritian Creole)</td>
    <td>5.1</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+1.1</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>re (Réunion Creole)</td>
    <td>14.8</td>
    <td>+1.5</td>
    <td>+1.4</td>
    <td>+1.4</td>
    <td>+3.6</td>
    <td>+3.6</td>
  </tr>
  <tr>
    <td>cs (Seychellois)</td>
    <td>11.2</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+2.6</td>
    <td>+2.6</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>5.9</td>
    <td>+0.9</td>
    <td>+0.9</td>
    <td>+0.9</td>
    <td>+2.5</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>16.8</td>
    <td>+0.4</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+3.8</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>10.9</td>
    <td>+0.4</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+3.8</td>
    <td>+2.5</td>
  </tr>
</table>

Table 17: M→D BLEU scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The overall best score for each language is bolded and the best score in each paradigm is underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>the1st</th>
    <th>strand1ng</th>
    <th>strand1ng+clnd</th>
    <th>-shall</th>
    <th>-clnd</th>
  </tr>
  <tr>
    <th colspan="7">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latn (Indonesian)</td>
    <td>41.5</td>
    <td>-0.4</td>
    <td>-0.3</td>
    <td>-0.4</td>
    <td>+1.0</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>14.2</td>
    <td>+2.0</td>
    <td>+0.4</td>
    <td>+0.8</td>
    <td>+5.9</td>
    <td>+5.5</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>11.0</td>
    <td>+0.8</td>
    <td>+1.2</td>
    <td>+0.9</td>
    <td>+2.6</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>msa_Latin (Malay)</td>
    <td>3.2</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+0.5</td>
    <td>+1.1</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>msi_Latin (Mentawai)</td>
    <td>2.8</td>
    <td>-1.5</td>
    <td>-1.2</td>
    <td>-1.4</td>
    <td>+0.0</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>cel_Latin (Cebuano)</td>
    <td>10.0</td>
    <td>+2.2</td>
    <td>+2.1</td>
    <td>+2.2</td>
    <td>+2.2</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>zsm_Latin (Standard Malay)</td>
    <td>58.5</td>
    <td>+0.1</td>
    <td>-0.1</td>
    <td>+0.1</td>
    <td>+0.5</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>25.6</td>
    <td>+2.5</td>
    <td>+5.6</td>
    <td>+6.2</td>
    <td>+7.4</td>
    <td>+7.4</td>
  </tr>
  <tr>
    <td>isk_Latin (Iban)</td>
    <td>7.8</td>
    <td>+1.9</td>
    <td>+1.6</td>
    <td>+1.8</td>
    <td>+1.3</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>fij_Latin (Fijian)</td>
    <td>7.7</td>
    <td>-0.2</td>
    <td>-0.1</td>
    <td>-0.4</td>
    <td>+0.6</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>plg_Latin (Malagasy)</td>
    <td>2.5</td>
    <td>+1.0</td>
    <td>+1.0</td>
    <td>+0.9</td>
    <td>+0.7</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>7.7</td>
    <td>+0.7</td>
    <td>+0.9</td>
    <td>+0.8</td>
    <td>+4.4</td>
    <td>+4.4</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>11.9</th>
    <th>-1.0</th>
    <th>-0.9</th>
    <th>-1.0</th>
    <th>+1.1</th>
    <th>+1.2</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>7.7</th>
    <th>-0.6</th>
    <th>-0.4</th>
    <th>-0.5</th>
    <th>+2.1</th>
    <th>+2.2</th>
  </tr>
  <tr>
    <th colspan="7">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>58.1</td>
    <td>+0.0</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.7</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>31.6</td>
    <td>+0.5</td>
    <td>+0.3</td>
    <td>+0.8</td>
    <td>+0.5</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>aqj_Arab (Tirizi-Adeni Arabic)</td>
    <td>33.7</td>
    <td>+0.2</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>arh_Arab (Tunisian Arabic)</td>
    <td>20.6</td>
    <td>-0.6</td>
    <td>-0.5</td>
    <td>-1.0</td>
    <td>+0.2</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>arj_Arab (Jordanian Arabic)</td>
    <td>37.4</td>
    <td>+2.2</td>
    <td>+1.7</td>
    <td>+2.1</td>
    <td>+2.2</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>33.2</td>
    <td>-1.5</td>
    <td>-1.0</td>
    <td>-1.3</td>
    <td>-0.3</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>arv_Arab (Iraqi Arabic)</td>
    <td>27.7</td>
    <td>+0.0</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.6</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>21.5</td>
    <td>-1.8</td>
    <td>-1.7</td>
    <td>-1.8</td>
    <td>+0.4</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>28.9</td>
    <td>+0.3</td>
    <td>+0.1</td>
    <td>+0.3</td>
    <td>+0.0</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>31.5</th>
    <th>-0.7</th>
    <th>-0.5</th>
    <th>-0.6</th>
    <th>+0.2</th>
    <th>+0.2</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>21.2</th>
    <th>-1.8</th>
    <th>-2.3</th>
    <th>-1.9</th>
    <th>+0.4</th>
    <th>+0.4</th>
  </tr>
  <tr>
    <th colspan="7">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>32.6</td>
    <td>+1.2</td>
    <td>+1.0</td>
    <td>+0.7</td>
    <td>+0.5</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>29.0</td>
    <td>+1.7</td>
    <td>+1.9</td>
    <td>+2.0</td>
    <td>+1.1</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>44.0</td>
    <td>+2.1</td>
    <td>+2.1</td>
    <td>+2.1</td>
    <td>+2.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>46.9</td>
    <td>-1.9</td>
    <td>-1.7</td>
    <td>-1.6</td>
    <td>-3.7</td>
    <td>-3.7</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>24.0</td>
    <td>-2.0</td>
    <td>-1.9</td>
    <td>-1.8</td>
    <td>-4.8</td>
    <td>-4.8</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>18.5</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+0.6</td>
    <td>+0.5</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>52.6</td>
    <td>-2.6</td>
    <td>-2.3</td>
    <td>-2.3</td>
    <td>-2.1</td>
    <td>-2.1</td>
  </tr>
  <tr>
    <td>occ_Latin (Occitan)</td>
    <td>31.1</td>
    <td>-2.6</td>
    <td>-2.6</td>
    <td>-2.1</td>
    <td>-0.5</td>
    <td>-0.5</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>31.8</td>
    <td>-1.5</td>
    <td>-1.5</td>
    <td>-1.4</td>
    <td>-2.0</td>
    <td>-2.0</td>
  </tr>
  <tr>
    <td>lmo_Latin (Lombard)</td>
    <td>17.4</td>
    <td>+1.4</td>
    <td>+2.4</td>
    <td>+1.7</td>
    <td>+7.6</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <td>teo_Latin (Toscan)</td>
    <td>26.9</td>
    <td>+2.7</td>
    <td>+2.7</td>
    <td>+2.9</td>
    <td>+4.6</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <td>sca_Latin (Sicilian)</td>
    <td>16.7</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+0.3</td>
    <td>+7.4</td>
    <td>+7.4</td>
  </tr>
  <tr>
    <td>sed_Latin (Sardinian)</td>
    <td>16.4</td>
    <td>-0.5</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>+6.9</td>
    <td>+6.8</td>
  </tr>
  <tr>
    <td>lat_Latin (Latin)</td>
    <td>17.7</td>
    <td>+2.4</td>
    <td>+2.4</td>
    <td>+2.6</td>
    <td>+2.7</td>
    <td>+2.7</td>
  </tr>
  <tr>
    <td>lij_Latin (Ligurian)</td>
    <td>17.4</td>
    <td>+2.9</td>
    <td>+3.2</td>
    <td>+3.2</td>
    <td>+10.0</td>
    <td>+9.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>29.5</th>
    <th>-0.1</th>
    <th>+0.2</th>
    <th>+0.0</th>
    <th>+2.4</th>
    <th>+2.5</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>17.1</th>
    <th>+1.4</th>
    <th>+1.5</th>
    <th>+1.5</th>
    <th>+2.9</th>
    <th>+2.9</th>
  </tr>
  <tr>
    <th colspan="7">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>31.7</td>
    <td>-0.8</td>
    <td>-1.4</td>
    <td>-1.4</td>
    <td>-2.5</td>
    <td>-1.7</td>
  </tr>
  <tr>
    <td>uaz_Latin (Northern Uzbek)</td>
    <td>3.8</td>
    <td>+0.6</td>
    <td>+0.8</td>
    <td>+1.0</td>
    <td>+3.2</td>
    <td>+3.3</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>6.1</td>
    <td>+2.1</td>
    <td>+2.1</td>
    <td>+2.0</td>
    <td>+3.5</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>ash_Latin (Ashgabati)</td>
    <td>9.8</td>
    <td>+2.2</td>
    <td>+2.2</td>
    <td>+2.0</td>
    <td>+2.0</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>ctb_Latin (Crimean Tatar)</td>
    <td>11.9</td>
    <td>+1.5</td>
    <td>+1.0</td>
    <td>+1.4</td>
    <td>+2.9</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>7.9</th>
    <th>+1.2</th>
    <th>+1.1</th>
    <th>+1.5</th>
    <th>+2.9</th>
    <th>+2.9</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>7.9</th>
    <th>+1.2</th>
    <th>+1.1</th>
    <th>+1.5</th>
    <th>+2.9</th>
    <th>+2.9</th>
  </tr>
  <tr>
    <th colspan="7">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>34.8</td>
    <td>-1.5</td>
    <td>-1.5</td>
    <td>-1.5</td>
    <td>-2.5</td>
    <td>-1.7</td>
  </tr>
  <tr>
    <td>bne_Deva (Chhattisgarhi)</td>
    <td>23.3</td>
    <td>-2.2</td>
    <td>-2.2</td>
    <td>-2.9</td>
    <td>+2.2</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>18.2</td>
    <td>-1.2</td>
    <td>-1.3</td>
    <td>-1.2</td>
    <td>+2.4</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>17.0</td>
    <td>-1.7</td>
    <td>-1.9</td>
    <td>-1.7</td>
    <td>+2.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>mad_Deva (Marathi)</td>
    <td>17.2</td>
    <td>-2.2</td>
    <td>-1.9</td>
    <td>-1.9</td>
    <td>+2.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>20.4</th>
    <th>-1.8</th>
    <th>-1.8</th>
    <th>-1.9</th>
    <th>+2.8</th>
    <th>+2.7</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>20.4</th>
    <th>-1.8</th>
    <th>-1.8</th>
    <th>-1.9</th>
    <th>+2.8</th>
    <th>+2.7</th>
  </tr>
  <tr>
    <th colspan="7">Creole Language Family</th>
  </tr>
  <tr>
    <td>htl_Haitian</td>
    <td>8.9</td>
    <td>+20.5</td>
    <td>+20.8</td>
    <td>+20.6</td>
    <td>+18.7</td>
    <td>+18.2</td>
  </tr>
  <tr>
    <td>gof (Guadeloupean)</td>
    <td>6.4</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.6</td>
    <td>+3.0</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>mart1259 (Martinican)</td>
    <td>4.7</td>
    <td>+0.9</td>
    <td>+1.0</td>
    <td>+1.1</td>
    <td>+2.8</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>acl (Saint Lucian Patois)</td>
    <td>2.6</td>
    <td>+2.9</td>
    <td>+2.9</td>
    <td>+2.0</td>
    <td>+3.5</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>gph (French Guianan Creole)</td>
    <td>4.2</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.7</td>
    <td>+2.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>lou (Louisiana Creole)</td>
    <td>9.6</td>
    <td>+1.8</td>
    <td>+1.6</td>
    <td>+2.5</td>
    <td>+2.4</td>
    <td>+2.4</td>
  </tr>
  <tr>
    <td>mlt_Maltese</td>
    <td>7.4</td>
    <td>+0.7</td>
    <td>+0.6</td>
    <td>+0.7</td>
    <td>+2.7</td>
    <td>+2.7</td>
  </tr>
  <tr>
    <td>rcf (Réunion Creole)</td>
    <td>20.9</td>
    <td>-0.5</td>
    <td>-3.0</td>
    <td>-3.0</td>
    <td>-2.7</td>
    <td>-2.7</td>
  </tr>
  <tr>
    <td>csy (Seychellois)</td>
    <td>5.7</td>
    <td>+0.8</td>
    <td>+0.3</td>
    <td>+0.8</td>
    <td>+2.9</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>7.3</th>
    <th>+2.1</th>
    <th>+1.6</th>
    <th>+1.6</th>
    <th>+3.0</th>
    <th>+3.4</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>7.3</th>
    <th>+2.1</th>
    <th>+1.6</th>
    <th>+1.6</th>
    <th>+3.0</th>
    <th>+3.4</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>19.7</th>
    <th>+1.5</th>
    <th>+1.0</th>
    <th>+1.9</th>
    <th>+2.9</th>
    <th>+2.9</th>
  </tr>
</table>

Table 18: M→D BLEU scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The overall best score for each language is bolded and the best score in each paradigm is underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>fbtri</th>
    <th>randaug-shell</th>
    <th>randaug-cloud</th>
    <th>-shell</th>
    <th>-cloud</th>
  </tr>
  <tr>
    <th colspan="7">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>97.4</td>
    <td>-0.1</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.1</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>70.1</td>
    <td>+0.9</td>
    <td>-0.0</td>
    <td>-0.1</td>
    <td>+2.1</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>71.6</td>
    <td>+0.7</td>
    <td>-0.2</td>
    <td>-0.3</td>
    <td>+2.0</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>msd_Latin (Malay)</td>
    <td>77.4</td>
    <td>+0.8</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+3.1</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>mr_Latin (Marathi)</td>
    <td>25.7</td>
    <td>-0.6</td>
    <td>-0.6</td>
    <td>-0.6</td>
    <td>-0.5</td>
    <td>+3.2</td>
  </tr>
  <tr>
    <td>cch_Latin (Cebuano)</td>
    <td>64.2</td>
    <td>+0.5</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>+4.8</td>
    <td>+4.1</td>
  </tr>
  <tr>
    <td>zwe_Latin (Swahili Malawi)</td>
    <td>86.5</td>
    <td>+0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>+0.0</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>73.2</td>
    <td>+0.6</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>+1.6</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>itc_Latin (Ilocano)</td>
    <td>56.1</td>
    <td>+1.0</td>
    <td>+0.2</td>
    <td>-0.1</td>
    <td>+3.0</td>
    <td>+2.6</td>
  </tr>
  <tr>
    <td>bl_Latin (Balinese)</td>
    <td>27.7</td>
    <td>+0.3</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>+0.9</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>plh_Latin (Plautdietsch Malagasy)</td>
    <td>44.3</td>
    <td>+2.8</td>
    <td>+1.6</td>
    <td>+1.3</td>
    <td>+5.8</td>
    <td>+3.1</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>55.5</td>
    <td>+4.8</td>
    <td>+1.8</td>
    <td>+0.9</td>
    <td>+4.0</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>72.6</th>
    <th>+0.7</th>
    <th>-0.1</th>
    <th>-0.1</th>
    <th>+4.5</th>
    <th>+2.9</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>46.9</th>
    <th>+0.7</th>
    <th>+0.2</th>
    <th>-0.1</th>
    <th>+5.1</th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th colspan="7">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>80.6</td>
    <td>-0.2</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>+0.0</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>76.0</td>
    <td>-0.2</td>
    <td>-0.4</td>
    <td>-0.3</td>
    <td>+0.7</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>acw_Arab (Ta'izzi-Aden Arabic)</td>
    <td>77.6</td>
    <td>-0.2</td>
    <td>-0.5</td>
    <td>-0.5</td>
    <td>+0.3</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>arab_Arab (Tunisian Arabic)</td>
    <td>68.8</td>
    <td>-0.2</td>
    <td>-0.5</td>
    <td>-0.5</td>
    <td>+1.7</td>
    <td>+1.2</td>
  </tr>
  <tr>
    <td>ajp_Arab (Jordanian Arabic)</td>
    <td>73.2</td>
    <td>-0.5</td>
    <td>-0.5</td>
    <td>-0.5</td>
    <td>+1.4</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>72.8</td>
    <td>-0.2</td>
    <td>-0.6</td>
    <td>-0.7</td>
    <td>+1.2</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>ar_Arab (Najdi Arabic)</td>
    <td>80.0</td>
    <td>-0.1</td>
    <td>-0.5</td>
    <td>-0.5</td>
    <td>+0.0</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>arj_Arab (Jemeni Arabic)</td>
    <td>60.8</td>
    <td>+0.1</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>+1.2</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>are_Arab (Egyptian Arabic)</td>
    <td>71.4</td>
    <td>+0.0</td>
    <td>-0.5</td>
    <td>-0.6</td>
    <td>+1.4</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>72.6</th>
    <th>-0.1</th>
    <th>-0.5</th>
    <th>-0.5</th>
    <th>+1.3</th>
    <th>+0.8</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>71.5</th>
    <th>-0.1</th>
    <th>-0.5</th>
    <th>-0.5</th>
    <th>+1.5</th>
    <th>+1.0</th>
  </tr>
  <tr>
    <th colspan="7">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>86.3</td>
    <td>-0.1</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>+0.0</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>85.2</td>
    <td>-0.3</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>-0.2</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>87.8</td>
    <td>-0.2</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>-0.2</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>89.0</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>rot_Latin (Romanian)</td>
    <td>88.2</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>gl_Latin (Galician)</td>
    <td>87.6</td>
    <td>-0.1</td>
    <td>-0.0</td>
    <td>-0.0</td>
    <td>-0.1</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>87.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>oc_Latin (Occitan)</td>
    <td>80.0</td>
    <td>+0.0</td>
    <td>-0.4</td>
    <td>-0.4</td>
    <td>+0.2</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>lae_Latin (Ladin)</td>
    <td>79.8</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>lnm_Latin (Lombard)</td>
    <td>44.5</td>
    <td>-2.3</td>
    <td>-1.0</td>
    <td>-1.0</td>
    <td>+19.3</td>
    <td>+17.1</td>
  </tr>
  <tr>
    <td>vev_Latin (Venetian)</td>
    <td>54.0</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+16.9</td>
    <td>+16.5</td>
  </tr>
  <tr>
    <td>scs_Latin (Sicilian)</td>
    <td>44.1</td>
    <td>-0.8</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+18.9</td>
    <td>+15.9</td>
  </tr>
  <tr>
    <td>sed_Latin (Sardinian)</td>
    <td>37.9</td>
    <td>-0.9</td>
    <td>-0.2</td>
    <td>-0.5</td>
    <td>+21.4</td>
    <td>+17.5</td>
  </tr>
  <tr>
    <td>fre_Latin (Corsican)</td>
    <td>29.5</td>
    <td>+0.2</td>
    <td>-0.7</td>
    <td>-0.7</td>
    <td>+19.0</td>
    <td>+16.0</td>
  </tr>
  <tr>
    <td>ljt_Latin (Ligurian)</td>
    <td>45.0</td>
    <td>-0.8</td>
    <td>-0.1</td>
    <td>-0.0</td>
    <td>+19.1</td>
    <td>+15.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>67.9</th>
    <th>-0.6</th>
    <th>-0.2</th>
    <th>-0.5</th>
    <th>+8.3</th>
    <th>+7.0</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>44.7</th>
    <th>-1.3</th>
    <th>-0.3</th>
    <th>-0.5</th>
    <th>+19.4</th>
    <th>+16.5</th>
  </tr>
  <tr>
    <th colspan="7">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>86.7</td>
    <td>-0.1</td>
    <td>-0.3</td>
    <td>-0.5</td>
    <td>-0.2</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>uaa_Latin (Northern Uyghur)</td>
    <td>39.5</td>
    <td>+0.8</td>
    <td>-0.1</td>
    <td>-0.0</td>
    <td>+5.0</td>
    <td>+3.8</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>34.4</td>
    <td>-1.8</td>
    <td>-1.3</td>
    <td>-1.1</td>
    <td>+0.1</td>
    <td>+4.2</td>
  </tr>
  <tr>
    <td>ayr_Latin (Azerbaijani)</td>
    <td>60.8</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+4.2</td>
    <td>+4.1</td>
  </tr>
  <tr>
    <td>ctf_Latin (Crimean Tatar)</td>
    <td>55.7</td>
    <td>-0.3</td>
    <td>-0.2</td>
    <td>-0.3</td>
    <td>+7.7</td>
    <td>+4.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>47.6</th>
    <th>-0.6</th>
    <th>-0.3</th>
    <th>-0.5</th>
    <th>+8.0</th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>47.6</th>
    <th>-0.6</th>
    <th>-0.3</th>
    <th>-0.5</th>
    <th>+8.0</th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th colspan="7">Indic Language Family</th>
  </tr>
  <tr>
    <td>lan_Deva (Hindi)</td>
    <td>86.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.0</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>lan_Deva (Chhattisgarhi)</td>
    <td>66.4</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.3</td>
    <td>+6.2</td>
    <td>+6.7</td>
  </tr>
  <tr>
    <td>lan_Deva (Bhojpuri)</td>
    <td>63.6</td>
    <td>-0.0</td>
    <td>-0.1</td>
    <td>-0.1</td>
    <td>+6.5</td>
    <td>+7.2</td>
  </tr>
  <tr>
    <td>lan_Deva (Maithili)</td>
    <td>69.0</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
    <td>+6.1</td>
    <td>+6.4</td>
  </tr>
  <tr>
    <td>lan_Deva (Mathili)</td>
    <td>57.9</td>
    <td>+0.2</td>
    <td>+0.0</td>
    <td>-0.1</td>
    <td>+13.1</td>
    <td>+14.0</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>64.2</th>
    <th>+0.0</th>
    <th>-0.0</th>
    <th>-0.1</th>
    <th>+8.0</th>
    <th>+8.7</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>64.2</th>
    <th>+0.0</th>
    <th>-0.0</th>
    <th>-0.1</th>
    <th>+8.0</th>
    <th>+8.7</th>
  </tr>
  <tr>
    <th colspan="7">French-Creole Language Family</th>
  </tr>
  <tr>
    <td>ht_Latin (Haitian)</td>
    <td>73.6</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.7</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>gfq/Guadeloupean</td>
    <td>52.4</td>
    <td>-0.9</td>
    <td>-1.0</td>
    <td>-1.0</td>
    <td>+1.6</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>mar1259 (Martinican)</td>
    <td>50.7</td>
    <td>+0.1</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+1.0</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <td>act (Guadeloupe Creole)</td>
    <td>43.4</td>
    <td>-0.7</td>
    <td>-0.7</td>
    <td>-0.7</td>
    <td>+1.5</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>ger (French Guianese Creole)</td>
    <td>43.9</td>
    <td>-1.0</td>
    <td>-1.1</td>
    <td>-1.2</td>
    <td>+3.5</td>
    <td>+3.3</td>
  </tr>
  <tr>
    <td>lou (Louisiana Creole)</td>
    <td>43.4</td>
    <td>+0.9</td>
    <td>+0.9</td>
    <td>+1.0</td>
    <td>+6.0</td>
    <td>+5.7</td>
  </tr>
  <tr>
    <td>ant (Antillean Creole)</td>
    <td>45.0</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>+0.7</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>ref (Réunion Creole)</td>
    <td>47.1</td>
    <td>-1.4</td>
    <td>-1.2</td>
    <td>-1.2</td>
    <td>+2.7</td>
    <td>+1.8</td>
  </tr>
  <tr>
    <td>enr (St. Eustatius)</td>
    <td>54.6</td>
    <td>-0.4</td>
    <td>-0.3</td>
    <td>-0.2</td>
    <td>+6.2</td>
    <td>+5.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>47.6</th>
    <th>-0.3</th>
    <th>-0.3</th>
    <th>-0.3</th>
    <th>+3.6</th>
    <th>+3.3</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>47.6</th>
    <th>-0.3</th>
    <th>-0.3</th>
    <th>-0.3</th>
    <th>+3.6</th>
    <th>+3.3</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>60.0</th>
    <th>-0.1</th>
    <th>-0.2</th>
    <th>-0.2</th>
    <th>+5.5</th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>53.1</th>
    <th>-0.2</th>
    <th>-0.2</th>
    <th>-0.2</th>
    <th>+5.0</th>
    <th>+5.7</th>
  </tr>
</table>

Table 19: M→D COMET scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>fthr1</th>
    <th>randaug-shell</th>
    <th>randaug-cloud</th>
    <th>-shell</th>
    <th>-cloud</th>
  </tr>
  <tr>
    <th colspan="7">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Lam (Indonesian)</td>
    <td>+8.5</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>jav_Lam (Javanese)</td>
    <td>+6.5</td>
    <td>+0.5</td>
    <td>+0.8</td>
    <td>+1.0</td>
    <td>+6.2</td>
    <td>+6.2</td>
  </tr>
  <tr>
    <td>sun_Lam (Sundanese)</td>
    <td>+6.3</td>
    <td>+0.9</td>
    <td>+1.3</td>
    <td>+1.2</td>
    <td>+8.7</td>
    <td>+9.0</td>
  </tr>
  <tr>
    <td>sno_Lam (Samoan)</td>
    <td>+5.4</td>
    <td>-2.6</td>
    <td>-2.0</td>
    <td>-2.5</td>
    <td>+6.3</td>
    <td>+6.2</td>
  </tr>
  <tr>
    <td>mri_Lam (Maori)</td>
    <td>+4.0</td>
    <td>-6.7</td>
    <td>-5.6</td>
    <td>-6.1</td>
    <td>+1.7</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>ces_Lam (Cebuano)</td>
    <td>+6.1</td>
    <td>+3.7</td>
    <td>+3.5</td>
    <td>+3.2</td>
    <td>+9.4</td>
    <td>+9.3</td>
  </tr>
  <tr>
    <td>zen_Lam (Standard Malay)</td>
    <td>+8.7</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.3</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>tgl_Lam (Tagalog)</td>
    <td>+6.4</td>
    <td>-4.1</td>
    <td>-3.8</td>
    <td>-4.3</td>
    <td>-6.5</td>
    <td>-5.4</td>
  </tr>
  <tr>
    <td>ilo_Lam (Ilocano)</td>
    <td>+4.2</td>
    <td>-4.0</td>
    <td>-3.5</td>
    <td>-3.9</td>
    <td>+6.6</td>
    <td>+7.0</td>
  </tr>
  <tr>
    <td>fil_Lam (Filipino)</td>
    <td>+6.1</td>
    <td>-4.4</td>
    <td>-3.7</td>
    <td>-4.2</td>
    <td>-3.5</td>
    <td>-3.7</td>
  </tr>
  <tr>
    <td>plh_Lam (Plateau Malagasy)</td>
    <td>+4.6</td>
    <td>-1.0</td>
    <td>-1.1</td>
    <td>-0.9</td>
    <td>+0.9</td>
    <td>+1.6</td>
  </tr>
  <tr>
    <td>pag_Lam (Pangasinan)</td>
    <td>+4.1</td>
    <td>+2.3</td>
    <td>+3.1</td>
    <td>+3.1</td>
    <td>+13.8</td>
    <td>+13.6</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+5.2</td>
    <td>-2.0</td>
    <td>-1.6</td>
    <td>-1.8</td>
    <td>+3.8</td>
    <td>+4.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+4.9</td>
    <td>-2.0</td>
    <td>-1.6</td>
    <td>-1.8</td>
    <td>+3.3</td>
    <td>+5.5</td>
  </tr>
  <tr>
    <th colspan="7">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>+8.1</td>
    <td>+0.5</td>
    <td>+0.4</td>
    <td>+0.3</td>
    <td>+0.3</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>+8.2</td>
    <td>+0.3</td>
    <td>+0.4</td>
    <td>+0.5</td>
    <td>+1.0</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>aqj_Arab (Ta'izzi-Adeni Arabic)</td>
    <td>+8.3</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.5</td>
    <td>+1.0</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>aqj_Arab (South Levantine Arabic)</td>
    <td>+8.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+0.4</td>
    <td>+1.6</td>
    <td>+1.8</td>
  </tr>
  <tr>
    <td>ajp_Arab (North Levantine Arabic)</td>
    <td>+8.7</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>-0.4</td>
    <td>+0.2</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>arh_Arab (Hijazi Arabic)</td>
    <td>+8.5</td>
    <td>+0.7</td>
    <td>+0.8</td>
    <td>+0.7</td>
    <td>+0.7</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>+7.0</td>
    <td>+0.2</td>
    <td>+0.6</td>
    <td>+0.2</td>
    <td>+2.7</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <td>azr_Arab (Egyptian Arabic)</td>
    <td>+8.2</td>
    <td>-0.0</td>
    <td>+0.1</td>
    <td>-0.1</td>
    <td>+0.3</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+8.1</td>
    <td>+0.2</td>
    <td>+0.3</td>
    <td>+0.2</td>
    <td>+1.0</td>
    <td>+1.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+7.4</td>
    <td>+0.2</td>
    <td>+0.6</td>
    <td>+0.2</td>
    <td>+2.7</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <th colspan="7">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>+8.1</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>-0.1</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>+8.6</td>
    <td>-0.3</td>
    <td>-0.2</td>
    <td>-0.2</td>
    <td>-0.4</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>+8.6</td>
    <td>-0.7</td>
    <td>-0.8</td>
    <td>-0.9</td>
    <td>-1.2</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>ptg_Latin (Portuguese)</td>
    <td>+8.4</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>+0.8</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>+8.6</td>
    <td>-0.7</td>
    <td>-0.7</td>
    <td>-0.7</td>
    <td>-1.4</td>
    <td>-1.3</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>+8.5</td>
    <td>-0.5</td>
    <td>-0.2</td>
    <td>-0.5</td>
    <td>+0.1</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>+8.0</td>
    <td>-0.9</td>
    <td>-0.8</td>
    <td>-0.8</td>
    <td>-0.4</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>+7.9</td>
    <td>-0.3</td>
    <td>+0.2</td>
    <td>+0.1</td>
    <td>+3.3</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>+7.7</td>
    <td>-1.1</td>
    <td>-0.6</td>
    <td>-1.0</td>
    <td>+1.5</td>
    <td>+1.8</td>
  </tr>
  <tr>
    <td>mbd_Latin (Moldovan)</td>
    <td>+8.3</td>
    <td>+3.2</td>
    <td>+4.6</td>
    <td>+4.9</td>
    <td>+12.2</td>
    <td>+12.1</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>+7.0</td>
    <td>+3.9</td>
    <td>+4.3</td>
    <td>+4.0</td>
    <td>+7.6</td>
    <td>+7.9</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>+5.9</td>
    <td>+1.9</td>
    <td>+3.0</td>
    <td>+2.5</td>
    <td>+12.0</td>
    <td>+11.9</td>
  </tr>
  <tr>
    <td>ord_Latin (Sardinian)</td>
    <td>+5.9</td>
    <td>-0.1</td>
    <td>+1.4</td>
    <td>+1.0</td>
    <td>+12.0</td>
    <td>+12.2</td>
  </tr>
  <tr>
    <td>ltg_Latin (Ligurian)</td>
    <td>+5.9</td>
    <td>+4.1</td>
    <td>+5.0</td>
    <td>+4.8</td>
    <td>+11.9</td>
    <td>+12.2</td>
  </tr>
  <tr>
    <td>lij_Latin (Ligurian)</td>
    <td>+5.9</td>
    <td>+4.8</td>
    <td>+6.2</td>
    <td>+5.6</td>
    <td>+15.3</td>
    <td>+14.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+7.4</td>
    <td>+0.9</td>
    <td>+1.5</td>
    <td>+1.3</td>
    <td>+5.1</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+5.8</td>
    <td>+2.8</td>
    <td>+4.0</td>
    <td>+3.5</td>
    <td>+12.7</td>
    <td>+12.7</td>
  </tr>
  <tr>
    <th colspan="7">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>+8.7</td>
    <td>+0.2</td>
    <td>+0.3</td>
    <td>-0.0</td>
    <td>+0.1</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>uzn_Latin (Northern Uzbek)</td>
    <td>+5.6</td>
    <td>+1.4</td>
    <td>+1.7</td>
    <td>+1.3</td>
    <td>+7.6</td>
    <td>+7.7</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>+5.1</td>
    <td>+8.8</td>
    <td>+8.7</td>
    <td>+8.5</td>
    <td>+12.6</td>
    <td>+12.2</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>+7.0</td>
    <td>+2.6</td>
    <td>+2.5</td>
    <td>+2.4</td>
    <td>+5.3</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>crh_Latin (Crimean Tatar)</td>
    <td>+6.4</td>
    <td>+4.4</td>
    <td>+4.1</td>
    <td>+4.1</td>
    <td>+7.8</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+6.0</td>
    <td>+4.3</td>
    <td>+4.3</td>
    <td>+4.1</td>
    <td>+8.3</td>
    <td>+8.2</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+6.0</td>
    <td>+4.3</td>
    <td>+4.3</td>
    <td>+4.1</td>
    <td>+8.3</td>
    <td>+8.2</td>
  </tr>
  <tr>
    <th colspan="7">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>+8.3</td>
    <td>-1.0</td>
    <td>-1.0</td>
    <td>-1.0</td>
    <td>-1.1</td>
    <td>-1.0</td>
  </tr>
  <tr>
    <td>hne_Deva (Chittagari)</td>
    <td>+7.2</td>
    <td>-1.4</td>
    <td>-1.5</td>
    <td>-1.4</td>
    <td>+2.9</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>+7.5</td>
    <td>-2.5</td>
    <td>-2.8</td>
    <td>-2.5</td>
    <td>+1.9</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>+7.9</td>
    <td>-1.2</td>
    <td>-1.1</td>
    <td>-1.0</td>
    <td>+2.7</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <td>mal_Deva (Maithili)</td>
    <td>+7.4</td>
    <td>-1.8</td>
    <td>-1.6</td>
    <td>-2.0</td>
    <td>+3.9</td>
    <td>+4.2</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+7.6</td>
    <td>-1.7</td>
    <td>-1.8</td>
    <td>-1.7</td>
    <td>+2.8</td>
    <td>+3.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+7.6</td>
    <td>-1.7</td>
    <td>-1.8</td>
    <td>-1.7</td>
    <td>+2.8</td>
    <td>+3.0</td>
  </tr>
  <tr>
    <th colspan="7">French-Creole Language Family</th>
  </tr>
  <tr>
    <td>hat_Haitian</td>
    <td>+5.1</td>
    <td>+21.1</td>
    <td>+21.2</td>
    <td>+21.3</td>
    <td>+20.3</td>
    <td>+20.5</td>
  </tr>
  <tr>
    <td>gcd (Gadoucouy)</td>
    <td>+5.3</td>
    <td>+1.1</td>
    <td>+1.0</td>
    <td>+0.6</td>
    <td>+2.5</td>
    <td>+3.6</td>
  </tr>
  <tr>
    <td>mart1259 (Martinican)</td>
    <td>+4.9</td>
    <td>+4.2</td>
    <td>+3.2</td>
    <td>+4.1</td>
    <td>+6.7</td>
    <td>+7.2</td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>+4.2</td>
    <td>+6.4</td>
    <td>+5.8</td>
    <td>+6.4</td>
    <td>+6.3</td>
    <td>+7.2</td>
  </tr>
  <tr>
    <td>ger (French Guianese Creole)</td>
    <td>+4.6</td>
    <td>+7.5</td>
    <td>+4.4</td>
    <td>+3.8</td>
    <td>+7.5</td>
    <td>+8.2</td>
  </tr>
  <tr>
    <td>brt (British Creole)</td>
    <td>+4.9</td>
    <td>+3.3</td>
    <td>+3.2</td>
    <td>+3.2</td>
    <td>+9.7</td>
    <td>+10.4</td>
  </tr>
  <tr>
    <td>mfe (Mauritian)</td>
    <td>+4.5</td>
    <td>+9.7</td>
    <td>+9.8</td>
    <td>+9.8</td>
    <td>+10.1</td>
    <td>+10.4</td>
  </tr>
  <tr>
    <td>rcf (Réunion Creole)</td>
    <td>+4.8</td>
    <td>+2.8</td>
    <td>+4.5</td>
    <td>+3.6</td>
    <td>+6.3</td>
    <td>+6.2</td>
  </tr>
  <tr>
    <td>crs (Seychellois)</td>
    <td>+4.7</td>
    <td>+14.0</td>
    <td>+13.9</td>
    <td>+13.6</td>
    <td>+15.7</td>
    <td>+15.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>+4.7</td>
    <td>+6.2</td>
    <td>+5.7</td>
    <td>+5.7</td>
    <td>+7.5</td>
    <td>+7.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>+4.8</td>
    <td>+7.8</td>
    <td>+7.4</td>
    <td>+7.4</td>
    <td>+8.9</td>
    <td>+9.2</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>+6.2</td>
    <td>+0.5</td>
    <td>+0.5</td>
    <td>+0.4</td>
    <td>+5.0</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>+5.4</td>
    <td>+1.0</td>
    <td>+1.1</td>
    <td>+1.0</td>
    <td>+7.2</td>
    <td>+6.6</td>
  </tr>
</table>

Table 20: M→D COMET scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.
<table>
  <tr>
    <th></th>
    <th>-cloud</th>
    <th>-cloud-multsrcs</th>
  </tr>
  <tr>
    <th colspan="3">Arabic Language Family (FloRes Dataset)</th>
  </tr>
  <tr><td>acm_Arab</td><td>30.4</td><td>30.5</td></tr>
  <tr><td>acq_Arab</td><td>31.4</td><td>31.8</td></tr>
  <tr><td>aeb_Arab</td><td>25.4</td><td>26.0</td></tr>
  <tr><td>aip_Arab</td><td>33.5</td><td>35.2</td></tr>
  <tr><td>apc_Arab</td><td>30.3</td><td>32.4</td></tr>
  <tr><td>arb_Arab</td><td>37.1</td><td>36.0</td></tr>
  <tr><td>ars_Arab</td><td>36.1</td><td>35.2</td></tr>
  <tr><td>ary_Arab</td><td>19.7</td><td>23.8</td></tr>
  <tr><td>arz_Arab</td><td>27.9</td><td>29.4</td></tr>
  <tr><th>Average</th><th>30.2</th><th>31.1</th></tr>
  <tr>
    <th colspan="3">Romance Language Family</th>
  </tr>
  <tr><td>ast_Latn</td><td>31.6</td><td>33.4</td></tr>
  <tr><td>cat_Latn</td><td>33.4</td><td>35.8</td></tr>
  <tr><td>fra_Latn</td><td>40.9</td><td>41.4</td></tr>
  <tr><td>fur_Latn</td><td>24.4</td><td>24.1</td></tr>
  <tr><td>glg_Latn</td><td>34.1</td><td>36.4</td></tr>
  <tr><td>ita_Latn</td><td>32.0</td><td>33.5</td></tr>
  <tr><td>lij_Latn</td><td>25.6</td><td>25.0</td></tr>
  <tr><td>lmo_Latn</td><td>26.2</td><td>25.5</td></tr>
  <tr><td>oci_Latn</td><td>36.5</td><td>37.3</td></tr>
  <tr><td>por_Latn</td><td>43.8</td><td>45.2</td></tr>
  <tr><td>ron_Latn</td><td>38.2</td><td>40.4</td></tr>
  <tr><td>scn_Latn</td><td>26.1</td><td>25.5</td></tr>
  <tr><td>spa_Latn</td><td>30.7</td><td>30.9</td></tr>
  <tr><td>srd_Latn</td><td>24.1</td><td>23.5</td></tr>
  <tr><td>vec_Latn</td><td>32.7</td><td>31.0</td></tr>
  <tr><th>Average</th><th>32.0</th><th>32.6</th></tr>
  <tr>
    <th colspan="3">Turkic Language Family</th>
  </tr>
  <tr><td>azi_Latn</td><td>11.3</td><td>11.6</td></tr>
  <tr><td>crh_Latn</td><td>17.0</td><td>16.4</td></tr>
  <tr><td>tuk_Latn</td><td>9.4</td><td>9.7</td></tr>
  <tr><td>tur_Latn</td><td>29.2</td><td>27.7</td></tr>
  <tr><td>uzn_Latn</td><td>7.6</td><td>9.2</td></tr>
  <tr><th>Average</th><th>14.9</th><th>14.9</th></tr>
</table>

Table 21: Summary of adopting multiple HRLs as sources for the M→D approach on the model Aya-23. These results are computed over a 300 sample-subset of the FloRes-200 test set, for Aya-23

<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>fthr1</th>
    <th>-shell</th>
  </tr>
  <tr><td>bho_Deva</td><td>10.1</td><td>10.0</td><td>13.3</td></tr>
  <tr><td>hin_Deva</td><td>34.8</td><td>33.4</td><td>33.3</td></tr>
  <tr><td>hne_Deva</td><td>14.5</td><td>13.2</td><td>19.5</td></tr>
  <tr><td>mag_Deva</td><td>17.4</td><td>16.0</td><td>21.3</td></tr>
  <tr><td>mai_Deva</td><td>9.6</td><td>7.1</td><td>16.1</td></tr>
  <tr><th>Average</th><th>17.3</th><th>15.9</th><th>20.7</th></tr>
</table>

Table 22: Using indicorp to noise for languages in the Indic family. The M2M model was tested. These results are computed over a 300 sample-subset of the FloRes-200 test set.

<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>fthr1</th>
    <th>-shell</th>
  </tr>
  <tr><td>bho_Deva</td><td>10.1</td><td>10.0</td><td>13.3</td></tr>
  <tr><td>hin_Deva</td><td>34.8</td><td>33.4</td><td>33.3</td></tr>
  <tr><td>hne_Deva</td><td>14.5</td><td>13.2</td><td>19.5</td></tr>
  <tr><td>mag_Deva</td><td>17.4</td><td>16.0</td><td>21.3</td></tr>
  <tr><td>mai_Deva</td><td>9.6</td><td>7.1</td><td>16.1</td></tr>
  <tr><th>Average</th><th>17.3</th><th>15.9</th><th>20.7</th></tr>
</table>

Table 23: Using the NLLB dataset to noise the languages in the Indic family. The M2M model was tested. These results are computed over a 300 sample-subset of the FloRes-200 test set.

<table>
  <tr>
    <th>Language Family</th>
    <th>-cloud</th>
    <th>-cloud-sem</th>
  </tr>
  <tr><td>Austronesian</td><td>17.9</td><td>18.0</td></tr>
  <tr><td>Arabic</td><td>22.8</td><td>22.6</td></tr>
  <tr><td>Romance</td><td>30.5</td><td>30.5</td></tr>
  <tr><td>Turkic</td><td>11.8</td><td>12.0</td></tr>
  <tr><td>Indic</td><td>21.0</td><td>21.0</td></tr>
  <tr><td>French-Creole</td><td>12.1</td><td>-</td></tr>
</table>

Table 24: Additional semantic noisers as described in § E.1 for M→D are applied to the model M2M. Language family means were taken.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cont</th>
    <th>-func</th>
    <th>-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>38.9</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>22.9</td>
    <td>-0.2</td>
    <td>+0.2</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>22.9</td>
    <td>-2.6</td>
    <td>+1.0</td>
    <td>-1.5</td>
  </tr>
  <tr>
    <td>smo_Latin (Samoan)</td>
    <td>2.3</td>
    <td>+0.2</td>
    <td>+1.0</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>mri_Latin (Mori)</td>
    <td>2.3</td>
    <td>+0.5</td>
    <td>+1.2</td>
    <td>+2.3</td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>22.2</td>
    <td>-2.7</td>
    <td>-9.5</td>
    <td>-10.4</td>
  </tr>
  <tr>
    <td>zsm_Latin (Standard Malay)</td>
    <td>38.7</td>
    <td>-6.0</td>
    <td>-1.5</td>
    <td>-7.5</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>27.8</td>
    <td>-6.3</td>
    <td>-14.3</td>
    <td>-16.3</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>18.4</td>
    <td>+0.5</td>
    <td>+0.3</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>fji_Latin (Fijian)</td>
    <td>2.3</td>
    <td>+0.3</td>
    <td>+1.3</td>
    <td>+1.9</td>
  </tr>
  <tr>
    <td>plt_Latin (Plateau Malagasy)</td>
    <td>2.5</td>
    <td>+0.7</td>
    <td>+2.4</td>
    <td>+3.8</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>6.4</td>
    <td>-0.6</td>
    <td>+5.5</td>
    <td>+5.1</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>14.9</th>
    <th>-1.5</th>
    <th>-1.5</th>
    <th>-2.1</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>10.8</th>
    <th>-0.4</th>
    <th>-0.1</th>
    <th>+0.0</th>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>29.6</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>22.4</td>
    <td>-0.5</td>
    <td>+1.0</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta’izzi-Adeni Arabic)</td>
    <td>24.7</td>
    <td>-0.6</td>
    <td>+0.7</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>17.7</td>
    <td>-1.3</td>
    <td>+1.3</td>
    <td>+0.3</td>
  </tr>
  <tr>
    <td>apj_Arab (Jordanian Arabic)</td>
    <td>24.5</td>
    <td>-5.3</td>
    <td>+3.8</td>
    <td>+2.7</td>
  </tr>
  <tr>
    <td>ape_Arab (North Levantine Arabic)</td>
    <td>21.7</td>
    <td>-3.6</td>
    <td>+1.0</td>
    <td>-2.8</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>28.5</td>
    <td>+0.0</td>
    <td>+0.1</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>11.9</td>
    <td>-1.3</td>
    <td>+2.0</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>18.2</td>
    <td>+3.1</td>
    <td>+2.9</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>21.2</th>
    <th>-2.0</th>
    <th>+0.7</th>
    <th>-1.2</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>20.2</th>
    <th>-2.2</th>
    <th>+0.8</th>
    <th>-1.4</th>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>30.8</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>26.8</td>
    <td>-10.0</td>
    <td>-7.2</td>
    <td>-16.9</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>41.6</td>
    <td>-16.5</td>
    <td>+13.0</td>
    <td>-28.3</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>45.8</td>
    <td>-16.2</td>
    <td>-14.9</td>
    <td>-30.0</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>40.8</td>
    <td>-16.1</td>
    <td>-15.6</td>
    <td>-28.7</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>37.5</td>
    <td>-14.1</td>
    <td>-12.5</td>
    <td>-21.5</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>42.7</td>
    <td>-19.9</td>
    <td>+17.5</td>
    <td>-26.1</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>44.6</td>
    <td>-10.7</td>
    <td>-5.1</td>
    <td>-15.7</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>34.3</td>
    <td>-10.9</td>
    <td>-7.6</td>
    <td>-18.4</td>
  </tr>
  <tr>
    <td>lin_Latin (Lombard)</td>
    <td>9.6</td>
    <td>+1.6</td>
    <td>+5.7</td>
    <td>+7.3</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>17.2</td>
    <td>+1.7</td>
    <td>+8.0</td>
    <td>+9.7</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>7.3</td>
    <td>+1.6</td>
    <td>+8.5</td>
    <td>+10.0</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>5.3</td>
    <td>+1.8</td>
    <td>+10.5</td>
    <td>+13.8</td>
  </tr>
  <tr>
    <td>fur_Latin (Friulian)</td>
    <td>8.6</td>
    <td>+2.5</td>
    <td>+6.0</td>
    <td>+10.5</td>
  </tr>
  <tr>
    <td>liu_Latin (Ligurian)</td>
    <td>24.8</td>
    <td>+2.2</td>
    <td>+8.0</td>
    <td>+10.2</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>26.6</th>
    <th>-7.1</th>
    <th>-2.8</th>
    <th>-9.2</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>9.6</th>
    <th>+1.6</th>
    <th>+8.0</th>
    <th>+10.8</th>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>33.3</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzr_Latin (Northern Uzbek)</td>
    <td>1.6</td>
    <td>+2.2</td>
    <td>+1.0</td>
    <td>+5.0</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>2.4</td>
    <td>+3.3</td>
    <td>+1.6</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>7.4</td>
    <td>-0.2</td>
    <td>+0.1</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>crh_Latin (Crimean Tatar)</td>
    <td>8.5</td>
    <td>+1.3</td>
    <td>+1.3</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>5.0</th>
    <th>+1.4</th>
    <th>+1.0</th>
    <th>+3.0</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>5.0</th>
    <th>+1.4</th>
    <th>+1.0</th>
    <th>+3.0</th>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>33.4</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>15.2</td>
    <td>-1.3</td>
    <td>+13.9</td>
    <td>+9.9</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>10.1</td>
    <td>+0.3</td>
    <td>+6.1</td>
    <td>+6.4</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>17.2</td>
    <td>+0.8</td>
    <td>+12.8</td>
    <td>+12.8</td>
  </tr>
  <tr>
    <td>mar_Deva (Marathi)</td>
    <td>7.7</td>
    <td>+1.4</td>
    <td>+15.3</td>
    <td>+16.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>12.6</th>
    <th>+0.3</th>
    <th>+12.0</th>
    <th>+11.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>12.6</th>
    <th>+0.3</th>
    <th>+12.0</th>
    <th>+11.4</th>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat (Haitian)</td>
    <td>39.7</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>3.7</td>
    <td>+2.9</td>
    <td>+3.7</td>
    <td>+7.8</td>
  </tr>
  <tr>
    <td>mts (Mauritian)</td>
    <td>5.1</td>
    <td>+1.6</td>
    <td>+1.1</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>crs (Seychellois)</td>
    <td>11.2</td>
    <td>+0.8</td>
    <td>+4.1</td>
    <td>+5.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>5.9</th>
    <th>+2.5</th>
    <th>+3.7</th>
    <th>+5.6</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>5.9</th>
    <th>+2.5</th>
    <th>+3.7</th>
    <th>+5.6</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>16.8</th>
    <th>-1.4</th>
    <th>+1.5</th>
    <th>-0.8</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>10.9</th>
    <th>+0.9</th>
    <th>+4.3</th>
    <th>+4.5</th>
  </tr>
</table>

Table 25: D→M BLEU scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each D→M approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The best score for each language in the D→M paradigm is bolded and underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cont</th>
    <th>-func</th>
    <th>-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>41.5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>14.2</td>
    <td>+2.9</td>
    <td>+7.8</td>
    <td><u>+10.6</u></td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>11.0</td>
    <td>+3.6</td>
    <td>+10.1</td>
    <td><u>+11.4</u></td>
  </tr>
  <tr>
    <td>smo_Latin (Samoan)</td>
    <td>3.2</td>
    <td>-0.1</td>
    <td>+0.4</td>
    <td><u>+1.1</u></td>
  </tr>
  <tr>
    <td>mri_Latin (Mori)</td>
    <td>4.6</td>
    <td><u>+8.4</u></td>
    <td>-0.8</td>
    <td><u>+8.6</u></td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>14.0</td>
    <td><u>-0.0</u></td>
    <td>-3.0</td>
    <td>-1.7</td>
  </tr>
  <tr>
    <td>zsm_Latin (Standard Malay)</td>
    <td>36.5</td>
    <td>-3.7</td>
    <td><u>-1.2</u></td>
    <td>-4.6</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>25.6</td>
    <td><u>+4.7</u></td>
    <td>-12.5</td>
    <td>-12.4</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>7.5</td>
    <td>+0.5</td>
    <td>+0.8</td>
    <td><u>+2.4</u></td>
  </tr>
  <tr>
    <td>fji_Latin (Fijian)</td>
    <td>3.3</td>
    <td>+0.4</td>
    <td>-0.1</td>
    <td><u>+0.7</u></td>
  </tr>
  <tr>
    <td>plt_Latin (Plateau Malagasy)</td>
    <td>3.3</td>
    <td>+1.1</td>
    <td>+1.3</td>
    <td><u>+3.9</u></td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>7.3</td>
    <td>-1.0</td>
    <td><u>+5.1</u></td>
    <td>+4.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>11.9</th>
    <th>-0.1</th>
    <th>+0.7</th>
    <th><u>+1.5</u></th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>7.6</th>
    <th>+0.8</th>
    <th>+2.4</th>
    <th><u>+3.7</u></th>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>38.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>31.6</td>
    <td>-1.7</td>
    <td><u>-0.2</u></td>
    <td>-1.9</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta’izzi-Adeni Arabic)</td>
    <td>33.7</td>
    <td>-1.3</td>
    <td><u>+0.2</u></td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>26.7</td>
    <td>-2.8</td>
    <td><u>+0.2</u></td>
    <td>-2.6</td>
  </tr>
  <tr>
    <td>apj_Arab (Jordanian Arabic)</td>
    <td>37.4</td>
    <td>+2.1</td>
    <td><u>+0.6</u></td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>ape_Arab (North Levantine Arabic)</td>
    <td>33.2</td>
    <td>-6.7</td>
    <td><u>-1.2</u></td>
    <td>-8.2</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>37.1</td>
    <td>-0.1</td>
    <td><u>+0.0</u></td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>23.5</td>
    <td>-6.2</td>
    <td><u>+1.8</u></td>
    <td>-6.9</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>28.9</td>
    <td>+4.4</td>
    <td>-1.8</td>
    <td>+2.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>31.5</th>
    <th>-4.1</th>
    <th><u>-1.1</u></th>
    <th>-5.0</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>23.5</th>
    <th>-6.2</th>
    <th><u>+1.8</u></th>
    <th>-6.9</th>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>32.6</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>29.0</td>
    <td>-10.1</td>
    <td><u>-4.6</u></td>
    <td>-13.4</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>44.0</td>
    <td>-15.5</td>
    <td><u>+0.8</u></td>
    <td>-21.9</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>46.9</td>
    <td>-15.8</td>
    <td><u>+7.7</u></td>
    <td>-20.7</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>42.0</td>
    <td>-15.3</td>
    <td><u>+10.2</u></td>
    <td>-22.4</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>35.5</td>
    <td>-12.4</td>
    <td><u>-5.7</u></td>
    <td>-18.1</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>36.5</td>
    <td>-10.6</td>
    <td><u>+5.7</u></td>
    <td>-14.7</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>35.1</td>
    <td>-8.1</td>
    <td><u>+2.2</u></td>
    <td>-9.1</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>31.8</td>
    <td>-8.3</td>
    <td><u>+5.1</u></td>
    <td>-11.8</td>
  </tr>
  <tr>
    <td>lin_Latin (Lombard)</td>
    <td>17.4</td>
    <td>-2.0</td>
    <td><u>+2.0</u></td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>26.7</td>
    <td>-4.7</td>
    <td><u>+2.7</u></td>
    <td>-2.7</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>16.7</td>
    <td>-0.6</td>
    <td><u>+3.7</u></td>
    <td>+1.9</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>16.4</td>
    <td>-1.4</td>
    <td><u>+6.1</u></td>
    <td>+4.4</td>
  </tr>
  <tr>
    <td>fur_Latin (Friulian)</td>
    <td>17.7</td>
    <td>-0.4</td>
    <td><u>+4.5</u></td>
    <td>+4.4</td>
  </tr>
  <tr>
    <td>lij_Latin (Ligurian)</td>
    <td>17.4</td>
    <td>-0.5</td>
    <td><u>+4.8</u></td>
    <td>+4.3</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>29.5</th>
    <th>-7.6</th>
    <th><u>-1.8</u></th>
    <th>-8.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>17.1</th>
    <th>+4.0</th>
    <th><u>+2.7</u></th>
    <th>+3.1</th>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>31.7</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzr_Latin (Northern Uzbek)</td>
    <td>3.8</td>
    <td>+1.3</td>
    <td>+0.6</td>
    <td><u>+2.1</u></td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>6.1</td>
    <td>+0.7</td>
    <td>+0.9</td>
    <td><u>+2.0</u></td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>9.8</td>
    <td>-2.7</td>
    <td><u>-0.3</u></td>
    <td>-3.1</td>
  </tr>
  <tr>
    <td>crh_Latin (Crimean Tatar)</td>
    <td>11.9</td>
    <td>-1.4</td>
    <td><u>+0.0</u></td>
    <td>-0.8</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>7.9</th>
    <th>-0.6</th>
    <th><u>+0.3</u></th>
    <th>+0.0</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>7.9</th>
    <th>-0.6</th>
    <th><u>+0.3</u></th>
    <th>+0.0</th>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>34.8</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>23.3</td>
    <td>-3.3</td>
    <td><u>+6.0</u></td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>16.2</td>
    <td>-0.2</td>
    <td><u>+3.2</u></td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>24.9</td>
    <td>-0.3</td>
    <td><u>+5.5</u></td>
    <td>+5.3</td>
  </tr>
  <tr>
    <td>mar_Deva (Malathi)</td>
    <td>17.2</td>
    <td>-0.4</td>
    <td><u>+7.8</u></td>
    <td>+7.4</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>20.4</th>
    <th>-1.1</th>
    <th><u>+5.6</u></th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>20.4</th>
    <th>-1.1</th>
    <th><u>+5.6</u></th>
    <th>+4.4</th>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat (Haitian)</td>
    <td>8.9</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>2.6</td>
    <td>+0.2</td>
    <td>-0.1</td>
    <td><u>+0.5</u></td>
  </tr>
  <tr>
    <td>mtc (Mauritian)</td>
    <td>3.4</td>
    <td>-0.3</td>
    <td><u>+0.2</u></td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>crs (Seychellois)</td>
    <td>5.7</td>
    <td>-1.1</td>
    <td><u>+0.8</u></td>
    <td>-0.5</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>7.3</th>
    <th>-3.8</th>
    <th><u>-3.3</u></th>
    <th>-3.5</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>7.3</th>
    <th>-3.8</th>
    <th><u>-3.3</u></th>
    <th>-3.5</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>19.7</th>
    <th>-2.2</th>
    <th><u>+1.2</u></th>
    <th>-1.6</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>11.3</th>
    <th>-0.0</th>
    <th><u>+3.0</u></th>
    <th>+2.7</th>
  </tr>
</table>

Table 26: D→M BLEU scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each D→M approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The best score for each language in the D→M paradigm is bolded and underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cont</th>
    <th>-func</th>
    <th>-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>87.4</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>70.1</td>
    <td>+0.6</td>
    <td>-0.0</td>
    <td>-0.6</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>71.6</td>
    <td>-2.6</td>
    <td>+0.9</td>
    <td>-2.7</td>
  </tr>
  <tr>
    <td>sma_Latin (Samoan)</td>
    <td>27.4</td>
    <td>+7.5</td>
    <td>+3.9</td>
    <td>+13.6</td>
  </tr>
  <tr>
    <td>mri_Latin (Ori)</td>
    <td>25.7</td>
    <td>+7.2</td>
    <td>+7.9</td>
    <td>+17.0</td>
  </tr>
  <tr>
    <td>cdo_Latin (Cebuano)</td>
    <td>64.2</td>
    <td>+3.3</td>
    <td>+8.4</td>
    <td>+8.8</td>
  </tr>
  <tr>
    <td>zsm_Latin (Standard Malay)</td>
    <td>86.5</td>
    <td>-6.7</td>
    <td>-1.5</td>
    <td>-8.0</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>73.2</td>
    <td>-7.9</td>
    <td>-14.2</td>
    <td>-18.0</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>56.1</td>
    <td>+0.8</td>
    <td>+4.3</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>fij_Latin (Fijian)</td>
    <td>27.7</td>
    <td>+7.9</td>
    <td>+7.4</td>
    <td>+14.9</td>
  </tr>
  <tr>
    <td>plt_Latin (Plateau Malagasy)</td>
    <td>44.3</td>
    <td>-1.3</td>
    <td>-7.3</td>
    <td>-8.7</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>35.5</td>
    <td>+2.4</td>
    <td>+8.6</td>
    <td>+11.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>52.9</td>
    <td>+0.3</td>
    <td>-0.6</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>46.9</td>
    <td>+2.0</td>
    <td>+1.0</td>
    <td>+4.4</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>80.6</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>76.0</td>
    <td>-1.1</td>
    <td>+0.3</td>
    <td>-0.9</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta’izzi-Adeni Arabic)</td>
    <td>77.6</td>
    <td>-1.3</td>
    <td>+0.0</td>
    <td>-1.2</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>68.8</td>
    <td>-1.6</td>
    <td>+0.8</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>app_Arab (Algerian Arabic)</td>
    <td>73.2</td>
    <td>+6.5</td>
    <td>+0.4</td>
    <td>+6.9</td>
  </tr>
  <tr>
    <td>ape_Arab (North Levantine Arabic)</td>
    <td>72.8</td>
    <td>-4.8</td>
    <td>+0.6</td>
    <td>-4.3</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>80.0</td>
    <td>+0.0</td>
    <td>+0.0</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>80.8</td>
    <td>-2.3</td>
    <td>+2.4</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>71.4</td>
    <td>-4.4</td>
    <td>+0.6</td>
    <td>-3.9</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>72.6</td>
    <td>-2.7</td>
    <td>+0.5</td>
    <td>-2.3</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>71.5</td>
    <td>-3.1</td>
    <td>+0.6</td>
    <td>-2.6</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>86.3</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>85.2</td>
    <td>-18.6</td>
    <td>-8.1</td>
    <td>-26.7</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>87.5</td>
    <td>-20.8</td>
    <td>-10.6</td>
    <td>-30.6</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>88.0</td>
    <td>-18.9</td>
    <td>-10.2</td>
    <td>-28.6</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>88.2</td>
    <td>-20.8</td>
    <td>-12.9</td>
    <td>-32.3</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>86.6</td>
    <td>-18.6</td>
    <td>-9.2</td>
    <td>-27.8</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>87.1</td>
    <td>-19.9</td>
    <td>-27.7</td>
    <td>-47.7</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>80.0</td>
    <td>-10.2</td>
    <td>-2.8</td>
    <td>-13.3</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>79.8</td>
    <td>-14.0</td>
    <td>-7.3</td>
    <td>-21.3</td>
  </tr>
  <tr>
    <td>lin_Latin (Lombard)</td>
    <td>43.5</td>
    <td>+2.6</td>
    <td>+8.6</td>
    <td>+11.2</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>54.0</td>
    <td>+1.7</td>
    <td>+8.3</td>
    <td>+10.0</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>44.1</td>
    <td>+2.6</td>
    <td>+8.7</td>
    <td>+11.2</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>37.9</td>
    <td>+3.6</td>
    <td>+11.3</td>
    <td>+14.9</td>
  </tr>
  <tr>
    <td>fur_Latin (Friulian)</td>
    <td>42.5</td>
    <td>+6.4</td>
    <td>+8.2</td>
    <td>+14.8</td>
  </tr>
  <tr>
    <td>lij_Latin (Ligurian)</td>
    <td>45.0</td>
    <td>+3.3</td>
    <td>+11.0</td>
    <td>+14.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>67.9</td>
    <td>-8.7</td>
    <td>-1.2</td>
    <td>-10.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>44.7</td>
    <td>+2.8</td>
    <td>+9.0</td>
    <td>+11.5</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>86.7</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzn_Latin (Northern Uzbek)</td>
    <td>39.5</td>
    <td>+11.4</td>
    <td>+4.9</td>
    <td>+16.0</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>34.4</td>
    <td>+8.2</td>
    <td>+3.3</td>
    <td>+11.2</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>60.8</td>
    <td>-0.7</td>
    <td>-0.9</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>crh_Latin (Crimean Tatar)</td>
    <td>55.7</td>
    <td>+1.9</td>
    <td>+2.4</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>47.6</td>
    <td>+5.2</td>
    <td>+2.9</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>47.6</td>
    <td>+5.2</td>
    <td>+2.9</td>
    <td>+7.6</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>86.2</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>66.4</td>
    <td>-2.2</td>
    <td>+10.0</td>
    <td>+6.3</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>63.6</td>
    <td>+0.9</td>
    <td>+7.2</td>
    <td>+7.0</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>69.0</td>
    <td>+0.7</td>
    <td>+9.7</td>
    <td>+9.3</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>57.9</td>
    <td>+3.9</td>
    <td>+16.4</td>
    <td>+17.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>64.2</td>
    <td>+0.8</td>
    <td>+10.8</td>
    <td>+10.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>64.2</td>
    <td>+0.8</td>
    <td>+10.8</td>
    <td>+10.0</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>62.5</td>
    <td>+2.8</td>
    <td>+0.9</td>
    <td>-1.9</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>54.6</td>
    <td>+1.2</td>
    <td>+4.1</td>
    <td>+5.4</td>
  </tr>
</table>

Table 27: D→M COMET scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each D→M approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cont</th>
    <th>-func</th>
    <th>-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>88.5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>67.5</td>
    <td>+1.1</td>
    <td>+3.5</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>63.7</td>
    <td>+1.7</td>
    <td>+7.6</td>
    <td>+6.1</td>
  </tr>
  <tr>
    <td>sma_Latin (Samaran)</td>
    <td>58.4</td>
    <td>+3.9</td>
    <td>+8.5</td>
    <td>+5.3</td>
  </tr>
  <tr>
    <td>msr_Latin (Maori)</td>
    <td>40.5</td>
    <td>+3.6</td>
    <td>-2.8</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>61.1</td>
    <td>-2.2</td>
    <td>-7.7</td>
    <td>-7.7</td>
  </tr>
  <tr>
    <td>zsm_Latin (Standard Malay)</td>
    <td>85.7</td>
    <td>-4.3</td>
    <td>-1.0</td>
    <td>-5.1</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>76.4</td>
    <td>-2.3</td>
    <td>-17.0</td>
    <td>-19.2</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>48.2</td>
    <td>+2.6</td>
    <td>-2.4</td>
    <td>+0.4</td>
  </tr>
  <tr>
    <td>fij_Latin (Fijian)</td>
    <td>40.1</td>
    <td>+3.5</td>
    <td>-3.2</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>plt_Latin (Plateau Malagasy)</td>
    <td>46.1</td>
    <td>+2.2</td>
    <td>-4.5</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>43.1</td>
    <td>+0.4</td>
    <td>+7.7</td>
    <td>+7.4</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>55.2</th>
    <th>+0.5</th>
    <th>-1.7</th>
    <th>-0.3</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>49.5</th>
    <th>+1.9</th>
    <th>-0.1</th>
    <th>+2.4</th>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>86.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>83.2</td>
    <td>-1.5</td>
    <td>-0.2</td>
    <td>-1.7</td>
  </tr>
  <tr>
    <td>acp_Arab (Levantine Arabic)</td>
    <td>84.1</td>
    <td>+1.2</td>
    <td>+0.1</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>78.4</td>
    <td>-2.8</td>
    <td>-0.0</td>
    <td>-3.3</td>
  </tr>
  <tr>
    <td>ajr_Arab (South Levantine Arabic)</td>
    <td>83.7</td>
    <td>-8.0</td>
    <td>-1.8</td>
    <td>-10.1</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>82.7</td>
    <td>-6.2</td>
    <td>-1.0</td>
    <td>-7.7</td>
  </tr>
  <tr>
    <td>ary_Arab (Modern Arabic)</td>
    <td>85.4</td>
    <td>+0.1</td>
    <td>+0.1</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>ary_Arab (Modern Arabic)</td>
    <td>74.0</td>
    <td>-6.5</td>
    <td>-1.3</td>
    <td>-7.7</td>
  </tr>
  <tr>
    <td>azr_Arab (Egyptian Arabic)</td>
    <td>82.4</td>
    <td>-6.2</td>
    <td>-1.4</td>
    <td>-7.9</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>81.7</th>
    <th>-4.1</th>
    <th>-0.7</th>
    <th>-5.0</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>74.0</th>
    <th>-6.5</th>
    <th>-1.3</th>
    <th>-7.7</th>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>87.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>86.1</td>
    <td>-15.1</td>
    <td>-4.1</td>
    <td>-19.6</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>88.6</td>
    <td>-14.4</td>
    <td>-6.1</td>
    <td>-20.9</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>88.4</td>
    <td>-13.7</td>
    <td>-4.1</td>
    <td>-17.4</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>88.6</td>
    <td>-14.5</td>
    <td>-6.0</td>
    <td>-21.9</td>
  </tr>
  <tr>
    <td>gle_Latin (Irish)</td>
    <td>85.8</td>
    <td>-13.7</td>
    <td>-4.8</td>
    <td>-19.0</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>84.0</td>
    <td>-11.3</td>
    <td>-5.0</td>
    <td>-16.7</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>74.9</td>
    <td>-9.0</td>
    <td>-1.9</td>
    <td>-10.9</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>77.1</td>
    <td>-10.2</td>
    <td>-4.5</td>
    <td>-14.6</td>
  </tr>
  <tr>
    <td>limb_Latin (Lombard)</td>
    <td>58.8</td>
    <td>-3.7</td>
    <td>+3.2</td>
    <td>-1.5</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>70.3</td>
    <td>-7.0</td>
    <td>+1.5</td>
    <td>-7.0</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>59.5</td>
    <td>-2.1</td>
    <td>+2.7</td>
    <td>-1.2</td>
  </tr>
  <tr>
    <td>std_Latin (Sardinian)</td>
    <td>56.9</td>
    <td>-2.6</td>
    <td>+5.5</td>
    <td>+1.7</td>
  </tr>
  <tr>
    <td>far_Latin (Faroese)</td>
    <td>59.7</td>
    <td>-1.0</td>
    <td>+5.5</td>
    <td>+1.2</td>
  </tr>
  <tr>
    <td>lij_Latin (Ligurian)</td>
    <td>56.9</td>
    <td>-1.1</td>
    <td>+6.4</td>
    <td>+4.1</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>74.0</th>
    <th>-8.6</th>
    <th>-0.9</th>
    <th>-10.1</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>58.4</th>
    <th>-2.1</th>
    <th>+4.5</th>
    <th>+1.3</th>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>87.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzr_Latin (Northern Uzbek)</td>
    <td>56.3</td>
    <td>-2.2</td>
    <td>-0.6</td>
    <td>-2.3</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>51.2</td>
    <td>+1.2</td>
    <td>+2.2</td>
    <td>+2.9</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>70.5</td>
    <td>-9.8</td>
    <td>-1.5</td>
    <td>-11.0</td>
  </tr>
  <tr>
    <td>cfr_Latin (Crimean Tatar)</td>
    <td>64.8</td>
    <td>-3.8</td>
    <td>+0.5</td>
    <td>-3.2</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>60.7</th>
    <th>-3.7</th>
    <th>+0.1</th>
    <th>-3.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>60.7</th>
    <th>-3.7</th>
    <th>+0.1</th>
    <th>-3.4</th>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>88.3</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>77.2</td>
    <td>-2.9</td>
    <td>+3.2</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>bba_Deva (Bhojpuri)</td>
    <td>75.2</td>
    <td>-1.6</td>
    <td>+1.6</td>
    <td>+0.5</td>
  </tr>
  <tr>
    <td>mag_Deva (Magadhi)</td>
    <td>78.9</td>
    <td>-0.2</td>
    <td>+3.0</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>mai_Deva (Mahihi)</td>
    <td>74.6</td>
    <td>-0.6</td>
    <td>+4.6</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>76.5</th>
    <th>-1.3</th>
    <th>+3.1</th>
    <th>+1.7</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>76.7</th>
    <th>-1.3</th>
    <th>+3.1</th>
    <th>+1.7</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>65.9</th>
    <th>-3.5</th>
    <th>+0.1</th>
    <th>-3.3</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>56.6</th>
    <th>+0.3</th>
    <th>+2.9</th>
    <th>+3.3</th>
  </tr>
</table>

Table 28: D→M COMET scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each D→M approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.
<table>
  <tr>
    <th>Dictionary</th>
    <th>Medium</th>
    <th>Length (words)</th>
    <th>→ English</th>
    <th>→ Related Language</th>
    <th>Inflections</th>
    <th>Part-of-Speech</th>
  </tr>
  <tr>
    <td>Fijian<br>Ronald Gatty’s Fijian-English Dictionary (Gatty, 2009)<br>Geocities Fijian-English Dictionary<sup>21</sup></td>
    <td>PDF<br>table</td>
    <td>~ 7585<br>~ 867</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Ilokano<br>The Online Ilokano Dictionary Project<sup>26</sup><br>The Pinoy Dictionary<sup>27</sup></td>
    <td>mySQL database<br>web app</td>
    <td>~ 2060<br>unknown</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Samoan<br>University of Hawai’i at Manoa (Kobayashi et al., 2016)<br>New South Wales Department of Education and Training<sup>28</sup></td>
    <td>web app<br>PDF</td>
    <td>13703<br>~ 3160</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Maori<br>Te Aka (Moorfield, 2024)</td>
    <td>web app</td>
    <td>unknown</td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Turkmen<br>Peace Corps Turkmenistan (Garrett et al., 1996)<br>Türkmençe-iilişce sözlük<sup>29</sup></td>
    <td>PDF<br>web app</td>
    <td>~ 10000<br>~ 7179</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Crimean Tatar<br>Kırım Tatarca’dan Türkçe’ye Sözlük<sup>30</sup></td>
    <td>PDF</td>
    <td>~ 1440</td>
    <td></td>
    <td></td>
    <td></td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Tunisian Arabic<br>Derja Ninja<sup>31</sup><br>Peace Corps (Abdelkader et al., 1977)</td>
    <td>web app<br>PDF</td>
    <td>19,231<br>~ 6000</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Moroccan Arabic<br>Traductor Darja (El Ouamari, 2024)<br>learnmoroccan<sup>32</sup><br>Online English-Moroccan Arabic (Darjia) Dictionary<sup>33</sup></td>
    <td>web app<br>web app<br>web app</td>
    <td>unknown<br>unknown<br>unknown</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Egyptian Arabic<br>Lisan Masry (Green, 2020)</td>
    <td>web app</td>
    <td>unknown</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Sicilian<br>Michael San Filippo’s English-Sicilian Dictionary<sup>34</sup><br>Dieli (Sicilian, English) (Dieli, 2011)</td>
    <td>table<br>table</td>
    <td>678<br>~ 19152</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Friulian<br>Agenzie regionali pe longhe furfane<sup>35</sup></td>
    <td>web app</td>
    <td>unknown</td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td></td>
  </tr>
  <tr>
    <td>Ligurian<br>ligu.it<sup>36</sup></td>
    <td>web app</td>
    <td>unknown</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td></td>
  </tr>
  <tr>
    <td>Bhojpuri<br>(Bafna et al., 2024a)</td>
    <td>JSON</td>
    <td>21983</td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td></td>
  </tr>
  <tr>
    <td>Magahi<br>(Bafna et al., 2024a)</td>
    <td>JSON</td>
    <td>30784</td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td></td>
  </tr>
  <tr>
    <td>Maithili<br>(Bafna et al., 2024a)</td>
    <td>JSON</td>
    <td>12069</td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td></td>
  </tr>
</table>

Table 29: Lexicons specific to a language
<table>
  <tr>
    <th>Dictionary Source</th>
    <th>Total Words</th>
    <th>Functional Words</th>
    <th>Content Words</th>
    <th>Coverage (%)</th>
  </tr>
  <tr>
    <th colspan="5">French - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>238677</td>
    <td>643</td>
    <td>238034</td>
    <td>89.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>234647</td>
    <td>370</td>
    <td>234277</td>
    <td>38.2</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>217</td>
    <td>23</td>
    <td>194</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6636</td>
    <td>538</td>
    <td>6098</td>
    <td>84.1</td>
  </tr>
  <tr>
    <th colspan="5">Spanish - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>209274</td>
    <td>538</td>
    <td>208736</td>
    <td>90.7</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>205057</td>
    <td>349</td>
    <td>204708</td>
    <td>38.5</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>242</td>
    <td>29</td>
    <td>213</td>
    <td>1.3</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6880</td>
    <td>413</td>
    <td>6467</td>
    <td>85.1</td>
  </tr>
  <tr>
    <th colspan="5">Portuguese - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>162212</td>
    <td>507</td>
    <td>161705</td>
    <td>92.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>157815</td>
    <td>257</td>
    <td>157558</td>
    <td>35.3</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>277</td>
    <td>30</td>
    <td>247</td>
    <td>1.6</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6846</td>
    <td>422</td>
    <td>6424</td>
    <td>88.3</td>
  </tr>
  <tr>
    <th colspan="5">Catalan - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>75100</td>
    <td>441</td>
    <td>74659</td>
    <td>91.0</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>70435</td>
    <td>230</td>
    <td>70205</td>
    <td>31.3</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>253</td>
    <td>23</td>
    <td>230</td>
    <td>1.5</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6851</td>
    <td>398</td>
    <td>6453</td>
    <td>87.8</td>
  </tr>
  <tr>
    <th colspan="5">Romanian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>65578</td>
    <td>529</td>
    <td>65049</td>
    <td>89.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>59734</td>
    <td>250</td>
    <td>59484</td>
    <td>21.7</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>241</td>
    <td>22</td>
    <td>219</td>
    <td>0.9</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>7493</td>
    <td>505</td>
    <td>6988</td>
    <td>86.8</td>
  </tr>
  <tr>
    <th colspan="5">Standard Malay - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>43854</td>
    <td>354</td>
    <td>43500</td>
    <td>95.8</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>40501</td>
    <td>250</td>
    <td>40251</td>
    <td>44.1</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>5994</td>
    <td>271</td>
    <td>5723</td>
    <td>92.5</td>
  </tr>
  <tr>
    <th colspan="5">Galician - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>39031</td>
    <td>414</td>
    <td>38617</td>
    <td>91.2</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>33989</td>
    <td>88</td>
    <td>33901</td>
    <td>26.8</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>290</td>
    <td>24</td>
    <td>266</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6883</td>
    <td>489</td>
    <td>6394</td>
    <td>88.5</td>
  </tr>
  <tr>
    <th colspan="5">Magahi - Hindi</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>33143</td>
    <td>3274</td>
    <td>29869</td>
    <td>90.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>10</td>
    <td>0</td>
    <td>10</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>(Bafna et al., 2024a)</td>
    <td>30784</td>
    <td>3171</td>
    <td>27613</td>
    <td>51.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>4966</td>
    <td>427</td>
    <td>4539</td>
    <td>82.6</td>
  </tr>
  <tr>
    <th colspan="5">North Azerbaijani - Turkish</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>26518</td>
    <td>324</td>
    <td>26194</td>
    <td>93.5</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>19772</td>
    <td>95</td>
    <td>19677</td>
    <td>17.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8134</td>
    <td>331</td>
    <td>7803</td>
    <td>92.0</td>
  </tr>
  <tr>
    <th colspan="5">Asturian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>25267</td>
    <td>554</td>
    <td>24713</td>
    <td>91.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>19343</td>
    <td>94</td>
    <td>19249</td>
    <td>17.1</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>222</td>
    <td>27</td>
    <td>195</td>
    <td>1.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>7140</td>
    <td>543</td>
    <td>6597</td>
    <td>89.9</td>
  </tr>
  <tr>
    <th colspan="5">Bhojpuri - Hindi</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>24453</td>
    <td>1262</td>
    <td>23191</td>
    <td>87.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>38</td>
    <td>0</td>
    <td>38</td>
    <td>0.1</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>235</td>
    <td>26</td>
    <td>209</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>(Bafna et al., 2024a)</td>
    <td>21983</td>
    <td>1088</td>
    <td>20895</td>
    <td>50.3</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>5044</td>
    <td>515</td>
    <td>4529</td>
    <td>78.2</td>
  </tr>
</table>

Table 30: Part 1 —Total, functional, and content word counts across all lexicons. We also report the word type coverage, a measure of how extensively the lexicon documents a LRL. To calculate coverage percentage, we compared the CRL lexicon against the vocabulary size of the FloRes dev set.
<table>
  <tr>
    <th>Dictionary Source</th>
    <th>Total Words</th>
    <th>Functional Words</th>
    <th>Content Words</th>
    <th>Coverage (%)</th>
  </tr>
  <tr>
    <th colspan="5">Sicilian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>23755</td>
    <td>575</td>
    <td>23180</td>
    <td>90.2</td>
  </tr>
  <tr>
    <td>Dieli (Dieli, 2011)</td>
    <td>10380</td>
    <td>143</td>
    <td>10237</td>
    <td>11.8</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>17080</td>
    <td>206</td>
    <td>16874</td>
    <td>15.9</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>219</td>
    <td>22</td>
    <td>197</td>
    <td>1.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>7213</td>
    <td>528</td>
    <td>6685</td>
    <td>88.8</td>
  </tr>
  <tr>
    <th colspan="5">Northern Uzbek - Turkish</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>23307</td>
    <td>378</td>
    <td>22929</td>
    <td>94.5</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>16288</td>
    <td>116</td>
    <td>16172</td>
    <td>14.3</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8198</td>
    <td>350</td>
    <td>7848</td>
    <td>93.7</td>
  </tr>
  <tr>
    <th colspan="5">Occitan (post 1500) - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>21464</td>
    <td>401</td>
    <td>21063</td>
    <td>88.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>15708</td>
    <td>76</td>
    <td>15632</td>
    <td>16.5</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>291</td>
    <td>34</td>
    <td>257</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6877</td>
    <td>409</td>
    <td>6468</td>
    <td>86.5</td>
  </tr>
  <tr>
    <th colspan="5">Turkmen - Turkish</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>15425</td>
    <td>323</td>
    <td>15102</td>
    <td>94.4</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>8520</td>
    <td>64</td>
    <td>8456</td>
    <td>12.7</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>259</td>
    <td>21</td>
    <td>238</td>
    <td>1.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>7935</td>
    <td>339</td>
    <td>7596</td>
    <td>93.9</td>
  </tr>
  <tr>
    <th colspan="5">Maithili - Hindi</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>15154</td>
    <td>676</td>
    <td>14478</td>
    <td>89.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>16</td>
    <td>0</td>
    <td>16</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>(Bafna et al., 2024a)</td>
    <td>12069</td>
    <td>471</td>
    <td>11598</td>
    <td>39.7</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>5211</td>
    <td>467</td>
    <td>4744</td>
    <td>83.8</td>
  </tr>
  <tr>
    <th colspan="5">Venetian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>14531</td>
    <td>396</td>
    <td>14135</td>
    <td>89.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>8317</td>
    <td>109</td>
    <td>8208</td>
    <td>6.8</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6688</td>
    <td>355</td>
    <td>6333</td>
    <td>88.8</td>
  </tr>
  <tr>
    <th colspan="5">Lombard - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>13603</td>
    <td>432</td>
    <td>13171</td>
    <td>81.2</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>6565</td>
    <td>25</td>
    <td>6540</td>
    <td>1.3</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>336</td>
    <td>44</td>
    <td>292</td>
    <td>0.8</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6875</td>
    <td>380</td>
    <td>6495</td>
    <td>81.0</td>
  </tr>
  <tr>
    <th colspan="5">Ligurian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>12591</td>
    <td>363</td>
    <td>12228</td>
    <td>86.7</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>6191</td>
    <td>21</td>
    <td>6170</td>
    <td>6.3</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>216</td>
    <td>24</td>
    <td>192</td>
    <td>0.7</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6703</td>
    <td>356</td>
    <td>6347</td>
    <td>85.9</td>
  </tr>
  <tr>
    <th colspan="5">Crimean Tatar - Turkish</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>11213</td>
    <td>325</td>
    <td>10888</td>
    <td>93.6</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>3470</td>
    <td>51</td>
    <td>3419</td>
    <td>6.3</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>242</td>
    <td>18</td>
    <td>224</td>
    <td>1.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8260</td>
    <td>305</td>
    <td>7955</td>
    <td>93.3</td>
  </tr>
  <tr>
    <th colspan="5">Sardinian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>10720</td>
    <td>352</td>
    <td>10368</td>
    <td>87.0</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>4850</td>
    <td>52</td>
    <td>4798</td>
    <td>7.6</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6356</td>
    <td>342</td>
    <td>6014</td>
    <td>86.0</td>
  </tr>
  <tr>
    <th colspan="5">Friulian - Italian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>10596</td>
    <td>282</td>
    <td>10314</td>
    <td>87.5</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>5166</td>
    <td>75</td>
    <td>5091</td>
    <td>11.4</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>215</td>
    <td>23</td>
    <td>192</td>
    <td>1.5</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6160</td>
    <td>297</td>
    <td>5863</td>
    <td>86.5</td>
  </tr>
</table>

Table 31: Part 2 —Total, functional, and content word counts across all lexicons. We also report the word type coverage, a measure of how extensively the lexicon documents a LRL. To calculate coverage percentage, we compared the CRL lexicon against the vocabulary size of the FloRes dev set.
<table>
  <tr>
    <th>Dictionary Source</th>
    <th>Total Words</th>
    <th>Functional Words</th>
    <th>Content Words</th>
    <th>Coverage (%)</th>
  </tr>
  <tr>
    <th colspan="5">Awadhi - Hindi</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>10470</td>
    <td>1356</td>
    <td>9114</td>
    <td>31.8</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>12</td>
    <td>0</td>
    <td>12</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>(Bafna et al., 2024a)</td>
    <td>10462</td>
    <td>1355</td>
    <td>9107</td>
    <td>31.8</td>
  </tr>
  <tr>
    <th colspan="5">Tagalog - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>10176</td>
    <td>284</td>
    <td>9892</td>
    <td>79.5</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>4916</td>
    <td>76</td>
    <td>4840</td>
    <td>8.9</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>211</td>
    <td>20</td>
    <td>191</td>
    <td>1.5</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>5801</td>
    <td>253</td>
    <td>5548</td>
    <td>78.1</td>
  </tr>
  <tr>
    <th colspan="5">Egyptian Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>10077</td>
    <td>284</td>
    <td>9793</td>
    <td>93.4</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>2244</td>
    <td>26</td>
    <td>2218</td>
    <td>4.5</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>299</td>
    <td>0</td>
    <td>299</td>
    <td>1.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8203</td>
    <td>317</td>
    <td>7886</td>
    <td>93.2</td>
  </tr>
  <tr>
    <th colspan="5">Maori - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>9908</td>
    <td>208</td>
    <td>9700</td>
    <td>78.0</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>6700</td>
    <td>86</td>
    <td>6614</td>
    <td>16.5</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>306</td>
    <td>35</td>
    <td>271</td>
    <td>3.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>3851</td>
    <td>172</td>
    <td>3679</td>
    <td>75.1</td>
  </tr>
  <tr>
    <th colspan="5">Moroccan Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>9199</td>
    <td>408</td>
    <td>8791</td>
    <td>93.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>464</td>
    <td>2</td>
    <td>462</td>
    <td>0.7</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>308</td>
    <td>0</td>
    <td>308</td>
    <td>1.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8590</td>
    <td>445</td>
    <td>8145</td>
    <td>93.0</td>
  </tr>
  <tr>
    <th colspan="5">Najdi Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>9147</td>
    <td>117</td>
    <td>9030</td>
    <td>98.5</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>1</td>
    <td>0</td>
    <td>1</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>9146</td>
    <td>115</td>
    <td>9031</td>
    <td>98.5</td>
  </tr>
  <tr>
    <th colspan="5">Ta’izzi-Adeni Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8915</td>
    <td>232</td>
    <td>8683</td>
    <td>96.1</td>
  </tr>
  <tr>
    <th colspan="5">Tunisian Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>8813</td>
    <td>325</td>
    <td>8488</td>
    <td>94.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>237</td>
    <td>20</td>
    <td>217</td>
    <td>0.2</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>205</td>
    <td>0</td>
    <td>205</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8579</td>
    <td>343</td>
    <td>8236</td>
    <td>94.1</td>
  </tr>
  <tr>
    <th colspan="5">Mesopotamian Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>8705</td>
    <td>275</td>
    <td>8430</td>
    <td>95.1</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>6</td>
    <td>0</td>
    <td>6</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8702</td>
    <td>282</td>
    <td>8420</td>
    <td>95.1</td>
  </tr>
  <tr>
    <th colspan="5">South Levantine Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>8480</td>
    <td>336</td>
    <td>8144</td>
    <td>93.8</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>300</td>
    <td>24</td>
    <td>276</td>
    <td>1.0</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>242</td>
    <td>0</td>
    <td>242</td>
    <td>1.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>8196</td>
    <td>343</td>
    <td>7853</td>
    <td>93.8</td>
  </tr>
  <tr>
    <th colspan="5">Javanese - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>8439</td>
    <td>384</td>
    <td>8055</td>
    <td>92.3</td>
  </tr>
  <tr>
    <td>PanLex (Kamholz et al., 2014)</td>
    <td>2382</td>
    <td>79</td>
    <td>2303</td>
    <td>5.5</td>
  </tr>
  <tr>
    <td>Swadesh (Swadesh, 2015)</td>
    <td>219</td>
    <td>24</td>
    <td>195</td>
    <td>0.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (Dyer et al., 2013)</td>
    <td>6245</td>
    <td>319</td>
    <td>5926</td>
    <td>92.0</td>
  </tr>
</table>

Table 32: Part 3 —Total, functional, and content word counts across all lexicons. We also report the word type coverage, a measure of how extensively the lexicon documents a LRL. To calculate coverage percentage, we compared the CRL lexicon against the vocabulary size of the FloRes dev set.
<table>
  <tr>
    <th>Dictionary Source</th>
    <th>Total Words</th>
    <th>Functional Words</th>
    <th>Content Words</th>
    <th>Coverage (%)</th>
  </tr>
  <tr>
    <th colspan="5">North Levantine Arabic - Standard Arabic</th>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>8411</td>
    <td>416</td>
    <td>7995</td>
    <td>93.6</td>
  </tr>
  <tr>
    <th colspan="5">Italian - Italian</th>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>8118</td>
    <td>226</td>
    <td>7892</td>
    <td>98.3</td>
  </tr>
  <tr>
    <th colspan="5">Plateau Malagasy - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>8106</td>
    <td>433</td>
    <td>7673</td>
    <td>85.9</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>2420</td>
    <td>85</td>
    <td>2335</td>
    <td>3.5</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>5908</td>
    <td>383</td>
    <td>5525</td>
    <td>85.7</td>
  </tr>
  <tr>
    <th colspan="5">Sundanese - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>7640</td>
    <td>418</td>
    <td>7222</td>
    <td>91.7</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>1245</td>
    <td>51</td>
    <td>1194</td>
    <td>4.3</td>
  </tr>
  <tr>
    <td>Swadesh (<i>Swadesh, 2015</i>)</td>
    <td>301</td>
    <td>40</td>
    <td>261</td>
    <td>1.7</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>6633</td>
    <td>415</td>
    <td>6218</td>
    <td>91.4</td>
  </tr>
  <tr>
    <th colspan="5">Cebuano - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>6969</td>
    <td>263</td>
    <td>6706</td>
    <td>77.6</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>1441</td>
    <td>37</td>
    <td>1404</td>
    <td>3.3</td>
  </tr>
  <tr>
    <td>Swadesh (<i>Swadesh, 2015</i>)</td>
    <td>297</td>
    <td>27</td>
    <td>270</td>
    <td>1.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>5582</td>
    <td>238</td>
    <td>5344</td>
    <td>77.1</td>
  </tr>
  <tr>
    <th colspan="5">Morisyen - Haitian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>6513</td>
    <td>754</td>
    <td>5759</td>
    <td>36.0</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>90</td>
    <td>2</td>
    <td>88</td>
    <td>0.8</td>
  </tr>
  <tr>
    <td>Swadesh (<i>Swadesh, 2015</i>)</td>
    <td>218</td>
    <td>28</td>
    <td>190</td>
    <td>3.3</td>
  </tr>
  <tr>
    <td>Statistically Aligned Bible Data (<i>McCarthy et al., 2020</i>)</td>
    <td>6384</td>
    <td>766</td>
    <td>5618</td>
    <td>35.5</td>
  </tr>
  <tr>
    <th colspan="5">Pangasinan - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>6473</td>
    <td>416</td>
    <td>6057</td>
    <td>86.6</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>546</td>
    <td>49</td>
    <td>497</td>
    <td>2.2</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>6067</td>
    <td>402</td>
    <td>5665</td>
    <td>86.4</td>
  </tr>
  <tr>
    <th colspan="5">Iloko - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>6329</td>
    <td>238</td>
    <td>6091</td>
    <td>76.6</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>743</td>
    <td>30</td>
    <td>713</td>
    <td>2.9</td>
  </tr>
  <tr>
    <td>Swadesh (<i>Swadesh, 2015</i>)</td>
    <td>219</td>
    <td>23</td>
    <td>196</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>5696</td>
    <td>240</td>
    <td>5456</td>
    <td>76.3</td>
  </tr>
  <tr>
    <th colspan="5">Seselwa Creole French - Haitian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>6051</td>
    <td>635</td>
    <td>5416</td>
    <td>29.1</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>103</td>
    <td>2</td>
    <td>101</td>
    <td>0.7</td>
  </tr>
  <tr>
    <td>Statistically Aligned Bible Data (<i>McCarthy et al., 2020</i>)</td>
    <td>5983</td>
    <td>643</td>
    <td>5340</td>
    <td>28.9</td>
  </tr>
  <tr>
    <th colspan="5">Chhattisgarhi - Hindi</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>5293</td>
    <td>476</td>
    <td>4817</td>
    <td>80.8</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>11</td>
    <td>0</td>
    <td>11</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>5282</td>
    <td>467</td>
    <td>4815</td>
    <td>80.8</td>
  </tr>
  <tr>
    <th colspan="5">Saint Lucian Creole French - Haitian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>4867</td>
    <td>464</td>
    <td>4403</td>
    <td>50.6</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>85</td>
    <td>3</td>
    <td>82</td>
    <td>1.0</td>
  </tr>
  <tr>
    <td>Statistically Aligned Bible Data (<i>McCarthy et al., 2020</i>)</td>
    <td>4796</td>
    <td>462</td>
    <td>4334</td>
    <td>50.6</td>
  </tr>
  <tr>
    <th colspan="5">Samoan - Indonesian</th>
  </tr>
  <tr>
    <td>Combined</td>
    <td>4471</td>
    <td>155</td>
    <td>4316</td>
    <td>72.4</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>872</td>
    <td>39</td>
    <td>833</td>
    <td>4.1</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>3785</td>
    <td>138</td>
    <td>3647</td>
    <td>71.8</td>
  </tr>
</table>

Table 33: Part 4 —Total, functional, and content word counts across all lexicons. We also report the word type coverage, a measure of how extensively the lexicon documents a LRL. To calculate coverage percentage, we compared the CRL lexicon against the vocabulary size of the FloRes dev set.
<table>
  <tr>
    <th>Dictionary Source</th>
    <th>Total Words</th>
    <th>Functional Words</th>
    <th>Content Words</th>
    <th>Coverage (%)</th>
  </tr>
  <tr>
    <th colspan="5">Fijian - Indonesian</th>
  </tr>
  <tr>
    <td><b>Combined</b></td>
    <td>4280</td>
    <td>205</td>
    <td>4075</td>
    <td>77.5</td>
  </tr>
  <tr>
    <td>PanLex (<i>Kamholz et al., 2014</i>)</td>
    <td>630</td>
    <td>46</td>
    <td>584</td>
    <td>4.1</td>
  </tr>
  <tr>
    <td>Swadesh (<i>Swadesh, 2015</i>)</td>
    <td>222</td>
    <td>33</td>
    <td>189</td>
    <td>1.3</td>
  </tr>
  <tr>
    <td>Statistically Aligned FloRes Data (<i>Dyer et al., 2013</i>)</td>
    <td>3705</td>
    <td>150</td>
    <td>3555</td>
    <td>77.0</td>
  </tr>
</table>

Table 34: Part 5 —Total, functional, and content word counts across all lexicons. We also report the word type coverage, a measure of how extensively the lexicon documents a LRL. To calculate coverage percentage, we compared the CRL lexicon against the vocabulary size of the FloRes dev set.

<table>
  <tr>
    <th rowspan="2">feature</th>
    <th colspan="2">Aya</th>
    <th colspan="2">M2M</th>
  </tr>
  <tr>
    <th>lrc</th>
    <th>fi</th>
    <th>lrc</th>
    <th>fi</th>
  </tr>
  <tr>
    <td>hrl_wiki_count</td>
    <td>0.0</td>
    <td>0.07</td>
    <td>0.0</td>
    <td>0.02</td>
  </tr>
  <tr>
    <td>hrl_wiki_count</td>
    <td>0.0</td>
    <td>0.11</td>
    <td>0.0</td>
    <td>0.2</td>
  </tr>
  <tr>
    <td>chr_f_sim</td>
    <td>-0.03</td>
    <td>0.12</td>
    <td>-0.07</td>
    <td>0.26</td>
  </tr>
  <tr>
    <td>baseline_performance</td>
    <td>-0.3</td>
    <td>0.48</td>
    <td>-0.12</td>
    <td>0.45</td>
  </tr>
  <tr>
    <td>in_pretraining</td>
    <td>-1.72</td>
    <td>0.0</td>
    <td>-1.67</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>token_fertility_ratio</td>
    <td>-6.3</td>
    <td>0.21</td>
    <td>-3.66</td>
    <td>0.07</td>
  </tr>
</table>

Table 35: Random forest feature importance values and linear regression coefficients for different language features. lrc = linear regression coefficient; fi = feature importance value.

<table>
  <tr>
    <th>HRL</th>
    <th>Natural %</th>
    <th>Mean % over all CRLs</th>
  </tr>
  <tr>
    <td>ind</td>
    <td>24.5</td>
    <td>43.1</td>
  </tr>
  <tr>
    <td>arb</td>
    <td>27.7</td>
    <td>16.3</td>
  </tr>
  <tr>
    <td>ita</td>
    <td>40.3</td>
    <td>38.5</td>
  </tr>
  <tr>
    <td>tur</td>
    <td>16.8</td>
    <td>15.0</td>
  </tr>
  <tr>
    <td>hin</td>
    <td>39.7</td>
    <td>33.4</td>
  </tr>
</table>

Table 36: Comparing the natural percentage of functional words in an HRL against the mean % of words identified as function words over all its CRLs for D→M.
<table>
  <tr>
    <th>Language</th>
    <th>% la -cont</th>
    <th>% la -func</th>
    <th>% la -all</th>
  </tr>
  <tr><th colspan="4">Austronesian Language Family</th></tr>
  <tr><td>jar_Latin (Javanese)</td><td>42.9</td><td>32.2</td><td>75.1</td></tr>
  <tr><td>sas_Latin (Sasak)</td><td>40.2</td><td>17.7</td><td>71.9</td></tr>
  <tr><td>snu_Latin (Samoan)</td><td>33.2</td><td>52.2</td><td>85.4</td></tr>
  <tr><td>mri_Latin (Maori)</td><td>41.2</td><td>46.8</td><td>87.9</td></tr>
  <tr><td>cel_Latin (Cebuano)</td><td>26.8</td><td>29.7</td><td>75.7</td></tr>
  <tr><td>zsm_Latin (Standard Malay)</td><td>61.8</td><td>27.5</td><td>89.3</td></tr>
  <tr><td>tgl_Latin (Tagalog)</td><td>31.1</td><td>46.5</td><td>77.6</td></tr>
  <tr><td>ik_Latin (Ilokano)</td><td>22.6</td><td>31.0</td><td>73.6</td></tr>
  <tr><td>fj_Latin (Fijian)</td><td>35.5</td><td>50.4</td><td>85.9</td></tr>
  <tr><td>plh_Latin (Borneo Malay)</td><td>56.4</td><td>37.9</td><td>75.7</td></tr>
  <tr><td>pmg_Latin (Pangasinan)</td><td>22.7</td><td>47.3</td><td>70.0</td></tr>
  <tr><th>Average</th><th>35.8</th><th>43.1</th><th>78.9</th></tr>
  <tr><th colspan="4">Arabic Language Family</th></tr>
  <tr><td>acm_Arab (Mesopotamian Arabic)</td><td>40.2</td><td>17.9</td><td>58.1</td></tr>
  <tr><td>ajb_Arab (Jebel Arabic)</td><td>39.4</td><td>19.0</td><td>58.4</td></tr>
  <tr><td>ach_Arab (Tunisian Arabic)</td><td>43.9</td><td>13.4</td><td>57.3</td></tr>
  <tr><td>ajg_Arab (South Levantine Arabic)</td><td>42.8</td><td>17.4</td><td>60.2</td></tr>
  <tr><td>ajh_Arab (North Levantine Arabic)</td><td>38.4</td><td>18.5</td><td>56.9</td></tr>
  <tr><td>anj_Arab (Najdi Arabic)</td><td>39.6</td><td>20.4</td><td>60.0</td></tr>
  <tr><td>any_Arab (Moroccan Arabic)</td><td>42.2</td><td>15.9</td><td>58.4</td></tr>
  <tr><td>anm_Arab (Egyptian Arabic)</td><td>54.9</td><td>7.6</td><td>62.6</td></tr>
  <tr><th>Average</th><th>42.7</th><th>16.3</th><th>59.0</th></tr>
  <tr><th colspan="4">Romance Language Family</th></tr>
  <tr><td>spa_Latin (Spanish)</td><td>46.5</td><td>40.4</td><td>86.9</td></tr>
  <tr><td>fra_Latin (French)</td><td>51.9</td><td>35.8</td><td>85.6</td></tr>
  <tr><td>por_Latin (Portuguese)</td><td>52.2</td><td>32.1</td><td>84.3</td></tr>
  <tr><td>ron_Latin (Romanian)</td><td>42.4</td><td>33.4</td><td>75.8</td></tr>
  <tr><td>glg_Latin (Galician)</td><td>47.1</td><td>35.5</td><td>82.5</td></tr>
  <tr><td>cat_Latin (Catalan)</td><td>43.2</td><td>39.3</td><td>82.5</td></tr>
  <tr><td>oc_Latin (Occitan)</td><td>36.6</td><td>48.5</td><td>76.9</td></tr>
  <tr><td>anm_Latin (Lombard)</td><td>37.9</td><td>37.9</td><td>75.8</td></tr>
  <tr><td>veg_Latin (Venetian)</td><td>24.6</td><td>45.7</td><td>70.4</td></tr>
  <tr><td>scn_Latin (Sicilian)</td><td>35.1</td><td>36.5</td><td>71.6</td></tr>
  <tr><td>sen_Latin (Sardinian)</td><td>39.4</td><td>34.6</td><td>74.0</td></tr>
  <tr><td>srl_Latin (Sardinian)</td><td>28.5</td><td>48.8</td><td>77.3</td></tr>
  <tr><td>frg_Latin (Friulian)</td><td>41.2</td><td>28.8</td><td>80.0</td></tr>
  <tr><td>lig_Latin (Ligurian)</td><td>34.5</td><td>41.1</td><td>75.6</td></tr>
  <tr><th>Average</th><th>40.1</th><th>38.5</th><th>78.5</th></tr>
  <tr><th colspan="4">Turkic Language Family</th></tr>
  <tr><td>umz_Latin (Northern Uzbek)</td><td>49.2</td><td>13.7</td><td>62.9</td></tr>
  <tr><td>tan_Latin (Turkmen)</td><td>45.7</td><td>14.8</td><td>60.5</td></tr>
  <tr><td>anj_Latin (North Azerbaijani)</td><td>49.4</td><td>15.4</td><td>64.8</td></tr>
  <tr><td>chz_Latin (Crimean Tatar)</td><td>41.9</td><td>18.1</td><td>58.1</td></tr>
  <tr><th>Average</th><th>46.6</th><th>15.0</th><th>61.6</th></tr>
  <tr><th colspan="4">Indic Language Family</th></tr>
  <tr><td>hin_Deva (Hindi/Urdu)</td><td>34.4</td><td>42.9</td><td>77.3</td></tr>
  <tr><td>bho_Deva (Bhojpuri)</td><td>58.5</td><td>30.0</td><td>88.4</td></tr>
  <tr><td>mag_Deva (Magahi)</td><td>57.1</td><td>25.4</td><td>82.2</td></tr>
  <tr><td>mai_Deva (Marathi)</td><td>60.9</td><td>25.2</td><td>86.1</td></tr>
  <tr><th>Average</th><th>51.9</th><th>33.4</th><th>85.2</th></tr>
  <tr><th colspan="4">Creole Language Family</th></tr>
  <tr><td>adl (Saint Lucian Patois)</td><td>43.0</td><td>39.5</td><td>82.3</td></tr>
  <tr><td>mau (Mauritian)</td><td>34.9</td><td>37.9</td><td>72.8</td></tr>
  <tr><td>csv (Seychellois)</td><td>24.7</td><td>40.6</td><td>65.3</td></tr>
  <tr><th>Average</th><th>34.1</th><th>39.3</th><th>73.3</th></tr>
</table>

Table 37: Percentage of words switched out in D→M approaches for each language
<table>
  <tr>
    <th></th>
    <th>off-the-shelf</th>
    <th>-all-eng</th>
    <th>-cont-eng</th>
    <th>-func-eng</th>
  </tr>
  <tr>
    <td>ind (Indonesian)</td>
    <td>15.9</td>
    <td>10.1</td>
    <td>10.2</td>
    <td>13.4</td>
  </tr>
  <tr>
    <td>arb (Standard Arabic)</td>
    <td>29.5</td>
    <td>10.3</td>
    <td>10.2</td>
    <td>28.8</td>
  </tr>
  <tr>
    <td>ita (Italian)</td>
    <td>29.3</td>
    <td>14.4</td>
    <td>16.6</td>
    <td>22.8</td>
  </tr>
  <tr>
    <td>tur (Turkish)</td>
    <td>12.5</td>
    <td>3.7</td>
    <td>4.1</td>
    <td>7.6</td>
  </tr>
  <tr>
    <td>hin (Hindi)</td>
    <td>22.9</td>
    <td>4.9</td>
    <td>4.8</td>
    <td>19.5</td>
  </tr>
</table>

Table 38: D→M results for Aya-23 where low-resource words are swapped with English words. Experiment was performed on 300 FLORES examples.

<table>
  <tr>
    <th></th>
    <th>off-the-shelf</th>
    <th>-all-eng</th>
    <th>-cont-eng</th>
    <th>-func-eng</th>
  </tr>
  <tr>
    <td>ind (Indonesian)</td>
    <td>13.5</td>
    <td>11.1</td>
    <td>11.0</td>
    <td>11.0</td>
  </tr>
  <tr>
    <td>arb (Standard Arabic)</td>
    <td>20.9</td>
    <td>9.3</td>
    <td>9.3</td>
    <td>19.9</td>
  </tr>
  <tr>
    <td>ita (Italian)</td>
    <td>19.6</td>
    <td>14.3</td>
    <td>14.3</td>
    <td>16.8</td>
  </tr>
  <tr>
    <td>tur (Turkish)</td>
    <td>10.9</td>
    <td>4.6</td>
    <td>4.7</td>
    <td>5.3</td>
  </tr>
  <tr>
    <td>hin (Hindi)</td>
    <td>17.3</td>
    <td>5.4</td>
    <td>5.4</td>
    <td>12.9</td>
  </tr>
</table>

Table 39: D→M results for M2M where low-resource words are swapped with English words Experiment was performed on 300 FLORES examples.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cloud-cont</th>
    <th>-cloud-func</th>
    <th>-cloud-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>38.9</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>22.9</td>
    <td>+0.4</td>
    <td>+1.1</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>22.9</td>
    <td>+1.6</td>
    <td>+1.5</td>
    <td>+1.2</td>
  </tr>
  <tr>
    <td>sma_Latin (Malay)</td>
    <td>23.1</td>
    <td>+1.1</td>
    <td>+1.6</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>mri_Latin (Maori)</td>
    <td>2.3</td>
    <td>+2.4</td>
    <td>+1.4</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>22.2</td>
    <td>-1.9</td>
    <td>-7.3</td>
    <td>-9.1</td>
  </tr>
  <tr>
    <td>zun_Latin (Borneo Malay)</td>
    <td>27.7</td>
    <td>+0.2</td>
    <td>+2.2</td>
    <td>+7.7</td>
  </tr>
  <tr>
    <td>tel_Latin (Telugu)</td>
    <td>27.8</td>
    <td>-5.1</td>
    <td>-10.8</td>
    <td>-14.0</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>13.4</td>
    <td>+2.2</td>
    <td>-1.9</td>
    <td>-1.8</td>
  </tr>
  <tr>
    <td>fj_Latin (Fijian)</td>
    <td>2.3</td>
    <td>+2.4</td>
    <td>+1.4</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>pli_Latin (Plains Malagasy)</td>
    <td>2.3</td>
    <td>+2.4</td>
    <td>+1.4</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>6.4</td>
    <td>+2.5</td>
    <td>+7.7</td>
    <td>+6.5</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>14.9</td>
    <td>+0.1</td>
    <td>+0.3</td>
    <td>-1.3</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>10.8</td>
    <td>+1.4</td>
    <td>+1.1</td>
    <td>+0.9</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>29.6</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>22.4</td>
    <td>-0.1</td>
    <td>+1.5</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta izzi-Adeni Arabic)</td>
    <td>24.7</td>
    <td>-0.4</td>
    <td>+0.7</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>17.7</td>
    <td>-0.3</td>
    <td>+2.0</td>
    <td>+0.7</td>
  </tr>
  <tr>
    <td>ap_Arab (South Levantine Arabic)</td>
    <td>24.5</td>
    <td>-4.3</td>
    <td>+1.1</td>
    <td>-5.7</td>
  </tr>
  <tr>
    <td>aq_Arab (North Levantine Arabic)</td>
    <td>21.7</td>
    <td>-4.0</td>
    <td>+0.8</td>
    <td>-3.0</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>28.5</td>
    <td>-0.3</td>
    <td>+0.3</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>11.9</td>
    <td>+0.4</td>
    <td>+3.6</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>18.2</td>
    <td>-1.9</td>
    <td>+1.7</td>
    <td>-1.8</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>21.2</td>
    <td>-1.4</td>
    <td>+1.1</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.2</td>
    <td>-1.5</td>
    <td>+1.3</td>
    <td>-1.2</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>30.8</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>26.8</td>
    <td>-8.5</td>
    <td>-3.9</td>
    <td>-11.5</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>41.6</td>
    <td>-13.4</td>
    <td>+8.5</td>
    <td>-20.1</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>45.8</td>
    <td>-13.5</td>
    <td>+8.6</td>
    <td>-20.9</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>40.8</td>
    <td>-13.9</td>
    <td>-11.1</td>
    <td>-22.0</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>37.5</td>
    <td>-11.8</td>
    <td>-5.8</td>
    <td>-16.7</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>42.2</td>
    <td>-12.5</td>
    <td>+5.8</td>
    <td>-16.5</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>44.6</td>
    <td>-10.3</td>
    <td>+4.6</td>
    <td>-13.8</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>34.3</td>
    <td>-8.6</td>
    <td>-5.7</td>
    <td>-14.1</td>
  </tr>
  <tr>
    <td>lno_Latin (Lombard)</td>
    <td>9.6</td>
    <td>+9.2</td>
    <td>+12.9</td>
    <td>+11.1</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>12.2</td>
    <td>+5.2</td>
    <td>+11.4</td>
    <td>+7.9</td>
  </tr>
  <tr>
    <td>scr_Latin (Sicilian)</td>
    <td>7.3</td>
    <td>+8.7</td>
    <td>+13.1</td>
    <td>+12.1</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>5.3</td>
    <td>+10.2</td>
    <td>+17.8</td>
    <td>+18.0</td>
  </tr>
  <tr>
    <td>fur_Latin (Friulian)</td>
    <td>8.0</td>
    <td>+10.9</td>
    <td>+13.2</td>
    <td>+14.3</td>
  </tr>
  <tr>
    <td>li_Latin (Ligurian)</td>
    <td>9.8</td>
    <td>+9.9</td>
    <td>+15.9</td>
    <td>+15.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>26.6</td>
    <td>-2.7</td>
    <td>+2.0</td>
    <td>-4.3</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>9.6</td>
    <td>+9.0</td>
    <td>+14.1</td>
    <td>+13.1</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>33.3</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzn_Latin (Northern Uzbek)</td>
    <td>1.6</td>
    <td>+3.2</td>
    <td>+1.8</td>
    <td>+5.7</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>2.4</td>
    <td>+3.3</td>
    <td>+2.2</td>
    <td>+5.7</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>7.4</td>
    <td>+1.4</td>
    <td>+2.4</td>
    <td>+0.8</td>
  </tr>
  <tr>
    <td>cth_Latin (Crimean Tatar)</td>
    <td>8.5</td>
    <td>+2.4</td>
    <td>+2.9</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>5.0</td>
    <td>+2.6</td>
    <td>+2.3</td>
    <td>+3.9</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>5.0</td>
    <td>+2.6</td>
    <td>+2.3</td>
    <td>+3.9</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>33.4</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>15.2</td>
    <td>+2.6</td>
    <td>+14.3</td>
    <td>+10.9</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>10.1</td>
    <td>+3.9</td>
    <td>+7.7</td>
    <td>+8.1</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>12.2</td>
    <td>+5.3</td>
    <td>+14.0</td>
    <td>+16.8</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>7.7</td>
    <td>+8.4</td>
    <td>+16.0</td>
    <td>+16.8</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>12.6</td>
    <td>+5.1</td>
    <td>+13.0</td>
    <td>+12.2</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>12.6</td>
    <td>+5.1</td>
    <td>+13.0</td>
    <td>+12.2</td>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat_Haitian</td>
    <td>39.7</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>3.7</td>
    <td>+4.0</td>
    <td>+4.7</td>
    <td>+9.1</td>
  </tr>
  <tr>
    <td>mfe (Mauritian)</td>
    <td>5.1</td>
    <td>+2.8</td>
    <td>+2.2</td>
    <td>+3.6</td>
  </tr>
  <tr>
    <td>csf (Seychellois)</td>
    <td>11.2</td>
    <td>+2.1</td>
    <td>+4.5</td>
    <td>+3.1</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>5.9</td>
    <td>+3.7</td>
    <td>+4.6</td>
    <td>+6.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>5.9</td>
    <td>+3.7</td>
    <td>+4.6</td>
    <td>+6.1</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>16.8</td>
    <td>+1.1</td>
    <td>+3.7</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>10.9</td>
    <td>+3.7</td>
    <td>+6.2</td>
    <td>+5.6</td>
  </tr>
</table>

Table 40: M←D BLEU scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each M←D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The best score for each language in the M←D paradigm is bolded and underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cloud-cont</th>
    <th>-cloud-func</th>
    <th>-cloud-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>41.5</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>14.2</td>
    <td>+7.6</td>
    <td>+9.9</td>
    <td>+12.3</td>
  </tr>
  <tr>
    <td>sun_Latin (Sundanese)</td>
    <td>11.0</td>
    <td>+8.5</td>
    <td>+12.3</td>
    <td>+13.3</td>
  </tr>
  <tr>
    <td>sma_Latin (Malay)</td>
    <td>11.2</td>
    <td>+1.5</td>
    <td>+1.9</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>mri_Latin (Maori)</td>
    <td>4.8</td>
    <td>+2.0</td>
    <td>0.0</td>
    <td>+1.5</td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>14.0</td>
    <td>-0.3</td>
    <td>-2.1</td>
    <td>-0.7</td>
  </tr>
  <tr>
    <td>zam_Latin (Borneo Malay)</td>
    <td>8.4</td>
    <td>+3.8</td>
    <td>+8.5</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>25.6</td>
    <td>-8.8</td>
    <td>-13.0</td>
    <td>-11.5</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>7.5</td>
    <td>+2.3</td>
    <td>+2.4</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>fj_Latin (Fijian)</td>
    <td>3.3</td>
    <td>+1.8</td>
    <td>+0.9</td>
    <td>+2.2</td>
  </tr>
  <tr>
    <td>pli_Latin (Plains Malagasy)</td>
    <td>3.3</td>
    <td>+2.5</td>
    <td>+2.7</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>7.3</td>
    <td>+3.8</td>
    <td>+6.6</td>
    <td>+6.4</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>11.9</td>
    <td>+1.6</td>
    <td>+1.9</td>
    <td>+2.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>7.6</td>
    <td>+3.3</td>
    <td>+3.8</td>
    <td>+5.1</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>38.1</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>31.6</td>
    <td>-1.0</td>
    <td>+0.7</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta izzi-Adeni Arabic)</td>
    <td>33.7</td>
    <td>-1.0</td>
    <td>+0.2</td>
    <td>-0.5</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>26.7</td>
    <td>-2.2</td>
    <td>+0.4</td>
    <td>-1.9</td>
  </tr>
  <tr>
    <td>ap_Arab (South Levantine Arabic)</td>
    <td>31.4</td>
    <td>-9.2</td>
    <td>+4.7</td>
    <td>-11.7</td>
  </tr>
  <tr>
    <td>aqe_Arab (North Levantine Arabic)</td>
    <td>33.2</td>
    <td>-4.2</td>
    <td>+1.8</td>
    <td>-2.5</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>37.1</td>
    <td>-0.2</td>
    <td>+0.0</td>
    <td>+0.0</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>23.5</td>
    <td>-4.3</td>
    <td>+0.8</td>
    <td>-4.5</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>28.9</td>
    <td>-4.5</td>
    <td>+0.7</td>
    <td>-5.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>31.5</td>
    <td>-3.6</td>
    <td>+0.8</td>
    <td>-4.0</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>23.5</td>
    <td>-4.3</td>
    <td>+0.8</td>
    <td>-4.5</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>32.6</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>29.0</td>
    <td>-6.7</td>
    <td>-1.6</td>
    <td>-9.3</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>44.0</td>
    <td>-14.4</td>
    <td>+2.7</td>
    <td>-18.6</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>46.9</td>
    <td>-14.1</td>
    <td>+9.1</td>
    <td>-18.0</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>42.0</td>
    <td>-14.7</td>
    <td>-11.7</td>
    <td>-20.3</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>35.5</td>
    <td>-10.4</td>
    <td>+4.0</td>
    <td>-12.4</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>35.8</td>
    <td>-10.0</td>
    <td>+5.5</td>
    <td>-14.8</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>35.1</td>
    <td>-5.4</td>
    <td>+0.8</td>
    <td>-5.8</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>31.8</td>
    <td>-6.9</td>
    <td>+4.6</td>
    <td>-10.3</td>
  </tr>
  <tr>
    <td>iml_Latin (Lombard)</td>
    <td>17.4</td>
    <td>+3.9</td>
    <td>+8.5</td>
    <td>+5.1</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>17.7</td>
    <td>+0.9</td>
    <td>+0.9</td>
    <td>+0.2</td>
  </tr>
  <tr>
    <td>scr_Latin (Sicilian)</td>
    <td>16.7</td>
    <td>+4.5</td>
    <td>+7.8</td>
    <td>+5.3</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>16.4</td>
    <td>+4.6</td>
    <td>+10.0</td>
    <td>+8.2</td>
  </tr>
  <tr>
    <td>frc_Latin (Corsican)</td>
    <td>17.7</td>
    <td>+6.9</td>
    <td>+9.1</td>
    <td>+8.6</td>
  </tr>
  <tr>
    <td>li_Latin (Ligurian)</td>
    <td>17.4</td>
    <td>+6.4</td>
    <td>+13.2</td>
    <td>+10.1</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>29.5</td>
    <td>-4.0</td>
    <td>+6.5</td>
    <td>-4.9</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>17.1</td>
    <td>+5.3</td>
    <td>+9.8</td>
    <td>+7.4</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>31.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>uzn_Latin (Northern Uzbek)</td>
    <td>3.8</td>
    <td>+4.1</td>
    <td>+3.9</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>6.1</td>
    <td>+3.8</td>
    <td>+3.9</td>
    <td>+4.0</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>9.8</td>
    <td>-1.4</td>
    <td>+1.3</td>
    <td>-1.6</td>
  </tr>
  <tr>
    <td>cth_Latin (Crimean Tatar)</td>
    <td>11.9</td>
    <td>+0.8</td>
    <td>+3.3</td>
    <td>+1.0</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>7.9</td>
    <td>+1.8</td>
    <td>+3.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>7.9</td>
    <td>+1.8</td>
    <td>+3.1</td>
    <td>+2.1</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>34.8</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>hne_Deva (Chhattisgarhi)</td>
    <td>23.3</td>
    <td>-0.7</td>
    <td>+6.3</td>
    <td>+2.4</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>16.2</td>
    <td>+1.0</td>
    <td>+2.2</td>
    <td>+3.2</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>15.9</td>
    <td>+1.9</td>
    <td>+5.8</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>17.2</td>
    <td>+4.3</td>
    <td>+7.6</td>
    <td>+7.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>20.4</td>
    <td>+1.6</td>
    <td>+6.0</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>20.4</td>
    <td>+1.6</td>
    <td>+6.0</td>
    <td>+4.6</td>
  </tr>
  <tr>
    <th colspan="5">Creole Language Family</th>
  </tr>
  <tr>
    <td>hat_Haitian</td>
    <td>8.9</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>acf (Saint Lucian Patois)</td>
    <td>2.6</td>
    <td>+6.0</td>
    <td>+5.2</td>
    <td>+6.9</td>
  </tr>
  <tr>
    <td>mfe (Mauritian)</td>
    <td>3.4</td>
    <td>+4.7</td>
    <td>+3.5</td>
    <td>+3.4</td>
  </tr>
  <tr>
    <td>csf (Seychellois)</td>
    <td>5.7</td>
    <td>+7.2</td>
    <td>+9.6</td>
    <td>+7.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>7.3</td>
    <td>+2.6</td>
    <td>+2.8</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>7.3</td>
    <td>+2.6</td>
    <td>+2.8</td>
    <td>+2.5</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>19.7</td>
    <td>+0.4</td>
    <td>+2.9</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>11.3</td>
    <td>+3.6</td>
    <td>+5.7</td>
    <td>+5.1</td>
  </tr>
</table>

Table 41: M<rightarrow>D BLEU scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each M<rightarrow>D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25. The best score for each language in the M<rightarrow>D paradigm is bolded and underlined.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cloud-cont</th>
    <th>-cloud-func</th>
    <th>-cloud-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latin (Indonesian)</td>
    <td>87.4</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>jav_Latin (Javanese)</td>
    <td>70.1</td>
    <td>+0.4</td>
    <td>+1.3</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>sin_Latin (Sundanese)</td>
    <td>71.6</td>
    <td>+1.7</td>
    <td>+1.5</td>
    <td>+1.9</td>
  </tr>
  <tr>
    <td>sma_Latin (Malay)</td>
    <td>27.4</td>
    <td>+12.4</td>
    <td>+2.5</td>
    <td>+15.5</td>
  </tr>
  <tr>
    <td>mri_Latin (Maori)</td>
    <td>25.7</td>
    <td>+16.6</td>
    <td>+10.4</td>
    <td>+18.7</td>
  </tr>
  <tr>
    <td>ceb_Latin (Cebuano)</td>
    <td>64.2</td>
    <td>-2.0</td>
    <td>-5.6</td>
    <td>-7.9</td>
  </tr>
  <tr>
    <td>zun_Latin (Central Malay)</td>
    <td>86.5</td>
    <td>+6.1</td>
    <td>+1.4</td>
    <td>+7.9</td>
  </tr>
  <tr>
    <td>tgl_Latin (Tagalog)</td>
    <td>73.2</td>
    <td>-6.0</td>
    <td>-9.0</td>
    <td>-13.5</td>
  </tr>
  <tr>
    <td>ilo_Latin (Ilocano)</td>
    <td>56.1</td>
    <td>+2.6</td>
    <td>-1.4</td>
    <td>-1.2</td>
  </tr>
  <tr>
    <td>fj_Latin (Fijian)</td>
    <td>27.7</td>
    <td>+17.3</td>
    <td>+10.5</td>
    <td>+16.5</td>
  </tr>
  <tr>
    <td>pli_Latin (Eastern Malagasy)</td>
    <td>64.4</td>
    <td>+6.9</td>
    <td>+0.4</td>
    <td>+7.3</td>
  </tr>
  <tr>
    <td>pag_Latin (Pangasinan)</td>
    <td>35.5</td>
    <td>+11.9</td>
    <td>+15.9</td>
    <td>+15.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>52.9</td>
    <td>+4.7</td>
    <td>+2.6</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>46.9</td>
    <td>+7.2</td>
    <td>+4.4</td>
    <td>+6.6</td>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>80.6</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>76.0</td>
    <td>-0.5</td>
    <td>+0.6</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta'izzi-Adeni Arabic)</td>
    <td>77.6</td>
    <td>-1.1</td>
    <td>+0.0</td>
    <td>-1.1</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>68.8</td>
    <td>-0.8</td>
    <td>+1.8</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>ap_Arab (South Levantine Arabic)</td>
    <td>71.2</td>
    <td>-5.1</td>
    <td>+0.2</td>
    <td>-5.7</td>
  </tr>
  <tr>
    <td>ape_Arab (North Levantine Arabic)</td>
    <td>72.8</td>
    <td>-3.7</td>
    <td>+1.1</td>
    <td>-3.8</td>
  </tr>
  <tr>
    <td>ars_Arab (Najdi Arabic)</td>
    <td>80.0</td>
    <td>-0.3</td>
    <td>-0.3</td>
    <td>-0.3</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>60.8</td>
    <td>-0.4</td>
    <td>+3.7</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>arz_Arab (Egyptian Arabic)</td>
    <td>71.4</td>
    <td>-3.5</td>
    <td>+1.1</td>
    <td>-3.2</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>72.6</td>
    <td>-1.9</td>
    <td>+1.0</td>
    <td>-1.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>71.5</td>
    <td>-2.2</td>
    <td>+1.2</td>
    <td>-2.0</td>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latin (Italian)</td>
    <td>86.3</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>spa_Latin (Spanish)</td>
    <td>85.2</td>
    <td>-1.4</td>
    <td>-4.4</td>
    <td>-18.0</td>
  </tr>
  <tr>
    <td>fra_Latin (French)</td>
    <td>87.8</td>
    <td>-14.6</td>
    <td>-5.2</td>
    <td>-19.5</td>
  </tr>
  <tr>
    <td>por_Latin (Portuguese)</td>
    <td>88.0</td>
    <td>-13.5</td>
    <td>-4.1</td>
    <td>-17.6</td>
  </tr>
  <tr>
    <td>ron_Latin (Romanian)</td>
    <td>88.2</td>
    <td>-15.8</td>
    <td>-6.5</td>
    <td>-21.0</td>
  </tr>
  <tr>
    <td>glg_Latin (Galician)</td>
    <td>86.6</td>
    <td>-14.0</td>
    <td>-3.6</td>
    <td>-18.2</td>
  </tr>
  <tr>
    <td>cat_Latin (Catalan)</td>
    <td>83.7</td>
    <td>-13.2</td>
    <td>-5.5</td>
    <td>-18.9</td>
  </tr>
  <tr>
    <td>oci_Latin (Occitan)</td>
    <td>80.0</td>
    <td>-9.2</td>
    <td>-2.3</td>
    <td>-11.4</td>
  </tr>
  <tr>
    <td>ast_Latin (Asturian)</td>
    <td>79.8</td>
    <td>-11.6</td>
    <td>-3.9</td>
    <td>-15.4</td>
  </tr>
  <tr>
    <td>lim_Latin (Limburg)</td>
    <td>84.5</td>
    <td>+15.5</td>
    <td>+18.5</td>
    <td>+15.4</td>
  </tr>
  <tr>
    <td>vec_Latin (Venetian)</td>
    <td>54.0</td>
    <td>+10.9</td>
    <td>+16.5</td>
    <td>+10.5</td>
  </tr>
  <tr>
    <td>scn_Latin (Sicilian)</td>
    <td>44.1</td>
    <td>+15.3</td>
    <td>+18.8</td>
    <td>+16.5</td>
  </tr>
  <tr>
    <td>srd_Latin (Sardinian)</td>
    <td>37.9</td>
    <td>+17.8</td>
    <td>+23.4</td>
    <td>+22.3</td>
  </tr>
  <tr>
    <td>fra_Latin (Friulian)</td>
    <td>42.5</td>
    <td>+18.3</td>
    <td>+20.7</td>
    <td>+20.3</td>
  </tr>
  <tr>
    <td>il_Latin (Ligurian)</td>
    <td>45.0</td>
    <td>+14.5</td>
    <td>+20.7</td>
    <td>+18.3</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>67.9</td>
    <td>-1.0</td>
    <td>+5.9</td>
    <td>-2.6</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>44.7</td>
    <td>+15.3</td>
    <td>+19.8</td>
    <td>+17.3</td>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latin (Turkish)</td>
    <td>86.7</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>uzn_Latin (Northern Uzbek)</td>
    <td>39.5</td>
    <td>+13.7</td>
    <td>+7.4</td>
    <td>+17.0</td>
  </tr>
  <tr>
    <td>tuk_Latin (Turkmen)</td>
    <td>34.4</td>
    <td>+12.6</td>
    <td>+7.5</td>
    <td>+15.5</td>
  </tr>
  <tr>
    <td>azj_Latin (North Azerbaijani)</td>
    <td>60.8</td>
    <td>+2.6</td>
    <td>+5.1</td>
    <td>+1.4</td>
  </tr>
  <tr>
    <td>cth_Latin (Crimean Tatar)</td>
    <td>55.7</td>
    <td>+4.3</td>
    <td>+5.9</td>
    <td>+5.2</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>47.6</td>
    <td>+8.3</td>
    <td>+6.5</td>
    <td>+9.8</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>47.6</td>
    <td>+8.3</td>
    <td>+6.5</td>
    <td>+9.8</td>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>86.2</td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>hne_Deva (Chattisgarhi)</td>
    <td>66.4</td>
    <td>+3.7</td>
    <td>+11.7</td>
    <td>+8.1</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>63.6</td>
    <td>+6.8</td>
    <td>+9.8</td>
    <td>+8.8</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>60.0</td>
    <td>+8.5</td>
    <td>+11.3</td>
    <td>+10.3</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>57.9</td>
    <td>+14.2</td>
    <td>+18.3</td>
    <td>+18.2</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>64.2</td>
    <td>+7.8</td>
    <td>+12.8</td>
    <td>+11.4</td>
  </tr>
  <tr>
    <td>Baseline < 25 Average</td>
    <td>64.2</td>
    <td>+7.8</td>
    <td>+12.8</td>
    <td>+11.4</td>
  </tr>
  <tr>
    <td>General Average</td>
    <td>62.5</td>
    <td>+2.1</td>
    <td>+4.8</td>
    <td>+1.8</td>
  </tr>
  <tr>
    <td>General Baseline < 25 Average</td>
    <td>54.6</td>
    <td>+6.8</td>
    <td>+8.1</td>
    <td>+7.8</td>
  </tr>
</table>

Table 42: M→D COMET scores by language for the model M2M. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.
<table>
  <tr>
    <th>Language</th>
    <th>off-the-shelf</th>
    <th>-cloud-cont</th>
    <th>-cloud-func</th>
    <th>-cloud-all</th>
  </tr>
  <tr>
    <th colspan="5">Austronesian Language Family</th>
  </tr>
  <tr>
    <td>ind_Latn (Indonesian)</td>
    <td>88.5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>jav_Latn (Javanese)</td>
    <td>67.5</td>
    <td>+6.6</td>
    <td>+8.2</td>
    <td>+8.0</td>
  </tr>
  <tr>
    <td>sun_Latn (Sundanese)</td>
    <td>63.7</td>
    <td>+8.9</td>
    <td>+12.1</td>
    <td>+11.1</td>
  </tr>
  <tr>
    <td>smo_Latn (Samoan)</td>
    <td>35.4</td>
    <td>+11.4</td>
    <td>+6.8</td>
    <td>+10.5</td>
  </tr>
  <tr>
    <td>mri_Latn (Maori)</td>
    <td>40.5</td>
    <td>+9.0</td>
    <td>+0.9</td>
    <td>+8.0</td>
  </tr>
  <tr>
    <td>ceb_Latn (Cebuano)</td>
    <td>61.1</td>
    <td>+0.3</td>
    <td>-3.8</td>
    <td>-3.0</td>
  </tr>
  <tr>
    <td>zsm_Latn (Standard Malay)</td>
    <td>68.5</td>
    <td>+3.2</td>
    <td>-0.4</td>
    <td>+3.5</td>
  </tr>
  <tr>
    <td>tgl_Latn (Tagalog)</td>
    <td>76.4</td>
    <td>-9.4</td>
    <td>-15.6</td>
    <td>-14.7</td>
  </tr>
  <tr>
    <td>ilo_Latn (Ilocano)</td>
    <td>48.2</td>
    <td>+9.1</td>
    <td>+5.3</td>
    <td>+7.1</td>
  </tr>
  <tr>
    <td>fij_Latn (Fijian)</td>
    <td>40.1</td>
    <td>+9.7</td>
    <td>+1.9</td>
    <td>+7.5</td>
  </tr>
  <tr>
    <td>plt_Latn (Plateau Malagasy)</td>
    <td>46.1</td>
    <td>+8.0</td>
    <td>+2.9</td>
    <td>+8.1</td>
  </tr>
  <tr>
    <td>pag_Latn (Pangasinan)</td>
    <td>43.1</td>
    <td>+12.6</td>
    <td>+14.2</td>
    <td>+13.3</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>55.2</th>
    <th>+5.7</th>
    <th>+3.0</th>
    <th>+4.8</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>49.5</th>
    <th>+8.4</th>
    <th>+5.4</th>
    <th>+7.8</th>
  </tr>
  <tr>
    <th colspan="5">Arabic Language Family</th>
  </tr>
  <tr>
    <td>arb_Arab (Modern Standard Arabic)</td>
    <td>86.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>acm_Arab (Mesopotamian Arabic)</td>
    <td>83.2</td>
    <td>+0.1</td>
    <td>+0.9</td>
    <td>-0.0</td>
  </tr>
  <tr>
    <td>acq_Arab (Ta'izzi-Adeni Arabic)</td>
    <td>84.3</td>
    <td>+0.0</td>
    <td>+0.6</td>
    <td>-0.2</td>
  </tr>
  <tr>
    <td>aeb_Arab (Tunisian Arabic)</td>
    <td>78.4</td>
    <td>-0.7</td>
    <td>+1.4</td>
    <td>-0.8</td>
  </tr>
  <tr>
    <td>ajj_Arab (Levantine Arabic)</td>
    <td>83.7</td>
    <td>-5.1</td>
    <td>-0.8</td>
    <td>-0.1</td>
  </tr>
  <tr>
    <td>apc_Arab (North Levantine Arabic)</td>
    <td>82.7</td>
    <td>-3.4</td>
    <td>+0.0</td>
    <td>-0.4</td>
  </tr>
  <tr>
    <td>arj_Arab (Nijdi Arabic)</td>
    <td>85.4</td>
    <td>+0.8</td>
    <td>+0.7</td>
    <td>+0.6</td>
  </tr>
  <tr>
    <td>ary_Arab (Moroccan Arabic)</td>
    <td>74.0</td>
    <td>-1.7</td>
    <td>+1.6</td>
    <td>-2.8</td>
  </tr>
  <tr>
    <td>azr_Arab (Egyptian Arabic)</td>
    <td>82.4</td>
    <td>-3.6</td>
    <td>-0.3</td>
    <td>-4.6</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>81.7</th>
    <th>-1.7</th>
    <th>+0.5</th>
    <th>-2.3</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>74.0</th>
    <th>-1.7</th>
    <th>+1.6</th>
    <th>-2.8</th>
  </tr>
  <tr>
    <th colspan="5">Romance Language Family</th>
  </tr>
  <tr>
    <td>ita_Latn (Italian)</td>
    <td>87.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>spa_Latn (Spanish)</td>
    <td>86.1</td>
    <td>-9.4</td>
    <td>-2.1</td>
    <td>-11.6</td>
  </tr>
  <tr>
    <td>fra_Latn (French)</td>
    <td>88.6</td>
    <td>-9.4</td>
    <td>-4.2</td>
    <td>-13.3</td>
  </tr>
  <tr>
    <td>por_Latn (Portuguese)</td>
    <td>88.4</td>
    <td>-8.2</td>
    <td>-3.0</td>
    <td>-11.3</td>
  </tr>
  <tr>
    <td>ron_Latn (Romanian)</td>
    <td>88.6</td>
    <td>-10.6</td>
    <td>-5.1</td>
    <td>-15.0</td>
  </tr>
  <tr>
    <td>glg_Latn (Galician)</td>
    <td>85.2</td>
    <td>-8.4</td>
    <td>-1.8</td>
    <td>-10.2</td>
  </tr>
  <tr>
    <td>cat_Latn (Catalan)</td>
    <td>84.0</td>
    <td>-7.5</td>
    <td>-2.5</td>
    <td>-9.7</td>
  </tr>
  <tr>
    <td>ces_Latn (Czech)</td>
    <td>79.3</td>
    <td>+2.8</td>
    <td>+2.2</td>
    <td>+5.7</td>
  </tr>
  <tr>
    <td>ast_Latn (Asturian)</td>
    <td>77.7</td>
    <td>-5.4</td>
    <td>-1.3</td>
    <td>-8.7</td>
  </tr>
  <tr>
    <td>lmo_Latn (Lombard)</td>
    <td>58.8</td>
    <td>+7.5</td>
    <td>+12.6</td>
    <td>+8.1</td>
  </tr>
  <tr>
    <td>vec_Latn (Venetian)</td>
    <td>70.3</td>
    <td>+1.4</td>
    <td>+7.1</td>
    <td>+0.1</td>
  </tr>
  <tr>
    <td>scn_Latn (Sicilian)</td>
    <td>59.5</td>
    <td>+8.3</td>
    <td>+11.8</td>
    <td>+8.3</td>
  </tr>
  <tr>
    <td>srd_Latn (Sardinian)</td>
    <td>56.9</td>
    <td>+8.7</td>
    <td>+13.6</td>
    <td>+10.3</td>
  </tr>
  <tr>
    <td>fur_Latn (Friulian)</td>
    <td>59.7</td>
    <td>+9.8</td>
    <td>+12.6</td>
    <td>+10.5</td>
  </tr>
  <tr>
    <td>lij_Latn (Ligurian)</td>
    <td>56.9</td>
    <td>+11.7</td>
    <td>+16.5</td>
    <td>+12.7</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>74.0</th>
    <th>-1.0</th>
    <th>+4.0</th>
    <th>-2.4</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>58.4</th>
    <th>+9.2</th>
    <th>+13.4</th>
    <th>+10.0</th>
  </tr>
  <tr>
    <th colspan="5">Turkic Language Family</th>
  </tr>
  <tr>
    <td>tur_Latn (Turkish)</td>
    <td>87.1</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>uzn_Latn (Northern Uzbek)</td>
    <td>56.5</td>
    <td>+8.5</td>
    <td>+7.7</td>
    <td>+8.7</td>
  </tr>
  <tr>
    <td>tuk_Latn (Turkmen)</td>
    <td>51.2</td>
    <td>+11.9</td>
    <td>+12.5</td>
    <td>+11.8</td>
  </tr>
  <tr>
    <td>azi_Latn (North Azerbaijani)</td>
    <td>70.5</td>
    <td>-0.5</td>
    <td>+4.0</td>
    <td>-1.7</td>
  </tr>
  <tr>
    <td>crh_Latn (Crimean Tatar)</td>
    <td>64.8</td>
    <td>+3.8</td>
    <td>+7.7</td>
    <td>+3.7</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>60.7</th>
    <th>+5.9</th>
    <th>+8.0</th>
    <th>+5.6</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>60.7</th>
    <th>+5.9</th>
    <th>+8.0</th>
    <th>+5.6</th>
  </tr>
  <tr>
    <th colspan="5">Indic Language Family</th>
  </tr>
  <tr>
    <td>hin_Deva (Hindi)</td>
    <td>88.3</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>hne_Deva (Chattisgarhi)</td>
    <td>77.2</td>
    <td>-0.1</td>
    <td>+4.3</td>
    <td>+1.3</td>
  </tr>
  <tr>
    <td>bho_Deva (Bhojpuri)</td>
    <td>75.2</td>
    <td>+0.9</td>
    <td>+3.2</td>
    <td>+2.0</td>
  </tr>
  <tr>
    <td>mag_Deva (Magahi)</td>
    <td>78.9</td>
    <td>+2.1</td>
    <td>+3.7</td>
    <td>+3.0</td>
  </tr>
  <tr>
    <td>mai_Deva (Maithili)</td>
    <td>74.6</td>
    <td>+3.7</td>
    <td>+5.2</td>
    <td>+4.7</td>
  </tr>
  <tr>
    <th>Average</th>
    <th>76.5</th>
    <th>+1.6</th>
    <th>+4.1</th>
    <th>+2.8</th>
  </tr>
  <tr>
    <th>Baseline < 25 Average</th>
    <th>76.5</th>
    <th>+1.6</th>
    <th>+4.1</th>
    <th>+2.8</th>
  </tr>
  <tr>
    <th>General Average</th>
    <th>65.9</th>
    <th>+1.9</th>
    <th>+4.1</th>
    <th>+1.3</th>
  </tr>
  <tr>
    <th>General Baseline < 25 Average</th>
    <th>56.6</th>
    <th>+6.7</th>
    <th>+7.7</th>
    <th>+7.3</th>
  </tr>
</table>

Table 43: M→D COMET scores by language for the model Aya-23. Performance gains/losses are relative to off-the-shelf. Averages for each M→D approach are computed for each language family as well as for the general body of languages studied. These averages are recomputed for languages whose off-the-shelf BLEU score < 25.

Reviewer paper summaries: The research paper presents DialUp, a method to enhance machine translation performance for low-resource languages and dialects.

This research paper aims to address the challenges of translating low-resource languages and dialects in machine translation. The authors present DialUp, which consists of techniques to adapt pretrained models to dialectical data and vice versa, with the goal of improving translation performance.

The paper evaluates DialUp taking two models (Aya-23 and M2M-100) across 49 languages from six language families. DialUp consists of two paradigms: M->D, where the models are adapted to the dialectical data during training, and D->M, where the dialectal data, during inference, is adapted to the model. 


The results demonstrate that M->D provides consistent gains, especially for low-performing languages, while D->M offers significant benefits for select dialects. The study also analyzes errors and discrepancies in model performance in the Arabic language family, indicating areas for further improvement in the method.
Summary of strengths: DialUp offers a novel approach to addressing the challenges of  languages and dialects in machine translation, especially when the languages are low-resourced.

A comprehensive analysis of two models is performed, showing DialUp’s effectiveness. Additionally empirical results are shown for 49 languages, spanning across morphologically different language families.
Summary of weaknesses: The efficacy of the automatic data generation is not discussed in detail. Automatic language generation is the major component of the M->D method. However the quality of the generated dialectal data is not ascertained. While this might be an expensive process, even if such a quality check was performed for a single language-dialect pair, it would have strengthened the results seen for M->D.
Comments, suggestions and typos: The bars for M->D, D->M, and M<->D in Figure 2 were not visible when I used my browser (Edge). However, when I switched to Adobe Reader, they were fixed. This might just be a compiling or a Browser issue, but just something to consider.



Comments after Author Discussion

Thank you for the detailed response! I agree that evaluating the quality of the generated language is whole research direction in itself and beyond the scope of this paper.

In lieu of this, I kept the Overall Assessment as I rated the paper fairly high, but raised the Soundness score.
Overall assessment scores: 4