Limited-Resource Adapters Are Regularizers, Not Linguists
Authors: Marcell Fekete, Nathaniel Romney Robinson, Ernests Lavrinovics, Djeride Jean-Baptiste, Raj Dabre, Johannes Bjerva, Heather Lent
Accepted at: acl25

Limited-Resource Adapters Are Regularizers, Not Linguists

Marcell Fekete1  Nathaniel R. Robinson2  Ernests Lavrinovics1
E. Djeride Jean-Baptiste3  Raj Dabre4  Johannes Bjerva1  Heather Lent1
1Aalborg University, Denmark; 2Johns Hopkins University, USA;
3University of Ottawa, Canada; 4IIT Madras, India
{mrfe, hcle}@cs.aau.dk

Abstract

Cross-lingual transfer from related high-resource languages is a well-established strategy to enhance low-resource language technologies. Prior work has shown that adapters show promise for, e.g., improving low-resource machine translation (MT). In this work, we investigate an adapter souping method combined with cross-attention fine-tuning of a pre-trained MT model to leverage language transfer for three low-resource Creole languages, which exhibit relatedness to different language groups across distinct linguistic dimensions. Our approach improves performance substantially over baselines. However, we find that linguistic relatedness—or even a lack thereof—does not covary meaningfully with adapter performance. Surprisingly, our cross-attention fine-tuning approach appears equally effective with randomly initialized adapters, implying that the benefit of adapters in this setting lies in parameter regularization, and not in meaningful information transfer. We provide analysis supporting this regularization hypothesis. Our findings underscore the reality that neural language processing involves many success factors, and that not all neural methods leverage linguistic knowledge in intuitive ways.

1 Introduction and Background

For a majority of the world’s languages, data scarcity remains a barrier for reliable machine translation (MT). Cross-lingual transfer from high-resource to low-resource languages has been shown to help mitigate this (Zoph et al., 2016; Tars et al., 2021), but has still not closed the gap between high- and low-resource scenarios. In this work, we seek to exploit methods from parameter-efficient fine-tuning (PEFT) to further improve cross-lingual transfer for a Creoles, an under-served group of low-resource languages.

PEFT methods such as bottleneck adapters (Houlsby et al., 2019; Pfeiffer et al., 2020), LoRA (Hu et al., 2021), and (IA)3 (Liu et al., 2022a) are widely used to adapt language models to low-resource languages (Parović et al., 2022; Parovic et al., 2023; Rathore et al., 2023; Vázquez et al., 2024; Dementieva et al., 2025). This is in part because they are robust to hyperparameter choices and training data amounts, in contrast to full model fine-tuning (Pfeiffer et al., 2024). Some PEFT methods can successfully use monolingual data to improve MT performance; for example, Üstün et al. (2021) train denoising adapters for both source and target languages, inserting the adapters between the encoder and decoder layers of an MT model, respectively. By freezing all model weights except for the cross-attention of the decoder, they then carry out cross-attention fine-tuning (CAFT) with only a limited amount of parallel data. Chronopoulou et al. (2023a) also address data scarcity in adaptation to new domains following Wortsman et al. (2022) by “souping” (i.e., weight-space averaging) pretrained domain adapters, as

![Overview of the MT transfer experiments between English and Creoles. The ↑ path shows the use of source language adapter in the encoder and the ‘souped’ (weight-averaged) target and transfer language adapters in the decoder layers. The ↓ path shows the opposite direction.](page_1012_370_388_246.png)

Figure 1: Overview of the MT transfer experiments between English and Creoles. The ↑ path shows the use of source language adapter in the encoder and the ‘souped’ (weight-averaged) target and transfer language adapters in the decoder layers. The ↓ path shows the opposite direction.
in Equation 1 (where \( \theta_i \) represent the weights of each domain adapter). We combine the principles of these prior works in our paper.

\[
\theta_{soup} = \frac{1}{l} \sum_{i=1}^{l} \theta_i
\] (1)

Prior work has also catalyzed transfer by training language family adapters to leverage phylogenetic relations between languages (Faisal and Anastasopoulos, 2022; Chronopoulou et al., 2023b). Other factors of selecting transfer languages have been investigated in the past besides phylogeny. These include similarity in terms of explicit grammatical features like word order (Pires et al., 2019), similarity in terms of language representation space (Üstün et al., 2022), as well as lexical overlap (Pfeiffer et al., 2021; de Vries et al., 2022). Other metrics have been proposed to select candidate transfer languages, including Subword Evenness (SuE; Pelloni et al., 2022) which quantifies how uneven subword lengths are in a multilingual tokenizer: uneven implies successful transfer.

In this work we focus on MT for Creole languages, a set of low-resource languages with hundreds of millions of speakers who presently lack reliable MT systems, despite demonstrable need for them (Neto et al., 2020; Robinson et al., 2024). Creole language technologies have not been thoroughly researched or developed; yet several works that do exist inspired our decision to explore them (cf. our literature review in Appendix A). Although most Creoles are genealogically related to high-resource languages (\emph{e.g.}, English, French, Spanish), previous works have had mixed success leveraging transfer learning to boost performance for Creoles (Lent et al., 2022a, 2024; Robinson et al., 2022, 2023). No prior work has conclusively determined which languages to select for successful Creole transfer, explored PEFT in this application, nor compared linguistic and non-linguistic language relations in Creole transfer language selection. These aspects combined make Creoles a viable testbed for comparing transfer criteria.

We apply a new adapter method inspired by Üstün et al.’s (2021) approach to MT for three Creole languages. (See details in §2.) We experimented with this approach to compare a large variety of dimensions of linguistic relatedness and found that, contrary to our initial hypotheses, gains are virtually indistinguishable regardless of which transfer criterion we apply. Still more surprisingly, incorporating unrelated languages or noise to the Creole adapters yields comparable improvements. We conjecture that even untrained adaptors may assist cross-attention fine-tuning by acting as regularizers. We couple this hypothesis with analysis presented in §3.

We contribute both ① a novel regularization method combining adapter souping with cross-attention fine-tuning that surpasses baselines by 8 BLEU in some settings, and ② evidence that some of the benefit of adapters lies in regularization, independent of a wide range of linguistic information.

2 Methodology and Experiments

We follow Üstün et al. (2021) using their two-step methodology. First, we train monolingual denoising adapters separately for each language. At inference time, we add the relevant adapters to the encoder and decoder corresponding to the source and target languages, and fine-tune the decoder cross-attention on small amounts of parallel bitexts. Furthermore, we induce transfer by weight-averaging ("souping") Creole adapters with candidate transfer language adapters inspired by previous efforts (Faisal and Anastasopoulos, 2024; Wortsman et al., 2022; Chronopoulou et al., 2023a). The novelty lies in combining source and target language adapters, cross-attention fine-tuning and souping in a single method for improving low-resource MT. (See more details about our methodology in Section 2.3).

We focus our efforts on three Creole languages: Haitian, Papiamento, and Sango, spoken in total by roughly 18 million speakers in Haiti, the ABC Islands, and the Central African Republic, respectively.\footnote{According to Ethnologue.} This selection is primarily due to the intersection of data availability and support by NLLB-200, the MT model we apply in experiments (NLLB Team et al., 2022). Despite their unique contexts, these languages share commonalities in socio-historical development: they evolved from linguistic contact between European and African languages resulting from Western colonialism (Kouwenberg and Singler, 2009). As such, they can trace parts of their lexicon and grammar back to both European and African languages.

2.1 Selecting Languages for Transfer

We examine five principled approaches to select transfer languages, which range from linguistically motivated to purely data-driven. First, we
<table>
  <tr>
    <th>Dataset</th>
    <th>Domain</th>
    <th>Source</th>
  </tr>
  <tr>
    <td>MADLAD</td>
    <td>Web scrape</td>
    <td>Kudugunta et al., 2023</td>
  </tr>
  <tr>
    <td>NLLB-OPUS</td>
    <td>Web scrape</td>
    <td>NLLB Team et al., 2022</td>
  </tr>
  <tr>
    <td>FLORES-200</td>
    <td>Wikipedia</td>
    <td>NLLB Team et al., 2022</td>
  </tr>
</table>

Table 1: Datasets and domains used. We use MADLAD to train adapters, NLLB-OPUS to fine-tune cross-attention, and FLORES-200 to evaluate.

leverage language phylogeny, selecting likely Indo-European (IE; Rickford and McWhorter, 1997; Lefebvre, 2011) and Niger-Congo (NC; Hammarström et al., 2023; Mufwene, 2006; Hall, 1958; Faine, 1937) relatives for Haitian, Papiamento, and Sango. We also experiment with transfer between these Creole languages, as previous works have argued that Creole languages have linguistic commonalities with each other (Daval-Markussen and Bakker, 2012). We next select languages by similarity of documented typological features, calculated as lang2vec distances (Littell et al., 2017). For data-driven language selection, we determine language relatedness from similarity of NLLB-200 language embeddings (NLLB representations; NLLB Team et al., 2022), as well as Subword Evenness (SuE; Pelloni et al., 2022). For details of our language selection processes, see Appendix C.

2.2 Control experiments

As an experimental control, we identify three language groups with no linguistic relation to the Creole languages in question: Uralic languages, Dravidian languages, and CJK (i.e., Chinese, Japanese, and Korean). None of these groups share phylogeny or significant amounts of typological features with our Creole languages. The latter two also differ in terms of script. And the final group is distinct in that its languages also bear no phylogenetic relation to each other. As a final experimental control, we use a randomly initialized, untrained adapter (init). We use this untrained adapter in two ways: as a replacement for the Creole language adapter, and souped with the Creole language adapter. As Chronopoulou et al. (2023a) suggest for souping, we use the same random initialization for all untrained and trained adapters.

2.3 Experimental Details

Our experiments follow the same general steps. (1) We train all language adapters individually on denoising using monolingual data from MADLAD-400 (Kudugunta et al., 2023) that we preprocess and clean further.2 (2) We add the source language adapter s to the encoder and the target language adapter t to the decoder and freeze all adapter and model parameters. In adapter averaging experiments the init or the helper language adapters are souped with the s or t language adapters in this step as well. (3) We unfreeze the cross-attention of the decoder and use 10k parallel data from NLLB-OPUS (NLLB Team et al., 2022) to do cross-attention fine-tuning (CA-FT). (4) We evaluate MT from s → t on Flores-200 (NLLB Team et al., 2022) using mean BLEU and chrF scores. (See Table 1 for data composition.)

All experiments are conducted using a 600M-parameter distillation of the original 54B-parameter NLLB-200 model (NLLB Team et al., 2022) which has 12 encoder and 12 decoder layers, 16 attention heads, 1,024 dimensions, and a model vocabulary of over 250,000 sentence-piece tokens shared across all languages. We adopt the adapter training and the hyperparameters from Üstün et al. (2021), training bottleneck adapters on 10,000 training samples for a maximum of 100,000 training steps with early stopping, a batch size of 8, 4,000 warmup steps, and the Adam optimizer with a maximum learning rate of \(2e^{-4}\) with a gradient accumulation of 8 steps. All adapters initialized from the same random seed as Chronopoulou et al. (2023a) show that this is necessary for successful souping. We train CA-FT for 1 epoch with early stopping. We soup adapters with the same weight except when averaging with the init adapter where the weight of the Creole language to init adapter is 1 : 3, to closer approximate the other setups where the Creole language adapter is souped with three other adapters.

3 Results and Analysis

Here we present only Creole-to-English BLEU scores. We conducted preliminary experiments using only 800 aligned fine-tuning segments on these and the inverse translation directions, and saw the same trends shown in our results here. We decided to prioritize into-English MT for our main experiments after seeing small adapter gains across the board for into-Creole. (See Appendix D for our full results suite.) Table 2 shows BLEU improvements of our approaches over both baselines: the base MT model with CA-FT, and the s and t language adapters without souping (Üstün et al., 2021). The

2See Appendix B for preprocessing details.
<table>
  <tr>
    <th>Experiment</th>
    <th>hat → eng</th>
    <th>pap → eng</th>
    <th>sag → eng</th>
  </tr>
  <tr>
    <td>Base Model (CA-FT)</td>
    <td>33.37</td>
    <td>38.97</td>
    <td>10.89</td>
  </tr>
  <tr>
    <td>s and t Adapters</td>
    <td>32.33</td>
    <td>40.04</td>
    <td>11.40</td>
  </tr>
  <tr>
    <td>Untrained s Adapter</td>
    <td>37.07</td>
    <td>45.01</td>
    <td>14.91</td>
  </tr>
  <tr>
    <td>IE Transfer</td>
    <td>36.44</td>
    <td>46.35</td>
    <td>12.46</td>
  </tr>
  <tr>
    <td>NC Transfer</td>
    <td>36.06</td>
    <td>46.69</td>
    <td>12.29</td>
  </tr>
  <tr>
    <td>Creole Transfer</td>
    <td>35.25</td>
    <td>46.23</td>
    <td>12.76</td>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>36.54</td>
    <td>47.04</td>
    <td>13.07</td>
  </tr>
  <tr>
    <td>NLLB Vec</td>
    <td>35.80</td>
    <td>46.91</td>
    <td>12.80</td>
  </tr>
  <tr>
    <td>Subword Evenness</td>
    <td>36.36</td>
    <td>47.03</td>
    <td>13.12</td>
  </tr>
  <tr>
    <td>Mean Improvement</td>
    <td>+2.70</td>
    <td>+7.74</td>
    <td>+1.86</td>
  </tr>
  <tr>
    <td>Untrained Souping</td>
    <td>37.42</td>
    <td>46.34</td>
    <td>13.41</td>
  </tr>
  <tr>
    <td>Uralic</td>
    <td>37.06</td>
    <td>47.00</td>
    <td>13.58</td>
  </tr>
  <tr>
    <td>CJK</td>
    <td>36.41</td>
    <td>47.17</td>
    <td>13.33</td>
  </tr>
  <tr>
    <td>Dravidian</td>
    <td>36.55</td>
    <td>47.27</td>
    <td>13.27</td>
  </tr>
  <tr>
    <td>Mean Improvement</td>
    <td>+3.49</td>
    <td>+7.98</td>
    <td>+2.51</td>
  </tr>
</table>

Table 2: Mean BLEU scores for Creole to English MT experiments over a diverse set of conditions for selecting transfer languages, as motivated by phylogeny (♣), language group (♥), typological databases (🞂), model representations (▲), and tokenization (🞃). Experimental results are also compared against control experiments with unrelated languages (🞄) and untrained adapters (🞅). We highlight mean performance improvements over the Base Model (CA-FT) for both transfer and controls.

<table>
  <tr>
    <th>FT examples:</th>
    <th>800</th>
    <th>10k</th>
  </tr>
  <tr>
    <td>Base Model (CA-FT)</td>
    <td>45.45</td>
    <td>45.53</td>
  </tr>
  <tr>
    <td>s and t Adapters</td>
    <td>38.58</td>
    <td>41.92</td>
  </tr>
  <tr>
    <td>spa + por + oci Souping</td>
    <td>41.87</td>
    <td>43.74</td>
  </tr>
  <tr>
    <td>Untrained Souping</td>
    <td>43.97</td>
    <td>44.75</td>
  </tr>
</table>

Table 3: BLEU scores for Catalan to English MT.

improvements, however, are not particularly differentiated between various transfer and control scenarios, and notably, untrained adapters match or exceed performance of trained ones.

Regularization, Not Transfer? Two facets of our results indicate that cross-lingual transfer does not explain the success of our methodology. The first is that adapter souping improves performance over the baselines, regardless of the criteria used to select transfer languages. The unrelated control languages (i.e., Dravidian, Uralic, and CJK) achieve performance gains on par with languages selected through principled means (e.g., genealogy and typology), without marked differences in BLEU or chrF scores. This holds for both the source and target language adapters in the encoder and decoder, respectively. The second is the commensurate success of the untrained adapters in averaging. Souping adapters with ‘meaningful’ language adapters is no better than souping with random noise, indicating that cross-lingual transfer (or any information transfer) is likely not responsible for performance gains. These findings seem more consistent with previous works on regularization, which find that adding noise can improve a model’s ability to generalize and avoid overfitting (Hochreiter and Schmidhuber, 1994; Poole et al., 2014; Moradi et al., 2020) (which may be pertinent given both our limited train sets and train/test domain mismatch).

We perform multiple analyses to test our hypothesis that adapters are regularizing. First, we compared the use of untrained adapters to the use of adapters of known closely related languages. Because the close relatives of Creole languages are disputed, we turned to another relatively low resource language, Catalan, which has three close relatives also supported by NLLB-200. Spanish, Portuguese, and Occitan are closely related to Catalan by effectively every reasonable linguistic criterion. Table 3 shows different trends for Catalan (a language on which NLLB’s baseline performance is notably high). In this scenario, none of the adapter approaches improve performance over the baseline, once again indicating that the monolingual denoising adapter training on MADLAD data is not positively contributing. And once again, even with these highly attested close linguistic relationships, we see the random adapter comes nearest to recovering the baseline score in both data settings. This set of results underscores the possibility that CA-FT without randomized regularization may lead to overfitting (hence the performance drops). We also experimented souping only one or two related languages for both Catalan→English and Papiamento→English but saw no improvements.

We observed training loss, gradient norms, and dev loss for CA-FT of the experiments presented in this section. We compare untrained adapter souping (hypothesized regularization) with simple s and t adapters (unregularized), finding that the unregularized case has moderately higher gradient norms and dev loss, consistent with our hypothesis. The unregularized case also sees higher train loss, which is inconclusive regarding our hypothesis since an overfit model may be stuck in a local minimum or be overfit on a distribution varying from the CA-FT data. (See Appendix E.)
Figure 2: Parameter variance between the pretrained Creole adapters (represented by the outliers) vs the souped adapters of various experimental conditions (shown in the boxplots).

We also show that parameter variance decreases substantially when souping adapters compared to the pretrained Creole adapters (Figure 2), further supporting our regularization explanation.

Qualitative Evaluation Beyond computational analyses, we further assess the quality of translations across our experimental conditions. To start, a subset of the authors create a simple annotation guideline, inspired by MQM (Burchardt, 2013), to help identify errors regarding vocabulary, grammar, and other translation issues (e.g., addition or omission). Then, one author, a native Haitian speaker, carries out a linguistically-grounded manual evaluation of Haitian → English translation across the baseline of s and t Adapters, the IE Transfer, and the Untrained Souping. The evaluation involves 33 samples per condition presented to the annotator in three shuffled sets (i.e., 99 sentences in total) to avoid biasing him. The primarily focus falls on grammar and translation issues (For a condensed list of errors, see Table 9 in Appendix F). Although a small sample size, we observe that adapter souping results in less grammatical and lexical errors when applying the pretrained Haitian and English adapters with CA-FT. While both IE Transfer and Untrained Souping are comparable in reducing lexical errors, the latter does better in avoiding grammatical errors. This may indicate a comparative success of regularization when adding random noise that is also reflected in the +0.98 BLEU score Untrained Souping has over IE Transfer for Haitian → English (see Table 2). We believe that even a small manual translation evaluation demonstrates the value of involving native speakers in MT.

4 Conclusion

Inspired by prior success of monolingual adapters and CA-FT for low-resource MT, we corroborate prior findings that adapters can help by consistently improving BLEU scores for a set of Creole languages. However, we differ from prior research by finding no evidence of linguistic or informational transfer from adapters, even across a wide array of linguistic relationships. We hypothesize that most adapter aid in this setting is from regularization and find some analytical support for this.

Limitations

Scalability This work can only be applied to a small subset of all Creoles that are present in larger benchmarks (Lent et al., 2024; Robinson et al., 2024) due to the fact that they are low-resource languages – in certain cases, extremely so –, so there is an issue of data scarcity and a limited support by MT models. This data scarcity extends to also the African relatives of the Creoles we target in this paper. Our work is further limited by the limited coverage by typological resources. Grambank (Skirgård et al., 2023) supports only Haitian, not Papiamento and Sango, and while the lang2vec covers more languages, the coverage is limited and uneven. Despite these constraints, however, we believe our selection of Creoles (i.e., Haitian, Papiamento, and Sango) are diverse enough to make high-level hypotheses about transfer learning for Creoles; these languages have different relatives (e.g., inheriting large vocabulary from French, Portuguese, and Ngbandi, respectively) and are spoken across different regions (i.e., the Caribbean and Africa). Moreover, our experimentation over these languages is extensive, with careful investigation of 6 transfer scenarios, 1 init baseline, and 3 principled controls. So while this work cannot scale widely across Creoles, we investigate transfer scenarios for each Creole deeply.

Domain of Data Similar to many other low-resource languages, much of the available data for Creoles comes from the religious domain (e.g., Bible translations), which has a characteristic style that is typically not representative of language use in other contexts (Mielke et al., 2019). Moreover, religious domain content may not necessarily be culturally appropriate for some communities (Hershovitch et al., 2022; Mager et al., 2023). In order to be truly useful for end users, data sourced from a
wide variety of domains will ultimately be a necessary component for the development of robust MT systems for Creoles. Until more such data becomes available, we believe that work at the intersection of transfer learning and domain adaptation could be greatly beneficial for Creole NLP.

Difficulty in Verifying Neural Phenomena
While we find some analytical evidence that our adapters are boosting performance by regularizing fine-tuning, it is difficult to verify this with certainty. Regularization is a mathematical phenomenon that can be demonstrated rigorously, but such involved analysis is outside the scope of this current work; hence we leave these important analytical details to future work.

Ethics Statement
As Creole languages are often marginalized, we recognize the necessity of heightened ethical scrutiny for research involving these languages. We thus engage directly with considerations from Lent et al. (2022b), Mager et al. (2023), and Bird (2020), to ensure that our work is in accordance with the needs and desires of speakers. To this end, we note that speakers of many different Creole languages actively want and/or need improved MT technology for their language. In the context of Haitian Creole, for example, MT technology is highly desirable for facilitating interactions with non-Haitian speakers (e.g., the neighboring, Spanish-speaking Dominican Republic, and foreign workers at NGOs operating in Haiti). Additionally, MT has the potential to greatly improve access to information for Haitians. DeGraff et al. (2022) argues that the lack of access to education in Creole is a critical barrier to social and economic mobility in Haiti. Accordingly, reliable MT for Haitian Creole could make available high-quality educational resources from platforms such as edX, Coursera, MITx, and Udemy, for example. Thus, if MT systems could offer accurate translations into Haitian Creole, they would become transformative tools for advancing the country’s development, breaking down educational barriers, and fostering greater social equity. We note similar potential for MT for speakers of Sango in Central African Republic, Chad, and Democratic Republic of the Congo and (African Development Bank Group, 2024). And while the socioeconomic context of Aruba, Bonaire, and Curacao is quite different from that of aforementioned Haiti or Central African Republic, speakers’ desire for language technology for Papiamento is well documented through projects like papiamentu.ai.

Acknowledgments
MF, JB, and HL were supported by the Carlsberg Foundation, under the Semper Ardens: Accelerate program (project nr. CF210454); the same project also supported the annotation work performed by DJB. EL was funded by the Google Award for Inclusion Research program (awarded to JB and HL for the “CREOLE: Creating Resources for Disadvantaged Language Communities” project). We extend our thanks to Mike Zhang, Esther Ploeger, and the other members of AAU-NLP, for their helpful feedback on our manuscript.

References
David Adelani, Jesujoba Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajudeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emenezue, Colin Leong, Michael Beukman, Shamsuddeen Muhammad, Guyo Jarso, Oreen Yousuf, Andre Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin Ajibade, Tunde Ajayi, Yvonne Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anouluwopo Aremu, Perez Ogayo, Jonathan Mukibi, Fatoumata Ouoba Kabore, Godson Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire Memdokam Koagne, Edwin Munkoh-Buabeng, Valencia Wagner, Idris Abdulmunin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, and Sam Manthulu. 2022. A few thousand translations go a long way! leveraging pre-trained models for African news translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3053–3070, Seattle, United States. Association for Computational Linguistics.

David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Liggins, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emenezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anouluwopo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajudeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukibi, Verah Otiente, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwu Adeyemi, Gerald Muruki, Emmanuel Anebi, Chiamaka Chukwunweke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akin-
ode, Deborah Nabagereka, Maurice Katusime, Ayo-dele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaogheh Ahia, Bonaventure F. P. Dosso, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei. 2021. MasakhaNER: Named entity recognition for African languages. Transactions of the Association for Computational Linguistics, 9:1116–1131.

African Development Bank Group. 2024. Central african republic: New digital era dawns with debut of high-speed internet cable. African Development Bank Group.

Hamidreza Amirzadeh, Sadegh Jafari, Anika Harju, and Rob van der Goot. 2025. data2langvec: Data driven typological features completion. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6520–6529, Abu Dhabi, UAE. Association for Computational Linguistics.

Ruth-Ann Armstrong, John Hewitt, and Christopher Manning. 2022. JamPatoisNLI: A jamaican patois natural language inference dataset. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5307–5320, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Steven Bird. 2020. Decolonising speech and language technology. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3504–3519, Barcelona, Spain (Online). International Committee on Computational Linguistics.

Aljoscha Burchardt. 2013. Multidimensional quality metrics: a flexible system for assessing translation quality. In Proceedings of Translating and the Computer 35, London, UK. Aslib.

Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22–64, Edinburgh, Scotland. Association for Computational Linguistics.

Bernard Caron, Marine Courtin, Kim Gerdes, and Sylvain Kahane. 2019. A surface-syntactic UD treebank for Naija. In Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), pages 13–24, Paris, France. Association for Computational Linguistics.

Alexandra Chronopoulou, Matthew Peters, Alexander Fraser, and Jesse Dodge. 2023a. Adapter-Soup: Weight averaging to improve generalization of pretrained language models. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2054–2063, Dubrovnik, Croatia. Association for Computational Linguistics.

Alexandra Chronopoulou, Dario Stojanovski, and Alexander Fraser. 2023b. Language-family adapters for low-resource multilingual neural machine translation. In Proceedings of the Sixth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2023), pages 59–72, Dubrovnik, Croatia. Association for Computational Linguistics.

Christopher Clarke, Roland Daynauth, Jason Mars, Charlene Wilkinson, and Hubert Devonish. 2024. GuyLingo: The Republic of Guyana creole corpora. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 792–798, Mexico City, Mexico. Association for Computational Linguistics.

Raj Dabre and Aneerav Sukhoo. 2022. Kreol-MorisienMT: A dataset for mauritian creole machine translation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2022, pages 22–29, Online only. Association for Computational Linguistics.

Aymeric Daval-Markussen and Peter Bakker. 2012. Explorations in creole research with phylogenetic tools. In Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH, pages 89–97, Avignon, France. Association for Computational Linguistics.

Wietse de Vries, Martijn Wieling, and Malvina Nissim. 2022. Make the best of cross-lingual transfer: Evidence from POS tagging with over 100 languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7676–7685, Dublin, Ireland. Association for Computational Linguistics.

Michel DeGraff, William Scott Frager, and Haynes Miller. 2022. Language policy in haitian education. Journal of Haitian Studies, 28(2):33–95.

Daryna Dementieva, Valeria Khlyenko, and Georg Groh. 2025. Cross-lingual text classification transfer: The case of Ukrainian. In Proceedings of the 31st International Conference on Computational Linguistics, pages 1451–1464, Abu Dhabi, UAE. Association for Computational Linguistics.

Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online (v2020.3). Zenodo.

Jules Faine. 1937. Philologie creole: Études historique et étymologiques sur la langue creoles d’Haiti. Imprimerie de l’Etat, Port-au-Prince.

Fahim Faisal and Antonios Anastasopoulos. 2022. Phylogeny-inspired adaptation of multilingual models to new languages. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 434–452, Online only. Association for Computational Linguistics.
Fahim Faisal and Antonios Anastasopoulos. 2024. Data-augmentation-based dialectal adaptation for LLMs. In Proceedings of the Eleventh Workshop on NLP for Similar Languages, Varieties, and Dialects (VarDial 2024), pages 197–208, Mexico City, Mexico. Association for Computational Linguistics.

Robert E. Frederking, Ralf D. Brown, and Christopher Hogan. 1997. The DIPLOMAT rapid development speech MT system. In Proceedings of Machine Translation Summit VI: Systems, pages 261–262, San Diego, California.

Robert Hall. 1958. Creole languages and genetic relationships. Word, 14:367–373.

Harald Hammarström, Sebastian Nordhoff, Robert Forkel, Martin Haspelmath, and Sebastian Bank. 2023. Glottolog. Online database.

Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Pi-queras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022. Challenges and strategies in cross-cultural NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6997–7013, Dublin, Ireland. Association for Computational Linguistics.

Sepp Hochreiter and Jürgen Schmidhuber. 1994. Simplifying neural nets by discovering flat minima. In Advances in Neural Information Processing Systems, volume 7. MIT Press.

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, and Bruna Morrone. 2019. Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, page 10, Long Beach, California.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv preprint. ArXiv:2106.09685 [cs].

Silvia Kouwenberg and John Victor Singler. 2009. The Handbook of Pidgin and Creole Studies. John Wiley & Sons.

Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A. Choquette-Choo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. Madlad-400: A multilingual and document-level large audited dataset. Preprint, arXiv:2309.04662.

Claire Lefebvre. 2011. Substrate features in the properties of verbs in three atlantic creoles: Haitian Creole, Saramaccan and Papiamentu. In Claire Lefebvre, editor, Creoles, their Substrates, and Language Typology, pages 127–154. John Benjamins, Amsterdam.

Heather Lent, Emanuele Bugliarello, and Anders Søgaard. 2022a. Ancestor-to-creole transfer is not a walk in the park. In Proceedings of the Third Workshop on Insights from Negative Results in NLP, pages 68–74, Dublin, Ireland. Association for Computational Linguistics.

Heather Lent, Kelechi Ogueji, Miryam de Lhoneux, Orevaoghene Ahia, and Anders Søgaard. 2022b. What a creole wants, what a creole needs. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6439–6449, Marseille, France. European Language Resources Association.

Heather Lent, Kushal Tatariya, Raj Dabre, Yiyi Chen, Marcell Fekete, Esther Ploeger, Li Zhou, Ruth-Ann Armstrong, Abee Eijansantos, Catriona Malau, Hans Erik Heje, Ernest Lavrinovic, Diptesh Kanodia, Paul Belony, Marcel Bollmann, Loïc Grobol, Miryam de Lhoneux, Daniel Hershcovich, Michel DeGraff, Anders Søgaard, and Johannes Bjerva. 2024. CreoleVal: Multilingual multitask benchmarks for creoles. Transactions of the Association for Computational Linguistics, 12:950–978.

Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 8–14, Valencia, Spain. Association for Computational Linguistics.

Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mota, Tenghao Huang, Mohit Bansal, and Colin Raffel. 2022a. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning. arXiv preprint. ArXiv:2205.05638 [cs].

Zhengyuan Liu, Shikang Ni, Ai Ti Aw, and Nancy F. Chen. 2022b. Singlish message paraphrasing: A joint task of creole translation and text normalization. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3924–3936, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Manuel Mager, Elisabeth Mager, Katharina Kann, and Ngoc Thang Vu. 2023. Ethical considerations for machine translation of indigenous languages: Giving a voice to the speakers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4871–4897, Toronto, Canada. Association for Computational Linguistics.

Susanne Maria Michaelis, Philippe Maurer, Martin Haspelmath, and Magnus Huber, editors. 2013. APICS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig.

Sabrina J. Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, and Jason Eisner. 2019. What kind of language is hard to language-model? In Proceedings
of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4975–4989, Florence, Italy. Association for Computational Linguistics.

Ludovic Mompelat, Daniel Dakota, and Sandra Kübler. 2022. How to parse a creole: When martinican creole meets French. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4397–4406, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.

Reza Moradi, Reza Berangi, and Behrouz Minaei. 2020. A survey of regularization strategies for deep models. Artificial Intelligence Review, 53(6):3947–3986.

Salikoko Muwene. 2006. Pidgins and creoles. In Braj B. Kachru, Yamuna Kachru, and Cecil L. Nelson, editors, The Handbook of World Englishes, pages 313–327. Blackwell, Hoboken, NJ.

Shamsudeen Hassan Muhammad, Idris Abdulmumin, Seid Muhie Yimam, David Ifeoluwa Adelejani, Ibrahim Said Ahmad, Nedjma Ousidhoum, Abinev Ali Ayele, Saif Mohammad, Meriem Beloucif, and Sebastian Ruder. 2023. SemEval-2023 task 12: Sentiment analysis for African languages (AfriSenti-SemEval). In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2319–2337, Toronto, Canada. Association for Computational Linguistics.

Shamsudeen Hassan Muhammad, David Ifeoluwa Adelejani, Sebastian Ruder, Ibrahim Sa’id Ahmad, Idris Abdulmumin, Bello Shehu Bello, Monojit Choudhury, Chris Chinenye Emezue, Saheed Salahudeen Abdullahi, Anoluwapo Aremu, Alípío Jorge, and Pavel Brazdil. 2022. NaijaSenti: A Nigerian Twitter sentiment corpus for multilingual sentiment analysis. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 590–602, Marseille, France. European Language Resources Association.

Antonio Carvalho Neto, Fernanda Versiani, Kelly Pelizari, Carolina Mota-Santos, and Gustavo Abreu. 2020. Latin american, african and asian immigrants working in brazilian organizations: facing the language barrier. Revista Economia & Gestão, 20(55).

NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hefner, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejía-Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Samnun Spruit, Chau Tran, Pierre Andrews, Necip Fazıl Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation.

Kelechi Ogueji and Orevaoghene Ahia. 2019. Pidginummt: Unsupervised neural machine translation from west african pidgin to english. arXiv preprint arXiv:1912.03444.

Wuraola Fisayo Oyewusi, Olubayo Adekanmbi, and Olalekan Akinsande. 2020. Semantic enrichment of nigerian pidgin english for contextual sentiment classification. arXiv preprint arXiv:2003.12450.

Marinela Parović, Alan Ansell, Ivan Vulić, and Anna Korhonen. 2023. Cross-lingual transfer with target language-ready task adapters. In Findings of the Association for Computational Linguistics: ACL 2023, pages 176–193, Toronto, Canada. Association for Computational Linguistics.

Marinela Parović, Goran Glavaš, Ivan Vulić, and Anna Korhonen. 2022. BAD-X: Bilingual adapters improve zero-shot cross-lingual transfer. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1791–1799, Seattle, United States. Association for Computational Linguistics.

Olga Pelloni, Anastassia Shaitarova, and Tanja Samardzic. 2022. Subword evenness (SuE) as a predictor of cross-lingual transfer to low-resource languages. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7428–7445, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, and Edoardo Maria Ponti. 2024. Modular deep learning. Preprint, arXiv:2302.11529.

Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020. MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational Linguistics.

Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2021. UNKs everywhere: Adapting multilingual language models to new scripts. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10186–10203, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
Edoardo Maria Ponti, Helen O’Horan, Yevgeni Berzak, Ivan Vulić, Roi Reichart, Thierry Poibeau, Ekaterina Shutova, and Anna Korhonen. 2019. Modeling language variation and universals: A survey on typological linguistics for natural language processing. Computational Linguistics, 45(3):559–601.

Ben Poole, Jascha Sohl-Dickstein, and Surya Ganguli. 2014. Analyzing noise in autoencoders and deep networks. CoRR, abs/1406.1831.

Vipul Rathore, Rajdeep Dhingra, Parag Singla, and Mausam. 2023. ZGUL: Zero-shot generalization to unseen languages using multi-source ensembling of language adapters. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6969–6987, Singapore. Association for Computational Linguistics.

John R Rickford and John McWhorter. 1997. Language contact and language generation: Pidgins and creoles. In Florian Coulmas, editor, The handbook of sociolinguistics, pages 238–256. Blackwell, Malden, MA.

Nathaniel Robinson, Raj Dabre, Ammon Shurtz, Rasul Dent, Onenamiyi Onesi, Claire Monroc, Loïc Grobol, Hasan Muhammad, Ash Garg, Naome Etori, Vijay Murari Tiyalya, Olanrewaju Samuel, Matthew Stutzman, Bismarck Odoom, Sanjeev Khudanpur, Stephen Richardson, and Kenton Murray. 2024. Kreyòl-MT: Building MT for Latin American, Caribbean and colonial African creole languages. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3083–3110, Mexico City, Mexico. Association for Computational Linguistics.

Nathaniel Robinson, Cameron Hogan, Nancy Fulda, and David R. Mortensen. 2022. Data-adaptive transfer learning for translation: A case study in Haitian and jamaican. In Proceedings of the Fifth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2022), pages 35–42, Gyeongju, Republic of Korea. Association for Computational Linguistics.

Nathaniel Romney Robinson, Matthew Dean Stutzman, Stephen D. Richardson, and David R Mortensen. 2023. African substrates rather than european lexifiers to augment african-diapora creole translation. In 4th Workshop on African Natural Language Processing.

Hedvig Skirgård, Hannah J. Haynie, Damían E. Blasi, Harald Hammarström, Jeremy Collins, Jay J. Latarche, Jakob Lesage, Tobias Weber, Alena Witzlack-Makarevich, Sam Passmore, Angela Chira, Luke Maurits, Russell Dinnage, Michael Dunn, Ger Reesink, Ruth Singer, Claire Bowern, Patience Epps, Jane Hill, Outi Vesakoski, Martine Robbeets, Noor Karolin Abbas, Daniel Auer, Nancy A. Bakker, Giulia Barbos, Robert D. Borges, Swinthia Danielsen, Luise Dorenbusch, Ella Dorn, John Elliott, Giada Falcone, Jana Fischer, Yustinus Ghanggo Ate, Hannah Gibson, Hans-Philipp Göbel, Jemima A. Goodall, Victoria Gruner, Andrew Harvey, Rebekah Hayes, Leonard Heer, Roberto E. Herrera Miranda, Natalia Hübler, Biu Huntington-Rainey, Jessica K. Ivani, Marilen Johns, Erika Just, Eri Kashima, Carolina Kipf, Janina V. Klingenberg, Nikita König, Aikaterina Koti, Richard G. A. Kowalik, Olga Krasnoukhova, Nora L.M. Lindvall, Mandy Lorenzen, Hannah Lutzenberger, Tonia R.A. Martins, Celia Mata German, Suzanne van der Meer, Jaime Montoya Samamé, Michael Müller, Saliha Muradoğlu, Kelsey Neely, Johanna Nickel, Milna Norvik, Cheryl Akinyi Olouoch, Jesse Peacock, India O.C. Pearcy, Naomi Peck, Stephanie Petit, Sören Pieper, Mariana Poblete, Daniel Prestipino, Linda Raabe, Amna Raja, Janis Reimringer, Sydney C. Rey, Julia Rizaew, Eloisa Ruppert, Kim K. Salmon, Jill Sammet, Rhiannon Schembri, Lars Schlabbach, Frederick W.P. Schmidt, Amalia Skilton, Wikalider Daniel Smith, Hilário de Sousa, Kristin Sverredal, Daniel Valle, Javier Vera, Judith Voß, Tim Witte, Henry Wu, Stephanie Yam, Jingting Ye, Maisie Yong, Tessa Yu-ditha, Roberto Zariquey, Robert Forkel, Nicholas Evans, Stephen C. Levinson, Martin Haspelmath, Simon J. Greenhill, Quentin D. Atkinson, and Russell D. Gray. 2023. Grambank reveals global patterns in the structural diversity of the world’s languages. Science Advances, 9.

Maali Tars, Andre Tättar, and Mark Fišel. 2021. Extremely low-resource machine translation for closely related languages. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 41–52, Reykjavík, Iceland (Online). Linköping University Electronic Press, Sweden.

Ahmet Üstün, Alexandre Berard, Laurent Besacier, and Matthias Gallé. 2021. Multilingual unsupervised neural machine translation with denoising adapters. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6650–6662, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Ahmet Üstün, Arianna Bisazza, Gosse Bouma, Gertjan van Noord, and Sebastian Ruder. 2022. Hyper-X: A unified hypernetwork for multi-task multilingual transfer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7934–7949, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Raúl Vázquez, Timothee Mickus, Jörg Tiedemann, Ivan Vulić, and Ahmet Üstün, editors. 2024. Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024). Association for Computational Linguistics, St Julians, Malta.

Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, and Hai Leong Chieu. 2017. Universal Dependencies parsing for colloquial Singaporean English. In Proceedings of the 55th Annual Meeting
of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1732–1744, Vancouver, Canada. Association for Computational Linguistics.

Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 23965–23998. PMLR.

Fei Yuan, Yinquan Lu, Wenhai Zhu, Lingpeng Kong, Lei Li, Yu Qiao, and Jingjing Xu. 2023. Lego-MT: Learning detachable models for massively multilingual machine translation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11518–11533, Toronto, Canada. Association for Computational Linguistics.

Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575, Austin, Texas. Association for Computational Linguistics.

Just Zwennicker and David Stap. 2022. Towards a general purpose machine translation system for sranantongo. arXiv preprint arXiv:2212.06383.

A Literature Review

NLP research for Creole languages is still in relatively early stages. Some large-scale multilingual MT systems have limited Creole language support: NLLB (NLLB Team et al., 2022) supports five Creole languages, and Google Translate3 and Lego-MT (Yuan et al., 2023) each support eight. Other works have focused specifically on building MT for Creole languages Haitian (Robinson et al., 2022; Callison-Burch et al., 2011; Frederking et al., 1997), Singlish (Wang et al., 2017; Liu et al., 2022b), Nigerian Pidgin (Adelani et al., 2022; Ogueji and Ahia, 2019), Mauritian Creole (Dabre and Sukhoo, 2022), Sranan Tongo (Zwennicker and Stap, 2022), Guyanese Creole (Clarke et al., 2024), and Jamaican Patois (Robinson et al., 2022). Some others have focused on building Creole language-specific NLP tools outside of MT for Nigerian Pidgin (Muhammad et al., 2023, 2022; Adelani et al., 2021; Oyewusi et al., 2020; Caron et al., 2019), Jamaican Patois (Armstrong et al., 2022), and Martinican Creole (Mompelat et al., 2022).

More recently, researchers have developed large scale multi-Creole-language NLP applications. Lent et al. (2024) developed CreoleVal, a multitask benchmark for Creoles including an MT dataset and model for 28 Creole languages. Robinson et al. (2024) expanded on Lent et al.’s (2024) work, building genre-diverse MT datasets and models for 41 Creole languages of the Caribbean, Latin America, and Colonial Africa. Perhaps most pertinent to our current work are past studies into cross-lingual transfer to benefit Creole NLP. In tightly controlled experiments, Robinson et al. (2023) found indications that Haitian and Jamaican Patois’ Niger-Congo relatives may be more helpful as transfer languages than their Indo-European relatives, for MT applications. Lent et al. (2022a), however, did not identify clear advantages from a language’s relatedness to a Creole language in terms of its usefulness in transfer learning. Given the sparse research and even sparser positive results in the area, optimal transfer learning for Creole NLP remains an open question.

B Data Preprocessing

The MadLad corpus is organized in series of internet-scraped documents, which the MadLad team cross-checks for noise (Kudugunta et al., 2023). For training the monolingual adapters, we use the MadLad-Clean subsplit and we perform further preprocessing, since we noted after manual inspection that noise can still be present in the documents. Firstly, the MadLad documents are split into sentences using the SentenceSplitter4 library.

For each sentence in the document, we define the following rules, if any of the rules is breached, then the given sentence is not included in the final monolingual training corpus.

1. Sentence ends with a terminal punctuation mark (’, ’!’, ’ ”’, ’?’).
2. Ratio of numerical characters with respect to all other characters is less than \(0.25\).
3. Sentence starting with a bullet-point.
4. Sentence containing a *lorem ipsum* substring.
5. Ratio of stop-words with respect to all other tokens is less than \(0.6\) using whitespace tokenization.
6. Ratio of token count that does not contain alpha characters with respect to total token count is less than \(0.25\), using whitespace tokenization.

3https://translate.google.com/ as of 8 Feb., 2025
4https://github.com/mediacloud/sentence-splitter
7. Sentence length is less than 14 and more than 300 characters.
8. Ratio of capitalized characters is less than 0.2.
9. Ratio of alpha characters with respect to all other characters is more than 0.6.
10. Language identification does not match the expected language. For this we employ FastText model 5.

This preprocessing removed approximately 20-23% of sentences, depending on the language. The same preprocessing steps were also applied to NLLB parallel data used for cross-attention fine-tuning, where preprocessing removed approximately 30-50% of the sentences.

C Selecting Languages for Transfer

Table 4 displays the different transfer languages we selected in our experiments. Below we detail our processes of selecting them.

C.1 Phylogeny-based Transfer

Faisal and Anastasopoulos (2022) followed principles of language phylogenetics to inform their transfer learning experiments, training adapters on combined Indo-European data, and for subfamilies of Indo-European such as Germanic, Romance, Indo-Aryan, etc. To translate for French, for example, they stacked a French adapter on top of a Romance and then an Indo-European adapter at inference time. Creole phylogeny is heavily disputed, however (Kouwenberg and Singler, 2009). Linguists have debated whether Atlantic Creole languages such as Haitian and Papiamento are best understood as Niger-Congo (NC) languages with extensive Indo-European (IE) lexical borrowings (Lefebvre, 2011; Rickford and McWhorter, 1997), or as IE languages that developed with strong NC influence (Faine, 1937; Hall, 1958; Hammarström et al., 2023; Mufwene, 2006). Considering this lack of consensus, we select two groups of related languages for each Creole language: IE relatives and NC relatives. In this work, we made a selection based on APICS (Michaelis et al., 2013) and linguistic references.

<table>
  <tr>
    <th>Condition</th>
    <th>Language</th>
  </tr>
  <tr>
    <th colspan="2">Haitian</th>
  </tr>
  <tr>
    <td>Creole Language</td>
    <td>hat</td>
  </tr>
  <tr>
    <td>IE Relative</td>
    <td>fr, es, en</td>
  </tr>
  <tr>
    <td>NC Relative</td>
    <td>ig, ee, fon</td>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>grc, oc, fo</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>pap, epo, ltz</td>
  </tr>
  <tr>
    <th colspan="2">Papiamento</th>
  </tr>
  <tr>
    <td>Creole Language</td>
    <td>pap</td>
  </tr>
  <tr>
    <td>IE Relative</td>
    <td>nl, pt, es</td>
  </tr>
  <tr>
    <td>NC Relative</td>
    <td>kg, ak, fon</td>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>suz, sr, ug</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>hat, war, tgl</td>
  </tr>
  <tr>
    <th colspan="2">Sango</th>
  </tr>
  <tr>
    <td>Creole Language</td>
    <td>sag</td>
  </tr>
  <tr>
    <td>IE Relative</td>
    <td>fr</td>
  </tr>
  <tr>
    <td>NC Relative</td>
    <td>kbp</td>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>ig, yo, ca</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>kg, tsn, lin</td>
  </tr>
  <tr>
    <td>Subword Evenness</td>
    <td>kat, rus, zsm</td>
  </tr>
  <tr>
    <th colspan="2">Controls</th>
  </tr>
  <tr>
    <td>Uralic</td>
    <td>fin, est, hun</td>
  </tr>
  <tr>
    <td>CJK</td>
    <td>zho, jap, kor</td>
  </tr>
  <tr>
    <td>Dravidian</td>
    <td>te, ta, ka</td>
  </tr>
  <tr>
    <th colspan="2">Other Settings</th>
  </tr>
  <tr>
    <th colspan="2">Hungarian</th>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>fin, tur, als</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>fin, bul, hrv</td>
  </tr>
  <tr>
    <th colspan="2">Yoruba</th>
  </tr>
  <tr>
    <td>lang2vec</td>
    <td>tha, ibo, fon</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>ibo, hau, xho</td>
  </tr>
</table>

Table 4: All languages used for the various experimental settings.

5https://huggingface.co/facebook/fasttext-language-identification
and Haspelmath, 2013), and Grambank (Skirgård et al., 2023). The lang2vec library (Littell et al., 2017) assembles typological feature vectors from different sources, enabling measurement of holistic typological distance between languages. The drawbacks of lang2vec include the lack of equal coverage of languages in terms of the number of features, and the coarseness of the linguistic properties (Ponti et al., 2019). To supplant the lack of coverage, we use data2lang2vec which uses textual data to inform feature-prediction of the missing features (Amirzadeh et al., 2025).

C.3 Data-driven Language Selection Methods

NLLB Language Embeddings Typological similarity between languages can also be derived in a data-driven manner, e.g., from language embeddings. To derive the similarity between languages this way, we utilize the language embeddings of the distilled NLLB-200 model which implicitly contain much information with regards to the various languages especially with regards to morphosyntax and vocabulary.

Subword Evenness Introduced by Pelloni et al. (2022), Subword Evenness (SuE) is a metric quantifying the differences in the length of subword units for each individual language. It presents another data-driven approach to transfer language selection. In this metric, lower SuE scores have been demonstrated as a good predictor of transfer success to various low-resource languages. Thus, we select the 5 languages in the MADLAD dataset with the lowest SuE angles as calculated when using a BPE tokenizer, see Table 5.

<table>
  <tr>
    <th>Language Code</th>
    <th>Language Code</th>
    <th>Script</th>
    <th>SuE Angle</th>
  </tr>
  <tr>
    <td>-azb</td>
    <td>South Azerbaijani</td>
    <td>Arabic</td>
    <td>66.72</td>
  </tr>
  <tr>
    <td>kat</td>
    <td>Georgian</td>
    <td>Georgian</td>
    <td>72.57</td>
  </tr>
  <tr>
    <td>zsm</td>
    <td>Malay</td>
    <td>Latin</td>
    <td>79.80</td>
  </tr>
  <tr>
    <td>-kas</td>
    <td>Kashmiri</td>
    <td>Devanagari</td>
    <td>80.79</td>
  </tr>
  <tr>
    <td>rus</td>
    <td>Russian</td>
    <td>Cyrillic</td>
    <td>81.21</td>
  </tr>
</table>

Table 5: MADLAD Languages with the lowest SuE angles calculated using a BPE tokenizer. We note that MADLAD only has Norther Azerbaijani (Latin script), and not South Azerbaijani (Arabic script). Similarly, Kashmiri is only in MADLAD with the Arabic script, not with Devanagari.

C.4 Control Experiments with Unrelated Languages

To determine whether improvements over Creoles are truly the result of successful cross-lingual transfer, we compare the previously described methods against an equivalent adapter approach using entirely unrelated languages. We use three different sets of such unrelated languages. The first set is made up of the Uralic languages Finnish, Estonian, and Hungarian, which are written with the Latin script but have no phylogenetic relation to Haitian, Papiamento, and Sango. The second set consists of Chinese, Japanese, and Korean – so-called CJK languages –, which belong to three distinct language families and are written with three distinct scripts having little to no overlap with Atlantic and African Creole languages. The third group consists of the Dravidian languages Tamil, Telugu, and Kannada, which are closely related to each other but also written with three distinct scripts, and again are unrelated to the Creole languages in terms of phylogeny.

D Full Results

See Table 6 for our results translating between English and Creole languages using only 800 fine-tuning parallel segments.

E Training Loss

Table 7 compares loss when evaluating on a small subset of the NLLB CA-FT training data. Figure 3 indicates that the adapters we train learn successfully from the MADLAD-400 data on a selection from all language adapters.

F Native Speaker Evaluation

See Table 9 for a condensed categorization of translation errors under the Untrained Souping (🞂), the IE Transfer (🞃), and the s and t Adapters (s + t) conditions.
![Training loss curves for a subset of adapter training experiments.](page_324_671_1092_563.png)

Figure 3: Training loss curves for a subset of adapter training experiments.
<table>
  <tr>
    <th>Experiments</th>
    <th>en → hat</th>
    <th>en → pap</th>
    <th>en → sag</th>
    <th>hat → en</th>
    <th>pap → en</th>
    <th>sag → en</th>
  </tr>
  <tr>
    <th colspan="7">Baselines</th>
  </tr>
  <tr>
    <td>dNLLB-200 w/ CA-FT</td>
    <td>23.59</td>
    <td>23.56</td>
    <td>11.16</td>
    <td>33.36</td>
    <td>39.05</td>
    <td>10.62</td>
  </tr>
  <tr>
    <th colspan="7">Language Adapters</th>
  </tr>
  <tr>
    <td>Source + Target LA</td>
    <td>23.55</td>
    <td>26.82</td>
    <td>11.90</td>
    <td>31.18</td>
    <td>38.23</td>
    <td>12.19</td>
  </tr>
  <tr>
    <th colspan="7">Phylogeny-Inspired Adaptation</th>
  </tr>
  <tr>
    <td>Target + IE Relatives</td>
    <td>24.01</td>
    <td>29.35</td>
    <td>11.94</td>
    <td>35.24</td>
    <td>45.48</td>
    <td>13.51</td>
  </tr>
  <tr>
    <td>Target + NC Relatives</td>
    <td>23.98</td>
    <td>29.71</td>
    <td>11.93</td>
    <td>34.96</td>
    <td>46.13</td>
    <td>13.03</td>
  </tr>
  <tr>
    <th colspan="7">Typological Adaptation</th>
  </tr>
  <tr>
    <td>1lang2vec</td>
    <td><b>24.28</b></td>
    <td><b>30.21</b></td>
    <td>11.71</td>
    <td><b>35.63</b></td>
    <td><b>46.95</b></td>
    <td>13.79</td>
  </tr>
  <tr>
    <td>Target + Data-Driven (NLLB)</td>
    <td>24.09</td>
    <td>30.17</td>
    <td><b>12.01</b></td>
    <td>34.75</td>
    <td>46.09</td>
    <td>13.48</td>
  </tr>
  <tr>
    <th colspan="7">Miscellaneous</th>
  </tr>
  <tr>
    <td>Creole Transfer</td>
    <td>23.99</td>
    <td>29.87</td>
    <td>11.71</td>
    <td>34.24</td>
    <td>44.75</td>
    <td>13.48</td>
  </tr>
  <tr>
    <td>Subword Evenness</td>
    <td>24.23</td>
    <td>29.73</td>
    <td>11.80</td>
    <td>35.20</td>
    <td>46.20</td>
    <td><b>14.11</b></td>
  </tr>
  <tr>
    <th>Largest Improvement from dNLLB</th>
    <td>+0.69</td>
    <td>+0.65</td>
    <td>+0.85</td>
    <td>+2.27</td>
    <td>+7.90</td>
    <td>+3.49</td>
  </tr>
  <tr>
    <th>Largest Improvement from Üstün</th>
    <td>+0.73</td>
    <td>+3.39</td>
    <td>+0.11</td>
    <td>+4.45</td>
    <td>+8.63</td>
    <td>+1.92</td>
  </tr>
  <tr>
    <th colspan="7">Control</th>
  </tr>
  <tr>
    <td>Uralic</td>
    <td>24.31</td>
    <td>30.08</td>
    <td>11.44</td>
    <td><b>36.01</b></td>
    <td>46.36</td>
    <td><b>15.03</b></td>
  </tr>
  <tr>
    <td>CJK</td>
    <td>23.95</td>
    <td>29.18</td>
    <td>11.60</td>
    <td>35.64</td>
    <td>46.91</td>
    <td>14.71</td>
  </tr>
  <tr>
    <td>Dravidian</td>
    <td>24.21</td>
    <td>29.17</td>
    <td>11.69</td>
    <td>35.51</td>
    <td>46.66</td>
    <td>14.45</td>
  </tr>
  <tr>
    <td>Random (same init) (1 : 1)</td>
    <td>24.21</td>
    <td>28.89</td>
    <td>11.86</td>
    <td>36.07</td>
    <td>46.03</td>
    <td>14.79</td>
  </tr>
  <tr>
    <td>Random (same init) (1 : 3)</td>
    <td>24.10</td>
    <td>29.65</td>
    <td>11.55</td>
    <td>37.10</td>
    <td>46.66</td>
    <td>15.39</td>
  </tr>
  <tr>
    <td>Only random (same init)</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>36.84</td>
    <td>45.01</td>
    <td>14.90</td>
  </tr>
</table>

Table 6: Mean BLEU scores on Flores-200 (after 1 run). Results follow adapter training defaults used in the Üstün et al. (2021) paper. Encoder and decoder adapters are trained together and trained further using cross-attention fine-tuning (CA-FT), and helper and target language adapters are averaged (souped) together. These scores represent preliminary results, where significantly less data (800 segments) were used to do CA-FT, and thus these results do not correspond to Table 2.

<table>
  <tr>
    <th>Experiment</th>
    <th>hat → eng</th>
    <th>pap → eng</th>
    <th>sag → eng</th>
  </tr>
  <tr>
    <td>s and t Adapters</td>
    <td>0.594</td>
    <td>1.106</td>
    <td>1.908</td>
  </tr>
  <tr>
    <td>Untrained s Adapter</td>
    <td><b>0.476</b></td>
    <td><b>0.917</b></td>
    <td><b>1.796</b></td>
  </tr>
</table>

Table 7: Validation loss when evaluating on the CA-FT training data. Averaging with the weights with the same initialization results in lower loss values.
<table>
  <tr>
    <th>Experiments</th>
    <th>en → hu</th>
    <th>en → yo</th>
    <th>hu → en</th>
    <th>yo → en</th>
  </tr>
  <tr>
    <td>dNLLB-200</td>
    <td>22.59</td>
    <td>8.85</td>
    <td>29.51</td>
    <td>18.82</td>
  </tr>
  <tr>
    <td>1ang2vec</td>
    <td>21.85</td>
    <td><b>8.51</b></td>
    <td>33.38</td>
    <td>19.27</td>
  </tr>
  <tr>
    <td>NLLB</td>
    <td>21.75</td>
    <td>8.45</td>
    <td>33.31</td>
    <td>19.14</td>
  </tr>
  <tr>
    <td>SuE</td>
    <td><b>21.90</b></td>
    <td>8.37</td>
    <td>33.47</td>
    <td>19.56</td>
  </tr>
  <tr>
    <td>Random (init) (1 : 3)</td>
    <td>21.88</td>
    <td>8.47</td>
    <td><b>33.97</b></td>
    <td><b>20.70</b></td>
  </tr>
  <tr>
    <td>Difference</td>
    <td><b>−0.69</b></td>
    <td><b>−0.34</b></td>
    <td><b>+4.46</b></td>
    <td><b>+1.88</b></td>
  </tr>
</table>

Table 8: Mean BLEU scores with our methodology, as applied to Yoruba and Hungarian. These scores represent preliminary results, where significantly less data (800 segments) were used to do CA-FT, and thus not directly comparable to Table 2.

<table>
  <tr>
    <th>Error</th>
    <th>🞄</th>
    <th>🞅</th>
    <th>s + t</th>
    <th>Notes</th>
  </tr>
  <tr>
    <th colspan="5">Grammar Issues</th>
  </tr>
  <tr>
    <td>yo rann yo</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>Aspectual issue: English translation is passive, Creole is a voluntary activity</td>
  </tr>
  <tr>
    <td>te eksplore</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>Aspectual error: English passive which also affects the sentence meaning</td>
  </tr>
  <tr>
    <td>"many beautiful buildings to look at"</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and s + t ungrammatical</td>
  </tr>
  <tr>
    <td>"improve a considerable amount"</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and s + t ungrammatical</td>
  </tr>
  <tr>
    <td>"often provide some"</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 ungrammatical</td>
  </tr>
  <tr>
    <td>fatigé mwens postib</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 omits it; 🞅 and s + t to "fatigued less possibly"</td>
  </tr>
  <tr>
    <td>konesans yo bay la</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>to "they give/provide" with no clear subject (passive would be better)</td>
  </tr>
  <tr>
    <td>mannyè efikas</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>missing preposition "in" ("in efficient manner")</td>
  </tr>
  <tr>
    <th colspan="5">Translation Issues</th>
  </tr>
  <tr>
    <th colspan="5">Lexical errors</th>
  </tr>
  <tr>
    <td>nimevotation</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>s + t wrongly translates it to "naming system"</td>
  </tr>
  <tr>
    <td>benevola</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>"travel benevola" or "benevola travel" (in fact "charity")</td>
  </tr>
  <tr>
    <td>charabya</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>Means "nonsense", "gibberish"</td>
  </tr>
  <tr>
    <td>linets</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 to "lenses"; 🞅 to "links"; s + t not translating</td>
  </tr>
  <tr>
    <td>dyèl</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 to "deals"; 🞅 correctly; s + t to "dyed"</td>
  </tr>
  <tr>
    <td>patikilarite</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>s + t to "patikiliarity"</td>
  </tr>
  <tr>
    <td>kotidyen</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>s + t not translated</td>
  </tr>
  <tr>
    <td>setalo</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 to "several"; 🞅 to "most"; s + t to "satellite" (in fact "sometimes")</td>
  </tr>
  <tr>
    <td>bildinn</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 to "image"; 🞅 to "figure"; s + t to "picture" (in fact "building")</td>
  </tr>
  <tr>
    <td>batiman</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and 🞅 to "vessels"; s + t to "baths" (in fact "building")</td>
  </tr>
  <tr>
    <td>mannyè</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 correctly to "manner"; 🞅 and s + t to "manny"</td>
  </tr>
  <tr>
    <td>touch</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>translated to "touch" though it means "piano key"</td>
  </tr>
  <tr>
    <td>founi militè</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and 🞅 to "military foam"; s + t to "military fountains" (in fact "army ants")</td>
  </tr>
  <tr>
    <td>pyèj</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 to "hostage"; 🞅 and s + t to "pigeon" (in fact "trap")</td>
  </tr>
  <tr>
    <td>jako</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and 🞅 to "jaco"; s + t omitted/misinterpreted (in fact "parrot")</td>
  </tr>
  <tr>
    <td>eskrimè</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and s + t to "skirmish"; 🞅 to "skater" (in fact "fencer")</td>
  </tr>
  <tr>
    <td>parese</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>to "parasites" (in fact "sloths")</td>
  </tr>
  <tr>
    <td>pwomennen</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>to "promoted" (in fact "walking")</td>
  </tr>
  <tr>
    <td>eta espiri</td>
    <td>✓</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and 🞅 correctly to "state of mind"; s + t to "spirit state"</td>
  </tr>
  <tr>
    <td>Premye/Twazèm Group</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>to "First/Third Group" (in fact "First/Third Class")</td>
  </tr>
  <tr>
    <td>mache</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>to "walk" (in fact "migrate" in context)</td>
  </tr>
  <tr>
    <th colspan="5">Other errors</th>
  </tr>
  <tr>
    <td>kote pasaj enpótan yo</td>
    <td>x</td>
    <td>x</td>
    <td>x</td>
    <td>🞄 and 🞅 too literal; s + t also adding "their" instead of "the"</td>
  </tr>
</table>

Table 9: Condensed categorization errors under various translation conditions.🞄 refers to Untrained Souping,🞅 indicates IE Transfer, and s + t means s and t Adapters.

Reviewer paper summaries: They propose a method for MT for low-resource language using and adapter souping(adapter weight averaging across domains) with cross attention finetuning. 
Then, using control experiments on 3 Creole languages without linguistic relations to the original experiment Creole languages, they demonstrate that cross-lingual transfer does not affect the high bleu score of the proposed methodology, as the control experiments' performance is comparable to the results of the linguistically relevant languages.
Summary of strengths: Their proposed method based on cross-lingual transfer surpasses baselines on Bleu score in some settings.
They perform various experiments to confirm that adapters are not meaningfully transferring linguistic knowledge, but more performing as regularizers in the fine-tuning process.
Summary of weaknesses: N/A
Comments, suggestions and typos: I preferred to see an ablation study on the components of the proposed method as the first contribution is it's success on surpassing baselines on 8 Bleu scores.
Overall assessment scores: 3
Metareview score: 3.5
Metareview: The paper proposes a combination of adapter soups and cross-attention fine-tuning for transferring an MT model to low-resource languages. Analysis and ablations of their methods yields the result that linguistic similarity is not what contributes to transfer, and the authors hypothesize (and argue) that it’s more due to parameter regularization. The paper is a nice mixture of contributions to low-resource adaptation, as well as analysis and interpretability of what makes them work, leaving interesting avenues for future work in the community.
Summary of reasons to publish: The paper is clear and well-motivated. The authors engage both with a practical contribution (adapter souping with cross-attention fine-tuning) as well as with a more general contribution of why adapters work, balancing the ideas that drive the paper between practical methods and linguistic analyses.