Data-adaptive Transfer Learning for Translation: A Case Study in H aitian and Jamaican (2022)

Abstract: Multilingual transfer techniques often improve low-resource machine translation (MT). Many of these techniques are applied without considering data characteristics. We show in the context of Haitian-to-English translation that transfer effectiveness is correlated with amount of training data and relationships between knowledge-sharing languages. Our experiments suggest that for some languages beyond a threshold of authentic data, back-translation augmentation methods are counterproductive, while cross-lingual transfer from a sufficiently related language is preferred. We complement this finding by contributing a rule-based French-Haitian orthographic and syntactic engine and a novel method for phonological embedding. When used with multilingual techniques, orthographic transformation makes statistically significant improvements over conventional methods. And in very low-resource Jamaican MT, code-switching with a transfer language for orthographic resemblance yields a 6.63 BLEU point advantage.

## introduction and motivation machine translation (mt) for low resource languages (lrl) requires special attention due to data scarcity. often lrl mt is aided by knowledge transfer from languages with more abundant resources @xcite @xcite . in this work we report a case study showing that transfer techniques based on back-translation can improve poor scores in very low-resource settings, but they can be counterproductive with more abundant authentic data. we demonstrate this in the case of a lrl for which augmentation data in the same genre as authentic data is not available. we show that in some settings where authentic data amount renders back-translation less effective, multi-source mt methods @xcite are more reliable to make incremental improvements. in these settings, mt systems map from a small amount of data in a lrl and a larger amount of data in a related high resource language (hrl) to a target language (tgt), in order to improve lrl-to-tgt translation quality. (see §2.1.) in addition to applying these methods conventionally, we present novel techniques for harnessing syntactic, orthographic, and phonological similarities between source languages lrl and hrl. prior to training, we employ multiple tools to transform hrl data to resemble lrl orthography and syntax by harnessing language relatedness. for phonologically similar languages, we present novel phonological word embeddings via panphon @xcite and use these to initialize mt models to facilitate a model's learning the lrl from the hrl. we conduct these experiments in a case study of haitian-to-english mt. we also contribute a rulebased french-haitian (fra-hat) orthographic and syntactic engine that transforms french to haitian text with 59.5% character error rate (cer) and 1.60 bleu @xcite on a singlereference set of 50 sentences. to demonstrate how these techniques can be applied to other lrl, we adapt these strategies to jamaican and show significant improvements over baseline performance, including improvements of up to 6.63 bleu points. our findings suggest that despite back-translation's reputation for usefulness in some settings, it cannot result in usable mt in others, in which case other transfer methods are needed for further, albeit marginal, improvement. ## related work and approach we are not the first researchers to explore haitianto-english mt. frederking et al. (1998) developed early statistical systems for haitian mt and automatic speech recognition. in 2010 a devastating earthquake in haiti's capital caused a global humanitarian disaster. this catastrophe renewed international interest in haitian mt systems for disaster relief efforts, the deployment of which was a "widely heralded success story" @xcite . ## methodology and experiments our experiments use a hat-eng bitext with 189,182 aligned sentence pairs (lrl-eng) and a fra-eng bitext with 315,577 (hrl-tgt). these data come from broadcasts and literature produced by the church of jesus christ of latter-day saints, with small additions from opus foot_0 . because of overlap between the english portions of these two bitexts, we have an implicit fra-hat bitext of length 77,121. we have a large monolingual eng corpus of text from wikipedia, the toronto book corpus @xcite , and text scraped from reddit. this monolingual augmentation data is not the same genre as the authentic aligned text. this setting is not ideal for back-translation, but it is meant to represent the realistic circumstance that no augmentation data in the authentic text genre is available, which may be the case for many low-resource languages. all our models are attention-based @xcite , adapted from the annotated transformer @xcite , and trained using the adam optimizer (kingma and ba, 2017). hyperparameters are detailed in appendix a.1 because we are comparing data sets produced with different transfer methods, rather than model architecture or configuration, we used these same settings for all experiments. we outline our methodology for the established methods of back-translation and multi-source training ( §3.1 and §3.2) and then for our novel methods of linguistic transfer ( §3.3). ## results figure 2 shows translation performance scores across a progression of back-translation-based augmentation as discussed in §3.1. these techniques improve performance when the amount of authentic data is very small. but once it crosses a threshold, they become counter-productive. we do not identify the exact threshold, since we performed these experiments as a case study, and such a threshold would certainly vary, depending on the source language and training data genre. our objective here is to illustrate a conceivable setting in which backtranslation augmentation can hurt mt performance. in such circumstances, we note that there exist essource bleu bleurt hat 43.94* .6810* hat+fra 46.05* .7026* hat+spa 46.51* .7065 hat+fra+spa 46.41* .7131* hat+jpn 30.41 -.1554 results for multilingual source training experiments are in ## rapid adaptation to new languages we seek to apply these principles of orthographic, syntactic, and phonological transfer rapidly to new languages by exploring another case study: jamaican. jamaican (jam) is an even lower-resource language than haitian, with only 3.2 million native speakers foot_2 . we experiment with syntactic transfer in jamto-eng translation. in these experiments we used haitian in the hrl role because it is close to jamaican syntactically but distant from it in terms of lexicon and orthography. results in the top of as mentioned in §3.3, our method for phonological embedding is readily applicable to other languages. to apply it to jamaican, we created a new jamaican setting in epitran via 37 mapping rules. this step would be unnecessary, however, for adaption to any of the 77 languages supported by epitran. we applied phonological transfer in jam-to-fra translation, where we used english as the hrl because it is phonologically close to jamaican. results from phonological embedding in the bottom of in the absense of a rule-based orthographic automatic transliterator from english to jamaican, we sought to imitate the effects of orthographic transfer via code-switching. this is a method employed in multiple past works @xcite @xcite , however all of them employ code-switching by replacing source language (lrl) words with target langauge (tgt) words. in our experiments, we replace english (hrl) words with jamaican (lrl) words using a dictionary of 200 jamaican words with english translations. this causes the english augmentation text to resemble jamaican orthography more closely. of the methods we attempted to improve jam-to-fra translation, this was the most successful. as shown in the bottom of ## conclusion although back-translation transfer methods are effective in some mt settings, in others they are unable to improve mt performance beyond a threshold or result in usable translation. per our explorations, methods involving multilingual transfer from a hrl during training are able to make further improvements, even when more abundant authentic data yields higher baseline performance. in our experiments, employing strategies to transfer orthographic and syntactic information from the hrl outperform methods to transfer phonological information or no specific information. our experiments on haitian mt indicate the potential for future improvements and broad social impact. and our exploration of jamaican demonstrates the capacity of these techniques for rapid adaptation to new settings and improvements in low-resource domains more generally.