Data Augmentation Techniques for Machine Translation of Code-Switched Texts: A Comparative Study (2023)

Abstract: Code-switching (CSW) text generation has been receiving increasing attention as a solution to address data scarcity. In light of this growing interest, we need more comprehensive studies comparing different augmentation approaches. In this work, we compare three popular approaches: lexical replacements, linguistic theories, and back-translation (BT), in the context of Egyptian Arabic-English CSW. We assess the effectiveness of the approaches on machine translation and the quality of augmentations through human evaluation. We show that BT and CSW predictive-based lexical replacement, being trained on CSW parallel data, perform best on both tasks. Linguistic theories and random lexical replacement prove to be effective in the lack of CSW parallel data, where both approaches achieve similar results.

## introduction code-switching (csw) is the alternation of language in text or speech, which can occur across different levels of granularity: sentences, words and morphemes. csw is a common phenomenon in arabic-speaking countries, as in other multilingual communities. given that arabic is a morphologically rich language @xcite , speakers produce morphological csw, as illustrated below: algorithm+ implement+ ⇐ 'okay, i'll+implement the+algorithm right away' csw introduces a set of challenges to nlp systems, not least of which is data scarcity. this is attributed to csw being a predominantly spoken phenomenon, only recently increasing in written form on social media. data augmentation has proved to be a successful workaround for this limitation. researchers have investigated several techniques for csw data augmentation, including learning csw points @xcite , lexical replacements @xcite @xcite , linguistic theories @xcite @xcite , neural-based approaches @xcite @xcite @xcite , and machine translation (mt) @xcite . with increasing efforts in this area, we need more comparative studies to better understand the merits and requirements of different approaches. efforts along these lines include the work of @xcite , where different linguistic-driven and lexical replacement techniques were compared through human evaluation, but not for nlp tasks. @xcite propose the use of pointer-generator network and compare it against the equivalence constraint (ec) theory @xcite and random lexical replacement for lm, without human evaluation. @xcite compare multiple lexical replacement techniques covering human evaluation and performance on language modeling (lm), automatic speech recognition (asr), mt, and speech translation. @xcite compare using the ec theory and random lexical replacement for lm and asr, also reporting human assessments. in this work, we compare three main approaches: lexical replacements, linguistic theories, and back-translation (bt). we evaluate the approaches for both naturalness of csw generations and performance on mt, where we focus on csw egyptian arabic-english to english translation. the rationale for our focus on mt is the scarcity of work around data augmentation as opposed to lm and asr. furthermore, previous work on mt focuses on lexical replacements @xcite @xcite @xcite and bt @xcite , without substantial comparison between approaches. through our comparative study, we provide answers to the following research questions: we provide an overview on the investigated techniques. 1 our aim is to augment arabic-to-english parallel sentences, converting the source side of the parallel data from monolingual arabic to csw arabic-english, further extending the mt training data with csw instances. in figure 1 , we provide an example showing possible augmentations across techniques. more examples are shown in ## evaluation in this section, we present intrinsic evaluation, human evaluation, and extrinsic evaluation. ## discussion in this section, we revisit our rqs: rq1 -which augmentation techniques perform the best for mt? in the zero-shot setting, lex rand and linguistic theories achieve similar performance, with ec outperforming ml models. in the non-zero shot setting, bt outperforms all techniques, followed by lex p red . both techniques, being trained on real csw data, are able to generate more natural csw sentences, that could also be closer in csw style to arzen-st. ## insights into augmentations in this section, we present insights into the augmentations produced by the different techniques, further elaborating on their strengths and weaknesses. all examples mentioned in this section refer to the examples demonstrated in lexical replacements: the main drawback in lex dict is that the replaced words might not be correct translations within context, which can negatively affect the mt model. as shown in ## conclusion and future work we present a comparative study between different csw data augmentation techniques and their effectiveness for mt in both zero-shot and non-zeroshot settings. we show that in the zero-shot setting, random lexical replacement performs equally well as linguistic theories. in the case of non-zero shot setting, back-translation performs best, followed by csw predictive-based lexical replacement. both approaches also stand out in human evaluation, where we confirm a positive correlation between naturalness of augmentations and mt performance. however, both approaches are reliant on expensive and limited csw parallel data. overall, the set of approaches examined proves useful in alleviating data scarcity. each approach comes with particular merits and requirements, guiding the choice for different research needs. in future work, we plan on enhancing the back-translation approach to leverage larger amounts of english data. in parallel, we will investigate the effectiveness of generative ai to broaden the benchmark of approaches, and expand our study to cover other nlp tasks.