Can You Traducir This? Machine Translation for Code-Switched Input (2021)

Abstract: Code-Switching (CSW) is a common phenomenon that occurs in multilingual geographic or social contexts, which raises challenging problems for natural language processing tools. We focus here on Machine Translation (MT) of CSW texts, where we aim to simultaneously disentangle and translate the two mixed languages. Due to the lack of actual translated CSW data, we generate artificial training data from regular parallel texts. Experiments show this training strategy yields MT systems that surpass multilingual systems for code-switched texts. These results are confirmed in an alternative task aimed at providing contextual translations for a L2 writing assistant.

## introduction code-switching (csw) denotes the alternation of two languages within a single utterance @xcite . it is a common communicative phenomenon that occurs in multilingual communities during spoken and written interactions. csw is a well studied phenomenon in linguistic circles and has given rise to a number of theories regarding the structure of mixed language fragments @xcite @xcite @xcite . the matrix language frame (mlf) theory @xcite defines the concept of matrix and embedded languages where the matrix language is the main language that the sentence structure should conform to and notably provides the syntactic morphemes, while the influence of the embedded language is lesser and is mostly manifested in the insertion of content morphemes. the rise of social media and user-generated content has made written instances of code-switched language more visible. it is estimated that as much as 17% of indian facebook posts @xcite and 3.5% of all tweets @xcite are code-switched. this phenomenon is also becoming more pervasive in short text messages, chats, blogs, and the like @xcite . code-switching however remains understudied in natural language processing (nlp) @xcite , and most work to date has focused on token-level language identification (lid) @xcite and on language models for automatic speech recognition @xcite . more tasks are being considered lately, such as named entity recognition @xcite , part-of-speech tagging @xcite or sentiment analysis @xcite . we focus here on another task for csw texts: machine translation (mt). the advent of neural machine translation (nmt) technologies @xcite has made it possible to design multilingual models capable of translating from multiple source languages into multiple target languages @xcite , where however both the input and output are monolingual. we study here the ability of such architectures to translate fragments freely mixing a "matrix" and an "embedded" language into monolingual utterances. our main contribution is to show that for the two pairs of languages considered (french-english and spanish-english): (a) translation of csw texts is almost as good as the translation of monolingual texts -a performance that bilingual systems are unable to match; (b) such results can be obtained by training solely with artificial data; (c) csw translation systems achieve a near deterministic ability to recopy in the output target words found in the input, suggesting that they are endowed with some language identification abilities. using these models, we are also able to obtain competitive results on the semeval 2014 task 5: l2 writing assistant, which we see as one potential application area of csw translation. ## computing translations in context in this section, we evaluate csw translation for the semeval 2014 task 5: l2 writing assistant @xcite , which can be handled as an mt task from mixed data. ## related work ## conclusion and outlook in this study, we present a data augmentation method to generate artificial csw data. we have shown that artificial data generated could be used to train nmt systems to translate both monolingual and csw sentences (in one or even two different languages). with joint training of the two languages, we were able to build systems that were as good as a baseline bilingual system on monolingual texts, and much better for csw texts. our system does not need any explicit language identification and almost perfectly sorts out source tokens from target tokens in a csw utterance. another interesting feature of our system is that it always output monolingual translations. we finally report state-of-the-art results for the semeval l2 writing assistant task for es-en, while the related results for fr-en are still somewhat lagging behind the best scores. in the future, we would like to generate more realistic csw data from monolingual sentences using a translation model. we also plan to explore ways to translate csw texts simultaneously into both languages, so that the two decoding processes can mutually influence one another: in a first step in that direction, we have shown that training with a joint loss was actually beneficial for the translation into the two languages. another line of research would be to continue experimenting with realistic language data, also containing other phenomena such as morphological binding. finally, we also intend to study the somewhat more realistic condition where a mixture of languages a and b is translated into language c; we believe that the artificial csw generation methods developed in our work would also be effective for this task.