C o M e T : Towards Code-Mixed Translation Using Parallel Monolingual Sentences (2021)

Abstract: Code-mixed languages are very popular in multilingual societies around the world, yet the resources lag behind to enable robust systems on such languages. A major contributing factor is the informal nature of these languages which makes it difficult to collect code-mixed data. In this paper, we propose our system for Task 1 of CACLS 2021 to generate a machine translation system for English to Hinglish in a supervised setting. Translating in the given direction can help expand the set of resources for several tasks by translating valuable datasets from high resource languages. We propose to use mBART, a pre-trained multilingual sequence-to-sequence model, and fully utilize the pre-training of the model by transliterating the roman Hindi words in the code-mixed sentences to Devanagri script. We evaluate how expanding the input by concatenating Hindi translations of the English sentences improves mBART’s performance. Our system gives a BLEU score of 12.22 on test set. Further, we perform a detailed error analysis of our proposed systems and explore the limitations of the provided dataset and metrics.

## introduction code-mixing 2 is the mixing of two or more languages where words from different languages are interleaved with each other in the same conversation. it is a common phenomenon in multilingual societies across the globe. in the last decade, due to the increase in the popularity of social media and various online messaging platforms, there has been an increase in various forms of informal writing, such as emojis, slang, and the usage of code-mixed languages. due to the informal nature of code-mixing, codemixed languages do not follow a prescriptively defined structure, and the structure often varies with the speaker. nevertheless, some linguistic constraints @xcite have been proposed that attempt to determine how languages mix with each other. given the increasing use of code-mixed languages by people around the globe, there is a growing need for research related to code-mixed languages. a significant challenge to research is that there are no formal sources like books or news articles in code-mixed languages, and studies have to rely on sources like twitter or messaging platforms. another challenge with hinglish, in particular, is that there is no standard system of transliteration for hindi words, and individuals provide a rough phonetic transcription of the intended word, which often varies with individuals. in this paper, we describe our systems for task 1 of calcs 2021, which focuses on translating english sentences to english-hindi code-mixed sentences. the code-mixed language is often called hinglish. it is commonly used in india because many bilingual speakers use both hindi and english frequently in their personal and professional lives. the translation systems could be used to augment datasets for various hinglish tasks by translating datasets from english to hinglish. an example of a hinglish sentence from the provided dataset (with small modifications) is shown below: we propose to fine-tune mbart for the given task by first transliterating the hindi words in the target sentences from roman script to devanagri script to utilize its pre-training. we further translate the english input to hindi using pre-existing models and show improvements in the translation using parallel sentences as input to the mbart model. the code for our systems, along with error analysis, is public foot_0 . the main contributions of our work are as follows: the rest of the paper is organized as follows. we discuss prior work related to code-mixed language processing, machine translation, and synthetic generation of code-mixed data. we describe our translation systems and compare the performances of our approaches. we discuss the amount of codemixing in the translations predicted by our systems and discuss some issues present in the provided dataset. we conclude with a direction for future work and highlight our main findings. ## background code-mixing occurs when a speaker switches between two or more languages in the context of the same conversation. it has become popular in multilingual societies with the rise of social media applications and messaging platforms. in attempts to progress the field of code-mixed data, several code-switching workshops @xcite @xcite have been organized in notable conferences. most of the workshops include shared tasks on various of the lan-guage understanding tasks like language identification @xcite , ner @xcite , ir @xcite , pos tagging @xcite , sentiment analysis @xcite , and question answering @xcite . although these workshops have gained traction, the field lacks standard datasets to build robust systems. the small size of the datasets is a major factor that limits the scope of code-mixed systems. machine translation refers to the use of software to translate text from one language to another. in the current state of globalization, translation systems have widespread applications and are consequently an active area of research. neural machine translation has gained popularity only in the last decade, while earlier works focused on statistical or rule-based approaches. @xcite first proposed a dnn model for translation, following which transformerbased approaches @xcite have taken the stage. some approaches utilize multilingual pre-training @xcite @xcite ; however, these works focus only on monolingual language pairs. although a large number of multilingual speakers in a highly populous country like india use english-hindi code-mixed language, only a few studies @xcite @xcite have attempted the problem. enabling translation systems in the following pair can bridge the communication gap between several people and further improve the state of globalization in the world. synthetic code-mixed data generation is a plausible option to build resources for code-mixed language research and is a very similar task to translation. while translation focuses on retaining the meaning of the source sentence, generation is a simpler task requiring focus only on the quality of the synthetic data generated. @xcite started by exploring linguistic theories to generate code-mixed data. later works attempt the problem using several approaches including generative adversarial networks @xcite , an encoder-decoder framework @xcite , pointer-generator networks @xcite , and a two-level table variational autoencoder @xcite . recently, rizvi et al. ( 2021 ) released a tool to generate code-mixed data using parallel sentences as input. ## system overview in this section, we describe our proposed systems for the task, which use mbart @xcite to translate english to hinglish. ## results ## discussion in this paper, we presented our approaches for english to hinglish translation using mbart. we analyse our model's outputs and show that the translation quality can be improved by including parallel hindi translations, along with the english sentences, while translating english sentences to hinglish. we also discuss the limitations of using bleu scores for evaluating code-mixed outputs and propose using bleu normalized -a slightly modified version of bleu. to understand the codemixing quality of the generated translations, we propose that a code-mixing measure, like cmi, should also be part of the evaluation process. along with the working models, we have analysed the model's shortcomings by doing error analysis on the outputs generated by the models. further, we have also presented an analysis on the shared dataset : percentage of sentences in the dataset which are not code-mixed, the erroneous reference translations. removing such pairs and replacing them with better samples will help improve the translation quality of the models. as part of future work, we would like to improve our translation quality by augmenting the current dataset with parallel sentences with a higher degree of code-mixing and good reference translations. we would also like to further analyse the nature of code-mixing in the generated outputs, and study the possibility of constraining the models to generated translations with a certain degree of code-mixing.