IITP - MT at CALCS 2021: E nglish to H inglish Neural Machine Translation using Unsupervised Synthetic Code-Mixed Parallel Corpus (2021)

Abstract: This paper describes the system submitted by IITP-MT team to Computational Approaches to Linguistic Code-Switching (CALCS 2021) shared task on MT for English→Hinglish. We submit a neural machine translation (NMT) system which is trained on the synthetic code-mixed (cm) English-Hinglish parallel corpus. We propose an approach to create code-mixed parallel corpus from a clean parallel corpus in an unsupervised manner. It is an alignment based approach and we do not use any linguistic resources for explicitly marking any token for code-switching. We also train NMT model on the gold corpus provided by the workshop organizers augmented with the generated synthetic code-mixed parallel corpus. The model trained over the generated synthetic cm data achieves 10.09 BLEU points over the given test set.

## introduction in this paper, we describe our submission to shared task on machine translation (mt) for @xmath0. the objective of this shared task to generate hinglish (hindi-english code-mixed 1 ) data from english. in this task, we submit an nmt system which is trained on the parallel code-mixed english-hinglish synthetic corpus. we generate synthetic corpus in unsupervised fashion and the methodology followed to generate data is independent of languages involved. since the target hindi tokens are written in roman script, during the synthetic corpus creation, we transliterate the hindi tokens from devanagari script to roman script. code-mixing (cm) is a very common phenomenon in various social media contents, product description and reviews, educational domain etc. for better understanding and ease in writing, users * equal contribution 1 hindi words are romanized write posts, comments on social media in codemixed fashion. it is not consistent or convenient always to translate all the words, especially the named entities, quality related terms etc. but translating in code-mixed fashion required code-mixed parallel training data. it is possible to generate code-mixed parallel corpus from a clean parallel corpus. from the term 'clean parallel corpus', we refer to a parallel corpus which consists of the non code-mixed parallel sentences. generally noun tokens, noun phrases and adjectives are the major candidates to be preserved as it is (without translation) in the code-mixed output. this requires a kind of explicit token marking using parser, tagger (part of speech, named entity etc.) to find the eligible candidate tokens for code-mixed replacement. since this method is dependent on linguistic resources, it is limited to the high resource languages only. we introduce an alignment based unsupervised approach for generating code-mixed data from parallel corpus which can be used to train the nmt model for code-mixed text translation. the paper is organized as follows. in section 2, we briefly mention some notable works on translation and generation of synthetic code-mixed corpus. in section 3, we describe our approach to generate synthetic code-mixed corpus along with the system description. results are described in section 4. finally, the work is concluded in section 5. ## related works translation of code-mixed data has gained popularity in recent times. @xcite conducted experiments on translating arabic-english cm data to pure arabic and/or to pure english with statistical machine translation (smt) and neural machine translation (nmt) approaches. @xcite proposed an mt augmentation pipeline which takes cm sentence and determines the most dominating language and translates the remaining words into that language. the resulting sentence will be in one single language and can be translated to other language with the existing mt systems. @xcite have used code-mixing phenomenon and proposed a pre-training strategy for nmt. @xcite augmented the codemixed data with clean data while training the nmt system and reported that this type of data augmentation improves the translation quality of constrained words such as named entities. singh and solorio (2017); masoud et al. (2019); mahata et al. (2019) also explored various approaches which utilize linguistic resources (such as language identification etc.) to translate the code-mixed data. there have been some efforts for creating codemixed data. gupta et al. (2020) proposed an encoder-decoder based model which takes english sentence along with linguistic features as input and generates synthetic code-mixed sentence. pratapa et al. (2018) explored 'equivalence constraint' theory to generate the synthetic code-mixed data which is used to improve the performance of recurrent neural network (rnn) based language model. while winata et al. (2019) proposed a method to generate code-mixed data using a pointer-generator network, garg et al. (2018) explored seqgan for code-mixed data generation. ## system description in this section, we describe the synthetic parallel corpus creation, dataset and experimental setup of our system. ## results we train two models. baseline model which is trained on the gold standard corpus. second model on the synthetic code-mixed data. we upload our model predictions on the test set provided by organizers to shared task leaderboard 7 . the test set con ## conclusion in this paper, we described our submission to shared task on mt for @xmath1. we submitted a system which is trained on synthetic code-mixed corpus generated in unsupervised way. we trained an nmt model on the synthetic code-mixed corpus and gold standard data provided by organizers. on the test set, the model trained over the gold data provided by the workshop achieves 2.45 bleu points while the model trained over our generated synthetic cm data yields bleu score of 10.09. we believe that the proposed method to generate synthetic code-mixed data can be very useful for training mt systems in code-mixed settings as the proposed method does not require any linguistic resources to generate code-mixed data.