Gated Convolutional Sequence to Sequence Based Learning for E nglish-Hingilsh Code-Switched Machine Translation. (2021)

Abstract: Code-Switching is the embedding of linguistic units or phrases from two or more languages in a single sentence. This phenomenon is practiced in all multilingual communities and is prominent in social media. Consequently, there is a growing need to understand code-switched translations by translating the code-switched text into one of the standard languages or vice versa. Neural Machine translation is a well-studied research problem in the monolingual text. In this paper, we have used the gated convolutional sequences to sequence networks for English-Hinglish translation. The convolutions in the model help to identify the compositional structure in the sequences more easily. The model relies on gating and performs multiple attention steps at encoder and decoder layers.

## introduction language is a social phenomenon. the day-to-day interactions are made possible via language. the adaptive nature of languages and the flexibility to use multiple languages in one text message might help the speakers to communicate efficiently. this form of language interaction/contact is considered to be an essential phenomenon, especially in multilingual societies. in bilingual or multilingual communities, speakers use their native tongue and their second language during interactions. this form of alternation of two or more languages is called code-switching (cs) @xcite . through the advent of social media, people from around the world can connect and exchange information instantly. users from a multilingual community often express their thoughts or opinions on social media by mixing different languages in the same utterance @xcite . this mixing or alteration of two or more languages is known as code-mixing or code-switching @xcite . there are no standard grammar rules that are meant to be practiced in the code-switched text. the code-switched data often contain variations of spellings and grammar. the computational processing of code-mixed or code-switched data is challenging due to the nature of the mixing and the presence of non-standard variations in spellings and grammar, and transliteration @xcite . because of such linguistic complexities, code-switching poses several unseen difficulties in fundamental fields of natural language processing (nlp) tasks such as language identification, part-of-speech tagging, shallow parsing, named entity recognition, sentiment analysis, offensive language identification etc. to encourage research on code-mixing text, the computational approaches to linguistic code-switching (calcs) community has organized several workshops on language identification, named entity recognition @xcite . this task focus on machine translation in the code-switched environment in multiple language combinations and directions foot_0 . this paper presents a gated convolutional sequence to sequence encoder and decoder models @xcite for machine translation on the code-mixed text. we have used the convolutional model because of its sliding window concept to deal with contextual words and the convolutions to extract rich representations. the paper is organized as follows. section 2 provides related work on the code-switched text for machine translation. section 3 provides information on the task and dataset. section 4 describes the proposed work. section 5 presents the experimental setup and the performance of the model. section 6 concludes our work. there is relatively less research in the field of the machine translation of the code-switched text, partially due to the relative lack of structured corpora and also potentially because it also poses significant linguistic challenges such as ambiguity in language identification, spelling variations, informal style of writing, misplaced/skipped punctuation, etc. nonetheless, some researchers have provided datasets to enable research in code-mixed machine translation, specifically in hindi-english codeswitched scenario @xcite . @xcite presented a parallel corpus of the 13,738 code-mixed hindi-english sentences and their corresponding human translation in english. in addition, they also provided a translation pipeline built on top of google translate. the pipeline fragments the input sentence into multiple chunks and identifies the language of each word in the chunk before feeding it to google-translate. the pipeline gives a bleu-1 metric of 0.153 on the given english dataset. @xcite translated the 6,096 code-mixed english-hindi sentences into english and presented a translation augmentation pipeline. the pipeline is presented as a pre-processing step and can be plugged into any existing mt system. the preprocessed data is then given to translation systems like moses, google neural machine translation system (nmts), and bing translator, where the pre-processed data with nmts has outperformed all the baselines with a bleu score of 28.4. ## task description the goal of this task is the machine translation for code-switching settings in multiple language combinations and directions, such as involving english, hinglish, spanish, spanglish, modern standard arabic, and egyptian arabic languages. the codemixed dataset is obtained from comments/posts from social media. in this paper, we have focussed on the english-hinglish dataset. the english-hinglish code-mixed dataset has 8060 train, 952 dev, and 960 test with source, and target translations. the task is to translate the given english sentence into a code-mixed hindi-english sentence. the examples of the given english-hinglish translation are given in the in the first translation, one can see that the hinglish sentence has a mixture of non-standard variations of words such as komedee(comedy), edavenchar(adventure), eneemeshan(animation), and the second translation exhibits the switching of english and hindi phrases. in the third translation, the sentence is completely translated to hindi (with roman script). the fourth translation shows that no translation is followed. the above sentences depict the diversity of the code-mixed translations, thus making the research and translation of the codemixed text a complex task. ## the proposed work this section presents the proposed gated convolutional neural networks with encoder and decoder models for machine translation from english to code-mixed hinglish text. the encoder model encode the source sentence into a vector and the decoder model takes the encoder information and decodes the given target sentences. the encoded vector is also known context vector. the context vector can be visualized as an abstract representation of the entire input sentence. the vector is decoded by a decoder model that learns to output the target sentence. the context needs to contain all of the information about the source sentence. it can be done by using attention. our encoder and decoder attention-based models use convolutions to encode the source sentence and to decode it into the target sentence. the convolutional layer uses filters. these filters have a window size. for example, if a filter has a window size of 3, then it can process three consecutive tokens. this window helps in determining the context. the convolutional layer has many of these filters, where each filter will slide across the entire sequence by looking at all three consecutive tokens at a time. these filters will help extract different features in the given text and aid the machine translation model. the description of the encoder and decoder convolutional models is given in the subsequent subsections. ## experiments here, we demonstrate the performance of the machine translation systems on the code-mixed text. we experiment with the popular rnn based encoder-decoder machine translation and vanilla transformer models and evaluate their performance on the given english-hinglish machine translation task. we use bleu metrics to evaluate system performance @xcite . ## conclusion this paper presents the performance of a neural machine translation model for the shared task on codeswitched english-hinglish translation. the model uses the convolutional sequence to sequence-based neural network architecture to translate the given sequence. the contextual window and the stateof-the-art convolution model helped the model learn better representations from the text and improved the model's performance compared to rnn encoder-decoder and vanilla transformer models. in the future, we wish to use the pre-trained bert models and their ensembles and also consider other code-mixing factors such as pre-processing of the code-switched text to improve the quality of the code-switched machine translation.