Low Resource Similar Language Neural Machine Translation for T amil- T elugu (2021)

Abstract: This paper describes the participation of team oneNLP (LTRC, IIIT-Hyderabad) for the WMT 2021 task, similar language translation. We experimented with transformer based Neural Machine Translation and explored the use of language similarity for Tamil-Telugu and Telugu-Tamil. We incorporated use of different subword configurations, script conversion and single model training for both directions as exploratory experiments.

## introduction machine translation (mt) is a field of natural language processing which aims to translate a text from one natural language to another. the meaning of the source text must be fully preserved in the resulting translated text in the target language. recent years have seen significant quality advancements in machine translation with the advent of neural machine translation. for the translation task, different types of machine translation systems have been developed and they are mainly categorized into rule based machine translation (rbmt) @xcite , statistical machine translation (smt) @xcite and neural machine translation (nmt) @xcite . neural machine translation (nmt) shows high quality in terms of output fluency and translation quality, when large amounts of parallel data are available @xcite . unfortunately, for most language pairs, parallel data is either scare or non-existent. to overcome this, unsupervised mt (umt) @xcite focuses on utilising monolingual data to generate synthetic parallel training data. other techniques like backtranslation @xcite ,(hoang et al., 1 https://www.statmt.org/wmt21/similar.html 2018), @xcite or denoising @xcite also rely on parallel corpora of other language pairs and/or large quantities of monolingual data. this paper describes our experiments for very low resourced similar language translation. for our work, we focused only on tamil-telugu language pair (both directions) and participated in a constrained setting. we experimented only with transformer @xcite based neural machine translation throughout. along with it, to tackle high agglutination of both languages, we explored the morph @xcite induced sub-word segmentation with byte pair encoding (bpe) @xcite . similar to multilingual neural machine translation (mnmt), we explored the use of a tag trick, where a token like "< 2xx >" (xx is language code) is prefixed to each source sentence to indicate the desired target language @xcite . here, we trained a single model for both directions (tamil-telugu and telugu-tamil) on given parallel data and monolingual data under mnmt setting. the sections of the paper are organised as following: section 2 describes data, section 3 and 4 describe pre-processing and training configuration and in section 5 we talk about results and we conclude in section 6. ## data we utilised provided parallel corpora for @xmath0. apart form parallel corpus, we randomly selected 0.1m monolingual corpora from indiccorp monolingual corpus foot_0 for tamil and telugu. table-1 describes the training and development data (parallel and monolingual) used in all our experiments under constrained setting. data sents token type train tamil (parallel) 40,147 0.68m 74k telugu (parallel) 40,147 0.72m 90k development tamil (parallel) 1261 29k 9k telugu (parallel) 1261 30k 10k tamil (mono) 0.1m --telugu (mono) 0.1m -- ## training configuration throughout all experiments, we used transformer sequence to sequence architecture with the following configuration. for these experiments, we used shared vocab across trainings. we used opennmt-py @xcite toolkit with above configuration for our experiments. using the above described pre-processing and configuration, we performed experiments on word level, bpe level and morph + bpe level for input and output. the results are discussed in following result section. 5 result feature bpe dev script conversion (ta to te) -0.57 word -5.12 bpe 20k 6.07 morph + bpe 20k 6.25 morph + bpe (mnmt) 20k 6.65 table-2 and table-3 show that non-translation baselines are also low in terms of bleu scores which indicates that the task much harder even though languages are similar. the results show that for low resource settings, transformer network based mt models can be improved with morph based segmentation along with byte pair encoding for morph rich languages. also, forming it as a multilingual machine translation problem, along with monolingual data, it improves the quality of mt models. this may be due to language similarity and use of monolingual data, as it is helping models to do better generalization by learning better source language encoding and target language fluency. ## conclusion from our experiments, we conclude that linguistic feature such as morph based segmentation with subword segments along with mnmt is a promising approach for similar language translation.