Unified NMT models for the I ndian subcontinent, transcending script-barriers (2022)

Abstract: Highly accurate machine translation systems are very important in societies and countries where multilinguality is very common, and where English often does not suffice. The Indian subcontinent (or South Asia) is such a region, with all the Indic languages currently being under-represented in the NLP ecosystem. It is essential to thoroughly explore various techniques to improve the performance of such lowresource languages at least using the data available in open-source, which itself is something not very explored in the Indic ecosystem. In our work, we perform a study with a focus on improving the performance of very-low-resource South Asian languages, especially of countries in addition to India. Specifically, we propose how unified models can be built that can exploit the data from comparatively resource-rich languages of the same region. We propose strategies to unify different types of unexplored scripts, especially Perso–Arabic scripts and Indic scripts to build multilingual models for all the South Asian languages despite the script barrier. We also study how augmentation techniques like back-translation can be made useof to build unified models just using openly available raw data, to understand what levels of improvements can be expected for these Indic languages.

## introduction the indian subcontinent is a well-studied linguistic area @xcite , known as south asian sprachbund. the region is home to around a quarter of the world's population, with a total which is projected to reach more than 2 billion in a decade. despite this, the progress in natural language processing is significantly lacking for south asian languages (or indic languages). especially, machine translation is of core importance since south asia is largely a multilingual society, with more than 25 languages recognized officially across the subcontinent and more than 100s attested and spoken. although there are quite a few number of works which have released datasets for languages of india @xcite and studied multilingual models for the same @xcite , they are not exhaustively studied. in particular, the indic languages of other south asian countries like pakistan, nepal and sri lanka are almost never studied together with the languages of india and bangladesh. in this work, we aim to study all the available indic languages (of indo-aryan and dravidian families) of all the above countries together, precisely 15 south asian languages (listed in appendix a). especially, we propose a simple strategy to unify digraphic languages like hindi-urdu, sindhi and punjabi which are written in indic scripts in india and perso-arabic scripts in pakistan. we propose how one can build a script-agnostic encoder which can generalize well across different types of translation models, like code-mixed, roman (social media) and formal texts. we study for the first time in literature backtranslation-based nmt for all script-unified indic languages together, which provides significantly better performance than models trained only on parallel data, by using only freely available monolingual data. we finally provide brief recommendations for researchers working in this indic-nmt domain, and finally mention how this work can be extended and its future scope. ## related works training multilingual models for neural machine translation currently the go-to approach for significantly improving the performance of low-resource languages @xcite . especially sharing of sub-word vocabulary among related languages (of the same or similar families) is of more importance to exploit the inter-relationships between the languages @xcite , so that resource sharing from high-resource languages to low-resource languages is achieved. recent works @xcite have explored strategies to train multilingual nmt for 11 languages of india, both with and without shared vocabulary across languages, demonstrating that vocabulary sharing by script unification is significantly beneficial. it is also common to convert all the text across all languages to ipa (international phonetic alphabet) or any common script, especially in speech-to-text @xcite ) and text-to-speech @xcite to obtain a universal representation of text across any language/script. in the case of south asian languages, it is more convenient to map all scripts to a common indic script (like devanagari) which is capable of representing all phonemes used in the indic families @xcite . ## background this section sets provides the background required for the subsequent sections. ## discussions and conclusion we demonstrate in this paper various methods to achieve improvement in performance, especially across south asian languages which were not previously explored along with the languages of india. we believe our presented contributions are more of exploratory nature, and make fundamental proposals (like always building romanized models when the source side is indic). although the fact that a unified model results in better performance in low-resource scenarios has been discovered by many prior work and hence not surprising, our work merely focuses on quantitatively studying the improvement in the case of indic languages. in this section, we provide general suggestions for research groups working on nmt for indic languages. in general, to train model for any low-resource indic language to english, we recommend that data from all the languages is used to train a multilingual model. 7 especially, training a romanized model would be more beneficial, since it would be a scriptagnostic model, and hence easily generalize for code-mixed and social media texts. for training english to any low-resource indic language, it maybe be preferable to train familyspecific models when working under resourcecontrained settings. especially for languages of the countries pakistan, bangladesh, nepal and sri lanka, we highly recommend and encourage them to exploit the datasets made available by researchers of india. if possible, it is highly recommended to exploit the abundant monolingual data and train models using backtranslated data.