Efforts Towards Developing a T amang N epali Machine Translation System (2020)

Abstract: The Tamang language is spoken mainly in Nepal, Sikkim, West Bengal, some parts of Assam, and the North East region of India. As per the 2011 census conducted by the Nepal Government, there are about 1.35 million Tamang speakers in Nepal itself. In this regard, a Machine Translation System for Tamang-Nepali language pair is significant both from research and practical outcomes in terms of enabling communication between the Tamang and the Nepali communities. In this work, we train the Transformer Neural Machine Translation (NMT) architecture with attention using a small hand-labeled or aligned Tamang-Nepali corpus (15K sentence pairs). Our preliminary results show BLEU scores of 27.74 for the Nepali→Tamang direction and 23.74 in the Tamang→Nepali direction. We are currently working on increasing the datasets as well as improving the model to obtain better BLEU scores. We also plan to extend the work to add the English language to the model, thus making it a trilingual Machine Translation System for Tamang-Nepali-English languages.

## introduction machine translation (mt) systems today represent one of the biggest achievements of human mankind in the field of artificial intelligence (ai) and language technologies (lt). with the advancement of deep neural networks (dnn) and abundance of data, today's mt systems already claim more than 90% accuracy in the translation thus opening doors for the adoption and use of highly reliable translation systems. these systems serve multiple folds: (1) bridge the language barrier between different language speaking communities; (2) preserve languages that are not as popularly spoken or used in the younger generations and many more. if we talk about the tamang language, native to the tibeto-burman group of the sino-tibetan language family and spoken by about 1.35 million native speakers in nepal according to the 2011 census by the nepal government foot_0 , then we can say that it can greatly benefit from mt systems. this is further enunciated by the fact that tamang is one of the languages which is highly affected by the latest migration trends, both within and outside the country, in quest of better lives and opportunities. consequently, the language is not being spoken or learned by the younger generation and is being limited to the older generations thus creating a fear of extinction. on the contrary, in areas with dense tamang populations in nepal the primary language of communication is tamang and even the medium of teaching is tamang in these areas. needless to note, mt systems can open up the tamang community from such areas to the outer world via the knowledge sources available in nepali and english languages on the internet. in this paper, we discuss our efforts on developing a tamang↔nepali machine translation system. the biggest contribution of this work is the development of the parallel corpus of 15,000 parallel sentences in tamang and nepali from scratch. moreover, we have adopted the transformer architecture @xcite for a lowresource language pair (tamang-nepali) with quite a few language-based customizations of hyperparameters. we have achieved a bleu score of 27.74 for @xmath0.74 for @xmath1. ## challenges in parallel corpus development this is the first work of any kind in terms of nepali↔tamang machine translation; there is no related work in this regard. when we started working on the project, we did not have any parallel sentences for the tamang-nepali pair. hence, we started to look for any available resources for the language pair. the problem with tamang is that there is still no consensus in the community regarding the script for the language. there are a significant number of people in the tamang community advocating for the tamyig script which is very close to tibetan script. however, since the tamyig script is mostly confined to scriptures, holy books of the lamas and not taught widely in schools, a large chunk of the tamang community still uses the devanagari script for writing and print media. hence, we can find quite a few publications in tamang written in the devanagari script. based on the popular usage and availability, we also decided to opt for devanagari script-based tamang language texts. our very first preliminary attempt included seeing if we could use the bible translations in nepali 2 and tamang 3 as the parallel sentences. however, the resource had its own set of issues so, we dropped the idea of using this resource. we also tried using tamang songs and their translations but this attempt was not fruitful as the translations lacked wide coverage of texts of varying complexity in the languages. finally, we established contact with tamang linguists and experts as well as activists who had been working for the tamang language in different capacities for long and we decided to initially develop 15,000 parallel sentences for tamang-nepali. these sentences range from simple to medium and complex sentences and have been taken from different sources like day-to-day spoken communication texts, child storybooks, general articles from tamang language magazines, literary articles, etc. ## experimental setup in this section, we present the experimental settings used for training the mt architecture and models for experimenting and reporting the results. the models which varied with different vocabulary of pre-selected words (1k, 2.5k and 5k) were trained using fairseq foot_1 toolkit (version 0.9.0) @xcite . google colab foot_2 (free-tier) is used for training the mt architectures with hardware acclerator set to gpu. ## results after training all the models for 150 epochs, the best performing checkpoint 9 is used for each model with beam size of 5 @xcite and length penalty of 1.2 to measure the translation quality using the bleu 10 metric. the bleu score @xcite is a general way to evaluate the performance of machine translation system when there can be multiple right outputs. sacrebleu 11 (version 1.4.10) (post, 2018) is used to compute the bleu scores. bleu scores obtained after training on multiple vocabulary sizes (1k, 2.5k and 5k) are shown in ## discussion for a low-resource language like tamang, which has very negligible digital footprints, the achieved bleu scores are quite encouraging. besides, the gathered dataset (approx. 15k) can serve as a benchmark data for the tamang -nepali mt. we consider this as a significant contribution and achievement for the language pair. the deployed mt system is made available in the link 12 and is open for testing. we are really encouraged by the preliminary feedback that we have received from the community. ## future plans and road map we are in touch with the linguists and experts from the community and working towards further increasing the dataset. similarly, we will also be looking further into how the mt model can be further optimized via hyper-parameter tuning, vo-cabulary size modification, changes in the learning rate etc. the tamang community has been involved in the testing of the system throughout and they are quite satisfied with the gradual improvement of the quality of the system. we plan to increase the size of the parallel sentences to 100k via crowdsourcing and other methods. we also plan to extend the bi-lingual mt system to a trilingual one incorporating english to the system so that the tamang and the nepali community benefit from the knowledge sources available in english on the internet.