MUCS @Adap- MT 2020: Low Resource Domain Adaptation for Indic Machine Translation (2020)

Abstract: Machine Translation (MT) is the task of automatically converting the text in source language to text in target language by preserving the meaning. MT usually require large corpus for training the translation models. Due to scarcity of resources very less attention is given to translating into low resource languages and in particular into Indic languages. In this direction, a shared task called “Adap-MT 2020: Low Resource Domain Adaptation for Indic Machine Translation” is organized to illustrate the capability of general domain MT when translating into Indic languages and low resource domain adaptation of MT systems. In this paper, we, team MUCS, describe a simple word extraction based domain adaptation approach applied to English-Hindi MT only. MT in the proposed model is carried out using Open-NMT - a popular Neural Machine Translation tool. A general domain corpus is built effectively combining the available English-Hindi corpora and removing the duplicate sentences. Further, domain specific corpus is updated by extracting the sentences from generic corpus that contains the words given in the domain specific corpus. The proposed model exhibited satisfactory results for small domain specific AI and CHE corpora provided by the organizers in terms of BLEU score with 1.25 and 2.72 respectively. Further, this methodology is quite generic and can easily be extended to other low resource language pairs as well.

## introduction machine translation (mt) acts as a bridge for cross-language communication in natural language processing (nlp). it handles perplexity problems between two languages while preserving its meaning. mt was one of the initial tasks taken up by computer scientists and the research in this field is going on for last 50 years. mt task was initially handled with dictionary matching techniques and slowly upgraded to rule-based approaches @xcite . to resolve knowledge acquisition issues corpus based approaches became popular and bilingual parallel corpora was used to acquire translation knowledge @xcite . along with corpus based approaches, hybrid mt approaches also became popular as these approaches promise state-of-the-art result. the recent shift to large-scale analytical techniques has resulted in very significant improvements in the quality of mt. neural machine translation (nmt) -a corpus based approach has gained attention of the mt researchers. nmt is the task of translating text from one natural language (source) to another natural language (target) using most commonly, recurrent neural networks (rnn), specifically the encoder-decoder or sequence-to sequence models @xcite . further, unlike conventional translation systems, all parts of the neural translation model are trained jointly (end-to-end) to maximize the translation performance @xcite . in an nmt system, a bidirectional rnn, known as encoder is used by the neural network (nn) to encode a source sentence for a second rnn, known as decoder which is used to predict words in the target language. this encoder-decoder architecture can be designed with multiple layers to increase the efficiency of the system. now, nmt has become an effective alternative to traditional phrase-based statistical machine translation @xcite . ## domain adaptation for nmt dataset plays a crucial role in nn based translation models. huge amount of quality dataset for training results in good translation performance whereas small dataset results in poor translation performance. hence, if the dataset is small, effective management of such dataset for nn based translation will be the key for better translation performance. domain adaptation techniques that transfer existing knowledge to new domains as much as possible is one method in this direction. domain adaptation (da) is a sub-discipline of machine learning in which a model trained on a source distribution is used in the context of a different (but related) target distribution. in simple words, it is the ability to apply an algorithm trained in one domain to a different domain or updating one corpus using another corpus. while the big generic corpus will help to avoid out-of-vocabulary problem and unidiomatic translations, the small specialized corpus will help to capture terminology and vocabulary that is required for the translation @xcite . few effective da approaches which promise better translation performance are as follows: ## methodology despite the considerable advances in mt models, translation of low-resource languages is still an unresolved issue and da approaches are promising considerable performance in this direction. in the proposed work, a da approach of combining both generic dataset and domain specific dataset based on the vocabulary of domain specific dataset is used to conduct effective training and inference for translation using opennmt-a popular open source tool @xcite . opennmt accepts only primarily cleaned dataset as its input. therefore, noise such as initial space, end space, blank lines and special characters have been removed from the bilingual parallel corpus. this pre-processing is carried out for both generic corpus and domain specific corpora. then vocabulary of the domain specific corpora is constructed and sentences that contain any of the words in this vocabulary are extracted from the generic corpus. finally, these extracted sentences are added to the domain specific corpus and the updated corpus is used to train the translation model. ## experimental setup english to hindi mt is implemented using open-nmt which is considered as the most sophisticated generalized translation tool that provides easy modifications. as this model requires gpu, we set up this experiment in google colaboratory. translation experiments are carried out by continuous tuning of the model to conduct better training. initially, this model is trained using a huge generic corpus then the same set up is used for domain specific corpus. as the given domain specific corpora are very small to conduct efficient translation, training data of domain specific corpora is combined with generic corpus based on vocabulary of the domain specific corpora and the training is continued with the same set up. ## conclusion and future work in this english-hindi translation work, a huge generic corpus and small domain specific corpora are used for translation in opennmt. further, a simple domain adaptation technique is used to tackle translation issues of low-resource languages. as this approach is language independent it can easily be extended to other low-resource languages. further, these experiments have exhibited satisfactory results for both generic corpus and domain specific corpora. we would like to explore different preprocessing techniques that helps to translate low resource languages efficiently.