A dap NMT : Neural Machine Translation with Technical Domain Adaptation for Indic Languages (2020)

Abstract: Adapting new domain is highly challenging task for Neural Machine Translation (NMT). In this paper we show the capability of general domain machine translation when translating into Indic languages (English - Hindi , English - Telugu and Hindi - Telugu), and low resource domain adaptation of MT systems using existing general parallel data and small in domain parallel data for AI and Chemistry Domains. We carried out our experiments using Byte Pair Encoding(BPE) as it solves rare word problems. It has been observed that with addition of little amount of in-domain data to the general data improves the BLEU score significantly.

## introduction due to the fact that neural machine translation (nmt) is performing better compared to the traditional statistical machine translation (smt) models, it has become very popular in the recent years. nmt systems require a large amount of training data and thus perform poorly relative to phrase-based machine translation (pbmt) systems in low resource and domain adaptation scenarios @xcite . one of the challenges in nmt is domain adaptation, it becomes more challenging when it comes to low resource indic languages and technical domains like artificial intelligence(ai) and chemistry as these domains may contain many technical terms and equations etc. in a typical domain adaptation setup like ours, we have a large amount of outof-domain bilingual training data for which we need to train a nmt model, we can treat this as a baseline model. now given only an additional small amount of in-domain data, the challenge is to improve the translation perfor-mance on the new domain. domain adaptation became very popular in these times, but very few works have been carried out on technical domains like chemistry, computer science, etc. therefore we adopted two new technical domains in our experiments, those include artificial intelligence and chemistry provided by icon adap-mt 2020 shared task for english -hindi and hindi -telugu language pairs. in our approach first we train a general models(baseline models) which trains based on only general data, we test domain data (ai, chemistry) on this general model then we try to improve performance of this new domain by training another model which uses combined training data(general data + domain data). inspired from @xcite , we encode rare and unknown words as sequences of sub word units using byte pair encodings(bpe) in order to make our nmt model capable of open vocabulary translation, this is further discussed in 3.2. ## background & motivation domain adaptation has became an active research topic in nmt. @xcite proposed two approaches, continue the training of the baseline model(general model) only on the in-domain data (domain data) and ensemble the continue model with the baseline model at decoding time. @xcite proposed iterative dual domain adaptation framework for nmt, which continuously fully exploits the mutual complementarity between indomain and out-domain corpora for translation knowledge transfer. apart from these domain adaptation techniques, there exists some approaches which has domain terminology and how to use that in nmt. similarly @xcite proposed an approach on nmt decoding with terminology constraints using decoder attentions which enables reduced output duplication and better constraint placement compared to existing methods. apart from traditional approaches there is a stack-based lattice search algorithm, constraining its search space with lattices generated by phrase-based machine translation (pbmt) improves the robustness @xcite . @xcite proposed two instance weighting methods with a dynamic weight learning strategy for nmt domain adaptation. although huge amount of research exists in this area , there exists very few works on indian languages. as per our knowledge there is no work on technical domains like ours (artificial intelligence and chemistry). therefore there is a need to handle these technical domains and work on morphological rich and resource poor languages. ## approach there are many approaches for domain adaptation discussed in section 2. however the approach we adopted , falls under combining the training data of general domain and specific technical domain data. this is further discussed in section 3.3. our approach follows attention-based nmt implementation similar to @xcite and @xcite . our model is very much similar to the model described in @xcite and supports label smoothing, beam-search decoding and random sampling. the brief explanation about nmt is described in section 3.1. ## future work we would like to extend this work to possible technical domains and for more languages as well. we plan to explore many other approaches like transformer based models for technical domain adaptation. and try to incorporate linguistic features into the nmt models. ## conclusion for morphologically rich and resource poor languages like telugu it's very difficult to get the large amount of parallel corpus for technical domain. therefor there is a need to optimize our general models with available small amount of domain data. in this paper we showed an approach which combines little amount of technical domain data to the available general domain data and trains a model using bpe. for better translation quality on technical domain we used only domain data as validation and observed our approach is giving promising results.