# Enhancement of English-Bengali Machine Translation Leveraging Back-Translation
### Challenges and Contributions  
In this paper, in building our viable English-Bengali NMT engine, we employ back-translation while leveraging monolingual data. Back-translation is one of the state-of-the-art strategies used in NMT for addressing low-resource languages and monolingual corpora [19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29]. Notably, the architectures of translation and back-translation models are similar, simply altering/reversing the source and target corpora for the encoder and decoder. In back-translation, instead of extracting the test set from the datasets used for training and validation, it is recommended to use a part of the dataset for testing or directly use a monolingual target corpus. That improves the model generalization. Especially, the translated text and the monolingual target data in the last stage can be used as a pseudo-parallel corpus and can be appended with the original dataset, and the model can be trained with this augmented dataset, which can help train the model better. Therefore, with back-translation, we can obtain an augmented dataset. However, the new data can be regarded as data with noise because they are generated by models that may not yet be trained very well or not evaluated as well as human translators (assuming that human translators generate perfect translations). Since the original output of a translation model is a probability distribution of candidate words, to make the model more robust, different decoding methods are used, such as beam search [34], top-\(k\) random sampling[35] and random sampling with temperature\(T\)[36], and others. Notably, top-\(k\) random sampling and random sampling with temperature\(T\) are the most commonly used and optimal decoding methods than the beam search [35; 36]. Beam search was usually used in the early days of NMT evolution. Therefore, we can append the pseudo-parallel corpus with the original dataset obtained from back-translation with the decoding strategies. The experiment results (Section 4.4) show that back-translation with decoding strategies improves translation accuracy respectfully. Thus, the model can be trained with the augmented dataset, which can help enhance the generalization ability [37; 38; 39]. Especially, the process can be repeated further with the monolingual datasets from different sources. In the experimental study of this paper, we start our analysis by comparing two (2) architectures: LSTM and Transformer. Thereafter, we employ back-translation with the two decoding methods as stated earlier: top-\(k\) random sampling[35] and random sampling with temperature\(T\)[36]. The top-\(k\) random sampling samples from the top \(k\) predictions. For example, if \(k=3\), in this case, the model simply picks up the 3 words or tokens with the highest probabilities and sets the probabilities of the rests to \(-\infty\) (\(-10\),\(000\) in practice). Alongside, random sampling with temperature\(T\) is a factor to modify the randomness of candidate words. Notably, if we set \(k=+\infty\) in top-\(k\) random sampling or if we set \(T=1\) in random sampling with temperature\(T\), then the probability distribution is not modified, i.e., the model samples from all tokens in its vocabulary with their original probabilities. (It can be called \"no strategy\"). It is not resource-intensive and optimal as it samples from all the tokens. In this paper, in our study, we explore the optimal values for \(k\) and \(T\) respectfully. For more details, including the definitions of \(k\) and \(T\), we refer to Sections 3 and 4. Especially in our study, we evaluate the machine translation task using an automatic evaluation method called Bilingual Evaluation Understudy (BLEU) [12]. BLEU is the most commonly used evaluation metric in machine translation evaluation [2]. Given a sentence predicted by the model and the reference translation, BLEU measures their similarity. Notably, the BLEU score is based on precision (ranges from 0 to 1) while analyzing the similarity between the candidate translation and any translation reference(s).