# Enhancement of English-Bengali Machine Translation Leveraging Back-Translation
###### Abstract  
An English-Bengali machine translation (MT) application can convert an English text into a corresponding Bengali translation. To build a better model for this task, we can optimize English-Bengali MT. MT for languages with rich resources, like English-German, started decades ago. However, MT for languages lacking many parallel corpora remains challenging. In our study, we employed back-translation to improve the translation accuracy. With back-translation, we can have a pseudo-parallel corpus, and the generated (pseudo) corpus can be added to the original dataset to obtain an augmented dataset. However, the new data can be regarded as noisy data because they are generated by models that may not be trained very well or not evaluated well, like human translators. Since the original output of a translation model is a probability distribution of candidate words, to make the model more robust, different decoding methods are used, such as beam search, top-\(k\) random sampling and random sampling with temperature \(T\), and others. Notably, top-\(k\) random sampling and random sampling with temperature \(T\) are more commonly used and more optimal decoding methods than the beam search. To this end, our study compares LSTM (Long-Short Term Memory, as a baseline) and Transformer. Our results show that Transformer (BLEU: 27.80 in validation, 1.33 in test) outperforms LSTM (3.62 in validation, 0.00 in test) by a large margin in the English-Bengali translation task. (Evaluating LSTM and Transformer without any augmented data is our baseline study.) We also incorporate two decoding methods, top-\(k\) random sampling and random sampling with temperature \(T\), for back-translation that help improve the translation accuracy of the model. The results show that data generated by back-translation without top-\(k\) or temperature sampling (\"no strategy\") help improve the accuracy (BLEU 38.22, +10.42 on validation, 2.07, +0.74 on test). Specifically, back-translation with top-k sampling is less effective (\(k=10\), BLEU 29.43, +1.83 on validation, 1.36, +0.03 on test), while sampling with a proper value of \(T\), \(T=0.5\) makes the model achieve a higher score (\(T=0.5\), BLEU 35.02, +7.22 on validation, 2.35, +1.02 on test). This implies that in English-Bengali MT, we can augment the training set through back-translation using random sampling with a proper temperature \(T\). English; Bengali; machine translation; back-translation; random sampling; temperature 00000-0000-0000-0000-0000-0000-0000-00000-0000-0000-0000-0000-00000-00000-00000-00000-00000-00000-00000-000000-000000-000000-000000-0000000-target language, which has the same meaning as the input [1]. Most recent research on MT falls under neural machine translation (NMT), which employs various neural networks, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformer models, and so on. A typical NMT model consists of two main components: an encoder that transcribes the input sentence into a hidden representation and a decoder that generates the target sentence according to the hidden representation [2]. In particular, RNN-based models (basic RNN, LSTM, GRU, and other variants) were used in earlier days of NMT evolution [3; 4]. However, with RNN-based models, translation accuracy degrades when the sequence length and out-of-vocabulary or unknown words increase. Another issue is that the information that the decoder receives from the encoder is the last hidden state of the encoder. It especially contains the numerical summary of the input sequence. With this, it can handle the translation of short sentences but not the translation of long sentences since it gets a numerical summary of itself. As human beings, we may also have trouble translating a long sentence. To address these issues, an RNN-based model with an attention-based mechanism was introduced [5; 6; 7].