# Enhancement of English-Bengali Machine Translation Leveraging Back-Translation
## 5 Summary  
### Conclusion and Contributions  
In this article, we develop a machine translation method to translate English texts to Bengali texts. In particular, we have evaluated LSTM and Transformer architectures. The experiment shows that the Transformer model outperforms LSTM in both directions of the English-Bengali translation task. The performance enhancement indicates that we can exploit Transformer in designing better NMT architectures for low-resource language pairs like English-Bengali. We then investigate the effect of back-translation on the Transformer for the English-to-Bengali translation task. In our experiments, we analyze two decoding methods: top-\(k\) random sampling and random sampling with temperature \(T\). According to the results, back-translation with proper parameters improves translation accuracy through data augmentation. As the experiment shows, if we take top-\(k\) random sampling in back-translation, we can let all words in the vocabulary become candidates to achieve the best results. However, while the top-all (\(k=+\infty\)) setup is shown as the optimal case in the top-\(k\) strategy, it is theoretically equivalent to the \"no strategy\" setup, which can also be regarded as \(T=1\) case in temperature \(T\) sampling, where we can optimize \(T\) for even better results. Therefore, to obtain the best back-translation output, it is better to adopt random sampling with temperature \(T\), and explore the optimal value of \(T\) between 0 and 1. We observed that using random sampling with temperature \(T=0.5\) in back-translation makes  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Setup**} & **Checkpoint** & **BLEU Score** & **BLEU Score** \\  & **(Epoch)** & **(Validation)** & **(Test)** \\ \hline \multicolumn{4}{c}{Model comparison} \\ \hline LSTM, base & 49,000 & 3.62 & 0.00 \\ Transformer, base & 7000 & 27.80 & **1.33** \\ \hline \multicolumn{4}{c}{Top-\(k\) random sampling} \\ \hline Transformer, \(k=1\) & 43,000 & 19.98 & 1.15 \\ Transformer, \(k=5\) & 42,000 & 27.94 & 1.36 \\ Transformer, \(k=10\) & 42,000 & 29.43 & 1.36 \\ Transformer, \(k=+\infty\) (no strategy) & 44,000 & 38.22 & **2.07** \\ \hline \multicolumn{4}{c}{Random sampling with temperature \(T\)} \\ \hline Transformer, \(T=0\) & 43,000 & 19.98 & 1.15 \\ Transformer, \(T=0.25\) & 33,000 & 33.95 & 1.93 \\ Transformer, \(T=0.5\) & 34,000 & 35.02 & **2.35** \\ Transformer, \(T=0.75\) & 14,000 & 33.53 & 1.91 \\ Transformer, \(T=1.0\) (no strategy) & 44,000 & 38.22 & 2.07 \\ \hline \hline \end{tabular}
\end{table}
Table 3: English-Bengali NMT results on test set. We bold faced the best result for each set of comparisons. Note that in the base case, the training set is not augmented by back-translation.  
the model perform best. Therefore, we can augment the English-Bengali corpus by back-translation incorporating the decoding strategy of random sampling with temperature \(T\) over and over again, which helps the model gain a higher generalization ability. Finally, it can help improve the accuracy and quality of translation. However, in the evaluation, we did not get a very high BLEU score. Notably, in our experiments, we have built a simplified Transformer with only two blocks in the encoder and two blocks in the decoder, having only two heads in the multi-head attention because of the lack of powerful computing resources and to reduce the carbon footprint. Above all, our study shows how we can augment the datasets for low-resource languages and, finally, how to enhance the learnability of the model toward optimizing machine translation tasks.