# Enhancement of English-Bengali Machine Translation Leveraging Back-Translation
## 4 Experiments
### Experiment Results and Discussion  
In this section, we present the results of our experiment and their discussion. As Figure 4 shows, Transformer models on both translation directions converge rapidly and outperform LSTM models. LSTM models need much more training epochs and show lower scores. This phenomenon implies that we can choose better architectures, like Transformer, to optimize the translation. Notably, according to our observation, the Bengali-English Transformer (for back-translation) has a higher BLEU score than the En-Bn by LSTM. One probable explanation is that a single sentence in English corresponds to several translations based on the status of speakers, but that extra information is lost in the corpus. Meanwhile, the result is reversed for the Transformer, which warrants further research.  
We then use the best Transformer Bn-En checkpoint (11,000 epochs) to back-translate the Bengali part of the WikiMatrix dataset. We obtain 7 versions of the translation, corresponding to different generation strategies. For top-\(k\) random sampling strategy, we randomly sample from the \(k\) most probable candidate words when generating the translation, with \(k=1,5,10\). Another method is to randomly sample from all candidate words with the probability modified by temperature \(T\), with \(T=0.25\), 0.5, 0.75, and 1.  
As discussed earlier, \(T=0\) (frozen) means that the word with the highest probability survives, while other choices have zero probability. This case is equivalent to the top 1 random sampling (argmax). Similarly, \(T=1\) is equivalent to the \"no strategy\" setup because dividing by 1 does not change the probability. For large \(T\) (suppose \(T\rightarrow+\infty\)), the probability distribution becomes flat, which means the model degrades into an untrained situation (every word has the same probability). It destroys the modeled probability distribution, causing many writing errors to appear in the back-translated text. Therefore, we do not examine for a higher temperature.  
The translations are separately added to the training data. Thus, we have seven versions of augmented training sets. Next, we train a new Transformer model on each of  
Figure 4: BLEU scores on the validation set over epochs for different models.  
the seven augmented training data for 50,000 epochs, pick up their best checkpoints on the validation set, and compare their performance on the test set. Instead of extracting the test set from the datasets used for training and validation, we use a small part of the SUPara dataset (SUPara benchmark) for testing. With this approach, we expect to better examine the generalizability.  
Table 3 shows the results of different setups on the validation set and the test set. Note that \(k=1\) and \(T=0\) refer to the same case: the argmax sampling in back-translation, and that the top-all setup (\(k=+\infty\)) and the \(T=1\) setup are the \"no strategy\" cases. According to the test, we obtained three results. First, the Transformer model has much better performance than LSTM on the base training set. Therefore, Transformer is promising for translating low-resource languages. Second, we can use back-translation to boost the translation performance, but the adoption of the top-\(K\) random sampling strategy weakens the effect of back-translation. Third, back-translation with temperature \(T\) sampling achieves higher BLEU with \(T=0.5\), while too low or too high \(T\) values reduce the enhancement.