# Impact of Sentence Representation Matching in Neural Machine Translation
## 5 Results and Discussions
### Performance on State-of-the-Art Model  
Table 11 shows the BLEU score on the BiBERT model with and without our matching method. We downloaded the IWSLT'14 \(De\leftrightarrow En\) data from Pytorch ([[https://github.com/pytorch/fairseq/blob/main/examples/translation/prepare-iwslt14.sh](https://github.com/pytorch/fairseq/blob/main/examples/translation/prepare-iwslt14.sh)]([https://github.com/pytorch/fairseq/blob/main/examples/translation/prepare-iwslt14.sh](https://github.com/pytorch/fairseq/blob/main/examples/translation/prepare-iwslt14.sh)), accessed on 18 January 2022) and the code from BiBERT github link ([[https://github.com/felixxu/BiBERT](https://github.com/felixxu/BiBERT)]([https://github.com/felixxu/BiBERT](https://github.com/felixxu/BiBERT)), accessed on 18 January 2022). Only the matching method was added, and all other progress of training were the same as baseline. The improvement of performance is observed in every condition, including the one-way, dual training, and fine-tuning conditions, except \(En\to De\) with one-way training. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**BE \& WV (50,000 Tokens)**} & \multicolumn{4}{c}{**BE (119,547 Tokens)**} \\ \cline{3-10}
**Dataset** & **Sentence** & \multicolumn{2}{c}{**Word**} & \multicolumn{2}{c}{**Token**} & \multicolumn{2}{c}{**Word**} & \multicolumn{2}{c}{**Token**} \\  & & **Fr** & **En** & **Fr** & **En** & **Fr** & **En** & **Fr** & **En** \\ \hline Europarl & 1,737,355 & 38.95 & 48.32 & 99.71 & 99.84 & 7.01 & 9.35 & 78.03 & 89.40 \\ common-test & 22,960 & 87.53 & 89.49 & 99.51 & 99.63 & 17.88 & 31.85 & 79.40 & 89.86 \\ newstest & 3003 & 81.80 & 82.39 & 95.92 & 96.08 & 30.44 & 45.33 & 74.87 & 83.07 \\ nc-test & 2007 & 92.17 & 92.87 & 98.67 & 98.65 & 31.89 & 48.34 & 75.26 & 84.05 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Overlapping rate of sorts and tokens between imported embedding dictionary and data sets for robustness analysis. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**De \(\rightarrow\) En**} & \multicolumn{2}{c}{**En \(\rightarrow\) De**} \\ \cline{2-7}
**Method** & **Baseline**[20] & **+ Matching** & **Baseline**[20] & **+ Matching** \\ \hline One-Way (vocab size = 12 K) & 37.69 & **38.13** & **30.00** & 29.93 \\ Dual-Directional Training & 38.37 & **38.42** & 30.30 & **30.50** \\ + Fine-Tuning & 38.61 & **38.70** & 30.45 & **30.53** \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Comparison of dual-directional and ordinary (one-way) translation models in [20] including stochastic layer selection (\(K=8\)) with and without sentence representation mapping on IWSLT’14 \(De\leftrightarrow En\). \begin{table}
\begin{tabular}{l l c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**Best Performance in Each Set**} \\  & & & **Common-Test** & \multicolumn{2}{c}{**Newstest**} & \multicolumn{2}{c}{**Nc-Test**} \\ \hline Embedding & Networks & prec. & BLEU & prec. & BLEU & prec. & BLEU \\ \hline Random & LSTM & 5.72 & 10.96 & 14.08 & 21.30 & 16.73 & 27.29 \\ Embedding & LSTM + matching & **5.82** & **11.10** & **14.25** & 21.30 & 16.19 & 26.74 \\ (RE) & Transformer & 6.09 & 11.67 & 16.24 & 25.81 & 17.44 & 29.19 \\  & Transformer + matching & 6.09 & 11.61 & **16.28** & 24.96 & **18.13** & **29.32** \\ \hline Word2 Vec & LSTM & 5.77 & 11.12 & 14.55 & 21.75 & 16.67 & 26.57 \\ (WV) & LSTM + matching & **5.95** & **11.45** & **14.73** & 21.48 & 15.82 & **26.78** \\  & Transformer & 6.12 & 11.44 & 16.30 & 23.50 & 17.76 & 28.25 \\  & Transformer + matching & 6.10 & 11.33 & 16.07 & **24.60** & **18.70** & **28.46** \\ \hline BERT & LSTM & 4.55 & 7.54 & 10.40 & 12.62 & 10.90 & 14.21 \\ Embedding & LSTM + matching & **4.66** & **7.62** & **11.01** & **12.87** & **11.06** & **14.41** \\ (BE) & Transformer & 2.86 & 2.49 & 5.26 & 2.64 & 6.19 & 3.54 \\  & Transformer + matching & **3.30** & 1.86 & **6.25** & 2.36 & **6.97** & 2.74 \\ \hline \hline \end{tabular}
\end{table}