# Impact of Sentence Representation Matching in Neural Machine Translation
## 5 Results and Discussions
### Robustness Analysis  
Table 9 shows the change in performance which occurred by applying the matching method. The bold scores indicate the cases that showed improvement. In the overall embedding and model settings, at least half of the test cases increased the scores. Precision and BLEU were improved up to 0.99 and 0.94. Transformer showed better performance than LSTM, but LSTM showed more cases that improved their performance. In the comparison of results between embedding methods, BE showed critically lower performance than RE and WV embedding.  
In the observation, we could confirm that the matching method has more of a chance to cause positive effects in various models and embedding methods. The better performance of transformer compared to LSTM is consistently observed in current NMT literature. The benefit of the matching method is likely to be less positive in the transformer. This difference may be caused by different densities of information in the sentence representation generated by the encoder-transformer. The performance of the BE method is seriously worse than the others, but it is because of the big difference in the unknown word rate. The rate of commonly used tokens is a maximum of 9.36% in BE, while the others showed values as high as 48.32%, as shown in Table 10. The scale difference of input elements in embedded vectors was not a cause, as shown in Table 6.  
Figure 9 shows three examples of robust analysis tests. Most translations are semantically almost correct translations in comparison to their reference sentence. However, the styles of models before and after applying matching are distinguished as red and blue-colored texts. Even if the semantic meaning is almost equivalent, the matching method more strongly follows the translation of the reference sentence. This is because the matching method has the role of restricting the translation to not generate a completely syntactic form in addition to the original translation models.  
\begin{table}
\begin{tabular}{l l c c c c} \hline \hline  & & \multicolumn{4}{c}{**Performance of Best Validation Model**} \\  & & **Train** & **Test1** & **Test2** \\ \hline Data & Model & Prec. & Prec. & BLEU & Prec. & BLEU \\ \hline  & \(E[\delta]\) & 0.99 & 0.25 & 0.72 & 0.27 & 0.69 \\ \hline \multirow{2}{*}{En \(\rightarrow\) Fr} & NMT implementation of [9] & 65.89 & 17.99 & 22.88 & 15.26 & 23.64 \\  & NMT + matching & 64.09 & **19.62** & **24.41** & **16.20** & **24.74** \\ \hline \multirow{2}{*}{Fr \(\rightarrow\) En} & NMT implementation of [9] & 63.22 & 22.56 & 21.68 & 17.98 & 21.94 \\  & NMT + matching & **65.04** & 22.48 & **22.51** & 17.94 & **22.59** \\ \hline \multirow{2}{*}{En \(\rightarrow\) Es} & NMT implementation of [9] & 63.21 & 19.66 & 20.96 & 21.27 & 28.41 \\  & NMT + matching & **65.63** & **20.16** & **22.09** & **22.50** & **29.96** \\ \hline \multirow{2}{*}{Es \(\rightarrow\) En} & NMT implementation of [9] & 65.42 & 22.23 & 21.47 & 22.42 & 28.62 \\  & NMT + matching & 63.83 & 20.18 & 20.24 & 20.39 & 27.24 \\ \hline \multirow{2}{*}{En \(\rightarrow\) De} & NMT implementation of [9] & 54.85 & 17.46 & 12.07 & 13.61 & 12.58 \\  & NMT + matching & **61.24** & **20.53** & **14.87** & **17.00** & **15.81** \\ \hline \multirow{2}{*}{De \(\rightarrow\) En} & NMT implementation of [9] & 60.75 & 19.88 & 17.59 & 17.18 & 19.68 \\  & NMT + matching & 59.46 & 18.31 & 16.84 & 15.28 & 18.68 \\ \hline \hline \end{tabular}
\end{table}
Table 8: _Cont._