# Improving Non-Autoregressive Machine Translation Using Sentence-Level Semantic Agreement
## 4 Experiments
### Main Results  
The main results of our model on the three translation tasks are listed in Table 1. Our model achieved a significant improvement and outperformed the other fully NAT models. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & & **EN \(\rightarrow\) DE** & **EN \(\rightarrow\) RO** & **DE \(\rightarrow\) EN** \\ \hline
**AT Model** & **Transformer** & **27.2** & **33.70** & **34.50** \\ \hline \multirow{8}{*}{Iterative-Based NAT models} & NAT-IR(k = 10) [15] & 21.61 & 29.32 & 23.94 \\  & LaNAT (k = 4) [22] & 26.30 & / & / \\  & LevT (k \(\approx\) 6) [23] & 27.27 & / & / \\  & CMLM (k = 4) [24] & 25.94 & 32.53 & 30.42 \\  & CMLM (k = 10) & 27.03 & 33.08 & 31.71 \\  & JM-NAT (k = 4) [25] & 26.82 & 32.97 & 31.27 \\  & JM-NAT (k = 10) & 27.31 & 33.52 & 32.59 \\ \hline \multirow{8}{*}{Fully NAT models} & NAT [4] & 17.69 & 26.22 & / \\  & Hint-NAT [26] & 21.11 & / & / \\  & TCL-NAT [27] & 21.94 & / & 28.16 \\  & DCRF-NAT [6] & 23.44 & / & / \\  & Flowseq [28] & 21.45 & 29.34 & 27.55 \\  & CMLM & 18.05 & 27.32 & / \\  & GLAT [7] & 25.21 & 31.19 & 29.80 * \\  & CNAT [29] & 25.56 & / & 31.15 \\ \cline{2-5}  & NAT (NPD = 100) & 19.17 & 29.79 & 24.21 \\  & Hint-NAT (NPD = 9) & 25.20 & / & / \\  & DCRF-NAT (NPD) & 26.07 & / & 29.99 \\  & GLAT (NPD = 7) & 26.55 & 32.87 & 31.23 * \\  & CNAT (NPD = 9) & 26.60 & / & / \\ \hline \multirow{8}{*}{Ours} & GLAT & 25.02 & 31.09 & 29.74 \\  & SLSAv1 & 25.53 & 31.23 & 30.45 \\ \cline{1-1}  & SLSA & 26.06 & 31.40 & 31.14 \\ \cline{1-1}  & SLSA (NPD = 7) & 27.01 & 32.90 & 32.39 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The results of our model on the EN \(\rightarrow\) DE, EN \(\rightarrow\) RO, and DE \(\rightarrow\) EN translation tasks. “NPD” represents noisy parallel decoding. “/” indicates there are no values in this term. “*” denotes the results are obtained by our reimplementation. “k” denotes the number of decoding iterations. Although the performance of the SLSA fell behind the iterative-based models, because it only performs one decoding pass, it has a large speed advantage. In detail, the following observations can be obtained from Table 1:  
* Our model, SLSA, is based on the architecture of GLAT, but our model obtained the better results than GLAT. It had a significant improvement (about 0.85 BLEU+) on the EN \(\rightarrow\) DE task. In addition, on the DE \(\rightarrow\) EN task, it gained +1.3 BLEU over GLAT. The reason is that SLSA introduces the semantic agreement between the encoder and decoder. The semantic agreement in the encoder can ensure the encoder projects the representation of the source target sentences into the shared space. The semantic agreement between the source and target sentences can ensure the sentence-level representation of the generated sentence is consistent with the representation of the source sentence. Meanwhile, this semantic agreement gives the decoder a constraint, which limits the decoder to generating semantically relevant words. Furthermore, the result of our model was better than SLSAv1. SLSAv1 uses the mean square error to pull the similar representation closer, but it cannot push the dissimilar representations away. We think this is the reason for better results of the SLSA. * Compared with the other fully NAT models, our model achieved better results and remained simple. Hint-NAT and TCL-NAT need a pretrained autoregressive model to provide the knowledge for the non-autoregressive model. Although DCRF-NAT does not need a pretrained autoregressive model, it introduces the CRF module and uses viterbi to generate the target sentence, which may reduce the decoding speed when the target sentence is lengthy. However, SLSA only modifies the training process, and it can perform the vanilla decoding process. Compared with CNAT, on the WMT14 EN \(\rightarrow\) DE task, our model achieved a better result. * As for the NPD, it did have a critical effect on improving the performance of the NAT model, as the previous works [4; 25] pointed out.