Question: What experiment result led to conclussion that reducing the number of layers of the decoder does not matter much?
Evidence: Last, we analyze the importance of our second encoder ($enc_{src \rightarrow mt}$), compared to the source encoder ($enc_{src}$) and the decoder ($dec_{pe}$), by reducing and expanding the amount of layers in the encoders and the decoder. Our standard setup, which we use for fine-tuning, ensembling etc., is fixed to 6-6-6 for $N_{src}$-$N_{mt}$-$N_{pe}$ (cf. Figure FIGREF1), where 6 is the value that was proposed by Vaswani:NIPS2017 for the base model. We investigate what happens in terms of APE performance if we change this setting to 6-6-4 and 6-4-6. To handle out-of-vocabulary words and reduce the vocabulary size, instead of considering words, we consider subword units BIBREF19 by using byte-pair encoding (BPE). In the preprocessing step, instead of learning an explicit mapping between BPEs in the $src$, $mt$ and $pe$, we define BPE tokens by jointly processing all triplets. Thus, $src$, $mt$ and $pe$ derive a single BPE vocabulary. Since $mt$ and $pe$ belong to the same language (German) and $src$ is a close language (English), they naturally share a good fraction of BPE tokens, which reduces the vocabulary size to 28k.
The number of layers ($N_{src}$-$N_{mt}$-$N_{pe}$) in all encoders and the decoder for these results is fixed to 6-6-6. In Exp. 5.1, and 5.2 in Table TABREF5, we see the results of changing this setting to 6-6-4 and 6-4-6. This can be compared to the results of Exp. 2.3, since no fine-tuning or ensembling was performed for these three experiments. Exp. 5.1 shows that decreasing the number of layers on the decoder side does not hurt the performance. In fact, in the case of test2016, we got some improvement, while for test2017, the scores got slightly worse. In contrast, reducing the $enc_{src \rightarrow mt}$ encoder block's depth (Exp. 5.2) does indeed reduce the performance for all four scores, showing the importance of this second encoder.
FLOAT SELECTED: Table 1: Evaluation results on the WMT APE test set 2016, and test set 2017 for the PBSMT task; (±X) value is the improvement over wmt18smtbest (x4). The last section of the table shows the impact of increasing and decreasing the depth of the encoders and the decoder.
Last, we analyze the importance of our second encoder ($enc_{src \rightarrow mt}$), compared to the source encoder ($enc_{src}$) and the decoder ($dec_{pe}$), by reducing and expanding the amount of layers in the encoders and the decoder. Our standard setup, which we use for fine-tuning, ensembling etc., is fixed to 6-6-6 for $N_{src}$-$N_{mt}$-$N_{pe}$ (cf. Figure FIGREF1), where 6 is the value that was proposed by Vaswani:NIPS2017 for the base model. We investigate what happens in terms of APE performance if we change this setting to 6-6-4 and 6-4-6. 
The number of layers ($N_{src}$-$N_{mt}$-$N_{pe}$) in all encoders and the decoder for these results is fixed to 6-6-6. In Exp. 5.1, and 5.2 in Table TABREF5, we see the results of changing this setting to 6-6-4 and 6-4-6. This can be compared to the results of Exp. 2.3, since no fine-tuning or ensembling was performed for these three experiments. Exp. 5.1 shows that decreasing the number of layers on the decoder side does not hurt the performance. In fact, in the case of test2016, we got some improvement, while for test2017, the scores got slightly worse. In contrast, reducing the $enc_{src \rightarrow mt}$ encoder block's depth (Exp. 5.2) does indeed reduce the performance for all four scores, showing the importance of this second encoder.
FLOAT SELECTED: Table 1: Evaluation results on the WMT APE test set 2016, and test set 2017 for the PBSMT task; (±X) value is the improvement over wmt18smtbest (x4). The last section of the table shows the impact of increasing and decreasing the depth of the encoders and the decoder.
Answer: The answers are shown as follows:
* Exp. 5.1