Question: What improvement does the MOE model make over the SOTA on machine translation?
Answer: 1.34 and 1.12 BLEU score on top of the strong baselines in BIBREF3, perplexity scores are also better, On the Google Production dataset, our model achieved 1.01 higher test BLEU score